Zeno's Blog

The first few pages of Bronfort et al. certainly talk the right talk:

EBH [Evidence-Based Healthcare] is about doing the right things for the right people at the right time. It does so by promoting the examination of best available clinical research evidence as the preferred process of decision making where higher quality evidence is available. This reduces the emphasis on unsystematic clinical experience and pathophysiological rationale alone while increasing the likelihood of improving clinical outcomes. The fact that randomized clinical trial (RCT) derived evidence of potentially effective interventions in population studies may not be translated in a straight forward manner to the management of individual cases is widely recognized. However, RCTs comprise the body of information best able to meet existing standards for claims of benefit from care delivery. The evidence provided by RCTs constitutes the first line of recommended action for patients and contributes, along with informed patient preference, in guiding care. (page 4)

Indeed, the report frequently extols the virtues of RCTs as the only reliable way of knowing what works and what doesn’t work:

It has become generally accepted that providing evidence-based healthcare will result in better patient outcomes than non-evidence-based healthcare. The debate of whether or not clinicians should embrace an evidence-based approach has become muted. Put simply by one author: “…anyone in medicine today who does not believe in it (EBH) is in the wrong business.” (page 5)

…

The question is no longer “should” we embrace EBH, but “how”? (page 6)

…

This requires a healthy respect for which scientific literature to use and how to use it. “Cherry-picking” only those studies which support one’s views or relying on study designs not appropriate for the question being asked, does not promote doing the right thing for the right people at the right time. (page 6)

…

Perhaps most critical is the clinician’s willingness to change the way they practice when high quality scientific evidence becomes available. It requires flexibility born of intellectual honesty that recognizes one’s current clinical practices may not really be in the best interests of the patient. In some cases this will require the abandonment of treatment and diagnostic approaches once believed to be helpful. In other cases, it will require the acceptance and training in new methods.The ever-evolving scientific knowledge base demands that clinicians be accepting of the possibility that what is “right” today, might not be “right” tomorrow. EBH requires that clinicians’ actions are influenced by the evidence. Importantly, a willingness to change must accompany the ability to keep up to date with the constant barrage of emerging scientific evidence. (page 6)

Some interesting and important points there and ones that many sceptics have been banging on about for a very long time.

It then goes on to discuss what exactly makes good EBH and good RCTs in particular — all familiar stuff about placebo controls, randomisation, good methodology, representative populations, etc.

Oddly, there is absolutely no mention — never mind discussion — of blinding anywhere in the report. Some of the references are certainly to blinded trials, but they refer to only six single-blinded trials, four double-blinded trials and three that are not stated as one or the other. Of course, many of the other references (there are 322 in total) may well have been blinded, but it is surprising that this important area is not discussed, particularly given the frequent mumblings about the difficulties of blinding a manual intervention.

Then there is bias — but that’s a post for another time.

But why?

It is interesting to stop for a minute and consider why this report was commissioned by the GCC. Bronfort says:

The impetus for this report stems from the media debate in the United Kingdom (UK) surrounding the scope of chiropractic care and claims regarding its effectiveness particularly for non-musculoskeletal conditions. (page 4)

However, one of the commentators on the report, Scott Haldeman, says:

The chiropractic profession is to be congratulated on formulating this Evidence Report. (page 7)

I note these without further comment.

The scene is set

And thus the scene is set for the review:

Professor Bronfort was asked to consider every condition, sign and symptom that was mentioned on any of the websites in respect of which the GCC has received complaints. This included ADHD, difficulties with breastfeeding and dyslexia.

…as the GCC have told fellow skeptical activist, Skeptic Barista. In fact, Bronfort and his researchers (all chiropractic researchers or tutors in the USA) were supplied with the full list of claims I had submitted in my complaint, but with all names and website details removed.

Bronfort et al. considered 26 categories of conditions in three main categories of musculoskeletal, headache and non-musculoskeletal conditions (Figure 2 in the report):

Musculoskeletal

Spinal pain

• low back
• thoracic
• neck
• coccyx

Extremity pain

• shoulder
• elbow
• wrist
• hip
• knee
• foot/ankle

Other

• temporomandibular disorders
• fibromyalgia
• myofascial pain syndrome

Headache

• migraine
• tension type
• cervicogenic
• miscellaneous headache

Non-Musculoskeletal

• asthma
• pneumonia
• vertigo
• infant colic
• hypertension
• enuresis (bed wetting)
• dysmenorrhea (period pains)
• premenstrual syndrome
• otitis media (ear infection)

So are all the claims I complained about covered by these 26 categories?

Not exactly. There were far more claims that I questioned, but many didn’t end up in the Bronfort report. Why? The GCC told Skeptic Barista:

Where any condition does not appear in the report, this is because no relevant randomised controlled trials were identified.

So they found no RCTs for claims like sleeping and feeding problems, prolonged crying as well as no RCTs for ADHD, irritable bowel syndrome, tinnitus, dyslexia, eczema, , hyperactivity, ‘wellness’, catarrh and sinus problems, reflux, chronic fatigue, stroke, multiple sclerosis, anxiety, panic attacks, depression, digestive disorders, constipation, etc, etc, etc. The list is long. Very long.

It may be up to the courts to decide whether there is a not jot of evidence for these conditions, but there is clearly — even to professional chiropractic researchers — not a jot of robust scientific evidence.

Money, money, money

Bear in mind that Bronfort carried out no new trials: no adults or babies had their backs cracked for this report. All they did was search for existing  evidence in the form of systematic reviews, RCTs and guidelines, gather it all together, review it and publish their findings.

As Skeptic Barista has revealed, this review cost the GCC $20,000, just over £13,000, or looking at it another way, about £4 per chiropractor. Not much to pay really for finding out what claims, if any, have any kind of an evidence base. I have to wonder why this has not been done before either by the GCC, any of the chiropractic trade associations, any of the colleges that teach chiropractic, or any chiropractor wanting to ensure he or she was only making claims that were backed up by robust evidence. The papers are already there. The systematic reviews are available in Cochrane or in PubMed or chiropractic publications. Why has it taken a crisis to prompt such an essential review?

In fact, shouldn’t the evidence base for chiropractic have been established before the Chiropractors Act 1994 was passed? Did it not occur to anyone at that time that trying to pass laws to regulate something that had such a poor evidence base and for which such wild claims were being made was irresponsible and just asking for trouble? Did no one bother to ask whether the public, rather than being protected, were left more vulnerable after the Act? It would seem not. And it looks like few lessons have been learned in recent years.

However, now that we have a report — althought somewhat flawed — there can be fewer excuses.

The way forward

However, all is not rosy. There is still much work to be done in critically examining the Bronfort report, including looking at the details of how they came to some of their conclusions and whether those conclusions are a fair and balanced of the evidence. Indeed, there are certainly instances of coming to somewhat dubious conclusions from the available evidence. More about this in a later post.

However, the Bronfort report still leaves one big problem. The GCC’s CoP does not say that chiropractors must only make claims in line with the Bronfort report: it says that chiropractors:

may publicise their practices or permit another person to do so consistent with the law and the guidance issued by the Advertising Standards Authority.

The GCC have sent a copy of the report to the ASA, but they have not yet changed their guidance. The ASA have told me:

…the Complaints team can only take into account our official guidance when assessing complaints, so until this changes we will continue to work from the existing guidance.

The ASA have been fairly thorough and rigorous with scientific evidence in the past, so it would not come as a surprise if they rejected at least some of the conclusions of Bronfort.

Making a start

It’s good to see so many chiropractors now looking at what they are claiming and revising those claims. As part of my ongoing complaints (don’t worry, I’ll be writing all about it at some point), some have even let me see ‘before and after’ screenshots of their websites.

They’ve talked the talk, now they must walk the walk and ensure that all claims made by chiropractors are backed by robust evidence and in line with ASA guidance.