The random thoughts of a sceptical activist

Talking the talk

The first few pages of Bronfort et al. certainly talk the right talk:

EBH [Evidence-Based Healthcare] is about doing the right things for the right people at the right time. It does so by promoting the examination of best available clinical research evidence as the preferred process of decision making where higher quality evidence is available. This reduces the emphasis on unsystematic clinical experience and pathophysiological rationale alone while increasing the likelihood of improving clinical outcomes. The fact that randomized clinical trial (RCT) derived evidence of potentially effective interventions in population studies may not be translated in a straight forward manner to the management of individual cases is widely recognized. However, RCTs comprise the body of information best able to meet existing standards for claims of benefit from care delivery. The evidence provided by RCTs constitutes the first line of recommended action for patients and contributes, along with informed patient preference, in guiding care. (page 4)

Indeed, the report frequently extols the virtues of RCTs as the only reliable way of knowing what works and what doesn’t work:

It has become generally accepted that providing evidence-based healthcare will result in better patient outcomes than non-evidence-based healthcare. The debate of whether or not clinicians should embrace an evidence-based approach has become muted. Put simply by one author: “…anyone in medicine today who does not believe in it (EBH) is in the wrong business.” (page 5)

The question is no longer “should” we embrace EBH, but “how”? (page 6)

This requires a healthy respect for which scientific literature to use and how to use it. “Cherry-picking” only those studies which support one’s views or relying on study designs not appropriate for the question being asked, does not promote doing the right thing for the right people at the right time. (page 6)

Perhaps most critical is the clinician’s willingness to change the way they practice when high quality scientific evidence becomes available. It requires flexibility born of intellectual honesty that recognizes one’s current clinical practices may not really be in the best interests of the patient. In some cases this will require the abandonment of treatment and diagnostic approaches once believed to be helpful. In other cases, it will require the acceptance and training in new methods.The ever-evolving scientific knowledge base demands that clinicians be accepting of the possibility that what is “right” today, might not be “right” tomorrow. EBH requires that clinicians’ actions are influenced by the evidence. Importantly, a willingness to change must accompany the ability to keep up to date with the constant barrage of emerging scientific evidence. (page 6)

Some interesting and important points there and ones that many sceptics have been banging on about for a very long time.

It then goes on to discuss what exactly makes good EBH and good RCTs in particular — all familiar stuff about placebo controls, randomisation, good methodology, representative populations, etc.

Oddly, there is absolutely no mention — never mind discussion — of blinding anywhere in the report. Some of the references are certainly to blinded trials, but they refer to only six single-blinded trials, four double-blinded trials and three that are not stated as one or the other. Of course, many of the other references (there are 322 in total) may well have been blinded, but it is surprising that this important area is not discussed, particularly given the frequent mumblings about the difficulties of blinding a manual intervention.

Then there is bias — but that’s a post for another time.

But why?

It is interesting to stop for a minute and consider why this report was commissioned by the GCC. Bronfort says:

The impetus for this report stems from the media debate in the United Kingdom (UK) surrounding the scope of chiropractic care and claims regarding its effectiveness particularly for non-musculoskeletal conditions. (page 4)

However, one of the commentators on the report, Scott Haldeman, says:

The chiropractic profession is to be congratulated on formulating this Evidence Report. (page 7)

I note these without further comment.

The scene is set

And thus the scene is set for the review:

Professor Bronfort was asked to consider every condition, sign and symptom that was mentioned on any of the websites in respect of which the GCC has received complaints. This included ADHD, difficulties with breastfeeding and dyslexia.

…as the GCC have told fellow skeptical activist, Skeptic Barista. In fact, Bronfort and his researchers (all chiropractic researchers or tutors in the USA) were supplied with the full list of claims I had submitted in my complaint, but with all names and website details removed.

Bronfort et al. considered 26 categories of conditions in three main categories of musculoskeletal, headache and non-musculoskeletal conditions (Figure 2 in the report):

Musculoskeletal

Spinal pain

  • low back
  • thoracic
  • neck
  • coccyx

Extremity pain

  • shoulder
  • elbow
  • wrist
  • hip
  • knee
  • foot/ankle

Other

  • temporomandibular disorders
  • fibromyalgia
  • myofascial pain syndrome

Headache

  • migraine
  • tension type
  • cervicogenic
  • miscellaneous headache

Non-Musculoskeletal

  • asthma
  • pneumonia
  • vertigo
  • infant colic
  • hypertension
  • enuresis (bed wetting)
  • dysmenorrhea (period pains)
  • premenstrual syndrome
  • otitis media (ear infection)

So are all the claims I complained about covered by these 26 categories?

Not exactly. There were far more claims that I questioned, but many didn’t end up in the Bronfort report. Why? The GCC told Skeptic Barista:

Where any condition does not appear in the report, this is because no relevant randomised controlled trials were identified.

So they found no RCTs for claims like sleeping and feeding problems, prolonged crying as well as no RCTs for ADHD, irritable bowel syndrome, tinnitus, dyslexia, eczema, , hyperactivity, ‘wellness’, catarrh and sinus problems, reflux, chronic fatigue, stroke, multiple sclerosis, anxiety, panic attacks, depression, digestive disorders, constipation, etc, etc, etc. The list is long. Very long.

It may be up to the courts to decide whether there is a not jot of evidence for these conditions, but there is clearly — even to professional chiropractic researchers — not a jot of robust scientific evidence.

Money, money, money

Bear in mind that Bronfort carried out no new trials: no adults or babies had their backs cracked for this report. All they did was search for existing  evidence in the form of systematic reviews, RCTs and guidelines, gather it all together, review it and publish their findings.

As Skeptic Barista has revealed, this review cost the GCC $20,000, just over £13,000, or looking at it another way, about £4 per chiropractor. Not much to pay really for finding out what claims, if any, have any kind of an evidence base. I have to wonder why this has not been done before either by the GCC, any of the chiropractic trade associations, any of the colleges that teach chiropractic, or any chiropractor wanting to ensure he or she was only making claims that were backed up by robust evidence. The papers are already there. The systematic reviews are available in Cochrane or in PubMed or chiropractic publications. Why has it taken a crisis to prompt such an essential review?

In fact, shouldn’t the evidence base for chiropractic have been established before the Chiropractors Act 1994 was passed? Did it not occur to anyone at that time that trying to pass laws to regulate something that had such a poor evidence base and for which such wild claims were being made was irresponsible and just asking for trouble? Did no one bother to ask whether the public, rather than being protected, were left more vulnerable after the Act? It would seem not. And it looks like few lessons have been learned in recent years.

However, now that we have a report — althought somewhat flawed — there can be fewer excuses.

The way forward

However, all is not rosy. There is still much work to be done in critically examining the Bronfort report, including looking at the details of how they came to some of their conclusions and whether those conclusions are a fair and balanced of the evidence. Indeed, there are certainly instances of coming to somewhat dubious conclusions from the available evidence. More about this in a later post.

However, the Bronfort report still leaves one big problem. The GCC’s CoP does not say that chiropractors must only make claims in line with the Bronfort report: it says that chiropractors:

may publicise their practices or permit another person to do so consistent with the law and the guidance issued by the Advertising Standards Authority.

The GCC have sent a copy of the report to the ASA, but they have not yet changed their guidance. The ASA have told me:

…the Complaints team can only take into account our official guidance when assessing complaints, so until this changes we will continue to work from the existing guidance.

The ASA have been fairly thorough and rigorous with scientific evidence in the past, so it would not come as a surprise if they rejected at least some of the conclusions of Bronfort.

Making a start

It’s good to see so many chiropractors now looking at what they are claiming and revising those claims. As part of my ongoing complaints (don’t worry, I’ll be writing all about it at some point), some have even let me see ‘before and after’ screenshots of their websites.

They’ve talked the talk, now they must walk the walk and ensure that all claims made by chiropractors are backed by robust evidence and in line with ASA guidance.

doing the right things for the right people at the right time [5]. It does so
by promoting the examination of best available clinical research evidence as the
preferred process of decision making where higher quality evidence is available [6].
This reduces the emphasis on unsystematic clinical experience and pathophysiological
rationale alone while increasing the likelihood of improving clinical outcomes [7].
The fact that randomized clinical trial (RCT) derived evidence of potentially effective
interventions in population studies may not be translated in a straight forward manner
to the management of individual cases is widely recognized [8-10]. However, RCTs
comprise the body of information best able to meet existing standards for claims of
benefit from care delivery. The evidence provided by RCTs constitutes the first line of
recommended action for patients and contributes, along with informed patientdoing the right things for the right people at the right time [5]. It does so

by promoting the examination of best available clinical research evidence as the

preferred process of decision making where higher quality evidence is available [6].

This reduces the emphasis on unsystematic clinical experience and pathophysiological

rationale alone while increasing the likelihood of improving clinical outcomes [7].

The fact that randomized clinical trial (RCT) derived evidence of potentially effective

interventions in population studies may not be translated in a straight forward manner

to the management of individual cases is widely recognized [8-10]. However, RCTs

comprise the body of information best able to meet existing standards for claims of

benefit from care delivery. The evidence provided by RCTs constitutes the first line of

recommended action for patients and contributes, along with informed patient

10 Responses to Talking the talk

  • I am surprised that in their eagerness to strive for a rigorous evaluation of all the evidence, Bronfort et al didn’t find this paper on: Spinal manipulation for the treatment of headache http://bit.ly/cbYXSP. I’ll admit I did have to use a little known feature of the net called Google but I managed.

  • Chiropractic treatment is one of the most suitable treatment method for everyone and every stage of age. It is really amazing that anyone would have a new life after applying chiropractic.

  • Thanks for another excellent post.

    FYI, according to a letter that the GCC sent to the group Action for Victims of Chiropractic (AVC) in 2004, it would seem that robust evidence has never been a concern for chiropractors (at least not up until now). You can read the GCC’s entire letter in the June 2004 section of the AVC news page linked to below, but here’s an interesting excerpt from it:

    Quote
    “ No health profession limits its approach to treatment for which there is evidence, because for most interventions in healthcare there simply is not yet good evidence.”

    http://www.chirovictims.org.uk/victims/news.html

    No doubt the above was the reason why it was happy to declare in a November 2006 letter to the AVC group that:

    Quote
    “Adjustment of the atlas, craniosacral therapy and applied kinesiology fall within the […] definition of evidence-based care.”

    http://jdc325.files.wordpress.com/2009/06/gcc-ak.pdf

    IMO, the regulation of chiropractic was premature and happened because legislators (who are generally not well versed in matters of science) were duped by highly organised chiropractic lobby groups intent on securing statutory regulation in order to give the chiropractic industry in the UK an appearance of legitimacy, thereby increasing its profitability. Indeed, here’s what a former Chair of the GCC had to say on the subject of chiropractic statutory regulation back in 2004:

    Quote
    “In spite of strong mutual suspicion and distrust, the profession united under a group formed specifically to pursue regulation and secured the Chiropractors Act (1994)…..Regulation for a new profession will literally ‘legitimise it’, establishing its members within the community, making them feel more valued. In turn, this brings greater opportunity for more clients and a healthier bank balance.”

    Michael C. Copland-Griffiths, former Chairman of the General Chiropractic Council (European Journal of Oriental Medicine, Vol.2 No.6) http://web.archive.org/web/20060924183943/www.ejom.co.uk/backissues/vol2no6/feature1.html

  • Chiropractor Lawrenceville ga said:

    Chiropractic treatment is one of the most suitable treatment method for everyone and every stage of age. It is really amazing that anyone would have a new life after applying chiropractic.

    LOL! I think Dr. Ross Carter (the Gentle Doctor, apparently) of Lawrenceville Chiropractic in Georgia, USA needs to hire a new spammer! This one posted from Bangladesh.

  • Chris Gurr said:

    I am surprised that in their eagerness to strive for a rigorous evaluation of all the evidence, Bronfort et al didn’t find this paper on: Spinal manipulation for the treatment of headache http://bit.ly/cbYXSP. I’ll admit I did have to use a little known feature of the net called Google but I managed.

    I hope you had a rest after all that hard work! Thanks for finding it. I’m sure they’ll be a lot more criticism of the Bronfort report.

  • Zeno said: “I hope you had a rest after all that hard work! Thanks for finding it. I’m sure they’ll (sic) be a lot more criticism of the Bronfort report.”

    Oh, I’m sure there will! But only because it’s become a sport for you and your cronies, no better reason than that.

  • Sounds like a good enough reason to me.

  • Now that the GCC looks like it might start grasping the EBM nettle following the findings of the (controversial) Bronfort review, it’s interesting to note in an ex-chiropractor’s blog today that some chiropractors have been looking at the possibility of taking legal action against the GCC:

    Quote
    “I am a chiropractor who has been very supportive of the GCC in the past. However, the way they are handling these complaints makes me reconsider. It is now possible that chiropractors will find themselves found guilty of professional misconduct for following the example set by the GCC in their own leaflets and websites. This raises the question are the GCC fit for purpose?

    Interstingly if chiropractors suffer a loss of reputation or financial loss for following the GCCs example/advice it may open up the possibility of legal action or even a class action against the GCC.”

    See comments 5 and 6 here:
    http://chiropracticlive.com/how-coats-runs-the-council/how-are-the-general-chiropractic-council-and-the-british-chiropractic-council-going-to-deal-with-zeno-rsquo-s-complaints/

    Bearing in mind that the Bronfort review carried out no new trials, I wonder if patients could demand to be refunded up nine years retrospectively (since the date that statutory regulation came into force) for having been administered bogus treatments?

  • “…are the GCC fit for purpose?”
    That depends upon for what purpose one assumes that the GCC exist.
    There is divided opinion about their role, even within the GCC.

Leave a Reply

Your email address will not be published. Required fields are marked *


− one = 1

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>