The random thoughts of a sceptical activist

Bronfort on: Ear Infections

Time to take a closer look at some of the conclusions of the Bronfort report.

Otitis Media is, to the average parent, an ear infection — of the middle ear, to be precise. This can affect children and can be very painful and disturbing to both infant and parents. About one in five chiropractors I complained about made claims about ear or similar infections. It was also one of the childhood ailments that Simon Singh mentioned in the Guardian article the BCA had a hissy fit about.

Because of my complaints and because there appeared to be no definitive list (other than the ASA’s list of acceptable claims, of course) of what conditions were backed by robust evidence, the GCC commissioned five US chiropractic researchers to review the good evidence for all the conditions I complained about. I’ve already said something about the kinds of evidence Bronfort et al. decided should be included in the report: they were interested in only relying on quality evidence.

Bronfort’s evidence

Otitis Media

Definition

Otitis media is characterized by middle ear inflammation which can exist in an acute or chronic state and can occur with or without symptoms.

Diagnosis

Diagnosis of otitis media relies on otoscopic signs and symptoms consistent with a purulent middle ear effusion in association with systemic signs of illness [266].

Evidence base for manual treatment

Systematic reviews (most recent)

Hawk et al [230] found promising evidence for the potential benefit of spinal manipulation/mobilization procedures for children with otitis media. This was based on one trial [267]. Two other reviews specifically addressed spinal manipulation by chiropractors for non-musculoskeletal [247] and pediatric [248] conditions. Both found insufficient evidence to comment on manual treatment effectiveness or ineffectiveness for otitis media.

Evidence-based clinical guidelines

The American Academy of Pediatrics 2004 guidelines on the diagnosis and management of acute otitis media [268] concluded no recommendation for complementary and alternative medicine for the treatment of acute otitis media can be made due to limited data.

Recent randomized clinical trials not included in above

Wahl et al investigated the efficacy of osteopathic manipulative treatment with and without Echinacea compared to sham and placebo for the treatment of otitis media [269]. The study found that a regimen of up to five osteopathic manipulative treatments does not significantly decrease the risk of acute otitis media episodes. This study had a high risk of bias.

Evidence Summary (See Figure 7)

Inconclusive evidence in an unclear direction regarding the effectiveness of osteopathic manipulative therapy for otitis media [267,269].

Other effective non-invasive physical treatments or patient education

Patient education and “watch and wait” approach for 72 hours for acute otitis media [266,268]

Evidence summary

To summarise all the evidence cited (I’ll use Bronfort’s reference numbers to avoid confusion):

Ref (Lead author/publisher) Type Treatment Bronfort’s Conclusions
230 (Hawk) Systematic review, but only one relevant trial [267] Chiropractic care “promising evidence for the potential benefit”
247 (Ernst) Systematic review of one relevant trial Spinal manipulation by chiropractors “insufficient evidence to comment”
248 (Gotlib) Systematic review of one feasibility study Spinal manipulation by chiropractors “insufficient evidence to comment”
266 (SIGN) Guidelines Various “Did not consider chiropractic”
267 (Mills) RCT (n=57) Osteopathic manipulative treatment “The results of this study suggest a potential benefit of osteopathic manipulative treatment as adjuvant therapy in children with recurrent AOM”
268 (Pediatrics) Guidelines Various “No recommendations for complementary and alternative medicine (CAM) for treatment of [acute otitis media] are made based on limited and controversial data”
269 (Wahl) RCT (n=90) Osteopathic manipulative treatment (OMT) including cranial manipulation “OMT did not significantly affect risk [of having at least one episode of acute otitis media during 6-month follow-up] compared to sham”

So that’s three systematic reviews, two guidelines and one RCT.

One of those systematic reviews (Hawk) cited just one RCT (Mills).

The other two reviews (Ernst, Gotlib) said there was insufficient evidence to comment.

One of the guidelines (SIGN) didn’t even consider chiropractic and the other (Pediatrics) only gave chiropractic a passing mention, correctly lumping it in with homeopathy and other woo and said:

To date there are no studies that conclusively show a beneficial effect of alternative therapies used for [acute otitis media].

They found an additional RCT (Wahl) that had not been covered in the systematic reviews.

Bronfort examined all this evidence and concluded there was:

Inconclusive evidence in an unclear direction regarding the effectiveness of osteopathic manipulative therapy for otitis media

However, he only relied on the Mills and Wahl papers for this conclusion so we need to take a closer look at these two trials.

Sleight of hand

Ref 230 (Hawk) is a review titled Chiropractic Care for Nonmusculoskeletal Conditions: A Systematic Review with Implications for Whole Systems Research, but they were only able to find one RCT for otitis media (267, Mills). They did rate it highly on several standard checklists, including Jadad, and from this single study, Hawk concluded:

Evidence was promising for the potential benefit of manual procedures for children with otitis media

Note it says ‘manual procedures’. More of that later.

Delving deeper still…

Mills is a small RCT (n=57) titled: The Use of Osteopathic Manipulative Treatment as Adjuvant Therapy in Children With Recurrent Acute Otitis Media.

The clue, as they say, is in the title.

The objective of the study was:

To study effects of osteopathic manipulative treatment as an adjuvant therapy to routine pediatric care in children with recurrent acute otitis media (AOM). (My emphasis)

Note that when Bronfort introduces this study, he omits to mention the fact that this trial was for osteopathic manipulation and simply says it was for “spinal manipulation/mobilization procedures”. Indeed Hawk uses it in a review of chiropractic care.

However, the study itself is just a bit clearer on what the treatment was:

OSTEOPATHIC MANIPULATIVE TREATMENT

Osteopathic manipulative treatment was provided to the intervention group at each visit, as indicated by the osteopathic examination results and the child’s cooperation. Treatments lasted 15 to 25 minutes, which is usual in most practices. Treatments were gentle techniques on areas of restriction consisting of articulation, myofascial release, balanced membranous tension (according to teachings of William Garner Sutherland, DO, and others 25), balanced ligamentous tension, facilitated positional release, and/or counterstrain treatments. These techniques are familiar to most recently trained osteopathic physicians, but it is not in the scope of this article to describe them in detail. Despite some expected variation in their application by different physicians, we attempted to standardize their approach by using only physicians with teaching experience in OMT. No high-velocity (popping) techniques were used. The entire body, with attention to the head and neck, was included in the osteopathic evaluation and treatment.

No spinal adjustments. No high-velocity low-amplitude thrusts. No ‘gentle but specific’ adjustments. No back cracking. No popping. Not chiropractic.

But since the paper refuses to give details about exactly what the half-dozen osteopathic treatments were and how they were randomised (if indeed they were) and administered to the 25 infants that were in the OMT arm of the trial, it is impossible to even be sure what effect any of the osteopathic treatments had.

On the subject of blinding:

The parents knew the child’s treatment group, though every effort was made not to let this information affect pediatrician recommendations, which we knew by means of medical record review.

Mills concluded:

Although a larger study is needed, it appears that [osteopathic manual treatment] offers a potential benefit as adjuvant therapy for children with recurrent AOM.

In summary:

  • the treatment was osteopathy, not chiropractic;
  • it was small and low powered;
  • it looked at OMT as an adjuvant (ie as an additional treatment) to normal care by a pediatrician, including antibiotic use;
  • the parents were not blinded to the treatment;
  • there was no placebo group;
  • the subjects were six months to six years, therefore not applicable to newborns;
  • a larger study was recommended.

So what on earth made Bronfort think this was an appropriate quality study to include in a review of chiropractic?

Déjà vu

Of course, you may have realised by now that you’re read about this study before — it was in the BCA’s plethora (although they missed out the significant words “as an adjuvant therapy” in the title) that Gimpy took it apart in his blog — one of the many that demolished the BCA’s plethora within 24 hours of its publication.

Similarly, it was dismissed by Ernst in the BMJ as having nothing to do with chiropractic: it still has nothing to do with chiropractic.

So why did Bronfort include this study? We can only guess.

What I hadn’t noticed before was that both the Mills and the Hawk papers were included in the BCA’s plethora for otitis media (their references 9 and 11). Although they have tried to use Hawk to bolster other claims, this does look like a bit of double counting.

Next!

The only other paper Bronfort chose to rely on for his conclusion was reference 269 (Wahl). This is Echinacea purpurea and osteopathic manipulative treatment in children with recurrent otitis media: a randomized controlled trial.

This didn’t make an appearance in the BCA’s plethora, but it fails for the same reason as the Mills study — it’s about osteopathy, with a little bit of herbalism thrown in for good measure:

Treatment modalities were limited to cranial osteopathy, balanced membranous/ligamentous tension, and/or myofascial release (applied directly or indirectly). These treatments consist of gentle manipulations of the cranium, pelvis, diaphragm, and other structures. No high velocity or thrusting maneuvers were performed. At the discretion of the osteopathic physician, an osteopathic percussion hammer could also be used for treatment, which allowed gentle vibration in tissues at variable frequencies.

Not chiropractic, then either.

Note that Bronfort made no assessment of the quality of any of the RCTs it looked at:

While critical appraisal of the included reviews and guidelines would be ideal, it is beyond the scope of the present report.

They presumably weren’t getting paid enough or were in too much of a hurry.

However, even for osteopathy, they concluded:

A regimen of up to five osteopathic manipulative treatments does not significantly decrease the risk of acute otitis media.

OMT was not significantly associated with the risk of AOM

Sham

Even if this was all about osteopathy and not chiropractic, it’s difficult to see how Bronfort reached the conclusions he did, based on these two studies — to recap:

Inconclusive evidence in an unclear direction regarding the effectiveness of osteopathic manipulative therapy for otitis media

What Bronfort should have concluded was not that there was inconclusive evidence but that there was no evidence for the use of chiropractic for otitis media.

Not a jot.

I have to wonder why otitis media even made it into the report, rather than being rejected outright through lack of any applicable RCTs.

Ah! I know why.

Claims

So, what will chiropractors be claiming on their websites and other advertising for otitis media?

As the GCC told Skeptic Barista:

The GCC’s guidance to the profession mirrors that of the Committee of Advertising Practice Copy Advice Team ie that any claims for chiropractic must be based on best research of the highest quality. This will almost certainly mean randomised controlled trials that produce high or moderate quality positive evidence.

Or, as Skeptic Barista succinctly put it:

No RCT = No evidence = No claim

http://skepticbarista.wordpress.com/2010/03/14/chiropractic-evidence-claims-and-a-willingness-to-change/

15 Responses to Bronfort on: Ear Infections

  • *applause*

    Good work.

  • thanks for the interesting blog, its one of my favorites

  • Interesting that in their “other effective treatment” options they suggest “waiting for 72 hours”. I wonder how many of the “positive” results in all those other trials were really just a result of time, not action?

  • Nice work, thanks very much.

    It seems that Bronfort has been honing his skills in interpretive bias since I first first came across his work with this “systematic” review of chiropractic for low back pain http://www.ncbi.nlm.nih.gov/pubmed/15125860.

    You have to dig deeper now to find the sleights of hand.

    The Mills study is even less reliable than you make it out to be.

    It was on prevention of acute otitis media, and the outcomes measured were highly subjective, and therefore *very* vulnerable to bias in an unblinded study. Episodes of acute OM, surgery, and prescriptions of antibiotics superficially seem to be “objective”, but they all depend on the subjective diagnosis of AOM. A study done well before the Mills trial (will dig for the reference if I am pressed for it), took nearly 1000 children and asked ENT surgeons to assess if they needed surgery. The 50% that were thought to not need surgery, were then reassessed by another doctor. The 50% that were thought to not need surgery, were then reassessed by another doctor. And so on, until almost every child had been assessed as needing surgery. So, the decision whether a child needs grommets or not is completely random. (Rates of surgery for otitis media have fallen dramatically since that paper was published.)

    Dropout rates were higher than anticipated (about 15%), and were higher in the control group than the intervention group. Results were not analysed on an “intention to treat” basis, so this could hide an additional source of bias.

    I haven’t looked at the Wahl study, but would be surprised if it doesn’t have the same methodological weaknesses.

  • Wow, this is shocking in how unconclusive the evidence is. Thanks for posting, very informative.

  • Excellent post, once again, Zeno.

    @ Michael

    Yes, re Bronfort and bias, it’s interesting to note what Ernst has had to say about him the past. The last sentence of the quote below speaks volumes:

    Quote
    “Bronfort[10] concluded that SM and/or mobilization are viable options for treating low back and neck pain. Bronfort[13] concluded that SM has a better effect than massage and a comparable effect to prophylactic drugs for headache. Reid[21] found only limited evidence from methodologically poor trials for effectiveness in cervicogenic dizziness. All other conclusions agreed that the effectiveness of SM is not supported by the results from rigorous clinical trials]…..Three systematic reviews were related to SM for neck pain[10-12] of which one reached a a positive[10] overall conclusion and this was the same review which reached a positive conclusion regarding back pain. The most authoritative of the three reviews[12] stated that SM/mobilization is effective only when combined with other interventions such as exercise and as a sole treatment for neck pain, it is not of demonstrable effectiveness. Similarly, there are three systematic reviews of SM for headache.[13-15] While Bronfort et al.[13] concluded that SM is as effective as other interventions, the other two teams of reviewers[14,15] did not find conclusive evidence in favour of SM…..Our previous work[6] has shown that the conclusions of reviews of SM for back pain appear to be influenced by authorship and methodological quality such that authorship by osteopaths or chiropractors and low methodological quality are associated with a positive conclusion. It is perhaps relevant to note that all three of the overtly positive recommendations for SM in the indications back pain,[10] neck pain[10] and headache[13] originate from the same chiropractor.”

    Ref:
    E Ernst, P H Canter. A systematic review of systematic reviews of spinal manipulation J R Soc Med 2006;99:192-196
    http://jrsm.rsmjournals.com/cgi/content/full/99/4/192?ijkey=67d13fec77bd5ae1759151315cce803fdc962330

  • Excellent Blogging!

    Sets out in very clear terms just how little of the evidence chiropractors are referring to is actually chiropractic evidence!

    I’ve already asked for the General Osteopathic Councils verdict of the Bronfort report (as it mentions OMT). Their reply and how they view the report makes an interesting comparison to the views fo the GCC. Hopefully blogging on this tomorrow.

    Good work.

  • with the atheists:

    they start begging when they start dying…

    they PAY THE PRICE FOR ATTACKING THE SUPERNATURAL –

    with their LIVES…

    CRYSTAL NIGHT TONIGHT!

    Atheists,

    but you have NO ANSWER TO DEATH… therefore you FAIL…

    THE DEATH TRAP

    http://abcnews.go.com/Nightline/FaceOff/
    ********

    THE REAL QUESTION:

    DOES ATHEISM HAVE A FUTURE?

    AND THE ANSWER – NO!

    visit:

    http://www.clubconspiracy.com/forum/f30/does-ath-ism-have-future-no-11202.html#post66570

    Shermer – Harris – Myers – Dawkins – Randi VS. NOSTRADAMUS – EINSTEIN – MARKUZE

    you’re ANNIHILATED!

    Atheists,

    Repent and turn to God.

  • I’m surprised it took so long for someone like the above to comment on your site. The majority of the people in these skeptical blogs are radically against everything that hasn’t been explained,
    or approved by a scientist.
    Eventually, you get get the radical opposite to post and what is supposed to be an intelligent
    debate turns into something a childish melee.

  • @middleoftheroad

    There is nobody “like the above”, as you put it. He is unique and much beloved by Zeno and by me. Why – we’ve even given him his own forum
    here.

    So there’s no danger of anything turning into a childish melee. :-)

  • No RCT = No evidence = No claim = No way!

    I suspect this excerpt from the New Zealand Chiropractic association would be perceived by potential clients as a claim! And just look at those claims! Who needs traditional medicine!!(Check other similar claims and a rather one-sided ‘selection’ of ‘research’ at http://www.chiropractic.org.nz/research-literature )

    Other Health Issues and Conditions
    A smaller number of clinical trials alongside other preliminary studies, case reports and anecdotal evidence show that chiropractic may be effective in many different conditions such as colic, ear infection and other paediatric issues, HIV and immune problems, asthma, multiple sclerosis and other nervous system disorders, infertility issues, digestive and cardiovascular health etc. It is important to realise that chiropractic is not the treatment of any condition. The purpose of chiropractic care is to reduce interference to the nerve system, allowing the person to experience greater function and an overall improvement in quality of life.

  • Yes Andrew, I think skeptics in all countries should be looking the claims being made and the regulatory frameworks that chiros have to work within and see what can be done about about these outrageous claims. I bet every one of these regulatory frameworks — whether statutory or not — say something about ‘protecting the public’. They now need to be called to account.

  • Given the comprehensive demolition of Bronfort’s effort to show chiroquackery is useful for ear infections, I have to ask how much is down to ignorance and how much down to plain, straightforward dishonesty. Chiroquackery is a business so I know where my vote goes.

  • We often see comments like osteopathy is a different treatment to chiropractic or physiotherapy. Osteopathy, chiropractic and physiotherapy are professions – not treatments.

    Each may employ a wide range of treatments, some modestly evidence-based, most not at all, but if we are to examine what they offer we need to distinguish between the semantics of professional titles and the treatments provided by each profession (including overlaps).

Leave a Reply

Your email address will not be published. Required fields are marked *


6 × = twenty four

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>