Shedding more light on acceptable evidence

Pharmacies are in the news again this week. Not Boots this time, but that other well-known high street chemist, Lloyds Pharmacy.

An ASA adjudication, published today, upheld a complaint against a TV advert about their ‘light therapy device‘.  The advert claimed:

Hay fever seasons [sic] here again.  But heres [sic] something you might not have tried before, the Lloyds Pharmacy hay fever reliever.  Its [sic] been shown to help reduce symptoms like your runny nose and itchy eyes … Just pop it up your nose for a couple of minutes two or three times a day and start making the most of your summer.

Although I wouldn’t be entirely keen on sticking anything up my nose, I don’t suffer from hay fever. However, I do know it can be a miserable condition and anything that might help would be worth a try. But you’d want something for which there was good evidence for efficacy before you splashed out your hard-earned cash, wouldn’t you? And you’d want the seller to be able to provide that good evidence when asked?

Lloyds had evidence, didn’t they? Yes and no.

The claims they made were that their device would ‘help reduce the symptoms like your runny nose and itchy eyes’ and that, after just a few uses, the used would be able to start making the most of their summer.

So, the evidence would have to directly address those claims, wouldn’t it?

This is really where Lloyds failed.

The study they provided in evidence wasn’t conducted during the pollen season, so the ASA’s expert concluded that:

…it was difficult to extrapolate the results to that scenario.

Surprise, surprise. Evidence is only acceptable if it directly relates to the claims you are making.

However, the expert noted it:

…came from a credible source and was published in an internationally recognised peer reviewed journal.

…but we’re not told which journal it was.

Flawed methodology

Their expert also noted that the methodology was flawed in that the study did not use a method that was close to real-life conditions, including the method and frequency of delivery of pollen and what happened in low and medium pollen count days. The ASA also were also critical of the lack of information on what happened after 14 days — important if you’re making claims about lasting all summer long.

The study wasn’t a complete loss. It did appear to show ‘a reduction in sneezing, runny nose and eyes and itchy palate’ but it was:

…flawed to the extent that there was no baseline assessment, instead it was only subjective reporting of the effect of pollen on symptoms, which made it difficult to interpret whether light therapy had really altered the immune response. [The ASA’s expert] was concerned that it was not clear what subjects [sic] responses to pollen were prior to intervention with the device and therefore it was difficult to compare any later change in response to pollen.

No baseline? Then how do you know your treatment made any difference? That word ‘subjective’ is also important.

Additionally, the study only reported on the reduction in ‘sneezing, runny nose and eyes and itchy palate’, and that the device did not have an effect on other symptoms of hay fever, including itchy eyes, nose and throat. Since ‘itchy eyes’ was in the claims made, the study did not substantiate that claim.

There were other problems with the evidence to do with how it was known the participants had complied with the treatment (and, I assume, how those subjective improvements were recorded). The study relied on the participants filling in a diary card and being interviewed. The expert was concerned about the objectivity of this.

In summary, the ASA said:

We noted that our expert had raised considerable doubt in relation to the claims, raising questions about the study, such as the lack of an objective measure on which to base its findings and that its replication of real-life circumstances was limited for several reasons. These included the fact that there was no information on the effect of the device on symptoms on low or medium pollen count days or on its effect beyond 14 days.  In particular, we noted the study did not show the device to be effective on “itchy eyes”, as claimed in the ad.  We concluded that the ad was misleading.

The ad breached CAP (Broadcast) TV Advertising Standards Code rules 5.1.1 (Misleading advertising) and 5.2.1 (Evidence).

Action
The ad must not appear again in its current form.

A definite fail for the advertiser.

Unfortunately, the adjudication says nothing about whether the study was controlled and randomised.

There’s evidence…and then there’s good evidence

It’s not difficult to ensure that the evidence you hold (before you make any kind of claim, of course) is of a standard that the ASA might accept. Make sure it really does substantiate what you claim; make sure it is methodologically sound; make sure it is from a controlled and randomised study or studies; make sure it is published in an internationally recognised peer reviewed journal. All that doesn’t guarantee that the evidence will be solid enough, but anything less than that and you are bound to lose!

If you didn’t hold that standard of evidence when you were making claims, why on earth are you making those claims?

Just to make money perhaps?

6 thoughts on “Shedding more light on acceptable evidence”

  1. Nice summary of the ASA adjudication. I think that the study Lloyds Pharmacy cite is:

    Emberlin JCC, Lewis RAA. Pollen challenge study of a phototherapy device for reducing the symptoms of hay fever. Current medical research and opinion. 2009 May;Available from: http://dx.doi.org/10.1185/03007990903024699

    I’ve had a copy for a while, but never got around to blogging it. Still, the ASA’s expert has done a better job than I ever could!

  2. Just been re-reading this. They are still selling this gizmo; their current site says:

    “The Lloydspharmacy Hayfever reliever works by using phototherapy of two wavelengths. One wavelength promotes increased blood flow circulation and the second wavelength (red light therapy) suppresses the cells that release histamine and reduces inflammation, thereby relieving the irritating symptoms of hayfever.”

    Err.. no doubt there, then.

    And:

    “A study conducted by the National Pollen and Aerobiology Research the National Pollen and Aerobiology Research Unit, and published online by the Current Medical and Research Opinion concluded that the Lloydspharmacy Hayfever Reliever significantly reduced the main symptoms of hayfever – sneezing, runny nose, watering eyes and an itchy palate. In fact it’s so effective some people may feel a reduction in hayfever symptoms after using it just a few times. This device has been clinically proven to help to reduce susceptibility to some types of pollens, making it very useful for the severe pollen seasons forecasted over the next ten to fifty years.”

    [This is the Emberlin & Lewis study Adrian Gaylard linked to above]

  3. Thanks for that Dr Aust.

    I’ve just asked them whether it is CE marked and what EU Directives it meets — this could be something to be referred to the MHRA.

    “…the severe pollen seasons forecasted over the next ten to fifty years.”

    And I thought they had difficulty telling us what the pollen count was going to be like at the weekend…

Leave a Reply to Zeno Cancel reply

Your email address will not be published. Required fields are marked *


The reCAPTCHA verification period has expired. Please reload the page.

This site uses Akismet to reduce spam. Learn how your comment data is processed.