Zeno's Blog

Those champions of woo busting strike again. More ASA adjudications — to be formally published tomorrow — showing what is and isn’t acceptable as evidence required to substantiate claims made.

In an adjudication against a weight loss ‘system’, the ASA again emphasise the necessity for robust evidence:

The ASA noted the system was based on a theory that some foods could encourage the body to metabolise faster. We also noted that although IGEA referred to some studies, we had not seen robust evidence, such as peer reviewed or clinical trial evidence carried out on people, to show that there were “fat-burning foods”. We concluded that the claim was misleading.

Note yet gain where the ASA set the bar for evidence for claims made.

Note also that something based on a ‘theory’ alone isn’t good enough. Interestingly, the ASA also ruled:

We were also concerned that the ad did not make clear how the Bodytrim method worked…

This has obvious parallels with claims frequently made by chiropractors: they claim all sorts of conditions are caused by ‘subluxations’, but fail to give any clear substantiation of how chiropractic actually can correct/improve/cure/heal/help these conditions, other than bland statements that it does. It would appear that this would not satisfy the ASA.

In a second adjudication, the ASA rule:

We noted IntraMed did not hold any studies specifically on the advertised titanium stockings to demonstrate their effects on people who wore them, and considered such evidence would be necessary to substantiate the ad’s efficacy claims.

Evidence must be held that demonstrates that the specific product must have been trialled on the people it was being advertised or sold to, otherwise it is deemed not to have been substantiated. A parallel here might be that even if some chiropractic manipulation had been tested on adults, it would fail this substantiation test if advertised for, say, babies and small children.

In another one about a misleading advert for an omega-7 supplement for vaginal dryness in menopausal women:

We noted the study considered the “Effects of seabuckthorn oil capsule” on 25 women aged from 37 to 66, with typical symptoms of itching, burning, liquid secretion and dryness in the genital tract mucosa, but that 13 of the women were post menopausal, and considered that the sample size was therefore limited.

Small studies aren’t sufficient, particularly when over half the participants were not relevant!

In an adjudication against an advert for an oil that claimed to help reduce stretch marks in pregnant women, the advertisers submitted some surveys as their plethora of evidence for the claims they made:

The ASA took expert advice in relation to the evidence which included the three studies submitted in support of each claim. We noted the results of the two surveys but did not consider them relevant to substantiating the efficacy of the product.

The advertisers also claimed:

They believed the scale of the product’s success among consumers and pharmacists alike was in itself evidence in support of the claims in the ad; it would not be so successful if it was not effective.

Simply an ad populum fallacy and shows a complete and total misunderstanding of — or contempt for — what constitutes evidence. How on earth did they think that was ever going to be good enough?

Interestingly, the advertisers brought up a single blind trial that supported the efficacy of a different (but presumably similar) product. However, the ASA’s expert noted that:

…other published, peer-reviewed studies suggested such products were not effective. He therefore considered that, the claim constituted a breakthrough claim for which a high standard of evidence was required.

So submitting one trial that might support your claims while ignoring others that didn’t was cherry picking and not acceptable. Where have we heard that before?

However, even that study was problematic to the ASA’s expert:

…it suffered from not being double blinded and there was no control for a placebo effect. Although Union-Swiss believed a single-blinded trial was sufficient to support the claim, the expert maintained that a double-blind trial controlled for some of the potential pitfalls that could arise with single or non-blinded trials and was therefore a more robust methodology. He said the mean results appeared to have been skewed by a few subjects recording relatively large changes. The majority of scars showed no change or worsening after 12 weeks. The expert concluded that the evidence was not sufficient to support a breakthrough claim.

Double-blinding is important. Results skewed by outliers are not acceptable.

On evidence submitted for another claim, the ASA ruled:

While we noted the study results showed a trend towards a slight effect, we understood that those results were obtained by way of a self-assessment study. We noted, however, the study was single blinded and there was no control for a placebo effect. In view of that, we considered that the study was insufficiently robust to support the claim and concluded that it could mislead.

Yet again, wholly insufficiently robust evidence.

The expert said the magnitude of change recorded was small and the number of subjects demonstrating no change at all was high; the ‘best’ result was from the self-assessment analysis which was not a reliable indicator in a single-blinded trial. He concluded that the evidence was not sufficiently robust to support the claim.

…

Because that inference was unsupported by evidence, we concluded that it could mislead.

Customer satisfaction surveys are just not good enough.

The last one for today: an advert for yet another weight loss management product where the advertiser got into a bit of a hissy fit:

Goldshield stated that it believed that the requirement of rule 8.4.2 was for advertisers to be able to substantiate claims in ads for slimming products but, if no claims were made, the rule did not apply. It believed that, because it did not make a claim, the advertisement did not breach the rule.

“We didn’t make any claims, so we can’t have broken your rules!” Hmmm… The product was called ‘LIPObind’ and the ASA ruled that even the product name on its own constituted a claim:

We noted that the Notes to rule 5.2.1 (Evidence) advised that a claim could be implied or direct, written, spoken or visual and that the name of a product or service may in itself be regarded as a claim.

The ASA has already been very clear on what constitutes a claim.

Yet another bad week for woo pushers.