Extraordinary Claims Require Extraordinary Evidence

Sometimes I don't think people understand what constitutes evidence, and what it means to support a claim with evidence.  Sometimes I don't think people understand the reason to be skeptical of a product.

Sometimes I don't think people understand what this blog is about.  Some people seem to think that this blog is about exposing scams, but that's only partly true.

 The purpose of this blog is stated very clearly over in the right sidebar.  Specifically, it states that the purpose "is to investigate various claims and products in the industry, and challenge their evidence basis."

Just because I challenge the evidence presented for a product or claim, doesn't necessarily mean I think the product is a scam.  It only means there is not enough evidence to support the claims being made about the product. 

So what makes me call something B.S.?

Here's where I call B.S...if the level of evidence being provided for a product doesn't match the claims being made about that product.

Let me repeat that.

 I consider something B.S. if the level of evidence being provided for a product doesn't match the claims being made about that product.

Logic dictates that extraordinary claims require extraordinary evidence.

If I tell you that I went to the grocery store today, you probably wouldn't demand much evidence for me to support that.  Yes, I may be lying, but since people go to the grocery store every day, it is an ordinary claim and thus there is little reason for you to be skeptical of my claim.

Now, I tell you that I went to the grocery store and saw Bigfoot

Suddenly, my claim is not so believable.  People don't go to the grocery store and see Bigfoot every day.  In fact, you never hear about people going to the grocery store and seeing Bigfoot.  This is an extraordinary claim.  Thus, the level of evidence necessary to support that claim is beyond the level of evidence necessary to support a claim I went to the store.  You would require extraordinary evidence...other eyewitnesses, video, photos, hair samples, etc.

This holds true in the field of health and wellness.  This field is full of products with bold claims, but not bold evidence to support those claims.  The evidence generally doesn't come even close to the claims being made.

In other words, in the health and wellness industry, extraordinary claims are often accompanied by only ordinary evidence.

If I told you my Bigfoot story, you wouldn't believe me because I wouldn't have sufficient evidence.  Yet, when it comes to health and wellness, people are willing to believe things on the flimsiest of evidence.

Take the product Cobroxin, which I discussed in my last blog post.  The claims being made about this product are quite extraordinary....that it causes significant pain relief for moderate to severe chronic pain, across a wide variety of medical conditions.  And this is all without major side effects.

That is quite a bold claim.  But, the level of evidence provided for that claim doesn't match up.  There are no published, double-blind, placebo controlled trials, published in major journals on Cobroxin.  The only evidence provided are numerous testimonials, as well as outdated, poorly designed studies on cobra venom components in mainly Chinese journals (which are notorious for bad science).

Compare that to your typical OTC or prescription pain reliever such as Tylenol.  There are numerous double-blind, placebo-controlled trials on OTC and prescription pain relievers, many of which are published in major journals.  Also, the claims being made for these products are certainly not as bold as that made for Cobroxin.  There are no magical claims of significant pain relief across a large number of medical conditions.  And there are also no unrealistic claims being made about a lack of side effects.

The fact is, the evidence provided for standard OTC and prescription pain relievers is quite good, and matches well with the claims that are made about these products.  Yet the Cobroxin believers claim Cobroxin is even better, on a lower standard of evidence.

And that's why Cobroxin goes on my B.S. list....because the level of evidence doesn't meet the claims being made.

 So my question for all the Cobroxin believers....why should Cobroxin be held to a lower standard evidence than OTC and prescription pain relievers, when the claims being made about it exceed the claims being made about OTC and prescription pain relievers?

And that same question goes for any health product out there.  Too often, health products are held to much lower standards of evidence than they should be.

But that will all change with this blog.

4 Responses to “Extraordinary Claims Require Extraordinary Evidence

  • You mean that wasn’t really Bigfoot that I saw at the Wal-Mart yesterday? I thought he had come in to buy some of the tabloids with pictures of him. Next time I’ll ask for his autograph, so I can give you some evidence.

    Jim Purdy

  • Hmm. I’ve been discussing E.C.R.E.E. with some other skeptics. I think the major pitfall is that what constitutes “extraordinary” is very subjective.
    Another thing if you look at Bayesian inference is the marginal probability
    http://en.wikipedia.org/wiki/Bayesian_inference
    P(H|E) = P(E|H)*P(H)/P(E)
    P(H) is called the prior probability. If this is low we would say a hypothesis is extraordinary.
    P(E | H) is called the conditional probability of seeing the evidence E if the hypothesis H happens to be true. If this is high the evidence is extraordinary
    P(E) is called the marginal probability of E: the a priori probability of witnessing the new evidence E under all other possible hypotheses. This is basically the possibility of some other explanation.
    From the equation we can see even if P(E|H) is low and P(H) is low (extraordinary claims with poor evidence) if P(E) is low enough the overall probability can still be pretty high. To use the bigfoot example, if I was to examine your testimony and I knew there could be no other explanation for the figure you saw.(I am sure you were not lying or hallucinating or saw something else.) Then overall the probability might be pretty high.

    This is very pedantic and has nothing to do with health (just good reasoning) but I’ll post it anyway.

Trackbacks & Pings

Leave a Reply Text

Your email address will not be published. Required fields are marked *