Thinking Better, Part 2: Confirmation Bias

 

Smart people believe weird things because they are skilled at defending beliefs they arrived at for nonsmart reasons.” – Michael Shermer

I know that most men, including those at ease with problems of the greatest complexity, can seldom accept the simplest and most obvious truth if it be such as would oblige them to admit the falsity of their conclusions which they have so proudly taught of others, and which they have woven, thread by thread, into the fabrics of their life.” – Leo Tolstoy

The most difficult subjects can be explained to the most slow-witted man if he has not formed any idea of them already, but the simplest thing cannot be made clear to the most intelligent man if he is firmly persuaded that he knows already, without a shadow of a doubt, what is laid before him.” – Leo Tolstoy

The term confirmation bias refers to the tendency to search for or interpret new information in a way that confirms our beliefs, while avoiding information or interpretations that conflict with our beliefs.  In fact, numerous studies have shown that we place too much value on confirmatory information.  All of the following are terms for, or forms of, confirmation bias:

  • Belief bias
  • Belief preservation
  • Belief overkill
  • Hypothesis locking
  • Non-falsifiable hypothesis
  • Polarization effect
  • Positive bias
  • Tolstoy syndrome
  • Selective thinking
  • Myside bias
  • Morton’s demon

Here is one example of confirmation bias.  Let’s say that you believe during a full moon that there is an increase in emergency room admissions.  You take notice of admissions during a full moon, but are inattentive to the moon when admissions occur during other nights of the month.  Your tendency to do this over time unjustifiably strengthens your belief in a relationship between the full moon and accidents.

There is no lack of examples of confirmation bias in the realm of health.  For example, opponents of the artificial sweetener aspartame (or Nutrasweet) often cite a 1996 study showing a link between increased brain tumor frequency and aspartame consumption.  From 1980 to 1985, there was a rapid rise in aspartame consumption in the U.S.  In 1985, there was a jump in brain tumor frequency.

Aspartame critics cite this as evidence that increased aspartame consumption caused an increase in brain tumors.  Ignoring the fact that the critics are commiting the post hoc, ergo propter hoc fallacy, the critics are committing confirmation bias by selectively choosing the time period of 1975 onward.  When we expand our time frame, starting at 1973, we get a different story:

The fact is, brain tumor frequency was already increasing before aspartame was introduced to the market.  Despite the flaws in this study, and despite a follow-up study showing no relationship between aspartame and brain tumors, aspartame opponents continue to cite this study as evidence of aspartame’s supposed danger.

There are many other examples of confirmation bias in the world of health.  For example, Dr. Wayne Westcott wrote an article supporting his belief that you would increase your metabolic rate by 50 calories for every pound of muscle you added to your body.  However, if you were to read the research Dr. Westcott cited to support his belief, you would find it doesn’t support his belief at all.  One of the studies he cited did not show an increase in muscle mass.  And while there was an increase in fat-free mass (remember that fat-free mass and muscle are not the same thing), the increase in fat-free mass did not correlate with the increase in metabolic rate.  In the other study he cited, there was no correlation between the increase in fat-free mass and metabolic rate.  Yet Westcott claimed a relationship existed.

In my review of chapter 14 of Good Calories, Bad Calories, I pointed out how the author relied heavily on old scientific data to support his thesis, despite the fact that more well-designed and more recent studies refuted his thesis.  Another example of confirmation bias is in a debate I had with an individual regarding carbohydrate.  This individual strongly believed that a diet high in carbohydrate would result in fat gain and obesity.  I pointed out that the traditional Okinawans consumed a high carbohydrate diet and did not have high obesity rates.  This individual then claimed it was  a low-carbohydrate diet because it was a low-calorie diet.  I then showed how the carbohydrate intake was an average of 379 grams per day.  He then shifted the goalposts and said the carbohydrates were mostly fibrous and indigestible.  I then pointed out that this wasn’t true and that the staple food was the sweet potato.  He then shifted the goalposts again and said the data was from 1949.  By this time, I was tired of debating him, but the fact is that it was based off of 6 decades worth of data.  This individual continued to shift the goalposts and come up with new ad hoc rationalizations as to why the Okinawan diet didn’t really refute his preconceived belief regarding carbohydrate.

In another discussion I had with this same individual, he had stated how the insulin response from protein was due to the protein being converted to glucose (sugar) first, which would then stimulate your pancreas to produce insulin.  I gave him a study showing amino acids (the building blocks of protein) to directly stimulate the pancreas to produce insulin.  However, this individual continued to reassert his same belief.

Polarization Effect

One effect  of confirmation bias is known as the polarization effect.  This is where mixed or neutral evidence is used to bolster an already established and biased point of view.  A study was performed in 1979 on this effect.  Students were given a series of studies in favor or against capital punishment.  Students felt the studies in favor of their point of view were superior to the contradictory studies.  However, all of the studies had the exact same methodology, only different results.

The polarization effect can also be found in the world of health.  For example, when the Women’s Health Initiative study results on diet were published, individuals of different dietary philosophies used this study to bolster their own point of view:

Dean Ornish:  “It’s critically important to understand the limitations of this study and continue to advise our patients about the benefits of low-fat, whole-foods nutrition.”

Atkins supporter:  “The myth of the health low-fat diet seems to have crumbled for good.”

An Exercise

The English psychologist Peter Wason, who coined the term “confirmation bias”, published an experiment in 1960 where he challenged subjects to identify a rule applying to triples of numbers.  I have used this same experiment on my audience when lecturing on confirmation bias.  I would give everyone a sheet of paper that looks like the following:

I would ask the audience members to guess what rule I was using to come up with the number sequence, and write it in the third column.  I would also ask them how certain they were of the rule, and write that in terms of a percentage (e.g., 50% certain) in the last column.  I would have the audience go down each row, and write a new sequence of numbers that they thought conformed to the rule.  I would go around and write a smiley face if their sequence of numbers fit the rule, or a frown face if the sequence didn’t fit the rule.  Often, a typical series of responses would look like this:

Many people would end up 100% certain that the rule was “Counting up by two’s”.  However, this was not the rule.  As shown in the example above, the individual only sought confirmatory information regarding the rule.  The person thought the rule was “counting up by two’s” from the beginning, and continued to test sequences of two.  But the person never made an attempt to falsify the rule with other number sequences.  When we consistently attempt to falsify the rule, we end up with the correct answer of “counting up”:

This exercise illustrates the importance of constantly challenging our own beliefs, and leaving our ideas open to falsification when the evidence dictates.  This is the way science is supposed to work…by formulating hypotheses, and then attempting to test and falsify those hypotheses.  Any hypothesis will generate a set of predictions.  If one or more of those predictions do not hold true, then we consider the hypothesis falsified.  However, it is human nature to often seek out confirmatory information, which is why we often cling to certain beliefs even when the evidence would dictate otherwise.

It is the peculiar and perpetual error of the human understanding to be more moved and excited by affirmatives rather than negatives” – Francis Bacon

  18 Responses to “Thinking Better, Part 2: Confirmation Bias”

  1. Another awesome post, James.

      

  2. “I pointed out that the traditional Okinawans consumed a high carbohydrate diet and did not have high obesity rates.” Furthermore, there is a plethora of scientific research data contradicting the carbohydrate obesity hypothesis.

    The confirmation bias is strong in every field study.

    This article should be required reading for everyone.

    Couple of references that offer detailed expamples and explainations for confirmation bias:

    How We Know What Isn’t So by Thomas Gilovich
    Why People Believe Weird Things by Michael Shermer

    J Hale

      

  3. Great post. I would add, however, that people also tend NOT to pay attention to detail and in so doing believe something to be true when perhaps it is not true.

    As an example, in your aspartame example you state, “Proponents of the artificial sweetener aspartame often cite a 1996 study showing a POSSIBLE link between increased brain tumor frequency and aspartame consumption.”

    You also state, “Aspartame critics cite this as evidence that increased aspartame consumption MAY have caused an increase in brain tumors.”

    I think the main point is that a certain group of people believe there MAY be a POSSIBLE link between aspartame and brain tumors. This does NOT mean there IS a link, only that there could be one.

    Therefore, the language is clearly suggesting that there could be something going on but to know this for certain additional research is required and at least according to the way you phrased it, nobody is actually stating that there IS a link. That’s jumping to conclusions.

    At the end of the day, paying attention to detail, i.e., the actual language used in research (and elsewhere), would do everybody a lot of good. Sadly, even someone with a doctorate, such as Westcott, apparently forgot this very important point.

    AH

      

    • I think the main point is that a certain group of people believe there MAY be a POSSIBLE link between aspartame and brain tumors. This does NOT mean there IS a link, only that there could be one.

      I probably should have worded my statement differently in my article. If you were to do a web search to see what the critics say about aspartame, you would find that they use the 1996 study as evidence of a DEFINITIVE link, and they will claim with certainty that it caused the increase in brain tumors. There is, in fact, a very large amount of aspartame alarmism on the web.

        

    • I changed the wording now to more accurately reflect how aspartame critics actually view the study

        

  4. I see, James. I didn’t mean to criticize your work, though.

    By the way, I’ve been trying to find a way to have your site alert me when comments are posted on a particular blog/thread that I’m interested in. I wasn’t able to find it. If you indeed don’t have that available, perhaps you’d consider adding such a feature (if possible).

    AH

      

  5. I’m not so sure the shifting goal posts tactic is an example of confirmation bias, so much as it is evidence of a deliberate attempt at misrepresentation. Confirmation bias generally results in ignoring information that goes counter to one’s (predetermined) beliefs. But moving the goal posts indicates some acknowledgement that valid counter-evidence exists. The topic is changed so as to explain away this acknowledged contradiction to their firmly held belief. I am not sure whether those who engage in this tactic are being deliberately misleading and disingenuous, or if they are so deeply entrenched in their beliefs that they don’t see the verbal contortions required to defend it.

      

    • CarbSane,

      Yes, the whole “shifting the goalpost” phenomena can be difficult to classify. I put it under the category of confirmation bias, because I consider goalpost shifting as a method for creating and supporting a non-falsifiable hypothesis. Goalpost shifting is very similar to the tactic that ESP believers use when confronted with studies that fail to support the existence of ESP. They will state, “The hostile thoughts of the researchers are interfering with ESP transmission, which is why the experiments fail to support ESP.” In other words, they will come up with every sort of ad hoc excuse imaginable as to why the experiments fail to show that ESP exists. This is very similar to examples described in my article.

        

  6. Another fantastic revisit of basic principles. The experiment is a great example. Thanks!

      

  7. If I could change the world … I would teach this stuff in the 5th grade!!! Much more useful in life than knowing things like all the state capitols (which I can still recite, not that anyone ever asks).

      

 Leave a Reply

(required)

(required)

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>