Thinking Better, Part 2: Confirmation Bias

"Smart people believe weird things because they are skilled at defending beliefs they arrived at for nonsmart reasons." - Michael Shermer

"I know that most men, including those at ease with problems of the greatest complexity, can seldom accept the simplest and most obvious truth if it be such as would oblige them to admit the falsity of their conclusions which they have so proudly taught of others, and which they have woven, thread by thread, into the fabrics of their life." - Leo Tolstoy

"The most difficult subjects can be explained to the most slow-witted man if he has not formed any idea of them already, but the simplest thing cannot be made clear to the most intelligent man if he is firmly persuaded that he knows already, without a shadow of a doubt, what is laid before him." - Leo Tolstoy

The term confirmation bias refers to the tendency to search for or interpret new information in a way that confirms our beliefs, while avoiding information or interpretations that conflict with our beliefs.  In fact, numerous studies have shown that we place too much value on confirmatory information.  All of the following are terms for, or forms of, confirmation bias:

• Belief bias
• Belief preservation
• Belief overkill
• Hypothesis locking
• Non-falsifiable hypothesis
• Polarization effect
• Positive bias
• Tolstoy syndrome
• Selective thinking
• Myside bias
• Morton's demon

Here is one example of confirmation bias.  Let's say that you believe during a full moon that there is an increase in emergency room admissions.  You take notice of admissions during a full moon, but are inattentive to the moon when admissions occur during other nights of the month.  Your tendency to do this over time unjustifiably strengthens your belief in a relationship between the full moon and accidents.

There is no lack of examples of confirmation bias in the realm of health.  For example, opponents of the artificial sweetener aspartame (or Nutrasweet) often cite a 1996 study showing a link between increased brain tumor frequency and aspartame consumption.  From 1980 to 1985, there was a rapid rise in aspartame consumption in the U.S.  In 1985, there was a jump in brain tumor frequency.

Aspartame critics cite this as evidence that increased aspartame consumption caused an increase in brain tumors.  Ignoring the fact that the critics are commiting the post hoc, ergo propter hoc fallacy, the critics are committing confirmation bias by selectively choosing the time period of 1975 onward.  When we expand our time frame, starting at 1973, we get a different story:

The fact is, brain tumor frequency was already increasing before aspartame was introduced to the market.  Despite the flaws in this study, and despite a follow-up study showing no relationship between aspartame and brain tumors, aspartame opponents continue to cite this study as evidence of aspartame's supposed danger.

There are many other examples of confirmation bias in the world of health.  For example, Dr. Wayne Westcott wrote an article supporting his belief that you would increase your metabolic rate by 50 calories for every pound of muscle you added to your body.  However, if you were to read the research Dr. Westcott cited to support his belief, you would find it doesn't support his belief at all.  One of the studies he cited did not show an increase in muscle mass.  And while there was an increase in fat-free mass (remember that fat-free mass and muscle are not the same thing), the increase in fat-free mass did not correlate with the increase in metabolic rate.  In the other study he cited, there was no correlation between the increase in fat-free mass and metabolic rate.  Yet Westcott claimed a relationship existed.

In my review of chapter 14 of Good Calories, Bad Calories, I pointed out how the author relied heavily on old scientific data to support his thesis, despite the fact that more well-designed and more recent studies refuted his thesis.  Another example of confirmation bias is in a debate I had with an individual regarding carbohydrate.  This individual strongly believed that a diet high in carbohydrate would result in fat gain and obesity.  I pointed out that the traditional Okinawans consumed a high carbohydrate diet and did not have high obesity rates.  This individual then claimed it was  a low-carbohydrate diet because it was a low-calorie diet.  I then showed how the carbohydrate intake was an average of 379 grams per day.  He then shifted the goalposts and said the carbohydrates were mostly fibrous and indigestible.  I then pointed out that this wasn't true and that the staple food was the sweet potato.  He then shifted the goalposts again and said the data was from 1949.  By this time, I was tired of debating him, but the fact is that it was based off of 6 decades worth of data.  This individual continued to shift the goalposts and come up with new ad hoc rationalizations as to why the Okinawan diet didn't really refute his preconceived belief regarding carbohydrate.

In another discussion I had with this same individual, he had stated how the insulin response from protein was due to the protein being converted to glucose (sugar) first, which would then stimulate your pancreas to produce insulin.  I gave him a study showing amino acids (the building blocks of protein) to directly stimulate the pancreas to produce insulin.  However, this individual continued to reassert his same belief.

Polarization Effect

One effect  of confirmation bias is known as the polarization effect.  This is where mixed or neutral evidence is used to bolster an already established and biased point of view.  A study was performed in 1979 on this effect.  Students were given a series of studies in favor or against capital punishment.  Students felt the studies in favor of their point of view were superior to the contradictory studies.  However, all of the studies had the exact same methodology, only different results.

The polarization effect can also be found in the world of health.  For example, when the Women's Health Initiative study results on diet were published, individuals of different dietary philosophies used this study to bolster their own point of view:

Dean Ornish:  "It's critically important to understand the limitations of this study and continue to advise our patients about the benefits of low-fat, whole-foods nutrition."

Atkins supporter:  "The myth of the health low-fat diet seems to have crumbled for good."

An Exercise

The English psychologist Peter Wason, who coined the term "confirmation bias", published an experiment in 1960 where he challenged subjects to identify a rule applying to triples of numbers.  I have used this same experiment on my audience when lecturing on confirmation bias.  I would give everyone a sheet of paper that looks like the following:

I would ask the audience members to guess what rule I was using to come up with the number sequence, and write it in the third column.  I would also ask them how certain they were of the rule, and write that in terms of a percentage (e.g., 50% certain) in the last column.  I would have the audience go down each row, and write a new sequence of numbers that they thought conformed to the rule.  I would go around and write a smiley face if their sequence of numbers fit the rule, or a frown face if the sequence didn't fit the rule.  Often, a typical series of responses would look like this:

Many people would end up 100% certain that the rule was "Counting up by two's".  However, this was not the rule.  As shown in the example above, the individual only sought confirmatory information regarding the rule.  The person thought the rule was "counting up by two's" from the beginning, and continued to test sequences of two.  But the person never made an attempt to falsify the rule with other number sequences.  When we consistently attempt to falsify the rule, we end up with the correct answer of "counting up":

This exercise illustrates the importance of constantly challenging our own beliefs, and leaving our ideas open to falsification when the evidence dictates.  This is the way science is supposed to work...by formulating hypotheses, and then attempting to test and falsify those hypotheses.  Any hypothesis will generate a set of predictions.  If one or more of those predictions do not hold true, then we consider the hypothesis falsified.  However, it is human nature to often seek out confirmatory information, which is why we often cling to certain beliefs even when the evidence would dictate otherwise.

"It is the peculiar and perpetual error of the human understanding to be more moved and excited by affirmatives rather than negatives" - Francis Bacon