Do you ever shout at the TV, radio or newspaper when a reporter or vested interest representative quotes some superficial statistics and draws completely wrong conclusions. With increasing age, I seem to be doing this more often!  This problem isn’t confined to the media. Misinterpreting simplistic numbers because of  poor qualitative knowledge is a common business mistake.

I’ve often warned people about this risk, but I often wished I had a simple example to illustrate the problem.  Thanks to Tim Harford I’ve now got one.  Britain’s Daily Mail shouted this headline:

Cracked it! Woman finds six double yolk eggs in one box beating trillion-to-one odds

The Daily Mail reported a woman’s astonishment at cracking six eggs to find them all double-yolked.  If the odds of a hen laying an egg with a double yolk is 1 in 1000, the Mail argues, superficially correct, then the chance of two consecutive eggs being double-yolked is 1 in a thousand thousand, and so on, meaning the odds of all six eggs in the carton being double-yoked is 1 in 1 trillion in traditional British nomenclature (quintillion if you use the more prevalent American scale).  The report sparked a flurry of paid media and social media comment on the phenomenon.  Some people argued that the statistics must be wrong, because they’d also, and more than once, bought cartons full of double-yolk eggs.  This kicked off speculation on a variety of causes, eg. flock genetics - extremely unlikely given the way the egg industry operates (high volume hatcheries are separate from production farms).

The answer is more prosaic.  Egg industry workers can spot double-yolk eggs by handling, and put them aside for themselves.  If, due to normal statistical variation, they collect too many, the excess eggs tend to be put back into the packing process together.

Good designers don’t just rely on statistics.  They observe what really goes on.  Toyota uses a technique called “standing in the circle” - literally drawing a circle on the ground and standing in it for hours, silently observing what goes on during a production process and then asking the workers why they did what they did. (That requires mutual trust and  shared desire for knowledge and improvement).  Before and after designing a new visitor experience, Click Suite’s interactive media designers watch what people do without comment, and then seek explanations.

Too many people use superficial statistics without knowledge of the underlying situation.  They make business decisions based on false premises.  As Alexander Pope once wrote in “An Essay on Criticism” (1709):

“A little learning is a dangerous thing; drink deep, or taste not the Pierian spring: there shallow draughts intoxicate the brain, and drinking largely sobers us again.”

But remember, in business as in life, you can’t make an omelette without cracking some eggs!