The Bullet Holes That Weren't There
Survivorship bias is one of the most useful thinking tools a kid can own, and once you see it you spot it everywhere: success stories, 'they don't make them like they used to', even falling cats.
- • The famous version is simplified. Wald did not just say 'armour the opposite spots': his group issued 8 technical memos on damaged aircraft, estimating how likely a hit to each area was to bring a plane down.
- • Accounts differ on whether it was Navy or Army planes, and the well-known picture of a plane covered in red dots is a modern illustration, not Wald's data.
- • A 1987 study of 132 fallen cats found those falling more than 6 floors had fewer injuries. Survivorship bias is one explanation; another is that cats relax at top speed, and a 2004 study found higher falls did cause worse injuries.
Planes come home from battle peppered with bullet holes. Where would you add armour? Vote now.
In the Second World War, military officers faced exactly this puzzle. Armour is heavy, so each plane could carry only a little. Their plan sounded obvious: bolt it on where the holes are.
Then Abraham Wald, a mathematician who had fled the Nazis to New York, asked an awkward question. Which planes are you looking at?
Follow his chain. Step one: the officers only saw planes that made it back. Step two: a plane hit in the engine or fuel system usually didn't. It went down over enemy land or into the sea. Step three: so the holes on returning planes only showed where a plane can be hit and keep flying. The engines looked clean because the planes hit there were missing.
The safest-looking spots were the deadliest. Wald worked out how to estimate where the lost planes were hit, and the engines got the armour.
This trap is called survivorship bias: you study the winners while the losers vanish from your data. It may even explain a cat mystery. Cats falling more than 6 floors seemed less hurt than cats falling lower. Or did the worst-hurt cats simply never reach the vet?
So, where are the missing planes in your life?