About Falling Cats
What is survivorship bias?
If you’ve been following our last few blog posts, you might have wondered what’s up with the “Falling Cats” thing. There is a brief explanation on the Falling Cats landing page, but for those who are interested, here’s a longer one…
Studies of how cats fall provide an interesting way to think about success, failure, and how survivorship bias might lead us to erroneous conclusions. Basically, in the late eighties, some scientists looked at the survival rates and injuries of cats after falling from various heights, and they noticed something odd. The higher the fall, the more likely the cat was to survive, and with less severe injuries. Based on these findings, they concluded that when cats fall from around six or seven stories or higher, they hit a terminal velocity, and they have time to relax. They spread out into the position of a flying squirrel and, in effect, paratroop to safety.
Many people still believe this conclusion to be the best explanation of this phenomenon. But, there is another possible explanation.
What if the higher the fall, the more likely the cat is to die immediately upon impact? Wouldn’t that also make it less likely that anyone would bother bringing it to the hospital to be counted in the study? If so, the results of the 1987 study are skewed because the researchers only looked at the cats that survived long enough to make it to the hospital. But what about those dead cats? What happens when we take them into account?
Whether we are looking at cats, brands, products, ideas, startups, or just about anything else, we are often looking at the survivors, the success cases, the ones that made it far enough for us to take notice. In those cases, we may be missing out on a lot of important information by ignoring the failures. And if we start to draw conclusions based on the survivors, we may tend toward over-optimism, believing that more brands, products, ideas, startups, etc. are successful than actually are.
We might also make the commonly made mistake of assigning causation to correlations and giving too much significance to traits of the survivors that are not particularly relevant to their success, especially if the failures share the same traits. For example, if all the cats do the flying squirrel maneuver, even the dead ones, it might be something else (like weight, or age, or breed) that accounts for the difference between the cats that make it and the cats that don’t.
It might even just be a matter of luck.
So, how do we avoid survivorship bias?
We need to look for the dead cats. Not literally, of course. But when looking at examples of success, we need to be mindful of the corresponding failures and try to learn from both. Some of the next Falling Cats blog posts will focus on ideas, businesses, and other things that didn’t make it. Let us know if you have any "dead cat" examples that we should take a look at.