Hindsight bias: How we overestimate our prediction abilities
People tend to overestimate their own predictive power when events have already occurred. This is commonly known as “hindsight bias”, or the “knew it all along” phenomenon.
It is easy to be wise after the event.” – Arthur Conan Doyle
Your baseball team is down by 2 runs in the 8th inning, but your power hitting 1st baseman is coming up to bat in the 9th and he is due for a big hit, so you casually remark to a friend that you’ve “got a good feeling about this.” Sure enough, your team wins the game. You knew there was something inevitable about that comeback. Remarkable.
Loathing ambiguity, the mind is predisposed, after the fact, to invent stories that fit unexpected outcomes. Your baseball team won? You remember thinking how that 1st baseman was due.
You are reading a mystery novel, and there are three suspects: Joe, Mary and Steve. Joe seems nervous. Mary seems evasive. Steve seems shifty. At the end of the novel, it is revealed that Steve is the culprit. You recall your thoughts on Steve’s shiftiness. You were able to foresee his guilt very early in the story.
The mind has a hard time reconstructing views it may have held in the past, which are difficult to access, and has a tendency, instead, to replace these with newer, more recently acquired states of knowledge. There were 3 suspects? Once you became aware of who the culprit was, you selectively focus on your specific thoughts about about the culprit.
Stories about the past, when they are altered to fit our observations, feed overconfidence in our ability to make predictions in the future, reinforcing the illusion of our predictive skill. Given these conditions, it is naturally difficult for us to assess how good we really are at making predictions.
Psychological research from early in 1970s explored hindsight bias, and found powerful evidence of its existence.
Game: Tie the strings together!
Consider the following problem. The picture below depicts a closed room, with 2 strings hanging from the ceilings, a ladder, and a wrench. You are asked to tie the ends of the strings together, but you cannot reach both of them at the same time. Given the objects at your disposal, how do you do it?
Now let’s review the solution: you can tie the wrench to the end of one string and make it swing like a pendulum. You climb the ladder to the other one string and when the wrench swings close enough, you can grab it and tie two strings together. Pretty straightforward, right?
This game was designed by Dennis Crouch, a professor at the University of Missouri School of Law. What Crouch did was divide students in his class into two groups. Students in the first group were given the question, and then were asked to rate the nonobviouseness of the solution, rated on a scale of 1~10, where 1=obvious and 10=nonobvious. Students in the second group were given BOTH the question AND the solution, and then were also asked to rate the nonobviouseness of the solution (again, on a 10 point scale).
The class exhibited a clear hindsight bias. Students who were not given the solution chose relatively higher nonobviouse rates, while students who were given solution chooses relatively higher obvious rates, since they knew it all along.
The danger of hindsight bias is that it reduces our ability to learn from our own mistakes. If we are always overconfident by virtue of our hindsight, and can’t acknowledge our failures at prediction, we are ignoring valuable feedback that can help us identify when we are being overconfident. If we don’t understand our overconfidence, we can’t address and remedy it.
Applications in Finance:
In a great master’s thesis, “Behavioral Biases of Investment Advisors – The Effect of Overconfidence and Hindsight Bias,” by Anita Seppala, the author conducts a series of fascinating experiments which illuminate how hindsight bias can trip up professional advisers.
In a Finnish study, subjects, who were financial advisers, were given some financial information, and then were asked to predict the future return of an asset. In the study, 83% predicted the asset would increase in value, 6% predicted no change, and 11% predicted the asset would decline in value.
It was then revealed to the advisers that the asset had subsequently declined in value. Now the advisers were asked to remember, with the benefit of hindsight, whether they had successfully predicted the drop in price. The results?
A whopping 31% claimed they had successfully predicted the decrease (recall that only 11% had actually made the correct call ahead of time). This means that a full 20% said they had correctly anticipated the movement of the stock, when in fact they had made the opposite call, which was wrong!
Clearly at least 20% of the advisers had an unrealistically high opinion of their judgment in this case. They overestimated their abilities, and underestimated how surprised they were by the direction the asset moved. This might lead to excessive risk-seeking behavior by financial advisers, who might exhibit high confidence in themselves and their opinions, even when such confidence might be completely unwarranted in light of their true skills.
These tendencies likely hold, on some level, for investors generally. And if you want to eliminate the pernicious effects of your cognitive errors, you must be careful not to systematically reinterpret the past and falsify your investing acumen.
- Dagmar Stahlberg and Frank Beller, We knew it all along: Hindsight Bias in Groups, Organizational Behavior and Human Decision Processes, Volume 63, Issue 1, July 1995, Pages 46–58