Algorithm Aversion — Why people don’t follow the model!

//, Uncategorized/Algorithm Aversion — Why people don’t follow the model!

Algorithm Aversion — Why people don’t follow the model!

By | 2017-08-18T17:10:46+00:00 July 29th, 2015|Behavioral Finance, Uncategorized|17 Comments

There are many studies showing that models beat experts, including the meta-study “Clinical versus mechanical prediction: A meta-analysis” by Grove et al. (2000).

However, given this knowledge that models beat experts, forecasters still prefer to use the human (expert) prediction as opposed to using the model.

Why is this?

A recent paper by Dietvorst et al. (2014), titled “Algorithm Aversion: People Erroneously Avoid Algorithms After Seeing Them Err” examines this phenomenon. Here is the abstract of the paper.

Research shows that evidence-based algorithms more accurately predict the future than do human forecasters. Yet when forecasters are deciding whether to use a human forecaster or a statistical algorithm, they often choose the human forecaster. This phenomenon, which we call algorithm aversion, is costly, and it is important to understand its causes. We show that people are especially averse to algorithmic forecasters after seeing them perform, even when they see them outperform a human forecaster. This is because people more quickly lose confidence in algorithmic than human forecasters after seeing them make the same mistake. In 5 studies, participants either saw an algorithm make forecasts, a human make forecasts, both, or neither. They then decided whether to tie their incentives to the future predictions of the algorithm or the human. Participants who saw the algorithm perform were less confident in it, and less likely to choose it over an inferior human forecaster. This was true even among those who saw the algorithm outperform the human.

Here is an interesting example (from the paper) to describe why this may occur:

Imagine that you are driving to work via your normal route. You run into traffic and you predict that a different route will be faster. You get to work 20 minutes later than usual, and you learn from a coworker that your decision to abandon your route was costly; the traffic was not as bad as it seemed. Many of us have made mistakes like this one, and most would shrug it off. Very few people would decide to never again trust their own judgment in such situations.

Now imagine the same scenario, but instead of you having wrongly decided to abandon your route, your traffic-sensitive GPS made the error. Upon learning that the GPS made a mistake, many of us would lose confidence in the machine, becoming reluctant to use it again in a similar situation. It seems that the errors that we tolerate in humans become less tolerable when machines make them.

We believe that this example highlights a general tendency for people to more quickly lose confidence in algorithmic than human forecasters after seeing them make the same mistake. We propose that this tendency plays an important role in algorithm aversion. If this is true, then algorithm aversion should (partially) hinge on people’s experience with the algorithm. Although people may be willing to trust an algorithm in the absence of experience with it, seeing it perform—and almost inevitably err—will cause them to abandon it in favor of a human judge. This may occur even when people see the algorithm outperform the human.

The paper goes on to show that as human confidence in the model increases, humans are more likely to use the model (even if they have viewed the model and have seen it fail). This is described in the figure below.

algorithm aversion

Source: “Algorithm Aversion: People Erroneously Avoid Algorithms After Seeing Them Err” by Dietvorst et al. (2014)

However, note that even when people have “much more” confidence in the models, around 18% of people still use the human prediction!

Our Thoughts:

We are a firm that believes in evidence-based investing and understand that (in general) models beat experts. However, most people prefer the human option after seeing a model (inevitably) fail at some point in time.

How does this tie to investing?

If we are trying to beat the market through security selection, an investor has three options: use a model to pick stocks, use a human , or combine the two. Inevitably, the model will underperform at some point, since no strategy wins all the time (if a strategy never failed, everyone would invest and the edge would cease to exist). When a model underperforms for a certain time period, it does not mean that the model is inherently broken. In fact, the model could have simply failed over some time period, but the long-term statistical “strength” of the model remains intact. Steve, the human stock-picker, will also under-perform at some point; however, Steve can probably tell a better story over a beer as to why he missed the mark on last quarter’s earnings, that pesky SEC investigation, etc. And since drinking a beer with stock-picker Steve is a lot more fun than drinking a beer with an HP desktop, we will probably give Steve the benefit of the doubt.

Successful investors understand that models will fail at times; however, being able to stick with the model through thick and thin is a good strategy for long-term wealth creation. For the rest of us, there’s always stock-picker Steve. Cheers.

  • The views and opinions expressed herein are those of the author and do not necessarily reflect the views of Alpha Architect, its affiliates or its employees. Our full disclosures are available here. Definitions of common statistics used in our analysis are available here (towards the bottom).
  • Join thousands of other readers and subscribe to our blog.
  • This site provides NO information on our value ETFs or our momentum ETFs. Please refer to this site.

Print Friendly, PDF & Email

About the Author:

Jack Vogel, Ph.D., conducts research in empirical asset pricing and behavioral finance, and is a co-author of DIY FINANCIAL ADVISOR: A Simple Solution to Build and Protect Your Wealth. His dissertation investigates how behavioral biases affect the value anomaly. His academic background includes experience as an instructor and research assistant at Drexel University in both the Finance and Mathematics departments, as well as a Finance instructor at Villanova University. Dr. Vogel is currently a Managing Member of Alpha Architect, LLC, an SEC-Registered Investment Advisor, where he heads the research department and serves as the Chief Financial Officer. He has a PhD in Finance and a MS in Mathematics from Drexel University, and graduated summa cum laude with a BS in Mathematics and Education from The University of Scranton.


  1. umair usman July 29, 2015 at 4:08 pm

    how do we know that the efficacy of the model has permanently changed?? ie how often must we update it??

    • Jack Vogel, PhD July 29, 2015 at 4:30 pm

      Great question, unfortunately the answer depends on the model. For example, a model of the growth rate of flip-phone sales would have needed to be adjusted after Apple launched the iPhone; whereas (I imagine this to be true) the sales growth of the local beer store is pretty consistent year to year. Depending on what one is attempting to model will determine how often the model needs to be assessed (and possibly changed).

      • umair usman July 30, 2015 at 9:26 am

        makes good sense .
        Fundamentals , fundamentals, fundamentals…the search never stops.
        Appreciate the reply

        • August 2, 2015 at 3:45 am

          Fundumentals are an indicator but on top of that one needs to consider risk.

          Is a model just a reflection of the risk-reward ratio sought?

          Value, is a vague term when the risk-reward ratio is not clearly defined.

          • umair usman August 2, 2015 at 3:49 am

            this is a good point as well.
            i dont think anyone ever says that Value is the end all, but people assume so thanks to the ‘buffet effect;

          • Wesley Gray, PhD
            Wesley Gray, PhD August 2, 2015 at 1:00 pm


            In the end, any signal–value, momentum, volatility, etc.–needs to serve as a proxy for expectation errors in the market. One “beats the market” by front-running expectations of the market, not necessarily by simply buying “value” stocks. Fundamentals, technicals, sentiment, and so forth, only work to the extent they aren’t already discounted into the current price. They don’t work just because they sound good or they are an interesting story.

            When it comes to value, specifically, we would argue that well-designed value signals work because they capture a systematic “overreaction to bad news” expectation error–investors, on average, throw the baby out with the bath water when it comes to the cheapest stocks in the market. These signals don’t work just because they are based on price/fundamental ratios, it just happens to be the case that price/fundamentals serve as a proxy for bad market expecations and we can use these signals to frontrun the market, on average.

            An recent example of this type of research is from Piotroski and So 2012 or Jack’s dissertation.

      • Kumar V July 31, 2015 at 12:59 pm

        well, most models are based on past data and theories about why they work. unless the model attempts to ‘learn’ there will always be a good reason to question the model when the human suspects paradigmatic shifts (eg touchscreen iPhone vs flip phones)

        • Jack Vogel, PhD July 31, 2015 at 1:24 pm

          I agree, one should always question the model.

      • Kumar V July 31, 2015 at 1:06 pm

        the other point is that models are built by humans. as much as we like to package models as ‘evidence’ based, there is a lot of art in massaging and interpreting the data, not to mention the creative aspect of coming up with theories for why something has been true in the past. therefore, as i see it, models serve to systematize human beliefs or biases! that is why IMHO it makes sense to always view models with some suspicion and override them when the risks of going with the model don’t seem to be worth it. (i agree with you in general that it depends on the model, but i haven’t seen much in terms of learning models out of the traditional finance academia)

  2. Steve July 29, 2015 at 11:14 pm

    Hey, I’m not a stock picker! 😀

    I like the suggestion by Greenblatt in “the little book…” for those who cannot resist stock picking…to at least keep within the best stocks of the model. After all, if you’re only after (say) 10 stocks from a decile of 100 stocks (perhaps doing this a few times a year)…then it shouldn’t matter TOO much whether you pick them via darts or try a bit of stock picking.

    At least doing it that way, you might discover that you actually have alpha of your own! And if (more likely) not…no harm done (so the theory goes)

    • Jack Vogel, PhD July 30, 2015 at 9:47 am

      Here are the results from Greenblatt’s test of model vs self-managed:

      May 1, 2009 – April 30, 2011:

      Self-managed = 59.4%
      S&P 500 = 62.7%
      Model = 84.1%

      Small sample size, but interesting results!

      • Steve July 30, 2015 at 8:35 pm

        Very interesting, and I think it shows that people often can’t, “follow the model”. But it doesn’t change what I said above about sticking with the model’s stocks and either randomly throwing darts or doing some stock picking – *as long as* the model rules are still stuck to. Re-reading the morning star article highlights to me that the problems were mainly to do with, “self-managed investors sold stocks without replacing them, held more cash, and/or stopped updating the strategy on a periodic basis after the markets and their portfolio declined for a period of time” – which is not, “following the model”.

        An investor with a stock picking inclination (perhaps they fancy themselves as capable of finding insights into the fundamentals or perhaps they fancy themselves as being able to spot trends well on price charts etc)….as long as they still stuck to the rules (e.g. hold a minimum 20 stocks, hold for one year and replace etc)….I maintain that the results should be no different than choosing 20 stocks at random from the top decile (which reminds me of the test you guys did – a monte carlo of 30 stocks picked from the value decile). And *if* that investor happens to have some hidden, innate Buffett like skills…who knows?

        • Jack Vogel, PhD July 31, 2015 at 7:58 am

          I agree, picking 20 stocks from the model is not bad, but at some level, you still need to “follow the model”

  3. Michael Milburn August 1, 2015 at 2:13 am

    One of my issues with “following the model” is that as I watch trades come and go, I am always questioning it – and coming up with reasons why the model should’ve been able to avoid that bad trade. (Ex: Currently Mylan is one of my bad mean reversion trades, and I’m wondering about a rule about not trading anything w/ hostile takeover bid underway… SUNE also a mean reversion trade that has quickly turned ugly, and it was a company I had a bad opinion of to start with, but I decided to follow the model anyhow. It would’ve been better if I didn’t already have views of the Solar Energy space.)

    So I guess I need a better rule about locking down a model and just trading it as is – vs. making adjustments on the fly. It’s difficult to resist watching trades unfold and wondering – if I’d just have filtered for X, Y, or Z maybe I could’ve avoided the bad trade. Most of my testing tells me “sorry, you just gotta eat the losses because trying to avoid them hurts the model in other ways”, but resisting the tweaking is difficult when your gut tells you “I don’t care what the model says, this stock sucks, don’t buy it.”

    • Michael Milburn August 1, 2015 at 2:26 am

      …not to mention that I know how simple things are underneath the hood of the model, and that there’s really no knowledge at work. It’s just looking for patterns that in the past are associated with higher probability trades.

    • drmark27 August 11, 2016 at 10:07 am

      One thing I’ve considered a number of times is that if I’m going to make a change in how you manage a trade then I need to consider the implications of managing a large sample of trades the same way. For any given trade, I can always identify one thing that, if done differently, could have generated better results. I consider this a form of “curve-fitting,” though. If I don’t have data from a sufficient sample then I really have nothing at all.

Comments are closed.