The subject of asset bubbles and market crashes has fascinated me for more than 20 years. As an options market maker for Susquehanna International Group (“SIG”), extreme price movements were a daily source of concern. I sat next to Jeff Yass for years and watched him manage option positions in thousands of different stocks. Almost daily he would be celebrating a big win in a stock that had an unusually large move (SIG loves to own the “teenie” puts).

At some point, a very interesting question popped into my head:

Why is it that 10-sigma events happen all the time?

Current risk models and option pricing models suggest that these events should happen almost never.^{(1)}

The science of risk management and derivative pricing revolves around one thing: estimating future prices of assets. For over 50 years the world of finance has defaulted to using a normal distribution to estimate the probabilities of future price moves. Why? Is it a physical law of the universe that asset prices follow a normal distribution? There are plenty of other statistical distributions that asset prices could follow.

*So, why use the Normal distribution?*

Sadly, the answer is that the normal distribution is mathematically neat and easy to use. It doesn’t work, but it’s easy to use. Some of the smartest people in the world have lost billions of dollars because they wanted to use an easy formula (i.e. Long Term Capital Management).

Over the years people began to realize that the normal distribution does a terrible job in predicting future price moves. But, instead of trying to find a distribution that does work, people decided to try to mangle the normal into something useful. They squeezed and stretched the normal with intelligent sounding terms like leptokurtosis and skew. It does work better now, but still not great.

## Is There A Better Model For Stocks Than the Normal Distribution?

For many years I have been using a different distribution that works much better, and has made a fortune for the companies in which I have worked. This formula is simple, easy to use, and has the added benefit of being able to arguably identify market bubbles before they burst. As you probably know, when an asset is in the midst of a bubble, the probability of an unusually large downside move is greatly increased. The good news is that the options world does not know how to identify bubbles, and often prices the out-of-the-money puts too low.

The normal curve is one member of a family of distributions known as “stable distributions.” If we assume mean = 0 and variance = 1, the characteristic equation for this family is f(t) = e\frac{-t^a}{2}. The difficult part of using this equation is estimating the parameter α, which can vary 0 < α ≤ 2. When α = 2, you get a normal distribution, when α = 1 you get a Cauchy distribution, and when 1 < α ≤ 2 you get a Pareto-Levy distribution. Financial engineers have defaulted to assuming α=2 at all times, but there is no reason to believe this accurately models asset returns. History shows us that a more accurate distribution should be more peaked around the mean, and have fatter tails. Using a Pareto-Levy distribution with 1< α < 2 produces just such a curve (fig. 1).

This distribution does what financial engineers have been trying to do to the normal.

Now, as mentioned, using α = 2 does not produce accurate estimates for future price moves. So, the question becomes what value of α should be used? The potential answer to this question is surprisingly simple and elegant. This α can be calculated based on historical price returns of each individual asset you are modeling. In fact, the answer is to use different α’s for different assets depending on their individual historical returns.

## Introducing the Hurst Exponent

The method of calculating α uses a statistical procedure called R/S (range over scale) analysis, which is then used to calculate a Hurst Exponent for the data in question.^{(2)} Harold Hurst used this method to calculate how high to build the Aswan Dam on the Nile River in Egypt in the early 1900’s. He recognized that historical Nile River flooding data (which went back thousands of years) was not normally distributed, but displayed a long-term “memory.” This exponent has been given the symbol H in his honor. This method of statistical analysis is not widely used, but there is plenty of literature on the subject.

The Hurst Exponent (and R/S analysis) basically measures how fast a data set is scaling over time. The parameter H varies from 0 < H ≤ 1. The key information that H reveals is that if H=0.5 then the data set is scaling as a purely random system would scale. For 0.5 < H ≤ 1 the data set is scaling faster than random, and for 0 < H < 0.5, the data set is scaling slower than random. For our purposes, the focus will be on assets where H > 0.5. This means that the price is scaling in time faster than random (i.e. the price is going up too quickly relative to the daily price moves). An asset whose price slowly increases in this fashion might be contradictory to The Efficient Market Hypothesis, making the normal distribution inappropriate for use in predicting future price moves.

As discussed earlier we want to use the Pareto-Levy distribution but we don’t know the correct α to use in the formula. Amazingly, the Hurst exponent is related to the α through the simple formula α = H^-1. So, when H = 0.5, then α = 2. This means that if we do R/S analysis of historical prices and determine they are random, then we can use the normal distribution. If we do R/S analysis and determine that the prices are not random then we simply use the Pareto-Levy distribution and calculate the correct α. The key thing to understand is that the closer α comes to 1 (which means H approaches 1), the fatter the tails get.

The question now is how to use H once it is calculated. In my experience, this Hurst Exponent usually falls between .4 and .8 (0.4 < H < 0.8), and is often near 0.5, indicating that the asset’s returns are random. In the cases where H is near 0.5 the widely used normal distribution is fine to use. The most interesting cases occur when 0.6 < H < 0.8, which may indicate a large amount of “herding,” or “crowding” in the asset. To summarize, I categorize assets as follows:

- 0.4 < H < 0.6 Asset is approximately random
- 0.61 < H < 0.7 Moderate bubble forming (i.e. moderate herding)
- 0.71 < H < 0.8 Critical bubble forming (i.e. critical herding)

The key to understanding these categories is that when 0.61 < H < 0.8 there is a *hidden* amount of volatility and risk associated with this asset, because there is a potential bubble forming. Remember, this Hurst range translates to 1.25 < α <1.64, and this range of alpha has INFINITE variance. The market completely misunderstands and misprices this hidden tail risk.

So, when H > 0.7, a bubble might be forming because prices are increasing in a non-random way. This price action is often not be caused by news or corporate events, but by non-fundamentals-based market participants.

## What Does the Hurst Exponent Say About the Markets Today?

The history of bubbles and crashes in markets is well documented, but rarely is there agreement about bubbles until after they burst. With this simple formula, bubbles can be mathematically identified and categorized BEFORE they burst. In addition, options traders should consider this formula in their pricing algorithms, otherwise, the out-of-the-money puts will likely be mispriced.

As of 10/18/2017, the S&P 500 index currently has an H exponent of 0.71, and is therefore in the critical bubble category. The range of distributions are arguably much larger than what the normal distribution would suggest. Arguably, the market has a lot of hidden risk right now, and the VIX at 10 is deceiving. That said, markets can remain irrational for a long time…

- The views and opinions expressed herein are those of the author and do not necessarily reflect the views of Alpha Architect, its affiliates or its employees. Our full disclosures are available here. Definitions of common statistics used in our analysis are available here (towards the bottom).
- Join thousands of other readers and subscribe to our blog.
- This site provides
**NO**information on our value ETFs or our momentum ETFs. Please refer to this site.

References

1. | ↑ | Just for context a 32% volatility stock that drops 20% in a day is a 10-sigma event. I’m pretty sure everyone has seen that happen. |

2. | ↑ | Here is a great site on calculating the Hurst Exponent. |

ShollyNovember 15, 2017 at 8:20 amInteresting blog post.

When you write “As of 10/18/2017, the S&P 500 index currently has an H exponent of 0.71, and is therefore in the critical bubble category.”, may I ask over what timeframe the exponent of H was calculated? It would also be interesting to overlay a chart with this rolling H value underneath the S&P 500 index chart.

Again, thanks for sharing this.

Sholly

ShollyNovember 15, 2017 at 10:02 amOn what timeframe (days) is the Hurst exponent calculated for the S&P index in your blog post?

AptenodyteNovember 17, 2017 at 10:33 pmHello can I ask what data period you use to get SPX H as 0.71?

Using matlab and 1 year of daily total return data on SPY etf I get H=0.4768.

Longer periods give me H’s that are even lower. Regards.

Wes Gray, PhDNovember 20, 2017 at 7:31 amI’ll have Keith respond

Keith KlineNovember 20, 2017 at 8:42 amAptenodyte, I used 1 year of daily returns on the spy (adjusted for dividends). I do my calculation in an excel spreadsheet which is not very efficient, but it does allow you to “see” every phase of the calculation. I have spoken to many people over the past week who are trying to implement the algorithm in either python or matlab. It is very difficult to trouble shoot this algorithm in these languages. If you are really interested I would highly suggest taking the time to build the algorithm in excel, get it working there, then automating with matlab. If you have trouble in excel, you can contact me directly and I can help you get it working.

Keith KlineNovember 20, 2017 at 11:11 amFor all those who are trying to calculate H on your own please keep in mind the following:

There are two types of Hurst Exponent that can be calculated: “Hurst with mean”, and “Hurst without mean”.

Most algorithms calculate “Hurst with mean”. This would be appropriate for asking questions like “Is the deviation from average annual rainfall random, or is there some kind of long range memory?”

For asset prices we don’t want to know about the deviation around average daily returns, we want to know the deviation in actual price. For example, if an asset went up 1% every day for a year the deviation from the mean would be 0, and your R/S would be 0. Despite the R/S being 0, the stock price would have gone up dramatically and I would certainly consider that stock to be in a bubble.

The easiest way to fix these algorithms would be to simply set the mean to 0 and then it should give you the appropriate Hurst Exponent.

ParkerdgNovember 28, 2017 at 12:48 pmCan you provide some additional background to the following statement: “Amazingly, the Hurst exponent is related to the α through the simple formula α=H^-1.”

Keith KlineNovember 28, 2017 at 2:26 pmParkerdg,

In fractal geometry the hurst exponent is directly related to fractal dimension. There is a wide range of academic research that can help clarify this point. One such article can be found at this link:

http://www.tandfonline.com/doi/abs/10.1080/096031001300313956

A quote from the abstract of this paper reads “The unique feature of Levy-stable family distributions is the existence of a relationship between the fractal dimension of the probability space and the fractal dimension of the time series. This relationship is simply expressed in terms of Hurst exponent (H), i.e. α = 1/ H.”

I would recommend reading some of these papers if you are looking for a detailed explanation, as it would be too long to reply in this blog. Of course if you still have questions please feel free to reach out to me directly.

Thank you for the question.

ParkerdgNovember 28, 2017 at 8:30 pmThank you very much for the interesting post and also for providing the additional reference to help in better understanding this relationship between Hurst and return distribution.

Scott TeresiJanuary 7, 2018 at 2:31 pmCan this be taken to practical conclusion, e.g. taking an analysis of stock index data (including dividends) going back to 1929, what might a true 95% probability event in the U.S. stock market be (normally a 2-sigma event)?

Morningstar reports a standard deviation for Vanguard 500 Index (VFINX) as 15.07 over the last ten years, using a normal distribution. (They compute this using monthly total return data and then the monthly standard deviations are annualized.) So that suggests a 2-sigma event of -30.14% (or +30.14%) erroneously should happen less than 95% of annual periods!

Intuitively, I would expect to see greater than -50% drops during 5-10% of annual periods (every 10-20 years) but I wonder what the Pareto-Levy distribution with Hurst exponent derived from all available U.S. stock market history would reveal.

January 24, 2018 at 10:31 pmHi. It’s seems strange for me that you apply the R/S method to the SP500 instead of the SP500 returns. Did I miss something?

Keith KlineJanuary 25, 2018 at 1:32 pmThe R/S method is used on returns not prices. You are correct.