Factor ETF Due Diligence: A New Challenge for Some Investors

Factor investing seems to be everywhere, but the topic is still misunderstood by large swaths of the investing public. Tommi Johnsen and I would like to help bring clarity to the topic and help this blog empower investors through education. We also have a new book which will focus on ten of the more robust and evidence-based investing ideas presented via academic and practitioner research.⁽¹⁾

Factor-based strategies have attracted $500 Billion ⁽²⁾ via ETF vehicles (as of the end of 2015) and are expected to grow to 2.4 Trillion by 2025, according to Blackrock.⁽³⁾

Among the main reasons for the success and popularity of ETFs are their versatility, liquidity, cheapness, and tax-efficiency.  These attributes provide an opportunity for the retail investor to access investment options on par with the quality often only available to institutional investors. In addition, institutional investors can leverage the structure to provide greater flexibility in their portfolio construction.

Recent surveys are showing that institutions are starting to increasingly use ETFs, although they still represent a small percentage of their total assets at approximately 1%⁽⁴⁾. In fact, despite the brute force evidence that high-cost active management fails to produce consistent alpha, institutional investors dedicate a lot of time and resources to the search of star managers and they pay them high fees for the promise of benchmark out-performance.

Only time will tell if ETFs will slowly start to represent a higher percentage of institutional investors’ portfolios, but we predict that robust ETF due diligence will be a new feature of the institutional investor toolkit.

Why is ETF strategy due diligence so important?

While ETFs are relatively new, the requirement that investors understand what they are buying hasn’t changed. Know what you own and why you own it — process matters.

We discuss a recent article by Ducoulombier, Goltz and Ulahel⁽⁵⁾ on the methodological differences across multi-factor index offerings. In particular, they review the robustness and consistency of the multi-factor indices and the level of diversification among the various options.

Asness et al. (2015)⁽⁶⁾ demonstrate that style diversification is a fundamental ingredient in portfolio construction. In fact, they find a negative correlation (average of -0.6) between value and momentum (long/short portfolios). This is a powerful finding because it provides guidance to investors on how to build more robust portfolios with exposure to a diversified set of factors.

As a matter of fact, multi-factor ETFs are the newest addition to a plethora of individual smart beta products. On top of diversification benefits, according to Ducoulombier et al. (2016), investors allocating to multi-factor solutions may benefit from implementation benefits (rebalancing trades crossing and turnover reduction).

We find the Ducoulombier et al. (2016) study interesting on various levels, but most importantly we think that every investor should be aware of the staggering differences that there can be among the various options. Differences between the methodologies used to construct the underlying indices and the academic evidence on factor investing can be striking.  Table 1 and Table 2 in the paper are particularly enlightening. In Table 1, the authors present an overview of the different methodologies used by the index provider to construct portfolios. In table 2, the reader can observe the differences with the standard academic definition of factors.

This is a key point because most factors defined in the academic literature have been subject to extensive empirical analysis with long time frames for back-testing, cross-sample markets robustness checks and with careful screening for datamining and various biases.

However, as highlighted in Table 2, most factor indices are based on different factor definitions and calculations from the more standardized academic ones. While this does not necessarily mean it is the wrong approach, investors need to be careful in making sure that the back-testing and cross validation analysis was done in a rigorous manner. Otherwise, the potential for data-mining and associated failures of prediction increases (Harvey and Liu, 2015).⁽⁷⁾

In particular, two type of biases can become problematic:

1. Selection bias, if there is a specific aim at selecting the best in sample factor performer;
2. Overfitting bias, when providers utilize composite scoring approaches (Novy-Marx, 2015)⁽⁸⁾

Table 3 shows some example of composite scores used in multi-factor indices.

Table 1: Extract from Ducoulombier et al. (2016)

Table 2: Extract from Ducoulombier et al. (2016)

Table 3: Extract from Ducoulombier et al. (2016)

Two other things that the authors warn investors against are the inconsistency of the design across the multi-factor indices for different factors as well as the inconsistency of the design across time.

ETFs are democratizing factor investing, but investors should be careful when selecting the investment vehicle to gain exposure to the various factors. In Ducoulombier et al. (2016), the authors conclude that “the methodological principle behind index construction should become a key area of attention in the assessment of these indices, […] as well as the consistency in in index design [… ] and ultimately, the robustness of the performance presented in back-tests.”

We’ll leave the last word to Lo (2016):⁽⁹⁾

The single biggest challenge created by the smart beta revolution is the potential for misleading investors and portfolio managers through back-test bias. This is especially true when, 1) the number of managers/model/track records grows; 2) the signal to noise ratio declines; 3) decisions become more dependent on simulated performance statistics rather than live track records”

All three of the conditions mentioned by Lo (2016) are present in the industry of smart beta. We predict that soon enough “fund pickers,” will change their title to “ETF pickers.” Let’s hope there are educational tools and outlets become available to help investors sift through this opaque and confusing environment.