How You Sort Matters in Sorting Factor Portfolios
Non-standard errors capture uncertainty due to differences in research design choices. We establish substantial variation in the design choices made by researchers when constructing asset pricing factors. By purposely data mining over two thousand different versions of each factor, we find that Sharpe ratios exhibit substantial variation within a factor due to different construction choices, which results in sizable non-standard errors and allows for p-hacking. We provide simple suggestions that reduce the average non-standard error by 70%. Our study has important implications for model selection exercises.