This paper acknowledges the pivotal role of earnings calls in disseminating value-relevant information, with particular emphasis on the Q&A segment. However, it confronts the inherent challenge posed by the unstructured nature of language in these calls, complicating quantitative analysis. In response, the authors innovate by introducing a novel measure designed to grasp the subtleties of information embedded in the dialogue. This measure, the Human AI Difference (HAID) is based on discerning semantic differences between responses to identical questions from corporate CEOs and CFOs, and those generated by advanced Large Language Models (LLMs), such as ChatGPT.
The authors ask the following questions:
Are high HAID values more likely to convey positive news rather than negative news?
Are high HAID values positively associated with earnings growth?
Are high HAID values positively associated with abnormal future stock returns and abnormal trading volume?
Is HAID negatively related to stock illiquidity measures?
What are the Academic Insights?
By studying a sample of 104,932 earnings conference calls for the period of 2004 – 2020, the authors find:
YES – higher values of HAID are more likely to convey positive news rather than negative news, aligning with managers’ incentives to emphasize positive developments and conceal adverse information
YES – higher values of HAID are indeed correlated with future earnings growth
YES – higher HAID is associated with higher absolute cumulative abnormal stock returns and abnormal trading volume around earnings calls, indicating that investors respond more actively to calls with higher HAID
YES – higher HAID is associated with smaller analyst forecast errors and less cross-analyst disagreement after earnings calls
YES – HAID is negatively related to stock illiquidity measures, such as bid-ask spread and Amihud Ratio, indicating that higher HAID is linked to improved liquidity. This implies that HAID contributes to better price discovery and reduced information asymmetry
The authors conduct robustness tests by applying the methodology to alternative LLMs, Google Bard, and an open-source LLM, and find qualitatively similar results, reinforcing the generalizability of the findings.
Why does this study matter?
This study contributes to our understanding of the role of language and information in financial markets. In fact, it demonstrates that the HAID measure effectively quantifies the information content in earnings conference calls, and higher HAID values are associated with more positive market outcomes, improved analyst behavior, and enhanced liquidity.
The Most Important Chart from the Paper:
The results are hypothetical results and are NOT an indicator of future results and do NOT represent returns that any investor actually attained. Indexes are unmanaged and do not reflect management or trading fees, and one cannot invest directly in an index.
A significant portion of information shared in earnings calls is conveyed through verbal communication by corporate managers. However, quantifying the extent of new information provided by managers poses challenges due to the unstructured nature of human language and the difficulty in gauging the market’s existing knowledge. In this study, we introduce a novel measure of information content (Human-AI Differences, HAID) by exploiting the discrepancy between answers to questions at earnings calls provided by corporate executives and those given by several context-preserving Large Language Models (LLM) such as ChatGPT, Google Bard, and an open source LLM. HAID strongly predicts stock liquidity, abnormal returns, number of analysts’ forecast revisions, analyst forecast accuracy following these calls, and propensity of managers to provide management guidance, consistent with HAID capturing new information conveyed by managers. Overall, our results highlight the importance of using LLM as a tool to help investors unveil the veiled – penetrating the information layers and unearthing hidden insights.
Dr. Elisabetta Basilico is a seasoned investment professional with an expertise in "turning academic insights into investment strategies." Research is her life's work and by combing her scientific grounding in quantitative investment management with a pragmatic approach to business challenges, she’s helped several institutional investors achieve stable returns from their global wealth portfolios. Her expertise spans from asset allocation to active quantitative investment strategies. Holder of the Charter Financial Analyst since 2007 and a PhD from the University of St. Gallen in Switzerland, she has experience in teaching and research at various international universities and co-author of articles published in peer-reviewed journals. She and co-author Tommi Johnsen published a book on research-backed investment ideas, titled Smarte(er) Investing. How Academic Insights Propel the Savvy Investor. You can find additional information at Academic Insights on Investing.
Performance figures contained herein are hypothetical, unaudited and prepared by Alpha Architect, LLC; hypothetical results are intended for illustrative purposes only. Past performance is not indicative of future results, which may vary. There is a risk of substantial loss associated with trading stocks, commodities, futures, options and other financial instruments. Full disclosures here.