Can chatbots, like ChatGPT, be used to interpret and condense lengthy financial disclosures into less complex, shorter but relevant documents? Can they answer questions and ease the access for investors to germane information in an understandable format? In a world where information is abundantly available to investors and the general public alike, the use of AI models to facilitate and improve decision-making is a worthy research topic. The authors of this study analyze these and other questions and conclude in the affirmative.
Bloated Disclosures: Can ChatGPT Help Investors Process Information?
- Alex G. Kim, Maximilian Muhn, Valeri V. Nikolaev
- SSRN, Working Paper
- A version of this paper can be found here
- Want to read our summaries of academic finance papers? Check out our Academic Research Insight category.
What are the research questions?
- Are AI language models, like ChatGPT, effective at reducing the length of corporate disclosures?
- Is the loss of information between the original and the summary produced by the AI model significantly different?
- Does informational bloat vary from firm to firm?
- How does the market react to informational bloat?
What are the Academic Insights?
- YES. Without any constraints on the modelling, the chatbot produced a summary that was 20% of the original, on average. A reduction in length of 80% is exceedingly large and suggests there are large gains in information processing to be had. That of course, assumes the loss in informational content is inconsequential.
- NO. Using the Loughran and McDonald (2011) sentiment measure where positive and negative word counts reflect sentiment, the results indicate the AI produced summary is consistent and more pronounced than the original disclosures. If the sentiment in the original is positive (negative), for example, the summary document is even more positive (negative). The authors argue that if firms and their executives pull their punches by using unduly precautionary terms, then the original disclosures may contain minimal information or are “largely boilerplate”. In any case, while the summary document may lose information, what is retained in the summary is apparently a better measure of actual sentiment. Truly a remarkable result.
- YES. Using the size of the reduction in the AI model summary as a measure of informational bloat, the authors report a significant variation in bloat from firm to firm. Variance decomposition of the data indicates that the variability in bloat is explained by time and industry effects at 30%-40% and 60%-70% of the variability is firm-specific. There was also significant variability within the same firm on a year-to-year basis. Generally, bloat is higher when firm circumstances are negative. That is when a firm reports negative net income, exhibits negative sentiment and negative market reactions to significant events. One question arises. Is management baffled, bewildered or just obfuscating as hypothesized in Li (2008)?
- NEGATIVE. Note Figure 3, the 2-day cumulative abnormal returns on the publication date of the original disclosure (either MD&A or conference calls) underperform the AI model produced summaries.
Why does it matter?
This research makes a number of contributions including:
- Establishing the economic usefulness of generative AI-based techniques in analyzing and summarizing unstructured textual data in a useful way;
- Developing a novel measure of the degree to which textual information contains redundancies and excessive details;
- Adding to the literature on disclosure quality and its economic consequences. Documenting that firms with bloated disclosures exhibit lower price efficiency.
The most important chart from the paper
The results are hypothetical results and are NOT an indicator of future results and do NOT represent returns that any investor actually attained. Indexes are unmanaged and do not reflect management or trading fees, and one cannot invest directly in an index.
Generative AI tools such as ChatGPT are expected to disrupt numerous industries and could fundamentally alter the way economic agents process information. We probe the economic usefulness of these tools in extracting information from complex corporate disclosures using the stock market as a laboratory. We use the GPT language model to summarize textual information disclosed by companies in their annual reports (MD&A) and during conference calls. Unconstrained summaries are dramatically shorter compared to the original disclosures, whereas their information content is amplified. When the originals have a positive (negative) sentiment, the summary becomes more positive (negative). More importantly, the summaries’ are more effective in explaining stock market reactions to the disclosed information. Motivated by these findings, we propose a novel measure of disclosure “bloat.” We show that bloated disclosure is associated with adverse capital market consequences, such as lower price efficiency and higher information asymmetry. Finally, we show that the model is effective at targeted summaries that distinguish between financial and non-financial (ESG) performance.