Can artificial intelligence (AI) models help investors quantify vague firm risks through textual analysis? In their October 2023 paper entitled “From Transcripts to Insights: Uncovering Corporate Risks Using Generative AI”, Alex Kim, Maximilian Muhn and Valeri Nikolaev explore the value of generative AI tool ChatGPT 3.5 in quantifying firm risks based on politics, climate change and AI as conveyed in earnings conference call transcripts. For each of the three risks, they generate: (1) risk summaries based solely on the transcripts, and (2) risk assessments in full context based on the transcripts plus all ChatGPT training data. They consider risk analysis both within (before September 2021) and outside (January 2022 through March 2023) ChatGPT’s training period. They test the import of ChatGPT-based risk assessments via 5-factor (accounting for market, size, book-to-market, profitability and investment effects) alphas of hedge portfolios that are that are long the fifth (quintile) of stocks with the highest assessed risks and short the quintile with the lowest. Using earnings transcripts and monthly returns for a broad sample of U.S. stocks during January 2018 through March 2023, they find that:
- Time-varying, firm-specific data account for the bulk of variation in firm political, climate and AI aggregate risks.
- Both during and after the ChatGPT training period, future stock market implied and realized return volatilities exhibit strong positive relationships with aggregate ChatGPT political and climate change risk assessments.
- There is limited evidence that the more recent aggregate ChatGPT AI risk (see the chart below) predicts stock market volatility.
- For all three kinds of risk, gross alphas increase almost regularly from monthly reformed lowest to highest risk quintiles.
- Annualized gross alphas for extreme quintile hedge portfolios as specified above are:
- 5.3% for political risk.
- 6.7% for climate change risk.
- 6.4% for AI-related.
The following chart, taken from the paper, tracks AI summary and assessment aggregate risk levels over the full sample period. As might be expected, this risk mostly grows over time, spiking at the end with advances in AI.
In summary, evidence from a short sample indicates that hedge portfolios based on ChatGPT-derived political, climate change and AI firm risk measures generate material gross 5-factor alphas.
Cautions regarding findings include.
- The sample, and especially subsamples, are very short in terms of variety of market conditions.
- Alphas are gross, not net. Accounting for monthly portfolio reformation frictions would reduce returns and alphas. Use of equal weighting exacerbates this concern due to the generally negative relationship between firm size and stock liquidity.
- The methodology used is beyond the reach of most investors, who would bear administrative costs and fees for delegating to a fund manager.
- It may be more accurate to describe the soft risks explored in the paper as “perceived risks.”
