Investing Expertise

Does finance professor David Kass, who presents annual lists of stock picks on Seeking Alpha, make good selections? To investigate, we consider his picks of “10 Stocks for 2020”, “16 Stocks For 2021”, “12 Stocks For 2022”, “10 Stocks For 2023” and “10 Stocks For 2024”. For each year and each stock, we compute total (dividend-adjusted) return. For each year, we then compare the average (equal-weighted) total return for a David Kass portfolio to that of  SPDR S&P 500 ETF Trust (SPY). Using dividend-adjusted returns from Yahoo!Finance for SPY and most stock picks and returns from Barchart.com and Investing.com for three picks during their selection years, we find that: Keep Reading

Can large language models (LLMs) mimic expert economic forecasters? In their December 2024 paper entitled “Simulating the Survey of Professional Forecasters”, Anne Hansen, John Horton, Sophia Kazinnik, Daniela Puzzello and Ali Zarifhonarvar employ a set of LLMs (primarily GPT-4o mini) to simulate economic forecasts of experts who participate in the Survey of Professional Forecasters. Specifically, they:

1. Provide the LLMs with detailed participant characteristics (demographics, education, job title, affiliated organizations, alma maters, degrees, professional roles, location and social media presence) and then prompt the LLMs to mimic forecaster personas.
2. Ask each persona to respond to survey questions using real-time economic data and historical survey responses.

They further explore which persona characteristics affect forecast accuracy. They address the issue of potential LLM look-ahead bias by telling the models to use only information available at the time of forecasting. Using the specified forecaster persona and economic/historical forecast data, they find that:

Keep Reading

Can large language models (LLM) handle complex financial reasoning tasks that require multi-step logic, market knowledge and regulatory adherence? In his December 2024 paper entitled “Large Language Models in Finance: Reasoning”, Miquel Noguer I Alonso surveys and extends techniques for enhancing LLM reasoning capabilities. He presents detailed finance-specific coding examples, including dynamic portfolio optimization, scenario stress testing, regulatory compliance analysis and credit risk assessment. He addresses key challenges in scalability, interpretability and bias mitigation. Based on his knowledge and experience with LLMs and other analysis tools, he concludes that:

Keep Reading

Recent research in accounting and finance finds that large language models (LLM) beat humans on a variety of related tasks, but the black box nature of LLMs obscures why. Is LLM outperformance real? In his December 2024 paper entitled “Caution Ahead: Numerical Reasoning and Look-ahead Bias in AI Models”, Bradford Levy conducts a series of experiments to open the LLM black box and determine why LLMs appear to perform so well on accounting and finance-related tasks. He focuses on numerical reasoning and look-ahead bias. Based on results of these experiments, he finds that:

Keep Reading

Do annual stock picks from the Meta AI large language model beat the market? To investigate, we ask Meta AI to pick the top 10 stocks for each of 2020-2024 based on information available only before each year. For example, we ask Meta AI to pick stocks for 2020 as follows:

“Limiting yourself strictly to information that was publicly available by December 31, 2019, what are the 10 best stocks for 2020?”

We then repeat the question for 2021, 2022, 2023 and 2024 stock picks, each time advancing the information restriction to the end of the prior year. For each year and each stock, we compute total (dividend-adjusted) return. For each year, we then compare the average (equal-weighted) total return for a Meta AI picks portfolio to those of  SPDR S&P 500 ETF Trust (SPY) and Invesco QQQ Trust (QQQ). Using end-of-year dividend-adjusted closing prices for SPY, QQQ and each of the specified years/stocks (with all five queries occurring on January 12, 2025) from Yahoo!Finance, we find that:

Keep Reading

Do annual stock picks from the ChatGPT large language model beat the market? To investigate, we ask ChatGPT to pick the top 10 stocks for each of 2020-2024 based on information available only before each year. For example, we ask ChatGPT to pick stocks for 2020 as follows:

“Limiting yourself strictly to information that was publicly available by December 31, 2019, what are the 10 best stocks for 2020?”

We then repeat the question for 2021, 2022, 2023 and 2024 stock picks, each time advancing the information restriction to the end of the prior year. For each year and each stock, we compute total (dividend-adjusted) return. For each year, we then compare the average (equal-weighted) total return for a ChatGPT picks portfolio to those of  SPDR S&P 500 ETF Trust (SPY) and Invesco QQQ Trust (QQQ). Using end-of-year dividend-adjusted closing prices for SPY, QQQ and each of the specified years/stocks (with all five queries occurring on January 12, 2025) from Yahoo!Finance, we find that:

Keep Reading

How can researchers ensure that large language models (LLM), when tasked with time series forecasting, do not inject look-ahead bias and thereby inflate measured predictive power? In his brief November 2024 paper entitled “Look-Ahead Bias in Large Language Models (LLMs): Implications and Applications in Finance”, Miquel Noguer I Alonso addresses sources of LLM look-ahead bias in financial time series forecasting and proposes strategies to mitigate it. Based on logic and knowledge of LLM development, he concludes that: Keep Reading

A reader asked: “I am wondering how come you have not rated Motley Fool guys. Any insight?” To augment the test of Motley Fool public stock picks in “‘Buy These Stocks for 2019’ Forward Test”, we evaluate stock picks for 2021, 2022, 2023 and 2024 via “10 Top Stocks That Will Make You Richer in 2021”, “7 Stocks That Could Make You Richer in 2022”, “Got $1,000? 5 Sensational Stocks to Buy to Start 2023 With a Bang” and “10 Top Stocks to Buy in 2024”. For each year and each stock, we compute total (dividend-adjusted) return. For each year, we then compare the average (equal-weighted) total return for a Motley Fools picks portfolio to that of  SPDR S&P 500 ETF Trust (SPY). Using end-of-year dividend-adjusted closing prices for SPY and each of the specified years/stocks from Yahoo!Finance (except for Kirkland Lake Gold, for which prices are from Barchart.com), we find that:

Keep Reading

Do “great stock picks” from Forbes beat the market? To investigate, we evaluate stock picks for 2022, 2023 and 2024 via  “10 Great Stock Picks for 2022 from Top-Performing Fund Managers”, “20 Great Stock Ideas for 2023 from Top-Performing Fund Managers” and “10 Best Stocks For 2024”. For each year and each stock, we compute total (dividend-adjusted) return. For each year, we then compare the average (equal-weighted) total return for a Forbes picks portfolio to that of SPDR S&P 500 ETF Trust (SPY). Using end-of-year dividend adjusted prices from Yahoo!Finance for the specified years/stocks, we find that: Keep Reading

Generative large language models (LLM), such as ChatGPT, are best known for conversational summation of complex information. Their use in financial forecasting focuses on discrete news sentiment signals of positive (1), neutral (0) or negative (-1). Is there a way to extract more granularity in LLM sentiment estimates? In their October 2024 paper entitled “Cut the Chit-Chat: A New Framework for the Application of Generative Language Models for Portfolio Construction”, Francesco Fabozzi and Ionut Florescu present Logit Extraction as a way to replace discrete LLM sentiment labels with continuous sentiment probabilities and apply results to ranking stocks for portfolio construction. Logit Extraction exploits the inner workings of LLMs to quantify sentiment strength. They test it on four LLMs: Mistral, Llama, ChatGpT-3.5 and ChatGPT-4. Their benchmark model is the specialized, quantitative FinBERT. They compare the abilities of each LLM to those of FinBERT in replicating human-assigned sentiment labels and generating long-short portfolio risk-adjusted returns, with and without Logit Extraction. Inputs are initial-release headlines from news alerts covering a single company published from 30-minutes before market open on the previous day to 30-minutes before market open on the day of trading during January 2010 through October 2020. They aggregate headlines published on non-trading days for long-short trading the next trading day. Portfolio trades occur at the open each trading day and are limited to stocks in the news the day before (an average of 46). Using the specified 216,837 news headlines and associated daily returns across 263 unique firms, they find that: Keep Reading