Mimicking Economic Expertise with LLMs

Can large language models (LLMs) mimic expert economic forecasters? In their December 2024 paper entitled “Simulating the Survey of Professional Forecasters”, Anne Hansen, John Horton, Sophia Kazinnik, Daniela Puzzello and Ali Zarifhonarvar employ a set of LLMs (primarily GPT-4o mini) to simulate economic forecasts of experts who participate in the Survey of Professional Forecasters. Specifically, they:

1. Provide the LLMs with detailed participant characteristics (demographics, education, job title, affiliated organizations, alma maters, degrees, professional roles, location and social media presence) and then prompt the LLMs to mimic forecaster personas.
2. Ask each persona to respond to survey questions using real-time economic data and historical survey responses.

They further explore which persona characteristics affect forecast accuracy. They address the issue of potential LLM look-ahead bias by telling the models to use only information available at the time of forecasting. Using the specified forecaster persona and economic/historical forecast data, they find that:

(more…)