A reader asked: “Have you ever undertaken a study of the EIA Short Term Energy Outlook (STEO) or other EIA forecasts to evaluate the degree of accuracy one could reasonably expect from them?”
CXOadvisory.com generally focuses on U.S. equities for this kind of analysis. A few observations, though:
The Energy Information Administration (EIA) modeling is highly quantitative, so any analysis of accuracy should be primarily statistical rather than qualitative.
The EIA methodology appears to be largely regression-based (see the “Model Documentation” section at the bottom of “Analysis Articles and Model Documentation”). Regression methodologies tend to underestimate volatility and lag this volatility.
It would be reasonably straightforward (but time-consuming) to compare forecasted and actual numbers for each forecasted item from the archive provided by EIA. However, spreadsheets are available only back to August 2004. Working from PDFs for older data makes setup very time-consuming. Also there is a shift from quarterly to monthly reports in 1997.
EIA may well have done its own assessment of out-of-sample forecast accuracy. You could direct the question to one of their experts.
