A reader requested a test of the stock market forecasts/targets at the end of the archived “Pictures of a Stock Market Mania” articles at Alan M. Newman’s Stock Market Crosscurrents, which promises ” powerful commentary and unique perspectives that cannot be found anywhere else.” Using the forecasts for the S&P 500 index from the 40 archived articles spanning 2/24/01 through 3/15/09, along with contemporaneous actual S&P 500 index data, we find that:
The Crosscurrents forecasts/targets frequently include qualifications/embellishments that makes testing difficult, as follows:
- Many forecasts/targets offer ranges of the S&P 500 index rather than definitive values. For testing, we use the middle of ranges.
- Many forecasts/targets include assignment of odds for their accuracies. Some have “even odds,” which seems to make them non-forecasts. Others have low odds, which seems to make them anti-forecasts. We ignore the odds qualifications, except for three instances involving alternative outcomes/odds for a single targeted result.
- Some embellishments specify the months during which highs or lows are likely to occur or make other kinds of forecasts; we record those embellishments (right or wrong) but ignore them in calculating quantitative forecast accuracy.
- Some forecasts/targets are reiterations across several articles. We included reiterations as separate forecasts since the forecaster has more information for each subsequent instance.
- Forecast/target horizons generally shrink during each calendar year. Forecasts over short horizons should be more accurate than those over long horizons. We ignore forecast horizon in this test, mostly because grouping forecasts by horizon yields very small subsamples.
- One intra-year forecast predicts a high for the year below the high already observed for the year; we assume this forecast is for the balance of the year rather than the entire year.
The table below summarizes test results. It includes forecaster comments on odds and other embellishments so that readers can decide for themselves the meaningfulness of the forecasts/targets and the appropriateness of the test methodology. The table also includes some untested targets for 2009.
Across all tested Crosscurrents forecasts/targets, the average error is -12%, meaning that the forecasts have tended to be too low by a substantial amount during the test period. The average absolute error is 20%, meaning that the forecasts/targets are either too high or too low by 20% on average. The median absolute error is 10%, so a few very bad forecasts make the average absolute error high.
For benchmarks, we refer to our test of The Annual Business Week Stock Market Forecasts. For the years 2001-2008 of that test, the average absolute error for stock market experts predicting the year-end S&P 500 index is 18%, while that for a mechanical extrapolation of the historical trend is 15%. In comparison, the accuracy of the entire sample of Crosscurrents forecasts/targets is a little below average. However, many of the Crosscurrents forecasts have shorter horizons than those from the study of Business Week experts, suggesting that the Crosscurrents forecasts should be more accurate.
Some method of incorporating comments on odds into forecasts/targets might improve the result for Crosscurrents.
In summary, the accuracy of the Crosscurrents forecasts/targets for the S&P 500 index during 2001-2008 is probably below the average accuracy achieved by other stock market experts.
In an 8/3/09 email, Alan Newman contested this review, which he regards as “grossly unfair” because it is based upon commentaries from “a FREE web site that is updated only every three months” and because there are “huge problems” with the analysis. See his full complaint and the response for details.
For assessments of the forecasting accuracy of other investing experts, see Guru Grades. For other research on expertise, see Blog Synthesis: The Wisdom of Analysts, Experts and Gurus.
