In his article entitled “The Seer-Sucker Theory: The Value of Experts in Forecasting” from the June/July 1980 issue of Technology Review, Scott Armstrong investigates the general supply of and demand for expertise across several disciplines. Based upon his survey of decades of research in multiple fields (including financial markets, psychology, health care, politics, sports), he concludes that:
- Expertise, above a minimal level, and forecasting accuracy are unrelated, and accuracy may even drop after a certain level. Most people can attain this minimal expertise quickly and easily. (See the figure below.)
- Because improved forecasting accuracy for experts shows up only in some studies with large samples, claims of accuracy by a single expert are of no practical value. No studies show an important advantage for expertise.
- These results come in spite of a possible bias in favor of publishing research that supports conventional wisdom (in this case, that experts are expert).
- People may pay big bucks for expert forecasts not because the forecasts are accurate, but because the buyers want to avoid responsibility for their own decisions.
- The higher (lower) an individual’s self-assessed level of expertise, the less (more) likely the individual is to use disconfirming information to alter beliefs.
- Experts are often unaware of how they make predictions.
- The key to improving forecasting accuracy is probably an active search for disconfirming evidence. Rather than an advocate of a belief, be an arbiter of alternative beliefs.
The following figure, taken from the paper, illustrates that attaining even a relatively low level of expertise offers a relatively high level of accuracy in forecasting change; beyond this minimal level, additional expertise does not improve forecasting accuracy and may decrease it.
In summary: “No matter how much evidence exists that seers do not exist, suckers will pay for the existence of seers.”
