Many gurus express certitude in their forecasts for asset classes (markets) and specific assets and for their associated investment strategy recommendations. How can they be so sure? If they are not really sure, why would they say they are? In his July 2010 paper entitled “Policy Analysis with Incredible Certitude”, Charles Manski categorizes incredible analytical practices that underlie certitude. His context is public policy, but substitution of “investors” for “public” and “investment strategy” for “policy” seems apt. Based on past study of problems that limit credible prediction, he proposes that:
- There are four categories of incredible analytical practices:
- Conventional certitudes (conventional wisdom) – predictions (indicators) that experts generally accept as accurate, but are not necessarily accurate.
- Dueling certitudes – two contradictory predictions that competing experts present as exact, with no expression of uncertainty (leading to conflicting strong investment strategy recommendations).
- Conflating science and advocacy – developing arguments (assumptions) that support an investment strategy rather than an investment strategy that supports evidence-based arguments, while portraying the deliberative process as scientific.
- Wishful extrapolation – drawing a conclusion about some future situation based on historical tendencies and untenable assumptions (ignoring differences between the historical and future situations, and emphasizing in-sample over out-of-sample testing).
- Incredible analytical practices rest on untenably strong assumptions. The credibility of inference decreases as the strength of embedded assumptions increases (equating future and past situations is a very strong assumption).
- Incentives tempt experts to trade analytical credibility for certitude by using assumptions far stronger than they can persuasively defend:
- The academic community rewards experts who produce strong novel findings.
- An impatient investor community rewards experts who offer simple analyses supporting belief in unequivocal trading strategies.
- An expert can mitigate the tension between credibility and strength of assumptions by posing alternative assumptions of varying strengths and layering the associated inferences according.
In summary, for the sake of realism, investment strategy developers should rigorously examine the defensibility of any assumptions embedded in their inference processes.
In counterpoint, might the temptation to make bold, precise (incredible) predictions derive from real behaviors of prediction consumers?
See “Why Gurus Go to Extremes” for a related discussion based on game theory and “Expert Political Judgment: How Good Is It? How Can We Know? (Chapter-by-Chapter Review)” for an in-depth empirical study of political expert forecasting accuracy.
