Are “intuitive statistics” good enough for investing? In their brief March 2007 paper entitled “We Don’t Quite Know What We Are Talking About When We Talk About Volatility”, Daniel Goldstein and Nassim Taleb report the results of a simple test of the ability of portfolio managers, traders, quantitative analysts and financial engineering graduate students to distinguish between two widely used measures of volatility: mean absolute deviation and standard deviation. Based on responses from 87 individuals to a survey question giving the mean absolute deviation for a normal distribution of stock returns and asking for the standard deviation, they find that:
- Only 3 of 87 respondents provide the correct value for the standard deviation.
- The ratio of too-low answers to too-high answers is 6.5:1, with the typical response 25% below the actual standard deviation.
- Real-life asset returns generally do not have normal distributions, and extrapolating the reasoning used by most respondents to some “fat tailed” asset returns would result in underestimation of the standard deviation by 90%. Confusion regarding measures of market variability is therefore consequential for decision making.
In summary, sloppiness in applying statistics can lead to severe misestimates of variability. People should rely on definitions, not intuitions, in assessing volatility.
