Is use of a sampling interval much shorter than input variable measurement interval a useful statistical practice in financial markets research? In the April 2018 update of their paper entitled “Long Horizon Predictability: A Cautionary Tale”, flagged by a subscriber, Jacob Boudoukh, Ronen Israel and Matthew Richardson examine statistical reliability gains from overlapping measurements of long-horizon variables (such as daily or monthly sampling of 5-year returns or 10-year moving average earnings). They employ the widely used cyclically adjusted price earnings ratio (CAPE, or P/E10) for some examples. Based on illustrations and mathematical derivations, they conclude that:
- Long-horizon variables have inherently small sample sizes. For example, the number of independent observations of 5-year returns in a 50-year sample is only 10.
- Using overlapping measurements of long-horizon variables offers only marginal benefit. For example:
- Using monthly measurements of 5-year returns in a 50-year sample generates 600 overlapping observations, but consecutive monthly observations have 98.3% of data in common.
- Reuse of overlapping data limits the increase in effective number of independent observations to just 12.
- CAPE is a stock index divided by a 10-year moving average of earnings, neither of which varies much from month to month. Monthly autocorrelation of CAPE is therefore close to one, and there is practically no gain in statistical reliability from increasing sampling frequency from five years to one month. In other words, the increase in the effective number of independent observations from overlapping data is too small to boost predictive power materially.
- The method most used by practitioners to adjust for overlapping observations (and available in standard statistical analysis packages), the Newey-West estimator, is unsuitable when input variable measurement interval is large relative to sample length.
In summary, researchers who use sampling intervals much shorter than input variable measurement intervals greatly exaggerate reliability of findings, strongly undermining use of long-horizon variables such as CAPE for stock market forecasting and timing.
For additional perspective, see “Chapter 1: Some Statistical Practices that Make Sense”.
