Modeling the Level of Snooping Bias in Asset Pricing Factors

Is aggregate data snooping bias (p-hacking) in financial markets research a big issue or a minor concern? In their June 2021 paper entitled “Uncovering the Iceberg from Its Tip: A Model of Publication Bias and p-Hacking”, Campbell Harvey and Yan Liu model the severity of p-hacking based on the view that there are, in fact, both some true anomalies and many false anomalies. This view contrasts with other recent research that models the severity of p-hacking by initially assuming that there are no true anomalies. They test their model on a sample of 156 published equal-weighted long-short anomaly time series and 18,113 comparable datamined equal-weighted long-short strategy time series, focusing on series exhibiting alphas with t-statistics greater than 2.0. They present in detail differences in conclusions for the initial assumption that there are some true anomalies and the initial assumption that there are no true anomalies. Applying their model to the specified time series, they find that:

(more…)