Why don’t machines rule the financial world? In his September 2017 presentation entitled “The 7 Reasons Most Machine Learning Funds Fail”, Marcos Lopez de Prado explores causes of the high failure rate of quantitative finance firms, particularly those employing machine learning. He then outlines fixes for those failure modes. Based on more than two decades of experience, he concludes that:
- To address the complexities of discovering a true investment strategy, developers should not work in isolation (a portfolio of individuals), but as specialists within assembly line teams. They should focus on expected recovery time for drawdowns of any size, because recovery time failures indicate a latent strategy weakness.
- Do not rely exclusively on asset returns for strategy development. Use a combination of returns (series that are stationary but retain no path memory) and prices (series that retain path memory but are non-stationary).
- Do not use time intervals in constructing samples. Instead, use intervals that measure information content such as number of trades, volume of trades, dollar value traded, order imbalance or entropy impact of trading.
- Do not use testing methods that ignore the path followed by prices, thereby missing conditions that would have stopped out positions (such as fund illiquidity, margin calls or risk intolerance). Include profit-taking limits, stop-loss limits and loss-of-patience (strategy expiration) limits.
- Do not assume that series of test observations are independent and identically distributed, because strategy parameters often rely on overlapping data. Instead, weight each observation as a function of absolute log returns uniquely attributable to it.
- Do not assume that training (in-sample) and testing (out-of-sample) data sets are separable by a simple date. Purge from the training set all observations whose strategy parameter measurements overlap with those in the testing set.
- Do not ignore backtest overfitting, which generates false positives (strategies that are lucky within the training data set). To mitigate, use a deflated Sharpe ratio (DSR) for backtest evaluation. In addition to return average and standard deviation, DSR considers non-normality of returns, length of the training sample, number of independent trials and intensity of data snooping.
In summary, investment strategy developers should be experts in the meaning of financial data and the effects of modeling assumptions on that meaning.
Cautions regarding conclusions include:
- The presentation is a statement of beliefs. The author does not prove that his recommendations are reliably sufficient for discovering persistently successful investment strategies.
- Implementing the recommendations implies costs to be debited from the performance of strategies so discovered.
