We occasionally get requests from readers to review the claims stated by an online investing/trading advisory service or investment manager. Most such web sites do not provide enough information to perform a quantitative review. Here is compilation of key points to consider in evaluating the claims of advisory services:
Read the general disclaimer (often linked at the bottom of site pages). Does it say something like: “It should not be assumed that the trading recommendations or advice will be profitable or that they will not result in losses.” Do you believe the marketers or the lawyers?
Find the assumptions made about claimed returns:
Are claimed returns cherry-picked or do they comprise steady outperformance over a reasonably long period of time? (There is often not enough information to tell.)
Does past performance commence with a strong return burst (perhaps based on backtesting) and then moderate (perhaps in real trading)? Such a scenario may indicate picking a start date at the beginning of a non-repeating lucky streak. At the end of the streak, the luck is gone.
If the offeror manages multiple services, are they all outperforming, or just the one being promoted? The latter case suggests the possibility of luck rather than skill in the promoted fund.
Are claimed returns from real trading or from a hypothetical backtest? Look for disclaiming statements such as: “Hypothetical performance results have many inherent limitations… One of the limitations is that they are generally prepared with the benefit of hindsight.” Exhaustive backtesting (data mining) likely discovers non-repeating lucky streaks rather than reliable patterns. Moreover, hypothetical trade timing may assume favorable exit points that a real-life trader is very unlikely to achieve systematically.
What are the assumptions about trading frictions? Look for disclaiming statements such as: “Performance reflects gross profit or loss and is exclusive of commissions, trading fees and subscription costs.” Assumptions about trading frictions (including about buy and sell prices with respect to the bid-ask spread) can have very large impacts on profitability for strategies that involve frequent trading and/or options. Also, in general, the less money you have to invest, the more trading frictions depress returns.
Are the claimed returns clearly tied to an amount of capital required to implement them, or are capital requirements hazy? If the frequency of an offeror’s recommended trades is sometimes so slow that assumed position sizes leave you with cash, or so fast that you do not have the cash to exploit them all, then your portfolio returns may be substantially lower than the offeror’s per-trade statistics.
Are the claimed returns for closed positions only, suggesting the possibility of a bow wave of bad trades kept open and excluded from analysis.
How do the offeror’s claims square with your beliefs about the level of financial markets efficiency and about human nature? Specifically:
Why have market makers, hedge fund managers and other sophisticated investors not discovered and extinguished the related market inefficiency?
If the offeror can reliably generate very large returns, why is he bothering to sell and administer an advisory service?
Is there credible research that supports the reliable outperformance of the type of strategy employed by the offeror?
Is there credible research that supports the existence, in general, of reliable returns of the magnitude claimed? In other words, does it seem too good to be true?
In summary, this compilation of suggestions and questions can help investors/traders assess the credibility of returns claimed by advisory services.
