A reader commented and asked: “It seems there are indeed systems in Collective2 that make money on a walk-forward basis even considering trading frictions. VT26 is one of them (for autotrading at Collective2, you can assume on average $8.50 per contract per round turn). How does this stand versus your skeptical approach?” The author of VT26 describes it briefly as a 100% automated volatility breakout system with priorities on steady profits and small drawdowns. He provides further description of the strategy in “Report on Systematic Portfolio VT26.” Using trade-by-trade data for VT26 for the period 4/9/08 through 3/8/10 (2,279 closed trades over 23 months across ten futures markets), we find that:
Some metrics for the sample of VT26 trades are:
- Trading frequency is roughly 100 trades per month.
- Average trade duration is less than two hours.
- Average gross profit per trade is $63.
- Average net profit per trade for an autotrading follower (assuming $8.50 per contract per round turn) would be $50.
Is profitability consistent over time?
The following chart shows the average gross and net profits per trade by calendar quarter for VT26 over the entire sample. There is some indication that strategy profitability is weakening over time, suggesting a lucky start, but the trend is irregular.
A more precise interpretation, given the volatility-dependence of the strategy, is that VT26 effectiveness may vary with market volatility state. It may have been most effective in the second half of 2008 because of dramatically increasing and elevated volatilities across markets. If so, the two-year sample period, involving relatively high volatilities and dramatic volatility shifts, may not include enough variety in market volatility states for a thorough test of the strategy. A longer sample period incorporating many volatility regimes might produce different results.
Is profitability consistent across markets?
The next chart shows the average gross and net profits per trade by market for VT26 over the entire sample. Results show that the strategy:
- Works best for crude oil and gold.
- Works moderately well for German equities and currency.
- Does not work for U.S. small and midcap equities, the yen or the pound (perhaps even working the wrong side of trades for the pound).
The fact that the strategy works to different degrees in different markets, and does not work in several markets, undermines belief in any general underlying principle describing how financial markets work. Factors/characteristics specific to the crude oil and gold futures markets (and German markets?) may help explain inconsistencies.
Is profitability consistent for long and short trades?
The final chart shows the average gross and net profits per trade for long trades and short trades over the entire sample. Results show that the VT26 strategy signals are substantially more effective for short trades than long trades. The disparity is perhaps related to market volatility states during the sample period.
Combining this finding with that in the prior chart, we find that the average gross (net) profit per short trade in gold is $241 ($233) over the sample period.
Note that if a large number of traders were autotrading the system, simultaneous trades might crowd and drive buy prices up and sell prices down for laggards, depressing aggregate profitability for followers.
In summary, evidence from simple tests of available data indicates that VT26 may be a good strategy for fairly high-frequency trading in some futures markets, better for short trades than long trades, with effectiveness perhaps dependent on market volatility state.
Generally, Collective2 is like a large trading room. Even for walk-forward data with vig, stochastic trading processes would produce some winners among a large group of traders (as in roulette). The degree to which trading outcomes are random is a measure of the difficulty of isolating reliable performance. Incomplete/biased sampling obscures the degree of randomness. In this case, as noted above, the VT26 sample may not incorporate a broad enough variety of market volatility states. Also, ex post selection of the strategy may incorporate data snooping bias by disregarding the implications about luck from many comparable strategies in the “trading room” that did not do so well (perhaps even crashed, burned and disappeared). Ex ante selection is much more convincing (and, obviously, more exploitable) than ex post. VT26 could now be selected ex ante for further testing.
