We review here the performances of 27 professional and six amateur investors/traders who have participated in 12 rounds of MSN Money’s “Strategy Lab” since late 2001. “Strategy Lab” involves a series of multi-month tests of the investment strategies of six participants at a time. The rules “give each trader a hypothetical $100,000 portfolio to manage. The traders are free to deploy the cash using their unique strategies as they see fit. Some might choose to invest all of their money on the first day. Some might never be fully invested… Before the strategists can make a move in their portfolios, they have to tell you about it in their journal entries.” Here’s how they have done:
Results comprise a total sample of 66 investing/trading records for:
- Round 7 (12/3/01-10/11/02)
- Round 8 (1/11/03-8/15/03)
- Round 9 (8/26/03-3/11/04)
- Round 10 (3/18/04-10/14/04)
- Round 11 (10/22/04-2/25/05)
- Round 12 (4/28/05-10/21/05)
- Round 13 (10/31/05-6/1/06)
- Round 14 (6/16/06-12/21/06)
- Round 15 (1/8/07-6/28/07)
- Round 16 (7/16/07-1/18/08)
- Round 17 (1/28/08-7/25/08)
- Round 18 (8/1/08-2/13/09)
The following chart depicts the average performance of participants by round, along with the concurrent performance of the value-weighted S&P 500 index for comparison. Other indexes may be more appropriate benchmarks for some participants, depending on investing style. Participants in aggregate have beaten the S&P 500 index six rounds out of 12.
The next chart depicts the ranges of participant returns by round, with returns for the S&P 500 index indicated for reference. The average range of participant returns across all rounds is about 37%. This large variability demonstrates that different stock selection rationales produce very different results.
Other points of interest are:
- Participants individually outperform the S&P 500 index 45 times (63%) and underperform 27 times (38%). The seven amateur instances are five outperforms and four underperforms. Again, different benchmarks may be better suited for some participants. Relative performance is a combination of market timing and stock picking. In general, when participants do not fully deploy all cash, they tend to underperform (outperform) a rising (falling) market.
- Across all rounds, participants average a 2.0% return per round, compared to -2.0% for the S&P 500 index. However, a sample of 12 is very small, and one new observation could substantially change results.
- Across all 12 rounds, the correlation between the average participant return in a round and the return on the S&P 500 index for that round is 0.75. The R-squared statistic is 0.56, indicating that overall market returns determine 56% of participant returns. Excluding Round 7 boosts the R-squared statistic to 0.66 (with the large change indicative of a very small sample).
- Some participants have complained about the suboptimizing effect of the Strategy Lab announcement rule that prevents them from trading quickly.
In summary, the portfolio returns of “Strategy Lab” participants since late 2001 vary widely, but aggregate results offer some support for the belief that experts on average can consistently outperform the broad stock market.
As of 2/13/09, MSN Money states that “‘Strategy Lab’ is going on hiatus,” with no new rounds scheduled.
See Guru Grades for a snapshot of the accuracy of various experts in predicting the direction of the U.S. stock market, including links to evaluations of their individual commentaries. See also Blog Synthesis: The Wisdom of Analysts, Experts and Gurus for more general research on the performance of expert investors.
