Jack Schannep, editor of The DowTheory.com, has several times requested that we reconsider parts of our assessment of his specific stock market forecasts, as follows:
Jack Schannep wrote on 12/23/05:
I had a minute so I thought I would check out if the ‘grading’ of my Letter changed from the April 10th “pedestrian” to the current “tentatively good,” but as you know the earlier one is not still posted. So I looked in on the recent one to see if I agreed with your pluses and minuses, and the results are . . . . I DO agree with you on most, particularly when you look to intermediate-term or short-term trading, but a half dozen I would submit you might review (with an eye to considering as pluses). They are:
9/3/03 “A ways to go” WAS up 4.3% and 9.3% after 63 and 253 days???
3/9/04 “Stick with the major (bullish) trend” – a year later +5.2%
4/8/04 “Still in a bull market” – yes, and we STILL are.
6/4/04 “Strong earnings growth is more of a positive factor than…” +6.4% a year later.
7/9/04 “the second half of the year will see even more highs”. Written at 10435 Dow, 1141 S&P, and 6605 NYSE (you’ll note I always include market levels in my Letters for later accountability), the prior ’04 highs had been 10737, 1157, and 6780. In the second half they rose to “even more highs” of 10783, 1211, and 7250. (63 days +2.6%, 253 days +9.8%).
I know, I know, I’m nit-picking. However, an extra six pluses changes the positive/negative ratio from 60/40 to 80/20!!! Either way, it’s your website, your judgments, and I have no complaints.
Our response:
Our 12/6/05 updated assessment of Jack’s forecasts added a “winning streak” of five straight pluses to his record of 4/10/05. Hence, we upgraded the overall assessment from “pedestrian” to “tentatively good.” Jack’s point that changing six items from minus to plus would improve his accuracy rate from 60% to 80% is an important one. This sensitivity emphasizes that the forecast sample we have for him is small, accounting for our use of the word “tentatively” in the overall assessment.
As noted at Guru Grades, in performing assessments of forecasts we rely on public information and try to put ourselves in the place of an individual investor reading them. How would such an individual react to each forecast? When an expert issues forecasts monthly, as does Jack, we tend to focus on the behavior of the market over the next month as the most important measure of the value of the forecast. With that in mind, here are our point-by-point responses to the items Jack questions:
The following is the assessment of Jack’s 9/3/03 forecast. For all of these items the four percentages represent the performance of the S&P 500 index over the 5, 21, 63 and 254 trading days after the publication date. We understand Jack’s comment, but in this case we reacted to “All…indicators are in full BUY mode.” It was not a good week or month to buy indices. We are leaving this one as a minus.
Next is the assessment of Jack’s 3/9/04 forecast. In this case, we would rather have been told to get out and stay out for a while. We think the risk-adjusted return on the index here does not justify being in stocks. We are leaving this one as a minus
Next is the assessment of Jack’s 4/8/04 forecast. We understand Jack’s literal point in his comment on this one, but there is substantial erosion of capital here after a month and still after three months. We would have felt like we were moving backward. Even after a year, the return on the index is anemic. We are leaving this one as a minus.
Next is the assessment of Jack’s 6/4/04 forecast. During 2004, S&P 500 aggregate operating earnings rose more than 20% year-over-year. The Federal Reserve began the present cycle of interest rate hikes in June 2004. One month and three months after the forecast, the index was down. We think many would agree that rising interest rates/inflation fears likely trumped the good earnings news over most of the year. We are leaving this one as a minus.
Next is the assessment of Jack’s 7/9/04 forecast. On this one, we yield. The S&P 500 index was 1113 on 7/9/04 and finished 2004 above 1200. The timeframes in the chart do not capture late 2004. We will change this one to a plus.
In checking on his comments, we found that we failed to rate Jack’s 5/6/04 forecast (next table element) during the December update of the review of his forecasts, even though sufficient S&P 500 index data was available. As noted above, S&P 500 aggregate operating earnings in 2004 rose more than 20% year-over-year. The Federal Reserve began the present cycle of interest rate hikes in June 2004. We think that interest rate hikes/fears likely dampened the stock market’s response to these strong earnings. We are making this one a minus.
The net effect of these changes is that Jack’s positive/negative ratio increases from 60/40 to 61/39. We have updated Guru Grades accordingly.
Jack Schannep wrote on 10/16/06:
I can’t believe you still use the 5-day timeframe when judging my letter, particularly since that is not my focus and you usually don’t receive the synopsis of my monthly letter within five days of publication. It is posted on the last day of each month and dated the next day, the 1st of the coming month.
I guess the 21-day timeframe is fair although that isn’t my focus either. The 5-day part of the evaluation process would seem to be the basis for the ‘minus’ on several dates (see 1/7/05, 3/4/05, and 4/8/05), if so I would hope you will re-evaluate them using the more appropriate timeframes??
I WILL acknowledge that the ‘minus’ for my cautiousness on your 8/12/06 issue IS warranted (and well deserved!). Incidentally, I reversed that outlook last Thursday.
One final thought — You marked down my 12/2/05 letter because (I suppose) 10940.55 was not reached in December (it was reached three (3) trading days later on 1/6). The rest of the quote was/is correct. That warrants a ‘minus’???
Our response:
For the reasons Jack cites, we have not considered the 5-day stock market return in grading his forecasts. However, because his forecasts are no more frequent than monthly, we have redone his grades with a 126-day (6-month) return substituted for the 5-day return. Depending on the specific forecast, the 126-day return may significantly affect the evaluation.
The following is the revised assessment of Jack’s 1/7/05 forecast. For all of these items the four percentages represent the performance of the S&P 500 index over the 21, 63, 126 and 254 trading days after the publication date. We are leaving this one as a minus because the Dow Jones Industrial Average (DJIA) (Jack’s focus) did not come close to setting an all-time high during 2005, for which this entry offered an outlook. In fact, staying out of broad market indexes was not a bad idea for most of 2005. The “no new highs” for the S&P 500 and the NASDAQ was not a challenging call.
The following is the revised assessment of Jack’s 3/4/05 forecast. We are leaving this one as a minus because broad market returns are negative at one, three and six months, and the 12-month return is well below average.
The following is the revised assessment of Jack’s 4/8/05 forecast. We are switching this one to a plus, but it is a close call. Returns at three and 12 months are acceptable, but returns at one and six months are not.
The following is the revised assessment of Jack’s 12/2/05 forecast. We are also switching this one to a plus. Jack is correct that he missed the timing of the first DJIA target by only one month, and the index reached the second target this month. The zero six-month return from the broad market gives pause, but the forecast is reasonably specific.
The net effect of these specific changes, plus other changes from reviewing all of Jack’s other forecasts with the six-month return data, is an increase in his positive/negative ratio from 60/40 to 62/38. We have updated Guru Grades accordingly.
Jack Schannep wrote on 1/30/10:
The recent update seems to show the same numbers for me (52,33,19) as last month, even as an entry is now rated that was not previously….my count comes up to 50,32,18 for 64%. I see one unrated (11/7/06) that should be a plus and two of the recent unrated ones (too soon to judge?). Is your adding calculator lying to you or am I missing something?
Our response:
The new accuracy rate (as of 1/30/10) is 33/52 = 63.46%, up from 32/51 = 62.75%. Excel rounds both to 63%.
The entry from 11/7/06 is probably of interest to those reviewing your past comments, but it is not gradable per the usual Guru Grades time-scale. It is mostly right now, pending the Dow hitting 16,000 now that the recession is over.
The bullish forecasts implied by the 6/12/09 and 10/16/09 entries need more seasoning. There are quite a few open comments among other experts like the one from 6/12/09, either calling a low in March 2009 or calling a resumption of the bear market to a new low.
As always, we invite readers to make their own judgments.
We appreciate Jack Schannep’s taking the time to review evaluation details.
