Sunday, February 04, 2007

Testing Period Affects Results

I've been complaining about this for some time - the fact that the market keeps going up and for a filter designer, especially one that uses TA exclusively, it makes it almost too easy. I've warned you in these pages that some of the results of these filters are probably more a result of market effects than of my genius.

To be fair I've written some pretty crummy ones too - so I know that you can write a loser even in a bull market but I do want to assure you - if I believe that the results are being cooked by market forces I will either not publish the filter or I will warn you ahead of time that the results are not exactly kosher.

To give you an example - last week I wrote about a new filter that I had designed looking for Blow-off bottoms and I said that it tested pretty good. In fact I said I tested it over three overlapping periods and it printed a fairly good ROI in each.

This morning I retested that filter against two periods - the most recent 90 day period and a 90 day period last year when the market was in decline. Here are the results of those two tests.

3-31/8-01 Win% = 29% and ROI = -299%
9-29/2-01 Win% = 56% and ROI = 158%

As you can see - for several months last year the markets were in decline and even a fairly good filter was striking out with regularity.

Regardless - if you are using the output from even the best known stock advisers out there you really have to know which way the market as a whole is moving before you can understand your personal results. In a bull market - everyone is a genius.

In another post - hopefully this weekend - I'll try to answer the question - can we use a good filter to forecast market health?

1 comment:


Sometimes results are not what you like.