How many years of data to generate strategies on?

Hi guys,

 

Just wondering if my data goes back to 2003, should I test from 2003 to 2015, leaving 2 years for OOS tests or does it make no difference if i just use say 2010 to 2015? Also, does testing on a longer of period data mean the EA will last a longer time?

 

And lastly, how often should i update my data using Tick Downloader?

Thanks everyone, appreciate the support.

Have a great week ahead.

Hi Shawn, I personally use the following:

1. Five years: IS

2. Five years: OOS

3. Four years: 2nd OOS

 

I do not leave data for a 3rd OOS test, as many advocate. And note that the other periods are not what is commonly advocated; but I’ve found these periods to work out quite well so far.

 

The more years of data you use, the better developed your EAs can be AND the more data you have to verify using OOS tests. A longer period of data will make your EA more solid.

 

I would guess updating your data once a month is OK. The more recent your data, the better of course, but it must be balanced with how much time it takes.

Hi qattack, thanks for your reply.

 

I have a few questions. Is leaving out the 3rd OOS simply because 1 year is insufficient or that the idea of a 3rd OOS is redundant? I would love to discuss more on this.

 

Also, what are your reasons for having an OOS = to IS range and using so many years for the 2nd OOS test?

 

I guess the 2nd and 3rd OOS would try to filter strategies which are profitable recently as well as on a yearly basis, but then that again might be illogical as we cannot expect strategies to be profitable year on year?

 

Hope to hear from you soon. Thanks.

Yes, I leave out the 3rd OOS test both because *IN MY OPINION (and contrary to many who know much more about SQ)* 1 year is too small of a sample size AND we already have two OOS periods — and in my case both of these periods are much longer than what is normally set.

 

And there is a third reason: I would prefer to absorb the 3rd OOS period into the other two so that those are given more accuracy.

 

I have found that strategy generation on five years is yields an abundance of strategies that survive further testing. And again, this leaves more data for the 1st OOS period. Thus, more strategies will be weeded out before you start further testing. As a nice side effect, it is easier to compare the initial IS and OOS results if you wish, though I’ve completely stopped viewing these stats as they really don’t mean anything once you’ve harvested the strategies.

 

If we had more data to work on — maybe four more years — I would include a 3rd OOS test. But the 2nd OOS is where all the real testing is done, and the 1st OOS is important to verifying initial results…both much more important sample periods.

I use 5 years IS 5 years OOS,

IS=05/05/2003 ~ 01/01/2009
OOS=01/01/2009 ~ 01/01/2013
#2 OOS=01/01/2013 ~ END OF HISTORY

the #2 OOS comes very last i dont use it at all when i generate strategies.

 

good luck 🙂