Building strategies waiting time

I have a scenario for low efficiency in Building strategies, and would like to seek someone’s comment on similar experience or advice me if I am doing this correctly.  Please bear with me this may be a bit long, because I like to discuss about the details in 1 posting instead of scattering them pieces by pieces.

 

I am testing a forex symbol starting from 03.01.2006 to 31.01.2017, and I have the In-sample data covering 03.01.2006 to 31.12.2013.  On my initial trials, I picked H4 period to hope for faster completion by using “Selected Timeframe only” during the strategy building process.  I obtained 500 strategies within an hour, with excellent results, a lot of them survived in the Retesting procedures, and finally around 10+ strategies passed the WFO test with handsome results in Profit, PF, Ret/DD, etc.  During these robustness testing steps, I continued to use “Selected Time Frame only’.  However, none of them gave the same equity curve shapes after I placed them on TradeStation, to the extent that none of these strategies worth considered for trading.

 

Then, I conducted the Retesting procedures on these 10+ strategies again by “1 minute data”order to reach better precision.  Less strategies survived after WFO test, then I found that I had to scrap all of them away after seeing their performances on TradeStation.

 

For the sake of comparisons on testing nature, I restarted the Building strategies procedures on the same set of data by selecting time frames D1, H1, M30, M15, M5 by picking “Selected Timeframe only”.  Then, a number of strategies passed after WFO with handsome results when I picked “Selected Timeframe only” during the retesting steps.  But I scrapped all of them away after seeing their results on TradeStation.  Finally, I picked “1 minute data” to retest these D1, H1, M30, M15, M5 selected timeframe generated strategies, none of them survived after WFO tests.  I seemed to validate that the testing result pattern was the same as H4 that I began with.

 

It appears to me that it’s not viable to save computing time by using the “fastest” selections in either Building strategies process nor the Retesting process, that triggers me to wonder what are these selections actually good for, if I will never be able to use them when the tested strategies are not even qualified for simulated trading purpose.

 

So, I am restarting the Building strategies on same data set, Period H1 vs “1 minute data” selection, and I will be using this setting for retesting.  Hopefully, they will give me the results that are at least good for simulation runs in TradeStation.

 

Now, I am waiting for the Building strategies to complete.  Over the past 12 hours, they generated 106 strategies.  The memory consumption is okay, as I capped the setting at 2Mbytes and the displayed PC memory usage chart is steady which means the SQ probably won’t hang up before it completes the current run.  I dedicated another PC for the current testing run, and I don’t dare to use the PC to open up other programs in order to avoid complications on the memory usage.  Given the current steady speed, it looks like it may take 3 days to complete this run in order to reach the initial 500 strategies set.

 

In the Building Strategies blocks, I picked around 60% of the total selections.  So, if I had selected all the blocks, I guess it may take 5 days to complete generating the initial 500 strategies set.

 

We have a serious performance issue here.  If we have to use 5 days to test on just one period H1 vs 1 minute data, it will be even slower if one selects Tick simulation or Real Tick.  Further, if one tries to test all 7 possible periods for only 1 symbol vs the 1 minute or slower data, does it mean it will take 1.5 months non-stop runs to complete 7 sets of initial strategies?  This is just for 1 symbol, and there are close to 90 symbols available in TDD, which seems to be mission impossible for testing out all symbols within the TDD universe, let alone that if one may be importing additional sources of data series for testing.

 

 

I hope I had given a comprehensive description on the testing experience, and I will be grateful if any SQ gurus out there can advice me whether there may be smarter ways in handling this please.

 

Thank you !!!

I thought you could only generate Trade station strategies using selected time frame. It is the same with NInjatrader I can only use selected time frame. If I use strategies that are generated in Metatrader format and then saved as Ninjatrader they do not work at all or give very bad result this is because there is differences in how metatrader works with indicators and how Tradestation/Ninjatrader works with indicators since they are calculated on bar open for Metatrader and on bar close on Tradestation plus some other differences as well. The best comparable results for me and Ninjatrader is if I generate strategies with trade on bar open only .But trade on bar open setting really limits the types of strategies that can be found, if you do not have special bar types (renko, etc) with a “true open” to play with which can give nice strategies using setting trade on bar open.

 

Strategies that end up in the data bank and how many depends on your Ranking options. After avail you learn how look for certain strategies and then you only have to use a few settings plus you also know what ranking options to use in order to have strategies that most likely will pass the RT and WFO test’s.

Hello Mabi, thanks for sharing.  Yes, I remember the difference in Metatrader and TradeStation trade handling.  Therefore, I did not select (This Bar Open / Today Open / Enter a Market) in the Entry rules.  Would that take care of the issue by avoiding the Open prices ?  Also, for all the 3 backtest engine (MT/NT/TS) selections, I noticed that we are also allowed to select any of the all the 5 different Test precisons.  I am wondering if MT / NT / TS will take just as long if I pick the 1-minute precision.  I will try them out afterwards.  May be I am doing something wrong in this setting, because over the past 48 hours of my current non-stop run, it only collected 379 strategies in the data bank, should I stop them ? 

 

For the ranking Options, I picked something very basic, which are Net Profit, Number of trades, PF, Return/DD, % Wins, and 500 as maximum to store in Databank.  For example, I am using %Wins < 30% for dismissal.  If I pick 50%, less strategies will be passed through but I guess it will just take even longer to collect 500 strategies.  My understanding is, the more stringent conditions are set, then the more suitable strategies will be generated but will take longer time for the genetic programming to complete.  Is my understanding correct or did I miss out some concepts please ?      

Well if you make strategies with tradestation “uploaded” data you should not need to exclude anything this should be accounted for. What i meant was that since we can not use minute resolution wile generating strategies today using tradestation data the accurazy is just as bad as if you use tradestation. If you just use trade on bar open then the accuracy is very high and will reflect live trading results ( tradestation will trade on bar open only since the code will say so). If you use limit orders the biggest problem is that a certain percentage will not be filled even thought the backtest engine think so. There is also a problem with exits some time the backtest engine think you get filled on a stop or more common think that you got the profit first but in reality you actually got stopped out which can happen when the target is closer then the stop since then the backtest engine assumes that you got reach the target before the stop( on the same bar) which you actually do not 50% of the time.

Thanks Mabi on the limit/stop elaboration.  I am only using the TDD data now, because they have much longer FX data history than the TS database.  So, if I disable the limit/stop and then use trade on bar only selections in the building blocks, plus, I pick TradeStation format as the data back-test engine on these TDD datas, and then pick “selected time frame only”, am I supposed to be getting something very close to actual historical equity curve in TradeStation please ?

dont really know you have to try it. If TDD  data is in the same formate as tradestation SQ will  se it as TS data. If it se it  as MT4 data and you can change to minute resolution  you still will have a problem. In order to generate using tradestaion back test engine i think you have to convert it to tradestation formate. It does not matter if you manually change it in the program. At least that is not the case with Ninjatrader.  Ask support.