Test precison discrepances in Build strategies and Retest

Hello, I am bumping into a scenario where it shows big differences between Test precision selections.

 

I picked a symbol and used H4 timeframe, picked “Selected Timeframe only (fastest)” to improve the running speed, and also tried out “1 minute data (slow)” to validate if the results are closed to each other.  Here are the differences :

 

 

 

“Selected Timeframe only” vs “1 minute data”

 

Time spent in generating 30 strategies in Build strategies mode : approx. 5 mins vs 5.5 hours

Strategies remained after varying timeframe & symbol Retest ï¼Å¡ 26 vs 6

Strategies remained after the randomized Robustness tests : 21 vs 0

Average run time in each randomized robustness tests : 30 seconds vs 300 seconds

 

 

I tried to wait for 500 generated strategies for comparison, but it just takes too long to wait, and settled for 30 strategies in this study first.

 

In addition, the performance statistics (Ret/DD, Profit factor, Net Profit) of “Selected Timeframe only” results far outpaced “1 minute data”. 

 

Is the result of “Selected Timeframe only” results too good to be true, or should I accept them please ?

Hello,

 

it depends on how a strategy works after all. Sometimes “too optimistic” results with selected timeframe precision can be caused by intra-bar fills. These are not correctly evaluated with selected timeframe so 1 minute or smaller data will be more accurate

 

Generally when you look for simple strategies like for example strategies using market order entries and simple PT and SL orders then it is usually enough to use “selected timeframe” precision. With advanced orders you need higher testing resolution which takes more time though to process

I see, thanks Tomas.  If I don’t allow intra-bar fills during the Build strategies stage, for example, I only select limit/stop orders based on plus/minus Closing price.  Then, during the Retest and WFO stages, I consistently use the Selected Time-frame only.  Will I be able to minimize this “Too optimistic” effect ?  The bottom line is, when I first migrate the code to TradeStation, I like to see the historical equity curve as close to SQ as possible at that point, although I know it can never be precisely identical.

‘‹Hello, is it possible to post a couple screen captures here ?  I can’t find the button that allows me to attach 2 JPG images here.  I wish to provide 2 screen caps for further discussion.  I was running WFO on a strategy under Selected Time-frame (H4) and the 1-minute precision separately.  The Selected Time-frame shows perfect result (100%) in the WFO test, but the 1-minute selection failed miserably in WFO.  The difference between them is too big to ignore.  The strategy was generated by using Selected Time-frame, and it passed through all the OOS/Robustness tests with nice results.  I am wondering if I should abandon the Selected Time-frame selection from now on please ?