Back Testing Difference between StrategyQuant and MT4

(I think there are many other threads with the same topic, but since my issue has not been solved, therefore I have created this thread here)

 

The issue I have been facing is that the back testing results of StrategyQuant  and MT4 are totally different, not slightly.

 

StrategyQuants (SQ) creates a lot of profitable strategies with quite high stability, but when tested the same EA in MT4, almost all of them are making significant losses.

Without understanding the reason of difference, the output of SQ is almost no meaning. Therefore I would like to understanding the reasons.

 

 

Followings are current situation of me.

 

– I have chosen the back testing engine of SQ as “MetaTrader”  

– For the back testing data for SQ,  I use USDJPY data from 2010-2016, those are divided by Training, Validation, and Out of Sample. I use the same period of data on MT4 also.

– I have picked up 10 strategies generated from SQ, which are profitable with high level of robustness in Training, Validation, and Out of Sample period. On SQ, used 20 monte-carlo simulation for each strategy and all shows profitable with 100% confidence level.

– As for the spread, I intentionally set much higher spread than my broker provides for SQ, and put the actual spread into MT4. Therefore, MT4 results should be more profitable that that of SQ.

– However, when I put those SQ generated EAs to MT4, all the 10 strategies make significant losses. Some EAs even make the fund to zero.

 

I am not sure what is making the differences and struggling.

Waiting for your hints and solutions.

Just adding an information: my broker is 6 digit broker and the price is like 112.676 for USDJPY.  This might be a reason?

HI, please can you attach some strategy and more information about your mt4 backtest (screenshots,mt4 reports and etc.)? Which data you use in mt4?

Hello,

 

yes, please attach at least one of your strategies that is causing you troubles and provide as many info as you can so we can help you find where the cause is. There can be many reasons for significant differences such as:

• using different data (mainly different time zone setting MT4 vs SQ3)
• using different backtesting precision. Tick data testing in SQ3 and “bar open” precision in MT4
• incorrect (different) EA setting – please check your EA setting is correct when executed in MT4
• using very complex EAs (too many indicators incorporated etc)

Just adding an information: my broker is 6 digit broker and the price is like 112.676 for USDJPY.  This might be a reason?

just an FYI, thats not a 6 digit broker, thats a 3 digits…

you start counting from the decimal, this is to figure out if you got full pip or fractional pip broker

That question reminds me of issues I found when I began running tests comparing SQ with MT4 results.

1. First problem was found by examining the EA logs output in MT4. They reported “unable to find indicator…”. I simply had to import the indicators used in SQ.

2. You have to make sure you use the same quality of data. I used tick data obtained from Birt’s Data Suite. Although I develop using 1M data, I always do final tests using tick data.

3. Spread in SQ is expressed in pips while it’s expressed in pipettes in MT4. In other words if you built your strategies with spread=2 in SQ you will need to test with spread=20 in MT4.

 

Right now that’s all I can think of, but I am sure there are other possible issues, as I found out the hard way that a lot can go wrong.

yeah…. SQ3 is too buggy,

lets just wait for SQ4

yeah…. SQ3 is too buggy,

lets just wait for SQ4

Actually I would not say that.
It does what I bought it for. I have so far built 6 strategies which, when set up correctly, resulted in virtually identical results in MT4. So it works. 

It could have a better memory cleanup, and also for some strange reason, even though the work is done, it keeps eating CPU cycles. So, yes, it contains some bugs, but none that prevents me from doing a lot of productive work.

Actually I would not say that.
It does what I bought it for. I have so far built 6 strategies which, when set up correctly, resulted in virtually identical results in MT4. So it works. 

It could have a better memory cleanup, and also for some strange reason, even though the work is done, it keeps eating CPU cycles. So, yes, it contains some bugs, but none that prevents me from doing a lot of productive work.

Sure SQ3 does what it should be doing, but i noticed “small” bugs like when using Hours or Today Open, High/Low/Open/Close of the Day,

when using those blocks in your development,

 

if you’ll test your SQ strategy and compare it to the MT4 test, you will see BIG changes, so yeah…, i wont use those blocks ever again till SQ4 comes out,

 

and to your remainder or attention if you still do not know,

 

SQ3 wont use in consideration the Leverage/Margin + Swaps, so keep this in mind it can influence your performance much….