multiple testing

Hi,

It’s well known that the methodology used in strategy quant introduce false discovery of the strategies.

http://link.brightcove.com/services/player/bcpid2207682327001?bckey=AQ~~,AAAAt6-8xFk~,r8nyyutNWdB9cOzqmiiiQxC0oXMNGP7d&bclid=2204845638001&bctid=3878525144001

http://eranraviv.com/multiple-testing/
 

http://www.financial-math.org/

see also David H. Bailey, Jonathan M. Borwein, Marcos Lopez de Prado and Qiji Jim Zhu, “Pseudo-mathematics and financial charlatanism: The effects of backtest over fitting on out-of-sample performance,” Notices of the American Mathematical Society, May 2014, pg. 458-471

Can you introduce the measure probability of backtest overfitting ??

Krzysztof

Sorry mate, but that´s all blah blah and good and old grey theory! All my strategies that SQ has found and that I have additionally curve matched to the GREATEST degree, DO work nicely going forward for months:) All I am making sure is to re-optimize them steadily, but always on the full set of data for each pair (14 years in my case).

So don´t worry about what someone thinks will NOT work, just go ahead and try yourself, that´s the best you can do. Don´t stick with the nay-sayers and grey theory, you are just wasting your time in which you could have created good strategies:)

Sorry mate, but that´s all blah blah and good and old grey theory! All my strategies that SQ has found and that I have additionally curve matched to the GREATEST degree, DO work nicely going forward for months:) All I am making sure is to re-optimize them steadily, but always on the full set of data for each pair (14 years in my case).

 

So don´t worry about what someone thinks will NOT work, just go ahead and try yourself, that´s the best you can do. Don´t stick with the nay-sayers and grey theory, you are just wasting your time in which you could have created good strategies:)

So you think that you are smarter than all those maths guys with a lot of experience….in age of 34….My experience with finance markets is like 20 years and unfortunately info from papers what i posted just confirms my view. Anyway the question was sent to the designers of SQ if they have implements additional measure. Any diploma in maths or any hard science and research experience ??

Krzysztof

another possibility is to implement deflated sharpe ratio. For sure having either Probability of Backtest overfitting or deflated sharpe ratio would increase value of your product as none of competitive products have it.

https://www.youtube.com/watch?v=KKduDAbZ4Uk&feature=youtu.be

Krzysztof

So you think that you are smarter than all those maths guys with a lot of experience….in age of 34….My experience with finance markets is like 20 years and unfortunately info from papers what i posted just confirms my view. Anyway the question was sent to the designers of SQ if they have implements additional measure. Any diploma in maths or any hard science and research experience ??

 

Krzysztof

Smarter or not, I don´t care. I make money the way I described to you, that´s all that counts in the end, that´s what we are all here for, not what someone says should theoretical NOT work. So you can stick to finding theoretical reasons and hypothesis why SQ won´t work for you, or you follow my method (or the method of many others here) and make some money.

So any chances to get this indicator ??

Krzysztof

Sorry mate, but that´s all blah blah and good and old grey theory! All my strategies that SQ has found and that I have additionally curve matched to the GREATEST degree, DO work nicely going forward for months:) All I am making sure is to re-optimize them steadily, but always on the full set of data for each pair (14 years in my case).

 

So don´t worry about what someone thinks will NOT work, just go ahead and try yourself, that´s the best you can do. Don´t stick with the nay-sayers and grey theory, you are just wasting your time in which you could have created good strategies:)

😀

Mathematicians Against Fraudulent Financial and Investment Advice (MAFFIA)

:rolleyes:

Maybe you can use it in the snippets framework.  in EA Analyzer 4.0, you can write your own java code (which will be found in SQ4 as well).  So in theory (I did not read these papers), you can wirite the calculation yourself in SQ4.

On a side note: I agree with geektrader – our main measure is to make money.  So if I am actually consistently making money, then I can have many papers saying that I should not be profitable.  There is a reason the brains in universities have proven they cannot make it in the real world.  One case in point is Long Term Capital. They are supposed to be some of the smartest brains in Finance but they failed.  Did other people make money using their same strategies during that same period.  Yes.  

With that said, that is not to say that we cannot learn from these academic papers and why Mark has put this snippet functionality into the software, so that we can test these theories they put out and see if it will make us money.

-Stearno

geek trader does not make money

http://www.myfxbook.com/members/geektrader/quantclimber-private/871117

all those ups and down of equity curve  this is just a movements within its variance – only by generating a lot of trades and long observation period you can verify if some system is profitable.

I don’t know if it is possible to extend functionality of SQ but i know that it is very buggy – MT4 backtest results and SQ backtest results almost never match to big extend. Also functionalities of EA analyzer are done it that way that fool you with backtest results.

https://strategyquant.com/forum/topic/3692-portfolio-optimization/

https://strategyquant.com/forum/topic/3476-script-example/

Krzysztof

No thats geektrader system i believe. He claims making money using SQ….

Well. To each his own when it comes to opinions.

Please don’t take offense when someone has a different opinion, and please do not attack someone.

The part where your side broke down is the big assumption you are making is that you have seen the results of all his systems / manual trading. I already know that he does not publish his successful systems.

So showing what you think is his track record will in no way help you win an argument nor demise his opinion for me.

Thank you for sharing your idea. Since this is pre-sales forum, I assume you are evaluating to buy or not. To that I can attest from experience that SQ does work as advertised, and to answer your original question: SQ4 will be able to measure using various calculations if you can enter them using the quant editor.

Good luck to you on your trading journey!

-Stearno

Sent from my HUAWEI MT7-TL10 using Tapatalk

Well. To each his own when it comes to opinions.

Please don’t take offense when someone has a different opinion, and please do not attack someone.

The part where your side broke down is the big assumption you are making is that you have seen the results of all his systems / manual trading. I already know that he does not publish his successful systems.

So showing what you think is his track record will in no way help you win an argument nor demise his opinion for me.

Thank you for sharing your idea. Since this is pre-sales forum, I assume you are evaluating to buy or not. To that I can attest from experience that SQ does work as advertised, and to answer your original question: SQ4 will be able to measure using various calculations if you can enter them using the quant editor.

Good luck to you on your trading journey!

-Stearno

Sent from my HUAWEI MT7-TL10 using Tapatalk

 

OK, I got this link from somebody, saying its his track record, on the link it says clearly geektrader and quantclimber. Anyway i will be only convinced that it works if i see independent verified (like fxbook) track record from quite long period of time with many trades. Sadly that he don’t publish his track  record as in this case ‘that it makes money’ its only words…..

Krzysztof

Does SQ generate strategies? Yes

Do the resulting strategies produce useable expert advisors? Yes

Are there bugs and little annoyances with SQ? Yes, but no show stoppers

Can we prove the output is fool proof and there is no risk of failure? No

On balance, is the product worthwhile? Yes, in my opinion.

Can SQ be improved? Yes, a new version is under development.

Hope this helps.

Are there bugs and little annoyances with SQ? Yes, but no show stoppers

 

If there is a mismatch between MT4 backtest results and SQ backtest results to big extent as it is a case it is definitely showstopper.

If you can’t set the range in EA analyzer or make kind of walk forward test for created portfolios or ‘what if’ results its also a showstopper.

The way how the portfolio master and ‘what if’ is designed force you to believe in backtest results as you can not make forward test of this functionality.

Because of all this ‘the backtest generated’ level of optimism of SQ users is growing……

Krzysztof

If there is a mismatch between MT4 backtest results and SQ backtest results to big extent as it is a case it is definitely showstopper.

 

If you can’t set the range in EA analyzer or make kind of walk forward test for created portfolios or ‘what if’ results its also a showstopper.

 

The way how the portfolio master and ‘what if’ is designed force you to believe in backtest results as you can not make forward test of this functionality.

 

Because of all this ‘the backtest generated’ level of optimism of SQ users is growing……

 

Krzysztof

I  have found in general, SQ backtest results and MT4 results align.  If they are not, I suspect you are not backtesting in MT4 correctly.  Getting MT4 to backtest propertly needs third party support such as Birt’s Tick Data Suite.

Sounds like you’ve made your mind up, SQ and EA analyzer are not the products for you at present.  Maybe some future versions will align with what you are looking for.

Good luck in your trading.