What is optimization?
The idea behind an optimization is simple. First you have to have a trading system - for example a simple moving average crossover: If EMA(10) crosses above EMA(20) go long, otherwise go short.
In almost every trading system there are some parameters (indicator periods, constants to compare, etc.) that influence the system performance. The optimization means to find the optimal values of these parameters (giving highest profit or best Return/DD ratio or other desired parameter).
For example, would it be better to use rule "EMA(10) crosses above EMA(20)" or rule "EMA(15) crosses above EMA(50)"?
Optimization can help you find the values that returned the best performance in the past.
What is Walk-Forward optimization/analysis?
Walk-Forward optimization is generally a special type of backtest that is composed of multiple smaller backtests on optimizaiton periods. These optimization periods are split over the whole backtesting period and are always followed by Out of Sample tests with the optimized parameters.
It is a technique in which you optimize the parameter values on a past segment of market data, then verify the performance of the system by testing it forward in time on data following the optimization segment, and the process can be repeated over subsequent time segments.
How Walk-Forward optimization works
in walk-forward optimization, the data are divided into a configurable number of periods (5 in this example). Each period consists of optimization part and run part.
The program starts with optimization period 1. It will run the simple optimization on optimization period 1 to find the best parameter values. These parameter values are then applied to run period 1 - strategy is trading with the optimized parameters found in previous step.
At the end of run period 1, the system again runs simple optimization on a part of data marked as optimization period 2. It finds the best set of parameter values and they are again used for trading in run period 2.
This continues until the period 5, which is also the end of history data used in test.
Walk-Forward optimization simulates how you could work with the strategy during real trading - you can optimize it on some historical data and then trade it with the optimal values. After some time you'd want to reoptimize it and let it trade again.
What Walk-Forward optimization/analysis tells you?
It basically tells you if the startegy is robust enough and if its performance can be improved by reoptimization.
If strategy performance is worse during reoptimization than the original non-optimized startegy, it is a big signal to watch for curve fitting.
On the other side, if Walk-Forward optimized strategy performs better than non-optimized version on the same data, it tells you that :
- Your strategy will benefit from optimization, so you should periodically reoptimize it to get the best performance
- It also means that the startegy is robust enough to cope with market changes (using reoptimization) and there is a big chance it will work also in future.
Walk-Forward optimization example in StrategyQuant
Performing Walk-Forward optimization in StrategyQuant is simple, in the next few lines I'll show the complete process.
For simplicity we'll use EMA Cross strategy in this example. Note that this strategy in the basic form is NOT profitable, and reoptimiation will not help it, but it is simple enough to demonstrate how optimization works.
You can download the strategy using the link below - Click with right mouse button and choose Save as...
- Loading a strategy for optimization
- Setting optimization values
- Configuring walk-forward runs
- Checking the results
- Interpreting the results
- Description of advanced WF score components
Step 1: Loading a strategy for optimization
First of all, you have to switch to Optimizer window and load the strategy you want top optimize.
For this example we'll use simple EMA_Cross strategy that goes long when faster EMA crosses above slower EMA, and go short when faster EMA crosses below slower EMA. After you loaded the strategy, it is added also as Original strategy to the Optimization results databank.
You can double click on the Original strategy and then go to Results -> Source code to see its rules.
Make sure you check the Put values to parameters checkbox so that you see that variables pLongEMA_1, pLongEMA_2, pShortEMA_1, pShortEMA_1 are used to store indicator parameters. In our optimization we'll try to find optimal values of these parameters.
There's still one small problem. We can see that the strategy uses different parameters for long and for short direction. We can use it like this if we want to find optimal values independently for long and short side, but for our example we'd like to use the same parameter for long and short side.
We can do it in program Tools -> Options -> Strategy parameters.
If you check the first checkbox, it will use the same parameters for long and short direction (providing the rules are the same). Click OK to store the settings and refrest the source code.
Step 2: Setting optimization values
To set up values that will be optimized we have to go to Settings -> Parameters
Here you can see the list of all strategy parameters that are available for optimization. Optimization simply means trying different values of input parameters.
For every parameter you want to optimize you have to check the line of the parameter and choose Start, Stop and Step values. The optimizer will iterate the value from Start to Stop, taking Steps. Original value is also configurable, it will be used to retest the original strategy. You can use this value to compare performance of new results with the "original" settings.
The Number of tests value shows us how many tests have to be performed to test all the combinations of the values.
It is possible that your parameters table will contain much more parameters, it could look like this:
This is another powerful feature of StrategyQuant. It allows you to optimize not only strategy parameters, but also other trading options, such as how many trades to take per day, or what should be the time range for trading. These settings are normally a part of Strategy Options, but you can also optimize their values.
Step 3: Configuring walk-forward runs
We have to specify also the walk-forward settings. In this example we'll use 30% Out Of Sample (Run) period and 6 reoptimization steps.
Out of Sample %
this means how much of the whole period is left for run. If we'll set it to 30 %, it means that in each period 70% of the data are used for optimization, and 30% will be used for trading using the optimized values.
Walk Forward Runs
this means how many optimization runs there will be, which means how many times we'll reoptimize the strategy.
It is also possible to specify the optimization (In Sample) and run (Out of Sample) periods by exact days, you can do it by checking Define specific number of days.
Step 4: Checking the results
Walk-Forward optimization takes longer than a simple one, because there a 6 (or more) optimization steps instead of just one.
When it finishes, we'll see that we have only two results in the databank - Original strategy and Walk-Forward result.
When we'll double click on the result in the databank, the strategy results will open.
We can see that the strategy failed in walk-forward test, based on our robustness score settings.
Robustness score is fully customizable. We can set all the conditions we want to watch in the Robustness score components table (1) and set their boundary values.
The main Robustness score threshold value (2) means how many of these scores used must pass in order for WF result to be considered successful.
On the right from this table we can check the results for each optimization and run period:
You can see that only 2 out of 6 runs ended up in profit.
We can also check the equity chart:
Blue line represents reoptimized strategy, while thinner gray line represents original non-optimized strategy.
Interpreting the results
How should we interpret these results?
First of all, it is clear this particular strategy doesn't have better result when reoptimization is used.
Surely this strategy isn't profitable in its original form and it WASN'T MADE PROFITABLE BY REOPTIMALIZATION.
But what if we'd reoptimize the strategy more often? Or if we'll use different Out of Sample %? How can we tell what is the best reoptimization period for strategy?
This is where Walk-Forward Matrix comes to play - check the next article.
Description of advanced WF score components
There are several special score components that you can use in the Robustness score components table to determine if the Walk-Forward optimization was successful or not.
WF Stability components allow you to compare performance between optimization and run (averaged by number of days, so it is not dependent on relative size of each period).
Usually we can expect that strategy performance AFTER optimization is worse than the optimized part, because optimized part already was optimized for the best performance. The conditions below allow us to set boundaries how big performance decrease we are willing to accept:
- WF Net Profit Stability - Net profit performance in run vs in optimization part (in percent).
Value above 100% means that strategy performs better in run than in optimization part.
Let's say you specify condition WF Net Profit Stability > 60%.
This means that performance in run part (after optimization) should be at least 60% of performance in optimization period. So, for example, if the strategy made $1000 in optimization period, it should make at least $600 or more after optimization period to pass this condition.
This is important to evaluate because we want our strategy to perform well after we optimize it and this condition allows us to control this - in our condition we let the optimization pass only if the startegy performs at least at 60% of the optimized performance.
- WF Drawdown Stability - Drawdown in run vs optimization part (in percent).
Value above 100% means that strategy has worse drawdown in run than in optimization part.
Let's say you specify condition WF Drawdown Stability Stability < 200%.
This means that drawdown in run part (after optimization) should be less than 200% of drawdown in optimization period. So, for example, if the strategy had drawdown $400 in optimization period, it should have drawdown smaller than $800 after optimization period to pass this condition.
This is an opposite example to Net Profit - we want drawdown to be as small as possible, but we can allow strategy to have worse drawdown after optimization.
- WF Return/DD Stability - Average Return/DD ratio in run vs optimization part (in percent).
Value above 100% means that strategy has better Return/DD ratio in run than in optimization part.
- WF Sharpe Ratio Stability - Average Sharpe ratio in run vs optimization part (in percent).
Value above 100% means that strategy has better Sharpe ratio in run than in optimization part.
- WF Profit Factor Stability - Average Profit Factor in run vs optimization part (in percent).
Value above 100% means that strategy has better Profit Factor in run than in optimization part.
- WF Annual % Return Stability - Annual % profit in run vs optimization part (in percent).
Value above 100% means that strategy performs better in run than in optimization part.
- Number of profitable runs - How many Walk-Forward runs (in percent) were profitable
- Max profit in one run - Maximal value of Net Profit in all the runs.
- Max profit in one run as % of total - Maximal Net Profit in all the runs as percentage of total profit.
Ideally we don't want any run to have too big % share on the final profit because it would mean the rest of optimiztion periods were not effective.
- Max Drawdown in one run - Maximal value of Drawdown in all the runs.
- Max % Drawdown in one run - Maximal % value of Drawdown in all the runs.
- Max Stagnation in days - Maximal stagnation (trades without new equity high) in days.
- Max Stagnation in % - Maximal stagnation (trades without new equity high) in percent of all trading days.
- Min trades in one run - the smallest number of trades in all the runs.
Ideally we want each run to have reasonable number of trades to make the results valid.