Curve fitting of EA (overfitting) countermeasures

☆Backtesting is not reliable?

Frequently asked questions include“Is there any test results from around 2000 onwards?”We often receive questions like this, but by performing optimization, any currency can be made to show excellent test results.

With a single EA, even for all five currencies, it is possible—though it takes a bit of time—to create an EA that shows a right-upward booming profit graph from around 2000. This is possible by using an original method.

This can be considered a special skill, and it is possible to create a right-upward EA even for all 10 currencies simultaneously.

However, the problem starts here.

Even an EA with an extremely optimized median can have subtle discrepancies in real operation, andit may be difficult to produce resultsin reality.

If the past test period is long, it becomes more reliable for the futureto a certain extent, but in practice, test results are not always very reliable and should not be overtrusted.

☆ Why backtests are not so reliable

EA products come with backtests, and aren’t most of them showing an upward-right graph?

Some reviews lament, “There is a large gap between test results and real-world operation.”There have been times when even with good test results, real performance was poor.

I have personally experienced phenomena where the test results looked good but real performance was not.

“Finally completed! Amazing verification results! This should work!”

and when I boldly applied it to real trading, it often didn’t go well.

An EA optimized to perfection did not allow the market to stay winning forever.

If an EA that was optimized on past data could win, wouldn’t that feel odd? It isn’t that simple.

☆ Even if test results are excellent, why does it fail in real life?Why?

An EA optimized for a past period iscurve-fitted,

・It works only under certain market conditions and becomes useless when the conditions change.

・For example, if the tested currency's period was a rising-buy market, in real markets a clear selling scenario might occur, yet tests would repeatedly judge buying.

If the test period is biased toward buying or selling,the test results may look excellent while the actual EA is biased.

In other words, because the test period must be specified,it is nearly impossible to optimize tests with a completely balanced buying and selling period.

Conclusion: “At least good test results are an important basis for credibility, but they alone cannot determine superiority.”

This is the conclusion.

☆ Daily results of ultimate past verification

☆ This entire folder is MT4

For nearly five years, I pursued “a universal median,” installing MT4 on more than 40 computers, pushing the SSDs to the edge (SSD health down by about 60%), and thoroughly combining various indicators for optimization.

Probably there is no one in Japan who has done so much “past verification” and “optimization.”

I placed about 50 indicators in a row and optimized all 10,000 combinations. I even pushed to such an extreme level.

As a result, the conclusion was that the initial trend reversal around near the head-and-shoulders and inverse head-and-shoulders patterns provided the best entry points, but even thenit still didn’t make me consistently win.

※ While the entry points were indeed good, they alone were not enough; simply fitting a past market into numbers was insufficient.

While still far better than random trading,

even if you formulate a system based on past markets, it remains incomplete and unreliable... The past is just the past...

This led to the conclusion.

At this point, nearly four years of time and money had passed, and I felt despair, as if my whole body was shrouded in darkness.

☆ Yet past verification is not a waste

Some who reach the same conclusion think that past markets with verified results are entirely different from the real markets on the right side of the chart,

and that past verification is a waste; only watch the live market

But I think that is not entirely correct.

To win in future markets, you need a fixed, rule-based logic

and that logic mustalso work in the past.

In other words,to prove whether the logic works in the future, rigorous past verification is still necessary.。

“Past data is not reliable, yet past verification of the logic is necessary.”

How can we resolve this delicate balance and create EA that keeps winning?

After five years of trials, finally,

I found a way to create EA that works in real markets

That method is to prioritize something other than past verification

I plan to publish this someday.

When this is done,

thus avoiding excessive optimization (overfitting) in the past,test results will no longer be the absolute highest, but,

intentionally in that state rather than temporarily,to continuously produce results,

we have completed an EA capable of fundamentally judging the market.

☆ Building an ideal trading system with EA (automatic)

And all the past five years of verification were not in vain.

Because I tried various indicators and combined them, I learned what to combine from entry to profit-taking to be effective.

Programming and forming a fixed method like “When this happens, enter” (logic) into an EA is quite feasible.If you tend to shy away from EAs, it might be because you think it looks difficult to handle yourself, or because products’ EAs do not fully disclose their logic, leaving you unsure how entries and exits are decided.

From a developer’s perspective, there are far more options than discretionary (personal judgment) trading, enabling perfect and ideal trades, thus

EA tends to outperform discretionary trading in many aspects..