[Episode 13] Backtesting breaks if you overfit it too much.
There are moments of wonder when you improve the logic.
Just by adding one condition, the backtest performance can improve dramatically.
Win rate goes up.
Drawdown decreases.
The profit and loss curve becomes an even cleaner upward slope.
At first glance, it looks like a great success.
However, having validated 22 years of USD/JPY data, I become cautious about such results.
I distrust any logic that has become too good.
Backtests are dealing with the “past.”
There is something we must not forget here.
What backtests are looking at is not the future.
It is historical charts.
In other words, the more conditions you add, the more you can tailor to past price moves.
In reality, if you only look at the past, you can keep tightening the numbers further.
However, there is no guarantee that such improvements will apply to the future.
“Improvement” and “overfitting” are different.
When I create a logic, after adding a condition, I always ask myself:
“Is this really an improvement?”
Or,
“Have I merely tailored it to a single past moment?”
This distinction is extremely important.
Because a logic optimized only for the past can have its expected value collapse if the market environment changes even slightly.
Even if the numbers look good on the surface, you may be losing “room for the future” behind them.