Do Not Trust Backtesting Too Much — Five Reasons It Might Diverge From Real-World Operation
“Backtesting looks great, but in live trading you can’t win.” Or vice versa, “Backtests bankrupt, live trading is fine.” When using EAs, you will eventually encounter this gap. Today, from the developer’s viewpoint, I’ll summarize the mainfive reasonswhy this happens.
Many EAs have mechanisms to stop new entries around important indicators like NFP or CPI. However, MT5 Strategy Tester cannot reference the economic calendar correctly. Thereforethese filters do not work in backtesting, and you may enter during indicators’ releases in live trading even when you would skip them. You may be swept into sudden moves, causing backtests to deteriorate and, for martingale-like strategies, even bankruptcy. In live trading, this is where the filters work. This is the biggest reason for the phenomenon “tests bankrupt, live trading is fine.”
Backtests often run with fixed spreads, while real spreads widen by time of day and by indicators. Especially for scalping, spread is the lifeblood of profit. A single change in test spread settings can make the results appear entirely different. Conversely, testing with unrealistically tiny spreads can yield excellent results that don’t materialize live.
In tests, orders almost always fill as ideal. In live trading, prices slip and orders may be rejected. The biggest differences occur when the market moves rapidly. Relying on “clean executions” from testing can cause some loss in live results; it’s safer to expect some degradation.
Gold (XAUUSD) prices, spreads, and tick sizes vary by broker. Additionally, the quality of historical data (modeling quality) used in tests affects results. With the same EA and settings, numbers can vary depending onthe testing environment. Be aware of this possibility.
This is the opposite gap. A “too-clean backtest” that perfectly fits past data by tuning parameters tends to break down in the future. The more overfitted the backtest, the more skeptical one should be. My insistence on 11 years of testing is to avoid “overfitting that only suits short periods.”
- In the testereconomic indicator filters don’t work(they do in live trading)
- Differences in spreads, execution, and datacause results to vary by environment
- Overly perfect backtestsshould be suspected as curve-fitting
- Approach live trading withdemo, minimal lot size, and a safety marginto gradually progress
Backtests are “maps of the past.” They’re convenient but not the exact path of live trading. Assume there will be deviations, verify filters are effective, test small before going live. This is the most practical way to maintain a long-term relationship with EAs.
If you have questions like “Is this really okay in backtests or live trading?”, join the free community “Asahina Lab.” Let’s tackle common concerns together, step by step.
※This article is provided for information purposes and is not investment solicitation. The performance results shown are historical and do not guarantee future profits. FX and CFD trading involve risk. Please make investment decisions at your own risk.