An EA created with AI fails in forward tests—the person who fooled themselves the most is the creator
Introduction: The one who is deceived the most is the creator
If you rely on AI, EA can be made in minutes. When you run a backtest, you get an asset curve that rises to the right. However, when you take it to forward testing in actual markets, you don’t win. It’s said that cases like this are increasing.
The reason is not that the AI is bad. The backtesting tool has a tendency for “the creator to show their own work in a favorable light.” AI can repeat that tendency much faster than humans, and as many times as needed.
In this article, I’ll explain four reasons why AI-created EAs fall in forward tests, and introduce five checks you can do yourself before forward testing.
Reason 1: AI makes backtests look good as you ask
When making an EA with AI, a common conversation goes like this.
- “The backtest results are like this. Make it better.”
- “This time period tends to lose, so add exit conditions.”
- “Set the stop loss and take profit to the values that yield the best performance.”
AI faithfully responds to this request. And the result always looks better. Since you’re adding conditions after you’ve seen losing trades, it’s natural that the results improve.
The problem is when those conditions are tailored only to “this past period.” The future market will lose in different ways on different days. The more you tailor to the past, the better the backtest, and the wider the gap with forward testing. This is called overfitting (curve fitting).
It isn’t that AI is bad; the request itself—“make the backtest look good”—becomes an instruction for overfitting.
Reason 2: How many times you tested is invisible
Rolling a die five times and getting all sixes is rare. But if 10,000 people do the same thing, some will necessarily end up that way. Bringing just those people and saying, “These people are dice masters,” is what happens in many backtests.
In conversations with AI, the number of attempts increases in an instant. Change the indicator, change the time frame, change the currency pair. Each small test adds up to hundreds of combinations.
What remains in the end is only the “best one result.” The fact that there were hundreds of attempts is not written anywhere in the backtest report. Therefore, the creator themselves cannot distinguish whether it was a lucky hit or not.
I myself ran 129 tests in one exploration, and five passed the adoption criteria. But upon inspection, all five were the same phenomenon captured with slightly different conditions. This story is“Five passing tests, but one true cause”as written in the article.
The remedy is simple. Keep a record of how many tests you ran, and the more trials you had, set a stricter criterion for “not by chance.”
Reason 3: Supported by a few big wins
Let’s look at a real example. I backtested the sample EA “Moving Average” that comes with MT5, on EURUSD 1-hour chart, from January to September 2026. Anyone can reproduce the same results.
The result: 193 trades, total +$958.87, profit factor (PF) 1.55. The equity curve also rises steadily. At first glance, it’s a pretty good EA.
But what happens if you remove the biggest wins one by one, starting from the largest profits?
| Removed biggest wins | Remaining trades | Total profit (USD) |
|---|---|---|
| 0 | 193 | +958.87 |
| 1 | 192 | +686.40 |
| 2 | 191 | +443.18 |
| 3 | 190 | +279.05 |
| 4 | 189 | +119.85 |
| 5 | 188 | −34.27 |
Removing five out of 193 trades, about 2.6% of the total, makes the overall result negative. The median profit per trade is −1.40 dollars, so a standard single trade is a small loss.
This shape itself isn’t inherently bad. However, if the handful of big wins doesn’t come in forward test over a few months, the performance becomes negative. When forward testing fails, it’s often not that the EA broke, but that the original performance was just that kind of result.
By the way, this EA has a 95% confidence interval of average P/L from −1.50 to +11.80 dollars, crossing zero. Statistically speaking, this means that even an non-advantaged EA can produce this level of performance by chance.
Reason 4: Premises about costs and duration are too lax
When you look at reports for the same sample EA, you’ll often see total fees listed as zero. MT5 tester calculates without fees unless you set them. In real accounts, there are fees and spreads can widen beyond the tester’s assumptions.
For many trades with small per-trade profits, this difference can wipe out profits. I’ve even seen case where the assumed fees in validation differ from the broker’s official figures, increasing real costs by about 64%.
Another factor is the period. If you split this sample EA into first half and second half, you get the following.
| Period | Trades | Total profit (USD) | Average per trade | PF |
|---|---|---|---|---|
| First half | 96 | +846.16 | +8.81 | 1.84 |
| Second half | 97 | +112.71 | +1.16 | 1.15 |
Most of the profits are in the first half, and the second half’s average is only about 13% of the first half. Looking at the total, it seems profitable over nine months, but in reality, it’s possible the second half simply matched a favorable market. Forward testing starts from where the second half began.
Five checks before forward testing
All of these can be verified with the backtest report and a spreadsheet.
- Remove the top 3–5 biggest wins and see if the total remains non-negative.If it goes negative, those few wins may not appear in forward testing.
- Check the median profit per trade and the number of trades.If there are fewer than 30 trades, that’s too few to judge.
- Split into first half and second half; are both positive?If only one half is profitable, it may have just matched that period’s market.
- Even with real account fees and spreads, is there still profit?Also check that the tester isn’t showing zero fees.
- How many different combinations did you try before selecting this setting?Include the conditions you tested with AI. The more, the stricter you should read items 1–4.
If you’re asking an AI, don’t request “make the backtest look good.” Instead, ask, “Is this backtest not a coincidence according to these five checks above?” The same AI will then take on the role of identifying weaknesses.
Summary
- AI faithfully responds to requests like “make backtest look good,” which directly leads to overfitting
- How many ways you tested is not recorded in the report. Keep track yourself
- Even removing a few big wins, splitting into halves, and including real costs, check whether there is still profit
Forward testing is a time-consuming process to determine whether backtests were due to chance. By doubting it yourself beforehand, you can reduce the detour of waiting months to see an EA that you already know will fail.
To continue, see Part 2“Numbers to look at before the Profit Factor”It’s written there. The five checks are summarized inPart 3 “Checklist before listing an EA”and please read as well.
Bonus: How to perform these checks collectively
The numbers in this article come from my own inspection tool“EA Robustness Check”If you provide MT5 backtest reports, it consolidates the five checks, gives a verdict, and highlights points to watch in a written report. The judgment criteria are fixed, and outcomes cannot be adjusted after viewing the results. If you’d rather not manipulate spreadsheets, take a look.
For those who find it hard to doubt themselves, we also offer a testing service.A third-party check of EA backtest robustness14,800 yen (Skijan). We’ll accept up to three MT5 reports, summarize findings in a PDF, and return it within three business days. The judgment criteria are fixed and cannot be changed if the result isn’t favorable.
Note: This article introduces the author’s approach to validation and does not advocate buying or selling any particular symbol, method, or EA. Past validation results do not guarantee future performance.