EA Engineer's EA Course【003】How to Find a Good EA

Backtesting and Forward Testing

I think the metrics that people care about—win rate, number of trades, PF, RF, and so on—vary by person, but one of the consistently important elements when choosing any EA is whether the long-term backtest results are公開されているかどうか？ (whether they are published).

When EA developers come up with a trading idea or method, they use historical data to verify whether there is an edge. In other words, a backtest.

How the EA is developed is something I will talk about later.

Backtesting is the process of confirming, as the name suggests, “What would have happened if this method had been used in the past?” Using future data is called forward testing.

Future data isn’t literally obtained by traveling to the future with a time machine; it is also about checking whether the system would function in actual operation—even for data that didn’t exist at development time—when operated in real trading.

Backtests have all data ready, so even a year-long test finishes in an instant, but forward testing,当然, for determining results, takes a year to complete.

The longer the period for both backtests and forward tests, the more trustworthy they are, but for forward testing you have to wait for years to know the results.

The idea of not being able to use a good idea for years is a waste of time while you wait. The advantage you’ve found might fade as others notice it.

One way to resolve that dilemma is the results of a long-term backtest.

Be careful not to overdo it

Even though it’s historical charts, if you know one logic has worked for three or five years, the probability that it will work for at least the next two to three years is high, unless something extreme happens.

If you operate with a high balance or anything irrational, you can’t complain about earning for two years with an investment of tens of thousands of yen. And you can leave it alone, too.

Of course, it isn’t 100% guaranteed, so you should diversify risk and run multiple EAs in parallel—i.e., a portfolio concept becomes necessary.

What you need to be careful about here is over-optimization (curve fitting, overfitting).

It would take too long to explain in detail here, but in short, if you trade with many parameters and conditions, the likelihood that future conditions align is low, so reproducibility of results is also low.

That’s also why I don’t use indicators in discretionary trading.

Checkpoints

So, to summarize…

• Long-term backtest
Are the results公開されているか？ (published)
• Simplicity
As mentioned in the over-optimization section, the simpler the logic, the higher the reproducibility.
• Ease of understanding
Ideally, the method should be simple, but there are limits. If there were an easy, highly profitable simple logic, you wouldn’t be struggling. So even if it’s somewhat complex, you need to be able to understand and be convinced by the theory used in the method. Are you being deceived by a complex logic using advanced mathematics?

By paying attention to these, you can reduce failures and improve your EA selection accuracy.

NO! Authority worship

Be careful not to jump into EAs that seem amazing just because they claim “a real current Tokyo University graduate thought up the logic, so you should just follow this high-IQ guy!”—a “seems great but you don’t understand it” phenomenon.

Don’t become an authority worshiper just because someone is a doctor or a lawyer, etc.

Be careful of explosive profits

There are EAs that cherry-pick only the parts that function well and trumpet “explosive profits.”

Some even falsify backtest data. At that point it’s fraud.

To check whether backtest results are falsified, see whether the developer and the publisher are different people; whether there are intermediaries or brokers involved.

In short, if the site published by the developer themselves is not checked by a third party, it’s not trustworthy.

Also, even if they are different people, confirm that they are not colluding. If that cannot be determined, don’t engage.

So, add to the checkpoints above…

- Whether the results have been manipulated

And I would like to end this article here.

■ My developed EA concepts and operation policy

EA Craftsman’s EA (Three Arrows) is here