What does the "AI" in an "AI-powered EA" actually mean? — A discussion on generative AI and machine learning

When you choose an EA, how do you feel when you see explanations like “AI equipped,” “AI determines the best timing,” or “AI perceives the environment”?

To be honest, at first I was excited by the two letters “AI.” It sounds impressive, and it felt like it could read the market more intelligently than humans.

But after looking at many EAs and even making some myself, I realized: many EAs labeled “AI equipped” are quite different from the AI we might imagine.

Today I’ll talk about that. If you like, read it without being on guard!

First of all, there are kinds of “AI”

Even though we call it AI, in the world of EAs there are two main types that matter.

One is, Generative AI. The kind that you can have a conversation with in text, like ChatGPT, Claude, or Gemini. When people say “AI” these days, this is usually what they mean.

The other is, Machine Learning. A system that automatically learns patterns from past data, such as “when this kind of movement occurred, the market tended to move in this way.”

These two share the same name “AI,” but what they can do inside an EA is completely different. Distinguishing them changes how sales pages look to you.

“ChatGPT decides the market”—please wait a moment

First, from the Generative AI side.

“AI that uses ChatGPT and can judge real-time buys/sells” — if you see this kind of description, I take a step back. I’ll honestly write the reasons within my understanding.

・Speed doesn’t match. The smarter an AI answers, the longer it takes to reply. For entry decisions in a market where every second counts, it’s honestly hard to catch in time.
・Costly. If you query the AI for every trade, the number of calls adds up and costs balloon.

　・Not always the same answer. Generative AI can give slightly different answers to the same question. You want your trading rules to be “same situation, same decision,” but that can wobble.

　・No special information source. Generative AI learns from general information in the world. It doesn’t hold “secret information” that makes you superior in the market.

That’s why I’m cautious about the claim that “Generative AI makes real-time buy/sell decisions inside an EA.”

Of course, it’s very useful for tasks like assisting humans with market analysis through dialogue or for broad market environment recognition. But handing over the entire entry decision for automated trading to Generative AI is a different matter, in my view.

Then what about machine learning? — This one can become “real”

On the other hand, I think machine learning can genuinely be useful AI within an EA—at least at present.

I’ll try to explain it in simple terms. When humans handcraft trading rules, there’s inevitably a sense that “this number feels good.” There are many conditions that determine market situations (for example price momentum, time of day, proximity to recent highs and lows, etc.), and there’s a limit to how well humans can combine them to win—relying on gut feel alone has limits.

Machine learning finds how to combine these conditions directly from past data. It learns patterns like “when these conditions align, buying tended to go up” or “this pattern was better to skip.” It learns these tendencies automatically from a large number of past examples.

One common method is called a “decision tree”. In short, it connects many small Yes/No forks to finally decide Buy/Skip. A single tree is coarse, but if you prepare many trees with slightly different properties and take a majority vote, the results become much more stable.

…Actually, my EA “Mikazuki USDJPY” uses this approach. It combines 290 decision trees to quantify the edge of each trade, and then has AI analyze that to determine the entry. (I’ll go into detail about this in another episode, as it’s a lengthy topic.)

…However, there is a big pitfall in machine learning: overfitting.

Overfitting — a student who only memorized past questions

Imagine a student who, before an exam, memorizes only the past questions. If the same questions appear, they score perfectly. But when faced with slightly different, unseen problems, they can’t do anything.

Overfitted ML is the same. It will perfectly match past charts used for learning, so backtest results look stunning. Yet, in unseen future markets, it can break easily.

My biggest caution is not about backtests that look too clean, but about black-box models that claim to “continuously optimize automatically.” If it’s that clean, I doubt it isn’t overfitting on past data.

So how can we prevent this “memorization”? Let me share a little from the creator’s perspective.
The basic approach is to make AI solve “unseen problems.” Split past data into a before and after, train on the before portion, and test whether performance holds in the unseen after portion. Technically called out-of-sample testing, it’s a test of whether the model works beyond its training range. If you want to be more thorough, you can perform walk-forward analysis by gradually shifting the time window and repeatedly training and testing.
The criterion is simple. If it’s excellent during the training period but collapses in the new period, that’s a red sign for overfitting. If it performs reasonably in both periods, it’s closer to genuine.

My own EA is published after verifying performance in a period not used for training.
If it collapses there, I won’t publish it no matter how perfect the backtest looks.

Where EA and AI will go from here

Now, a look to the future. This is my personal view, but—AI will continue to become smarter, faster, and cheaper. I wrote earlier that generative AI isn’t suitable for real-time trading, but that’s mostly due to current technical limits. In a few years, those barriers may be much lower, and the day when “ChatGPT decides an EA” isn’t a joke may come sooner than you think.

Still, I don’t think the day will come when AI perfectly predicts the market like a crystal ball. The market isn’t a world where past patterns continue exactly into the future. And it’s tricky that if everyone uses the same AI and same approach, that approach will eventually stop working. No matter how AI evolves, the traps of overfitting and the truth that “the future isn’t just a continuation of the past” will probably remain.

So the EA world will likely change like this: using machine learning will become more common. Eventually, “AI equipped” won’t be a special selling point; instead, the question will be—How sincerely are you using that AI and how do you manage risk?. Not the billboard, but the substance and attitude. It’s a little understated, but I expect it that way.

And one more thing. When AI becomes smarter, it benefits not only the “makers” and the sellers but also us who buy. We buyers can also team with AI as much.Read the numbers on sales pages with your own AI, and question them rigorously—such a discerning tool will become easier and smarter to use. The same weapon will be given to both sellers and buyers. I think that’s a fairly fair and healthy future.

In conclusion

My own EA, “Mikazuki USDJPY” is also truly a machine-learning model. So I’m not against AI as mere ornament. On the contrary, I’m a user who relies on it.

But because I use it, I’m also wary of its limits (tendency to overfit). So Ihave my own EA diagnosed by another AI to find its flaws. The diagnostic prompt I distributed previously was created for exactly that purpose.

AI is not a magical tool. But when used with understanding of its limits, it can be a truly useful, precise instrument. Not blindly trusting, not outright denying—this measured approach feels right to me.

This connects to the earlier “future talk,” but the point to watch is probably unchanged going forward.“Whether it is AI or not” is less important than “the substance and the risks.”. Is there an upper limit to losses? Are backtest conditions realistic? Is there a track record of real-world operation? Judge based on these practical aspects, not the four-character billboard. Not stopping thinking just because it’s AI is your best defense, I think.

If you’re curious which kind of AI your potential EA uses, try the diagnostic prompt from last time to check. You might be surprised to find that the billboard and the content differ.

ーーーーーーーーーーーーーーーーーーーー
For more about Mikazuki USDJPY and its real-world performance
https://www.gogojungle.co.jp/systemtrade/fx/79530