If you ask AI to "search for the edge," it will surely find it — so I reversed the order of creating the EA.

Last time, I wrote about creating an EA with a win rate of 100% and zero losses by myself.

Over four and a half years of backtesting, I traded 371 times and had zero losses. And when I hit a period I had never touched before, the profit factor collapsed to 0.34.

What I did was run 7,362 parameter combinations from one end and just pick the one with the best performance.

If you tailor to the past, you can manufacture numbers.That was the conclusion last time.

So, where should the edge come from? This time, I’ll talk about that.

Using generative AI worsens this problem

Right now, more people are using generative AI for EA development. I am one of them.

However, if you ask AI, “Find the edge from this data,”AI will surely find something.

Whether what it finds is real or not, patterns will emerge. If the data is long enough, there will be plenty of coincidental matches.

What I did by running 7,362 combinations with a script is the same as what you’re doing. The only difference is speed,and the faster it is, the more dangerousyou could say.

“AI analyzed it” by itself is no guarantee.It might just have quickly found something that can be found.

Therefore, I’ve reversed the order

What I’m trying now is this procedure.

1. Do discretionary trading myself and keep records.Record entry and exit times, direction, rationale, and what I was looking at at the time. Preserve not just wins and losses but the contents of the judgments.

2. Compare the records with the charts and have generative AI analyze them.Have it express in words what conditions are common to winning trades and what differs from losing trades. I’m using Claude.

3. Turn the resulting conditions into indicators.What I’m careful about here isnot to have orders issued. It’s purely a display tool. When conditions are met, it should tell us; that’s all it should do.

4. Trade manually based on that signal.This is the time-consuming part. I’ll actually trade until I have enough data points to judge.

5. If the results are good, describe them and turn them into an EA.Finalize the signal definitions, filters, and exit conditions, and encode them in code.

Why this order?

There is only one reason.

Backtesting can be redone any number of times.Look at the losing areas, add conditions to avoid them, and run again. If you repeat this, you can achieve a continuously upward curve for 10 years. I actually did this in the previous article.

However, I cannot reproduce the records of my past judgments later on.

When I made those trading decisions, I did not know the subsequent price movements. The judgments I made when I didn’t know are recorded as they happened.

I believe there is little data that cannot be rewritten conveniently afterwardto fit your needs.

In the previous article I wrote—“From 2016 to 2021 I did not touch it at all, but I have no way to prove that.”

In my own trading records, this problem does not exist. Timestamps remain.

However, there are limits to this method as well

To be honest,I have not yet reached a conclusion with this method.

Limit 1: It’s unclear whether it works on past markets.Because the data is taken positively, long-term verification doesn’t really exist. I can only say that it works in the most recent market conditions.

Still, I don’t see this as a drawback.“Works in the most recent market” is, in itself, a fact.It is far more reliable than numbers tailored to the past decade.

The issue is what to do when that regime—the nature of the market—changes. Therefore,I design it to enter only in those environments. The goal is not an EA that wins in all market conditions.

Limit 2: When the n is insufficient, nothing can be said.This is the most time-consuming part. With results in the tens of trials, even if winning, you can’t tell with certainty what’s coincidence.

The previous article’s EA traded 371 times and had a 100% win rate, but that still isn’t meaningful.What matters more is how those counts were gathered, not the counts themselves.

Limit 3: Discretionary records themselves have biases.Humans remember winning trades in more detail. The granularity of records changes by situation. Sometimes losing trades are forgotten.

Therefore records should be filled mechanically in the moment, not summarized later, or they cannot be used as data.

Limit 4: Overfitting can creep into the AI analysis step.This is the biggest weakness. As stated at the top, AI will find something if you search for it.

So I say,do not treat AI-generated conditions as discoveries.They are hypotheses. Hypotheses remain hypotheses until verified in forward-testing trades.

The value of this procedure lies not in making AI search for edges, but in the order of forward-testing what it has searched for.If you reverse this, you’ll end up with the same thing as the previous article.

Where I am now

I’ve created an indicator and started trading on its signal. The n is still insufficient.

I don’t know if it will work.If it doesn’t work, I’ll write that as well.

Like the previous EA distribution, if I hide the progress and only show the results, it becomes an unverifiable number.

About continuing this article

From here, I’m adding the actual procedure I’m using.

What I write in the records, what I pass to the AI and what I ask for,and what I avoid asking. Where I decide to stop.

It’s still a method whose conclusion has not been reached, so this isn’t a story of “this will make you win.”I’ve made it so that anyone who wants to try the same thing can run it themselves.

It’s free, but you need to sign up.