AI-type Chart Prediction EA Development Notes 2 | Fell into the swamp of past data optimization. Continuing like this has no meaning.
Hello, this is Tsumo.
This article isNotes on AI-like Chart Prediction EA Development, Part 2.
Last time, I wrote about AI-like chart prediction EA reaching “the point where it first works.”
A long-term backtest was run on EURUSD M15.
Total number of trades: 1,682.
Win rate: 54.82%.
However, net profit was negative.
PF was 0.87.
In other words,
Operational verification passed.
Profitability failed.
That was the current situation. (note)
This time, we continue from there.
I’ll write honestly.
I fell into a swamp.
Look for similar shapes in past charts.
Then observe the subsequent price movements.
If the past similar pattern rose, BUY.
If it fell, SELL.
This idea itself is quite interesting.
However, when I repeatedly backtested, dangerous swamps began to reveal themselves.
That is,an optimization loop to past data.
The current approach can be fitted to past data.
But it is highly likely that the logic will not generalize to the future.
Here, I’ll sort this out head-on.
If you’re interested in MTP introduction, please also take a look here.
MTP: Master【Free】
https://www.gogojungle.co.jp/tools/indicators/79103?via=usersMTP: Slave【Paid】
https://www.gogojungle.co.jp/tools/indicators/79106?via=users
What happened
AI-like chart prediction EA changes results quite a bit when you tweak parameters.
Number of historical searches.
Number of recent bars used for similarity判定.
Minimum similarity.
Minimum number of similarity samples.
Minimum win rate.
Minimum expected value.
Weight of price-change components.
Weight of candlestick shapes.
Weight of technical indicators.
Weight of time window.
TP.
Maximum holding time.
Opposite-sign exit.
Wait-close exit.
Changing these settings moves the backtest results.
A little better.
Worse.
Then a little better again.
Breaks down over another period.
Breaks when changing currency pairs.
Breaks when changing timeframes.
If you revert settings, you get a different result again.
At first, it’s fun.
“With this setting, PF goes up.”
“If you lower the minimum similarity a bit, the number of trades increases.”
“Changing TP improves profit/loss.”
“Changing the maximum holding time reduces the maximum loss.”
These kinds of discoveries exist.
But a sense of unease arises partway through.
Is this really improving the logic?
Or is it just fitting to past data?
This is the point.