Episode 17: Make another incision on the completed ACE
ACE Gold EA Ver.1 -GENESIS-
After running long backtests,
failing many times,
increasing defenses,
missing again,
we finally reached a single completed form.
Normally, development would end here.
We would sell the finished EA,
and just watch the results.
That might have been acceptable.
But as I looked at the eight and a half years of backtests repeatedly,
there was something I couldn’t stop worrying about.
ACE has “strong periods” and “weak periods”
In the end, it still makes a profit.
So ACE itself isn’t the problem.
However, when you look closely at that eight and a half years,
it didn’t win in exactly the same way all the time.
Periods when it ran well.
Periods when it couldn’t advance much.
And,
periods where ACE clearly struggled.
Even if, looking long term, it is profitable,
saying “it’s winning, so OK”
and ending there
would leave the reason forever unknown.
What I wanted to know was,
ACE, why did it lose at that point?
This is what I sought.
I don’t want to break the finished product
Here lies a big issue.
The more you try to improve an EA,
the more easily it breaks.
Find a losing trade,
“If I ban this here”
“If I add this condition”
and fix,
the backtest numbers can be made to look arbitrarily good.
But that does not mean
it would be a strong EA in the future as well.
Knowing the past answers,
it might have simply been tailored to the past.
This is classic overfitting.
Therefore, in Ver.2,
we decided on one rule from the start.
Do not touch the winning ACE.
What we change is
only the parts that can be proven to be wrong.
It’s not about “I want more profit, so I change.”
It’s about finding that “this movement has a clear problem.”
and only then,
to intervene there.
Ver.1 is not broken.
Keeping GENESIS as the baseline,
and comparing from there one by one.
Ver.2 is not an “improved version”
The purpose of Ver.2 development is not to create a deluxe version with added features.
It isn’t just to increase defenses.
It isn’t about adding more entries.
There is only one goal.
To find moments when ACE makes mistakes.
Only those mistakes,
gradually reduce them.
If profits increase, great.
The number of trades may decrease.
There may be no improvement at all.
That’s okay too.
Rather than deciding the answer first,
we will reexamine the eight and a half years of records from the start.
GENESIS is complete.
But,
ACE’s research is not finished.
We place the finished ACE on the operating table,
and now,
not “how to win more”
but,
“where does ACE judge wrongly?”
to investigate.
ACE Gold EA Ver.2
development begins.
Next time
“Do not touch the winning ACE”
Improving backtests is easy.
Just add conditions to the losing spots.
But if you repeat that,
ACE will become an EA tailored to the past.
The first development rule decided in Ver.2
must never be violated.
Next time, we start from there.