Episode 12: Why did you remove the defense you made to protect?
Last time,
ACE, which had been completed in 2 years and 1 month,
was taken into a market trend of about 8 years and a half.
However,
the ACE supposed to be completed turned into zero gain (zero ca).
By adding defense,
we tried again.
Even so,
a second zero gain.
We investigated the causes further,
reassessed the defense,
and in the third backtest,
ACE finally ran through the 8 years and a half.
It was long.
This time for sure, it would be completed.
It wouldn’t be surprising to think so.
However I began the opposite here.
I removed the defense I had painstakingly built.
Doubting what was built to survive
ACE, which had zeroed twice,
is the defense that carried it to an 8-year-and-a-half completion.
If you think normally,
there is no reason to remove it.
But,
precisely because it could finish,
a different question arose.
Is this defense really all necessary?
Defense,
does not simply stop bad trades.
If you make the conditions stricter,
dangerous entries decrease.
But at the same time,
it may also stop future big winning positions that would meet those conditions.
Future big-winning positions
that would be blocked by those conditions.
If the defense meant to protect ACE,
it might also削って ACE’s original profits.
So this time,
we added functions
and began ablation testing to compare by removing them
.
From adding tests to removing tests
We called such tests
ablation
The idea is simple.
Run ACE with defense ON and ACE with defense OFF,
under the same conditions.
If removing it greatly worsens,
that defense was necessary.
If removing it makes no difference,
there is room to doubt its necessity.
And,
removing makes it better?
The defense we painstakingly built,
we now remove with our own hands.
Right before completion,
a test to decompose ACE again began.
Removing defense improved performance
The result was,
contrary to expectations.
ACE that kept many defenses.
And,
ACE with some additional defense removed.
When compared,
the removed ACE performed better.
Net profit is,
+2,136,214 yen → +2,612,522 yen.
Difference is,
+476,308 yen.
PF is,
1.40 → 1.49.
RF is,
2.16 → 2.64.
Additionally, the number of trades was,
2,359 → 2,220.
Even though defenses were reduced,
there was no increase in reckless trades.
Rather, the number of trades decreased,
and final profit and PF, RF rose.
Removing the things built to protect made ACE stronger.
This was a fairly surprising result.
“Protecting” and “strengthening” are not the same
Here,
we return to the question raised in Episode 7.
EA becomes safer the more it protects?
ACE’s answer was not a simple YES.
A strong defense has the power to prevent losses.
But at the same time,
it also prunes the buds of profits.
Stopping dangerous entries.
But if you layer too many conditions,
entries that should have grown greatly in the future,
will be stopped at the same gate.
Increasing defensive capability and
strengthening the EA itself are not the same thing.
Only keep what is necessary
Therefore,
we did not remove all defenses.
The weaknesses found by two zeros cannot be erased.
Necessary defenses remain.
On the other hand,
remove those that over-protect and suppress ACE’s true power.
Not adding, but choosing.
ACE development
changed a little here.
Instead of adding functions each time you find a problem,
keep only what is truly necessary.
Thus,
ACE ran through another 8 years and a half.
And,
it has come closer to completion than ever before.
This time,
completed.
—I thought so.
But,
it wasn’t over yet.
If you investigate,
weaknesses still can be found.
There are places where improvements seem possible.
If you add new rules,
past backtests could be even better.
Then,
will you put everything you found into ACE?
Or,
will you deliberately not include them?
What remained in the end was not a program problem.
Where to stop development.
That judgment.
Next Episode Preview
Why stop ACE at completion when improvements are still possible?
削り防御を削り、
ACE ran through 8 years and a half.
But,
if you investigate, weaknesses are still found.
If you add new rules,
backtests might be made even better.
Nevertheless I,
did not put everything into ACE.
Next time, Episode 13.
The boundary between “making it better” and “over-fitting to the past.”
And,
ACE GOLD EA Ver.1 — GENESIS —
The current ACE is finally determined.