Even though win rate is 99.5%, abandoned
# Even with a win rate of 99.5% I discarded it
In the previous article,
**"A winning EA" can be made easily. What’s hard is a trustworthy EA**
I wrote about that.
This time, I’ll pick one of the most emblematic events from that discussion.
During the validation process,
**a 99.5% win rate**
setting appeared.
Just looking at the numbers, it’s quite appealing.
However, in the end, I did not adopt that setting.
This time,
**why I discarded such a high-win-rate setting**
will be written as specifically as possible.
---
## At first, it did look promising
There were three methods under consideration.
- AEF
- PVRF
- RMB
Among them, from the start, AEF and PVRF had zero survival candidates.
On the other hand, only RMB met the formal freeze criteria, leaving **4 cases**.
The important thing here is that
**these "4 cases" are not 4 separate strategies**
as a point of fact.
All four were four parameter settings within a single method called RMB.
At this point, it appears like this formally:
- AEF: 0 cases
- PVRF: 0 cases
- RMB: 4 cases
In other words,
**of the 3 methods, only RMB has 4 candidate settings remaining**
as a configuration.
At first glance,
it may seem that
“RMB seems to have captured something.”
Moreover, among them was a setting with a 99.5% win rate.
Normally, that’s a number that moves you.
---
## Yet, there was a sense of unease from the start
What I don’t want you to misunderstand here is that
**it wasn’t discarded solely on intuition**
The actual process consisted of three stages.
1. **Realization**
I felt a discomfort with the numbers
2. **Decomposition**
I examined the breakdown of those numbers numerically
3. **Cross-checking**
I checked whether it matched deterioration patterns I had already confirmed in the past
In other words, rather than ending with a vague sense of suspicion,
**I used the initial discomfort as a doorway to decompose the structure and compare it with known failure patterns**
as the sequence.
This order was quite important.
---
## When I investigated, all four collapsed in the same direction
Looking at the four settings in detail revealed common characteristics.
---
### 1. The exit was tied to the grid edge
All four settings were tied to a configuration where
**TP at minimum, SL at maximum**
was set at the edge.
This is a very risky shape in validation.
Profit-taking is very close, stop-loss is very far.
With this shape,
- Small profits are easy to take
- Large losses occur only occasionally
so the perceived win rate tends to look abnormally high.
In other words,
**high win rate does not necessarily mean superiority**
but rather,
**it tends to create an exit shape that looks favorable**
with a high probability.
---
### 2. Behind the 99.5% win rate was a loss/win ratio of 15.7x
When you only look at the 99.5% win rate, it seems like a strategy that hardly loses.
But the reality was different.
In this setting,
**the size of a single loss was 15.7 times that of a single win**
There you have it.
So structurally, it’s
- win small most of the time
- occasionally big losses
This makes the win rate look attractive while the profit–loss structure is very biased.
What becomes visible here is
**high win rate and quality of the strategy are separate issues**
A win rate of 99.5% does not necessarily prove strong predictive power.
It may simply be that
**it stacks small wins and pushes big losses to the back**
This was exactly the structure you could read here.
---
### 3. BE presence/absence did not change the result
Another decisive factor was how BE was handled.
BE is moving the entry to break-even, i.e., moving the stop to a position where a certain amount of unrealized profit exists.
Logically, BE on or off should affect the result to some extent.
Yet for these four settings,
**BE on or off did not change the result.**
This wasn’t because it felt unnatural.
The reason was clear.
**Because the stop-loss was attached to the maximum side and too far, BE never reached its trigger**
In other words,
- TP at minimum × SL at maximum structure
- Results unchanged even with BE
are not two separate abnormalities.
They were just different manifestations of the same root cause: “exiting too wide.”
Once this was understood, I judged the settings quite risky.
---
### 4. Four settings seemed present, but were actually highly redundant
Further investigation showed that the entries appearing in RMB with N=30 and N=60
had a
**Jaccard overlap rate of 0.976**
This is a very high figure.
In short,
**even though four settings appeared to remain, they were entering at almost the same places**
Not four separate ideas; instead, settings close to each other within one method were producing nearly identical signals.
At this point,
the impression of “4 cases remaining” and “multiple winning paths” collapses considerably.
---
## They matched known deterioration patterns
From the above decomposition, five points become clear:
1. **TP at minimum × SL at maximum**
2. **99.5% win rate**
3. **loss/win ratio of 15.7x**
4. **BE presence/absence yields unchanged results**
5. **entry overlap rate between N=30/60 is 0.976**
And importantly,
these weren’t first-found anomalies for me.
In another validation,
**“Looks strong but is actually deteriorated”**
patterns had already been confirmed.
The four settings in this article matched those known patterns.
So my judgment was not
“I discarded it because it somehow looked suspicious.”
but
**“After checking against previously confirmed failure patterns, I did not adopt it.”**
As a result,
I effectively discarded all four RMB settings,
and determined that the total survival candidates for all three methods were zero.
(Source: `hypothesis_backlog.md:2320-2331, 2366-2367`)
---
## It wasn’t that I discarded it because it had a high win rate
To summarize this story in one line,
**I didn’t discard it because the win rate was 99.5%.**
Rather, the opposite is true.
**I discarded it because investigating why it reached 99.5% undermined its credibility**
If there were genuinely high-quality edge,
- the exit wouldn’t cling to extreme ends
- BE settings would matter
- similar settings wouldn’t yield the exact same results
- you wouldn’t be counting nearly identical signals four times
That would be the expected pattern of a strong edge.
But this case was the opposite in every respect.
That’s why I did not adopt it.
---
## From this experience, I built an automatic detection system
This matter wasn’t just “discard once and done.”
To avoid repeatedly picking up the same superficially surviving patterns,
later I added mechanisms to detect things like
- edges clinging to grid edges
- extreme asymmetry between win rate and profit/loss
- BE invariance
(Source: `hypothesis_backlog.md:2330, 2366-2367`)
In other words, this matter was
not only about discarding one strategy, but
**creating auditing rules for what candidates to mechanically doubt**
as well.
I think this was a very significant takeaway.
---
## PF>1 behaved the same
As mentioned in the first piece, something similar happened with PF as well.
In another extended grid,
Of the 1,728 cells that passed PF>1.0,
**1,714 cells, 99.2%**
were mathematical artifacts caused by extreme asymmetry between TP and SL.
(Source: `hypothesis_backlog.md:6362-6373`)
In other words,
Not just win rate.
PF can also be made to look good under certain conditions.
So now we ask,
> What was the PF value?
Before that,
> Would random entries yield a similar PF if the exit is the same?
We started to focus on that.
---
## Good numbers and credibility are separate things
When developing automated trading,
the moment good numbers appear, your expectations rise.
- Win rate is high
- PF is high
- It looks like a rising trend
All of these are appealing.
But if you adopt them without seeing what structure produced those numbers,
you’ll regret it later.
The 99.5% win rate in this case was exactly that kind of issue.
The numbers are flashy.
However, when you look at the structure,
**it’s not about predictive power but about an exit bias creating the appearance of excellence**
.
So I discarded it.
---
## The most important takeaway from this story
What I want to emphasize most is
**when you see good numbers, you should try to break them down**
.
In automated trading development, the difficulty isn’t to produce promising results.
The hard part is to actively doubt whether those results are truly trustworthy.
The 99.5% win rate is strong.
But the stronger the number, the more you should doubt it.
Without that stance, backtests easily mislead you.
---
## In closing
In the previous article, I wrote about traps in backtesting overall.
This time I gave a concrete example and explained
**the reasons I did not adopt even a 99.5% win rate**
.
In this series going forward, I plan to write based on concrete cases about
- how seemingly good hypotheses broke down
- how paper-backed hypotheses fall apart in actual markets
- how flaws in validation code or evaluation methods can change conclusions
— and more.
Rather than stories of success,
**stories of why they were discarded**
may be more useful in the end.
I hope this will provide some hints for others who are building EA or conducting backtests.
Note: This article does not advocate any specific EA, financial product, or individual trading decision. It records the validations I conducted and the methods used.
Note: This article is a revised re-post of an article published on Note.