If you choose at your convenience later, the grades will appear to be good. It’s a story we nearly fell into ourselves.

The signs "Domestic standard" and "Overseas legend" alone inspire a certain level of trust. However, the weight of a signboard and the results when actually checked with data can be two different things.

Previous: The Ichimoku chart and the Turtles—when challenged with standard and legendary methods, the results were surprising

About this series (Verification Journey, 6 installments)

• Episode 0: Why we started the research lab (published)
• Episodes 1-2: We challenged the conventional path (published)
• Episodes 3-4: We combined indicators and even relied on AI (published)
• Episodes 5-6: What we saw when challenging domestic and international standards (published)
• Episodes 7-8 (this article): Changing the perspective revealed a crucial lesson ← right here
• Episodes 9-10: What we ultimately arrived at

Introduction

Up to here, we've repeated verification from various angles—using a single indicator, multiple combinations, AI, and domestic/foreign standard methods—but none of these alone could be said to have a clear edge.

Thus we decided to question the very premise of “which method to use.” Perhaps every method has a state of the market where it works well and a state where it doesn’t, and simply applying it uniformly without distinguishing those states was itself the reason things didn’t go well. That is what we thought.

Episode 7: Regime-adaptive (use according to market state)

What we tried was the idea of first distinguishing whether the current market is in a big directional move (trending market) or moving within a certain range (range-bound market). Technically, this is called “determining the regime (market state).”

【Image: 09_regime_switch.png planned to be inserted here (owner will paste manually)】

After identifying the regime, we use a trend-following method if it’s a trending market, or a range-bound method aiming at rebounds within the range if it’s a range market. Rather than entrusting everything to one method, the idea is to switch tools according to the situation. This was a clear shift from our previous thinking.

Episode 8: Important lessons learned in the verification process

During this verification process, we realized something very important that goes beyond the methods themselves. It is that “the way you test has dangerous pitfalls.”

Specifically, when validating multiple assets, if you later pick the “best-performing settings” for each asset individually, the apparent performance can look better than the actual capability.

【Image: 09_overfitting_trap.png planned to be inserted here (owner will paste manually)】

To put it another way, it’s like rewriting the test answers after the fact. It makes it look as if the result was always the initial target for that asset. We too realized during repeated verifications that we were unknowingly approaching this method, so we fixed all the conditions fairly and re-ran the verification.

【Image: 09_real_vs_fake.png planned to be inserted here (owner will paste manually)】

What we learned from trying it

The idea of regime-adaptive methods is by no means incorrect in principle. However, when we redo the verification under fair conditions, we have not yet reached a point where we can assert a stable edge based on this alone.

On the other hand, the lesson learned from the verification process—“performance is inflated when chosen after the fact”—was a much larger gain than the discovery of the method itself. Without this lesson, at some point we might have believed in a superficially good performance and released something dishonestly.

What we’ve felt so far

The phrase “today, one viewpoint has changed” is the motto of this research lab, and these two episodes embody that exactly. In terms of discovering a method, we haven’t reached a conclusion yet. However, we have gained a deeper understanding of the integrity of verification itself—how to determine when you can trust the results.

This lesson will apply to all future verifications.

Next time, we will actually answer the question many readers have: “If the win rate is low, shouldn’t we do the opposite?”, and we will finally share one key clue that our lab has ultimately arrived at.

Disclaimer

• The contents shared in this article do not advocate specific trades or provide investment advice. It is for information sharing only.
• Verification results based on past data do not guarantee the same results in the future. Markets always change.
• Whether you actually trade or not is a decision entirely at your own risk.
• The indicators and tools introduced in the future are one of the materials for judging market movements, and using them does not guarantee profit.

Next time we will bring you “Episodes 9-10: What we ultimately arrived at.” If you’re curious, please also visit the product page of the Indicator Institute.