Why bother distinguishing the "period viewed" from the "period not viewed"

Introduction

In the previous article, we explained at a high level what the so-called robustness test is designed to verify. From here, we will examine its contents one by one. The first topic is a test called the "IS/OOS split."

What IS/OOS split is

IS (In-Sample) refers to the "observed period," while OOS (Out-of-Sample) refers to the "unobserved period." When our AI Strategy Builder explores combinations of indicators, it divides usable data into two periods in advance. All condition and numeric adjustments are made only within the "observed period," and we do not touch the "unobserved period" at all.

After the conditions are solidified, we first use data from the "unobserved period" to check whether the same performance appears there as well. This is the basic flow of the IS/OOS split.

Why divide on purpose

The reason is simple. If you adjust conditions while looking at data from all periods, you can fine-tune any combination to work well specifically for that period. This is not actual skill; it is just adjusting to the data after the fact.

By setting aside the "unobserved period" from the start, we can fairly verify afterward whether the conditions are truly effective or whether they were merely aligned with the price movements of that period by chance. If good results appear only in the observed period and collapse in the unobserved period, we judge that the condition's effectiveness was likely due to chance rather than a true ability.

What this concept implies

This division is the first barrier we use to "question" things. Even if nice numbers appear, if they only come from the observed period, nothing has been proven yet. The sense is that only when the same tendency is preserved in the unobserved period can we finally stand at the starting line.

What I have felt so far

The idea of separating the "observed period" from the "unobserved period" is by no means difficult. However, many verifications skip this obvious extra step, which is unfortunately common. In the next article, we will discuss the second test, the "Monte Carlo Shuffle," and explain why swapping the order of trades reveals something meaningful.

Disclaimer

・The contents presented in this article do not constitute an endorsement of specific trades nor financial advice. It is solely sharing information.

・Verification results based on past data do not guarantee the same results in the future. Markets are always changing.

・Whether you actually engage in trading should be decided by you and you alone.

・The indicators and tools introduced in the future are among the factors to consider when viewing the market, and using them does not guarantee profits.

If you're interested, please also take a look at the product page for the Indicator Research Institute.