Forward performance of EA and the relationship between randomness and statistics

Recently, in forward testing,

・A short-term EA that performs well
・A short-term EA that performs poorly

have emerged, so
today I would like to write about this area (the relationship between randomness and statistics).

In Forward tests of EAs,
whether in a short period or a long period,
it is natural to expect that better performance is preferable.

However,
when looking at the “results of short-term forward tests,”

one needs to be somewhat careful.

If the forward period of the EA is short,
the influence of “randomness” becomes strong.

And,
as the forward period becomes longer,
the influence of randomness gradually diminishes,
and the influence of “statistics” becomes stronger.

Although the term “statistics” is a bit hard to understand,
this “statistics” means the opposite of randomness, namely that it is statistically more likely to occur

in a certain pattern. (You may also read this as “statistical advantage.”)

To put it more simply,
when the forward period is short,
the image is like rolling a die and getting two consecutive sixes.
→ It is plausible by chance.

And,
when the forward period is long,
the image is like rolling a die and getting ten consecutive sixes.
→ It is hardly possible by chance; only when there is a statistical advantage (statistical edge) can such results occur.

As described above,

what our EAs require is not randomness, but “statistical edge” (statistically favorable outcomes), so

when looking at short-term forward results,
it may be the case that

・they are winning
・they are losing

in either case,
the results are mainly showing a strong influence of randomness,
and you should be aware of this.

Furthermore, as the forward period becomes longer,
the influence of randomness diminishes,
and the EA's inherent [advantage] leads to
a clearer expression of the strength of statistical advantage (statistical edge).

Therefore,
when evaluating forward performance of EAs,
it is natural to wish for good performance in the short term as well,
but
short-term results inevitably contain
a strong influence of randomness,
so please keep that in mind.

Therefore,
for example, when re-adjusting a malfunctioning EA’s logic,
and so on,
if the short-term performance is poor, you might keep changing the logic.

However, that is not ideal; instead, run it for a certain period to

properly determine whether there is a statistical advantage or not, and
only if there is none, consider reviewing the logic;
this is considered the correct approach, in our view.

Thus,
as a purchaser of an EA, you likely want
“first, for the short term, to improve quickly” (then, to win long-term as well).
This is very understandable, but
if you judge the EA’s quality solely on short-term results and decide immediately,
you may end up in a negative long-term outcome; thus,
please keep in mind the above approach in the back of your mind.

Sincerely,