Validation using EA's historical data looks at "period" and "quality." A calculation example where the same 100 trades yield PF of either 2.33 or 1.00

I am the director of the FX Holy Grail Research Institute. I am a 50-year-old systems engineer, and I keep a record of testing and overturning automated trading methods that look “likely to win” using historical data.

On the product page for an EA (a program that automatically trades FX), there is a tab named “Backtest.” Backtest refers to verification using past data. Today, before looking at the performance on past data, I want to discuss two numbers to look at: “period” and “quality.”

The numbers appearing in this article are all calculation examples for explanation.They are not my own verification results or performance.

Conclusion

The performance of past-data verification depends onhow long a period was tested and how finely the data was sampledin terms of how trustworthy it is.

There are two numbers to look at.

1. Period:From which year and month to which year and month was the test conducted
2. Quality:How finely the data used for verification the price movements are aligned (expressed as a percentage)

The second item is easy to overlook.If the quality is low, the counting of wins and losses for the same trades can change. In the calculation example, the same 100 trades yielded PF of 2.33 or 1.00.

What is past-data verification

Past-data verification (backtest) is running the EA on past price movements to produce results showing what performance would have been during that period. This is done with the verification feature of trading software.

What matters here is thatthe trading software can only reproduce past data down to the resolution you have locally.

With only an hourly chart, you can’t tell which comes first

Past price movements are stored as fixed-time bars of open, high, low, and close. This unit is called a “bar.”

Trading software (MT5) usually uses bars as fine as every minute, but for clarity, here I will explain with hourly bars. The order within a minute-long bar is the same issue.

Here is a calculation example. Suppose there is an EA that buys USD/JPY and sets take-profit at +5 pips and stop-loss at -5 pips (that is, it auto-closes if the price moves to the line you set). One pip is 0.01 yen.

In one hourly bar, suppose the high is 7 pips above the purchase price, and the low is 7 pips below. The take-profit line and the stop-loss line are both reached.

If the price moved up first, then at take-profitWinwould occur. If the price moved down first, at stop-lossLosswould occur. Merely looking at the high and low does not tell you which came first.

PF changes depending on which you count as first

PF (the sum of profits from winning trades divided by the sum of losses from losing trades; 1.0 means break-even) is used for comparison.

Condition: 100 trades, trade volume 10,000 units (a 1-pip move yields 100 yen), take-profit +5 pips, stop-loss -5 pips. Of the 100 trades, 80 have a clear win or loss (50 wins, 30 losses). The remaining 20 trades reach both lines within a single bar, and it is unclear which came first. Trading costs are not included.

If you count all 20 as “take-profit came first,” you get 70 wins and 30 losses, PF 2.33, and a net profit of 20,000 yen. If you count them as half and half (10 wins and 10 losses), you get 60 wins and 40 losses, PF 1.50, and a net profit of 10,000 yen. If you count all as “stop-loss came first,” you get 50 wins and 50 losses, PF 1.00, and a net zero. If you include trading costs, it would be negative.

The same EA, but the counting method for the 20 trades is what changed.In reality, which one was truly first cannot be known without finer price movement data within the bar.

The smaller the target price range per trade, the greater this effect. Within one hour, USD/JPY can easily move 5 pips up or down. If you target 50 pips per trade, it becomes much rarer for both lines to be reached within one hour.

What is quality

Quality is a percentage indicating how finely the data used for verification are aligned. In MT5/MT4 (the FX trading software used to run the EA), verification reports label it as “History Quality” for MT5 and “Model Quality” for MT4.

If the data are finely aligned, you can reproduce which price moved first within a single bar in a way close to actual price movements. If fine data are missing, the trading software fills in the bar’s internal movements with a fixed procedure. That can cause the outcomes of trades like the 20 trades above to differ from reality.

The higher the quality, the more trustworthy the past-data results become.When quality is low, interpret the results as containing estimation within the bar itself.

GogoJungle listing rule: the lower the quality, the longer the period required

On GogoJungle, there are rules about the past-data verification period required when listing an EA, depending on quality. This is based on a response I received on 2026-09-24 after inquiring for listing preparation. The wording on GogoJungle’s FAQ page differs, so those listing should verify the current rule with GogoJungle.

If quality is 80% or higher, more than 2 years; if quality is below 80%, more than 7 years of verification is required. Lower quality demands longer testing periods.

One note on interpretation.Two years and seven years are minimums for listing.Meeting the minimum is not the same as ensuring the performance will not deteriorate in periods not yet observed. In my own verification, I predefine a criterion tested with 12 years of data.

On GogoJungle’s product page, look here

Product pages have tabs such as “Backtest” and “Forward Test.” I verified the tab names on two best-selling product pages on 2026-09-26. When I opened those two pages again on 2026-10-01, both listed the verification period, but only one listed the quality value. Even best-selling products may not display quality.

After opening the product page, look for the following two items under the “Backtest” tab or similar.

1. Verification period.From which year and month to which year and month. Is it exactly two years, or more than ten years? If not listed, ask the seller on the “Community” tab; if there is no answer, do not use that performance for judgment.
2. Quality value (%).In the trading software’s report, this corresponds to “History Quality” or “Model Quality.” If not listed, ask the seller.

If quality is below 80% or you still can’t get an answer, then look at the “About the Strategy” tab. This includes the take-profit width and stop-loss width.

The items are named “Take Profit” (width of profit-taking) and “Maximum Stop Loss” (width of stop-loss). The smaller these widths, as in the calculation example above, the more the performance within a bar varies depending on order.

September 29 article:Why a 90% win-rate EA is dangerous

In my verifications, when unsure, I assume stop-loss comes first

In my verification program, when both lines can be reached within a single bar and it’s unclear which comes first,I treat stop-loss as coming first.In the above calculation example, this is the strictest way to count PF at 1.00.

The reason is that the “even if you include trading costs, it should not deteriorate in unobserved periods” condition of the Holy Grail is included. A method that looks favorable when counted that way may deteriorate in real trading. Only continue with methods that remain viable when counted unfavorably.

Now I am comparing my own verification program’s results, including this counting method, with the MT5 trading software’s verification results for each trade. This is to identify where they diverge. I do not display the final numbers until they are verified.

How to treat results that do not state period, quality, or trading costs. This is one of five formats where past-data performance may deteriorate in real trading, and I discuss it in a paid article (980 yen).

Paid article “Why past-data performance of EAs deteriorates in live trading”:https://www.gogojungle.co.jp/finance/navi/articles/128308

In conclusion

In today's calculation example, the condition under which performance deteriorates was when you counted trades that could be favorable but whose order within the bar was unclear. Even the same EA can turn a 20,000 yen net profit into break-even simply by changing the counting method.

The calculation examples and diagrams were created with AI, and the numbers were cross-checked by my program.

I cannot promise that the Holy Grail will be found. Still, I believe in its existence and am seeking a trading rule that remains viable even when counted unfavorably, including trading costs and unseen periods. For me, the Holy Grail is a trading rule that does not deteriorate even when accounting for costs and unobserved periods.

The small-wins-with-low-quality-data performance changes from PF 2.33 to 1.00 depending on how you count the 20 trades. When you next open a product page, first check whether period and quality are listed before the performance numbers. If not, that too is material for judgment. If you can identify one instance where the counting method that appears favorable is misleading, it will help narrow down where to search for the Holy Grail.

Glossary

• EA: A program that automatically trades FX
• Past-data verification (backtest): Running the EA on past price movements to produce results
• Bar: A unit representing a fixed time interval’s open, high, low, and close
• Quality: A percentage indicating how finely the data used for verification are aligned. In MT5 it is called “History Quality”; in MT4, “Model Quality”
• MT5・MT4: Names of FX trading software used to run EAs. They also have past-data verification features
• PF: Total profit of winning trades ÷ total loss of losing trades. Below 1.0 means loss
• Stop loss: Automatically close a trade when the price moves to the predetermined line
• Take profit: Width of profit-taking. The tab name in GogoJungle’s “About the Strategy”
• Maximum stop loss: Width of stop loss. Same tab name
• Sen (pips): 0.01 yen. For a 10,000-unit trade, a 1-pip move equals 100 yen
• 10,000 units: The unit of trade size. For USD/JPY, 10,000 dollars’ worth
• Trading costs: Money charged on every trade. The difference between bid and ask (spread) and fees
• Forward test: Results from running the EA after listing on a non-funded practice account. On GogoJungle, this is done without using real money

This article is not investment advice. All numbers are calculation examples for explanation.