Does the Holy Grail exist?

In FX trading, the term "holy grail" is generally equated with a surefire method, and the common stance is "there is no surefire method, in other words there is no holy grail." I used to think that as well.

This chart is quite well made. By looking at it, you can instantly see that EA with long operating periods and high operating rates are excellent. A high operating rate reflects a higher expected value or actual performance.

Most notably on the upper right is tagosaku USDJPY, operating period 12 years 10 months, operating rate 64%. The total period Profit Factor (PF) is 1.25, and the win rate is 38.01%. It isn’t particularly high, but the profit/loss curve is almost steadily rising, which means that simply running the EA and leaving it could be profitable, which could be called a kind of holy grail. However, it seems to be updated frequently, which may be contributing to its stable results. The current version is 26-2, and the backtest results shown span from 2005 to 2017.

 

Backtest PF is roughly between 1.15 and up to 5, but there are some as low as 1.02, 0.97, or even 0.73. In the “USDJPY Great Reversal” series, the backtest PF is below 1, while forward test and official operation are good and have buyers (see below). The error in the PF value from backtesting is likely large, and while it is better than lottery-aid, I was surprised.

When PF is small, the sign of the FTI can flip or take on abnormally large values, so I omitted those with backtest PF less than 1.10 and plotted only one year, as shown below. Plotting extremely large or small values directly would be hard to read, so values above 200% are clipped to 200% and values below -200% are raised to -200% for plotting.

The points are almost scattered everywhere. Although the FTI does not show a clear convergence to 100% as days pass, its lack of change can be interpreted as PF being maintained on average. In other words, newer products may be more in tune with the current market, so it is not necessary to assume they are preferable.

The median FTI is 59% at 0-4 months, 36% at 5-8 months, and 37% at 9-12 months. If the FTI is positive, it means profits. If 36% or more, it seems safe, so more than half of Gogojan’s EAs could be profitable even with long-term operation.

Conversely, it is not correct to think that a negative FTI means it is not a “semi” holy grail. When calculated over short periods, even a semi holy grail can have a negative FTI. For tagosaku USDJPY, the win rate by month is only 63%. The win rate by year is 77%, so at least six months to two years of FTI are necessary to evaluate an EA properly, to avoid mistakenly eliminating a semi holy grail.

Thus, among the EAs sold on Gogojan, there may be many semi holy grails that profit from long-term operation. Those with proven sales performance are only a small portion. Not selling does not mean it won’t profit. There may be a windfall in leftovers.

When searching for semi holy grail EAs, be careful about sample size. You are often drawn to flashy win rates and profit rates right after release, but those have small sample sizes and poor reliability, so selecting them would be foolish. It is generally said that several hundreds of samples are necessary, so across days, at least 180 days, i.e., half a year, is required to properly evaluate.

Having many days alone does not guarantee enough. If trades per month are only a few, the effective sample size is greatly reduced. EAs like the ones in the figure are not suitable for evaluation in this sense.

Similarly, EAs with low win rates and small losses with large profits also cannot be properly evaluated with short forward tests because the number of winning trades is too small, so avoid them.