[Development Log] What happens if you run a backtest without verifying it — The true cost that isn’t visible
1. Verification is not limited to backtesting
When introducing an EA, many users perform a strategy tester and check profits and win rate, and if favorable numbers are obtained, they commonly proceed to live operation as is.
However, true verification encompasses a broader scope. It is only when you confirm what each parameter means, under what assumptions those values are derived, and whether those assumptions hold in your trading environment that the verification can be considered complete. Many users finish by just checking the numbers, but this is only part of the verification process.
2. Hidden cost (1): Misunderstanding of parameters
A typical example is using the initial value of the StopLoss as-is. Because brokers display currency pairs with different decimal places (4 digits/5 digits), the same numerical value “300” can mean 30 pips on one account and 3 pips on another. If you use the initial value without confirming the digits, you may end up taking ten times more risk than expected.
The same issue applies to inputting lot sizes. Even if you intend to enter “0.1,” depending on the account’s minimum lot size and margin calculation specifications, there are cases where the position size becomes much larger than expected.
3. Hidden cost (2): Gap between the verification environment and live trading environment
Most backtests are run assuming ideal execution conditions. There are settings where spreads are fixed and there is no slippage. In real brokers, spreads typically widen at certain times and there is a certain delay in execution.
If you move to live trading without checking this difference, trades that were profitable in backtests can effectively become losses due to differences in commissions and spreads. Even if the difference per trade is small, hundreds of trades per year can gradually erode the expected value you had in mind.
4. Hidden cost (3): Cost of handling unexpected behavior
If you start operating without understanding the meaning of parameters, when the market moves in an unanticipated way you have no criteria for judging whether the behavior is normal or abnormal.
Even in situations where a drawdown can be considered acceptable if you understand the logic, lack of understanding can lead you to stop the EA at the wrong time or close positions manually. This is a loss caused by the operator’s judgment, independent of the logic’s performance, and is included in the broad concept of “hidden costs.”
Note that such hidden costs occur on a axis separate from the superiority or inferiority of the logic itself. No matter how excellent the logic, if the operator does not understand the meaning of the parameters, the true performance cannot be realized. Conversely, even a relatively simple logic, if operated with full understanding, can more easily reproduce near-backtest results in live trading. The ability to select a logic and the attitude to operate it with understanding are both required.
5. Case where verification is omitted
Imagine a user who introduces an EA, checks only the backtest numbers, and immediately starts live trading without verifying parameter units or their own account specifications at all.
As a result, due to differences in display digits, entries occur with many times the expected lot size, and within a few weeks they accumulate drawdowns exceeding the amount they had deemed acceptable for the year. Although there was no problem with the logic itself, only due to misunderstanding of parameters, losses that could have been avoided were incurred.
6. Items to check at minimum
In practice, the following items are recommended as a minimum check. First, verify the units and meanings of each parameter through the manual or the source code. Second, confirm the display digits (4-digit/5-digit) on your account and the lot calculation specifications. Third, understand how much the backtest assumptions (spread, fill model) diverge from your trading environment. Fourth, before moving to live trading, verify on a small-lot or demo account that the behavior is as expected.
Although each item is mundane, whether you skip this step or not can cause a large difference in actual results even for the same logic.
The decision to omit verification may seem like a time saver at first glance. However, in most cases the later cost is greater. Skipping a few tens of minutes of verification at the start can force you to deal with drawdowns for weeks. This is the most glaring issue with skipping verification.
Also, such insufficiency in verification is not limited to a single EA. When running multiple EAs together, misunderstandings of each EA’s parameters can accumulate and cause the overall portfolio risk to exceed expectations. When operating in combination, in addition to checking individual parameters, it is desirable to pre-calculate the total risk as a whole.
To fully realize the performance of the logic, it is essential to operate after understanding not only the numbers but also the meaning and the underlying assumptions of the parameters.
If you would like to consult, please contact the GoGo Jungle consultation desk below.