Why is market forecasting possible with AI machine learning?

Structure of this Article

1. Why AI can predict FX markets
   1. Three bases that support the prediction
   2. Reasons why deep learning is suitable
   3. Four reasons it still may miss
   4. Domains where results are tangible in practice
   5. Predictability by time horizon
   6. Summary of Part 1
2. The potential of AI machine learning in market prediction and a future expanding with ONNX
   1. Current state and limits of FX × AI machine learning
   2. Why now “ONNX”?
   3. How ONNX unlocks the future of systematic trading
   4. Challenges in adoption and approaches to success
3. AI trading traps: overfitting and excessive optimization
   1. Definitions and mechanisms of occurrence
   2. Practical countermeasures
   3. Summary: the past does not perfectly predict the future

Part 1

Why can AI predict FX markets?

What machine learning captures is not “the future price” but slight statistical biases left in prices. That bias certainly exists, but in many cases its magnitude is small and often washed out by trading costs.

1. Three reasons it can be predicted

2. Why deep learning is well-suited

Traditional neural networks struggled with long sequences because past information faded with gradients, preventing reference to candles more than a few steps back. The following two architectures solved this.

LSTM

Learns three gates—forget, input, and output—and decides what to keep or discard. The cell state acts as long-term memory, retaining states over 100 steps or more.

Transformer

Self-attention learns which time points are currently influential. The ability to view multiple time scales (minutes to days) makes it well-suited for multi-scale series like forex.

A common benefit of both is that they can extract relationships from raw sequences without handcrafting features. Conversely, they can also pick up meaningless relationships.

3. Four reasons it may still miss

4. Domains where practical gains exist

While aiming for direction is a tough target, the following three areas show relative success.

Ultra-short-term statistical edge

In timeframes of minutes to hours, technical supply-demand factors dominate. The idea is to amplify small edges through many trades, with ultra-low latency and low costs. The insistence on low latency ties directly to the motivation for native execution via ONNX in Part 2.

Market regime classification

Instead of predicting up or down, classify whether the current market is trending, in a range, or in turmoil. Switching strategies based on classification reduces misalignment between trend-following and mean-reversion.

Anomaly and volatility detection

Detecting impending shifts to reduce positions as a risk management measure. Not a profit-seeking feature, but directly helps limit maximum drawdowns.

5. Predictability by time horizon

Minutes to hours		Significant but cost-dependent
Days to weeks		Grey area
Months to years		Almost impossible
Volatility		Relatively good at predicting
Timing of crashes		Impossible

The bar length indicates relative practicality rather than exact numbers. Predicting direction becomes harder as the time horizon lengthens, while predicting the magnitude of moves becomes more stable.

6. Summary of Part 1

AI is a tool to tilt probabilities, not a weapon that guarantees market predictions. The tilt itself is genuine but small and will degrade with market changes if left alone. Ongoing retraining and a mechanism to detect when the edge is gone are essential to call it a usable system.

When trying it in practice, at minimum these decisions should be made in advance:

• Test in a demo environment for at least one month, including actual spreads and slippage
• Set a stop if Sharpe ratio is below 1.0 or maximum drawdown exceeds 15%
• Monthly, measure win rate and track deviations from theoretical values
• If performance worsens for two consecutive months, redesign rather than tweak
• Do not rely on a single model; combine strategies with different characteristics

Part 2

The potential of AI machine learning in market prediction and a future expanded by ONNX

To forecast the complex price movements of financial markets, AI (artificial intelligence), machine learning, and deep learning have been rapidly adopted in the world of system trading. However, even with advanced predictive models, running them on actual trading platforms with no delay posed a major technical hurdle.

Breaking the wall between “development environment” and “execution environment,” ONNX (Open Neural Network Exchange) is expanding the possibilities for next-generation automated trading systems (EAs and cBots).ONNX is the key.

1. FX × AI machine learning: current status and limits

Machine learning models, led by deep learning, excel at extracting nonlinear patterns from vast historical time series that are hard for humans to recognize.

• Time-series prediction (RNN/LSTM, etc.):From past price movements and volatility, probabilistically predict the next candle direction.
• Reinforcement learning (RL):In a trading environment, reward the agent for “maximizing profit and minimizing drawdown,” iterating to learn optimal trading rules.
• Feature engineering:Use not only price data but also multiple indicators (moving averages, stochastics), order book information, macro indicators, etc., as multi-dimensional inputs.

Traditional challenge: implementation barriers
Even if you build excellent predictive models in Python (TensorFlow or PyTorch), platforms like MT4/MT5 or cTrader do not have built-in Python execution. The common solution has been to connect Python scripts to trading platforms via Web APIs or sockets (ZeroMQ, etc.), but latency and the complexity of building and maintaining the operating environment have been major barriers to practical use.

As seen in Part 1, the edge AI can offer is only a few points. Even a price improvement of a few pips due to latency can erase that edge. Execution environment issues carry the same weight as model accuracy.

2. Why ONNX now?

ONNX is an open format for interoperating models across different ML frameworks. It serves as a common language for AI models.

By leveraging ONNX, developers can train models in Python and simply export the finished model as a.onnxfile. This file acts as a self-contained inference engine, enabling the model structure and weights to be reproduced and run for inference on platforms where Python is not present.

3. The future ONNX unlocks for system trading

The greatest benefit of ONNX isnative execution in trading platforms. Recently, major development environments like MQL5 (MT5) and C# (cTrader) have strengthened support for directly loading and executing ONNX models.

Comparison	Traditional communication-based (API/ZeroMQ)	ONNX-based native execution
Execution speed (latency)	Has latency from communication overhead	Comes within the platform, extremely fast
Environment setup	Requires Python environment and separate server setup	.onnxOnly needs to be embedded into the platform
Distribution/sales ease	Requires complex configuration on the user side
Backtesting	Hard to test precisely within the platform	Easy to validate against past data with strategy testers, etc.

4. Adoption challenges and approaches to success

Technical hurdles are decreasing, but to translate AI into profit, you must address domain-specific challenges.

• Avoiding overfitting:Models that fit past charts perfectly do not work in forward testing on unknown markets. Require strict training/validation splits and robust regularization for noisy FX data.
• Market regime shifts (environment changes):Volatility and trend characteristics can change abruptly. Implement monitoring to adapt to current market conditions and retrain regularly.
• Hybrid operation:Rather than outsourcing all decisions to AI’s black box, combine AI predictions with proven technical rules (moving average crossovers) and strict risk controls (fixed SL/TP) in a hybrid approach to reduce operational risk.

Chapter 3

AI trading traps: Overfitting and excessive optimization (curve fitting) and countermeasures

In FX system trading and AI development, the most frequent and most fatal pitfall is the tendency to fit past data too closely.

In the context of AI/machine learningoverfittingand, in the context of traditional technical indicators used for EA developmentexcessive optimization (curve fitting)both refer to the fundamental problem of lacking adaptability to unknown markets, though their mechanisms and remedies differ.

1. Differences in definitions and mechanisms

Excessive optimization (curve fitting)

• Context:EA (expert advisors) or rule-based system trading development.
• Cause:The developer tweaks parameters (e.g., moving average periods, Stochastic thresholds) to maximize performance for a specific historical period, often manually or with an optimizer.
• Symptoms:Backtests show a clean uptrend, but when moving to live/forward testing, drawdown occurs. The rules fit past noise too perfectly.

Overfitting (AI)

• Context:Development of predictive models using machine learning or deep learning.
• Cause:The model’s capacity (depth, parameters) is so high that it memorizes not only general patterns but also specific noise and outliers in the training data.
• Symptoms:Training accuracy can reach 99%, but validation and test accuracy drop markedly.

2. Practical countermeasures

Defensive strategies differ by problem and development approach.

How to prevent curve fitting in EA development

1. Walk-forward analysis:Split data into multiple periods; optimize parameters in one period and test in the immediately following validation period, repeating to assess parameter robustness.
2. Look for parameter plateaus:Instead of selecting a single best value (e.g., MA 21), choose a plateau around it (e.g., 19–23) where profits remain stable; sharp peaks are likely random.
3. Reduce model flexibility (fewer rules):More rules increase overfitting risk on historical data. Simpler rules tend to generalize better to unknown markets.

How to prevent overfitting in AI/machine learning

1. Strict data splitting (Hold-out / Cross Validation):Split into training, validation, and testing sets. For time-series FX data, avoid data leakage by using time-ordered splits (Time Series Cross Validation) instead of random splits.
2. Regularization (L1/L2):Penalize large weights to prevent reliance on a few features.
3. Early stopping:Stop training when validation performance starts to degrade, even if training accuracy keeps improving.
4. Dropout:Drop a subset of neurons during training to encourage the model to learn more robust, generalizable features.

3. Summary: the past does not perfectly predict the future

Both curve fitting in EAs and AI overfitting share the same fundamental premise: “the financial market is a noisy, non-stationary environment.”If you forget this, you fail.No matter how advanced the AI model (ONNX) is, if it’s in an overfitted state, the system will collapse in a short time. Rather than chasing backward-test beauty, focus on robustness in unknown markets—the true challenge and appeal of AI trading development.

As stated in Part 1, the “slight edge” should be protected in Part 2’s execution environment and verified in Part 3’s validation procedures. Only with these three elements can AI machine learning-based FX prediction become a usable system.

This article is intended as a technical explanation and does not constitute investment advice. The figures cited are general guidelines and can vary greatly by currency pair, broker, and time period.


1. Products currently on sale here
2. Products currently on sale here