Financial AI / AI Agent Research and Development

Even if Test 7/7 Passes, We Won't Say It's “Finished”—Three Types of Evidence to Distinguish in Production Deployment

On September 17, we deployed a guard to stop late re-entries into GOLD13. Behavioral tests passed at 7/7, and we confirmed blocking from the fifth bar onward in replays. Still, we do not treat it as complete. Code tests, playback of past data, and real-market live trading each have different proven scopes.

This post is a development log of the indicators and automated trading software (EA) I developed, researched and validated with AI agents on actual accounts. We record successes, failures, and unverified items separately, and we also publish the operational status on a 24/7 weekday trading drive.

Evidence 1: Behavior Test of the Logic Itself

First, I confirmed that the decision conditions behave as coded. The 4th bar is allowed, the 5th is blocked, when there are 6 bars or more the episode resets, exclude warn1/2・LEGACY_HISTORY・stale, separate BUY and SELL, do not double-count on the same bar, and do not proceed to model judgment or orders after a block—these seven items all passed.

What this shows is that the conditional branching works as expected. It cannot prove spreads, communication delays, or actual market biases.

Evidence 2: Replay of Past Episodes

Using records from September 17, 2026, I confirmed that an episode corresponding to ticket 29721052 (1st) is permitted, and an episode corresponding to ticket 29721491 (8th) is blocked. This demonstrates that the guard operates on data formats that were actually saved.

On the other hand, replay does not fully reproduce real-time production judgments, as inputs order, missing data, connection delays, restarts, and other conditions that occur only in production remain.

Evidence 3: Production Natural Trading

Ultimately, what is needed is to stop the entrance on the 5th bar onward when same-direction warn0 occurs naturally, ensure guard_rejected / skip_warn0_late_cluster is stored, and ensure signal and order do not occur. Furthermore, I will verify corresponding chart recordings, telemetry, heartbeat, and order history at the same timestamps.

The evidence for this natural trading is not yet collected at deployment time. Therefore, even if implemented, I will not state that “production effect proven.”

Reason for Limiting the Scope of Changes

What I touched this time is the entry-entry guard. No changes were made to EA5, NORMAL, SOUL1 operation, existing position management, SL/settlement, lots, SL/TP, Quality Gate, AutoTrading, OBS, or YouTube streaming. There was one unsettled position at deployment, but I did not change the policy or settlement actions for that position.

Changing multiple points simultaneously makes it impossible to identify causes of improvement or deterioration. In financial AI research, we prioritize limiting the scope of a single change and keeping a reversible state and a comparison baseline.

Post-Deployment Monitoring Items

• Whether periodic runs end without exceptions
• Whether the blocking logs do not duplicate after a restart
• Whether we mistakenly stop valid candidates among 1–4 bars
• How blocking candidates from the 5th bar onward move thereafter
• Whether recorded video, telemetry, and order history timestamps match

The 21:15 periodic run ended normally with no_breakout / exit=0. Going forward, we will leave unproven items until the relevant episode occurs, and update the evaluation with the evidence we have.

There is no corresponding natural trading video yet, so we will not repurpose videos from other trades. Once sufficient evidence is gathered, I will add an article with the videos included.

GOLD13 Sales Page
Weekday 24-Hour Trading Drive

This article is a development and verification log and does not guarantee profits or future performance.

Why You Shouldn’t Mix Types of Tests

Unit tests verify conditional branching correctness, replay tests verify compatibility with saved data, and production natural trading verifies the entire operation including communication, timing, and restarts. If you apply the success of any one to others, unverified gaps may become hidden.

For example, even if a trade did not occur in a replay, production may have another process that picks up an old signal. Conversely, even if production stops once, it does not guarantee correct handling of duplicate bars or opposite-direction orders. I limit the questions I can answer to each piece of evidence.

Decide on Rollback First

In production deployment, waiting to see what happens after a problem occurs is too late, so I define rollback conditions to cover increasing exceptions, incorrect blocking of valid candidates, unintended signal or order generation, and heartbeat stops. I preserve the original source, settings, and log locations so I can revert to the previous state during investigation.

Financial AI runs 24 hours, so I assume incidents when humans are not watching. I define stop and recovery conditions simultaneously, and test not only successes but failure paths as well.

Numbers to Watch in Monitoring

• Total candidates and the ratio of approvals vs blocks
• Distribution of episodes by number of bars
• Counts by reason for blocks
• Difference between signal generation count and order transmission count
• End codes and processing time of periodic runs
• Counts of duplicate processes before and after restarts

Without these numbers, it ends up as “trading somehow decreased.” I will track the influence of changes from entry to order with numbers and verify that only the intended places changed.

Why Publicize Unproven Items

In research and development articles, describing only completed features can mislead decisions. I will clearly state that the fifth-bar block in natural trading is unconfirmed, there is no corresponding video yet, and measurements across multiple episodes remain.

Explicitly marking unproven items shows what to look at next to update the evaluation. When new evidence appears, we can compare with past articles and prevent showing only successes.

Next Production E2E

When the relevant episode occurs, I will continuously verify from candidate generation to blocking, persistence, non-generation of signal, and non-sending of orders. I will use video only from the candidate time and publish it with telemetry and heartbeat aligned. I will not conclude the superiority of the rules until multiple observations align.

Pass / On-Hold / Fail Judgment Table

In future deployments, if the conditional branching passes, I will label it “Single-Segment Pass,” and if it can be reproduced with saved data, “Replay Pass,” and if natural trading aligns video and order logs, “Production E2E Pass.” I will not call the intermediate stages “complete.”

If there is any exception, missing log, mismatch between signal and order, or duplication after restart, it is on hold or failed. After fixing the problem, I will re-run not only the failed case but all seven items to ensure no other part is broken.

In the article, I will not expose changes or internal information without limit; I will present input conditions readers can judge, expected results, actual measurements, and any unconfirmed items. The publication standard is to disclose not only success rates but also reproducible procedures and stop conditions.

Publication Standard:In production E2E, I will link candidate times, guard results, signal counts, order counts, position changes, recording files, and heartbeat to the same ticket or episode. If even one item cannot be addressed, the judgment is not possible. Until production natural trading can be reproduced multiple times, single successful cases will not be used as proof of the entire logic.

If I am unsure about a case, I will not mark it as pass by guesswork and will defer it to the next observation. I prioritize leaving verifiable failure records over adding speculative success stories.