Every analysis specialist in M2M passes through three gates before its cards reach users. The default path has no exceptions; the owner-override path is documented at the bottom of this page and the few cases where it has been used are logged on /honesty.
Phase 0 disclosure (2026-05-22): We are operating in Phase 0 with n=2 closed live autotrader trades over the trailing 30 days. The automated calibration-rollback mechanism described below requires n≥30 resolved outcomes per strategy to fire — at current sample size it cannot make a determination. Manual review by the owner is the active control. We surface this so you can evaluate live cards with the correct context. See the bottom of this page for the owner-override cases logged in this Phase.
Before a specialist produces any cards, its strategy is backtested using walk-forward methodology. Parameters are optimized on in-sample historical data, then frozen and tested on unseen out-of-sample data. The backtest must document methodology, sample size, and performance across different market regimes.
Strategies that show strong in-sample performance but significant out-of-sample degradation are shelved — they memorized the training period rather than finding a durable edge.
After passing backtest validation, the specialist runs in shadow mode for a minimum of 30 days. During this period it produces cards that are visible only to the M2M team — users never see them. Outcomes are captured automatically via market data observation.
Shadow mode tests whether the specialist performs as expected on live, real-time data. Backtests use historical data with known outcomes; shadow mode faces genuine uncertainty. This is the gap where most strategies fail.
After 30+ days of shadow mode with satisfactory calibration, the M2M team explicitly approves the specialist for live release. Cards from this specialist become visible to users. This approval is logged with a timestamp and cannot happen automatically.
The calibration audit compares predicted confidence to realized outcomes. Cards are grouped into 5-percentage-point confidence bands (e.g., 70–74%, 75–79%) and the realized win rate for each band is compared to the predicted confidence.
The audit flags a failure when the gap between predicted and realized exceeds 10 percentage points over 30+ resolved outcomes in a band. The target end-state is automatic rollback: cards in the 75%+ band winning less than 55% over 30+ outcomes should automatically pull the responsible specialist to shadow.
Current state (Phase 0, 2026-05-22): rollback is manual.The system surfaces calibration drift through the weekly outcomes cron + the per-strategy disclosure on /honesty. The owner reviews and flipscard_visibility toshadow when a strategy fails. The automated rollback worker that does this without human review ships when sample size per strategy reaches n=30; today the autotrader has n=2 closed live trades, so the worker would have no data to act on.
What this means for you: rollback IS happening — it just happens through owner review until sample-size enables the automated path. Every kill is logged on /honesty with the date and reason. Mean Reversion (killed 2026-04-26) is the most recent example.
Every card — shadow and live — has its outcome captured automatically. M2M observes market data to determine whether entry was filled, whether the exit target or stop was hit, the resulting P&L, maximum favorable excursion (how good it got), maximum adverse excursion (how bad it got), and time to resolution.
Outcome data feeds the calibration audit, the track record page, and future improvements to the confidence formula. No card escapes outcome tracking.
The owner can override a gate when the operational cost of waiting for the full sequence exceeds the risk of early activation. Every override is logged here with date, what was overridden, and what safety controls remain active. Phase 0 currently has the following overrides on record:
Bear-direction options trades were previously pinned to paper-mode pending re-backtest at PF ≥ 1.3 on the 2024-05 → 2026-05 hold-out. Owner explicitly overrode that gate and removed the pin. Historical backtests through 2026-04-14 showed bear branches of swing/squeeze/NR7 at PF 0.65–0.97. Bear strategies are now active subject to the same loss caps, sector cap, wash-sale, 5-factor gate, drawdown circuit breaker, and the 3-day forward-test paper window that gate any new strategy type. The Mean Reversion bear branch remains killed independently of this override.
A new deterministic options-contract ranker was wired into the autotrader entry path with the Gate 1 backtest verdict INSUFFICIENT_DATA (n=17 paired observations vs the n≥50 spec). The owner promoted it as the primary selector because the prior LLM-based selector showed the same R-multiple tracking gap that Gate 1 was designed to detect (median option R 0.038 vs underlying R 0.470). The new selector is auditable and deterministic; the LLM selector remains as a fallback. Live results will be re-evaluated against Gate 1 after the autotrader accumulates n=50 closed live trades.
Source of truth for kill/promotion events is trader_audit_log (logged programmatically) and individual commit messages in the GitHub repo.