Accuracy audit of the live producer on the live crypto_shortterm DB (emitted_signals × outcomes), clean both-sides era 2026-06-28 → 2026-07-02 (BTC n=920 fires; 4 alt dry lanes n=1,756). Headline: live emit silently dropped the validated edge-vs-ask trade condition — a fidelity gap vs the Phase-0-gate-clearing rule worth +60–90% EV/fire and roughly half the fires, fees, and FAK exposure. Also: EMIT_MAX_ASK unset on all 10 lanes, a folded calibration audit (healthy overall, four localized deviations), and cross-asset calibration transfer confirmed OOS.
For Agents — actionable outcomes
- Fidelity fix (code): restore the net-edge-vs-ask gate in
emit— fire only ifp_cal − (ask + slip) − fee > 0(optionally +δ margin). This re-instates the rule the walk-forward validated; it is NOT a new regime and does NOT require a gate re-run.- Config fix (deploy, immediate): set
EMIT_MAX_ASK(backtested cap 0.85) on all 10 lanes — currently unset everywhere; the live UP lane fired at asks up to 0.979.- Calibration: frozen map stays frozen. τ-conditional / per-side / σ-conditional refits are refit-era items — each requires refit + walk-forward re-gate per project rules (crypto-shortterm).
- Data hygiene: always filter
side IN ('Up','Down')onemitted_signals— legacy'BUY'era rows otherwise silently mis-score (see Gotcha — emitted_signals.side literal changed between eras).
The critical fidelity gap — emit never compares p_cal to the ask
The rule that cleared the Phase-0 gate (crates/analysis/src/bin/walkforward.rs:32, trade_decision_slipped) only trades when:
p_cal − (ask + 0.01 slip) − fee(0.07·p·(1−p)) > 0
Live emit.rs fires on is_regime(τ, p_cal) alone — it never compares p_cal to the ask. Every backtest that passed the multiple-testing-corrected bar (crypto-shortterm-phase0-walkforward, crypto-shortterm-phase0-selectivity, crypto-shortterm-phase0-depthaware) had the edge-vs-ask condition in the loop; the live producer does not. This is the definition of a fidelity gap: the deployed strategy is not the validated strategy.
Consequence
~26% of live BTC fires had zero or negative model-vs-market disagreement (pred ≤ ask) — gross EV ≈ +0.002/share, i.e. negative net of fees. These fires burn spend cap, fee drag, and FAK exposure for nothing.
Evidence (BTC clean era, n=920)
Fires bucketed by pred − ask, where pred = folded conviction prob (p_cal for Up, 1 − p_cal for Down):
| bucket | share of fires | avg ask | realized EV/share (gross) |
|---|---|---|---|
| ≤ 0 | 26% | 0.918 | +0.002 (≈ negative net of fees) |
| bucket 1 (small +Δ) | — | — | +0.03 |
| … monotone … | — | — | rising |
| Δ > 0.20 | n=148 | — | +0.28 (win rate 0.818) |
EV is monotone in disagreement — exactly the shape the validated rule exploits, and consistent with crypto-shortterm-loss-filter-analysis-2026-06-26 (the edge IS model-vs-market disagreement; agreement gates like pm_ask≥0.80 were rejected there because they delete it — this gate is the opposite: it requires disagreement).
Simulated gate on the same live fires:
| gate | fires | EV/fire (gross) | gross total |
|---|---|---|---|
| none (live behavior) | 920 | 0.0905 | 83.3 |
| pred − ask ≥ 0.03 | 521 | 0.1446 (+60%) | 75.4 (≈ equal net of fees) |
| pred − ask ≥ 0.05 | 440 | 0.1715 (+90%) | — |
Out-of-sample confirmation (4 alt dry lanes)
The hypothesis was formed on BTC only, then tested untouched on the eth/sol/xrp/doge dry lanes:
- All fires: n=1,756, EV 0.0704/share, gross total 123.7.
- Gate ≥ 0.03: 942 fires, 0.138/share, gross total 130.0 — HIGHER in absolute terms: the dropped fires averaged −0.0078/share gross, i.e. negative even before fees.
Verdict
Restoring the edge-vs-ask condition is strictly better: ≈ half the fires, +60–90% EV/fire, equal-or-higher total, and half the FAK exposure / fee drag / spend-cap burn. Fidelity fix, not a new regime — no walk-forward re-gate required (it re-instates the exact validated rule).
EMIT_MAX_ASK unset on ALL 10 lanes
- The live UP lane fired at asks up to 0.979; 43% of its last-5-day fires were above the backtested 0.85 cap.
- The −EV pocket is
ask > 0.90per thea27b852backtest. - Immediate config fix via @deploy (env, no code). Note this is a ceiling (don’t overpay), not the rejected agreement-floor from the loss-filter analysis.
Calibration audit (frozen 20-bin map, folded, BTC clean era, n=920)
Overall healthy — bucket predictions track realized within noise, with four localized deviations:
| # | finding | detail |
|---|---|---|
| a | τ 60–90s fires under-confident | pred .866 → realized .936, z≈2.9. Only 12% of fires land there — 74% of fires trigger at the first crossing into τ<150s. |
| b | DOWN side skews | 0.855-bucket under-confident (realized .898, z≈+2.5); ≥0.965 bucket possibly over-confident (realized .886, n=44). |
| c | lowest-σ quartile over-confident | −3.8pt, z≈1.6 — consistent with EWMA σ lag. σ = EWMA λ=0.94 over Chainlink prints decimated to 10s (oracle-quantized); a Binance-tick σ would be richer. |
| d | extreme disagreement fades | pred−ask > 0.15 realizes −2 to −5pt below pred → haircut sizing there, don’t skip — EV is still highest in that bucket. |
Refit-era items (do NOT do casually)
τ-conditional map, per-side maps, σ-conditional correction — all require a refit + walk-forward re-gate per the frozen-calibration rule. The frozen map stays frozen until then.
Cross-asset transfer — BTC calibration generalizes
BTC’s frozen calibration applied unchanged to the eth/sol/xrp/doge dry lanes (n≈430–445 each): pred ~0.88 vs realized 0.865–0.903 — within ~2 SE everywhere. The oracle-lag edge generalizes across assets; per-asset refits are fine-tuning, not blockers. See crypto-shortterm-multi-asset-status-babylon-2026-07-02 for the rollout state.
Method notes
- Query path: duckdb → Supabase
postgres_queryserver-side, read-only (Ad-hoc live-DB access via duckdb (no psql)). - Era hygiene:
side IN ('Up','Down')filter mandatory — the legacy'BUY'era (Jun 25–28 v1 rows) produced a fake 15% win-rate scare this session before being excluded. - “Folded” = Up and Down fires mapped to a single conviction probability (
p_calfor Up,1−p_calfor Down) so both sides share bins.
Related
- crypto-shortterm — project index (frozen regime + calibration rules this audit enforces).
- crypto-shortterm-pnl-attribution-corrections-2026-07-03 — label footing for any P&L work off this session: join
outcomeson(asset, window_id)(notwindow_idalone), and userealized_pnl_usd(CLOB/winner-flag), NOTresolved_up, as the P&L label. Also REFUTES the prior “#1031 PM realized_pnl bug / 7 false losses” claim (zero-delta — we lost all 7; the artifact was a flipped token-side cross-ref, not a PM bug). - crypto-shortterm-competitor-wowitsamazing-refuted-2026-07-03 — both corrects and corroborates this session: the “sole skill-verified winner / 100% BTC-5m” wallet (
0xd02b…) has zero selection edge and is a net loser (true USDC −$84k) — the oracle-state tier’s motivating record does not exist; and its lifetime taker-fee bleed empirically VALIDATES this note’s net-edge-vs-ask gate (the exact0.07·eff·(1−eff)term) on ~190k real fills. - crypto-shortterm-phase0-walkforward, crypto-shortterm-phase0-selectivity, crypto-shortterm-phase0-depthaware — the validation the live emit must stay faithful to.
- crypto-shortterm-loss-filter-analysis-2026-06-26 — disagreement-edge framing; why this gate is not the rejected agreement gate.
- crypto-shortterm-emit-guardrails — the emit safety-guardrail family this fix joins.
- crypto-shortterm-fak-nomatch-root-cause-2026-07-02 — FAK exposure this gate halves.
- crypto-shortterm-multi-asset-status-babylon-2026-07-02 — the 10-lane rollout the config fix applies to.
- crypto-shortterm-data-sources — duckdb access pattern + side-literal era gotcha.