Levandor

Phase-0 Edge Validation — Leakage Debunk or Confirm (2026-06-12)

4 min read

Three-pronged leakage debunk-or-confirm pass on Phase-0 test data, run by validate.rs (commit 4b3176d). Verdict: OFFLINE forecast edge confirmed (promising, de-risked) — it survives away from the leakage zone. But T19 is NOT passed/resolved — it remains OPEN: this is an offline backtest with optimistic fills, and live-capturability is UNCONFIRMED. Remaining gates before any GO: full purged walk-forward + deflated Sharpe, a capacity/depth read, and — decisive for a latency edge — a forward paper-trade.

Binary

crates/analysis/src/bin/validate.rs — added in commit 4b3176d on main of wowjeeez/pmv2-crypto-algo. Report written to docs/superpowers/phase0-validation-report.md.

Design

Runs against TEST data only (latest 30% of windows, same train/test split, calibration map, and fee model as edge.rs). Dataset: 198,733 clean rows, 568 train windows, 243 test windows.

Three probes:

  1. τ-stratification — gated strategy + all-trades baseline side-by-side, per tau bucket.
  2. Slippage sensitivity — gated strategy on the ≥60s subset across a range of slip values.
  3. Baseline explanation check — forces the all-trades baseline on the ≥60s subset (raw p_fair, no gate) to isolate leakage from genuine edge.

Results

≥60s Strategy (non-leaky region)

Slipavg_pnl/traden_trades
0.000+1.61%43,961
0.010+0.81%43,961

≥60s Baseline (forced, raw p — leakage-blind)

MetricValue
avg_pnl/trade−3.21%

The decisive offline signal

The ≥60s baseline is strongly negative (−3.21%) while the gated strategy stays strongly positive (+1.61% at zero slip, +0.81% at 1% slip). This contrast establishes a genuine offline forecast edge: P_fair provides a real signal beyond the leakage zone, not just reading a near-determined result. It does not establish that the edge is capturable live at size — the fills here are optimistic (top-of-book, 1s-stale book), which overstates a latency-style edge.

Leakage Zone Contribution

  • 0–60s share of total strategy PnL: 45.6%

This confirms the prior suspicion: nearly half of the all-trades +2.08% baseline (edge analysis) came from the leakage zone. But removing that zone still leaves a material positive edge.

Why the Original Baseline Was Contaminated

The all-trades baseline (+2.08%, logged in crypto-shortterm-phase0-edge-analysis) was a red flag because a no-edge strategy should net ≈ −fee. The contamination mechanism:

  • As τ→0, P_fair = Φ(ln(Ŝ/K)/(σ√τ)) collapses onto the realized outcome (σ√τ→0 blows up the argument to ±∞).
  • The 0–60s bucket contributed 45.6% of total strategy PnL despite being a small share of windows.
  • The ≥60s baseline being −3.21% (the full-fee drag with no edge) is exactly what a genuine no-signal region looks like — confirming that the +2.08% all-trades baseline was indeed inflated by leakage.
  • The gated strategy’s +1.61% in that same ≥60s region proves the gate itself is adding value, not just noise.

VERDICT

T19 Exit Gate: OPEN — offline edge confirmed, live-capturability UNCONFIRMED

T19 is NOT passed/resolved. Status: offline edge confirmed (promising, de-risked); live-capturability unconfirmed.

What IS established: a real OFFLINE forecast edge exists away from the leakage zone. The gated strategy at ≥60s returns +1.61%/trade at zero slip and +0.81%/trade at 1% slip, while the ≥60s forced baseline is −3.21%. The earlier +2.08% all-trades baseline was indeed near-expiry (0–60s) leakage — that specific concern is resolved.

What is NOT established: that this edge is capturable live at size. Reasons this offline result overstates what’s grabbable:

  • Optimistic fills. Top-of-book only, no depth/capacity (observations don’t store sizes), and the book is ~1s stale vs P_fair. For a latency-style edge (“beat the lagging book”), an offline backtest inherently overstates what’s grabbable — you’re filling against quotes that, live, would have moved or been taken.
  • Slippage-fragile. The edge halves from 1.6%→0.8% at 1% fill cost — thin and cost-sensitive.
  • Single train/test split, not the full purged/embargoed walk-forward + deflated Sharpe + t≈3 the spec requires.

Remaining gates before any GO (spec promotion ladder):

  1. Full purged / embargoed walk-forward + deflated Sharpe + t≈3.
  2. A capacity/depth read — requires recording book sizes (not stored today).
  3. The decisive one for a latency edge — forward paper-trade (real-time, real fills, no money).

Caveats

Remaining limitations before capital

  • Optimistic fills (the core caveat for a latency edge). Top-of-book only, no depth/capacity modeling (sizes aren’t recorded), and a ~1s-stale book vs P_fair. An offline backtest of a “beat the lagging book” edge structurally overstates the live-grabbable PnL.
  • Slippage-sensitive — edge halves (1.6%→0.8%) at 1% fill cost; thin/fragile.
  • Single temporal split — purged/embargoed walk-forward with multiple anchors + deflated Sharpe + t≈3 still required.
  • The 45.6% PnL share from the 0–60s region means the aggregate strategy still has leakage exposure — the walk-forward must run on the ≥60s subset or model the leakage zone separately.

Graph View

Backlinks