Levandor

Asian-Afternoon Shadow Rehearsal — Fill-Degradation Verdict (CONDITIONAL GO) + the look-ahead-bug lesson

10 min read

UPDATE 2026-06-22 — this shadow engine IS the weather_as lane now (own repo, pmv2 reorg)

This rehearsal became the Asian-weather (weather_as) producer: source weather_as, subject pmv2.order.weather_as.entry (was the legacy weather_shadow / weather.shadow), packaged in the PRIVATE repo pmv2-weather-as-algo at /coding/pmv2/as_weather_algo (committed e9415dc, pushed). The orchestrator shadow_engine.py lives there with the untangled modules. The research record (the report + research/shadow/) stays in the renamed US repo pmv2-weather-us-algo at /coding/pmv2/us_weather_algo — so the research/... paths below resolve under /coding/pmv2/us_weather_algo/, not the old weather_bet. The CONDITIONAL-GO has since been downgraded to LIVE-MEASUREMENT-ONLY by the win-label re-score (caveat #1 discharged): weather-gate-winlabel-2026-06-19. pmv2 is now multi-producer → single-executor (position_manager); full current state + the pmv2-contracts envelope: pmv2-reorg-and-weather-as-2026-06-22.

A separate Python shadow engine (now the weather_as producer; research lives in research/shadow/ under /coding/pmv2/us_weather_algo) for the Asian-afternoon repricing-lag strategy, backtested for fill degradation — does the edge survive our later/worse fills (detection + latency-L entry) vs the traders’ fills we’re copying? Verdict: CONDITIONAL GO on a live dry-run shadow loop (L=5 best); NO-GO on Rust productionization — and since downgraded to LIVE-MEASUREMENT-ONLY by weather-gate-winlabel-2026-06-19. The headline engineering takeaway is a look-ahead bug caught by an economic invariant — the first pass reported a physically-impossible result and was rejected by the verifier. Repo report: research/reports/weather-asia-shadow-rehearsal-2026-06-19.md; engine + data: research/shadow/ (shadow_engine.py, rehearsal_backtest.py, rehearsal_results.json).

CONDITIONAL GO — and even this EV is an optimistic CEILING

The corrected edge survives our fills on the cohort where the fill was provably realizable, but: (1) it is ~4× smaller than the buggy first number (0.44/); (2) the cohort is thin (n=14–19 per latency, per-city n=1–5); (3) the gate selects on convergence, not win — admitted EV is conditioned on “the book will reprice toward 1.0”, which correlates with eventual YES, so the headline EV is an optimistic upper bound, not the realizable strategy edge. The 0.95–1.0 win rates on the per-city splits are the tell. Treat this as a feasibility signal, not a validated edge. The capital gate is the live dry-run loop + re-scoring the gate against a true WIN label — not this backtest.

What it is

A standalone shadow rehearsal for the Asian-afternoon late-settlement / repricing-lag play — distinct from the live US wx-signal engine (see weather-bet-live-status). Three stages:

  1. Obs-driven causal lock (engine.causal_lock, LOCK_HOUR=15, NO_RISE_SECS=5400) on the causal IEM running-max — obs at-or-before the lock only. (Unchanged from the validated engine; verified correct.)
  2. Ex-ante gate screen (predict.predict_proba >= 0.68) — the lean classifier from weather-exante-gate-2026-06-19. Target is tradeable_10 = P(book converges toward ~1.0 within 180 min with lag > 10 min) — i.e. P(convergence), NOT P(win) (load-bearing — see caveats).
  3. CLOB entry at entry_ts = lock_ts + L·60, priced from CLOB prices-history.

The backtest measures fill degradation: our detection + latency-L entry fill vs the traders’ actual fills on the same admitted buckets. Runnable end-to-end via shadow_engine.py --backtest. Interpreter: research/gate/.venv/bin/python (py3.13, sklearn 1.9.0 — unpickles gate_model.pkl cleanly).

THE ENGINEERING LESSON — a look-ahead bug caught by an economic invariant (reusable)

The first pass reported +0.44/$ with NEGATIVE fill degradation — i.e. our later fill BEAT the traders’ fills. That is physically impossible: a fill placed L minutes after a real trade, into the same upward-converging book, cannot execute better than that trade. The impossibility was the signal.

Root cause — a non-causal price snap (look-ahead):

  1. yes_price_at snapped to the nearest price point — including points in the future of the entry timestamp.
  2. The price-history fetch window ran to +2h past entry, so future candidates existed to snap to.

Caught by: the adversarial verifier’s mandatory invariant — our-fill EV must be ≤ trader-fill EV (we cannot beat the people we are copying). The buggy run violated it (negative degradation = invariant breach) and was rejected as look-ahead.

The fix (five parts):

  1. price_history_causal — bounds endTs = entry_ts so no post-entry candidates exist to snap to.
  2. yes_price_at_causal — takes the latest point with t ≤ entry_ts, with a 120 s staleness cap (stale snaps rejected as no_fill_stale).
  3. Realizable-fill clamp: our_fill = max(causal_mid, trader_prior_paid) — removes residual mid-vs-ask optimism (prices-history returns the mid, not the ask we’d cross).
  4. In-harness apples-to-apples trader-fill ceiling — computed on the same admitted buckets from the trader’s actual paid price, replacing a hardcoded 0.14 magic constant. Because our_fill ≥ trader_prior_paid by construction, our_ev ≤ trader_ev is now a derived assertion, not a hope.
  5. Hard GO-gate: main() aborts to INVALID (look-ahead suspected) unless the invariant holds at every L (plus snap ≤ 120 s and degradation ≥ −0.02).

Lesson: backtest fill prices MUST be strictly causal — bound the fetch window to entry, snap only to at-or-before points, cap staleness. And an economic invariant (“we can’t beat the people we’re copying”) is a powerful, cheap look-ahead detector — far more robust than eyeballing the EV. When a copy/shadow strategy appears to outperform its source, suspect look-ahead first.

The numbers (post-fix, anchored cohort, zero cost)

Headline EV is over the anchored cohort — admitted entries that have a trader-prior-paid anchor (real fill evidence on that book). Unanchored entries (no trader trade at-or-before entry, n≈19–20/L) have no realizability proof and are reported separately, not in the headline.

L (min)n (anchored)our-fill net EV/$trader-fill ceiling EV/$fill degradationinvariant our ≤ trader
119+0.07440.1932+0.1188OK
5 (best)16+0.13660.2183+0.0817OK
1514+0.11630.2074+0.0911OK
  • Fill degradation is positive at every L — our fill is genuinely worse than the trader’s, as physics requires. The pre-fix run had negative degradation (the look-ahead tell); that is gone.
  • The invariant our_ev ≤ trader_ev holds at every L (0.074 ≤ 0.193, 0.137 ≤ 0.218, 0.116 ≤ 0.207). It is now partly construction-enforced via the clamp — at L=5, 8 of 16 fills are clamped to the trader-prior price (n_fill_clamped_to_trader_prior), so they lean on the clamp rather than an independent measurement (caveat).

Robustness to cost (fee + slippage sweep)

LEV @0/0EV @1c fee + 1c slipEV @2c fee + 2c slip (worst)
1+0.0744+0.0517+0.0290
5+0.1366+0.1139+0.0912
15+0.1163+0.0940+0.0716

Positive across the entire fee/slip grid at all three latencies (worst cell anywhere = +0.0290 at L=1; worst at L=5 = +0.091). L=5 is the best feasible latency and the most robust.

Per-city after degradation (L=5, net EV/$)

citynEV/$win ratemean fill
chengdu2+0.2901.000.775
paris2+0.1731.000.853
qingdao5+0.1201.000.893
seoul4+0.1091.000.902
atlanta3+0.0901.000.917

At L=5 every covered city is positive. The only negative city anywhere is san-francisco at L=1 (−0.256, n=3). Per-city n is 1–5 — directional, not statistical. The uniform 1.00 win rates are exactly the convergence-selection tell flagged above.

GO / NO-GO

(a) Live dry-run shadow loop → GO. The corrected edge is positive and survives realistic costs at L=1/5/15, L=5 strongest. The packaged engine locks, maps stations, resolves, and screens with verified feature-parity. Run it in dry-run against live signals to measure real fillability — the backtest’s anchored cohort is only a proxy; the live loop is the real test.

(b) Rust productionization → NO-GO (not yet). Hold until all three clear:

  1. The live loop reproduces positive EV on a larger sample (anchored n=14–19 is far too thin to commit engineering).
  2. The gate is re-scored against a true WIN/+EV label (today it selects on convergence — the EV is an optimistic ceiling until this is fixed).
  3. tokyo + wuhan coverage is filled (currently uncovered — see below).

Productionizing a separate Rust engine on 16 trades and a convergence-selected gate is premature.

Method (corrected)

research/shadow/rehearsal_backtest.py, via shadow_engine.py --backtest (REHEARSAL_CACHE_ONLY=1 uses the IEM cache, no network for obs):

  • Lock: engine.causal_lock on the causal IEM running-max (obs ≤ lock only). Unchanged, verified.
  • Entry: entry_ts = lock_ts + L·60.
  • Fill (FIXED): mid = CLOB prices-history YES price strictly at-or-before entry_ts (yes_price_at_causal); rejected no_fill_stale if the nearest at-or-before point is > 120 s away. Fetch window bounded endTs = entry_ts (price_history_causal) — no post-entry candidates.
  • Realizable-fill clamp (NEW): our_fill_price = max(mid, trader_prior_paid), trader_prior_paid = the largest price an observed trader actually paid on the same YES token at ts ≤ entry. A fill placed L min after a real trade on the same converging book cannot execute below it.
  • Trader-fill ceiling (FIXED): computed in-harness, apples-to-apples on the same admitted buckets as (payoff − trader_prior_paid)/trader_prior_paid. Replaces the hardcoded 0.14. our_fill ≥ trader_prior_paid by construction ⇒ our_ev ≤ trader_ev must hold absent look-ahead.
  • Hard gate (NEW): main() emits INVALID (look-ahead suspected) and refuses GO unless our_ev ≤ trader_ev + 0.02 at every L, max snap ≤ 120 s, and degradation ≥ −0.02. GO additionally requires best_ev > 0. (Run output: invariant_our_le_trader_ok: true, lookahead_flags: [], max snap 57–66 s.)
  • Resolution / parity / rounding: unchanged (verified; assert_feature_parity True across all 9 covered cities + tokyo/sf samples).

Coverage

value
Engine-city highest-temp market-days (universe)858
IEM-replayed650 (75.8%)
Cities covered9 / 11 — atlanta, busan, chengdu, paris, qingdao, san-francisco, seoul, shenzhen, taipei
Not coveredtokyo, wuhan (alphabetically last; IEM rate-limited the China stations before reaching them)
Locked + resolved days604
hong-kongcorrectly excluded (Observatory source, no ICAO, iem_id=null, 0.1°C-vs-whole-°C mismatch)

Load-bearing caveats

  1. The gate selects on convergence, not win. Target tradeable_10 = P(converge), not P(win). Admitted EV is conditioned on “book will converge,” which correlates with eventual YES — a selection bias that inflates the absolute EV. The trader-ceiling and invariant bound fill realizability, but the headline EV is an optimistic ceiling, not the realizable edge. The 0.95–1.0 per-city win rates are the tell. Re-score against a true WIN/+EV label before sizing capital. → DONE (2026-06-19), and it confirmed the ceiling: weather-gate-winlabel-2026-06-19 re-scored on P(WIN) and the EV collapsed ~+0.14/**, which loses to a per-city win-rate lookup — downgrading this CONDITIONAL-GO to LIVE-MEASUREMENT-ONLY (no capital, no Rust, until a live loop beats an ungated/per-city baseline).
  2. The invariant is now partly construction-enforced. our_fill = max(mid, trader_prior_paid) means clamped fills (8/16 at L=5) satisfy our_ev ≤ trader_ev by construction, not by independent measurement — they lean on the clamp.
  3. Thin cohort. Anchored n = 14–19 per latency; per-city n = 1–5. CIs wide; feasibility signal, not a sized backtest.
  4. The book reprices past us ~half the time. Price-reject (book already > 0.95 at entry) = 40.6% (L=1) / 45.7% (L=5) / 49.7% (L=15) of locked days. Adverse selection — we miss ~half the opportunities; the days we can enter are the genuinely-uncertain ones.
  5. Single-season data. One season; no cross-season generalization evidence (same limitation as weather-netcost-precheck-2026-06-19).
  6. Coverage gap. tokyo + wuhan absent (9 of 11, not all engine cities).
  7. HK precision. Hong Kong resolves to 0.1°C vs whole-°C bucketing — unresolvable with the IEM/engine stack, correctly excluded.

Where this sits in the fillability arc

This is the fill-degradation test for the Asian-afternoon slice — the practical “can we actually capture it” follow-on to:

  • weather-lag-distribution-2026-06-19 — established the repricing-lag layer, the ex-ante-gate requirement, and that winners pay a ~2-tick premium.
  • weather-exante-gate-2026-06-19 — the P(tradeable) classifier this engine screens with (and whose convergence-vs-win semantics is the load-bearing caveat here).
  • weather-netcost-precheck-2026-06-19 — showed gated EV survives costs but on the traders’ fills (an upper bound), and flagged the Asian-afternoon city concentration. This rehearsal is the direct attempt to quantify the fill haircut that pre-check left unmodeled.

Next

  1. Dry-run the live shadow loop at L=5 against real signals → measure realizable fillability + reject rate live. This is the capital gate, not the backtest.
  2. Re-score the gate against a WIN/+EV label (or tune the threshold on net EV, not convergence precision) before any sizing.
  3. Fill tokyo + wuhan IEM coverage (off-peak, to dodge the mesonet throttle) → re-run the full 11-city cohort.
  4. Only then consider Rust productionization.

Graph View

Backlinks