Levandor

Measure Before Mitigating — the Priority-Tiers Miss

4 min read

I shipped “priority tiers” (a staggered fire) to fix a slowdown I attributed to depth cannibalization. Measurement proved the hypothesis wrong: 90.5% of failures were price-moved-past-limit, not no-liquidity. The 400ms stagger was measured actively harmful and neutralized to TIER_DELAY_MS = 0. The durable lesson: a plausible mechanism is not evidence — instrument and measure before mitigating.

For Agents

The measurement verdict that overturned this lives in VERDICT — measured 2026-07-09. The R0-R6 telemetry harness (per-stage latency decomposition, Pub-Ts transit, FAK-no-match as its own error class) was built to answer “where does the time go” — see commits on feat/fanout-latency-r-items. The real fix (R2/R3) is deliberately UNSTARTED, gated on 24h of R1 latency-decomposition data so the target is picked from measurement.

The Wrong Hypothesis

With 4 wallets live, trading appeared “much slower.” I hypothesized depth cannibalization: wallets fan out, receive the same entry signal, and race into the same thin CLOB book — the first FAK eats the liquidity and the rest grind re-priced retries.

It was a coherent, mechanistic story: fan-out → race → FAK exhaustion. I designed and shipped priority tiers on the strength of it: a staggered fire where tier-2 wallets sleep (priority - 1) * TIER_DELAY_MS (400ms) before fetching the book, so they’d trade the leftover depth.

It was built on a wrong hypothesis.

What Measurement Actually Showed

The crypto agent ran the decisive query (read-only duckdb join of pm_fak_attempts × emitted_signals × pmv2_autotrade_orders):

  • 90.5% of failures are price-moved-past-limit (best_ask > limit at fire), not no-liquidity.
  • 88 instances of four-of-four wallets all filling — which directly disproves a depth cap of ~1-2 wallets/signal.
  • The real cost is PRE-BUILD latency: emit→signal gap p50 2.31s.
  • The ~50% fail rate predates wallets 3 and 4 existing — it cannot be caused by their fan-out.

Full verdict, tables, and provenance: VERDICT — measured 2026-07-09.

Outcome — Stagger Neutralized

The 400ms stagger measured actively harmful: it walked tier-2 wallets deeper into the ask-reprice window against a frozen shared max_price. levi/gyula ROI came in at −7.3% / −7.6%, below breakeven.

The mitigation made things worse

The stagger ate the valid-until window and delayed fills into worse prices. A mitigation aimed at a mechanism that wasn’t the real one didn’t just fail to help — it degraded the wallets it was meant to help.

Fix: TIER_DELAY_MS = 0. The tier machinery remains but is inert:

  • tier_delay(1) == Duration::ZERO.
  • Gated if !delay.is_zero(), so tier-1 is byte-identical to the pre-tiers path.

The real fix (R2/R3, the latency/limit path) was deliberately left UNSTARTED pending 24h of the R1 latency-decomposition telemetry, so the target gets picked from data rather than from a story.

Lessons

Durable lessons

  • Measure before mitigating. A plausible mechanism is not evidence. I had a coherent story (fan-out → race → FAK exhaustion) and it was simply wrong.
  • Instrument first. The R0-R6 harness (per-stage latency decomposition, Pub-Ts transit, FAK-no-match as its own classify_error class) exists precisely because we could not answer “where does the time go” without it. Build the measurement before the mitigation.
  • Watch for a mitigation that makes things worse. The stagger ate the valid-until window and delayed fills into worse prices — a negative-value change shipped on a wrong premise.

Related Deploy Hazard — main Moved Under a Const I Didn’t Touch

A build handed to deploy can roll BEFORE a retraction lands

My R0-R6 branch (feat/fanout-latency-r-items) was cut before TIER_DELAY_MS = 0 landed on main, so it still carried 400 and would have silently regressed the fix if rolled — on a const the branch doesn’t even touch.

Rules:

  • Before handing a branch to deploy, check whether main moved under you on a const/config you don’t modify. Rebase or cherry-pick the fix forward.
  • Say “don’t ship this” in the same message as the handoff, never after. A retraction sent later can lose the race to the deploy.

Graph View

Backlinks