Levandor

Phase-1 Live Signal Emitter Architecture

7 min read

Phase-1 live signal emitter built at SHA 5e92c7f; Directive struct aligned to pmv2’s confirmed schema at SHA c12c830. The emitter shares the same observe engine as Phase-0 via a refactored library, applies the winning regime from crypto-shortterm-phase0-selectivity, and publishes structured directives to NATS or shadow-logs them when no NATS credentials are present.

DRY Refactor: observe/src/lib.rs

The observe crate was refactored from a monolithic observe/main.rs into a shared library so both the observe and emit binaries can reuse the same engine without duplication.

Extracted into crates/observe/src/lib.rs:

  • EngineState struct
  • Observation struct
  • floor_window function
  • make_window function
  • to_f64 function
  • All shared constants

Both observe/src/main.rs and emit/src/bin/emit.rs now import from observe::lib.

New Files

PathPurpose
crates/observe/src/lib.rsShared engine library (extracted from main.rs)
crates/observe/src/calibration.rsCalibrationMap — load calibration.json, expose calibrate(p_fair) -> p_cal
crates/observe/src/directive.rsDirective struct + to_json() serialization
crates/observe/src/bin/emit.rsEmitter binary — the Phase-1 live signal emitter
migrations/0003_emitted_signals.sqlcrypto_shortterm.emitted_signals table schema
crates/analysis/src/bin/cal_export.rsExports calibration data from DB to calibration.json

Regime Filter

A window passes and fires a directive only if both conditions hold:

tau_secs ∈ [60, 150)   AND   |p_cal - 0.5| >= 0.30

p_cal = calibrate(p_fair) using the frozen calibration.json loaded at startup. This is identical to the winning cell from the selectivity sweep (t=4.61, 5/5 OOS folds positive — see crypto-shortterm-phase0-selectivity).

Single-Fire-Per-Window Guard

A HashSet<window_id> prevents double-firing on the same window. An additional EMIT_MAX_FIRES cap (default 50) provides a session-level ceiling to bound runaway firing.

NATS Integration

When NATS_URL is set in the environment, the emitter connects via async-nats = "0.37" (added to workspace) and publishes directives as JSON to subject "cohort.signals".

When NATS_URL is absent (shadow mode), the emitter logs the directive text instead of publishing. No NATS credentials are required to run or test the emitter in shadow mode.

Database Recording

Every fired directive is recorded to crypto_shortterm.emitted_signals with window_id as the primary key. This table was created by migrations/0003_emitted_signals.sql.

Directive Schema Aligned to pmv2 — Commit c12c830

The provisional Directive struct was replaced with the confirmed pmv2 schema. See pmv2 Directive Schema Alignment (c12c830) below for the full changeset.

pmv2 Directive Schema Alignment (c12c830)

Commit c12c830 replaced the provisional Directive struct in crates/observe/src/directive.rs with the pmv2-confirmed schema. This clears the earlier “ONE-SPOT SWAP REQUIRED” blocker.

Confirmed Directive Fields

FieldTypeNotes
idStringFormat: "clt-{window_id}" — idempotent on retry of the same window
sourceStringFixed value: "crypto_shortterm_latency_test"
schema_versionu32Always 1
actionStringAlways "BUY" in v1 UP-only path
market_slugStringPolymarket market slug
condition_idStringPolymarket CLOB condition ID (0x… hex string) — extracted from Gamma API response
yes_token_idStringString, not a number — avoids i64 overflow on large Polymarket token IDs
tick_sizef64From Gamma API; fallback constant DEFAULT_TICK_SIZE = 0.001
min_sizef64From Gamma API; fallback constant DEFAULT_MIN_SIZE = 5.0
neg_riskboolFrom Gamma API market object
max_pricef64Tick-rounded, clamped: floor(min(ask + EMIT_PRICE_BUFFER, 0.99 - tick_size) / tick_size) * tick_size
valid_until_tsi64Default = window close (win_open + 300); overridable via EMIT_VALID_SECS env var
bucket_labelOption<String>Optional; τ-bucket label for downstream routing

Removed Fields (Provisional → Confirmed)

The following fields from the original provisional struct were removed: token_id, side, size, emit_ts.

UP-Only v1 Gate

The emitter only fires when p_cal > 0.5 (UP conviction). DOWN conviction (p_cal <= 0.5) is logged as skipped but never published. The constant EMIT_BOTH_SIDES = false gates the unimplemented DOWN path at compile time.

max_price Formula

max_price = floor(min(ask + EMIT_PRICE_BUFFER, 0.99 - tick_size) / tick_size) * tick_size

Tick-rounded and clamped to stay below 0.99. EMIT_PRICE_BUFFER is a configured constant controlling how aggressively the limit order reaches through the ask.

NATS Dedup Header

On publish, the emitter sets Nats-Msg-Id = directive.id. pmv2 uses this header as the idempotency/dedup key — retrying the same window_id will not produce duplicate processing on the consumer side.

valid_until_ts and Latency Signal

valid_until_ts defaults to window close (win_open + 300). Setting EMIT_VALID_SECS to a tighter value causes pmv2 to drop slow round-trips (directive arrived after valid_until). This turns valid_until_ts into a latency signal: pmv2’s drop rate under various EMIT_VALID_SECS values directly measures the end-to-end latency distribution of the emit → pmv2 path.

polymarket.rs Changes

crates/observe/src/polymarket.rs now extracts the following fields directly from the Gamma API market object:

  • condition_id (the 0x… hex string)
  • neg_risk (bool)
  • tick_size (f64)
  • min_size (f64)

Fallback constants are applied when the Gamma API response omits a field.

Migration 0004

migrations/0004_directive_schema.sql adds id and yes_token_id columns to crypto_shortterm.emitted_signals and makes size nullable (size is no longer in the Directive struct).

NATS Subject

Directives are published to subject "cohort.signals". This is unchanged from the initial build.

calibration.json Not Yet Generated

The emitter requires calibration.json to be present at startup. This file is not yet generated. Before deploying emit:

  1. Ensure the DB has enough OOS windows of recorded observations and outcomes.
  2. Run crates/analysis/src/bin/cal_export.rs against the live DB.
  3. Place the resulting calibration.json where the emit binary can find it at startup.

Without this file the emitter will fail to start.

Test Coverage

8 new unit tests added, all passing:

  • tau boundary cases: 59s (below regime), 60s (in), 150s (out), 151s (out)
  • Conviction boundary cases: |p_cal - 0.5| at 0.29 (below threshold) and 0.30 (at threshold)
  • Side and price selection logic
  • Directive JSON shape (field names and values)
  • CalibrationMap round-trip
  • Single-fire-per-window guard (HashSet dedup)

Build, clippy, and fmt all clean across all crates. 0 failures.

Pending Before Deployment

  1. Confirm Polymarket CLOB condition ID schema with @polymarket — swap market.slug in directive.rs.
  2. Run cal_export against DB to generate calibration.json.
  3. Set NATS_URL in environment (or run in shadow mode first to verify directive shape).

Pre-Live Code Review + Hardening (bc38734)

A pre-live safety/correctness code review of the signal emitter — before it can place real money — was run against schema-align pin c12c830. It caught 3 BLOCKERS + 3 HIGH plus several medium/low findings. All fixed in commit bc38734. 21 tests green; gates clean. The full finding-by-finding breakdown with file changes lives in crypto-shortterm-emit-guardrails; the durable GOTCHAS are summarized below.

BLOCKER — condition_id slug fallback

If Gamma omits conditionId, the code had fallen back to the market slug. A slug is NOT a valid 0x condition_id, so pmv2’s dedup + settlement would break. Fix: skip the fire (return None) unless a real 0x condition_id is present.

BLOCKER — wrong-token risk

The YES/UP token was selected by array index clobTokenIds[0]. If Gamma reorders the array, you’d silently BUY the wrong outcome. Fix: select the token by matching the parallel outcomes label ("up"/"yes"), and validate the token id parses as u128.

BLOCKER — guardrails reset on restart

The single-fire-per-window set and the EMIT_MAX_FIRES cap were in-memory only → a crash or deploy mid-run could re-fire already-fired windows or blow past the cap. Fix: hydrate both from the emitted_signals table on startup (crash-durable).

HIGH — stale / zero / NaN ask

A stale, zero, or NaN ask could drive a bad max_price (to_f64 fell back to 0.0). Fix: added a book staleness gate (5s), a binance staleness gate (30s), and finite/>0 ask guards before pricing.

HIGH — NATS publish failure still marked the window fired

A NATS publish failure still marked the window as fired → the signal was silently lost. Fix: only mark fired after publish AND db-insert both succeed; otherwise retry on the next tick (idempotent via Nats-Msg-Id).

Medium / Low findings (also fixed)

  • MEDIUM — valid_until_ts ≥ now+10s guard; 5s discover HTTP timeout; market-race assert (book window_id == directive window_id); calibration p-clamp.
  • LOW — calibration embedded via include_str!; emitted_signals schema reconciled (migrations 0004 + 0005).

Pattern

The build → review → fix loop is the standing pre-live discipline for any money-path code in this project. The review findings above are durable GOTCHAS worth re-reading before writing any future trade-execution code. A final review of deltas is planned right before the live flip.

Remaining gates to a live latency test

  1. Andras merges pmv2 #62 + the source migration.
  2. Deploy polymarket-fetch.
  3. Deploy crypto-shortterm @ bc38734 + add the emit service.
  4. Shadow-validate.
  5. Andras flips live with tiny caps.

Graph View

Backlinks