Levandor

weather-AS Deploy — Pattern B Containerized Cron Jobs

5 min read

How as_weather_algo (pmv2-weather-as-algo) deploys to polymarket-infra: containerized Pattern B via the apps Ansible role. Answered by @deploy in babylon weather-shadow #338; implemented 2026-06-23. Companion to pmv2-reorg-and-weather-as-2026-06-22 and weather-as-babylon-identity-2026-06-22.

Update 2026-06-23 — run-shape flipped to a continuous service

The run shape changed from two OnCalendar cron jobs to a continuous kind=service running python shadow_engine.py --loop. See [[#update-2026-06-23—continuous-service---loop|Update 2026-06-23 — Continuous Service (--loop)]]. The “Run Shape — Two Cron Jobs” and “Secrets / Env” sections below are superseded.

Decision

Convention = containerized “Pattern B” via the apps Ansible role. NO host venvs, NO host systemd units.

  • Image pushed to Google Artifact Registry: europe-west3-docker.pkg.dev/<project>/levandor-apps/pmv2-weather-as-algo:<sha>.
  • Deploy = add a deploys/apps.yml entry, then dispatch deploy-apps.yml.
  • The apps role auto-injects OTel env (OTEL_EXPORTER_OTLP_ENDPOINT=http://host.docker.internal:4318, OTEL_SERVICE_NAME=<entry>) — OTel wiring is zero-config once the Python OTLP SDK is initialized in main().

Update 2026-06-23 — Continuous Service (--loop)

Supersedes "Two Cron Jobs" and "Secrets / Env" below

The run shape flipped from two OnCalendar cron jobs to a continuous kind=service running python shadow_engine.py --loop.

Why

  • The fleet-wide #354 startup-announcement directive converged (every pmv2 service announces on boot to telegram.outbound), and deploy is minting a weather_as NKey with PUB on telegram.outbound (#366). A continuous service announces cleanly once on boot; an OnCalendar cron would either announce on every 30-min run or not at all.
  • Symmetry with weather-US, which also pivoted to Python (as_weather_algo format) and runs as a --loop kind=service.

What --loop does internally

  • A cohort scan every SCAN_INTERVAL_SECS (default 1800).
  • A daily settle once per UTC day at/after SETTLE_HOUR_UTC (default 12).
  • Both env-tunable.
  • Boot announce publishes the canonical telegram.outbound envelope (#362: msg_id/text/chat/parse_mode/kind) with text component: pmv2-weather-as-algo started with version: <sha> <title>. sha/title come from GIT_SHA/GIT_TITLE Docker build-args (wired in Dockerfile + build-push.yml per deploy #358).

Consequence — the lane is no longer NATS-free

The boot announce needs NATS_URL + the weather_as NKey seed (NATS_NKEY_SEED) with PUB on telegram.outbound. Order publishing stays OFF (no --live); the NKey is only for the announce until the publish leg arms. The announce is best-effort — missing creds → startup_announce:false, never crashes.

2026-06-24 — live --loop crash fixed (%Z strptime, @ 7939da2)

The first live --loop deployment spammed loop_error: time data '... PDT' does not match format '... %Z' — a strftime→strptime round-trip through %Z (can’t parse PDT/JST/CST) aborted the whole scan tick on any locked city. Fixed by reconstructing the local datetime from lock_ts + IANA zone. See [[weather-as-selfmeasure-pnl-fixes-2026-06-23#2026-06-24—live---loop-crash-z-strptime-cant-parse-abbreviated-zone-names-fix—7939da2|2026-06-24 — Live --loop crash: %Z strptime can’t parse abbreviated zone names (fix @ 7939da2)]].

Implementation status

Built TDD, full suite green (22 tests on Python 3.14).

  • shadow_engine.py — loop / _announce_startup / _git_version + --loop wired.
  • publisher.py — DRY _connect_and_publish + publish_telegram_outbound.
  • Dockerfile — ARG/ENV GIT_SHA/GIT_TITLE, CMD --loop.
  • build-push.yml — gitmeta step + build-args.
  • Correction posted to deploy in weather-shadow (reply to #349).

Run Shape — Two Cron Jobs, Not a Service

Superseded 2026-06-23 — see [[#update-2026-06-23--continuous-service---loop|Update 2026-06-23 — Continuous Service ( --loop)]]. Kept for history.

Chosen = kind=job + OnCalendar (two cron jobs). Why: the strategy is windowed (Asian-afternoon lock windows) and entry-only, so cron fits the cadence far better than a long-running process.

scan job

python shadow_engine.py --scan — a NEW entrypoint added this session.

  • Sweeps the whole Asian cohort (12 stations in asia_stations.json, skipping ones with iem_id=null like Hong Kong).
  • Per city: computes the station-local date, runs lock → gate → size → self-measure record. NO NATS publish.
  • Idempotent because selfmeasure.record_signal dedups by (yes_token, entry_ts) and entry_ts is the deterministic causal-lock ts — so running every ~30 min through the afternoon is safe.
  • Suggested UTC scan window ~06:00–15:00 (cohort is UTC+8/+9, lock window local ~15:00–17:00).

settle job

python shadow_engine.py --settle — once daily, ~16:00 UTC, after the prior Asian day’s markets resolve.

Build Notes

  • The --scan entrypoint was built TDD (test_scan.py, 6 tests, pass on Python 3.14), reusing existing lock_from_trace + _measure_gated — no duplicated lock/gate logic.
  • pytest lives in a new requirements-dev.txt, kept OUT of the lean runtime requirements.txt.

Container artifacts added in-repo

  • Dockerfile — python:3.13-slim (the gate pickle needs scikit-learn 1.9.0 / pandas 3.0.3 / CPython 3.13+); non-root; ships gate/gate_model.pkl + asia_stations.json.
  • .dockerignore — excludes research/, caches, .mcp.json, logs.
  • .github/workflows/build-push.yml — WIF → GAR, tags sha + latest.

Secrets / Env

Superseded 2026-06-23 — the lane is NO LONGER NATS-free

As of the --loop flip, the boot announce requires NATS_URL + NATS_NKEY_SEED (weather_as NKey, PUB telegram.outbound). Order publishing stays OFF. See Consequence — the lane is no longer NATS-free.

(Historical, --scan/--settle era.) Dry-run jobs need NO secrets/env. Config + self-measure only hit public Gamma/CLOB/IEM endpoints; the NATS publish leg is dormant and unused by --scan/--settle. env_file stays empty until the publish leg arms.

Update 2026-06-23 — M7/M8/M9 now FIXED

The P&L-correctness bugs (M7 fill-vs-limit, M8 non-binary-terminal leakage + void economics, M9 cost hardcoded 0) plus the L7 cache-poisoning bug are resolved — see weather-as-selfmeasure-pnl-fixes-2026-06-23 (TDD, 44 tests green). The accumulated self-measured P&L is now trustworthy as an edge signal, not just plumbing validation.

(Historical) Accumulated P&L validated plumbing, NOT the edge

Before the 2026-06-23 fixes, the dry-run’s self-measured P&L was affected by open code-review bugs M7/M8/M9 (fill-vs-limit mismatch, non-binary-terminal leakage, cost hardcoded 0), so early deploy validated the plumbing, NOT the edge (see HANDOVER.md PART 2). Now resolved.

Still pending from @deploy / Andras (as of 2026-06-23)

  • WIF provider + SA literals for CI
  • the weather_as NKey (PUB telegram.outbound)
  • @Andras’s terraform ci_github_repos grant

Graph View

Backlinks