Codebase-wide correctness pass over polymarket-fetch triggered by the operator spotting data inaccuracies in the Telegram bot. A 6-domain audit (scoring/confidence, gauge/display, rank, pipeline/watcher math, all SQL, bot/notifier/self-wallet) surfaced ~50 issues; the operator-visible and correctness-critical ones were fixed across 6 commits and a follow-up revalidation pass confirmed every fix correct with no regressions or data loss.
For Agents
Read this before touching display/ranking/holdings/SQL in polymarket-fetch. Durable gotchas, most operator-visible:
Rank display must use MONTH window. The Polymarket leaderboard UI defaults to the month tab, not all-time. Display rank = COALESCE(rank_month, rank_week, rank_day), NOT COALESCE(rank_all, …). Fixed at all 5 query sites.
Always filter redeemable=false when aggregating holdings.pmv2_positions / pm_positions keep resolved rows until the next snapshot prune — unfiltered queries surface stale resolved positions as “held” with near-zero stakes that round to $0.
consolidate_gauge groups by (event_merge_key, outcome_index), not outcome_index alone — otherwise distinct candidate markets’ Yes/No merge into a nonsensical “No — for what?“.
Band = skill, “N× avg” = conviction; they are orthogonal. The displayed gauge band is trader_score = edge × consistency. A huge bet by a NEUTRAL-skill trader still shows NEUTRAL — conviction is now surfaced separately as · N× avg (stake / avg_bet_size).
repo::load_event_winners is SHARED with validated scoring — see pmv2-pipeline-bug-audit. The winner-selection change in this pass (require payout>0, deterministic tiebreak) DID touch scoring inputs and was operator-approved + revalidated.
State: HEAD f42c7f4, all pushed.@deploy keeping the bot on current :latest; CRM asked to reconcile its dashboard against the same data points.
Trigger
The operator spotted concrete data inaccuracies in the live Telegram bot output (wrong rank, $0 dust lines, a “No — for what?” gauge label, and a $220k bet showing NEUTRAL). Rather than spot-fix, this kicked off a codebase-wide correctness pass across 6 domains:
scoring / confidence
gauge / display
rank
pipeline / watcher math
all SQL
bot / notifier / self-wallet
~50 issues surfaced. The operator-visible and correctness-critical subset was fixed; the rest triaged (some deferred, see below).
(One commit in the C-series pairs with C1/C2; net = 6 fix commits, HEAD f42c7f4.) A separate revalidation pass confirmed all fixes correct with no regressions or data loss.
Displayed rank used ALL-TIME, PM UI defaults to MONTH
Displayed rank was COALESCE(rank_all, …) (all-time), but the Polymarket leaderboard UI defaults to the MONTH window. The operator’s “#9 vs #12” discrepancy was the all-time vs month rank gap.
Fix → COALESCE(rank_month, rank_week, rank_day) at all 5 query sites.
GOTCHA (durable): the PM leaderboard UI default is the month tab. Match it when displaying rank.
2. $0 dust lines — missing redeemable filter
Resolved positions linger as "held" → round to $0
Bet-holder queries read pmv2_positions with no redeemable=false filter, so resolved positions lingered as “held” with stale/past end-dates and near-zero stakes that fmt_usd rounded to $0.
Fix → redeemable=false AND initial_value>1 on the cohort/holder queries.
GOTCHA (durable):pmv2_positions / pm_positions keep resolved rows until the next snapshot prune. Always filter redeemable when aggregating holdings.
3. “No — for what?” — gauge merged distinct candidates
consolidate_gauge grouped by outcome_index alone
consolidate_gauge grouped by outcome_index alone, merging distinct candidate markets’ Yes/No into a label with no referent.
Fix → group by (event_merge_key, outcome_index) where event_merge_key strips date / year / integer tokens so true date-variants (e.g. Iran by-june-7 / -15 / -30) merge but distinct candidates stay separate. Conservative: never over-merge distinct names.
KNOWN EDGE:event_merge_key strips the token “may” (a month name), so a candidate slug containing the word “may” could mis-key. Low risk; hardening would be to strip months only when adjacent to a day/year token.
4. yungstalin $220k = NEUTRAL — band ignores conviction
Band is SKILL, not conviction
The displayed band is SKILL (trader_score = edge × consistency) and ignored conviction (bet size vs the trader’s own norm). A $220k bet from a NEUTRAL-skill trader correctly shows NEUTRAL by skill, but the operator expected the size to register.
Fix → show · N× avg (stake / avg_bet_size) next to the band in gauge, first-mover, and overlap output.
KEY SEMANTIC: band = skill; “N× avg” = conviction. They are orthogonal — surface both.
load_event_winners random winner on cancelled / 0-payout markets — was corrupting SCORING (feeds compute_trader_metrics). Fix → require payout>0 + deterministic tiebreak. This touches the shared winner-selection path — see pmv2-pipeline-bug-audit; change was operator-approved and revalidated.
Leaderboard-update names sent as unescaped HTML — Telegram parse failure on <>&.
clamp_to_telegram_limit byte-vs-char bug.
pmv2 collect / positions made non-fatal in execute_pipeline — a transient API error was aborting the whole scheduled run.
self_positions redeemable filter — false overlap alerts on resolved markets.
user_name subqueries LIMIT 1 without ORDER BY — nondeterministic name across the 4 leaderboard intervals.
non-pmv2 flatten_resolution_rows missing dedup — the same 371-classON CONFLICT crash documented in pmv2-pipeline-bug-audit, here on the non-pmv2 path.
These were approved because they fix genuine bias, not just bugs. They alter pmv2-two-tier-trader-scoring outputs slightly.
conviction() default 0.5 → 0.0 — no bet history = no weight, not a phantom-median 0.5.
consistency_factor returns 0 when overall edge ≤ 0, and dropped the 0.01 floor — break-even traders were getting inflated consistency; tiny edges were deflated by the floor.
avg_won / avg_lost / avg_bet_size now per-POSITION (buys aggregated per condition/outcome) not per trade-fill — the correct baseline for bet_confidence.
Mop-up (Fix-D, f42c7f4)
Critical: empty-set prune guard is load-bearing
Anonymous-sender rate-limit now denies (was bypassing the limit entirely).
Ex-cohort positions pruned each cycle via delete_positions_not_in_wallets, DOUBLE-guarded against empty-set whole-table wipe. CRITICAL: <> ALL('{}') is TRUE for all rows, so passing an empty wallet set would delete the entire table. The empty-slice early-return is load-bearing — do not remove it.
event_id = None copy-cap now bucketed under condition_id (was uncapped).
mark_copy_resolved made idempotent.
Partial-fetch min-success floor — collect bails if >25% of trade-fetches fail, so a degraded cohort never replaces a good one. fetch_candidate_trades now distinguishes failure from genuine-zero.
Deferred (noted, NOT fixed)
Structural — left for later
~3500-trade API offset cap truncates prolific traders’ history → span_days shrinks → long-term factor under-credits them. Structural; needs a pagination rework.
date-only end_date parsed as UTC midnight → countdown can be ≤24h off. No clean fix without per-market timezone data.
Process gotcha — rtk mangles clippy -D warnings
The CI command cannot be run through rtk
The rtk shell wrapper MANGLEScargo clippy ... -- -D warnings — it passes -D as an input filename. The real CI command must be run via the full cargo path: