Levandor

solv1-alpha Validation Backtest — Build Complete

8 min read

solv1-alpha — a Rust walk-forward validation harness (workspace member alpha/ in repo ~/coding/slv1, sibling to the now-shelved solv1-rig) — is built and green. Its whole job is to render a structured go/no-go verdict on the Solana meme-coin copy-trading thesis from historical wallet data, before any live latency rig or capital is committed. 16 tasks, subagent-driven on Opus (each task Opus-implemented + Opus-reviewed), plus a final whole-branch review that returned 1 must-fix (now applied).

For Agents

This harness is offline analysis only — it reads historical trades and simulates paper fills; it never signs or sends a transaction. It answers “is there a copyable edge at all?”, which is upstream of the latency question solv1-rig measures. solv1-rig is shelved pending this verdict.

The build is complete AND the first real verdict is now in: NO-GO on naive copy-trading. The original RPC run stalled on an ingestion blocker (not a code bug — see solv1-alpha-data-source-pivot-2026-07-04); after pivoting to the GMGN-ingester + 1s-kline decay, run f02ed44f (3wk train / 1wk test) rendered a verdict: cohort +23.0% vs a random meme basket +109.5% → NO-GO, but the edge concentrates entirely in position/long-holds (+296.1%) while scalps bleed (−20.5%). All analysis stages (score → filters → cohort → decay → mirror → benchmarks → verdict) ran as designed.

Key integrity facts: the final review confirmed walk-forward integrity is sound — no train/test leak, and the split boundary is applied consistently across all 10 query sites. The train/test split is anchored on run.created_at − test_weeks (never on data max). The verdict prints a survivorship upper-bound caveat verbatim — every headline number is a ceiling, because the wallet universe is leaderboard-selected. Storage: 7 solv1_alpha_* tables on the Supabase mgmt project (mkofmdtdldxgmmolxxhc).

What Shipped

One crate (alpha/, tokio async, sqlx/Postgres), a single binary with the full pipeline plus a report readout. 16 feature tasks + 1 whole-branch-review must-fix, subagent-driven on Opus, every task individually reviewed. 48 tests green, exercised against a live throwaway Postgres.

Commit range 67183a6..2f1f69b — 17 commits (16 feat(alpha) + the closing fix(alpha)):

StageCommitModule(s)
scaffold: workspace member, config :6543 guard, CLI skeleton4f87bbcconfig.rs main.rs
solv1_alpha schema v1 (applied to mgmt via MCP)442621fmigrations/0002_solv1_alpha.sql
core types + batched trade sinke83e80etypes.rs db.rs
Chainstack RPC client (pagination + bounded retry)de7faaerpc.rs
swap parser from token-balance deltas (per-venue fixtures)9c5f2abswap.rs
GMGN/Dragon universe fetch (+ seed-list fallback)964ac71universe.rs
resumable RPC indexer → solv1_alpha_trades74e1c10index.rs
training-window scoring (SQL), split boundary enforcede7cdc74score.rs
walk-forward lookahead guard4d21363cohort.rs
wash / self-dealing filtera35a3c3filters.rs
copy-bait sell-into-followers filter028a358filters.rs
age/funding filter + cohort selectiona0401ccfilters.rs cohort.rs
follower-fill decay from subsequent fills737369edecay.rs
paper-mirror test-window trades (decay + fees)863a4c5mirror.rs
SOL-hold + random-basket benchmarksbc99bd5bench.rs
structured go/no-go verdict + full-pipeline run70a3078run.rs
must-fix: skip reversed-order mirror positions; guard sol_size in capacity_sweep2f1f69bmirror.rs bench.rs

Pipeline

Ten stages, offline, end-to-end in run.rs:

Validation pipeline (offline, historical → verdict) 

graph LR
    U["universe<br/>GMGN leaderboard"] --> IDX["index<br/>RPC history + swap parse"]
    IDX --> SPLIT{{"walk-forward split<br/><b>anchor = created_at − test_weeks</b>"}}
    SPLIT -->|train| SCORE["score<br/>net-SOL-flow PnL,<br/>win-rate per token"]
    SCORE --> FILT["adverse-selection filters<br/>wash · copy-bait · age/funding"]
    FILT --> COH["cohort select<br/>precedence-ordered cut_reason"]
    COH --> GUARD["lookahead guard<br/>non-tautological<br/>trade_count recompute"]
    GUARD -->|test window| MIR["paper-mirror<br/>decayed fills + fees<br/>sol_size sweep"]
    SPLIT -->|test| MIR
    MIR --> BENCH["benchmarks<br/>SOL-hold + random basket"]
    BENCH --> V["go/no-go verdict<br/><b>4 conditions + caveat</b>"]
    style SPLIT fill:#3d2020,stroke:#c1666b,color:#fff
    style GUARD fill:#3d2020,stroke:#c1666b,color:#fff
    style V fill:#2d2d2d,stroke:#e9c46a,color:#fff

The two red nodes are the anti-leak spine — see Walk-Forward Integrity. Cohort selection is trained on the train window only; the paper-mirror only ever fills against the test window.

Walk-Forward Integrity

This is the load-bearing property — a backtest that leaks future information into wallet selection produces a fictitious edge. The final whole-branch review specifically audited this and confirmed it sound:

  • Split anchored on run.created_at − test_weeks, never on max(timestamp) of the data. Anchoring on the data’s own maximum would silently move the boundary as more history arrives; anchoring on the run clock keeps the test window a fixed, honest hold-out.
  • The split boundary is applied consistently across all 10 query sites. Every SQL that touches the boundary uses the same comparison — no stage accidentally reads the full range. This uniformity was the specific thing the review verified.
  • Wallet scoring and cohort selection see the TRAIN window only. The cohort (which wallets we would copy) is chosen without any visibility into test-window outcomes.
  • Lookahead guard is non-tautological. It re-computes trade_count from the test window independently rather than reusing the training figure, so the guard cannot trivially pass by construction — it can actually fail and drop a wallet whose activity doesn’t survive the split.

Survivorship is not eliminated, only bounded

The wallet universe comes from a GMGN 7d-PnL leaderboard — i.e. wallets that already won. No walk-forward split can remove that selection; it can only prevent additional leakage. The verdict therefore prints a survivorship upper-bound caveat verbatim: every headline PnL/edge number is a ceiling, and live forward performance will be strictly lower. This caveat is a first-class output, not a footnote.

Adverse-Selection Filters

Copy trading fails not on math but on who you copy. Three filters strip wallets whose realized edge is unrepeatable-by-you, each stamping a cut_reason resolved in precedence order (so a wallet cut for multiple reasons reports the dominant one):

  • Wash / self-dealing — round-trip density: wallets churning the same token in tight in/out cycles are trading against themselves (or inflating volume), not finding alpha.
  • Copy-bait — profit-weighted sell-into-tracked-inflows: wallets whose profits concentrate in selling into the very inflows a copier would create. Their edge is the copiers; mirroring them is buying their exit.
  • Age / funding (sybil-proxy) — wallet age and funding-graph shape as a proxy for sybil / insider clusters that can’t be reproduced by an outside follower.

Cohort selection consumes the filtered set; the go/no-go verdict then checks that filtered performance exceeds raw (condition 4 below) — i.e. the filters must earn their keep, not just shrink the sample.

Paper-Mirror & Benchmarks

  • Paper-mirror simulates a follower copying the selected cohort across the test window only. Entry/exit fills are decayed from the wallet’s actual subsequent on-chain fills (not a fixed haircut) — the follower lands after the leader, so the realized price is modeled from what actually traded next. Fills are net of fees, and a sol_size capacity sweep measures how the edge erodes as position size scales (a real strategy must survive size). The must-fix 2f1f69b hardened this: reversed-order positions are skipped and sol_size is guarded in the sweep.
  • Benchmarks give the verdict something to beat:
    • SOL-hold — buy-and-hold SOL over the same window (the opportunity-cost floor).
    • Reproducible random meme basket — a seeded random basket of meme tokens, so “did we beat picking at random?” is answerable and re-runnable deterministically.

Go/No-Go Verdict

run.rs emits a structured verdict gated on four conditions, all of which must hold:

  1. Beat SOL-hold by ≥ 10 percentage points — clear the opportunity-cost floor with margin.
  2. Beat the random meme basket — the edge is wallet-selection, not just meme-beta.
  3. Edge is not scalp-only — survives the sol_size capacity sweep, i.e. isn’t an artifact that vanishes at tradeable size.
  4. Filtered cohort outperforms raw — the adverse-selection filters add value.

Alongside the verdict the run prints the survivorship upper-bound caveat verbatim and 10 known limitations, so no reader mistakes a GO for a guarantee.

Storage

  • Supabase mgmt project (ref mkofmdtdldxgmmolxxhc), 7 tables prefixed solv1_alpha_ — schema migrations/0002_solv1_alpha.sql, applied live via the Supabase MCP and mirrored in-repo. Distinct from the solv1_-prefixed tables the rig owns on the same project.
  • Same connection discipline as the rig: Supavisor session mode :5432 (config hard-fails on :6543), RLS-on / FORCE-off, owner-writes. Full rationale: supabase-from-gcp-connection-gotchas.

Status & Verdict

  • Build: on git main (local), harness commit range 67183a6..2f1f69b. 48 tests green against a throwaway Postgres.
  • Verdict: IN → NO-GO on naive copy-trading. Run f02ed44f (3wk train / 1wk test, 28-day GMGN harvest, local Docker Postgres): cohort of 8 net +23.0% (real-kline-decayed, after fees) vs a random 50-meme basket +109.5% and below the SOL-hold+10pp bar. Key finding: the edge is entirely in position/long-holds (+296.1%); scalps lose −20.5% to entry decay; filtered−raw +14.8pp (filters add value). Survivorship upper bound → the NO-GO is hard. Refinement to test next: copy positions only, never scalps.
  • Path taken to the verdict: the first RPC run stalled on an ingestion blocker (non-archival Chainstack node + bot-polluted leaderboard — solv1-alpha-data-source-pivot-2026-07-04); the GMGN-ingester (browser-harvest + 1s-kline decay, now BUILT) replaced indexing tasks A4–A7, and the analysis stack A8–A16 (score → verdict) ran reused unchanged.

Graph View

Backlinks