Levandor

EU/HU Tender Scraping Activity Log

6 min read

For Agents

Reverse-chronological session log. Newest entries at top, grouped by date (## YYYY-MM-DD). Each bullet: one piece of work, short summary, wikilinks to docs touched. Updated by obsidian-documenter on every project doc write. Read by historian at bootstrap (top ~15 entries).

Log was stale until 2026-06-09

Entries between the 2026-05-04 init stub and the 2026-06-09 reconciliation were never recorded as the code shipped. The ~late-April build history below was backfilled on 2026-06-09 from git (feature/tender-pipeline, merged via PR #5), not captured live. Dates track the commit window and are approximate.

2026-06-15 — Phases 2 & 3 built (on master, NOT pushed)

State: local master, not pushed; 2 operator blockers

Both phases are committed + code-reviewed clean (web tests + build green) but not pushed, and not live end-to-end. Full writeup: qualification-phase-2-3.

  • Phase 2 — company profile DONE: editable singleton company_profile (id integer pk default 1 check (id = 1), migration 20260615093853_company_profile applied live to mkofmdtdldxgmmolxxhc), Settings CompanyProfileForm, useCompanyProfile / useUpdateCompanyProfile (upsert {id:1}), en/hu i18n. Gotcha: form clobbered unsaved edits on focus-refetch → fixed with a hydrate-once guard. — qualification-phase-2-3
  • Phase 3 — on-demand qualification DONE (mirrors /disclosure): tables tender_qualification_request (queue) + tender_qualification (one verdict/tender, upsert on tender_id), RLS authenticated SELECT/INSERT + service_role ALL (migration 20260615163827_tender_qualification applied live). tender-qualify edge fn DEPLOYED: verifyAuth bearer-or-secret → newest-pending → claim-race → load tender+profile → empty-criteria short-circuit to unknown (no LLM) → else Claude claude-opus-4-8 thinking:adaptive + structured output. — qualification-phase-2-3
  • Phase 3 landmines: VERDICT_SCHEMA must avoid minimum/maximum/minLength/pattern/recursion or the raw fetch 400s; parse the content[] block where type==='text' (adaptive thinking puts the thinking block first, so content[0] is wrong). — qualification-phase-2-3
  • Phase 3 frontend: lib/tender-qualification.ts (parsers/tone/i18n-key helpers), hooks (useRequestTenderQualification, useTenderQualificationRequest polling 8s while pending/running w/ 15-min bound + verdict invalidation on done, useTenderQualification(s)/List), TenderQualifyButton + QualificationVerdictPanel + eligibility-prose on TenderOverviewTab, en/hu i18n via t(). — qualification-phase-2-3
  • Blocker 1: ANTHROPIC_API_KEY NOT set in Supabase secrets → every real LLM assessment lands status='error' ("ANTHROPIC_API_KEY not set"). Fix: supabase secrets set ANTHROPIC_API_KEY=sk-ant-... --project-ref mkofmdtdldxgmmolxxhc (same secret backs _shared/llm.ts). — qualification-phase-2-3
  • Blocker 2: web SPA deploy parked on wrangler login / CLOUDFLARE_API_TOKEN → Phase 1+2+3 frontend not live. Plus: edge fn not self-verifiable (browser-driven invoke) and empty-criteria short-circuit untestable (all 101 live tenders have eligibility_criteria, 0 null). — qualification-phase-2-3

2026-06-09 — Refocus to Hungary/EKR-only + Qualification feature, Phase 1 shipped

Direction change — pricing intelligence is OUT

This session supersedes the old EU-wide / pricing-model direction. The pipeline is now Hungary / EKR-only, and the strategic goal shifts from “undercut the awarded value” to “does Levandor qualify to bid on this tender?“. The unbuilt pricing-intelligence half (Szerződéstár / CoRe / enrich-pricing / score / FX) is no longer the plan — see TOPICS for the superseded theme.

New direction

  • Refocused discovery from EU/TED → Hungary/EKR-only. TED is removed end-to-end; EKR Eljárástár is the sole source.
  • New feature: an on-demand LLM qualification assessment of each tender’s eligibility criteria against an editable company profile — mirroring the CRM’s /disclosure analyzer pattern (queue + verdict UI). This replaces pricing intelligence as the value thesis.
  • Executed via subagent-driven development with two-stage review per group; pushed to origin/master (~36 commits, both gates green).
  • New design docs live in the repo (not the vault — do not duplicate here): docs/superpowers/specs/2026-06-09-hungarian-tender-qualification-design.md and docs/superpowers/plans/2026-06-09-tender-hu-only-phase1.md.

Phase 1 — DONE, on master

  • Full TED removal across the pipeline package + tender-sync edge fn + web UI. tenders.source CHECK is now 'ekr'-only (kept the ted_publication_number / ted_url fields, which EKR also populates).
  • New config.ts centralizing CPV codes + tag map + defense buyer allowlist/keywords + accent-folding isDefenseBuyer helper.
  • Defense-sector OR-gate inside the EKR source: cpvOk || defenseOk before yield, so defense-sector buyers (Magyar Honvédség, Védelmi Beszerzési Ügynökség) are ingested + tagged defense_sector even with non-IT / null CPV (cpvCodes relaxed to allow empty).
  • Eligibility prose captured: 6 alkalmassagiKovetelmenyReszletek fields + documents flow types → adapter → zod → upsert into new tenders.eligibility_criteria / tenders.documents jsonb columns.
  • Live migration 20260609192522_tender_hu_only applied to project mkofmdtdldxgmmolxxhc: purged 42 TED tenders + their sync rows, rewrote the CHECK to 'ekr'-only, added the 3 jsonb columns, recreated tenders_view.
  • Gates: pnpm --filter @levandor/tender-pipeline test 20/20 green; pnpm --filter web build exit 0.

The tenders table is now EMPTY

The migration purged all 42 TED rows and 0 EKR rows were ever persisted historically, so the table is empty. EKR ingestion still needs a real verification run before the feature is usable.

Pending (Phases 2–4)

  • Phase 2 — editable company_profile singleton + Settings form.
  • Phase 3 — on-demand tender-qualify (Opus 4.8 + structured outputs), queue + verdict UI cloning /disclosure.
  • Phase 4 — null-criteria enrichment from PDF / TED-notice fallback.

Gotchas to carry forward

  • Defense is not in EKR as CPV. There are no CPV-35 / military procurement codes in EKR — only defense-sector civilian buyers (caught via the buyer allowlist/keywords + isDefenseBuyer), not weapons/CPV-35 tenders.
  • Eligibility criteria are free-text Hungarian prose (the alkalmassagiKovetelmenyReszletek fields). They need LLM extraction in Phase 3 — they are not structured/machine-comparable as captured.
  • Seed defense roots unconfirmed: HM EI Zrt and Katonai Nemzetbiztonsági Szolgálat in the allowlist are not yet confirmed against live EKR buyer names — validate during the first ingestion run.

2026-06-09 (earlier — status reconciliation)

  • Status reconciliation: documented that tender discovery + tracking UI shipped and merged (feature/tender-pipeline, 84ae43f, PR #5) while the entire pricing-intelligence half remains unbuilt. Recorded the spec-vs-code divergence (SQLite/Drizzle/FTS5 → Supabase/Postgres; 5 sources → 2; enrich-pricing + score stages → never built). Feature is dormant / operator-driven (not requested via the cross-agent handoff channel). Original plan: source-strategy-spec + implementation-guide; divergence and unbuilt-remainder index: TOPICS.

2026-04-27 (UI completion + API-drift fix)

  • 04b5b3b tender UI defaults to open / closing-soon filter; added a status filter sidebar.
  • 7bb267e API-drift fix: adapted the pipeline to TED Search API v3 and changed EKR API response shapes - the first time upstream APIs drifted and broke ingestion. A future re-run may need another such fix.
  • e1ed0c6 sync button wired into the tenders page.
  • 84ae43f / 4b2518b merged PR #5: completed the CRM tender UI (web/src/pages/tenders/, Tender*.tsx components, /tenders + /tenders/:tenderId routes into App.tsx + Sidebar) and merged feature/tender-pipeline into master.

2026-04-26 (pipeline + schema + runner build)

  • 441f0f1 … f38e05d scaffolded @levandor/tender-pipeline at packages/tender-pipeline/: Zod schemas for Tender and Contract (the Contract schema is exported but nothing ingests it yet - a placeholder for the unbuilt Szerződéstár stage), plus throttle + pagination utils.
  • 4a32a7a / 9d9c978 adapters: adapters/ted.ts (TED Search API v3) and adapters/ekr-procedure.ts (EKR Eljárástár public JSON API).
  • e12fcfb / e6ecd8c stages: stages/normalize.ts and stages/dedupe.ts (Levenshtein fuzzy matching), both with tests. No enrich-pricing or score stage was built, despite the spec calling for them.
  • f1b0d8c compose runner with time-budget support; 4db195b CPV reference table populated from the EKR dictionary; bd1f73d DB helpers (db/ client, sync-state, tender upsert).
  • db8a7b3 tender-sync edge function (web/supabase/functions/tender-sync/index.ts) for incremental daily sync - 120s budget, service-role key, optional x-webhook-secret, 7-day lookback. This is the runner; there is no standalone SQLite process (contrary to spec).
  • e8adb81 cli/backfill.ts entry point with resume support; 2aefaf7 migrations 003_tender_pipeline.sql (tenders, cpv_codes) + 004_tender_tracking.sql (tender_tracking lifecycle enum watching→evaluating→bidding→decided, tenders_view, FTS search_vector).

2026-05-04

  • Initialized activity log. (Stub - the build above actually predates this entry; the log simply was not kept current until the 2026-06-09 reconciliation.)

Graph View