KB consolidated generation phase 3 — design

Generate consolidated how-to articles from a cross-source cluster: feed its member sections to claude -p, get one merged, fully-cited article that flags where sources disagree. On-demand, draft → review → publish.

Roadmap position

Phase 1 (DONE): vectorize (3,252 chunks in Qdrant).
Phase 2 (DONE): section clustering + /semantics explorer (194 clusters, 24 cross-source in clusters.json).
Phase 3 (THIS): consolidated claude -p generation from a cluster. clusters.json is the retrieval bundle.

Goal

Turn “these 4 manuals all cover differential replacement” into ONE consolidated how-to that merges the sources, cites every fact, and highlights disagreements — generated on demand from the /semantics explorer, reviewed as a draft, then published into the knowledgebase.

Non-goals

No batch/auto generation (on-demand, one cluster at a time).
No auto-publish (draft → human review gate → publish).
No editing of the source manuals (read-only over them).
No image generation (only carry through source images).
No fine-tuning / RAG re-embedding (uses the existing cluster members).

Key decisions

Decision	Choice
Trust model	Cite-everything, never invent — every spec/step/number tagged with its source; conflicts shown both-ways
Trigger	On-demand, one cluster at a time (from `/semantics` or CLI)
Workflow	Draft (preview URL, “unreviewed” banner) → Approve → publish to KB; or Discard
Generator	`claude -p --model opus` (non-Sonnet per house rule), headless on the box
Execution	Background job (like OCR ingest) — status-polled; serialized; power-flaky box
Accuracy guard	Prompt contract + automated number-token spec-check + human review
Published location	`docs/consolidated/<slug>/` — browsable, vectorized on next index
Runs as	`levander` (claude logged in as levander; KB app is levander)

Architecture

Adds a generator to /home/levander/kb-vectors/ (it already holds clusters.json + section access) and draft/publish routes to the knowledgebase app.

~/kb-vectors/
  genprompt.py    pure: build the claude prompt from a cluster + its member section markdown
  speccheck.py    pure: extract number tokens; flag draft numbers absent from source text
  generate.py     job: load cluster -> gather member .md + images -> claude -p --model opus
                  -> write draft (markdown + images + meta) -> run speccheck -> status
  kbgen.py        CLI: `cluster <id>` (generate draft), `list-drafts`, `publish <slug>`, `discard <slug>`
  drafts/<slug>/  draft.md, meta.json (cluster id, sources, speccheck warnings, state), images

~/knowledgebase/
  app.py          + POST /generate/<cluster_id>  (enqueue a generation job; returns job id)
                  + GET  /drafts                  (list drafts)
                  + GET  /drafts/<slug>           (render draft HTML: banner + speccheck warnings + article)
                  + POST /drafts/<slug>/publish   (move to docs/consolidated/<slug>/, rebuild)
                  + POST /drafts/<slug>/discard    (delete draft)
                  + GET  /api/genjobs/<id>        (job status)
  /semantics      + a "Generál" button per cluster -> POST /generate/<id> -> job page -> draft

Prompt contract (genprompt.py)

Input: cluster meta (label, members: each {manual_id, page_url, heading}) + each member’s full section markdown (text with inline ![](img) refs), loaded from docs/<page_url without trailing slash>.md.

The prompt instructs claude (paraphrase):

You are consolidating multiple car-repair manual sections into ONE how-to article on <label>.
RULES (hard):
1. Use ONLY facts present in the provided sources. Add nothing not in them.
2. Tag every specification, step, torque value, measurement, and part number with its source in the form [<manual_id>].
3. Reproduce all numbers, units, torque values, and wire colors VERBATIM — never round, convert, or infer.
4. Where sources DISAGREE on a value or step, present BOTH with their tags and a ⚠ marker — do not pick one.
5. Carry through relevant source images: keep their ![](<image>) refs where the text references them, with the source tag.
6. If the sources don’t cover something, omit it — do not fill gaps.
Output: clean markdown, # <label>, a short intro naming the sources, then the consolidated procedure. No preamble/postamble.

Each source is delimited and labeled: === SOURCE [<manual_id>] — <heading> === then its markdown.

generate.py (background job)

Load clusters.json, find cluster by id; gather member section .md paths (docs/<manual_id>/<page>.md) and their image files.
Build prompt via genprompt.build_prompt(cluster, sources).
Run claude -p --model opus with the prompt on stdin (or -- arg); capture stdout = draft markdown. Timeout generously; the box is slow/shared.
Slug = safe_slug(label). Write drafts/<slug>/draft.md; copy every image referenced in the draft (resolved from the source dirs) into the draft dir; write meta.json {cluster_id, label, sources:[manual_id...], state:"draft", speccheck:[...], created}.
Run speccheck.check(draft_md, concatenated_source_text) → list of {number, context} warnings for numbers not found in sources; store in meta.
Single background worker (GPU-free but claude is heavy + shared; serialize with a queue like ingest.py). Job states: queued → generating → done/failed.

speccheck.py (pure accuracy guard)

numbers(text) -> set[str] — extract number tokens (integers, decimals, with optional unit suffix like 40, 1.5, 35nm; normalize whitespace/case for comparison).
check(draft, sources_text) -> list[dict] — for each number token in draft not present in sources_text, return {number, context} (the ~40 chars around it). Empty list = every draft number traces to a source. This is a FLAG surfaced in the review UI, not a hard block.

Draft / review / publish (knowledgebase app)

GET /drafts/<slug> renders: a prominent ”⚠ AI-generált — ellenőrizd a források alapján” banner, the speccheck warnings (if any), Approve/Discard buttons, then the rendered draft markdown (images served from the draft dir), and links to the source sections.
POST /publish → move drafts/<slug>/ into docs/consolidated/<slug>/ (as a normal KB manual folder: the draft.md becomes the section page(s), images alongside), run build_site(). Now browsable; picked up when kbvec index re-runs. Add a persistent source-cited header to the published page.
POST /discard → delete the draft dir.
Drafts and published articles carry the AI-generated provenance (banner on draft; header note + source links on the published page) — never presented as a manual.

Failure / ops notes

claude -p shares the box with Frigate/OCR/vectorize and the box is power-flaky → single serialized worker, one generation at a time; a job lost to a reboot is just re-run (idempotent by slug; re-generating overwrites the draft).
Read-only over the source manuals and Qdrant; only writes drafts/ and (on publish) docs/consolidated/.
claude failure / empty output → job failed with the stderr tail; no draft published.
speccheck warnings never block; they inform the human gate.
--model opus explicit (never Sonnet); if unavailable, job fails loudly rather than silently downgrading.
Publish is the only step that changes the live KB; it’s human-gated and atomic (build swap already exists).

Testing / verification

genprompt unit: prompt contains all member manual_id tags, the label, the rules, and each source’s markdown delimited/labeled.
speccheck unit: a draft number absent from sources is flagged; a present one is not; unit-suffixed and decimal numbers handled; empty when all trace.
safe_slug/paths: publish target is realpath-contained in docs/consolidated (reuse the existing traversal guard).
generate (real): generate for a known cross-source cluster (e.g. a brake or cooling cluster); assert the draft cites ≥2 distinct source tags, contains a # <label> heading, carries at least one source image, and speccheck runs (report its warnings).
Faithfulness spot-check (the real proof): read the generated draft against its sources — every torque/spec/number present is tagged and traceable; a genuine disagreement (if any) is shown both-ways with ⚠. Report an honest read.
Draft→publish e2e: /drafts/<slug> renders with banner + warnings; publish moves it to docs/consolidated/<slug>/ and it renders in the KB (200) with images; discard removes a draft.
No regression: existing KB routes/tests still pass; source manuals untouched.

Risks

Risk	Mitigation
Hallucinated spec (wrong torque)	never-invent prompt + per-number speccheck flag + human review gate + linked originals
Model picks one side of a conflict	rule 4 (both-ways + ⚠); spot-checked in testing
Dropped/garbled source image	copy only images actually referenced, resolved from source dirs; missing → omit, don’t break
Heavy claude run destabilises the box	single serialized worker, one at a time; re-runnable after a reboot
Published article mistaken for a manual	AI-provenance banner/header + source links; lives in a separate `consolidated/` folder
Model unavailable / silent downgrade	`--model opus` explicit; fail loudly, never publish a failed/empty generation

Levandor

Explorer

KB consolidated generation (phase 3) — design