Levandor

kick-fix.com archive ingest — HTML→markdown for the knowledgebase

3 min read

Ingested a 112 MB offline archive of the defunct kick-fix.com Suzuki Sidekick/Tracker repair site into the 2026-07-24-knowledgebase under docs/kick-fix/. Because it is a downloaded archive of a dead site (no live crawling), there is no ToS/robots concern. Result: 25 topics, 127 articles, 741 images, 8 text-layer PDFs ingested; 27 image-only PDFs deferred for OCR. The hard-won lessons here are reusable for any old table-layout HTML → markdown conversion.

For Agents

Script: /home/levander/knowledgebase/kickfix_ingest.py on telep-mainframe (runs in the knowledgebase venv: markdownify + beautifulsoup4 + pypdfium2). Output: ~/knowledgebase/manuals-src/docs/kick-fix/ (25 topic folders → mkdocs auto-nav). Clean-regenerate each run: rmtree docs/kick-fix before writing so re-runs leave no stale files. Counts: 25 topics / 127 articles / 741 images / 8 text-layer PDFs; 27 image-only PDFs deferred to OCR.

The two that cost real time

markdownify SILENTLY DROPS <img> inside <table> cells

Old sites use <table> for layout, not tabular data, so nearly every figure lived inside a table cell — and markdownify drops those images with no error. Initial output had 11 image refs for 122 pages while all the images had been copied to disk, just never referenced. Fix: before calling markdownify, .unwrap() the layout containers (table, tbody, tr, td, th, center, font) so text and images flow linearly in document order; rewrite each <img src> to the copied basename in place; wrap each img in its own <p>. This restored inline placement — 11 → 528 refs. A trailing ## Figures section is kept only as a minimal safety net for any img markdownify still drops.

"See photo here" TEXT links ( <a href="X.jpg">) are separate from inline <img> — 365/664 were broken

Beyond inline images, articles link to photos as anchor hrefs (<a href="foo.jpg">see photo</a>). Those targets were never copied → 365 of 664 were broken. Fix: also walk every <a href>; if the href (URL-decoded, ../ resolved cross-topic against the archive tree) points at an existing image, copy it under its basename and rewrite the href to the basename (keep it as a text link, do not inline it). Leave external http(s)/mailto links untouched; drop dead archive-local hrefs. Result: 365 → 0 broken, image files 449 → 741.

Other gotchas

  • URL-decode and resolve ../ for both src and href. Paths carry %20 and cross-topic ../ references that must be resolved against the archive tree before checking existence/copying.
  • Skip framesets and <200-char prose stubs — navigation frames and near-empty pages aren’t articles.
  • Clean-regenerate every run (rmtree docs/kick-fix) so re-runs don’t accumulate stale files.
  • Mechanical conversion can’t invent structure. Some source articles are genuine run-on blobs with few <br>; the converter preserves the breaks that exist but can’t manufacture missing ones. An optional claude -p reformat pass was considered and DEFERRED — the mechanical output was good enough for the user.

Why this feeds vectorization

kick-fix is community repair knowledge that overlaps and complements the official manuals. Once in the knowledgebase it was picked up by 2026-07-24-kb-vectorize-complete and now blends into cross-source semantic search (e.g. “depressurize the fuel line” surfaces kick-fix’s injectors de-pressurize guide alongside the Vitara Engine Fuel section).

Graph View

Backlinks