Homelab Topics

For Agents

Living index of themes for this project. Each H2 is a topic; bullets are wikilinks to related notes. Updated by obsidian-documenter when documenting work. Read by historian at bootstrap. Topics kept alphabetical.

Access & Remote Management

• telep-router — SSH via Tailscale MagicDNS; on macOS a shell wrapper breaks bare ssh, use /usr/bin/ssh explicitly
• Access — Frigate UI over Tailscale Serve (--https=443 → 127.0.0.1:8971), real LE cert, tailnet-only; iOS Private Relay breaks MagicDNS in Safari
• 2026-07-24-global-dashboard — home.taild4189d.ts.net, the tailnet landing dashboard linking Kamerák / Tudásbázis / Top képek / Router
• filestash — Windows drive browser over Tailscale Serve (--https=8445 → 127.0.0.1:8334), tailnet-only
• Prerequisite router access — mainframe→router SSH added via LAN-only key-only dropbear + /home/levander/.ssh/router_alarm (router was Tailscale-SSH-only, ACL-blocked)
• 2026-08-12-krakensdr-doa-rig — KrakenSDR rig: ssh levander@raspi.local (eth0 192.168.1.200); DoA UI over Tailscale at http://kraken-rig.taild4189d.ts.net:8080/doa — the :8080 port is REQUIRED (missing it → port 80 → refused, looks like an ACL problem but isn’t)
• 2026-08-15-telep-lan-split-horizon-caddy — internet-independent LAN access to every service via *.telep.lan (wildcard DNS on telep-router → mainframe, fronted by Caddy :80 on telep-mainframe): files/frigate/kraken/top/print/whiteboard/ruview/go2rtc.telep.lan etc. Works with no internet; Tailscale stays the remote path. Complements tailnet-service-exposure-convention
• 2026-08-15-dhcp-outage-duplicate-reservation-postmortem — recovery access when the LAN/DNS is down: reach the router directly over Tailscale at ssh root@100.69.112.32 and the mainframe at ssh levander@100.115.209.87 (MagicDNS may not resolve without the router’s DNS)
• 2026-08-15-admin-portal-passkey-olivetin — ⭐ admin.levandor.io: iPhone-only passkey-gated admin panel (Caddy → oauth2-proxy OIDC → OliveTin ssh actions), IdP = Pocket-ID at id.levandor.io. Actions ssh to mainframe .123 / router root@100.69.112.32 / pi .200. Caddy MUST bind 192.168.1.123 (else :443 collides with Tailscale 100.115.209.87:443); remote/off-LAN access deferred (binds LAN IP only). Status/health actions now call a single telep-selftest script over ssh (/usr/local/bin/telep-selftest [dns|airplay|camwall|services|cameras|health|alerts|all]) — logic kept OUT of the YAML to dodge OliveTin’s Go-template engine choking on literal {{ }}
• 2026-08-17-home-dashboard — ⭐ home.telep.lan “where is what” live-status service dashboard (gethomepage/homepage, Next.js, /home/levander/homepage/, network_mode: host, binds 127.0.0.1:3010). One URL to find every service + its fallback (raw IP / tailnet port / ssh target) during an outage; siteMonitor: up/down dot per tile. Reachable TWO ways (redundant): http://home.telep.lan via the :80 Caddy, and https://telep-mainframe.taild4189d.ts.net:8450 via tailscale serve — the tailnet path is deliberately Caddy-independent so the board survives a Caddy outage. Outage-safe: no icon:/CDN fetches, local widgets only. Must set HOMEPAGE_ALLOWED_HOSTS
• caddy-boot-race — 🔴 runbook: ALL *.telep.lan + admin/id.levandor.io down after a reboot = Caddy lost a :443 bind-race against the LAN IP coming up; systemctl status caddy → fix ip_nonlocal_bind=1 + systemctl start caddy (+ check oauth2-proxy up else admin 502s)
• 2026-09-02-lg-tv-network-control-presence — 📺 TV control from the admin portal: ⚠️ SUPERSEDED — the TV buttons now live DIRECTLY ON HOMEPAGE, embedded via the gethomepage iframe service widget pointing at /opt/tv-control/tv-http (127.0.0.2:8102, tailnet https://telep-mainframe.taild4189d.ts.net:8451, POST /api/{on,off,camwall,pair}, GET → 405); they were moved off OliveTin because the user wanted them on Homepage, not because of any OliveTin limitation. Historical: TV be / TV ki / TV → HDMI1 OliveTin actions used the ssh 192.168.1.123 '...' pattern. 🔴 OliveTin 3000.19.0 REMOVED /api/StartActionByGet/<id> — the API is now Connect-RPC at /api/olivetin.api.v1.OliveTinApiService/<Method>, where ❓ whether StartActionByGet is usable over GET is DISPUTED and was never settled — one investigation tested it and got 405 Allow: POST, another found both StartActionByGet and StartActionByGetAndWait registered in the binary but did not test them; record as unresolved, not fact. What is established: action identifiers are now bindingId UUIDs minted at config-load time so any hardcoded trigger URL rots on restart → the one-click “dashboard tile fires an OliveTin action” pattern is dead on 3000.x; a internal/webhooks / execOnWebhooks subsystem exists in the binary but its YAML schema is unconfirmed, so the tile just links to the panel
• 2026-09-05-freecad-mcp-502-dead-container-and-qt-event-loop-wedge — ✅ CORRECTION: “cad is not Tailscale-SSH-enabled” does NOT mean a person on site. Only telep-mainframe and telep-router carry Tailscale SSH, but cad is not a separate machine — it is a userspace tailscaled sidecar node on telep-mainframe, as are knowledgebase / drive / chatcut / orcaslicer / bambuddy, so ssh 100.120.203.1 (any sidecar’s tailnet IP) lands on telep-mainframe, where the docker stack lives anyway. Consistent with the raw forwarder in 2026-08-31-tailnet-plaintext-port-hardening (:22 → the mainframe’s own sshd). A full FreeCAD recovery was performed entirely remotely on 2026-09-05.

Agents & Control-Planes

• 2026-08-08-telep-master-agent-impl — master control-plane agent LIVE on the HOST telep-mainframe — an always-on, phone-drivable Claude that mirrors the Incus container control-planes and adds systemd autostart. control-plane.service (User=levander, Type=forking, Restart=always, enabled) runs claude remote-control --name control-plane-telep-mainframe --spawn same-dir in tmux; came up headless, shows Connected · obsidian · main in the Claude app Code tab, verified from the phone. ⚠️ the app displays the auto-SPAWNED session name (e.g. …-velvety-star), NOT the device --name. Vault cloned to /home/levander/obsidian (write deploy key id_ed25519, repo-local core.sshCommand); obsidian-sync.timer (5min/15min) does bidirectional pull-rebase+commit+push via /usr/local/bin/obsidian-sync.sh. Harness parity: path-rewritten historian+obsidian-documenter at /home/levander/.claude/agents/ + host ~/.claude/CLAUDE.md (Mac-only sources absent, degrade gracefully). 🔴 SECURITY: trust dialogs pre-seeded (~/.claude.json) but --dangerously-skip-permissions NOT used — tool permissions stay ACTIVE → destructive ops prompt in the app = human-in-the-loop; CAVEAT passwordless sudo means an approved sudo = instant root (guardrail is approval, not a sandbox)

• 2026-08-22-control-plane-oauth-zeroed-creds — control-plane agent OAuth error root-caused to a zeroed ~/.claude/.credentials.json (all tokens empty, expiresAt=0); empty refreshToken meant no silent refresh → fix = interactive claude login on the box (ssh -t, real TTY — the ! prefix / plain ssh fail); the service shares the creds file so Restart=always picks up new tokens with no restart. Gotcha: off-LAN only the slow DERP-relayed Tailscale path works (ConnectTimeout 30 s+)

• 2026-08-08-telep-master-agent-design — the design/spec for the above: motivation (the 2026-08-08 UPS reboot cycle left Frigate jammed with no remote way to drive the box healthy), the Remote Control mechanism, components table (tmux, vault clone, control-plane.service, obsidian-sync.timer, trust pre-seed, worklog, harness parity, skills), the deliberate active-permissions safety posture, known gotchas (Docker/Incus FORWARD, subagent path hardcoding, multi-writer sync ceiling, no WOL, PID-vs-pgrep supervision), and success criteria

• 2026-08-03-telep-router-factory-reset-recovery — the autonomy blast-radius lesson the master-agent’s active-permissions posture is built to avoid: a prior autonomous agent took the site’s only DNS/gateway down and forced a factory reset with no backup

• 2026-08-26-cad-designer-agent — a SECOND always-on Claude agent on the host, dedicated to parametric CAD: cad-designer.service (User=levander, Type=forking, Restart=always, enabled) runs claude remote-control --spawn same-dir in workspace /home/levander/cad-agent with the FreeCAD MCP wired (http://127.0.0.1:9876/mcp, local loopback not the tailnet hairpin → ✔ Connected). Appears in the Claude app Code tab as session cad-designer-telep (deterministic — sets --remote-control-session-name-prefix, unlike the control-plane’s random name). Shares the box’s ~/.claude/.credentials.json with the control-plane (no separate login), trust + MCP-enable pre-seeded per-project in ~/.claude.json; permissions stay ACTIVE. 🔴 GOTCHA: a Type=forking tmux unit sharing the control-plane’s DEFAULT tmux server never persists (no new daemon for systemd to track → ExecStop fires → restart thrash) — give it its OWN socket -L cad-designer. Drives the FreeCAD workstation export pipeline: export_all helper → STEP/STL/3MF in /exports, browsable at https://cad.taild4189d.ts.net:8080/ (nginx exports-http + tailscale serve --https=8080), auto-synced to the Mac ~/cad-exports/ (launchd rsync, 90 s)

• 2026-08-31-telep-kb-mcp-server — the knowledgebase as MCP tools: telep-kb-mcp.service (FastMCP streamable-http, /home/levander/telep-kb-mcp/server.py, 127.0.0.1:9099) wrapping knowledgebase.service :8092 / kb-qdrant, on its own tailnet node https://knowledgebase.taild4189d.ts.net:8443/mcp and fronted by the Aperture connector telebkb (https://ai.taild4189d.ts.net/v1/connectors/telebkb/, 307→upstream). Tool surface deliberately mirrors the historian subagent’s “named sources, explicit selection” pattern — get_collections() / search(collection, query, limit, folder) / get_note(path) / list_topics(), with an unknown collection returning {error, available} instead of throwing so the model self-corrects from the error rather than needing a discovery call first

• 2026-08-31-nextcloud-drive-code-server-runbook — drive-mcp.service: WebDAV-backed MCP at https://drive.taild4189d.ts.net:8444/mcp, loopback 9100, served from the existing drive sidecar (not a new node), acting as a single shared identity; secrets in /etc/drive-mcp.env (0600, not beside a compose dir). ⚠ FastMCP needs allowed_hosts or it 421s behind tailscale serve — same trap as 2026-08-31-telep-kb-mcp-server

• 2026-08-31-openchatcut-chatcut-deploy — OpenChatCut MCP at https://chatcut.taild4189d.ts.net/api/external-mcp/mcp (Streamable HTTP, Bearer, protocol 2025-06-18, serverInfo.name=openchatcut), registered in Claude Code at user scope, Connected. ⚠ initialize works headless but timeline/editing tools need a bound target_project + generally a connected browser (open the web UI). Token in /home/levander/openchatcut/.env.local (OPENCHATCUT_MCP_TOKEN, 0600)

• 2026-09-01-bambuddy-slicer-api-pipelines-mcp — bambuddy-mcp for Aperture: because the ai node isn’t self-hosted it needs a network MCP, but bambuddy-mcp (MrMebelMan/bambuddy-mcp, 731 endpoints via list_categories/search_tools/execute_tool/find_printer meta-tools) is stdio-only → self-hosted on the mainframe wrapped in an stdio→streamable-HTTP bridge (mcp-proxy) and path-mounted onto the existing bambuddy node (serve --set-path=/mcp, no new node). Endpoint https://bambuddy.taild4189d.ts.net/mcp, verified from personal-mac (mainframe can’t self-hairpin). ⚠ /mcp has no auth of its own — tailnet + tag:telep ACL is the only gate and execute_tool can drive the printer; no Funnel

• 2026-09-03-remarkable-paper-pure-claude-integration — 🔮 PLANNED, device not purchased: reaching a reMarkable Paper Pure from Claude Code via remarkable-mcp (SamMorrowDrums, MIT, uvx remarkable-mcp). Four transports — local dir / USB web / cloud (paid Connect subscription) / SSH — and only SSH exposes remarkable_author, the tool that writes native ink (draw/add_page/create_document) byte-identically to the canvas Save button; writes are FIFO-serialized through one dispatcher behind a shared xochitl restart barrier (defer_restart=True + remarkable_refresh() batches bulk imports into one restart). 🔴 Enabling Developer Mode FACTORY-RESETS the tablet (Paper Pro / Pro Move / Pure, unlike rM1/rM2) → decide on SSH before putting anything on the device. ⚠️ Every transport is PULL — no webhook, no watch; “live” = current when queried. 🔒 --http has no auth, rejects wildcard binds and documents 127.0.0.1 — conflicts with the 127.0.0.2 rule in 2026-08-31-tailnet-plaintext-port-hardening; expect the same 421 DNS-rebinding trap as 2026-08-31-telep-kb-mcp-server / drive-mcp. REMARKABLE_ROOT_PATH scopes the server to one folder; --read-only kills every write tool

Alerting & Notifications

• Alerting — Telegram bot “Dezsi az őr” → supergroup; frigate-notify polls the Frigate HTTP API every 15s (no MQTT); dedup is zone-aware
• 2026-07-17-intruder-alarm — same “Dezsi az őr” bot; auto-arm/disarm + 🚨 BETŐRŐ! intruder escalation (1×day/2×night) + snapshot; sole getUpdates consumer for enrollment/commands ⛔ RETIRED 2026-09-07 — unit and script removed from live paths, archived (not deleted) at /root/retired/2026-09-07-intruder-alarm/; restore = mv both back + daemon-reload. This note is now a historical design record. See 2026-09-07-alert-source-inventory §6.
• 2026-07-23-frigate-viewer-alert — same frigate-notify bot+chat: Telegrams 👁 <device> opened Frigate when someone opens the camera UI; identity from tailscale serve’s X-Forwarded-For → tailscale device name; per-source 5-min session cooldown
• 2026-07-28-frigate-notify-camera-exclude — silencing alerts for one camera (telep_cam3, aimed indoors): the correct key is frigate.cameras.exclude (under the frigate: block, by camera id); koanf silently ignores misplaced keys so verify against a real event, not “Config validated!”
• 2026-08-15-camera-alerts-dead-tailscale-magicdns-docker-dns — 🔴 alerts SILENTLY DEAD ~17 days (2026-07-29→2026-08-15): Tailscale MagicDNS on the host broke Docker container external DNS so frigate-notify couldn’t resolve api.telegram.org. 🎣 the Already notified on this zone flood is a RED HERRING (otter cache, 1 h TTL, caches BEFORE the send → a failed send masquerades as dedup); the real error is a buried WRN server misbehaving (docker logs … | grep WRN). FIX = pin dns: [192.168.1.1, 1.1.1.1] on the frigate-notify service. Runbook camera-alerts-not-firing
• 2026-08-15-admin-portal-passkey-olivetin — telep-selftest alerts — a canary for the alerting pipeline: checks frigate + frigate-notify up, that api.telegram.org is reachable FROM INSIDE the frigate container (docker exec frigate curl …), the last “Alert sent” timestamp, and counts misbehaving DNS errors in the last 30 min — i.e. it directly watches for the MagicDNS→container-DNS outage recurring. Exposed as the OliveTin “Riasztások (Telegram)” action
• 2026-08-10-top-kep-community-voting-system — scheduled Telegram voting reminders to group -1004475187307 via ~/top_kep_remind.py (token read at runtime from ~/nvr/frigate-notify/config.yml, not in source): daily 20:00 (only if a new candidate since the watermark), Sun 19:30 weekly, daily 21:00 monthly (fires only when tomorrow is the 1st). Three Persistent=true systemd timers; --dry is side-effect-free
• 2026-08-03-cam-stall-recovery-and-casino-alert — casino/gambling DNS DETECTOR (alert half) on telep-router: /etc/casino-alert.sh (busybox-ash) follows /tmp/dnsmasq-queries.log with tail -F … | while read line, matches a gambling regex inside the loop (busybox grep has NO --line-buffered), applies a 1 h per-(client|domain) cooldown via /tmp/casino-alert-state, and Telegrams the same frigate-notify bot via uclient-fetch (token read from ~/nvr/frigate-notify/config.yml, not hard-coded). procd service /etc/init.d/casino-alert (respawn, enabled); verified end-to-end. Blocking (/etc/casino-block.conf) is deferred — a dnsmasq restart risks another DNS outage AND re-stalls the weak .139 camera
• 2026-08-26-frigate-notify-cat-alerts — cat detections weren’t notifying: cat was tracked in Frigate but missing from frigate-notify’s alerts.labels.allow. The filter = TWO independent lists — frigate.cameras.exclude (telep_cam3/4 out → only cam1/2 notify) AND alerts.labels.allow (was person,car, now person,car,cat). RULE: a new tracked object must be added to BOTH Frigate objects.track AND frigate-notify labels.allow. Config /home/levander/nvr/frigate-notify/config.yml (v0.5.4, WEB API poll 15s, MQTT off). Gotchas: after a Frigate restart frigate-notify briefly logs /api/review error=500 (recovers on its own); it can get STUCK re-processing the same review_id for an EXCLUDED camera every 15s → docker restart frigate-notify clears both
• 2026-09-07-session-handover — ⭐⭐ ALL homelab alerts rerouted Telegram → Matrix into the encrypted room telep-ertesitesek (!gCuOI7uLN2JNqp5XR2voIsisfO0CXxBECMBh7CqynqQ, m.megolm.v1.aes-sha2). Relay /opt/matrix-relay/matrix-relay + matrix-relay.service, matrix-nio 0.26.0 / vodozemac 0.10.0 (nio 0.26 dropped libolm; libolm3 present but unused), key store /opt/matrix-relay/store. Binds 127.0.0.2:8118 AND 172.18.0.1:8118 — the docker-bridge bind is required because frigate-notify on nvr_default cannot reach host loopback; verified NOT tailnet-exposed. 🐞 Two real bugs: nio only re-shares a megolm session when it EXPIRES ⇒ late joiners never got keys (fix: refresh device lists + rotate on member-device change; ignore_unverified_devices=True is the 0.26 API), and a persisted sync token makes the incremental sync return no rooms ⇒ the join check failed forever (fix: force a full sync at startup). Routing: bambuddy provider id=2 webhook on, id=1 Telegram DISABLED but not deleted (rollback); frigate-notify webhook on, telegram off; intruder-alarm.py untouched/inactive. 🚩 frigate-notify’s native matrix backend deliberately NOT used — no persistent volume beyond config.yml ⇒ its crypto store is wiped every restart ⇒ recurring “unable to decrypt”; one E2EE identity, one persistent store. 🐛 Payload URLs use the container-internal http://frigate:5000 the relay cannot resolve — rebuild against the public tailnet URL. ⚠️ A synthetic test alert (“Telep Cam9”, 9999999999.000000-nope) was posted into the LIVE room and made the user think a real detection had failed — test alerts belong in a scratch room or must be redacted. ✅ RESOLVED — images and finish photos now attach. ⚠️ The suspected incomplete info block was NOT the cause; the real bug was a concurrency race — _prepare() mutated nio’s shared users_for_key_query from the HTTP thread while sync_forever consumed it, so one side called keys_query() empty ⇒ LocalProtocolError: No key query required. ⇒ the session died BETWEEN the text and image sends (RuntimeError: Session is closed). Fix = read-only device_store inspection (never mutate nio’s key-query state) + text and image in ONE coroutine (3 attempts, text tracked so retries don’t duplicate) + fallback snapshot.jpg → thumbnail.jpg → /api/<camera>/latest.jpg. bambuddy key = finish_photo_url (relative path, on print_complete/print_failed/print_stopped); 🚩 both thumbnail endpoints return the SLICER RENDER — the camera capture lives only under /archives/{id}/photos/{filename}; PNG magic-byte support added. 🟡 Still unproven: the live print_complete hook has never fired with a photo (bambuddy’s test endpoint sends a generic payload without the key) — the first real print confirms it. ✅ Test alerts now belong in #relay-scratch:chat.taild4189d.ts.net. 🔴 And the reason that matters: cameras.exclude is enforced in frigate-notify, UPSTREAM — the relay has no camera filter and posts anything POSTed to :8118, so direct posts reach the LIVE room regardless of camera. ⚠️ All 41 relay-sent events were redacted from the live room — there is NO alert history before this point
• 2026-09-07-alert-source-inventory — ⭐⭐ THE alert map: six producers across telep-mainframe + telep-router. ✅ Migrated: bambuddy (provider id=2), frigate-notify (webhook → 172.18.0.1:8118/frigate). 🚧 Still on Telegram at audit time (migration IN PROGRESS then, ✅ DONE and verified later the same day — see the UPDATE at the end of this entry): frigate-fps-watchdog (/usr/local/bin/frigate-fps-watchdog.py, creds /etc/nut/telegram.env; fires most, 12+ real alerts Sep 4), the NUT outage handler (/etc/nut/nut-outage-handler.sh via upsmon NOTIFYCMD), the top-kép reminders (3 timers; a family nudge, not security — own room), frigate-viewer-alert (highest volume; ⚠️ fires on ANY tailnet request to Frigate, not just /live/; 🎣 its rate is UNPROVABLE from logs — notify.send() prints only on FAILURE). ⛔ Not migratable: router casino-alert (infra-blocked — relay binds 127.0.0.2 + 172.18.0.1, neither reachable from the router — and running TWICE, so every hit double-sends) and intruder-alarm (interactive bot: inline keyboards + callback MAC enrollment; Matrix has no equivalent ⇒ rewrite, not migration — leave dormant or retire). 🔴 HEADLINE: Telegram fails exactly when it is needed — 31 of 34 UPS sends FAILED, all inside the Aug 7 / Aug 18 outage windows (power out ⇒ WAN down ⇒ the alert can’t leave the building); Continuwuity is local, so the UPS migration is a CORRECTNESS FIX, not tidiness. 🔴 Silently broken: smartd alerts go NOWHERE (no MTA installed at all), netdata notifies nobody (all 28 SEND_* = "NO"), no OnFailure= on any of ~70 custom units. ✅ CORRECTS the vault twice: print-guard and tv-presence are NOT alert producers (bambuddy-API-only / no HTTP client at all — “never fired” is about their action paths), and the relay’s total lack of filtering is now source-CONFIRMED (source = the URL path as a bare label) ⇒ unauthenticated + unvalidated; any nvr_default container can post into the live room. 🎣 telep-selftest will keep showing up in Telegram greps — it is a reachability probe, not an alerter. ✅ UPDATE 2026-09-07 — sources 1–4 are MIGRATED, verified and running: frigate-fps-watchdog → [fps-watchdog], nut-outage-handler.sh → [ups], top_kep_remind.py → [top-kep], frigate-viewer-alert → [viewer-alert], all on http://127.0.0.2:8118/<label>. Telegram code kept INTACT but INERT (bambuddy id=1 precedent) ⇒ each is a one-line swap with the exact rollback line recorded; .bak beside all five files; 34 unit tests pass; alert semantics UNCHANGED (viewer-alert still fires on any tailnet request to Frigate — proven by a GET / delivery). 🛡️ Live room never touched — all testing on a throwaway :8119 relay with its own device + crypto store (since removed), live relay PID 1564638 never restarted, grep -c 'sent event' → 0 for the whole window. 🟡 STILL UNPROVEN: the :8118 production path was never exercised end to end — only the port digit differs from what was tested, grep-confirmed but not traffic-tested; the first real alert is the test (20:00 top-kep, or viewer-alert the next time anyone opens Frigate). 🟡 No real UPS event simulated; no unattended timer fire yet. ⛔ Source 6 intruder-alarm is RETIRED (2026-09-07) — archived, not deleted, at /root/retired/2026-09-07-intruder-alarm/ (unit + script + README); systemctl status → “Unit intruder-alarm.service could not be found”; restore is mv back + daemon-reload. Nothing depended on it — grep-verified: the only hit outside its own files was a comment in /home/levander/ruview/scripts/c6-presence-watcher.py plus two coincidental matches in ruview/examples/research-sota/. 🧹 Side effect: its Unknown key 'StartLimitIntervalSec' in section [Service] warning on every daemon-reload is gone, and there is no wider misconfiguration — camwall.service, camwall-x.service, nvidia-cdi-refresh.service all have that key correctly in [Unit]. 🔑 Token-block readers are now top_kep_remind + frigate-viewer-alert (kept for rollback) + the duplicate top-kep-remind.py — the count changed, the do-not-delete verdict did not.

Backups & Restore

• 2026-09-07-session-handover — ⭐⭐ Matrix homeserver backup, built AND restore-rehearsed. /usr/local/sbin/continuwuity-backup.py, root crontab 30 3 * * * (deliberately offset from bambuddy’s 03:00), dest /home/levander/backups/continuwuity/ (dir 0700, archives 0600, ~222 KB), 7-day retention with pruning. Native online RocksDB backup ⇒ zero recurring downtime, triggered non-interactively by SIGUSR2 via admin_signal_execute = ["server backup-database"] — verified empirically (docker kill -s SIGUSR2 → Created database backup #1 … in 47 files, no admin room, no human). 🔴 Two findings that would have made a naive backup UNRESTORABLE: backup-database emits a RocksDB BackupEngine store (meta/, private/, shared_checksum/) that is NOT an openable database, with no restore admin command — so the script materialises it into a plain db/, making recovery tar xzf + cp; and the native backup covers only the database, so without media/ and archive/ the server refuses to boot (“Failed to verify media integrity”). ✅ Restore actually rehearsed: extract → throwaway container → /_matrix/client/versions 200 → users list-users returned all five accounts. 🔴 STILL NO OFF-SITE BACKUPS — Matrix backups, bambuddy backups and live data are all on the same 3.6 T LVM volume; decision made for restic → Cloudflare R2 with client-side encryption (restic 0.18.0 is in Debian 13’s repo but NOT installed; rclone v1.60 is installed with only a gdrive remote). Needs an R2 bucket, an S3-compatible token, and a restic repo password in a password manager — it is UNRECOVERABLE. ⚠️ /opt/matrix-relay/store is backup-critical: lose it and every previously-sent alert is permanently undecryptable for everyone

Bandwidth & Traffic Metering

• 2026-07-22-wifi-usage-strip — live per-device WiFi bandwidth on the camwall; wifi-usage.service polls iw dev <ap> station dump every 5s and diffs byte counters into Mbps
• [[2026-07-22-wifi-usage-strip#2-bytes-come-from-iw-not-iwinfo|2. Bytes come from iw, not iwinfo]] — iwinfo assoclist has no byte counters, iw station dump does; per-device metering needs no nftables accounting, conntrack parsing or nlbwmon, and iw ships on the router
• SESSION-HANDOVER — the tc HTB 15 Mbit cap on 192.168.1.135, persistent via /etc/hotplug.d/iface/99-throttle-fm; bmon for live per-interface rates
• 2026-07-28-net-monitor — cron probe (probe.sh every 5 min → netlog.csv) logging RTT/loss + an hourly 20 MB throughput/bufferbloat sample to characterize suspected fixed-wireless peak-hour degradation
• 2026-08-31-episodic-wan-degradation — the first real payoff of the netlog: 35 days of /home/levander/net-monitor/netlog.csv classified by day (samples < 100 Mbit = degraded) revealed multi-day degradation episodes, worst individual samples 3.0-17.2 Mbit and three 100%-loss samples on Aug 26. ⚠ the 20 MB sample is small enough that TCP slow-start inflates variance — trust DAY-LEVEL aggregates, not single rows; likewise never conclude from a single paired Mac-vs-mainframe comparison in an episodic-fault environment. RF baseline since captured (2026-08-31): weak 5G cellular signal (SINR ~9.5 dB, RSRP -102 dBm). ✅ RF logger now DEPLOYED (/home/levander/net-monitor/ne200_signal.py, cron */15 → rflog.csv, correlate vs netlog.csv by timestamp): the NE200 “GDPR encrypt” login is fully reverse-engineered — POST /cgi_gdpr?9 (?9 REQUIRED, its absence WAS the 71014), HTTP/0.9, RSA-512 sign + AES-128-CBC, username user not admin, TokenID header, signal OID DEV2_LTE_SERVING_CELL_INFO. ✅ Live readings confirm the root cause in real time: serving N78 5G on QPSK downlink with RSRP swinging -100→-108 dBm in ~2 min (⚠ trust nr_dl_mod/nr_rsrp/nr_cqi, not nr_sinr — its raw units are unclear)
• 2026-09-04-starlink-wan-migration-dish-telemetry — 🛰️ the successor logger after the WAN moved to Starlink: /home/levander/net-monitor/starlink_status.py → starlink.csv, cron */5 as levander, deliberately mirroring the net-monitor/NE200 conventions (HDR constant, header only if CSV absent, %Y-%m-%dT%H:%M:%S%z, blanks for missing, never crashes a row). Client = grpcurl v1.9.4 static binary (Debian 13’s PEP 668 made Python grpcio not worth a venv + toolchain for one RPC; reflection means no .proto files), calling SpaceX.API.Device.Device/Handle {"get_status":{}} on :9200. 🔴 -emit-defaults is MANDATORY or every healthy-state zero value silently vanishes. Cutover throughput baseline: 235–279 Mbps wired, 18.3–35.3 ms, 0% loss (vs the NE200’s 3–17 Mbit episodes). ⚠️ .err takes success rows too and at */5 grows ~4× faster than the old */15 logger, with no rotation. The NE200 */15 cron line was removed (backup /root/levander.crontab.bak.1788486106) but ne200_signal.py + 342 rows of rflog.csv were deliberately preserved as the evidence base

Bluetooth & Vehicle Diagnostics

• 2026-07-31-telep-mainframe-ax210-ble-scan-broken — telep-mainframe’s onboard Intel AX210 (hci0) saw ZERO BLE devices — not even stray phones — because NO ANTENNA is plugged into the M.2 card (WiFi+BT share the u.FL connector; no antenna = no RF receive). Controller reports UP RUNNING/le (init needs no antenna) but hcitool lescan → Set scan parameters failed: Input/output error. OS-side fixes (btusb reload, USB rebind, firmware check, kernel 6.12.95→6.12.100, reboot) were an ineffective detour. Fix = plug the antenna into the AX210 u.FL (USB BLE dongle → hci1 only as fallback). Blocked the thermoprint appliance. Lesson: zero devices incl. strays = check the antenna FIRST
• 2026-07-30-obd2-elm327-bridge-rfcomm-fix — ELM327 OBD2 dongle → RFCOMM→TCP bridge on the Mac (obdbridge, launchd com.levander.obdbridge, ELM327-over-TCP at 100.83.222.120:35000; point a phone OBD app there as a WiFi adapter). Recurring “phone can’t connect / RFCOMM open fails”: the bus-powered dongle power-cycles (ignition off) → macOS SDP cache goes stale (system_profiler shows < Braille ACL > not Serial Port) → bridge falls back to hardcoded RFCOMM channel 1 → openRFCOMMChannelSync failed: ret=0x-1ffffd44 (kIOReturnError). Fix (device-scoped, spares BT keyboard/mouse): blueutil --disconnect/--connect 00-1d-a5-68-98-8b + launchctl kickstart the bridge; watch bridge.err for RFCOMM connected→client connected

CAD & 3D Modeling

• 2026-08-26-freecad-cad-workstation — ⭐ the FreeCAD headless GUI + noVNC + MCP stack on the dedicated cad tailnet node: MCP https://cad.taild4189d.ts.net:8443/mcp, live noVNC preview, /exports → browsable listing (:8080) + self-hosted Online3DViewer (:8090, STEP via OCCT WASM), cad_export.py one-call STEP+STL+3MF helper, launchd auto-pull to ~/cad-exports/.
• 2026-09-02-freecad-mcp-rpc-refused-gui-not-running — 🔴 [Errno 111] Connection refused from every FreeCAD MCP tool = the FreeCAD GUI isn’t running. The XML-RPC server lives on the GUI thread, so no GUI ⇒ no RPC ⇒ all tools dead on both the direct MCP server and the aperture route. Fix = “Start RPC Server” in the freecad-mcp workbench, in person (cad is not Tailscale-SSH-enabled).
• 2026-09-02-freecad-mcp-gui-dispatch-timeout-modal-dialog — 🧩 execute_code times out while get_rpc_status says healthy — the status call deliberately does not use the GUI thread, so healthy (and ping()→True, list_documents()→[]) proves nothing. Real error only visible via raw XML-RPC: GUI dispatch timed out after 90s. Cause = FreeCAD crashed earlier → its Document Recovery modal dialog owns Qt’s main thread and starves the dispatch queue. Diagnose xwininfo -root -tree, fix xdotool windowactivate <id> (non-destructive; window id changes each run; trailing BadWindow = already gone). Do NOT restart the container — ⚠️ PARTIALLY CORRECTED 2026-09-05, see the next bullet: DISPLAY is :1 not :0, and closing the dialog is not always enough. Also: FreeCAD cannot be launched via docker exec (AppRun line 13 hardcodes QT_QPA_PLATFORM=xcb) — only docker restart freecad / s6 works; curl :9875 → HTTP 501 is the healthy answer (XML-RPC is POST-only)
• 2026-09-05-freecad-mcp-502-dead-container-and-qt-event-loop-wedge — 🔴 ⭐⭐ Two stacked FreeCAD failures behind one 502, and the CORRECTION to the modal-dialog note. https://cad.taild4189d.ts.net:8443/mcp 502’d on both POST /mcp and GET / because the freecad-mcp proxy container had been Exited (128) for three days — docker inspect gave "failed to create shim task: ttrpc: closed", OOMKilled=false and RestartCount=0: the containerd shim died before the task was ever created, so restart: unless-stopped had nothing to restart. docker start freecad-mcp fixed it. Then the second half: dismissing the Document Recovery dialog is NOT always sufficient — xdotool windowclose removed the window (verified gone from the tree) and execute_code still timed out at 90 s with gui_dispatch still saying healthy; after ~3 days behind the modal the Qt event loop was simply dead. ✅ Real fix = MOVE /config/.cache/FreeCAD/v1-1/Cache/FreeCAD_Doc_*/ into a dated stash, THEN docker restart freecad (RPC back in ~10 s). ⚠️ DISPLAY is :1, not :0. 🟡 10 stashed recovery dirs await the user’s decision.
• 2026-08-26-cad-designer-agent — the always-on on-prem Claude that drives the FreeCAD MCP.

Cameras & Surveillance

• 2026-09-05-session-handover — 🔴 north camera 192.168.30.139 is limited by MULTIPATH, not congestion — and channel tuning CANNOT fix it. It sits at −63 dBm / SNR 40 yet negotiates only 8.6–11 Mbit/s, while the south unit gets 114.7 Mbit/s at −47 dBm / SNR 57: good RSSI with a collapsed PHY rate is the signature of reflections in a metal-and-dense-concrete building. Real fixes = an AP closer with wired backhaul, a directional antenna to reject off-axis reflections, or moving either endpoint a metre or two to change the reflection geometry; more TX power and wider channels both make multipath WORSE. Context: camwall flashing was a watchdog re-roll loop driven by stalled camera streams; after the ch11→ch1 move the south camera recovered instantly but north needed ~/tapo-ctl/reboot-cams-onvif.py 192.168.30.139, after which all four cams returned to ~5 fps and recording resumed

• telep-mainframe — Frigate 0.17.2, ONNX YOLOv9-s on an RTX 3080, events-only recording, birdseye HDMI wall

• RTSP stream map — Tapo TC47 dual-lens: /cam1/ and /cam2/ are the same lens. Fixed = /stream1+/stream2, PTZ = /stream6+/stream7

• Camera VLAN (telep-cc) — isolated 192.168.30.0/24, hidden SSID, no internet, NTP DNAT’d back

• 2026-07-17-intruder-alarm — watches Frigate person/car events to escalate intruders and siren the TV while armed ⛔ RETIRED 2026-09-07 — unit and script removed from live paths, archived (not deleted) at /root/retired/2026-09-07-intruder-alarm/; restore = mv both back + daemon-reload. This note is now a historical design record. See 2026-09-07-alert-source-inventory §6.

• 2026-07-18-birdseye-nvenc-blue-glare — the camwall’s live panes play Frigate’s birdseye restream; NVENC encoder corruption there = “blue glare” on the TV wall (isolation method for the whole RTSP→mpv→TV pipeline)

• 2026-07-22-wifi-usage-strip — the camwall’s bottom strip (y 1880–2140, below TOP KÉPEK): live per-device WiFi rates drawn as an mpv osd-overlay, never over the camera image; uses osd-overlay rather than camwall-reload so playback is untouched

• 2026-07-23-frigate-viewer-alert — “who is looking at my cameras”: follows Frigate’s nginx access log via docker logs -f, alerts on every UI session by tailnet device name; internal automation (XFF -) excluded by construction

• 2026-07-28-camwall-4-substream-composite — second camera (ÉSZAK 192.168.30.139) added → 4 cameras; the TV wall moved OFF Frigate birdseye (which flip-flops between 2 and 4 cams) to an explicit 4-substream 2×2 composite in mpv (DÉL left, ÉSZAK right)

• Latency fix (2026-07-29) — 4 tiled single-input mpv instances — camwall latency fix (~10–13 s → ~1–2 s): the single-composite movie= lavfi filtergraph was buffering; rearchitected into 4 tiled single-input low-latency mpv instances (per-quadrant --geometry, --profile=low-latency --cache=no). ⚠️ two regressions: WiFi strip now only in the BL pane (cosmetic) + watchdog now covers only the IPC-socket-owning pane, not all 4 (security-relevant)

• 2026-07-28-frigate-notify-camera-exclude — the indoor-aimed telep_cam3 (ÉSZAK – Fix) excluded from frigate-notify alerts via frigate.cameras.exclude

• 2026-07-31-camwall-lag-24ghz-wifi-bufferbloat — ⭐ REAL root cause of camwall lag: saturated 2.4 GHz camera WiFi (98.9% busy → bufferbloat), NOT mpv/go2rtc. Ping to both cameras 350–420 ms/0% loss; self-congestion from 4 continuous HD record streams on one channel. Fix = wire Ethernet/PoE or 5 GHz; changing channel won’t help. Supersedes the player-tuning latency fix

• 2026-08-03-cam-stall-recovery-and-casino-alert — recovering a WEDGED go2rtc RTSP stream: the ÉSZAK/.139 dual-lens (telep_cam3/telep_cam4) dropped to ~0.7 fps after dnsmasq/firewall restarts dropped its weak WiFi mid-stream and stuck its Tapo RTSP session slots (stale sessions don’t close). A fresh ffprobe was fine and a Frigate restart did NOT recover it — the fix is a camera reboot over ONVIF (SystemReboot, port 2020; pytapo can’t — needs cloud creds, the nvr42vhy1 account is ONVIF/RTSP-only) + systemctl restart camwall.service (mpv holds the last frozen frame, no auto-reconnect). A reboot resets the isolated camera’s clock → daily ONVIF SetSystemDateAndTime cron. Scripts: /home/levander/tapo-ctl/{reboot-onvif,set-cam-time}.py

• 2026-08-04-telep-mainframe-airplay-receiver-uxplay — the box is now an AirPlay receiver (telep-tv, uxplay 1.71) that pauses the 4×substream camera grid while casting and restores it after. Enabled by splitting Xorg :0 out of camwall.service into a persistent camwall-x.service — camwall.service is now just a client of :0, so stopping the grid no longer kills X (uxplay needs :0 alive). Handoff driven by parsing real uxplay-1.71 stdout markers; an EXIT trap restores the grid on crash

• 2026-08-06-frigate-detect-record-jam-cpu-starvation — Frigate’s detect/record pipeline JAMS under CPU starvation and does NOT self-recover — a 20-min all-core stress test starved the detect+record processes; live tiles went black + latest.jpg froze. Tell (/api/stats): camera_fps normal ~5 but process_fps ~0.1, skipped_fps ≈ camera_fps, global detection_fps=0.0, + record.maintainer “Too many unprocessed recording segments” spam. Fix = sudo docker restart frigate. Distinct from the wedged-RTSP camera-side failure (2026-08-03-cam-stall-recovery-and-casino-alert, where a Frigate restart does NOT help). per-camera detect_fps=None is a display quirk — trust the GLOBAL detection_fps

• 2026-08-08-frigate-fps-watchdog — ⭐ per-camera FPS watchdog that auto-restarts Frigate on BOTH failure modes. /usr/local/bin/frigate-fps-watchdog.py + frigate-fps-watchdog.service (active, enabled, root, Restart=always) polls 127.0.0.1:5000/api/stats every 20s and trips on (a) frozen feed camera_fps<1.5 (normal ~5) or (b) detection jam process_fps<1.0 while camera_fps healthy → docker restart frigate. Fills the gap that the camwall watchdogs don’t watch per-camera fps. Anti-thrash: 3-poll debounce (~60s) + 120s post-restart cooldown + a circuit breaker (max 3 restarts/hr; 4th suppressed → Telegram-only, re-alert every 30min) so a genuinely-offline camera can’t loop. A breaker trip = go check for a WEDGED camera needing an ONVIF reboot (2026-08-03-cam-stall-recovery-and-casino-alert), which a restart won’t fix. Built after Frigate jammed twice on 2026-08-08 (detection jam then cam3 frozen at camera_fps=0.3)

• 2026-08-09-xiaomi-c302-tailnet-facetime-cam — ⭐ Xiaomi C302 (indoor pan-tilt, China-cloud) repurposed as a low-latency FaceTime/Zoom/Meet cam over the tailnet via a SECOND go2rtc instance (go2rtc-cam) + WebRTC — deliberately NOT in Frigate. Native xiaomi:// P2P source (H265), client = OBS macOS Window Capture of a Chrome app-window rendering webrtc.html?src=c302 → OBS Virtual Camera → FaceTime (sub-second). Latency floor is TRANSPORT (WebRTC sub-sec ≫ MSE ~1s ≫ RTSP+transcode). No local PTZ (no miio endpoint; motor = Mi cloud only). Privacy: hardware relay + red BELSŐ KAMERA AKTÍV banner. Full gotcha gauntlet incl. 70016=wrong-region, H265-black-in-OBS, CEF-renders-WebRTC-gray

• 2026-08-10-top-kepek-drive-curation-print-pipeline — ⭐ “Top képek → Drive” curation/print pipeline: browse Frigate detections, crop at NATIVE res, push to Google Drive for printing — self-serve in the top4-web.py picker (no more feeding timestamps to an agent). Adds batched infinite scroll (was a hard bug — dumped all ~1500 events into the DOM at once, each a live Frigate fetch; now 48/batch via IntersectionObserver on #sentinel), jump-to-date/time (Ugrás/Most, flexible new Date() parse → /events?from=<t-1800>&to=<t+7200>; server /events now takes optional from/to unix), and save-to-Drive (POST /save-full-to-drive?id= full bbox=0&quality=100 + crop-modal POST /save-crop-to-drive?id= at native res, NOT the 880×495 camwall letterbox → gdrive:top_képek/). Bulk: ~/top-kep-drive.py "<ts>" (all person in t-45s→t+75s, dedup >3s/cam → flat top_képek/archive/), ~/top-kep-id.py <eventid>. GOTCHAS: _0.95 filename score = confidence not a unique id (TIME is the key); Drive rate-limits HARD after bulk moves → run rclone reorgs detached; folder-by-id gdrive,root_folder_id=<id>:. Supersedes/extends the SESSION-HANDOVER top4 spec

• 2026-08-10-top-kep-community-voting-system — ⭐ LIVE: the tailnet-only Hungarian community “top kép” voting system that REPLACES the Google-Drive top-pics workflow. People vote favourite Frigate person snapshots per day (GET /nap) rolling up into weekly (/het) → monthly (/honap); voted pics saved LOCALLY (/srv/top-kep/img/YYYY-MM-DD/<event_id>.jpg, on first vote, bbox=0&quality=100) so they survive Frigate’s ~14-day purge. Built into ~/top4-web.py (a stdlib http.server app, NOT Flask; top4-web.service, :8090→:8443) + a stdlib-only ~/top_kep_store.py (SQLite /srv/top-kep/votes.db: pics/votes/reminder_state). Candidates = label=person events deduped to one per (camera, whole-second) keeping highest top_score. Identity = Tailscale-User-Login header (one vote/identity; POST /vote → 403 without it). Reminders via ~/top_kep_remind.py (--daily|--weekly|--monthly, token read at runtime from ~/nvr/frigate-notify/config.yml) on 3 Persistent=true timers (daily 20:00 watermark-guarded, Sun 19:30, daily 21:00 fires only when tomorrow=1st). Drive: weekly top4-export.timer DISABLED, new flow writes nothing to Drive (old picker’s Drive endpoints/helpers left intact, full rip-out deferred). Supersedes 2026-08-10-top-kepek-drive-curation-print-pipeline

• car filter, HD recording + HD upgrade — ⭐ top-kép build pass 2: mobile-first UI + tap-to-zoom lightbox, person/car filter, and REAL HD (2304×1296) pics. 🔴 The 720p ceiling was a RECORDING-layer bug: every camera both detected AND recorded the 720p SUB stream so no HD frame ever existed. Fixed via a Frigate reconfig — each telep_camN now detects on camN_sub (720p, role:detect) and RECORDS a new camN_main go2rtc stream (role:record) at 2304×1296 (lens map cam1 DÉL fix=.119/stream1, cam2 DÉL PTZ=.119/stream6, cam3 ÉSZAK fix=.139/stream1, cam4 ÉSZAK PTZ=.139/stream6; backup config.yml.bak-hqrec-1786445779; detect fps unaffected; storage ~3–4×). ⚠️ each Tapo now serves 4 concurrent RTSP pulls (2 lenses × sub+main) + ONVIF — watch camera-side connection limits. A lightbox “Magasabb minőség” button → /pic/<id>?hq=1 fetches the main-stream recording-snapshot (/api/<camera>/recordings/<start_time>/snapshot.jpg), caches to /srv/top-kep/hq/<id>.jpg, serves HD (fallback: recording-snapshot → event-clip ffmpeg frame → 720p; event_id charset-validated ^[0-9.]+-[A-Za-z0-9]+$); HD only for events AFTER the reconfig. Person/car filter via ?label=person|car (new label column, ALTER-TABLE migration). Verified live in a headless real browser at phone width

• 2026-08-04-telep-mainframe-tldraw-whiteboard-board-kiosk — the 4K HDMI TV gets a third mode: “board mode” — a chromium kiosk of the self-hosted tldraw whiteboard that toggles with the camera wall (tv board/tv cams; board-kiosk.service and camwall.service Conflicts= each other both directions, board-kiosk NOT enabled at boot). Reuses the camwall-x vs camwall split: stopping camwall doesn’t kill Xorg :0, so the kiosk attaches as a root client (DISPLAY=:0, no XAUTHORITY). ⚠️ matchbox (the fullscreening WM) is a child of camwall’s session → a kiosk that stops camwall must launch its OWN matchbox or chromium comes up as a floating half-window. Chromium flags: --no-sandbox (mandatory as root), --force-device-scale-factor=2 (4K legibility)

• 2026-08-11-camwall-freeze-and-cam-reboot-fixes — ⭐ camwall “stuck cameras” = MISSING per-pane freeze detection (CLOSES the open regression from 2026-07-28-camwall-4-substream-composite) + the camera-reboot tooling was silently dead. A quadrant froze on a static image with a stopped OSD clock for a long time while Frigate ingest stayed healthy (camera_fps ~5) → the Frigate FPS watchdog saw nothing (it watches ingest, not the display panes). ROOT CAUSE: camwall-mpv.sh runs 4 independent mpv panes but only cam2 had --input-ipc-server=/run/camwall.sock and camwall-watchdog.py polled that ONE socket → a stalled cam1/cam3/cam4 pane was undetectable + never re-rolled. FIX: per-pane sockets /run/camwall-cam{1,2,3,4}.sock, watchdog polls time-pos on ALL 4 with per-socket counters, re-rolls whole wall if ANY stalls (verified: pausing cam3 → re-roll ~33s later). Catches stalled-stream freezes, NOT repeated-frozen-frame-at-full-fps. ALSO: pytapo reboot-cam.py (hardcoded nvr42vhy1) now fails Invalid authentication data on both cams (Tapo KLAP breakage) → new ~/tapo-ctl/reboot-cams-onvif.py reboots via ONVIF SystemReboot() port 2020 with FRIGATE creds, waits for ping-back, then re-syncs the OSD clock via set-cam-time.py (RTC-less Tapos lose their clock on reboot; closes the ~40-min gap before the hourly cam-timesync.timer would)

• 2026-08-11-camwall-hdmi-mode-fallback-power-recovery — ⭐ camwall black/dead after a power outage = Xorg fell back to 640x480 on the HDMI output (no mode set), so the four 1920x1080 mpv panes couldn’t map across the 3840x2160 screen — the HOST never rebooted (on the UPS, uptime 3d17h); the power blip only restarted services. Neither guard caught it: the watchdog only watched HDMI connect/disconnect (stayed “connected”, wrong res) and it SKIPS null time-pos so the black panes weren’t a “stall”; every re-roll came back on the same 640x480 screen. TELL: xrandr = Screen 0: current 640 x 480 + HDMI-1 connected no geometry, while go2rtc/GPU were healthy → display-mode, not stream/GPU. RECOVERY: DISPLAY=:0 xrandr --output HDMI-1 --primary --mode 3840x2160 --pos 0x0 --fb 3840x2160 + systemctl restart camwall. DURABLE FIX (verified by fault injection): camwall-mpv.sh self-heals the mode on every start; camwall-watchdog.py now parses Screen 0: current WxH and re-rolls on resolution fallback (auto-restored to 4K in ~22s, POLL=10s). Note: host has no ffprobe; host ffmpeg=/usr/bin/ffmpeg (not the Frigate container’s); x11grab to screenshot the wall

• 2026-08-12-airplay-mdns-fix-printer-migration — ⭐ the long-parked “AirPlay broken” item RESOLVED: wired→WiFi mDNS discovery of telep-tv was blocked because telep-router’s telep1 BSSes had ap_isolate=1 in the running hostapd conf (uci/ubus said isolate:false — the values disagreed). Root cause = OpenWrt’s _wireless_set_brsnoop_isolation auto-injecting isolate 1 at config-gen time; the parked uci isolate=0 recipe does NOT clear it. Fixed by patching out line 309 of /lib/netifd/netifd-wireless.sh (telep1 de-isolated, telepcc/cams stays isolated). 🔴 reapply after any sysupgrade. Complements the avahi-side AirPlay fix in 2026-08-04-telep-mainframe-airplay-receiver-uxplay

• 2026-08-15-camera-alerts-dead-tailscale-magicdns-docker-dns — 🔴 person/car Telegram alerts DEAD ~17 days because Tailscale MagicDNS broke Docker container external DNS; only DÉL .119 (telep_cam1/2) is meant to alert, ÉSZAK .139 (telep_cam3/4) is excluded on purpose. ⚠️ Xiaomi C302 192.168.30.168 is REMOVED/dead — it was the only cam ever on the separate go2rtc-cam path; the Tapos go direct RTSP into Frigate (corrected 2026-08-09-xiaomi-c302-tailnet-facetime-cam to status/outdated)

• 2026-08-15-camwall-dead-br-cams-bridge-down-after-wifi-reload — 🔴 all 4 cameras fps=0 + frozen camwall root-caused to the router, NOT the cameras: a wifi reload (radio1 channel change) left the wireless-only bridge br-cams state DOWN so the cams gateway/DHCP 192.168.30.1 vanished — cameras stayed WiFi-associated but were L3-unreachable. 🎣 the ONVIF reboot ConnectTimeout’d (L3-dead, not bad cams) — a misleading symptom. ⚠️ same fps=0 symptom as the 2026-08-03 wedged-RTSP failure but different layer — branch on ip addr show br-cams (DOWN/no inet = router-side, fix ifdown/ip link up/ifup cams; UP but pane wedged = camera-side, fix ONVIF reboot). Then systemctl restart camwall. Also fixed: the OliveTin ONVIF action ran system python3 (no onvif module) → must use ~/tapo-ctl/venv/bin/python

• 2026-08-22-frigate-recording-retention-config-not-jam — ✅ “recording broken” was RETENTION CONFIG, not a jam (CORRECTS the wifi-reload/go2rtc-wedge hypothesis in telep-mainframe-handover). Resolved record config had continuous.days=0 + motion.days=0; only alerts/detections retention (keeps only segments overlapping a tracked person/car), so during quiet periods Frigate DISCARDED ffmpeg’s /tmp/cache segments instead of storing them → the “68 MB/day collapse” was just low activity. 🩺 diagnostic order: /api/stats camera_fps (detect) → docker exec frigate ls -la /tmp/cache (ffmpeg producing?) → /api/config record block; cache-has-segments + recordings/-empty + both days==0 ⇒ retention, not jam. FIX: /home/levander/nvr/frigate/config.yml (mounted /config, NOT /srv/frigate/config; backup config.yml.bak-20260822-catmotion) — added record.motion.days: 10 + cat to objects.track (coco-80). Cams = 2 Tapo dual-lens (.119=cam1/2, .139=cam3/4), detect camN_sub/record camN_main via go2rtc rtsp://127.0.0.1:8554. Loose end: 0-byte go2rtc_homekit.yml = source of stray c302/c302_h264 streams

• 2026-08-26-frigate-notify-cat-alerts — follow-up to the cat tracking added above: cat was tracked in Frigate but not in frigate-notify’s alerts.labels.allow, so cats never pinged Telegram until cat was added to that list on 2026-08-26 (backup config.yml.bak-add-cat + docker restart frigate-notify)

• 2026-08-31-cams-vlan-device-claim-failed-wifi-reload — 🔴 SECOND wifi reload cams outage (distinct from 08-15 carrier-down): the cams L3 interface stuck at ifstatus cams "up": false + errors: ["DEVICE_CLAIM_FAILED"] — br-cams device present but no 192.168.30.1. All 4 Frigate feeds dead + camwall.service hung in activating (start-pre) (its ExecStartPre gates on Frigate /api/version). 🩺 branch: br-cams DOWN qdisc noop=08-15 (ifup cams cycle), DEVICE_CLAIM_FAILED=this (ifup cams CANNOT clear it — only /etc/init.d/network restart). cams-guard’s per-minute ifup cams couldn’t self-heal; being hardened to escalate to network restart after 3 failures

• 2026-09-02-frigate-resource-tuning — ⭐ Frigate 0.17.2 resource tuning: birdseye.mode: continuous at width: 3840 was compositing + SOFTWARE-mpeg1-encoding a 4.1 MP canvas 24/7 with zero viewers (default is 1280×720) — the single largest ffmpeg CPU consumer in the stack; set to mode: objects + width: 1920 (half the pixels, verified the 3840x1080 encoder is replaced by 1920x1080). ⚠️ GOTCHA: with restream: true the birdseye encoder keeps running even under mode: objects because go2rtc holds a consumer attached — needs restream: false to truly idle (OPEN ITEM; the TV wall already moved off birdseye in 2026-07-28-camwall-4-substream-composite, so check consumers first). Also disabled a face_recognition that had burned 678 MiB VRAM since July while structurally unable to recognise anyone — /media/frigate/clips/faces/ EMPTY, GET /api/faces → {}, 0 of the last 500 events had a sub_label. Net −344 MiB VRAM / −723 MiB container RAM; 🔴 CPU INCONCLUSIVE, do not claim a win (the 149.81% “before” was a post-boot ps artifact). Untouched levers: /tmp/cache is an anonymous docker volume, not tmpfs (tmpfs: - /tmp/cache:size=1g), and record.motion.days: 10 keeps /srv/frigate/recordings at 496 GB — ⚠️ but that setting is load-bearing per 2026-08-22-frigate-recording-retention-config-not-jam

• 2026-09-02-lg-tv-network-control-presence — 📺 the camwall TV identified: LG 43UP75003LF, webOS 6, fw p20.03.53.45, 192.168.1.171 / LGwebOSTV.lan / ac:5a:f0:8b:48:da, controllable over the SSAP WebSocket (3000 plain → HTTP 101, 3001 TLS with a self-signed CN=LGE TV SSG cert → SSL verify must be OFF). ⚠️ webOS Auto Power Off kills the camwall every 4 hours — the set powers down after 4 h with no button pressed (5-min countdown first) and nobody ever touches the remote in front of a camera wall; disable at All Settings → General → Timers → Auto Power Off (+ Eco Mode screen-off). 🚧 STILL NOT DONE as of 2026-09-02 — this is the last open item on the whole TV build. Another way this TV goes black: 2026-08-11-camwall-hdmi-mode-fallback-power-recovery

• 2026-09-02-lg-tv-network-control-presence — 🩺 ⭐ “Is the camwall actually rendering?” — the standard check. Grab the live framebuffer with DISPLAY=:0 /usr/bin/ffmpeg -f x11grab and run per-quadrant signalstats YAVG. Healthy reference measured 2026-09-02: 4 live panes, YAVG 115–128 per quadrant, correct 3840x2160 mode. A black or frozen pane collapses its YAVG immediately. This is what proved the source was fine while the TV was blank, and pointed the investigation at the TV’s input rather than the GPU.

• 2026-09-02-lg-tv-network-control-presence — 📏 Diagnostic rule for an LG showing “no signal” while the source looks healthy: check the TV’s per-input connected flag FIRST, before touching modes, cables or Deep Colour settings. WebOsTvState.inputs reports it directly (com.webos.app.hdmi1 id=HDMI_1 connected=False / com.webos.app.hdmi2 id=HDMI_2 connected=True) — one SSAP status call settles which socket the source is really in.

• 2026-09-07-session-handover — camera-alert findings: frigate-notify’s telep_cam3/telep_cam4 exclusion is DELIBERATE and user-confirmed — do NOT “fix” it, and the documented consequence is that a genuine person on cam3 at 02:27 and 02:37 (79%, snapshot visually verified — a real person by the railing at night) produced NO alert. Real detections do carry images (4 of 5 frigate posts had image=True, 85,900 / 77,374 byte JPEGs); the only image-less one was a synthetic test alert wrongly injected into the live room. 🔴 CRITICAL scoping correction: cameras.exclude is a FRIGATE-NOTIFY setting enforced UPSTREAM of the Matrix relay, and the relay has NO camera filtering of its own — it posts whatever hits its webhook, so any direct POST to 127.0.0.2:8118 (testing, replay, a future integration) reaches the LIVE alert room regardless of camera. That is how a genuine cam3 detection was replayed for the 2026-09-07 image-attachment verification without contradicting the exclusion, and it is exactly how the “Telep Cam9” test alert got in. If a camera must never appear in a room, the relay needs its own filter — the exclude alone is not a guarantee. ⚠️ Inferred from the fixing agent’s report + the architecture, not verified on the box (one-line check: grep the relay for camera/exclude filtering)

Chat & Messaging

• 2026-09-07-session-handover — ⭐⭐ self-hosted Matrix homeserver: Continuwuity v26.8.1 (forgejo.ellis.link/continuwuation/continuwuity:v26.8.1, digest sha256:fdf3cd0f…, pinned). Compose /home/levander/matrix/, config continuwuity.toml, RocksDB data /home/levander/matrix/data; backend 127.0.0.2:8008 (127.0.0.1 refused); sidecar node chat (tag:telep) at https://chat.taild4189d.ts.net, tailnet-only, never Funnel. Federation OFF + open registration OFF, both proven with live 403s. Accounts @andras (admin), @phone, @mfalusi, @puliki, @alerts (bot) + built-in @conduit; passwords in /root/matrix-credentials.txt (0600). 🔴 server_name = chat.taild4189d.ts.net is PERMANENT — baked into the DB; changing it means wiping everything. 📦 max_request_size = 536870912 (512 MiB, 25× default) for uncompressed phone photos — proven with a real 400 MiB upload → HTTP 200. 🔴 The Admin Room must stay UNENCRYPTED — @conduit must read !admin in the clear and Matrix room encryption is one-way, so enabling it would permanently break user management. 📱 QR sign-in (MSC4108) is NOT implemented in v26.8.1 (rendezvous endpoints 404, no code in the tree) — but OAuth browser sign-in DOES work and both Element Desktop and Element X on iOS hand off to a browser (/_continuwuity/oauth2/*, grants authorization_code/refresh_token/device_code); MSC3882 get_login_token: true also present. 👤 User management via !admin users create-user|reset-password|list-users|make-user-admin|deactivate|suspend|logout in the Admins room — ⚠️ the reply prints the password in PLAINTEXT and it persists in the database; redact it. 🚩 Do not install conduwuit — it is ARCHIVED; and the fork rename is skin-deep (binary /sbin/conduwuit, data /var/lib/conduwuit, logs conduwuit_*, bot @conduit:…) 🔴 Gotcha (2026-09-07): Continuwuity’s /rooms/{room_id}/event/{event_id} IGNORES the room segment and resolves purely by event ID ⇒ a per-event “is this in room X?” check answers YES for anything that exists, in both directions. Trust the returned event’s own room_id field, page /rooms/{room}/messages?dir=b to audit a room, or count sent event lines in the relay journal. See 2026-09-07-alert-source-inventory.

Collaboration & Whiteboarding

• 2026-08-04-telep-mainframe-tldraw-whiteboard-board-kiosk — self-hosted real-time collaborative tldraw whiteboard (“Miro-type” board) on telep-mainframe with PDF annotation. /home/levander/tldraw-board: ONE Bun process serves the built Vite/React tldraw client AND the WebSocket sync + asset endpoints on port 8790 (tldraw-board.service, runs as levander, enabled). Tailnet http://telep-mainframe.taild4189d.ts.net:8447/, LAN http://192.168.1.123:8790/; rooms /r/<roomId> (default main). tldraw 5.2.5; useSync client + one TLSocketRoom per room over Bun WS; JSON snapshots at data/rooms/<roomId>.json; assets via PUT/GET /uploads/:id (not data-URLs) in data/assets/; PDF import = pdf.js page→PNG→locked stacked image shapes. ⭐ GOTCHA: tldraw 5.2.5 renders a BLANK canvas over HTTPS without a paid license (LicenseManager.isDevelopment needs non-https OR loopback OR NODE_ENV != production) → served over plain HTTP (tailscale serve --http, still WireGuard-encrypted over the tailnet); leaves a small “get a license” watermark; proper HTTPS needs a purchased licenseKey. Also a TV kiosk “board mode” — see the Cameras & Surveillance and Gotchas topics

Containers & Docker

• nvr) — frigate (stable-tensorrt image, but onnx detector) + frigate-notify; mount the whole /config dir or the event DB is wiped on recreate

• filestash — machines/filestash bound to 127.0.0.1:8334, state in filestash_data volume, Windows drive passed in at /mnt/data:ro

• The Windows drive — a bind-mount captures the host dir at container start: mounting /mnt/win after filebrowser started leaves /srv empty inside the container → mount first, then docker restart filebrowser

• 2026-08-06-frigate-detect-record-jam-cpu-starvation — sudo docker restart frigate is the fix for a CPU-starvation-jammed detect/record pipeline — after a sustained all-core load the frigate container’s frame queues + record-segment cache jam and stay jammed (container back to ~10% CPU) with global detection_fps=0.0; the restart clears it instantly (→ detection_fps 113). Image ghcr.io/blakeblackshear/frigate:stable-tensorrt, ports 5000/8971 on 127.0.0.1

• 2026-08-05-power-root-cause-nvme-damage-ups-kb-handover — ruview container (RuView WiFi-CSI through-wall sensing, image ruvnet/wifi-densepose ~43MB, simulated mode) on the mainframe at :8448/ui/index.html, then STOPPED and blocked on hardware — ⚠️ superseded 2026-08-11, see the next bullet

• 2026-08-11-ruview-esp32-csi-real-hardware — ruview RECREATED and running real ESP32-S3 CSI. 🔴 The blocker was that UDP 5005 was only EXPOSEd and never published — nothing could reach the CSI ingest port; ports can’t be added to a running container, hence the recreate (safe: it had NO volumes, only a regenerable 16K /app/data/session-secret). Now -p 5005:5005/udp, CSI_SOURCE=esp32 (was simulated; esp32 fails LOUD where auto might not — issue #937 removed the silent synthetic fallback), persistent -v /home/levander/ruview-data:/app/data, --model /app/data/models/model.rvf. ⚠️ The entrypoint PREPENDS the binary + its defaults (--source $CSI_SOURCE --tick-ms 100 --ui-path /app/ui --http-port 3000 --ws-port 3001 --bind-addr 0.0.0.0) whenever arg1 starts with -, so appending flags is safe and preserves ports/bind. ⚠️ SENSING_ALLOWED_HOSTS must list every hostname — DNS-rebinding Host-header validation returns HTTP 421 otherwise. ⚠️ the container creates /home/levander/ruview-data/models root-owned → chown before downloading or curl silently writes nothing (http=200 size=0)

• 2026-08-08-frigate-fps-watchdog — frigate-fps-watchdog.service auto-docker restart frigate when /api/stats shows a frozen feed (camera_fps<1.5) or a detection jam (process_fps<1.0), with a 3-restart/hour circuit breaker to protect against thrash on a genuinely-offline camera

• 2026-08-08-ups-shed-restore-hardening — kb-qdrant now runs --restart unless-stopped so docker brings it back on boot (previously it was explicitly docker stopped during a UPS shed and had no boot recovery → KB crash-looped 2360× against a missing Qdrant); belt-and-braces with the boot-time power-restore-reconcile.service

• 2026-08-09-xiaomi-c302-tailnet-facetime-cam — go2rtc-cam (alexxit/go2rtc, network_mode: host, restart: unless-stopped) at /home/levander/go2rtc-cam/ — a SECOND go2rtc instance alongside Frigate’s, on api :1984 / rtsp :8555 / webrtc :8556 because Frigate’s own go2rtc already holds 127.0.0.1:8554. 🔴 host-network binds all docker/incus bridges → WebRTC needs explicit candidates: (192.168.1.123:8556, 100.115.209.87:8556) or ICE advertises junk bridge IPs. go2rtc streams are LAZY (0 CPU until consumed) so a transcode fallback costs nothing idle

• 2026-08-15-admin-portal-passkey-olivetin — the admin-portal stack at /home/levander/admin-portal/ (Caddy host + oauth2-proxy + OliveTin + Pocket-ID containers). 🔴 oauth2-proxy OAUTH2_PROXY_UPSTREAMS must be the compose service name http://olivetin:1337 — 127.0.0.1:1337 is the oauth2-proxy container’s own loopback → 502. 🔴 jamesread/olivetin runs as USER olivetin (HOME /home/olivetin) but ssh key mounted at /root/.ssh → Host key verification failed/Permission denied until you add user: root to the service AND drop :ro on the ssh mount (so accept-new host keys persist)

• 2026-08-17-home-dashboard — the homepage container at /home/levander/homepage/ (gethomepage/homepage, Next.js): network_mode: host + HOSTNAME=127.0.0.1 PORT=3010 (3000 taken by ruview). 🔴 host networking is REQUIRED so its siteMonitor: health checks can reach 127.0.0.1:PORT backends + LAN/VLAN IPs directly. 🐛 config/services.yaml is YAML — a description: value with a colon-space (Windows C: (RO)) breaks parsing → QUOTE all descriptions. Must set HOMEPAGE_ALLOWED_HOSTS (rejects unknown Host headers)

• 2026-08-15-camera-alerts-dead-tailscale-magicdns-docker-dns — 🔴 any Docker container needing EXTERNAL DNS on telep-mainframe silently fails while the host /etc/resolv.conf is Tailscale MagicDNS (100.100.100.100) — the embedded resolver (127.0.0.11) forwards to it and MagicDNS misbehaves for containers (dial tcp: lookup … server misbehaving, docker exec … curl=http 000 while the host curl works). frigate-notify was the one that bit us (needs api.telegram.org); most other containers only need internal service-name resolution so they were fine. FIX = per-service dns: [192.168.1.1, 1.1.1.1] in ~/nvr/docker-compose.yml + --force-recreate (internal frigate name still resolves). Daemon-wide "dns":[...] in /etc/docker/daemon.json DEFERRED (restarts all containers)

• 2026-08-31-nextcloud-drive-code-server-runbook — code-server binds: exports:/home/coder/cad rw (no :ro, deliberately), /mnt/win/Users:/home/coder/windows ro, and Drive-home per user …/data/<user>/files:/home/coder/drive/<user> ro — ⚠ one bind per account, not one tree (a new Nextcloud user is invisible in the IDE until its line is added; quote paths containing @). Read-only reuses existing 0644 bits instead of loosening Nextcloud’s data-dir permissions; reversal is one compose line per account

• 2026-08-31-openchatcut-chatcut-deploy — OpenChatCut container openchatcut:local, hand-written Dockerfile (no upstream): FROM node:24-bookworm, apt git ffmpeg chromium fonts-liberation, ENV to use system chromium for Remotion (PUPPETEER_SKIP_DOWNLOAD=1, REMOTION_CHROME_EXECUTABLE=/usr/bin/chromium), ONNXRUNTIME_NODE_INSTALL=skip, runs the Vite dev server (no upstream prod server). Run (updated 2026-08-31 → host networking, was -p bridge): --restart unless-stopped --cpus=8 --memory=16g --network host --env-file .env.local -v …/data:/root/.openchatcut … npm run dev -- --host 127.0.0.2 — loopback-only bind, required so the app’s loopback-socket trust check passes (bridge showed the Docker gateway 172.17.0.1). Also bakes in patch-trust.mjs. 🔴 buildkit couldn’t resolve DNS (no MagicDNS 100.100.100.100 inherited) → build --network=host

• 2026-09-01-orcaslicer-tailnet-deploy — OrcaSlicer container lscr.io/linuxserver/orcaslicer:latest (KasmVNC desktop-in-browser): -p 127.0.0.2:8570:3000 --shm-size=1gb -e PUID/PGID=1000 -e TZ=Europe/Budapest -v /home/levander/orcaslicer/config:/config. KasmVNC serves HTTP on container 3000 (published to 127.0.0.2:8570); TLS added by tailscale serve (container’s 3001 self-signed HTTPS unused). --shm-size=1gb required for the browser/VNC session

• 2026-09-02-freecad-mcp-gui-dispatch-timeout-modal-dialog — freecad container internals on telep-mainframe: addon settings live in two files (/config/.FreeCAD/ and /config/.local/share/FreeCAD/freecad_mcp_settings.json), both remote_enabled:true / auto_start_rpc:true / allowed_ips="127.0.0.1,10.0.0.0/8,172.16.0.0/12,192.168.0.0/16". Network freecad_default = 172.22.0.0/16 (freecad .3, freecad-mcp proxy .5) — covered by the 172.16.0.0/12 entry. 🔴 remote_enabled:true is REQUIRED, not optional: with false the RPC binds the container’s 127.0.0.1 while docker-proxy forwards to the container’s eth0, so the published port can never reach it. The desktop app is started by s6 (/defaults/autostart = /opt/freecad/AppRun via startwm_wayland.sh) — docker exec cannot replace it. /exports = host /home/levander/freecad/exports

• 2026-09-05-freecad-mcp-502-dead-container-and-qt-event-loop-wedge — 🔴 ⭐⭐ restart: unless-stopped does NOT guarantee a container is running. freecad-mcp sat Exited (128) for three days at RestartCount: 0 with Error="failed to create task for container: failed to create shim task: ttrpc: closed", OOMKilled=false — the restart policy supervises a task, and the containerd shim failed before any task was created, so Docker never even attempted a restart. It 502’d the whole FreeCAD MCP endpoint. docker ps hides this — you need docker ps -a; docker start <name> is the fix. Never treat “it has a restart policy” as evidence it is up, and run docker ps -a early whenever a tailnet-served service 502s.

• 2026-09-01-bambuddy-slicer-api-pipelines-mcp — three new containers around bambuddy on telep-mainframe: (1) bambu-studio-api slicer sidecar (ghcr.io/maziggy/bambu-studio-api) — a bridge container published 127.0.0.2:3001:3000 (⚠ 127.0.0.2 because an ESP32 owns 127.0.0.1:3001; bambuddy reserves 3000/3002 for its virtual printer, sidecars use 3001 bambu / 3003 orca); (2) bambuddy-mcp-bridge (--network host, python:3.12-slim, mcp-proxy --host 127.0.0.2 --port 8091 -- bambuddy-mcp) — ⚠ pin mcp==1.29.1 in both the pip env and the uv-tool env (SDK 2.x removed request_ctx → breaks mcp-proxy, and the Server.list_tools decorator → breaks bambuddy-mcp). Secrets (bambuddy API key) in a chmod-600 .env

• 2026-09-02-frigate-resource-tuning — ⚠️ Frigate’s /tmp/cache is an ANONYMOUS DOCKER VOLUME, not tmpfs — it holds the 10-second pre-mux segments from all 4 cameras and is designed to be RAM-backed, but is currently churning /var/lib/docker on disk; fix with tmpfs: - /tmp/cache:size=1g on the compose service. Also 🔴 docker logs <container> replays FULL history — errors dated weeks earlier resurface after a restart and read as new; use --since. Measure containers with docker stats --no-stream sampled repeatedly, never ps %CPU

• 2026-09-02-lg-tv-network-control-presence — 🔴 ⭐ docker restart homepage does NOT apply config changes. A prerendered /app/.next/server/pages/en.html survives the restart, so services.yaml edits appear to be ignored. What actually applies them is curl http://127.0.0.2:3010/api/revalidate. This very likely explains past “I edited the dashboard and nothing happened” episodes. Also: 2xl:h-48 in a Homepage widget config is a dead class Tailwind never compiled — arbitrary responsive variants added via YAML aren’t guaranteed to exist in the built CSS.

CV & Diagram Understanding

• 2026-07-29-kb-wiring-extraction-v1-complete — v1 + v1.1 (scaled) hybrid classical-CV + vision wiring-diagram extractor (/home/levander/wiring-extract/): turns invisible scanned wiring diagrams into a searchable text sidecar + a best-effort, confidence-flagged netlist. cvtrace.py = Otsu→despeckle→TEXT mask (surya)→SYMBOL mask (vision boxes)→skeletonize→graph→dot-vs-crossing→union-find nets; legend.py decodes Suzuki wire colors. Naive skeletonize on 1632px scans = 161 false junctions from text+symbols; masking first → 37 (-77%). Division of labor: vision locates+reads, CV traces connectivity, vision fuses semantics. Scaled to 5door-supplement 55/55 (62+ pages total, ~377 Qdrant points): clean single-system circuits (037–043) trace high/med confidence, bundled/operation diagrams (060 body-electrical) conservative, routing/flowchart/block/mechanical labels-only — no fabrication (044 crank-angle is only a flowchart in the scan → labels-only)
• v1.1 (2026-07-29) — scaled to real content + graphical render layer — on-page render layer: hooks.py on_page_markdown appends a per-page “Kinyert huzalozási adatok (AI)” HTML connections table (Honnan|Huzalszín|Hová|Bizt.) + components/systems + AI-provenance from the .wiring.json (non-destructive), plus a NEW Mermaid graphical circuit renderer (“Áramköri gráf (AI)” graph LR: nodes=components, edges=decoded wire colors, LOW-confidence dashed, subgraph per subsystem, ~30-edge cap) via pymdownx.superfences + a mermaid fence in mkdocs.yml. ⚠️ markup-verified, NOT live-browser-confirmed yet. Folder/manual-root 404 fixed via gen_index.py landing-index.md generation (gen_index.py.bak-404)
• 2026-07-29-kb-wiring-extraction-design — the design: searchable labels (solid) + best-effort confidence-flagged connections; hybrid CV path-tracing for connectivity + vision for semantics; scan stays source of truth; per-item confidence + human review gate; scan resolution is the accuracy ceiling
• 2026-07-29-kb-wiring-extraction-plan — task-by-task plan with a Task-0 CV-tracing GO/NO-GO spike gate; CV under ocr/venv+scikit-image, vision subagent-driven, kb-vectors chunker.py patch, non-destructive
• cluzter — the human-in-the-loop successor to netlist extraction, in a different project. A browser wiring-diagram editor that loads a scanned page as a calibrated underlay (two-point calibration → world-mm-per-paper-mm) and traces over it by hand instead of importing an extracted netlist. Scope note: the retirement recorded in kb-agent-api kills the AI netlists, not the scans — the OCR’d diagram pages are cluzter’s authoritative underlay source. It also inherits this project’s finding that scan resolution is the ceiling (stripe letters B/R vs B/Bl fail ~40% at 1632px) — cluzter-wiring-gotchas
• 2026-07-30-kb-wiring-gallery-pivot-complete — the netlist extractor was RETIRED (too inaccurate for agents) and replaced with a consolidated scan gallery at /wiring-gallery/: 480 curated genuine wiring-diagram images (5 dedicated wiring manuals auto-included + vision-verified the ambiguous ones, dropping 147 photos/icons/exploded-views), 320px thumbs grouped vehicle→manual, each linking to the full source scan. Retirement is reversible (hook off, 65 .wires.txt moved to /home/levander/wiring-retired/, charts disabled, 2 manuals reindexed → marker count 0); the scans stay (accurate by construction) as cluzter’s underlay source. mkdocs-glightbox click-to-zoom fixed diagrams looking cut off in the ~60em content column
• 2026-08-06-kb-wiring-gallery-native-res-fix-and-astro-caching — the low-res gallery images were a THUMBNAILING artifact, not OCR — marker (marker-pdf/Surya) extracted the figure crops at native res, then gen_thumbs.py downscaled them to WIDTH=320, quality=72 and the gallery served the thumbnails (480 imgs, ~12–20KB each). Fix = copy the marker-native originals verbatim (image_abs in gallery_items.json) into the gallery dirs → 2.5× sharper (320px→822px). CEILING: source scans are only 150 dpi so ~822px is the real limit; marker *_meta.json has NO per-figure bboxes so re-cropping needs a full marker re-run, not a 300dpi upscale
• 2026-08-06-kb-marker-chunker-bug-and-ocr-bakeoff — ⭐ the KB’s “orphaned procedure” pages (generic H1 Removal and installation, no component name) are a marker per-page SPLITTER bug, NOT OCR — the true header survives in marker’s RAW ~/ocr/marker_out/<manual>/<manual>.md (e.g. # Idle Air Control (IAC) valve), recoverable deterministically without re-OCR. OCR bake-off: GOT-OCR2 (stepfun-ai/GOT-OCR-2.0-hf) recovers dropped headers + rebuilds mangled spec tables BUT regresses clean prose, emits LaTeX not GFM, useless on line-art → DECISION: patch, not full re-OCR; micro-re-OCR only for destroyed spec-table/code pages. Gotchas: JustVugg/colibri is an MoE-LLM engine NOT OCR; DeepSeek-OCR won’t install (flash_attn / transformers 4.4x vs 5.x)
• 2026-08-06-kb-embedding-classifiers-system-tags-and-image-types — ⭐ SigLIP zero-shot image-type classifier (google/siglip-so400m-patch14-384, ~2GB VRAM, ~26ms/img) types the wiring-gallery images into 8 classes → public/wiring-gallery/image_types.json → gallery filter pills (gallery renamed “Wiring & Schematics”, URL kept). KEY: the corpus is almost all scanned B&W LINE ART — of 480 imgs: line-drawing 222 / wiring 149 / flowchart 55 / connector 33 / other 17 / spec-table 3 / photo 1 / exploded 0. SigLIP softmax stays LOW on look-alike B&W classes → gate ≈0.35/0.10 (0.55 too strict)

DNS Monitoring & Watchlists

• telep-mainframe-handover — current-state record (no dedicated note yet): dnsmon — FastAPI+SQLite dnsmon.service (127.0.0.1:8099, own tailnet node dnsmon.taild4189d.ts.net) that ssh-tails the router’s dnsmasq query log, keeps 48 h (flagged rows forever), resolves device names via mDNS+DHCP+MAC-OUI, and drives a live monitor / /devices / /flagged / /watchlists / /context gambling-watchlist panel. Plus the router /etc/casino-alert.sh procd alerter now emitting Telegram alerts with device name + surrounding-domain context. Open: unify the two watchlists; watch.levandor.io passkey for off-tailnet; Private-Relay evasion (handle per-device, not LAN-wide).
• 2026-08-03-cam-stall-recovery-and-casino-alert — the original router casino/gambling DNS alerter (/etc/casino-alert.sh tailing /tmp/dnsmasq-queries.log).

Documentation & Knowledge Tooling

• 2026-07-29-kb-wiring-extraction-v1-complete — wiring diagrams become searchable + rendered: the chunker.py patch (kb-vectors) merges a page’s <page>.wires.txt sidecar onto that page’s chunks (same page_url, never its own page) so extracted labels/components/decoded colors get vectorized alongside the real page; scaled to 5door-supplement 55/55 (~377 Qdrant points), and each page now renders an on-page connections table + a Mermaid circuit graph via the on_page_markdown hook. Extends 2026-07-24-kb-vectorize-complete

• 2026-07-24-knowledgebase — tailnet-only manual library + OCR ingest portal; mkdocs-material static site of OCR’d PDF manuals (folder-tree browse, full-text search, dark mode, lazy images) + a Flask /upload portal that OCRs a PDF into the library

• How it works (the reusable parts) — chunked loading = split each manual by its # h1 headings into many small per-section pages (a 795 KB markdown would be one huge page) + a mkdocs on_post_page hook injecting loading="lazy" into every <img>; folder tree → mkdocs auto-nav; mkdocs builds to a temp dir then os.rename-swaps site/ (atomic, a broken source never takes down the live site)

• Library expansion (2026-07-24) — the expanded manual library (suzuki-vitara / chevy-tracker / suzuki-sidekick); marker → seed_import.py → build, manuals queued as CHAINED nohup scripts (.done sentinel) to serialize the single GPU; md5-dedup downloads BEFORE OCR

• 2026-07-24-kickfix-ingest — ingesting a 112 MB offline archive of the dead kick-fix.com repair site (HTML→markdown, 25 topics / 127 articles / 741 images) into docs/kick-fix/; reusable lessons for old table-layout HTML

• 2026-07-24-kb-vectorize-complete — semantic search over the whole library: 2839 chunks embedded (bge-large-en-v1.5, CPU torch) into Qdrant (kb-qdrant, collection manuals); cross-source search blends official manuals + kick-fix; idempotent per-manual (delete-by-filter + uuid5 ids)

• 2026-07-24-kb-semantics-phase2-complete — phase 2 (semantic dedup): fully-local SECTION clustering over the phase-1 vectors + a read-only “Szemantika” explorer, NO LLM. Mean-pool 3252 chunk vectors → ~1248 section vectors → HDBSCAN → 194 clusters / 24 cross-SOURCE / 631 noise; clusters.json regenerated on demand (kbclust.py build), explorer /semantics folded into the existing knowledgebase app (no new service); ranked by cross-source span; labels pluggable behind make_label for a future claude -p relabel; clusters.json = phase 3’s retrieval bundle

• 2026-07-25-kb-consolidated-gen-phase3-complete — phase 3 (consolidated generation) — DONE, full system complete: on-demand consolidated repair-article generation from a cross-source cluster via claude -p --model opus, in a draft → human review → publish workflow. “Generál” button on /semantics → serialized worker gathers member .md + images → never-invent prompt → draft (drafts/<slug>/) + advisory speccheck → review (AI-provenance banner + Közzététel/Elvetés) → publish to docs/consolidated/<slug>/. Code in knowledgebase/ (genprompt/speccheck/gen/kbgen), NO new service; cite-everything/never-invent, verbatim numbers, conflicts both-ways. Proven live (engine-cooling published, parking-brake draft pending)

• 2026-07-29-kb-semantic-search-frontend-complete — KB’s new front door: server-side SEMANTIC search replacing mkdocs’ slow client-side lunr. Runs in-process in the existing knowledgebase Flask app (CPU-torch + sentence-transformers + qdrant-client installed INTO the app venv, ~1.63 GB RSS; sys.path.insert to kb-vectors, warmed bge-large lazy singleton). GET /api/search → JSON hits (dedupe-by-page + snippet/highlight [XSS-safe] + classify + vehicle label) via 2026-07-24-kb-vectorize-complete’s search.py over Qdrant manuals; /+/search search-first UI (type badges, vehicle filter, keyboard nav, i18n), /browse keeps the mkdocs index. Warm ~54 ms; real semantic wins verified; graceful degradation (Qdrant down → 503, site stays up); 141 tests; retrieval-only (RAG synthesis is phase 2). New searchui.py; backups app.py.bak-search-fe/app.py.bak-search-ui

• 2026-07-29-kb-semantic-search-frontend-design — the design: search-first entry point, reuse the existing Qdrant/bge-large backend, extend the existing Flask app (no new service), retrieval-only in v1, phase-2 levers (synthesis, hybrid BM25 fusion, retire lunr)

• 2026-07-29-kb-semantic-search-frontend-plan — task-by-task plan with a Task-0 in-process-import GO/NO-GO gate (install CPU-torch into the app venv or pivot to a localhost search service); searchui.py pure helpers, /api/search route + warmed singleton, search-first UI + /browse, i18n chrome

• scanned-pdf-to-markdown-marker — the reusable ~/ocr/venv marker OCR pipeline the knowledgebase ingest calls as a subprocess (no duplicate torch/model install)

• Since last handover (2026-07-28) — KB Hungarian/English toggle — MID-BRAINSTORM, paused: Approach A chosen (mkdocs-static-i18n + Material language selector for the static site + a shared lang cookie for the Flask pages; HU/EN over UI chrome + cluster labels + generated consolidated articles; source manuals stay English). No spec doc yet — resume via brainstorming → writing-plans

• 2026-07-30-kb-hungarian-translation-complete — HU translation of the manuals COMPLETE for the prioritized set: 820/820 pages (956 total .hu.md incl. consolidated), served read-only at /hu/<page_url> (mkdocs-static-i18n suffix mode, English fallback), NOT search-indexed (search stays English). Manuals: geo-tracker-repair 434, workshop-1988-1998 293, 5door-supplement 61, supplement-61a40 23, sidekick wiring-1996 9, all kick-fix/* + consolidated. Skipped (low value, addable later): parts-catalogue, body-measurements, wiring-diagrams booklet, owners-1995, kickfix-docs. Faithfulness contract keeps numbers/units/torque/part-numbers/DTC/wire-colors/URLs/image-refs VERBATIM; > AI-forditas... banner per page

• 2026-07-30-kb-wiring-gallery-pivot-complete — wiring netlist extraction RETIRED → /wiring-gallery/ scan library (480 curated genuine wiring-diagram scans, glightbox click-to-zoom); /api/search no longer returns netlist text (marker count 0). The scans remain the authoritative wiring source; see kb-agent-api

• 2026-08-05-power-root-cause-nvme-damage-ups-kb-handover — KB migration status (parked, resume 2026-08-07): (a) HYBRID SEARCH DONE + LIVE — hybrid.py fuses a rank_bm25 keyword index (heading ×3 / manual ×2) with the existing Qdrant semantic search via Reciprocal Rank Fusion (k=60), wired backward-compatibly into /api/search (added rrf/match fields); legacy lunr /browse search retired; exact-title fix verified. (b) ASTRO 5 + STARLIGHT 0.36 REBUILD STAGED at ~/kb-astro/ (Bun, no Node) — ~2203 pages, all 6 folders, 68 landing pages, HU /hu/ locale, directory-URL byte-identical to mkdocs, image opt resumable from webp cache; wiring diagrams DROPPED. Full resume / atomic-cutover / rollback steps in the note

• 6. 2026-08-06 UPDATE — KB fixkick expansion LIVE, search-index PARKED, brownout root-cause upgraded — 2026-08-06: fixkick.com expansion LIVE (Astro site now the live one), search-index PARKED. KB grew 135 → 388 EN pages from fixkick.com (live successor to archived kick-fix.com): +~253 pages incl. 23 TSB bulletins (kick-fix/tsb/), Wayback-recovered dead pages (schematics-run 500s live but had a 2017 snapshot), new sections general/power-elect/tsb. 920 internal .html cross-links rewritten (0 broken remain), 344 dropped (kickfix_linkmap.json + ADDED_MANIFEST.json; backup kick-fix.bak-linkrewrite). Single-child folders flattened (folder URL serves the one page, child→folder redirect; flatten_redirects.json in astro.config) via build_content.py. Deterministic sidebar-focus deployed (astro.config head script scrolls .sidebar-pane to active on page-load/after-swap + double-rAF + MutationObserver to beat Starlight’s scroll-restore race — Playwright proof not yet run). Glossary/acronym query-expansion LIVE: hybrid.py from kb-vectors/glossary.json = 325 bidirectional entries (154 Vitara abbrevs + 171 fixkick JARGON), exact matches still first. Deploy = atomic swap dist→~/knowledgebase/site; rollback site.astro-prev, deeper site.mkdocs-bak. PARKED: only ~90/253 new pages embedded into Qdrant manuals — pages render but aren’t searchable (hybrid.py builds BM25 from Qdrant); resume via CPU-capped resume-safe index_new.py embed. New-page images not yet downloaded (build guard drops missing refs)

• 2026-08-06-kb-wiring-gallery-native-res-fix-and-astro-caching — Flask app.py cache-control for the Astro KB: Cache-Control: immutable for fingerprinted /_astro/*, no-cache for everything else — stops the post-redeploy stale-HTML / missing-image failure after the atomic dist→~/knowledgebase/site swap. Also the wiring-gallery native-res crop fix (see CV & Diagram Understanding)

• 2026-08-06-kb-marker-chunker-bug-and-ocr-bakeoff — ⭐ HEADLINE (2026-08-06 KB overhaul): orphaned procedure-fragment pages are a marker CHUNKER/SPLITTER failure, not OCR — recoverable from marker’s RAW output without re-OCR; OCR bake-off (GOT-OCR2) → DECISION to patch (deterministic header re-attach), not full re-OCR (targeted micro-re-OCR only for destroyed spec-table/code pages). SOTA OCR noted: GLM-OCR, DeepSeek-OCR 2, Dolphin (structure-first), PaddleOCR-VL

• 2026-08-06-kb-orphan-heading-recovery-subsystem — orphan-fix subsystem (kb-vectors/): orphans.py→orphans.json (235 found), pagesrc.py (locate page body in marker RAW → nearest non-generic heading), orphanfix.py (recover + confidence GATE + re-embed/re-tag), orphan_apply.py, pageflow.py (Inspect/Fix/Re-OCR/Apply/Rebuild). Result: 89 applied / 146 staged-uncertain (gated — wrong name worse than generic), 149 Qdrant points re-embedded+re-tagged (e.g. point 259 IAC valve Cooling→Fuel). Re-embed = new text→bge embed_passages→update vector+text payload in place preserving keys; re-tag via tagger.py. Reversible: .bak-orphanfix + staging/orphan-changelog.json

• 2026-08-06-kb-tools-ui-and-job-queue — per-page Tools UI + persistent job queue. app.py injects a floating “Tools” button on every page + /orphans, /api/orphans, /api/page-flow. Job queue jobstore.py = persistent JSONL kb-vectors/jobs.jsonl (survives restart/power-blip), GET /api/jobs + /jobs auto-refresh dashboard; every apply/re-OCR/fix/rebuild registers a typed job

• 2026-08-06-kb-section-code-extraction — FSM section-code extraction: codes (8A-9, 6E-92) parsed from contents tables + text → Qdrant payload section_codes/section_names/section_prefixes on 490 pages + kb-vectors/section_index.json (1254 codes). Prefix→system map (8A/8B/8C→Electrical, 6E/6C/6J/6K→Fuel&Emissions, 6A→Engine, 6B→Cooling, 7A–7F→Drivetrain, 3x→Steering&Susp, 1A/1B→HVAC, 9J→Body, 0x→Diagnostics). ~61% agreement with the text system-tags → complementary, not redundant

• 2026-08-06-kb-embedding-classifiers-system-tags-and-image-types — two embedding classifiers: (a) TEXT system-tags — bge doc-vectors vs 10 prototype vectors, cosine-assigned → Qdrant primary_tag/tags (7820 pts, tagger.py); gotcha — the Fuel prototype lacked idle-air-control terms so IAC mis-scored Cooling (augment the prototype). (b) IMAGE types — SigLIP zero-shot (see CV & Diagram Understanding). Corpus is almost all scanned B&W line art

• 2026-08-06-kb-build-backup-dir-leak-and-tsb-diagram-restore — build gotcha: build_content.py auto-discovered ALL dirs under manuals-src/docs incl. *.bak* → ~1646 stale pages (of 4566) served LIVE; fix = a list_manuals() helper excluding any name containing .bak. Lesson: content-tree backup dirs must be excluded from discovery. Also: FSM diagram images restored to 20 kick-fix/tsb legend pages via pdftoppm render (crops never captured on ingest)

• 2026-08-08-telep-master-agent-impl — the vault now has a second read-write clone on the host at /home/levander/obsidian (write deploy key), kept convergent with the Mac vault by obsidian-sync.timer (bidirectional pull --rebase --autostash + commit + push every ~15min). The master agent runs path-rewritten historian + obsidian-documenter subagents against this clone (/home/levander/.claude/agents/) — so the box can recall + document homelab work itself, degrading gracefully where Mac-only sources (claude-mem, auto-memory, obsidian CLI) are absent

• 2026-08-31-telep-kb-mcp-server — KB search reaches agents as MCP, not just HTTP (complements kb-agent-api’s /api/search). Two named collections: manuals 7816 pts, hybrid BM25+semantic; notes = the Obsidian vault, semantic only — because hybrid.py holds its BM25 index in module globals (_bm25/_docs/_facets) it can serve exactly one collection per process, and refactoring that to per-collection instances was deliberately deferred. Embeddings bge-large-en-v1.5, 1024-dim cosine, CPU-only torch; vector modules reached via app.py:8 sys.path.insert(0, "/home/levander/kb-vectors")

• 2026-08-31-obsidian-vault-qdrant-index-selection — the vault itself becomes searchable (kb-vectors/obsidian_index.py). Selection unit is a project (each dir under projects/, each top-level dir, vault-root .md as one pseudo-unit), excluded entirely if ANY note in it carries the frontmatter tag personal — opt-out, not opt-in. Inside an included unit LOG.md/TOPICS.md are skipped as scaffolding (homelab’s pair = 159 chunks at ~0.9 link density; generated near-pure wikilink lists that embed to noise and displace real answers, and are derived restatement of already-indexed notes), while index.md / moc overviews / Agent Landing.md are kept (0.09–0.34, real prose). Scope 28 units excluded / 16 included / 218 notes / ~979 chunks. Incremental + resumable: per-note sha256 content_hash in the payload, unchanged skipped, changed re-embedded and upserted immediately (not batched), vanished/newly-excluded points deleted, no delete_collection — verified run1 embed 2 → run2 zero → run3 only the edited note → run4 purge

• 2026-09-03-remarkable-paper-pure-claude-integration — 🔮 PLANNED: handwritten-notes-into-an-LLM evaluation for a (not yet bought) reMarkable Paper Pure. ⭐ OCR is the gate on whether ink is readable at all — REMARKABLE_OCR_BACKEND = auto|google|tesseract: Google Vision good at handwriting but needs an API key and is not offline (1,000 free req/month, then ~$1.50/1,000), Tesseract offline and free but designed for printed text and poor at handwriting. Typed text, PDF text and PDF annotations extract natively with no OCR — so “type it / annotate a PDF” sidesteps the whole problem. Read tools remarkable_read/browse/search/recent/status/image (PNG/SVG + optional OCR)/export (PDF or sectioned Markdown); documents auto-register as MCP resources under remarkable:///, remarkableimg:///, remarkablesvg:///

• 2026-09-07-session-handover — ✅ the knowledgebase IS the existing markdown renderer, and the record is corrected: Astro 5.14 + Starlight 0.36, built with Bun, project /home/levander/kb-astro/, canonical builder build_content.py — NOT convert.py, which is partial and produces a STALE site (independently re-confirmed by a second research pass); deploy kb_build.sh → atomic swap into ~/knowledgebase/site/ (1.6 GB built); knowledgebase.service (Flask + waitress) on 127.0.0.2:8092. ⚠️ The knowledgebase is NOT containerised — plain systemd; the only related container is kb-qdrant, previously unrecorded anywhere in the vault. ⚠️ The sidecar unit/socket/statedir are named kb while the hostname is knowledgebase. mkdocs-material → Astro (Aug 2026) specifically to preserve byte-identical directory URLs. ✅ CONFLICT RESOLVED: home.taild4189d.ts.net fronts gethomepage (→127.0.0.2:3010), not the old static home-portal — older vault notes are STALE; homepage also answers on the LAN at http://home.telep.lan via Caddy on :80; the orphaned portal (python3, 127.0.0.1:8093, pid 2716, 3.8 KB August index.html) is still running and should be killed. 🚩 gethomepage cannot host arbitrary pages — use the iframe service widget; docker restart homepage does NOT apply config (curl http://127.0.0.2:3010/api/revalidate); quote every description: or an unquoted colon-space blanks the dashboard; services.yaml has concurrent writers ⇒ surgical line insertion only. ⚠️ The requested markdown renderer was NOT built and the content to render was never specified

File Storage & Sharing

• filestash — in-browser file browser for the box’s Windows C: drive; NTFS mounted read-only at /mnt/win via ntfs3, served over tailnet at :8445

• mp4-missing-moov-forensics — a 1.6 GB SplitCam recording served off /mnt/win via Filestash was dead on arrival; how to trace a download’s origin (kMDItemWhereFroms) and chunk-md5 it against the source

• Drive inventory (2026-07-21) — mainframe drives: nvme0n1 1.8T Windows, nvme1n1 3.6T LUKS+LVM OS, sda 447G unused, sdb = Hiren’s BootCD PE USB stick

• 2026-08-10-top-kepek-drive-curation-print-pipeline — Google Drive staging via rclone from the box — curated gdrive:top_képek/ (flat), raw gdrive:top_képek/archive/ (flat, no month subfolders), ready-to-print folder referenced BY ID gdrive,root_folder_id=<id>: (rclone connection-string folder-by-id, --config /home/levander/.config/rclone/rclone.conf). ⚠ Drive rate-limits HARD after bulk moves (~18–60s/file) → run reorg/rename/flatten jobs detached (setsid nohup … </dev/null &, poll log), never timeout-wrapped SSH

• 2026-08-31-nextcloud-drive-tailscale-spec / 2026-08-31-nextcloud-drive-tailscale-plan — Nextcloud “Drive” on telep-mainframe (nextcloud + postgres + redis + onlyoffice compose stack at /home/levander/nextcloud/), identity via Tailscale tsidp + user_oidc, own drive tailnet node, CAD dir as external storage, whole store re-shared over Taildrive

• 2026-08-31-nextcloud-34-custom-file-action-registration — custom cadviewer app makes a .stl click open Online3DViewer instead of downloading; the NC 34 file-action API (window._nc_files_scope.v4_0.fileActions, context-object callbacks) + why _nc_fileactions silently does nothing. O3DV deep link resolves only for CAD-folder files (v1 limitation)

• 2026-08-31-nextcloud-drive-code-server-runbook — the operational runbook for the finished Drive stack: port map, occ invocation form, nextcloud-scan.timer (3 min: ownership normalization → files:scan --all), Taildrive shares (drive → data dir, cad → exports, both as root — writes land root:root, the timer is the mitigation, ≤3-min lag), OnlyOffice JWT/internal-URL requirements, verification checklist, every .bak + rollback

• 2026-08-31-nextcloud-drive-code-server-runbook — /windows external storage: storage id 2 → Drive /windows, read-only at four layers (kernel ntfs3 ro · docker bind :ro · Nextcloud readonly: "1" via a PermissionsMask storage wrapper · code-server files.readonlyInclude). ⭐ Nextcloud binds the whole C: drive (/mnt/win:/mnt/windows:ro) and narrows via datadir: /mnt/windows/Users, so widening Drive to all of C: is one command with no compose edit or restart (occ files_external:config 2 datadir /mnt/windows); code-server instead narrows in the bind (/mnt/win/Users:/home/coder/windows:ro). filebrowser retired — it was the only other door onto the same data (container stopped, volumes preserved, its serve :8445 removed)

• 2026-08-31-nextcloud-drive-code-server-runbook — scan timer restructured: files:scan --all --home-only + a targeted /CAD scan, because a naive --all with the windows mount exceeded 10 min and never completed on a 3-min timer; current form costs 0.83 s

• 2026-08-31-nextcloud-drive-code-server-runbook — file-action routing: .stl→O3DV, .csv/.docx→OnlyOffice (native defFormats, not a custom action), code→VS Code, .md/.txt→Nextcloud Text, fallback → VS Code, Download always last, “Open with” submenu on every file

• 2026-09-03-remarkable-paper-pure-claude-integration — 🔮 PLANNED: rmfakecloud (ddvk) as a fully self-hosted reMarkable sync backend on telep-mainframe — no vendor cloud, no Connect subscription, Paper Pure listed as supported, and WebDAV (Nextcloud/Owncloud) support pairs it directly with the existing Drive stack. ⚠️ File sync is tested only up to reMarkable software 3.27.1 — newer firmware untested, so a tablet auto-update is a real breakage risk. Setup installs rmfakecloud-proxy on the device (installer-rmpro.sh for Paper Pro/Pro Move via scp to 10.11.99.1; installer-rm12.sh for rM1/rM2), which generates a CA + *.appspot.com host cert into /usr/local/share/ca-certificates, runs update-ca-certificates and edits /etc/hosts — so it also requires Developer Mode, i.e. a factory reset. 🔴 v0.0.25 needs an extra /etc/hosts domain: eu.tectonic.remarkable.com. Supported: sync 1.0 + 1.5/2/3/4, send-by-email, handwriting recognition, screen sharing, WebDAV. Not supported: handwriting search, OneDrive (Dropbox/GDrive WIP)

Firewall & Security

• telep-router — zones: lan ACCEPT, wan REJECT + masq + mtu_fix, plus an isolated cams zone with no cams → wan
• Security Gaps — 8-char WPA3 PSK, dropbear password auth, LuCI on plain HTTP, plaintext DNS, unpatched 24.10.0
• telep-mainframe — all ports bound to 127.0.0.1; passwordless sudo is an accepted tradeoff; Secure Boot disabled to load the NVIDIA module
• Prerequisite router access — new LAN-only key-only dropbear on the router (PasswordAuth off) partially closes the “dropbear password auth” gap; bot token read from ~/nvr/frigate-notify/config.yml, never hard-coded
• 2026-08-03-cam-stall-recovery-and-casino-alert — the casino/gambling detector on the router: alert half deployed (/etc/casino-alert.sh procd service Telegrams gambling-domain DNS lookups by client IP), blocking (/etc/casino-block.conf sink list) deliberately deferred — it didn’t answer blocked queries after restart (auto-rolled-back) and every dnsmasq restart risks a DNS outage + re-stalling the weak .139 camera; TODO redo via addnhosts+SIGHUP (no restart)
• 2026-08-15-yubikey-gated-luks-fde — ⭐ root LUKS FDE is now YubiKey-GATED (replaced clevis/TPM). Possession model: key plugged in → unattended auto-unlock (survives mains cuts); pull it → udev sync+poweroff; boot without it → passphrase prompt. HMAC-SHA1 challenge-response initramfs keyscript (NOT FIDO2/systemd-cryptenroll — Debian initramfs-tools ignores those tokens, same reason clevis was used for TPM); slots 0=passphrase, 2=YubiKey, no TPM token. 🔴 ykman config = re-enumeration = a udev remove = poweroff → disarm 99-yk-lock.rules first; ONLY ONE key enrolled (add a backup); whole-box theft still boots WITH the key (deliberate — possession-only over a PIN). Enroll gotcha: pipe→luksAddKey DEV - fails (stdin eaten), use a /dev/shm keyfile. Pre-YK header backup SHREDDED (held the TPM token)
• 2026-07-24-luks-tpm-autounlock — ⚠️ SUPERSEDED 2026-08-15 (now YubiKey-gated, clevis/TPM binding removed). Historically: root LUKS2 TPM2 auto-unlocked via clevis (keyslot 1 = TPM empty PCRs, 0 = passphrase); with empty PCRs + Secure Boot off it protected bare-drive theft, not whole-box theft; reboot test PASSED in production (2026-07-24-psu-load-resets). Still-valid reusable facts: Debian initramfs-tools ignores systemd-cryptenroll TPM/FIDO2 tokens (use clevis / a keyscript); header-backup discipline; the orphaned-systemd-crypt header-lock gotcha
• Review-pass hardening (2026-08-10) — security/correctness review pass over the top-kép voting system (fixed + verified). 🔴 CRITICAL pre-existing bug: GRID_TAIL had doubled {{/}} braces but was concatenated RAW (not .format()ed) → they leaked into the <script> making it invalid JS → vote() never defined → client-side ❤ voting had NEVER worked (missed because earlier reviews curl’d the JSON endpoint, never rendered the page — lesson: check the SERVED JS). Fixed+verified: stored XSS (client camera unescaped → render_grid html.escape quote=True), concurrent toggle race (double-tap → IntegrityError/500 → atomic DELETE…rowcount+INSERT OR IGNORE), CSRF (/vote 403s on Sec-Fetch-Site: cross-site), DB perms (/srv/top-kep/votes.db 644→600, exposed voter emails), client error handling (if(!r.ok) guard), disk reclaim on un-vote-to-0. Confirmed SAFE: identity spoofing impossible (tailscale serve overwrites Tailscale-User-Login from the verified peer; app binds 127.0.0.1), bot token can’t leak via exceptions. Deferred: derive camera server-side, validate event_id charset
• 2026-08-08-telep-master-agent-impl — the phone-driven host agent’s deliberate security posture: trust dialogs are pre-seeded (remoteDialogSeen/hasTrustDialogAccepted=true in ~/.claude.json) so it launches headless, but --dangerously-skip-permissions is NOT used — tool permissions stay ACTIVE, so destructive/tool actions surface an approval prompt in the Claude app (human-in-the-loop). 🔴 The guardrail is approval, not a sandbox — and because the box has passwordless sudo (/etc/sudoers.d/levander), an approved sudo action from the phone is instant root. Mirrors the Incus container control-plane posture; pre-seeding widens launch, not authority
• 2026-08-09-xiaomi-c302-tailnet-facetime-cam — per-IP firewall scoping to keep the Tapo NVR cams air-gapped while one cam has internet. The C302’s Xiaomi P2P needs BOTH cams→wan (cloud key) and cams→lan (P2P return path) so it is not network-air-gappable — instead of opening the whole cams zone, the zone-wide forwardings were replaced with per-IP rules c302-wan/c302-lan (src_ip=192.168.30.168), restoring full air-gap for .119/.139. Privacy compensated physically: hardware relay + a red presence banner. Secrets (Mi pw, go2rtc V1: blob, cam token) kept out of the vault
• 2026-09-07-session-handover — 🔴 standing credential exposure: the Telegram bot token is plaintext in /home/levander/nvr/frigate-notify/config.yml and leaked into a chat transcript TWICE — Telegram now carries no alerts but the token can still post to the group, so rotate it; the bambuddy API key is likewise still unrotated; and a live @phone access token sits at /root/.mrelay/phone-session.json (0600) from a subagent’s decryption harness with no ongoing purpose — log the device out and delete it. bambuddy’s API remains unauthenticated with an EMPTY users table — the tailnet ACL is the only thing between the internet and a machine that heats to 280 °C. ✅ Positives: Matrix federation OFF and open registration OFF, both proven with live 403s; the relay’s :8118 binds verified not tailnet-reachable; secrets kept at 0600 (/root/matrix-credentials.txt, /etc/matrix-relay.env)
• 2026-09-07-alert-source-inventory — 🔴 the Telegram token leak is WIDER than recorded — THREE locations: ~/nvr/frigate-notify/config.yml, /etc/nut/telegram.env (0600 root), and hardcoded in plaintext in /etc/casino-alert.sh on telep-router, mode 0755 — WORLD-READABLE. Rotation must cover all three. ⚠️ Rotate ≠ delete: rotating leaves four alerters silently posting to a dead token (LOUD only for top_kep_remind — unhandled HTTPError 401 ⇒ visible in systemctl --failed), while deleting frigate-notify’s telegram: block raises ValueError in load_telegram_creds() ⇒ frigate-viewer-alert crash-loops on Restart=always. 🔴 The matrix-relay is UNAUTHENTICATED and UNVALIDATED — source-confirmed: the URL path is only a label, so it accepts a POST on ANY path and posts whatever it receives; ss -lntp shows exactly 127.0.0.2:8118 + 172.18.0.1:8118 on one pid (tailnet IP 100.115.209.87 not bound), so it is not tailnet-exposed — but any container on the nvr_default docker bridge can post arbitrary content into the LIVE alert room. ✅ UPDATE 2026-09-07 — ROTATION IS NOW SAFE. All four host alerters send via Matrix, so the Telegram path is inert rollback only; rotate all three locations (including the world-readable /etc/casino-alert.sh on the router, untouched and still needing it). 🔴 Deleting the telegram: block is STILL unsafe — load_telegram_creds() is deliberately still called to keep rollback available ⇒ deletion still crash-loops frigate-viewer-alert. ⚠️ And top_kep_remind is no longer the loud canary (it is on Matrix now), so a missed rotation location will now be completely silent. 🔴 Before rotating anything, note: Tailscale SSH logs the FULL command line to the journal — tailscaled writes the entire invocation, so any secret passed as a command-line argument is stored in the journal in PLAINTEXT. Set the new token by editing files or via stdin, never as an argument. ⚠️ And it compounds with the migration: top_kep_remind is on Matrix now, so it is no longer the loud HTTPError 401 canary ⇒ a missed location fails completely silently. A careless rotation can therefore leak the new token AND quietly leave a live one behind — verify all three locations by hand.

GPU & Hardware Acceleration

• telep-mainframe — RTX 3080: ONNX inference ~12ms / ~6% GPU, NVDEC decode (preset-nvidia), nvdec-copy in mpv so the drawtext filter can run

• TV wall (HDMI) — nvidia-drm modeset=1 is mandatory or the 3080 exposes zero display connectors

• 2026-07-18-birdseye-nvenc-blue-glare — the 3080’s NVENC encoder (not just NVDEC) can wedge and corrupt Frigate’s birdseye restream (h264_nvenc) blue/purple even at GPU idle; docker compose restart frigate clears it, blank hwaccel_args → libx264 for a durable fix (YOLO inference stays on CUDA)

• 2026-07-24-knowledgebase — the knowledgebase ingest reuses the ~/ocr/venv marker pipeline as a subprocess (marker_single, TORCH_DEVICE=cuda); a single background worker GPU-serialises OCR (one at a time, shared with Frigate) so occasional ingests coexist with detection on the 12 GB card

• 2026-07-24-kb-vectorize-complete — embeddings run on CPU torch deliberately (sidesteps the CUDA-13-vs-driver-550 trap and avoids GPU contention with OCR + Frigate); but maxing all 16 cores browned out the PSU → cap to 4 threads (see 2026-07-24-psu-load-resets)

• Running it on a CUDA GPU box (telep-mainframe) — driver 550.163.01 caps you at CUDA 12.4: pip’s default cu130 torch can’t init CUDA and PyTorch silently runs on CPU; fix is a cu126 build (minor-version compat), not a driver upgrade — driver 550 is load-bearing for Frigate. Reusable OCR venv at ~/ocr/venv; marker ~3.5 GB VRAM coexists fine with Frigate’s ~2.5 GB on the 12 GB card

• 2026-07-28-camwall-4-substream-composite — the 4-substream 2×2 camwall keeps --hwdec=no (software decode on the 12900K) because the 3080’s NVDEC corrupts these feeds to magenta under contention with Frigate

• 2026-08-04-telep-mainframe-airplay-receiver-uxplay — the uxplay AirPlay receiver has NO GStreamer HW H.264 decode on this box: Debian gstreamer1.0-plugins-bad ships no nvcodec (nvh264dec/nvdec absent) and there’s no VAAPI (vainfo empty) → decode falls to software avdec_h264 (gstreamer1.0-libav); scaling is still GPU via the sink; tolerable because AirPlay isn’t true 4K. Fullscreen: uxplay -fs leaves glimagesink rendering ~1:1 top-left — use xvimagesink (Xv rescales to the drawable every frame) + no WM + an xdotool force-resize of the telep-tv@telep-mainframe window to 3840x2160+0+0. Audio out is the NVIDIA-HDMI ALSA sink plughw:1,3 (card 1 dev 3, “LG TV SSCR2”)

• 2026-08-06-power-root-cause-gpu-12v-connector-stress-test-pass — ⭐ the recurring under-load hard-offs were an UNPLUGGED GPU 12V PCIe connector on the RTX 3080 (3 separate PSU cables, one bank disconnected), reconnected during reassembly. Graduated CPU+GPU stress test PASSED to 250W CPU / 450W GPU cap, zero crashes. Peak GPU ~411W, peak combined ~536W; GPU 86–87°C no throttle. Testing facts: CPU stays ~125W under sustained load because PL1=min(125,PL2) (PL2 is boost-window only — raise PL1 to stress the CPU rail); GPU FP32 matmul is compute-bound below the 450W cap. ⚠️ caps are NOT persistent — reset on reboot (mobo uncapped CPU default + GPU 320W stock)

• 2026-08-31-nvidia-drm-host-crash-embedding-pass — 🔴 a CPU-only job can still be taken down by the nvidia DRM driver: the full vault embedding pass hard-crashes telep-mainframe with nv_drm_revoke_modeset_permission+0x327/0x340 [nvidia_drm] via drm_file_free/drm_release (driver 550.163.01, RTX 3080, nvidia_drm modeset=Y, Xorg+mpv holding DRM fds). The identical trace predates the work (2026-08-22) so the driver bug is pre-existing, but the job coincided with 3 crashes in 3 attempts (last one logged no kernel output) — and since it is CPU torch that never opens a DRM fd, it is a trigger, not the caller. Real fix is a driver update off 550.163.01 or dropping nvidia-drm.modeset=1 — both blocked here (550 is load-bearing for Frigate/OCR CUDA 12.4; modeset=1 is mandatory or the 3080 exposes zero display connectors)

• 2026-08-31-telep-mainframe-mce-hardware-fault — ⚠ correction: nv_drm_revoke_modeset_permission [nvidia_drm] (550.163.01, Comm: vo = mpv/camwall) is a RED HERRING for the host crashes — it appears before and after the panic and on healthy boots. Two agents blamed it; both retracted. 2026-08-31-nvidia-drm-host-crash-embedding-pass is superseded

• 2026-08-31-telep-mainframe-mce-hardware-fault — ⚠ BIOS F31/F32+ enable Secure Boot + pre-boot DMA by default → the unsigned NVIDIA DKMS module stops loading → camwall dies. Disable Secure Boot after any flash. Prefer Q-Flash Plus (board controller, no CPU/OS). Not a microcode story: Debian intel-microcode already loads 0x3d

• 2026-09-01-orcaslicer-tailnet-deploy — GPU NOT passed through: OrcaSlicer’s 3D viewport runs on software GL, fine for slicing and keeps the RTX 3080 free for Frigate/OCR. Also 🔑 tailscale up --authkey with a truncated/mangled key silently yields NeedsLogin (looks like nothing happened), not NeedsMachineAuth — NeedsLogin = bad/incomplete key; NeedsMachineAuth = key fine, awaiting console approval

• 2026-09-02-frigate-resource-tuning — Frigate VRAM audit: 2148 → 1804 MiB (−344) by disabling a face_recognition that had never been used. ⚠️ disabling it does NOT remove frigate.embeddings_manager — in 0.17 that process also serves semantic search + LPR, so it halved 678 → 324 MiB rather than disappearing. Baseline for the healthy pieces on the RTX 3080: detector onnx/yolo-generic 640×640 = ~11–19% CPU + 394 MiB VRAM, detect streams 1280×720 @5fps on preset-nvidia + scale_cuda = ~2.7% CPU/cam, record ffmpegs -c:v copy = ~1.5% each. 🔴 The expensive thing was NOT on the GPU at all — birdseye’s jsmpeg live view is a software mpeg1 encode (ffmpeg -f rawvideo -video_size 3840x1080 … -codec:v mpeg1video) of a 4.1 MP canvas running 24/7

• 2026-09-02-lg-tv-network-control-presence — 🔴 ⭐ NVIDIA consumer GPUs have NO HDMI-CEC — there is no fix, ever. /dev/cec* and /sys/class/cec/ do not exist and never will (verified on driver 610.57.04); no driver option, module parameter or kernel module adds it, so cec-client over the camwall’s existing HDMI cable is a dead end — do not burn a session on it. Alternatives: a Pulse-Eight USB-CEC adapter (~EUR40) wired INLINE (GPU → adapter HDMI-in → adapter HDMI-out → TV; hanging it off a spare port makes “set active source” switch to the wrong input), an IR blaster, or the vendor network API (what was chosen). ⚠️ xrandr physical dimensions are a FALLBACK VALUE, not a measurement, under the NVIDIA blob — HDMI-1 reported 1600mm x 900mm (≈72–75”) for a set that is actually 43”, because the proprietary driver never populates /sys/class/drm/*/edid and 1600x900mm is a round 16:9 placeholder

• 2026-09-02-lg-tv-network-control-presence — 🔴 ⭐⭐ A GPU output name and a TV input number are UNRELATED NAMESPACES. xrandr reports HDMI-1 — that is the GPU’s port index and says nothing about which socket on the television the cable is in. The mainframe is physically in the TV’s HDMI 2, so switchInput HDMI_1 selected an empty socket and the set showed “no signal” while xrandr, the framebuffer and the camwall were provably healthy — a perfect false trail where both ends look fine and the screen is blank. Note xrandr under the NVIDIA blob misled on both the panel size (1600x900mm placeholder for a 43” set) and, by naming, the TV input: treat it as a GPU-side view only.

Gotchas & Learnings

• 2026-09-05-freecad-mcp-502-dead-container-and-qt-event-loop-wedge — 🔴 ⭐⭐ A restart policy is not a liveness guarantee, and a closed dialog is not a working event loop. Two independent traps in one outage: (1) restart: unless-stopped never fired for a container whose containerd shim died before task creation (ttrpc: closed, RestartCount: 0) — it stayed dead for 3 days; docker ps without -a hides exactly this. (2) xdotool windowclose removed the blocking modal — verified gone from the window tree — and execute_code still timed out at 90 s while gui_dispatch reported healthy: the thing that caused a wedge is not necessarily the thing still holding it. 🚩 Also: command -v a b c under dash only reports the FIRST argument, silently ignoring the rest — which made xdotool look uninstalled when it was present all along; use a loop or one command -v per binary. 🚩 And DISPLAY=:0 vs :1: a :0 socket can exist while xwininfo cannot open it, and the failure reads as “no windows”, not “wrong display”.

• 2026-09-05-session-handover — 🔴 busybox lacks timeout, and nc -z / ping -M LIE — this produced FOUR false diagnoses in a single session on telep-router. Test from the mainframe, not from busybox. 🔴 Frigate stores recordings in UTC — local 07:23 lives in the 05/ hour directory (and Frigate’s own export API left three of four exports stuck as orphaned in_progress rows with no worker process, so exports were built directly with ffmpeg -f concat -c copy). 🔴 xrandr HDMI-1 ≠ the TV’s HDMI 1 — the numbers are unrelated; switching to the wrong one blanks the TV to “no signal” while xrandr, the framebuffer and camwall all look perfectly healthy. ⚠️ ps -eo pcpu reports a LIFETIME average, not instantaneous CPU — worthless on a freshly booted box; use repeated docker stats --no-stream or top -bn2. ⚠️ aiowebostv caches power state — power_off() short-circuits on if not self.tv_state.is_on: return, so read live and verify the result rather than trusting the cache

• 2026-09-02-tv-presence-wifi-union-daemon — 🔴 never build LAN presence on arp-scan alone — a sleeping iOS phone stays WiFi-associated but stops answering ARP (confirmed twice independently on this LAN; see also 2026-07-17-intruder-alarm). ⚠️ OpenWrt/dropbear host key at 192.168.1.1 rotated → REMOTE HOST IDENTIFICATION HAS CHANGED from root on telep-mainframe (known_hosts is per-user — /root/.ssh/known_hosts, not levander’s); fix ssh-keygen -f /root/.ssh/known_hosts -R 192.168.1.1 then reconnect -o StrictHostKeyChecking=accept-new; new ED25519 SHA256:5WXvfnjJYyOyZa3h/iVwdsYrOb17SLqVFfXV6CLXugY. 🐍 extension-less python files can’t be loaded via importlib.util.spec_from_file_location (spec comes back with loader=None — the extension picks the loader); use importlib.machinery.SourceFileLoader + importlib.util.spec_from_loader.

• 2026-09-02-frigate-resource-tuning — 🔴 ⭐⭐ ps -eo pcpu is a LIFETIME AVERAGE since process start, NOT instantaneous CPU — on a box ~12 minutes into a fresh boot every reading has post-reboot startup churn permanently baked in (load avg was still settling 15.93 → 6.04 → 3.86), which is how “Frigate at 149.81% CPU” became a false alarm. Sample docker stats --no-stream repeatedly, or top -bn2 and trust the second iteration; never conclude from one ps %CPU. ⭐ Always divide by the core count before panicking — 149.81% on a 24-core box is ~6.2% of capacity. ⭐ A bad baseline poisons the whole before/after — the CPU result here is permanently INCONCLUSIVE because the “before” was measured during startup; the VRAM/RAM numbers survived only because they aren’t time-averaged. ⭐ docker logs <container> REPLAYS FULL HISTORY — grepping for errors after a restart surfaced entries dated 2026-07-27 that had nothing to do with it; always pass --since or read the timestamps before blaming your change

• 2026-08-31-nextcloud-34-custom-file-action-registration — ⭐⭐ A silent no-op is a SIGNAL, not the absence of one — a registration that yields neither an error nor an effect almost always means you wrote to a dead global. Nextcloud 34’s custom file action never appeared because it was pushed onto window._nc_fileactions, a string that appears in zero served bundles; the live registry is window._nc_files_scope.v4_0.fileActions (a Map). One grep across the actually-served bundles closed the case — do that before trusting any doc, tutorial, or prior code. ⭐ When you can’t reach the UI, extract the real predicate from the bundles’ source-map sourcesContent and drive it in a Node harness (here FileEntryMixin.ts’s enabledFileActions/defaultFileAction + downloadAction.ts) → genuine PASS/FAIL with no browser. ⭐ Second stacked defect: NC 34 calls enabled/exec with a single context object {nodes,view,folder,contents}, not positional args → Array.isArray(ctx) guards silently drop the action. ⭐ Match 3D files by extension, not mimetype (.stl/.step/.3mf are served application/octet-stream), and bump occ config:app:set theming cachebuster or clients keep a stale bundle forever. Failure signature to search for: XHR HEAD at downloadAction.ts:69

• 2026-08-30-telep-tv-airplay-10g-port-multicast-not-flooded — ⭐⭐ “unicast works but discovery doesn’t” = a MULTICAST-flooding problem, and a switch can drop multicast per-PORT — telep-tv AirPlay never appeared on Wi-Fi despite dns-sd -L resolving + ping/ssh working; the switch simply wasn’t flooding the box’s mDNS multicast from its 10G port to Wi-Fi (1G-port devices reached Wi-Fi fine). Reusable diagnosis: (1) tcpdump on the sender’s iface to prove the multicast is on the wire; (2) dns-sd -L (unicast) vs dns-sd -B (multicast browse) to split resolve-vs-discover; (3) browse a service ONLY the target advertises (_nut._tcp) so nothing else dedupes it — vs one from a known-good host (_ipp._tcp, the printer) — to isolate whose multicast is dropped; (4) check the bridge fdb PORT + link speed for the failing host. ⭐ Don’t let a plausible prior theory (here “5 GHz cross-band isolation”) anchor a NEW symptom — the AirPlay failure was band-independent, so it was never the 5 GHz issue

• 2026-08-26-router-reflash-mac-5ghz-dfs-and-bambu-offline — 🔴 ⭐ macOS CLI Wi-Fi tools LIE about association — networksetup -getairportnetwork en0 reports “You are not associated with an AirPort network” and system_profiler SPAirPortDataType redacts SSIDs even while Wi-Fi is fully up (valid IP + internet). It’s the Location-Services privacy restriction on CLI tools, not a disconnect — verify with ifconfig en0 (status active + inet) / ipconfig getifaddr en0 / route -n get default / ping instead. ⭐ macOS often won’t LIST a 5 GHz SSID that’s on a DFS channel (52–144) in the Wi-Fi menu (passive scan) while phones show it instantly — “it’s the Mac”; fix router-side with a non-DFS channel (36/40/44/48 or 149/153/157/161), no radar/CAC wait. ⭐ Bambu Studio’s log_iotc.txt ErrCode -12 (Not Initialized) is a symptom of an absent printer, not a cause — confirm reachability at the network layer with an SSDP M-SEARCH probe + nc -z <ip> 8883 990 6000 before touching Studio. ⭐ A router reflash resets the SSH host key AND /etc/dropbear/authorized_keys (→ known_hosts mismatch + key auth fails + password demanded) and re-registers Tailscale (new tailnet IP)

• 2026-08-15-telep-lan-split-horizon-caddy — 🔴 ⭐ Caddy that binds a specific LAN IP loses a boot-race and does NOT retry — it starts before 192.168.1.123 is assigned to enp5s0, fails bind: cannot assign requested address on :443, and stays failed (silently took ALL *.telep.lan + levandor.io domain access down for ~1.6 days after the YubiKey reboots). The bind <ip> (needed to dodge Tailscale’s :443) is what makes it IP-timing-sensitive. FIX = net.ipv4.ip_nonlocal_bind=1 (bind an address before it’s assigned) + a Restart=on-failure/After=network-online.target drop-in. Runbook caddy-boot-race

• 2026-08-17-home-dashboard — 🐛 a colon-space in a Homepage description: value breaks YAML parsing (Windows C: (RO) → bad indentation of a mapping entry) → quote all descriptions. Also: Homepage needs network_mode: host for its siteMonitor: checks to reach loopback/VLAN backends, and HOMEPAGE_ALLOWED_HOSTS set or it rejects unknown Host headers. Outage-safe pattern: no icon: (no CDN dependency), local widgets only, a second Caddy-independent tailscale serve path so the status board survives the outage it’s for

• 2026-08-15-yubikey-gated-luks-fde — 🔴 ⭐ cryptsetup luksAddKey DEV - reads the piped key as the EXISTING passphrase, not the new key — the pipe occupies stdin so the interactive prompt never comes and it fails No key available with this passphrase; write the new key to a /dev/shm keyfile so stdin stays free. ⭐ Debian initramfs-tools ignores systemd-cryptenroll FIDO2/TPM2 tokens (no systemd in the initramfs) → use clevis or a custom keyscript, not the modern FIDO2 path. ⭐ A ykman config re-enumerates the key = a udev remove event → any udev-remove automation (here a poweroff) fires mid-command; disarm the rule first. ⭐ De-risk an initramfs-unlock change with a zero-risk init-premount probe that runs the unlock logic and logs to /dev/kmsg while the OLD mechanism still does the real unlock (dmesg|grep), + cryptsetup luksOpen --test-passphrase --key-file - on the running system — only remove the fallback after both pass

• 2026-08-15-camwall-dead-br-cams-bridge-down-after-wifi-reload — 🔴 ⭐ A runtime wifi reload can silently drop a WIRELESS-ONLY OpenWrt bridge and it does NOT auto-recover — br-cams’ only member is the cams AP phy1-ap1, so a radio1 channel change (wifi reload) left br-cams state DOWN, qdisc noop with no inet and the static network.cams L3 (192.168.30.1/24) unapplied → whole cams VLAN + camwall down. ⭐ ifup <iface> alone does NOT bring a carrier-down wireless-only bridge back — needs a forced ifdown cams; sleep 2; ip link set dev br-cams up; ifup cams; sleep 5. ⭐ An identical symptom (fps=0 / frozen camwall) can have two totally different layers — camera-side wedged RTSP (2026-08-03-cam-stall-recovery-and-casino-alert, cams pingable, fix ONVIF reboot) vs router-side bridge-down (cams NOT pingable, fix the bridge); branch on ip addr show br-cams, don’t chase the cameras (the ONVIF reboot ConnectTimeout was a red herring — the cams were L3-dead, not broken). ⭐ A ~/tapo-ctl/ script under system python3 fails ModuleNotFoundError: onvif — the module lives only in ~/tapo-ctl/venv; all invocations (incl. OliveTin actions) must use ~/tapo-ctl/venv/bin/python

• 2026-08-15-router-wifi-tuning-htmode-band-penetration — ⭐ “WiFi is weak” ≠ “turn up the power” — TX power is usually already at the regulatory ceiling (here 5 GHz 23 dBm / 2.4 GHz 20 dBm on ETSI, HW 30 dBm only on DFS ch 100–140), so the UI “max” is literally true and power is not a lever. The real levers are channel WIDTH (narrowing HE80→HE40 concentrates the same power ≈ +3 dB range/penetration at half the peak rate) and band steering (move far/through-wall clients off 5 GHz onto 2.4 GHz — 5 GHz can’t penetrate walls). No single-router setting beats RF physics; dead zones need a 2nd AP with wired backhaul

• 2026-08-15-dhcp-outage-duplicate-reservation-postmortem — 🔴 ⭐ A duplicate dhcp-host IP is a FATAL dnsmasq error, not a silently-ignored one — adding a second reservation with the same IP (192.168.1.123) / MAC (d8:5e:d3:a7:05:d6) as the existing canonical one made dnsmasq log duplicate dhcp-host IP address … FAILED to start up and refuse to start at all → DHCP(:67) and DNS(:53) dead site-wide for ~8.5 h (mainframe + ~a dozen tailnet devices offline). Fix uci delete dhcp.telep_tv; uci commit dhcp; /etc/init.d/dnsmasq restart. Before adding any dhcp-host, uci show dhcp | grep <ip-or-mac> first. The AirPlay telep-tv receiver IS the mainframe, so its telep-mainframe reservation already covers that MAC/IP — a separate reservation is moot and is what caused the outage. Recovery when LAN is down: router ssh root@100.69.112.32, mainframe ssh levander@100.115.209.87

• 2026-08-15-admin-portal-passkey-olivetin — ⭐ oauth2-proxy 500 email in id_token isn't verified is a self-hosted-account problem, not a broken login — a Pocket-ID account you registered yourself has email_verified=0 (nothing verifies your own address) and oauth2-proxy rejects unverified emails by default; fix OAUTH2_PROXY_INSECURE_OIDC_ALLOW_UNVERIFIED_EMAIL=true. ⭐ Host key verification failed/Permission denied from a container’s ssh action can be a WRONG-HOME problem, not a stale host key — jamesread/olivetin runs as USER olivetin (HOME /home/olivetin) while the key + known_hosts were mounted at /root/.ssh, so ssh found no identity and no known_hosts; fix = user: root on the service AND drop :ro on the ssh mount (so accept-new persists new host keys). ⭐ An oauth2-proxy UPSTREAMS of 127.0.0.1:<port> is the PROXY container’s OWN loopback, not the app → 502; use the compose service name (http://olivetin:1337). ⭐ Empty oauth2-proxy approval_prompt falls back to force → OIDC invalid_request — set OAUTH2_PROXY_PROMPT=login; and Pocket-ID clients require PKCE (OAUTH2_PROXY_CODE_CHALLENGE_METHOD=S256). ⭐ Caddy grabbing :443 on all interfaces collides with Tailscale (100.115.209.87:443) → bind 192.168.1.123 in the site block; the tls { dns cloudflare … } block must be MULTI-LINE (single-line won’t parse); systemctl restart (not reload) after cert/env changes. ⭐ OliveTin runs every action’s shell: string through Go text/template BEFORE executing — a literal {{ }} (e.g. docker ps --format "{{.Names}}") is read as an OliveTin template var → can't evaluate field Names in type *tpl.actionTemplateContext; avoid {{ }} in shell: (or escape as Go-template literals), or keep logic out of YAML and call a script over ssh (telep-selftest). ⭐ From the mainframe the router is reachable ONLY on its LAN IP 192.168.1.1, NOT its Tailscale IP 100.69.112.32 (ping 100.69.112.32 from the mainframe times out) — yet OliveTin’s OWN container reaches the router fine on 100.69.112.32; the correct router address depends on WHERE the command runs (host telep-selftest → .1; OliveTin container → tailnet IP)

• 2026-08-11-ruview-esp32-csi-real-hardware — ⭐ EXPOSE is not -p — RuView’s CSI ingest on UDP 5005 was only EXPOSEd, so the port was unreachable and no sensor could ever have worked; ports cannot be added to a running container, it takes a recreate (check for volumes first — this one had none). ⭐ A prebuilt firmware binary can silently disable the feature you want: on a display-less ESP32-S3 (WROOM-1/DevKitC-1) the ADR-045 runtime panel probe false-positives with no TCA9554 + floating QSPI pins → display_is_active() returns true → main.c skips the MGMT+DATA promiscuous upgrade → CSI yield = 0 pps with no error; fix is the compile-time sdkconfig.defaults.devkitc overlay (# CONFIG_DISPLAY_ENABLE is not set), and the prebuilts were 2 versions stale anyway. ⭐ An ESP32 dev board’s two USB-C ports are not interchangeable — the UART bridge (CH343 1a86:55d3) flashes because it drives DTR/RTS; the native USB/OTG port (303a:*) failed No serial data received on both default_reset and usb_reset; flashing from Docker with --device /dev/ttyACM0 also sidesteps levander not being in dialout. ⭐ A “421 Misdirected Request” from your own app is Host-header / DNS-rebinding validation, not a proxy fault — add the new hostname to its allowlist. ⭐ A container-created data dir is root-owned → a user curl into it reports http=200 size=0, i.e. success status, zero bytes written. ⭐ A headline model accuracy may not measure what you assume — RuView’s “82.3%” is temporal-triplet representation accuracy, not presence accuracy (48 KB of weights, 12 min of training), and the authors retracted an earlier “100% presence” figure after finding it was measured on a single-class recording. ⭐ A sensor that never returns to baseline probably has no baseline — variance never dropping below ~200 in any condition traced to --calibrate (empty room) never having been run. ⭐ A test the tester can contaminate is not a test — three walk-tests were all invalidated by someone moving during the “still” controls; the fix was an overnight unattended recording

• 2026-08-09-xiaomi-c302-tailnet-facetime-cam — ⭐ Xiaomi cloud login 70016 登录验证失败 is a WRONG-REGION error, not wrong credentials — the request fails before the captcha/2FA challenge, so set the account region (de here) first; go2rtc’s WebUI login handles captcha + email 2FA once the region is right. ⭐ A go2rtc xiaomi:// source needs a real cloud LOGIN for a per-connection key — you can’t feed it a pre-extracted token, so the route is inherently NOT air-gappable (compensate physically: relay + firewall per-IP scoping). ⭐ P2P across a VLAN needs the LAN RETURN path — without cams→lan go2rtc logs read udp i/o timeout + 0-byte frames. ⭐ OBS RTSP Media Source can’t decode H.265 (black) → transcode, or better use WebRTC (real browser decodes H265 natively). ⭐ OBS Media Source black while VLC works = uncheck “Use hardware decoding” on macOS. ⭐ OBS’s built-in browser (CEF) renders WebRTC gray (known limitation) even though the same page is sub-second in Safari/Chrome → use a real Chrome app-window + OBS macOS Window Capture (keep the window un-minimized). ⭐ Latency floor is the TRANSPORT, not the cam — WebRTC sub-second ≫ MSE ~1s ≫ RTSP+transcode. ⭐ network_mode: host makes go2rtc advertise docker/incus bridge IPs as WebRTC ICE candidates → pin explicit candidates:. ⭐ No local PTZ on the C302 — no miio endpoint (UDP 54321 unreachable), motor is Mi-cloud only; auto-framing = an OBS face-track plugin. ⭐ PiotrMachowski’s tokens-extractor doesn’t save its console output — | tee

• 2026-08-08-ups-shed-restore-hardening — ⭐ Coupling “restore shed services” to only the NUT ONLINE event is a trap — if an outage outlasts the battery, the box does a graceful poweroff and the mains-return happens WHILE IT’S OFF, so upsmon never observes OB→OL and the restore handler never runs → shed services (here kb-qdrant, explicitly docker stopped) stay down and dependents crash-loop (KB looped 2360×). Fix pattern: extract restore into an idempotent script and also run it from a boot-time oneshot reconcile service so the box self-heals regardless of whether the event fired; give critical containers --restart unless-stopped; make dependents wait for readiness (poll 6333/readyz ~60s) instead of crash-looping. ⭐ sudo tee (and heredoc/SSH-stdin writes) can SILENTLY DROP a script’s #!/usr/bin/env bash shebang → the file execs under dash and chokes on bash-isms (here a bash array); transfer shebang-bearing files as base64 and decode on the far side. (Same class of first-line/quoting hazard as this homelab’s other remote-write gotchas)

• 2026-08-12-krakensdr-doa-rig — ⭐ A tool that conda activates in its start script silently launches under SYSTEM python over SSH/systemd — conda isn’t on a non-interactive PATH and ~/.bashrc isn’t sourced, so conda activate fails silently and the app crashes with a ModuleNotFoundError that looks like a missing dependency; source .../etc/profile.d/conda.sh && conda activate <env> explicitly before the script. ⭐ “Overdrive”/over-gain status ≠ a power problem — the KrakenSDR’s “Power Level: Overdrive” is RF ADC clipping (lower the gain); a genuinely weak PSU shows as USB dropouts in dmesg instead. ⭐ A device you expect to have a serial controller may not — the KrakenSDR is just 5× RTL2838 on an internal USB hub with NO CH340/serial (cal is over tuner GPIO); a CH340 on the bench was an unrelated ESP32. ⭐ A GitHub org path can 404 out from under you — krakenrf/krakensdr_pr moved to krakensdr_suite (dropping passive radar); use the mfkiwl/krakensdr_pr mirror. ⭐ A doc’s pinned dependency version can be wrong for your platform — the KrakenSDR docs say dash_bootstrap_components 1.1.0 (needs dash≥2) but this trixie/py3.9 stack needs 0.13.1, plus dash_devices which the UI imports as dash. ⭐ A URL without an explicit non-80 port looks like a firewall/ACL problem — kraken-rig.taild4189d.ts.net (no :8080) → port 80 → connection refused; the :8080 was just missing. ⭐ Antenna length mismatch is common-mode across identical elements → it does NOT corrupt DoA bearings, so you can legally test on a different band than the target. ⭐ RTL-SDR always shows a fake DC spike at the exact center frequency — offset-tune to tell a real signal from the artifact

• 2026-08-08-telep-master-agent-impl — ⭐ The Claude app’s Code tab shows the auto-SPAWNED session name, not the device --name — claude remote-control --name control-plane-telep-mainframe --spawn same-dir surfaces as e.g. …-velvety-star in the app; don’t look for the --name string (it identifies the device/host, the visible session is the spawned one). ⭐ A host harness copied from the Mac references Mac-only tools (claude-mem, auto-memory, obsidian CLI) that don’t exist on the box — path-rewrite the vault path in the subagent defs and rely on graceful degradation to file I/O + git. ⭐ Pre-seeding trust dialogs (~/.claude.json) widens launch, not authority — keep tool permissions ACTIVE (no --dangerously-skip-permissions); on a passwordless-sudo box an approved action is instant root, so approval IS the guardrail

• 2026-08-30-telep-tv-mdns-vanishes-after-uxplay-restart — ⭐ A service that stays active can still stop advertising on mDNS — uxplay 1.71 publishes its AirPlay/RAOP records via avahi-compat-libdnssd → avahi over D-Bus, and across a uxplay restart (or avahi/D-Bus hiccup) the compat layer doesn’t reliably re-register; uxplay keeps running while _airplay._tcp/_raop._tcp are silently gone (avahi is fine — it still advertises _nut._tcp / the printer). Diagnose with avahi-browse -rt _airplay._tcp returning nothing. FIX = order matters: restart avahi FIRST, sleep 2, then uxplay — so uxplay registers against a fresh avahi (bouncing uxplay alone can re-hit the race)

• troubleshooting — 2026-08-08: AirPlay dead after UPS switch + reboot — ⭐ (2026-08-08) AirPlay “appears but casting does nothing” = avahi announcing the service on ALL interfaces. With allow-interfaces commented out, avahi’s default advertises telep-tv.local on docker0 (172.17.0.1), the incus/veth bridges (172.18/19.0.1) and loopback (127.0.0.1) alongside the real LAN 192.168.1.123 — Apple clients latch an unreachable record. FIX: pin allow-interfaces=enp5s0 under [server], restart avahi; avahi-browse -rtp _airplay._tcp should then show ONLY the LAN record. ⭐ AirPlay “doesn’t appear at all” right after a reboot = a DHCP IP flap dropping the mDNS group — a dynamic noprefixroute lease briefly withdrawn+re-added makes avahi log Leaving mDNS multicast group ... Interface no longer relevant (here a ~35s dead window); the durable fix is a DHCP reservation for the box’s MAC so the address never flaps. ⭐ tcpdump was NOT installed on telep-mainframe — network/mDNS debugging silently returns empty output; verify tcpdump exists FIRST before trusting a “no packets” result on this host

• 2026-08-06-kb-marker-chunker-bug-and-ocr-bakeoff — ⭐ “unsearchable pages with a generic H1 and no component name” can be a document-SPLITTER bug, not bad OCR — a page-break emitted BETWEEN a component header and its procedure makes the # -split start a new page at the generic sub-heading, stranding the real name; the header survives verbatim in marker’s RAW output, so recovery is a deterministic text lookup, NOT a re-OCR. ⭐ A full re-OCR is the wrong lever — GOT-OCR2 fixes catastrophically-mangled spec tables but REGRESSES clean prose (char noise) + emits LaTeX not GFM + is useless on line-art; reserve vision re-OCR for the few genuinely destroyed table/code pages. ⭐ Vet an “OCR model” before installing — JustVugg/colibri is actually an MoE-LLM inference ENGINE, and DeepSeek-OCR needs flash_attn + transformers 4.4x (won’t install against 5.x)

• 2026-08-06-kb-embedding-classifiers-system-tags-and-image-types — ⭐ A cosine-to-prototype classifier silently loses items when a prototype OMITS their vocabulary — the Fuel prototype had no idle-air-control terms, so IAC-valve pages scored closer to Cooling and were mistagged; fix = augment the prototype’s text, not the assignment logic. ⭐ SigLIP softmax stays LOW when the classes look alike — on a corpus of near-identical scanned B&W line art (line-drawing vs wiring vs flowchart), a 0.55 gate rejects almost everything; ≈0.35/0.10 is right. Expect low absolute confidences and don’t read them as a broken model

• 2026-08-06-kb-build-backup-dir-leak-and-tsb-diagram-restore — ⭐ A site builder that AUTO-DISCOVERS every dir under the content root will happily serve your backup dirs — build_content.py picked up *.bak* (e.g. kick-fix.bak-linkrewrite) → ~1646 stale pages (of 4566) went LIVE; any content-tree walker must exclude .bak (or keep backups outside the content root). Same class of stale-content bug as the earlier “re-ingest must rmtree first or you get ghost pages”

• 2026-08-06-kb-orphan-heading-recovery-subsystem — ⭐ When auto-fixing metadata, GATE on confidence — a WRONG value is worse than a generic one — heading recovery auto-applied only 89/235 and staged 146 uncertain, because attaching the wrong component name actively misleads search (strictly worse than a generic Removal and installation). ⭐ Re-embed in place: same point ID, update vector + text payload, PRESERVE the other keys so nothing downstream breaks; keep it reversible with .bak files + a changelog JSON

• 2026-08-06-power-root-cause-gpu-12v-connector-stress-test-pass — ⭐ “resets under load, cool temps, no MCE” can be a physically UNPLUGGED GPU power connector, not the PSU or the mains — the RTX 3080 had one of its 3 separate (non-daisy-chained) 12V PCIe cables disconnected; a graduated stress test after reconnecting passed clean. ⭐ A CPU stays pinned ~125W under sustained load because PL1=min(125,PL2) — PL2 only applies in the brief boost window, so to actually stress the CPU rail you raise PL1, not PL2. ⭐ An FP32 matmul GPU stress load is compute-bound and won’t hit a high power cap (never reached 450W) — pick a load that saturates the rail you’re testing. ⭐ nvidia-smi -pl / MSR power caps are NOT persistent across reboot — the box reverts to the mobo’s uncapped CPU default + GPU stock cap; a persistent profile needs a boot-time unit

• 2026-08-06-frigate-detect-record-jam-cpu-starvation — ⭐ Frigate does NOT self-recover from a CPU-starvation jam — a sustained all-core load starves the detect+record processes, the frame queues + record-segment cache jam, and they stay jammed even after the load clears (container back to ~10% CPU). The signature in /api/stats is normal camera_fps (~5) but global detection_fps=0.0 with process_fps ~0.1 and skipped_fps ≈ camera_fps, plus record.maintainer “Too many unprocessed recording segments” spam; sudo docker restart frigate clears it. per-camera detect_fps=None is a display quirk in this build — trust the GLOBAL detection_fps. This is the SERVER-side failure (restart fixes it), unlike the camera-side wedged-RTSP one where a restart does nothing

• 2026-08-06-kb-wiring-gallery-native-res-fix-and-astro-caching — ⭐ Low-res gallery images were a downstream THUMBNAILING step, not the OCR — marker extracted native-res crops, then gen_thumbs.py shrank them to 320px/q72 and the gallery served the thumbnails; the fix was to serve the marker-native originals (image_abs in gallery_items.json), ~2.5× sharper. ⭐ Scan DPI is the hard ceiling — 150 dpi source ⇒ ~822px is the real limit; a 300dpi re-render is pointless upscaling, and marker *_meta.json has no per-figure bboxes so you can’t re-crop without re-running marker. ⭐ Serve fingerprinted assets Cache-Control: immutable and HTML no-cache to stop stale-HTML/missing-image after an atomic redeploy swap

• 6. 2026-08-06 UPDATE — KB fixkick expansion LIVE, search-index PARKED, brownout root-cause upgraded — ⭐ (2026-08-06) The canonical KB builder is ~/kb-astro/build_content.py, NOT convert.py — build_content.py is a full regen (rebuilds section indexes + auto-discovers folders); convert.py is a PARTIAL builder that produces a stale/incomplete site, do NOT use it for the real build. ⭐ Heavy CPU load TRIGGERS the brownout/reboot on this marginal mains — a full 24-core embedding reboots the box; CPU-capping to ≤500%/≤5 cores (CPUQuota=400%) stayed stable, so keep ALL heavy CPU/GPU work capped until the UPS is in (upgrades the “unstable mains” finding from correlation to confirmed cause→effect). ⭐ An empty TaskList does NOT mean a long background agent finished — a crawl agent ran ~83 min across reboots while TaskList showed no tasks, causing 3-way overlapping KB writes; rely on completion notifications and never run two site-rebuild/deploy agents at once. ⭐ New pages render before they’re searchable — hybrid.py builds its BM25 index FROM the Qdrant collection, so a newly-added page becomes searchable (keyword + dense) only after its embedding completes; index_new.py is resume-safe (deterministic IDs, skips already-embedded), relaunch after each reboot until 0 remaining

• 2026-08-05-power-root-cause-nvme-damage-ups-kb-handover — ⭐ Do NOT run heavy/write-heavy builds on a box with unstable mains — power-loss-during-write accumulates NVMe media errors (already 1617). Mitigation patterns used: resumable builds via an on-disk cache (Astro webp cache resumes from ~1455/5124 images) + atomic, rollback-able cutovers (swap the site/ dir, keep a .bak) so a cut costs minutes, not the whole job. ⭐ A crash signature of normal logs then INSTANT silence (no warning) = external power loss — a thermal/PSU-protection shutdown logs FIRST; the absence of any preceding line is the diagnosis. ⭐ The recurring nvidia-drm-drv.c:1220 nv_drm_revoke_modeset_permission kernel WARNING is known-benign cosmetic (fires on Xorg/modeset start), unrelated to crashes — don’t chase it

• 2026-08-04-telep-mainframe-tldraw-whiteboard-board-kiosk — ⭐ tldraw 5.2.5 renders a BLANK canvas over HTTPS without a paid license. Its LicenseManager.isDevelopment is only true when protocol is NOT https, OR host is loopback, OR NODE_ENV != production — so a plain-HTTPS non-loopback deploy with no licenseKey gates the canvas blank. Workaround = serve it over plain HTTP (tailscale serve --http), which is still WireGuard-encrypted over the tailnet (leaves a small “get a license” watermark); a real HTTPS fix needs a purchased tldraw license. ⭐ A kiosk that stops camwall.service must launch its OWN matchbox WM — matchbox (which fullscreens windows) is a child of camwall’s session, so without relaunching it chromium comes up as a floating half-window, not fullscreen. ⭐ chromium-as-root needs --no-sandbox (mandatory) and --force-device-scale-factor=2 to be legible on a 4K TV. ⭐ Because Xorg :0 is owned by the standalone camwall-x.service (not camwall), the kiosk attaches as a root client with just DISPLAY=:0 and no XAUTHORITY

• 2026-08-04-telep-mainframe-airplay-receiver-uxplay — ⭐ If a systemd service runs xinit directly, it OWNS the Xorg server — stopping the service kills X for everything. camwall.service ran xinit ... -- :0, so it couldn’t be stopped to hand :0 to a casting app without killing the whole display; fix = split Xorg into a standalone persistent camwall-x.service (xinit → sleep infinity, no WM) and make the grid a mere client (DISPLAY=:0 + BindsTo). ⭐ Don’t GUESS a tool’s log strings for state parsing — verify against a live session. The uxplay handoff first parsed guessed Open connections/TEARDOWN lines that uxplay 1.71 never prints on a macOS mirror stop → camwall never returned; the real markers are Accepted <ip> client on socket (connect), Begin streaming to GStreamer video pipeline (start), Connection closed for socket / raop_rtp_mirror->running is no longer true (end). ⭐ And macOS-verified markers may still differ for iPhone. ⭐ uxplay -fs doesn’t fill the panel with glimagesink (renders 1:1 top-left) → xvimagesink + xdotool window-resize

• 2026-07-31-telep-mainframe-ax210-ble-scan-broken — ⭐ A BLE scan that sees ZERO devices — not even stray phones — is an RF/antenna symptom, not a firmware bug. The AX210’s controller reported UP RUNNING and le support (chip init needs no antenna) while receiving nothing, because no antenna was plugged into the M.2 card’s u.FL connector (WiFi+BT share it). btusb reload / USB rebind / firmware-currency / kernel bump / reboot were all the wrong layer. Check physical RF (antenna) FIRST when a radio is “up but deaf.”

• 2026-07-30-obd2-elm327-bridge-rfcomm-fix — ⭐ A listening TCP port ≠ the underlying link is up — the ELM327 bridge accepts on 35000 even when the Bluetooth RFCOMM channel to the dongle never opened; always check bridge.err, not just lsof. ⭐ A bus-powered BT dongle that power-cycles rots the macOS SDP cache — after ignition-off the ELM327 re-advertises < Braille ACL > instead of Serial Port, so an SDP channel-ID lookup fails and the bridge falls back to (dead) RFCOMM channel 1 (kIOReturnError); the cure is a device-scoped blueutil --disconnect/--connect (not toggling the whole BT stack), then bounce the service to force a fresh SDP query — restarting the bridge ALONE re-runs the same failing lookup. ⭐ It is NOT a /dev/tty.* claimant problem, and blueutil --connect can block to a 2-min timeout (poll with non-blocking --is-connected)

• 2026-07-29-kb-wiring-extraction-v1-complete — ⭐ Naive CV path-tracing is a NO-GO on 1632px wiring scans — skeletonizing the raw binary gives 161 false junctions / 502 segments because TEXT (titles/labels/fuse ratings) and COMPONENT SYMBOLS (fuse coils/switch boxes) skeletonize into dense false-junction clusters (line-following works, connectivity doesn’t); mask text (surya auto-detect) + symbols (vision boxes) BEFORE skeletonizing → 161→37 junctions (-77%), symbols being the bigger contaminant. ⭐ Scan resolution is the ceiling, not the model — 1632×808 @ ~9 systems/page fails wire-color stripe letters (B/R vs B/Bl, ~60%); 300–600 DPI or single-circuit pages → ~90%+. ⭐ A diagram-page search win is real but scoped — the pilot went from unfindable to #1 within its manual, NOT global top-50 (a terse structured list can’t out-rank prose service pages); frame the goal as “diagram pages become retrievable”. ⭐ Vision+fusion is subagent-driven at ~13 min + ~110k tokens PER PAGE — that token cost, not the CV, is the binding scale constraint

• 2026-07-29-kb-semantic-search-frontend-complete — ⭐ Community mkdocs-material’s client-side lunr search is fundamentally unfixable at scale — a 5.4 MB / 6683-section index built in-browser hangs on “Initializing search”; prebuild_index is Insiders-only (removed from community), and dropping the HU search lang either doubles the index (reconfigure_search:false→13366 docs) or removes the HU site → no config path exists, the only fix is a server backend. ⭐ Installing CPU-torch INTO the app venv let the existing semantic backend run in-process (no new service, ~1.63 GB RSS) — lazy-singleton model + a startup warmup keep import app cheap. ⭐ Graceful degradation: Qdrant down → /api/search 503 but home + manual pages still 200 and the service didn’t crash (NRestarts=0) — the mkdocs site is independent of the search backend

• 2026-07-25-kb-consolidated-gen-phase3-complete — ⭐ Image basename collision across manuals silently swaps in the WRONG diagram — every marker-OCR’d manual reuses names like _page_62_Figure_16.jpeg, so a consolidated article pulling images from multiple manuals copies the wrong figure (a real accuracy failure); fix = rewrite refs to SOURCE-UNIQUE names (safe_slug(manual_id) + "__" + basename) + an image_map before generation. ⭐ A number-token spec-check’s “false flags” are mostly REAL verbatim-deviations (model reformatting 1,000→1000, 40→40.0, which the never-invent prompt forbids) → keep it ADVISORY, never a publish gate; human review is the backstop. Also: start_worker must be idempotent or concurrent workers break serialization; safe_slug guarantees a non-empty path component so a degenerate label can’t rmtree the whole consolidated/

• 2026-07-24-kb-semantics-phase2-complete — ⭐ HDBSCAN’s min_samples defaults to min_cluster_size → with min_cluster_size=2 unset, everything becomes noise (0 clusters); set min_samples=1 explicitly. ⭐ Cross-source “span” inflates when your id is <folder>/<manual> and one archive is split into many subfolders (kick-fix = 1 source but ~25 subfolders) → rank span by distinct TOP-LEVEL folder (manual_id.split("/",1)[0]), stoplist boilerplate labels, and treat 51% noise singletons as EXPECTED for a dedup goal; whole pipeline is READ-ONLY scroll → local clusters.json

• Gotchas — wifi-ifaces declare network='lan wan' (inert, but one step from bridging APs onto WAN); 8-char PSK; double-NAT; /rom at 100% is normal for squashfs

• filestash — Windows NTFS mount is manual (not fstab, won’t survive reboot); RW would corrupt the live/hibernating Windows install; iOS Private Relay breaks MagicDNS in Safari

• The Windows drive — NVMe device nodes flip across reboots (the Windows NTFS partition moved nvme1n1p2 → nvme0n1p2): lsblk -f every time, UUID= in fstab; and docker restart filebrowser after mounting or /srv looks empty in the container

• 2026-07-17-intruder-alarm — sleeping iOS phones stay WiFi-associated but stop answering ARP (arp-scan alone false-arms; assoc table saves it); failed scans skipped so blips can’t false-arm; camera SSID telep-cc (phy1-ap1) leaked the camera as an enrollable device; enabling router dropbear rotated its SSH host key

• 2026-07-18-birdseye-nvenc-blue-glare — camwall “blue glare” is NVENC-encode corruption of Frigate’s birdseye (distinct from the NVDEC decode magenta bug); wedges even at GPU idle; restart Frigate to clear, blank hwaccel_args for a durable libx264 fix; isolate by grabbing a still per pipeline layer

• mp4-missing-moov-forensics — moov atom not found does not always mean “untrunc can fix it”: a crashed recorder + NTFS preallocation leaves an mdat full of stale deleted-file bytes (max entropy, zero valid NAL chains) = unrecoverable. Verify the payload parses before investing in recovery tooling; Homebrew has no untrunc formula (2026-07)

• 2026-07-22-facekom-vpn-exit-node — a wrong peer IP looks exactly like a bad CA bundle when two OpenVPN servers share a subnet+port; a labelled/tab-delimited vendor auth.txt (jelszó:<TAB>pw) silently becomes part of the password → AUTH_FAILED; openvpn-client@.service’s process limit makes up/route-up scripts fatal (fork errno=11); Tailscale exit nodes exclude RFC1918; pin curl --resolve when testing route changes or anycast fools you; diff cert fingerprints before believing a “new” vendor bundle

• 2026-07-22-wifi-usage-strip — mpv destroys osd-overlay entries when the IPC client disconnects (and still replies {"error":"success"}) → the connection must be persistent and the event stream drained; station byte counters are AP-perspective so tx bytes is the device’s download while the WAN isn’t inverted; camwall-reload restarts playback so it’s unusable for live data; SSH ControlMaster masks a deleted key (drop the control socket to test); subprocess.run(text=True) can raise UnicodeDecodeError — a ValueError, not caught by except (SubprocessError, OSError)

• 2026-07-23-mains-power-shutdowns — a journal that stops dead with no Stopping/panic/thermal trace = power was physically removed, not a software or OS shutdown; to separate a PC PSU fault from a mains event, cross-check an independent always-on device’s boot time (date +%s minus /proc/uptime) — same-timestamp death = mains; a desktop that “stays off” while other gear silently recovers is the BIOS Restore on AC Power Loss = stay off policy, not a new failure

• 2026-07-23-frigate-viewer-alert — X-Forwarded-For is the LAST quoted nginx field before request_time=, not the first — leftmost-match is spoofable via a " injected into the User-Agent/Referer, so take findall(...)[-1]; never spawn tailscale status (or any ~10s subprocess) per log line — under an active viewer that’s a self-DoS, rate-limit miss-refreshes to ≤ once/60s; reset a docker logs -f reconnect backoff only after a line is read, and --since 0m so a daemon restart doesn’t replay stale alerts

• 2026-07-24-luks-tpm-autounlock — Debian 13 stock initramfs (initramfs-tools) ignores systemd-cryptenroll TPM2 tokens (update-initramfs warns ignoring unknown option 'tpm2-device' and still prompts at boot) → use clevis (clevis-initramfs+clevis-tpm2, clevis luks bind … tpm2 '{}') for TPM LUKS auto-unlock on Debian; adding ANY keyslot needs the existing passphrase (an agent can’t) and is slow, so an impatient Ctrl-C orphans a systemd-crypt* process that holds the header lock AND /dev/tpmrm0 and blocks the next bind (SIGKILL it, verify fuser/lsof clear); back up the LUKS header before touching keyslots

• 2026-07-24-knowledgebase — tailscale serve --https fails “no TailscaleVarRoot” (TLS handshake 000 even with -k, though TCP/ping succeed) when the second tailscaled has only --state=<file> — it has no var root for the LE cert; fix is --statedir=<dir>. Also: re-ingesting the same folder+name must rmtree the destination first or stale orphan section pages/images linger as ghost pages in nav/search; a bare mkdocs folder URL 404s by design (section pages have no folder index); the # split is on the ORIGINAL doc’s h1 count (few h1s → few coarse pages)

• 2026-07-24-kickfix-ingest — markdownify silently drops <img> inside <table> cells (old sites use tables for layout) → .unwrap() the layout containers first + rewrite src to the copied basename in place; “see photo” <a href="X.jpg"> text links are separate from inline <img> and were 365/664 broken → also walk <a href>, copy + rewrite existing images; URL-decode + resolve ../ cross-topic for both src and href; clean-regenerate each run

• 2026-07-24-psu-load-resets — TWO distinct power problems: mains outage (router dies too → UPS) vs PSU-can’t-sustain-load resets (router stays up → serialize/bigger PSU, a UPS does nothing); the router-uptime cross-check is the discriminator (same dead-stop journal signature either way); the default 16-thread CPU embedding reset the box in 2–6 min, OMP_NUM_THREADS=4 + serialize fixed it

• shell-gotchas-pgrep-self-match — pgrep -f/pkill -f match your own command line (worst over SSH, where the whole remote string is one argv): a completion monitor reported RUNNING 2.5 h after the job ended, and a pkill -f killed its own shell → exit 143 with the rest of the script silently skipped. Use PIDs, the [m]atch bracket trick, or a real artifact/sentinel

• The CUDA driver mismatch — torch does not error on a too-new CUDA build, it falls back to CPU; and pip install --index-url .../cu126 is a silent no-op unless you pip uninstall -y torch first (the version constraint is already satisfied). Debian 13 PEP 668 also blocks bare pip install → plain python3 -m venv

• Since last handover (2026-07-24) — on OpenWrt/busybox use setsid, not nohup (absent), to detach a self-reverting service swap (keep a persisted /overlay binary backup + auto-revert so a bad swap can’t lock you out of the router); a hand-installed package (router tailscale 1.98.9) is out-of-band so opkg still records the old version and a future opkg upgrade can clobber it

• Gotchas — the nine that cost real time: Secure Boot vs the DKMS-signed NVIDIA module; TensorRT detector dead on amd64 in Frigate 0.17; YOLOv9 export broken by torch ≥2.6 (pin 2.5.1); /cam1/ and /cam2/ are the same lens; Tapo silently truncates long passwords (and is Basic-auth only); Frigate’s DB lives at /config/frigate.db; birdseye draws no camera names; birdseye grid geometry follows the canvas aspect; iOS Private Relay vs MagicDNS

• 2026-07-28-ipv6-slow-internet — advertising IPv6 to a LAN with no working v6 upstream makes macOS “slow” (RFC 6724 / Happy Eyeballs stalls 2–7 s per dual-stack site before IPv4 fallback); diagnose with curl -w (connect slow, DNS instant) + curl -4 vs curl -6; fixed-wireless + double-NAT rarely carries v6 so don’t advertise it

• 2026-07-28-camwall-4-substream-composite — Frigate birdseye flip-flops between 2 and 4 cameras so it’s unusable as a fixed >2-cam wall source; composite substreams directly; mpv movie= rtsp URLs need colons escaped and can’t take rtsp_transport (benign 461); a movie= pane can freeze on go2rtc restart (watchdog covers only the primary input)

• 2026-07-28-frigate-notify-camera-exclude — frigate-notify’s koanf SILENTLY IGNORES misplaced/unknown keys — “Config validated! / App ready!” does NOT prove a filter works (a misplaced cameras.exclude validated and still alerted); verify suppression against a real event (Event dropped - Camera Excluded); the working key is frigate.cameras.exclude by camera id

• 2026-07-31-camwall-lag-24ghz-wifi-bufferbloat — ⭐ When lag survives a full player swap (mpv → go2rtc MSE/chromium), stop tuning the player — the bottleneck is upstream. Here it was 2.4 GHz WiFi bufferbloat: high latency + huge jitter + 0% packet loss = queueing under load, not a lossy link; confirm the interferer is self-congestion via iw survey (98.9% busy) + a clean noise floor (-106 dBm) — which means changing WiFi channel is useless. A local camera pinging 350–420 ms instead of <5 ms is the smoking gun. Fix is capacity (Ethernet/PoE or 5 GHz or less bitrate), never player config

• 2026-08-03-cam-stall-recovery-and-casino-alert — ⭐ A wedged RTSP stream is fixed at the CAMERA, not the server — go2rtc/mpv/Frigate won’t auto-recover a Tapo whose RTSP session slots got stuck when a dnsmasq/firewall restart dropped its weak WiFi mid-stream (a fresh ffprobe still works, but the stale sessions never close); the recovery is an ONVIF SystemReboot + camwall.service restart (mpv holds the last frozen frame). ⭐ pytapo reboot needs TP-Link cloud creds — an ONVIF/RTSP-only account (nvr42vhy1) must use the ONVIF device-mgmt service (port 2020) instead. ⭐ Rebooting an isolated (no-internet) camera loses its clock (no NTP) → keep a periodic ONVIF SetSystemDateAndTime cron. ⭐ busybox grep has NO --line-buffered → in a tail -F | grep log-watcher on OpenWrt, match INSIDE a while read line loop or alerts never fire

• 2026-08-03-telep-router-factory-reset-recovery — ⭐ Never blind-restart dnsmasq on the only gateway with a new/large config — a casino-blocklist deploy restarted it with a broken config, the agent died mid-run, and DNS went down site-wide → factory reset. Lessons: dnsmasq --test -C (test-parse) BEFORE bouncing the resolver; keep a sysupgrade -b backup (none existed → full rebuild, not a restore); apply network-critical router changes atomically so a mid-run crash rolls back instead of bricking resolution. Also: the log-rotate script was rewritten to truncate in place (no dnsmasq restart) so rotation can’t cause a DNS outage

• 2026-08-12-airplay-mdns-fix-printer-migration — ⭐ uci/ubus saying isolate:false does NOT mean the RUNNING hostapd conf agrees — on OpenWrt 24.10, _wireless_set_brsnoop_isolation in /lib/netifd/netifd-wireless.sh (~line 309) AUTO-INJECTS isolate 1 into the generated config for any BRIDGED, NON-ISOLATED AP whenever multicast_to_unicast is on (and it’s promoted to the top level as 1 by default), so /var/run/hostapd-phy*.conf ends up with ap_isolate=1 regardless of the uci isolate/multicast_to_unicast values → wired→WiFi mDNS (AirPlay) silently dropped. ⚠️ uci set …isolate='0' + wifi reload/wifi down;up/network reload all leave the conf BYTE-IDENTICAL — the uci option and changing the line-309 default both fail because the value is explicitly promoted, not merely unset. ⭐ When you suspect a generated config, DIFF the actual /var/run/hostapd-*.conf, don’t trust the uci/ubus abstraction. FIX = comment out the injection line + wifi down; wifi up; an explicit isolate=1 iface (the cams SSID) early-returns before that line so it STAYS isolated. 🔴 Patching a /lib/netifd/ system file is reverted by sysupgrade — reapply after every firmware upgrade (grep the line, line numbers drift). ⭐ CUPS libusb backend can’t claim a USB printer while the kernel usblp module holds it → unload + blacklist usblp (/etc/modprobe.d/blacklist-usblp.conf)

• 2026-08-31-obsidian-vault-qdrant-index-selection — ⭐⭐ Don’t threshold on a measured proxy when the thing you actually mean is already knowable. A link-density heuristic for deciding which vault pages to embed was tried and rejected: it split identical file roles incoherently — tatabanya TOPICS.md at 0.42 kept vs esp32 TOPICS.md at 0.67 dropped — because density tracks project size, not page quality. Excluding by filename/role (LOG.md, TOPICS.md = generated scaffolding) is both simpler and correct. ⭐ Generated index pages are actively harmful in a vector index, not merely useless: near-pure wikilink lists (homelab’s pair alone = 159 chunks at ~0.9 density) embed to noise and displace real answers in the top-k, while adding nothing — their content is derived restatement of notes already indexed. ⭐ Opt-out beats opt-in for privacy, but bites at the wrong granularity: one note tagged personal removes its entire project from the index — grep for the tag before debugging “missing” search results

• 2026-08-31-telep-kb-mcp-server — ⭐ A 421 Invalid Host header from a service behind tailscale serve is an APP setting, not a proxy bug. mcp ≥1.29 ships DNS-rebinding protection with an empty allowed_hosts (127.0.0.1 only) and serve faithfully forwards the original Host — so loopback works, the tailnet doesn’t, and the error body names no layer. Pass transport_security=TransportSecuritySettings(allowed_hosts=[...]). ⭐ Design lesson from the same note: an unknown-key error that returns {error, available} is better than an exception — it makes the recovery loop self-serve for a model, which is why the tool surface copies historian’s named-sources shape

• 2026-09-02-freecad-mcp-rpc-refused-gui-not-running — ⭐ On a tailscale serve / userspace-networking node, an OPEN port proves the PROXY is up, never the backend. nc -z cad…9875 reports open while nothing is bound on 127.0.0.1:9875 — the raw forwarder accepts the TCP handshake, fails to reach the backend, then drops. The discriminating probe is to speak a protocol: XML-RPC/http, XML-RPC/https and a raw TLS handshake all return RemoteDisconnected / SSLEOFError: UNEXPECTED_EOF_WHILE_READING (accept-then-close, zero bytes). Second lesson: Errno 111 is Linux ECONNREFUSED, macOS is 61 — the errno number alone localises the fault to the remote host. Third: “online in tailscale status” ≠ “SSH-able” — check the Tailscale-SSH-enabled machine list.

• 2026-08-31-nvidia-drm-host-crash-embedding-pass — ⭐ Total correlation is not causation, and a pre-existing trace is the tell: 3 crashes in 3 embedding runs looked damning, but the identical nv_drm_revoke_modeset_permission trace already appears from 2026-08-22, and the job is CPU-only torch that never opens a DRM fd — so it is a trigger perturbing a driver bug, not the caller in the stack. ⭐ Two traps that make such a hunt much worse: a OnBootSec= timer pointed at a crashing job = boot loop (disable it first), and pkill -f over Tailscale SSH matching the SSH wrapper’s own command string → you kill your own session and read it as another crash

• 2026-08-31-telep-mainframe-mce-hardware-fault — 🔴 /sys/fs/pstore is ALWAYS empty after an MCE panic and that is structural: efi_pstore_write runs in <#MC> context where the FPU is unavailable, kernel_fpu_begin_mask WARNs and the EFI variable write fails. Don’t try to “fix pstore” — netconsole is the only capture method that works

• 2026-08-31-telep-mainframe-mce-hardware-fault — 🔴 A one-shot thermal read proves nothing on a bursty box. core_id=16 looked hot at 54 °C once; continuous sampling gives avg 38.1 °C, mid-pack, with other cores peaking higher

• 2026-08-31-telep-mainframe-mce-hardware-fault — 🔴 A repeated, loud, plausible kernel WARNING is not automatically the cause. The nvidia_drm warnings appear on healthy boots and after the panic; two agents independently anchored on them and were wrong

• 2026-08-31-nextcloud-drive-code-server-runbook — 🔴 cadviewer is a MISNOMER — it handles all Nextcloud file actions, not just CAD. Deliberately not renamed: a Nextcloud app id is baked into install paths, enabled-apps config and asset URLs, so renaming costs a reinstall for zero functional gain

• 2026-09-02-freecad-mcp-gui-dispatch-timeout-modal-dialog — 🔴 ⭐⭐ A health endpoint that is designed to answer while the system is busy cannot prove the system is working. FreeCAD’s get_rpc_status returned gui_dispatch: healthy for 90 s at a time because it deliberately bypasses the very thread that was blocked — the port was listening, ping() was True, list_documents() returned cleanly, and every real call timed out. Ask what a green check actually exercises before trusting it; here only execute_code touched the blocked path. ⭐ Second lesson: a proxy layer flattens the backend’s error — the MCP said only “The operation timed out” while the raw XML-RPC call named the subsystem and the bound (GUI dispatch timed out after 90s). Re-run a timing-out call one layer down before theorising. ⭐ Third: an env var passed to docker exec loses to an export inside the entrypoint (AppRun line 13 QT_QPA_PLATFORM=xcb) — the override is silent, so read the launcher instead of permuting flags

• 2026-08-31-openchatcut-chatcut-deploy — 🔴 Vite 8 allowedHosts 403s the tailscale-serve hostname for BOTH UI and MCP (Blocked request. This host is not allowed) — the Vite equivalent of the FastMCP 421 Host-header trap; Vite 8 has no --allowed-hosts CLI flag, patch config/vite.config.ts (the server block is there, not a top-level config). Also: Vite ignores CRA-style HOST env → pass --host; onnxruntime-node postinstall downloads GPU libs on linux/x64 → ONNXRUNTIME_NODE_INSTALL=skip. 🔀 Editor trust-model trap (fixed 2026-08-31): the web editor 403 invalid request origin because state-changing requests need loopback socket + loopback Host + same-origin + Sec-Fetch — incompatible with tailscale serve. Fix = --network host (loopback-socket) + patch loopbackHost() to accept OPENCHATCUT_EDITOR_URL host (loopback-Host). Lesson: local-first apps behind serve fail in 3 Host/origin layers (allowedHosts, loopback-socket, loopback-Host) — same family as the FastMCP 421 trap

• 2026-09-02-lg-tv-network-control-presence — ⭐⭐ mDNS TXT records are the highest-yield way to identify an unknown device on the LAN — one avahi-browse -rt _airplay._tcp yielded exact model, firmware and MAC (model=43UP75003LF, manufacturer=LG Electronics, fv=p20.03.53.45, serialNumber=209MAAKHWU24_ac:5a:f0:8b:48:da), while SSDP M-SEARCH returned nothing at all and port scanning found open ports but could not name the device (contrast 2026-08-26-router-reflash-mac-5ghz-dfs-and-bambu-offline, where SSDP was the right probe — pick the discovery protocol the device class actually speaks). ⭐ Don’t size a display from xrandr on NVIDIA — 1600mm x 900mm is a placeholder, the panel was 43”. ⭐⭐ A client library’s connect() can have a PHYSICAL side effect — aiowebostv 0.10.0’s connect() always calls _check_registration(), so even a read-only status check against an unpaired TV throws a pairing prompt onto the screen; anything that “just checks state” must do a bare TCP probe first and refuse to open the WebSocket. Also 0.10.0 exposes no __version__ → importlib.metadata.version("aiowebostv"). ⭐ Watch for self-inflicted name collisions — telep-tv in mDNS is the mainframe’s own uxplay receiver, not the TV, and cost time as a red herring. ⭐ Debian 13 enforces PEP 668 → a venv is mandatory for any pip install on this host

• 2026-09-02-lg-tv-network-control-presence — 🐍 ⭐ Python idiom trap that hid a bug for several rounds: f"failed: {err or type(err).__name__}" tests the truthiness of the exception object, which is always True, so the fallback never fires and it interpolates str(err) — empty for exceptions raised with no message. Every failure printed pair failed: with nothing after it. Correct form is str(err) or type(err).__name__. Generalises: x or fallback on an object (exceptions, Path, most custom classes — all always truthy) never falls back; you must convert first.

• 2026-09-02-lg-tv-network-control-presence — ⏱️ Layered-timeout rule: an OUTER timeout must EXCEED the inner one, or it fires first and masks the real error. Concretely tv-http PAIR_TIMEOUT = 210 > tv PAIR_RECEIVE_TIMEOUT = 180. Any HTTP/subprocess wrapper around a slow operation needs its own timeout sized above the operation’s.

• 2026-09-02-lg-tv-network-control-presence — ⚠️ Don’t “fix” steady-state timeouts from post-wake symptoms. aiowebostv’s CONNECT_TIMEOUT = 2 is fine in steady state — measured 0/10 failures at 0.29–0.41 s over ten runs — and is only too short in the seconds after a device wakes. The right fix is a retry loop around the wake (CONNECT_ATTEMPTS = 6, CONNECT_RETRY_DELAY = 3), not a globally inflated timeout that makes every normal failure sluggish.

• 2026-09-02-lg-tv-network-control-presence — 🔌 webOS has TWO standby depths and only one of them closes the port. In Active Standby the TV keeps its network stack up and SSAP 3000 stays OPEN, so if not port_is_open(): send_wol() skips Wake-on-LAN entirely and falls back on SSAP power_on(), which LG sets honour unreliably — the whole of a reported “flaky power-on”. Send the magic packet unconditionally; it’s a harmless UDP broadcast when the device is already awake. Generalises: an open port is not proof a device is awake.

• 2026-09-02-lg-tv-network-control-presence — ⏳ A device can accept TCP seconds before its service will answer. After a real wake from deep standby, webOS opens 3000 well before SSAP responds — observed live: WoL woke the TV, then tv on, tv status and the input switch all raised TimeoutError within ~9 s while the set was booting perfectly and came up fine moments later. The commands reported failure while succeeding. Distrust a failure reported within seconds of a power-on.

• 2026-09-02-lg-tv-network-control-presence — 🤝 aiowebostv 0.10.0 gives a human exactly 10 SECONDS to accept the on-screen pairing prompt. Module globals in webos_client.py: CONNECT_TIMEOUT=2, RECEIVE_TIMEOUT=10, REQUEST_TIMEOUT=20, HEARTBEAT=5; _check_registration does ws.receive_json(timeout=RECEIVE_TIMEOUT) after the prompt appears. RECEIVE_TIMEOUT is NOT a WebOsClient.__init__ parameter (only connect_timeout is), so the only lever is monkeypatching the module global — which works because the constant is read at call time: webos_client.RECEIVE_TIMEOUT = PAIR_RECEIVE_TIMEOUT. Scope it to the pair command only (180), leaving normal commands responsive at 10 s.

• 2026-09-02-lg-tv-network-control-presence — 🏷️ Name a command after its INTENT, not after a port number at either end. The tv hdmi1 subcommand was removed rather than repointed to HDMI_2 — a command called hdmi1 that switches to HDMI 2 is a trap for the next reader. It is now tv camwall, backed by a single TV_INPUT = "HDMI_2" constant; tv hdmi1 exits 2 with unknown command.

• 2026-09-03-bambuddy-preheat-chamber-target-bug-and-clog-rca — 🚩 ⭐⭐ “Not available” is a CONTEXT DUMP, not a diagnosis — bambuddy logs Queue: printer %d not available - connected=%s, state=%s, awaiting_plate_clear=%s unconditionally as context; the real gate is if require_plate_clear and is_awaiting_plate_clear(...) (~L1991, ~L3320) and require_plate_clear was false, so awaiting_plate_clear=True could not have been the reason. Two more dead ends from the same session: state=FINISH is idle (idle = state.state in ("IDLE", "FINISH", "FAILED"), L3328), and a stale printing queue row wasn’t it either (busy_printers comes from SELECT printer_id FROM print_queue WHERE status='printing' — there were none). The authoritative signal was the preheat log lines. 🔧 Two more couplings worth knowing: pipelines cannot express preheat settings (print_queue has preheat_override/preheat_chamber_target_override, slicer_pipelines has neither) so a PC-FR pipeline run gets no chamber preheat at all; and zeroing a filament silently disables keep-warm for it (_apply_keep_warm: “Skips entirely for filaments that map to a 0 °C chamber target”)

• 2026-09-03-bambuddy-preheat-chamber-target-bug-and-clog-rca — 🌡️ ⭐⭐ NEVER auto-chain a temperature DROP between filaments — bambuddy has NO material-transition logic whatsoever (grepping the scheduler for material_change|filament_change|purge|previous_filament|last_filament returns only printer-state transitions; the queue will chain any two materials back-to-back with no guard). PLA→PC is SAFE (temp goes up, PLA residue melts and purges), PC→PC is BEST (batch them, chamber still warm), 🔴 PC→PLA is DANGEROUS — 280→220 °C, PC residue will not melt and carbonises, and this is what destroyed the H2S hot end. The cold end died separately, of PLA heat-creep caused by the 50 °C chamber / 90 °C bed bug above — the same spool killed both halves of the nozzle by two independent mechanisms. Enforcement: manual_start: true on the first PLA job after any PC job halts the queue for a human — that pause is the purge window. Contributing: PC-FR cannot be dried in the AMS — tray metadata says drying_temp: 80, AMS 2 Pro maxes at 65 °C

• 2026-09-03-bambuddy-preheat-chamber-target-bug-and-clog-rca — 🔌 ⭐ The Tapo P115 has NO lifetime energy counter over the local API — consumption_total returns None for every Tapo device, hard-coded in python-kasa’s kasa/smart/modules/energy.py. Any “total kWh” must be synthesized from daily values (here: persisted to /opt/tapo-bridge/state.json, monotonic and restart-safe, but starts at zero and under-counts if the bridge is down across midnight). ⚠️ python-kasa 0.10.2: Device.connect() has no credentials= parameter — they go in DeviceConfig(...) with DeviceConnectionParameters.from_values("SMART.TAPOPLUG", "KLAP", login_version=2, https=False). ⚠️ bambuddy’s generic rest backend defaults to POST — rest_method must be forced to GET. Units read from source, not guessed: power W, energy kWh, all multipliers 1.0

• 2026-09-04-starlink-wan-migration-dish-telemetry — 🔴🔴 grpcurl omits proto3 zero-values unless you pass -emit-defaults — popPingDropRate, fractionObstructed, currentlyObstructed, timeObstructed disappear exactly when they are zero, i.e. when everything is healthy, so a logger built without the flag looks fine and records nothing useful. More from the same session: 🚩 readyStates carries a legacy cady key that is permanently false on rev4_pez_proto1, so all(readyStates.values()) reports NOT_READY on a healthy dish (fix: explicit core set ("scp","l1l2","xphy","aap","rf")); 🚩 the dish reports noEthernetLink in alerts on EVERY row while ethSpeedMbps is 1000 — a Bypass-mode firmware artifact that makes the alerts column useless as a naive “any alert” trigger; 🚩 avg_prolonged_obstruction_interval_s can be the literal string NaN; 🚩 ping is useless as a tailnet reachability test (100.115.209.87 pings unreachable while routing fine over tailscale0 — ICMP filtered); 🚩 ip route get showing dev wan for a tailnet peer usually means an ACL denial, not a routing bug; 🚩 webapp.starlink.com is NXDOMAIN but is only a permissive CSP entry, not required; 🚩 grpcurl fails on :9201 because that port is gRPC-WEB (different framing — a raw POST returns 200 with proper gRPC headers, so it is healthy); 🚩 the Starlink mobile app cannot work in Bypass mode (it talks to the router, whose services are off) — use http://192.168.100.1; 🚩 CORRECTED 2026-09-05 — the TL-WA850RE is NOT a repeater: it is already a wired AP (SSID bandi, see [[telep-router#tp-link-tl-wa850re-added-as-a-wired-ap-ssid-bandi|TP-Link TL-WA850RE added as a wired AP (SSID bandi)]]). Its ~50 Mbps ceiling is a 100 Mb uplink port + single-band N300 radio + 2.4 GHz congestion, not repeater halving — the original claim was inferred from the model name, not measured

• 2026-09-07-session-handover — 🚩 conduwuit is ARCHIVED (Continuwuity = maintained fork, Tuwunel = competing live fork) but the rename is only skin deep — binary /sbin/conduwuit, data dir /var/lib/conduwuit, log modules conduwuit_*, admin bot @conduit:…, example config conduwuit-example.toml. 🚩 Distroless images cannot be docker exec’d ⇒ no shell/curl debugging and no meaningful healthcheck. 🚩 allow_announcements_check and allow_check_for_updates are ALIASES of one field — setting both is a fatal duplicate field that crashloops the container. 🚩 /.well-known/matrix/client 404s unless [global.well_known] client is set explicitly — it is NOT derived from server_name. 🚩 A RocksDB BackupEngine store is NOT an openable database and there is no restore command — materialise it back into a plain db/ or the backup is a paperweight. 🚩 matrix-nio only rotates megolm sessions on expiry — rotate manually on member-device change. 🚩 A persisted sync token makes incremental syncs return no rooms. 🚩 One sensor per AMS chamber ⇒ per-material humidity thresholds are STRUCTURALLY broken — the lowest threshold alerts forever regardless of what is loaded. 🎣 du on a RocksDB dir lies (76 MB reported vs 1.4 MB real — preallocated WAL + MANIFEST). 🎣 Excellent WiFi RSSI does not mean the device is reachable — the Tapo P115 sat at −39 dBm / SNR 64 with a valid lease while ping was 100% loss, ARP STALE and port 80 closed. 🔴 NEVER mutate matrix-nio’s shared users_for_key_query from another thread — sync_forever consumes it, and a double-consume makes keys_query() raise LocalProtocolError: No key query required., killing the session mid-send; let room_send handle key queries and inspect device_store read-only. 🚩 A dead Matrix session between two sends loses the second one SILENTLY — send related text + media in one coroutine and track which part already landed so a retry doesn’t duplicate it. 🚩 bambuddy’s thumbnail endpoints return the SLICER RENDER, not the camera capture — the finish photo exists only under /archives/{id}/photos/{filename}. 🎣 A confident root-cause hypothesis in a handover can be flat wrong — the “malformed info block” theory for the missing images survived a whole session before decryption showed the event was spec-compliant all along. 🔴 Filtering that lives in an UPSTREAM component does not protect the one downstream — cameras.exclude is a frigate-notify setting, the Matrix relay has no camera filter, and any direct POST to :8118 reaches the live alert room regardless of camera

• 2026-09-07-alert-source-inventory — 💡 four reusable rules from auditing every alerter. 🔴 curl -s without -f exits 0 on HTTP 4xx — any script treating curl’s exit code as delivery confirmation logs success while sending nothing (exactly why a token rotation would be silent for frigate-fps-watchdog and nut-outage-handler, which also discards the response body). 🚩 A “disabled but not deleted” rollback block can become LOAD-BEARING — frigate-notify’s telegram.enabled: false left token/chatid in place and three unrelated scripts now read their creds from it; disabling a feature does not mean its config is unreferenced — grep before deleting. 🔴 An alerter that shares a failure domain with the thing it monitors is not an alarm — UPS alerts over the WAN cannot survive a power cut; keep the alert path local to the failure. 🎣 “Never seen it fire” is only as strong as the state you kept — casino-alert’s cooldown state lives in /tmp and resets every boot, so its observation window proves nothing (unproven, not dead). ↔️ Also: migrating an alerter onto the relay needs NO relay change — any path is accepted as a source label with title/message or raw text, so it is a one-function URL swap to http://127.0.0.2:8118/<name>. 🔴 ⭐⭐ UPDATE 2026-09-07 — Continuwuity’s /_matrix/client/v3/rooms/{room_id}/event/{event_id} IGNORES the room in the path: it resolves purely by event ID, so /rooms/{scratch}/event/{id} cheerfully returns an event that lives in the live room, and vice versa. ⇒ a naive “is this event in room X?” check returns YES for ANY event that exists, and it fails in BOTH directions — inventing pollution that never happened AND hiding a genuine live-room leak. Hit for real (false positive on the first room check), confirmed with a control: a known live-room event id requested through the scratch-room path returned successfully, with room_id = the live room. ✅ Instead: read the event’s own room_id field (authoritative — the URL is not), page /rooms/{live}/messages?dir=b to audit a room, or count at the source (journalctl -u matrix-relay | grep -c 'sent event'). ⚠️ This undermines any per-event re-audit of the “Telep Cam9” pollution or the 41 redactions. ↔️ Also: migrating an alerter is a one-function URL swap — keep the old transport callable and record the exact rollback line (revert = one edit, not a rewrite); test a new transport against a SCRATCH relay instance, never the live one (own device, own crypto store, own port, torn down after); and 🔴 some “test” invocations have side effects that suppress future REAL alerts — a watermark, a debounce, a rate-limit window ⇒ check for state advancement first (top_kep_remind.py --daily advances a watermark and would have silently eaten that evening’s 20:00 reminder; --dry is the safe path). 🔴 ⭐⭐ Tailscale SSH logs the FULL command line to the journal. tailscaled writes the entire invocation of every Tailscale SSH command, which bites twice. (1) Your own grep command becomes a match — a command containing the literal StartLimitIntervalSec, run to check whether that warning had stopped, matched itself and made an already-fixed problem look unfixed. ⇒ when grepping the journal for a string, remember your own command is in there too: filter on the systemd[1]: prefix, or grep the message shape, not a bare keyword. (2) 🔴 Never pass a secret as a command-line argument over Tailscale SSH — it lands in the journal in plaintext (directly relevant to the pending Telegram rotation). ↔️ Also: retire by archiving, not deleting — intruder-alarm went to /root/retired/2026-09-07-intruder-alarm/ with unit + script + README, so restore is mv + daemon-reload; grep for dependents first and record the evidence. And StartLimitIntervalSec belongs in [Unit], not [Service] — a misplaced one emits Unknown key … on every daemon-reload; recurring noise of that shape is usually one bad unit, not a fleet-wide problem.

Handovers & Session Snapshots

• 2026-09-07-session-handover — ⭐⭐ 2026-09-05 → 07 single entry point: the self-hosted Matrix homeserver (Continuwuity v26.8.1 at chat.taild4189d.ts.net, federation + open registration off, 512 MiB uploads proven), its restore-TESTED nightly RocksDB backup, the Telegram → Matrix migration of every homelab alert, the AMS humidity-threshold rethink, a wedged Tapo P115, two camera/alert findings, and knowledgebase/homepage research (nothing built). ✅ Both alert defects FIXED and verified — Frigate snapshot attachments and bambuddy finish photos (root cause was a concurrency race killing the Matrix session mid-send, not the suspected malformed info block); 🟡 the live print_complete trigger has still never fired with a photo. ⚠️ All 41 relay-sent events were redacted from the live alert room — no alert history exists before this point. 🔴 Open: NO off-site backups (everything on one 3.6 T LVM volume — decision made for restic → Cloudflare R2, restic not yet installed), /opt/matrix-relay/store is backup-critical (lose it and all past alerts are permanently undecryptable), rotate the leaked Telegram bot token + bambuddy API key, delete the stray @phone token, power-cycle the Tapo P115, kill the orphaned home-portal
• 2026-09-05-session-handover — ⭐⭐ 2026-09-02 → 05 single entry point: LG TV network control + presence daemon, the bambuddy preheat chamber-target bug / clog RCA / print-guard / Tapo P115 bridge, the Starlink WAN migration + dish telemetry, the 2.4 GHz interference hunt (radio1 ch11→ch1) and the north-camera multipath finding, Frigate resource tuning, the reMarkable evaluation. Cross-links the dated notes rather than duplicating them. 🔴 Open: TV physically powered off + Auto Power Off still never disabled, Starlink mount printed but NOT installed (obstruction 3.3%), north camera .139 marginal from multipath, leaked bambuddy API key to rotate, bambuddy’s unauthenticated API / empty users table
• 2026-09-02-session-handover — 2026-09-02: 5G WAN root-cause + NE200 SINR logger, chatcut deploy, OrcaSlicer deployed-then-deleted, camwall crash-loop fix, bambuddy slicer-API/pipelines/MCP buildout, FreeCAD connector fix, NVIDIA 550 → 610.57.04 (the nvidia_drm crash fix + the dropped nouveau blacklist trap). Open items at the time: the AI image-editor (designed, not built), bambuddy MCP → Aperture, proving the driver fix
• telep-mainframe-handover — the living operations runbook (status-at-a-glance table + open items); refresh this every session
• 2026-08-29-session-handover — ⭐ 2026-08-22 → 29 single entry point: control-plane OAuth fix, Frigate retention + cat alerts, router 5 GHz cross-band client isolation + Bambu H2S, the FreeCAD CAD studio + Online3DViewer + cad-designer agent + export pipeline, KrakenSDR mobile-DF field-working, Filestash remount. Cross-links the individual dated notes; open items = KrakenSDR array_offset calibration + car 5V/5A power, Filestash reboot-safe mount, unresolved cross-band isolation, Bambu maintenance due
• 2026-08-12-session-handover — 2026-08-12: top-kép voting LIVE, Frigate HD recording, camwall watchdog + HDMI hardening, AirPlay + printer parked
• 2026-08-10-session-handover — ⭐ 2026-08-09/10 overnight single entry point: AirPlay fix (avahi → enp5s0), Frigate FPS watchdog, UPS shed/restore hardening, KB Qdrant restart, camera OSD clock DST fix, .139 weak-WiFi frame drops, camwall birdseye re-roll, offline printer, the master control-plane agent, Stirling-PDF (tag:telep node), the C302 tailnet FaceTime cam, the Top képek→Drive pipeline, and telep-router DHCP/VLAN/SSH changes. Summarises + links the individual dated notes; front-and-centre “needs you” list
• 2026-07-30-session-handover — 2026-07-29/30 KB overhaul / wiring gallery / camera / network snapshot
• SESSION-HANDOVER — the older rolling handover (per-date “Since last handover” sections)

Internationalization (i18n)

• 2026-07-28-kb-hungarian-i18n-design — the KB HU/EN design: mkdocs-static-i18n + Material language selector for the static site + a shared lang cookie for the Flask pages; HU/EN over UI chrome + cluster labels + generated consolidated articles; source manuals stay English (auto fallback); reuse the draft→review→publish gate for machine translations; number-token speccheck as the accuracy guard
• 2026-07-28-kb-hungarian-i18n-plan — the task-by-task i18n plan (string table, lang cookie/route, translate_labels.py, transprompt.py/speccheck, static fallback)
• 2026-07-30-kb-hungarian-translation-complete — manual translation COMPLETE: 820/820 prioritized pages (956 total .hu.md), served at /hu/<page_url> (suffix mode, English fallback), NOT search-indexed. ⭐ Crash-safe method: flush each .hu.md to disk the instant it’s translated — the first bulk-then-write attempt lost the whole run to a mid-run API drop; after flush-per-page + skip-if-exists, 3 API drops and a session-limit cost zero completed work. Faithfulness contract keeps all numbers/units/part-numbers/wire-colors/URLs/image-refs VERBATIM; > AI-forditas... banner per page. Two open judgment calls: page 198 paint-mark colors translated (revert if codes), page 202 dropped an OCR TOC cell
• 2026-08-31-nextcloud-drive-code-server-runbook — Nextcloud Drive: default_language=hu, default_locale=hu_HU, defaultapp=files; force_language deliberately unset so per-user language switching stays available

Linux Host Ops

• Host — Debian 13 bare metal; the installer ticks the desktop task by default (systemctl set-default multi-user.target); all sleep/suspend/hibernate targets masked; getty@tty1 disabled so mpv can own the TTY
• The Windows drive — NTFS Windows C: mounted read-only via in-kernel ntfs3 at /mnt/win; RO because live Windows hibernation would corrupt RW; manual mount, not in fstab (won’t survive reboot); device node unstable → lsblk -f, UUID=
• Mount procedure (verified 2026-07-21) — modprobe ntfs3 → mount -t ntfs3 -o ro <1.8T ntfs part> /mnt/win → docker restart filebrowser; proper fix is a systemd mount unit on UUID= ordered Before=docker.service
• 2026-07-23-mains-power-shutdowns — proving an overnight shutdown was external mains loss, not the host: the dead-stop journal signature (no Stopping/panic/thermal), ruling out software poweroff (crontabs, poweroff/suspend timers, atq, rtc wakealarm, masked sleep targets) and thermal (grep -iE "thermal|mce|Xid"), and cross-checking the router’s boot epoch
• 2026-08-04-telep-mainframe-airplay-receiver-uxplay — decoupling an Xorg :0 server from its consumer via systemd: a standalone camwall-x.service (exec xinit camwall-xsession.sh -- :0 vt1 -nolisten tcp -keeptty, Restart=always) runs Xorg + xset dpms/screensaver off + sleep infinity (no WM); the camera-grid camwall.service and the uxplay.service AirPlay receiver both attach as clients with Environment=DISPLAY=:0 + Requires/BindsTo/After camwall-x.service, so stopping either leaves :0 (and the other client) alive. uxplay runs as root with XDG_RUNTIME_DIR=/run/uxplay
• 2026-08-08-ups-shed-restore-hardening — boot-time reconcile pattern: a power-restore-reconcile.service oneshot (enabled, After docker + network-online.target) runs the same idempotent /usr/local/bin/power-restore.sh the NUT ONLINE handler uses, so state shed during a UPS event is always restored on the next boot even if the restore event never fired; [RECONCILE]-tagged logs distinguish boot reconcile from a live restore. Plus a knowledgebase.service.d/wait-qdrant.conf drop-in that polls a dependency’s 6333/readyz before start instead of crash-looping
• 2026-08-08-telep-master-agent-impl — systemd-autostarted always-on Claude agent on the host: control-plane.service (User=levander, Type=forking, Restart=always, enabled) launches claude remote-control --spawn same-dir inside a persistent tmux session (tmux supplies the PTY under systemd); self-recovers across the clevis+TPM2 auto-unlocked reboots like the other host services. Paired with obsidian-sync.timer (OnBootSec=5min/OnUnitActiveSec=15min) running a git pull-rebase-autostash + commit + push script
• 2026-08-31-nvidia-drm-host-crash-embedding-pass — 🔴 a systemd timer with OnBootSec= aimed at a host-crashing job is a BOOT LOOP — host returns, timer fires, host dies, and every cycle looks like a fresh spontaneous crash. Disable the timer before diagnosing anything that can take the box down; telep-kb-obsidian-sync.timer is currently disabled for exactly this reason. Corollary from the same hunt: design long jobs to upsert as they go rather than batching at the end, so a kill mid-run leaves durable partial progress (see 2026-08-31-obsidian-vault-qdrant-index-selection)
• 2026-08-31-telep-mainframe-mce-hardware-fault — CPU hardware fault on telep-mainframe: mce: CPUs not responding to MCE broadcast: 8-9 → Kernel panic … Not all CPUs entered broadcast exception handler; CPUs 8-9 = the two SMT threads of ONE physical P-core (core_id=16, i9-12900K). Trigger = the all-core embedding pass, not the cause. kernel.printk = 5 4 1 7 because the box boots quiet at console_loglevel 4 and was silently dropping every KERN_WARNING
• 2026-08-31-nextcloud-drive-code-server-runbook — /etc/fstab mounts /mnt/win by UUID: the NVMe device nodes swap across boots (nvme0n1p2 ↔ nvme1n1p2), so a device-node mount eventually targets the wrong disk
• 2026-09-04-starlink-wan-migration-dish-telemetry — Debian 13 enforces PEP 668, so adding one gRPC call to a stdlib cron script was solved with a static grpcurl v1.9.4 binary in /usr/local/bin (root:root, 0755) rather than a venv plus a compiled grpcio toolchain — server reflection removes the need for .proto files entirely. ⚠️ That binary is pinned to whatever was latest on install day, root-owned, and tracked by no package manager — nothing will ever update it. 🔎 Incidental duplication found on telep-mainframe: /home/levander/tapo-ctl already holds a venv with python-kasa 0.10.2 (plus a nightly 04:30 set-cam-time.py), the same version the Tapo P115 bridge installed into /opt/tapo-bridge/venv — and TP-Link credentials likely already exist on the box, which may make populating /etc/tapo-bridge.env trivial
• 2026-09-07-alert-source-inventory — 🔴 three host-level failure paths that alert NOBODY on telep-mainframe. smartd is configured but has no transport: /etc/smartd.conf uses -m root -M exec …/smartd-runner → /etc/smartmontools/run.d/10mail, but no MTA is installed at all (no sendmail/mail/mailx/msmtp/postfix/exim), /var/mail is empty and nothing appears in the journal ⇒ disk-health warnings go to /dev/null — the biggest hole on the box given the earlier NVMe damage. netdata notifies nobody — all 28 SEND_* methods set to "NO", a deliberate override of the stock YES defaults ⇒ the dashboard is pull-only, you must go and look. No OnFailure= on ANY of the ~70 custom systemd units ⇒ nothing alerts when a service dies; every “is it still running?” claim in this vault is a manual observation. 🧹 UPDATE 2026-09-07 — two systemd/journal facts from retiring intruder-alarm: StartLimitIntervalSec belongs in [Unit], not [Service] — a misplaced one emits Unknown key 'StartLimitIntervalSec' in section [Service] on EVERY daemon-reload; that noise stopped when the unit was removed, and ✅ no wider misconfiguration exists (camwall.service, camwall-x.service, nvidia-cdi-refresh.service all have it in [Unit]). And 🔴 tailscaled logs the FULL command line of every Tailscale SSH invocation to the journal ⇒ your own grep matches itself (filter on systemd[1]: or on the message shape) and any secret passed as an argument is journalled in plaintext.

Network Topology

• telep-router — br-lan 192.168.1.1/24, only 10g-copper has link; WAN is a DHCP client at 192.168.254.2 behind upstream 192.168.254.1 → double-NAT
• 2026-07-28-ipv6-slow-internet — the WAN is a TP-Link NE200 outdoor fixed-wireless unit, double-NAT’d; it carries no working IPv6, so the router’s LAN IPv6 advertisement (RA/DHCPv6) was disabled (dhcp.lan.ra/dhcpv6/ndp + network.wan6) to stop the macOS stall; undoing the double-NAT (bridge/DMZ the NE200) could restore real v6
• Camera VLAN (telep-cc) — third segment: 192.168.30.0/24, reachable from lan, reaches nothing
• 2026-07-22-facekom-vpn-exit-node — a fourth segment reachable on demand: the FaceKom office LAN 192.168.105.0/24 via the exit-vpn container’s OpenVPN tunnel (gateway 92.119.122.32), re-exported to the tailnet
• “VPN-only hosts” are a routing problem, not DNS — “VPN-only” FaceKom hosts (npm.facekom.net) are public IPs behind an nginx source-IP ACL, identical DNS answers everywhere: 403 from home WAN, 200 from the VPN
• 2026-08-12-krakensdr-doa-rig — the KrakenSDR Pi’s dual-interface layout: eth0 → home LAN (192.168.1.200, carries SSH) and wlan0 = a pre-existing NetworkManager “Hotspot” AP (10.42.0.0/24, wifi-hunter’s shared AP). The nft wlan0 reject rules are NM’s normal shared-hotspot NAT, NOT a firewall problem. Single WiFi radio → the car deploy flips wlan0 AP → client on the driver’s hotspot (can’t do both at once)
• 2026-08-12-airplay-mdns-fix-printer-migration — on raspi the wifi-hunter wlan0 NetworkManager “Hotspot” AP was brought DOWN (nmcli connection down Hotspot + connection.autoconnect no) to free the single WiFi radio for the future KrakenSDR car-deploy client link; note this stops wifi-hunter’s AP
• 2026-08-15-telep-lan-split-horizon-caddy — split-horizon local access: a *.telep.lan wildcard DNS record on telep-router (dhcp.@dnsmasq[0].address='/telep.lan/192.168.1.123') resolves every homelab name to the mainframe on the LAN, fronted by a Caddy v2 reverse proxy on telep-mainframe (:80, auto_https off) mapping names → backends. Internet-independent path to every service; Tailscale kept for remote. Complements tailnet-service-exposure-convention
• 2026-08-15-dhcp-outage-duplicate-reservation-postmortem — canonical LAN reservation reference: mainframe = 192.168.1.123 / enp5s0 / mac d8:5e:d3:a7:05:d6 (dhcp.@host[2] telep-mainframe); Pi (KrakenSDR rig) = 192.168.1.200 — don’t confuse them. Tailnet recovery IPs: router 100.69.112.32, mainframe 100.115.209.87
• [[telep-router#tp-link-tl-wa850re-added-as-a-wired-ap-ssid-bandi|TP-Link TL-WA850RE added as a wired AP (SSID bandi)]] — LAN inventory: TP-Link TL-WA850RE wired coverage AP at 192.168.1.101 (dhcp.@host[3] name=TL-WA850RE, eth/mgmt MAC ac:84:c6:1b:04:30, WiFi BSSID ac:84:c6:1b:04:31), bridged into br-lan on a 100 Mb port, broadcasting SSID bandi
• 2026-08-26-router-reflash-mac-5ghz-dfs-and-bambu-offline — LAN inventory: Bambu H2S printer reserved 192.168.1.202 (DHCP static host name='bambu', mac='50:31:23:c9:63:ba', dns='1' → resolves bambu.lan), added 2026-08-26. Topology gotcha: the H2S is dual-band (observed on 5 GHz telep1, NOT 2.4-only) — driver-level cross-band Wi-Fi isolation only blocks reachability when the printer is on 2.4 GHz telep1-2G while the Mac is on 5 GHz (wifi→ethernet works, wifi→wifi cross-band fails); both on telep1 (5 GHz) just works. Ethernet or same-band; a hostname/reservation does NOT change the route
• 2026-08-30-telep-tv-airplay-10g-port-multicast-not-flooded — ⭐ the router’s switch does NOT flood multicast arriving on the box’s 10G port out to Wi-Fi. telep-mainframe (192.168.1.123, MAC d8:5e:d3:a7:05:d6, single NIC enp5s0 @ 10000 Mbps) sits on br-lan port 2 = 10G; HP printer on port 5 = 1G; Mac on port 11 = phy0-ap0. Printer’s mDNS reaches Wi-Fi, box’s does not — the ONLY difference is 1G-port vs 10G-port. multicast_snooping=0 (floods), no flow_offloading, no packet_steering. Unicast unaffected; only multicast-based discovery (dns-sd -B) fails. This is the REAL cause of telep-tv being undiscoverable on Wi-Fi — NOT the 5 GHz cross-band isolation it was long blamed on (band-independent)
• 2026-08-31-episodic-wan-degradation — 🔴 the WAN itself degrades in multi-day EPISODES — the NE200 fixed-wireless uplink (upstream of telep-router, behind the double-NAT) collapses throughput 5-50x (normal 180-265 Mbit → 3-17 Mbit) for days at a time (Aug 19-22 = 55/100/100/82% of samples degraded), separated by clean periods. Latency stays healthy ~25-40 ms throughout ⇒ capacity/loss fault, not a latency fault. LAN is provably clean (Mac on lan3 1000baseT FD, 0 iface errors, 0% loss to gw, 111 MB forwarded cleanly); bufferbloat excellent (26.4→27.4 ms under load); NOT DNS (30 ms from all three resolvers); NOT a recurrence of 2026-07-28-ipv6-slow-internet (curl -6 fails in 2-32 ms instead of stalling 2-7 s — that’s the discriminator). ⚠ Separate hardware fault found on the same audit: router port 10g-sfp = 3,792 carrier_changes (next worst lan2 = 26), 252 rx_crc_errors — the only port with any, operstate=down/speed=65535, 414 KB rx in 5 days, ~1 s flap cycle; br-lan STP is disabled (stp_state=0) so it is NOT causing topology churn — bad/empty SFP module, remove it. Also lan5 linked at only 100 Mbit with 1 GB rx (unidentified device). ✅ RF root cause CONFIRMED 2026-08-31 from the NE200 web UI: it is a 5G NR cellular FWA CPE on a Telekom HU SIM (NOT a WISP link) running on a weak radio — SS-SINR ~9.5 dB, SS-RSRP -102 dBm on N78 3.5 GHz — which caps modulation and throttles throughput. ✅ Automated RF logger deployed 2026-08-31 (login solved) captured direct real-time proof: serving N78 cell on QPSK downlink (~4x throughput loss vs 256-QAM) with RSRP swinging -100→-108 dBm in ~2 min — the fast swings explain the episodic collapses. Open competing hypothesis for the MULTI-day blocks: a carrier data-cap throttle (784.5 GB used) fits better than pure RF jitter — check the Telekom plan before aiming the antenna
• 2026-09-04-starlink-wan-migration-dish-telemetry — 🛰️ ⭐⭐ CURRENT WAN: Starlink in Bypass mode, replacing the NE200 — this supersedes every “NE200 / double-NAT / 192.168.254.2” statement above. Router WAN 100.87.12.221/10, gateway 100.64.0.1; the NE200 at 192.168.254.1 is 100% unreachable and out of the path. ✅ Clean-bypass proof = one ARP comparison: 100.64.0.1 and the dish 192.168.100.1 share the SAME MAC 26:12:ac:1a:80:01 — the OpenWrt WAN port talks straight to the dish/bypass-bridge with no Starlink router routing in between. ⚠️ STILL CGNAT (100.64.0.0/10) → no inbound ports are possible, exactly as under the old double-NAT (tailnet/DERP paths unaffected; this is why the tailcat plan for the offline company Mac stands). 🆕 LAN inventory: the dish is 192.168.100.1 (web UI :80, also :8080, gRPC :9200, gRPC-web :9201), reachable from any LAN client once the /32 route exists. ❌ Do NOT renumber the LAN — 192.168.1.0/24 collides with nothing; the only real overlap is the 100.64.0.0/10 Tailscale/CGNAT clash and no local subnet change can fix it

OpenWrt Configuration

• telep-router — UCI-based config; hostname drift fixed 2026-07-13 (uci set system.@system[0].hostname); 174 packages, essentially stock
• IPv6 disabled on the LAN (2026-07-28) — disabled LAN IPv6 via UCI (dhcp.lan.ra/dhcpv6/ndp='disabled' + network.wan6.disabled='1', odhcpd restart + network reload); backups dhcp.bak-ipv6fix / network.bak-ipv6fix, reversible — 2026-07-28-ipv6-slow-internet
• 2026-07-31-wifi-qr-code-sheet — uci show wireless is the source of truth for ssid/key/encryption when building a guest Wi-Fi QR (telep1 sae, telep-cc psk2)
• 2026-08-03-telep-router-factory-reset-recovery — the full UCI config rebuilt from scratch after a factory reset: WiFi (telep1 sae both radios, telep-cc psk2 hidden→cams), cams network (br-cams 192.168.30.1/24, camera DHCP reservations), firewall cams zone + forwardings, DNS query logging (outage-safe in-place rotate), IPv6 LAN-disable re-applied, Tailscale reinstalled. Config backup now at mainframe:/home/levander/telep-router-config-backup-20260803.tar.gz
• 2026-08-15-telep-lan-split-horizon-caddy — added a wildcard local-DNS address record to dnsmasq: uci add_list dhcp.@dnsmasq[0].address='/telep.lan/192.168.1.123' (+ commit + restart) → every *.telep.lan resolves to the mainframe for LAN clients (router = their DHCP DNS); AAAA is NXDOMAIN (cosmetic, A works). Backend of this is a Caddy reverse proxy on telep-mainframe
• 2026-08-15-dhcp-outage-duplicate-reservation-postmortem — 🔴 a duplicate dhcp-host reservation (same IP/MAC) makes dnsmasq FAIL to start entirely — DHCP+DNS down site-wide. uci show dhcp | grep <ip-or-mac> before adding any reservation; the telep-mainframe reservation already owns .123/d8:5e:d3:a7:05:d6. Fixed with uci delete dhcp.telep_tv; uci commit dhcp; /etc/init.d/dnsmasq restart
• 2026-08-15-router-wifi-tuning-htmode-band-penetration — 5 GHz channel width narrowed for range: uci set wireless.radio0.htmode='HE40' (was HE80) + uci commit wireless; wifi reload. TX power (radio0 23 dBm / radio1 20 dBm) is already at the ETSI ceiling so it’s not adjustable; width is the lever. Reversible. Band steering (dawn package) is the recommended next step
• 2026-08-12-airplay-mdns-fix-printer-migration — the AirPlay mDNS fix is NOT a UCI change — OpenWrt auto-injects ap_isolate=1 into the generated hostapd conf (_wireless_set_brsnoop_isolation in /lib/netifd/netifd-wireless.sh) for bridged non-isolated APs with multicast_to_unicast on, and no uci set …isolate/multicast_to_unicast value clears it; fixed by commenting out line 309 + wifi down; wifi up (backup .bak). 🔴 a /lib/netifd/ patch is reverted by sysupgrade — reapply after every firmware upgrade
• 2026-08-15-camwall-dead-br-cams-bridge-down-after-wifi-reload — 🔴 a runtime wifi reload can drop a WIRELESS-ONLY bridge and it does NOT auto-recover: br-cams’ only member is the cams AP phy1-ap1, so a wifi reload (radio1 channel change) left br-cams state DOWN, qdisc noop with no inet — the static network.cams 192.168.30.1/24 was not reapplied. ifup cams alone does NOT fix it; the working recovery is ifdown cams; sleep 2; ip link set dev br-cams up; ifup cams; sleep 5. Durable guards recorded (not applied): hotplug hook to auto-ifup cams, or add a persistent dummy bridge member
• 2026-08-26-router-reflash-mac-5ghz-dfs-and-bambu-offline — keep the 5 GHz radio on a NON-DFS channel or macOS won’t list the SSID: iwinfo to find the 5 GHz radioX, then uci set wireless.radioX.channel='149' (or 36/40/44/48) ; uci commit wireless; wifi reload. Also: a reflash resets the SSH host key + /etc/dropbear/authorized_keys — re-add your pubkey (or use LuCI/console), and re-registers Tailscale (new tailnet IP)
• 2026-09-04-starlink-wan-migration-dish-telemetry — 🔴 /etc/init.d/network reload|restart was DELIBERATELY never run on telep-router — it would drop the whole site’s connectivity and the tailnet SSH used to manage the box. ✅ Reusable safe pattern: apply live with ip route, then write the uci config WITHOUT reloading, and add a hotplug script you can test in isolation. Concretely, the dish /32 was persisted two independent ways: a config route stanza (interface wan, target 192.168.100.1, netmask 255.255.255.255) in /etc/config/network — never verified, because nothing was reloaded — and /etc/hotplug.d/iface/99-starlink-dish (0755, ip route replace so it is idempotent) which WAS verified for real: route deleted → script run → route restored, with lan ifup and wan ifdown both correctly no-opping. LAN clients reach the dish via the wan-zone masquerade (firewall.@zone[1].masq='1') plus the default lan→wan forward

Power & Reliability

• 2026-08-15-telep-lan-split-horizon-caddy — 🔴 ⭐ a reboot can silently kill ALL domain access via a Caddy :443 boot-race — Caddy comes up before the LAN IP 192.168.1.123 is assigned, fails to bind, and doesn’t retry → every *.telep.lan + levandor.io name dead by-domain (found failed ~1.6 days after the 2026-08-15 YubiKey reboots). Same reboot-side-effect family as the AirPlay/DHCP-lease flap below. FIX = net.ipv4.ip_nonlocal_bind=1 + a restart/After=network-online drop-in. Runbook caddy-boot-race

• troubleshooting — 2026-08-08: AirPlay dead after UPS switch + reboot — ⭐ (2026-08-08) the UPS switch + reboot broke AirPlay — a reboot side-effect worth recording alongside the power work: enp5s0’s 192.168.1.123 is a DYNAMIC DHCP lease, so on boot the address flapped (withdrawn+re-added), avahi dropped the mDNS multicast group for ~35s and telep-tv briefly vanished. OPEN FOLLOW-UP: add a DHCP reservation on the router for MAC d8:5e:d3:a7:05:d6 so future reboots don’t flap the address (the 2026-08-03 factory reset — 2026-08-03-telep-router-factory-reset-recovery — likely dropped the prior reservation)

• 2026-07-23-mains-power-shutdowns — telep-mainframe’s recurring overnight shutdowns root-caused to brief mains power interruptions, confirmed by the telep-router (separate PSU) dying at the same instant; the PC only “stays off” because BIOS Restore on AC Power Loss = stay-off. Fix: ~600–900 VA UPS + NUT (logs sags, graceful shutdown); stopgap: flip the BIOS AC-loss policy (physical access only); also rule out an overloaded shared circuit

• Host — all sleep/suspend/hibernate targets masked (relevant when ruling out a software-initiated poweroff)

• 2026-07-24-psu-load-resets — a SECOND, distinct power fault: the PSU can’t sustain peak combined i9-12900K + RTX 3080 draw and hard-resets the box under full load (~3× in 30 min); NOT thermal, NOT mains (the router stayed up 14h+); the default 16-thread CPU embedding was the reliable trigger; mitigate with a 4-thread cap + serialized jobs, real fix is a bigger PSU. A UPS does NOT fix this (it fixes mains, 2026-07-23-mains-power-shutdowns)

• 2026-08-15-yubikey-gated-luks-fde — the reboot-recovery unlock is now YubiKey-gated (the key stays plugged in so unattended auto-unlock is preserved across mains cuts). ⚠️ Its live-lock does a clean sync+systemctl poweroff, deliberately NOT a hard sysrq cut — to avoid adding to the NVMe media-error count from power-loss-during-write. Auto-unlock still only helps once the box BOOTS (BIOS AC-loss + UPS handle that)

• 2026-07-24-luks-tpm-autounlock — ⚠️ SUPERSEDED by the YubiKey unlock above. Historically the reboot-recovery follow-up: root was LUKS2 passphrase-only so an unattended reboot stalled at the prompt (no dropbear-initramfs); clevis TPM2 auto-unlock unsealed the disk at boot (Debian stock initramfs ignores systemd-cryptenroll TPM tokens — a still-true fact); reboot test PASSED in production (2026-07-24-psu-load-resets reboot storm)

• Boot & reboot safety — reboot-safety facts verified 2026-07-31: the clevis+TPM2 binding has NO pcr_ids ({"hash":"sha256","key":"ecc"}), so kernel/firmware/bootloader updates do NOT break auto-unlock; clevis IS in the initramfs; no dropbear fallback exists (auto-unlock is the only unattended path — don’t enable PCR binding without adding one); NVIDIA is DKMS (550.163.01, auto-rebuilt for 6.12.100); the 6.12.95→6.12.100 reboot recovered camwall/knowledgebase/frigate/kb-qdrant cleanly — 2026-07-31-telep-mainframe-ax210-ble-scan-broken

• 2026-07-24-psu-load-resets — the PSU-under-load resets, a SECOND power problem distinct from the mains outages; the router-uptime cross-check discriminates the two; a UPS fixes mains, NOT this — serialize heavy jobs / cap CPU threads / bigger PSU

• 2026-08-05-power-root-cause-nvme-damage-ups-kb-handover — ⭐ the mains diagnosis CONFIRMED with NVMe SMART evidence — and the cuts are now DAMAGING the SSD. ~6 hard cuts in ~50 min (2026-08-05); nvme smart-log = 240 unsafe_shutdowns / 1440 power_cycles / 1617 media_errors (4% used, no critical warning) — power-loss-during-write is actively corrupting the drive. Ruled out thermal (49°C CPU / 55°C GPU, zero throttling) + hardware (no MCE/panic). FIX/DECISION: UPS ~2026-08-07 — pure-sine (mandatory for active-PFC PSU) + AVR + ~1500VA/900-1000W + USB/NUT auto-shutdown (CyberPower CP1500PFCLCD / APC Smart-UPS 1500); NUT is what stops the media-error bleed. TODO once stable: fsck + NVMe self-test, consider an electrician. The nv_drm_revoke_modeset_permission WARNING is known-benign — don’t chase it

• 2026-08-06-power-root-cause-gpu-12v-connector-stress-test-pass — ⭐ ROOT CAUSE FOUND + FIXED: the under-load hard-offs were an unplugged GPU 12V PCIe connector (RTX 3080, 3 separate PSU cables, one bank disconnected), reconnected during reassembly → graduated CPU+GPU stress test PASSED with zero crashes (up to 250W CPU / 450W GPU cap, peak combined ~536W, GPU 86–87°C no throttle). This supersedes the 2026-07-24-psu-load-resets “PSU can’t sustain load” theory and the 2026-08-05-power-root-cause-nvme-damage-ups-kb-handover §6c “heavy CPU triggers brownouts on marginal mains” theory for the load-triggered resets. Corsair 1000W PSU was NOT the fault. A UPS+NUT still helps genuine mains events; the deferred fsck+NVMe self-test still apply. ⚠️ OPEN TODO: the stress-test power caps reset on reboot — no persistent sane profile yet

• 2026-08-08-ups-shed-restore-hardening — ⭐ the first real UPS-outage post-mortem: shed/restore was NOT idempotent across a battery-exhausting outage. Overnight: ONBATT→shed (KB, Qdrant, camwall, AirPlay, jobs)→LOWBATT→graceful poweroff→mains returned WHILE THE BOX WAS OFF→BIOS auto-power-on→boot, but the NUT ONLINE/restore event never fired (upsmon wasn’t running to see it) → kb-qdrant (no boot recovery) stayed down → knowledgebase.service crash-looped 2360× against a missing Qdrant. HARDENING (all on the box): shared idempotent /usr/local/bin/power-restore.sh ([RECONCILE] logs) called by BOTH the NUT handler AND a new boot-time power-restore-reconcile.service (enabled, After docker+network-online); kb-qdrant → --restart unless-stopped; knowledgebase.service.d/wait-qdrant.conf polls 6333/readyz before start. Handler backup /etc/nut/nut-outage-handler.sh.bak-*. Extends the UPS/NUT work in telep-mainframe-handover §2

• 2026-08-11-camwall-hdmi-mode-fallback-power-recovery — ⭐ a power-event side-effect the host RODE OUT on the UPS (no reboot, uptime 3d17h) that still broke a service: after the HDMI renegotiation Xorg fell back to 640x480 with no mode set on HDMI-1, blacking out the whole camwall (4× 1920x1080 panes can’t map on a 640x480 root). Distinct from the boot-recovery failures — here only services restarted (Frigate, camwall-x/Xorg, camwall), and every re-roll came back on the same broken screen. Fixed with a self-healing xrandr --mode 3840x2160 on every camwall start + a resolution check in camwall-watchdog.py (auto re-rolls on resolution fallback, ~10–20s worst case). Filed under both Cameras and Power because the trigger was mains, not the display pipeline itself

• Since last handover (2026-07-24) — a daily ~06:00 mains die-off pattern (NEW, UNCONFIRMED): the box + router die together ~06:00 most mornings (Jul 19/23/24) = mains-level, suspect a scheduled utility relay (vezérelt/GEO tariff) or a timed high-draw appliance; confirm via journalctl --list-boots vs router reboots. Also: remote-wake (WOL) is NOT possible as configured (BIOS+NIC WOL off, no router wake tool, MAC unknown after lease expiry) → the real fix is BIOS Restore on AC Power Loss → Power On

• 2026-08-26-raspi-oom-zram-hunter-ap-433-retune — 📡 the 2 GB Pi 5 raspi/kraken-rig OOM-crashed twice running the memory-heavy KrakenSDR DAQ → added zram swap (zram-tools, zstd, PERCENT=50, zramswap.service) ≈ 1 GB compressed swap. 🔴 systemctl restart zramswap reports “failed” while the zram device is live (can’t re-init in place) but swap IS active (/proc/swaps); clean reset = swapoff /dev/zram0; zramswap stop; systemctl restart zramswap; fresh boot starts clean. Separately: a headless Pi that won’t boot is often just an unseated SD card (reseat before assuming corruption — bootfs FAT + ext4 root verified clean here)

• 2026-08-26-krakensdr-field-test-milestone — ⭐ CORRECTION: the kraken-rig’s “OOM crashes” were largely POWER BROWNOUTS. The Pi 5 + KrakenSDR DAQ browns out and resets on 5V/3A — and a generic PD-30W does NOT help (Pi only takes 5V → still 5V/3A). Needs a real 5V/5A: verified stable then (throttled=0x0 under load, usb_max_current_enable auto→1). Car: 12V→5V/5A buck + usb_max_current_enable=1, Kraken and phone on their OWN feeds (Kraken self-powered via its own USB-C, not off the Pi)

• 2026-08-31-telep-mainframe-mce-hardware-fault — the crashes that outlived the electrical era are a faulty CPU core, not power. Distinct from 2026-07-24-psu-load-resets / 2026-08-06-power-root-cause-gpu-12v-connector-stress-test-pass: those left no kernel trace, this one does. Instrumentation (both enabled): netconsole-target.service → telep-router /tmp/netconsole/kmsg.log (RAM-backed, copy off after a crash) + thermalwatch.service → /var/log/thermalwatch/samples.log (10 s, fsync’d per line, tags SUSPECT_core16_cpu8_9=). ✅ 43+ min stable with the sync timers disabled — mitigation, not a fix: the faulty core is untouched

• 2026-09-03-bambuddy-preheat-chamber-target-bug-and-clog-rca — 🔌 Tapo P115 → bambuddy energy bridge — BUILT, ENABLED, NOT STARTED, entirely UNTESTED (no plug was on the network). bambuddy has no native Tapo backend (only tasmota/homeassistant/mqtt/rest), so a python-kasa 0.10.2 shim at /opt/tapo-bridge/tapo-bridge (venv /opt/tapo-bridge/venv) adapts a P115 to the generic rest backend. Binds 127.0.0.2:8117 per the never-127.0.0.1 rule; bambuddy is NetworkMode: host so it can reach it — verified. /etc/tapo-bridge.env is 0600 root-only with placeholders. 🔴 consumption_total is None for ALL Tapo devices — lifetime energy is synthesized from /opt/tapo-bridge/state.json and starts at zero

• 2026-09-07-alert-source-inventory — 🔴🔴 the UPS alarm has NEVER worked during a power event. 31 of 34 UPS alert sends FAILED, every one of them inside the 2026-08-07 (6) and 2026-08-18 (25) outage windows — the failure is structural, not flaky: the power was out ⇒ the WAN was down ⇒ the Telegram alert could not leave the building. The only successes landed after power and network returned, i.e. after the human already knew. Continuwuity runs locally on the same box, so a Matrix alert still delivers on battery over LAN/tailnet ⇒ moving /etc/nut/nut-outage-handler.sh off Telegram is a CORRECTNESS FIX, not tidiness. 💡 Generalised: an alerter that shares a failure domain with the thing it monitors is not an alarm — keep the alert path local to the failure. ✅ DONE 2026-09-07 — /etc/nut/nut-outage-handler.sh now sends via the relay as [ups] (line 51 in send_alert(); rollback = send_matrix "$1" → send_telegram "$1"). 🟡 No real ONBATT/ONLINE/LOWBATT event has been simulated — the send function was proven with the three real message strings and the script run end to end through the safe unmapped COMMBAD branch, so the first genuine outage is still the real test.

Presence & Automation

• 2026-09-05-session-handover — ⚠️ both new automation paths are still UNPROVEN on real data: tv-presence’s arrival path has never fired (both tracked devices have been home throughout) and print-guard’s guard path has never fired. The presence union (router iwinfo assoclist + arp-scan) remains load-bearing — a sleeping iPhone is ABSENT from arp-scan but PRESENT in the assoclist, so arp-scan alone would switch the TV off with someone standing in front of it

• 2026-07-17-intruder-alarm — auto-arming alarm; presence = union of router WiFi association table (iwinfo assoclist) + arp-scan on enp5s0; dormant until ≥1 trusted device, arms after all absent 10 min, disarms instantly on return; TV siren via aplay -D plughw:1,3 (NVIDIA HDMI = ALSA card 1 dev 3) ⛔ RETIRED 2026-09-07 — unit and script removed from live paths, archived (not deleted) at /root/retired/2026-09-07-intruder-alarm/; restore = mv both back + daemon-reload. This note is now a historical design record. See 2026-09-07-alert-source-inventory §6.

• Device naming & enrollment — device names via mDNS avahi-resolve → dnsmasq reverse DNS → eszköz-<last4>; confirm-each enrollment over Telegram inline buttons

• Untrusted input: LAN device names — reuses the alarm’s ~/alarm/trusted.json names rather than inventing a second naming system; LAN-supplied names are an untrusted input path — ASS-escape and cap at 22 chars, and decode with errors="replace"

• 2026-08-09-xiaomi-c302-tailnet-facetime-cam — internal-cam-alert.service (/usr/local/bin/internal-cam-alert.py, root, DISPLAY=:0, After=camwall-x): pings the indoor C302 (192.168.30.168) every 3s (ACTIVE after 1 success, INACTIVE after 3 fails) and draws a persistent full-width red override-redirect banner BELSŐ KAMERA AKTÍV (120px white-on-#d00000, 3840x200+0+0, no WM, -topmost) on the camwall TV whenever the cam is online, clearing when its hardware relay cuts power — the visible half of the “physically isolate what you can’t network-isolate” privacy stance. Rides the same Xorg :0 as the camwall/uxplay overlays

• 2026-08-11-ruview-esp32-csi-real-hardware — WiFi-CSI presence sensing on real hardware (RuView + an ESP32-S3 node at yield=33–37 pps, ruview.taild4189d.ts.net). Presence / motion / RSSI / variance are real and responsive — everything else is not yet trustworthy: estimated_persons read 1 when 2 people were present (then 2, then 3, ~0.5 confidence throughout), samples can be internally contradictory (motion_level: absent + presence: true + estimated_persons: 3 at once), and with ONE node there is NO localisation (the 3D blob is feature magnitudes, not position — needs multiple nodes + --node-positions). 🔴 --calibrate (empty room) has never been run and variance never drops below ~200 in any condition = the signature of no empty-room baseline, so everything sits above threshold and everything reads as presence — the prime suspect and the next step. 🔴 Do NOT wire this into an alarm yet: a malformed sample every ~20–40 readings carries confidence ABOVE 1.0 (4.79, 12.96) and would false-trigger 2026-07-17-intruder-alarm-style logic. All three walk-tests so far were contaminated; the uncontaminatable test is an overnight record-csi-udp.py capture looking for diurnal structure

• 2026-09-02-tv-presence-wifi-union-daemon — ⭐⭐ tv-presence.service REBUILT: presence = router assoclist UNION arp-scan; the arp-only version was deleted. /opt/tv-control/presence (stdlib python3), 60 s poll, 900 s presence window, drives /opt/tv-control/tv on|off|camwall on transitions only (⚠️ the subcommand was renamed hdmi1 → camwall). 🔴 arp-scan alone is provably broken here — confirmed live 2026-09-02: iPhone 38:7f:8b:df:2a:79 PRESENT in iwinfo phy0-ap0 assoclist yet ABSENT from arp-scan --interface=enp5s0 --localnet (sleeping iOS stays WiFi-associated, stops answering ARP); daemon logged seen=router → seen=both across consecutive polls, i.e. an arp-only design would have switched the TV off with the owner in the room. Safety: presence starts None and the first poll is adopted without actuation (restart never power-cycles the TV); a failed poll is DISCARDED entirely, never read as “everyone left”; a failed tv on/off doesn’t latch. ✅ TV is now paired (2026-09-02 16:13) and tv on/off/camwall are verified against real hardware — but 🚧 the daemon’s own absent→present path has still NEVER fired for real; verified only by code inspection plus the identical subprocess mechanism working through tv-http.

• 2026-09-02-lg-tv-network-control-presence — ⭐ tv-presence.service (/opt/tv-control/presence): polls arp-scan on enp5s0 every 60 s, matches tracked MACs case-insensitively (38:7f:8b:df:2a:79 spider-web/iPhone, a4:40:e1:02:01:e9 F_M), 900 s (15 min) absence debounce; absent→present runs tv on then tv camwall (⚠️ renamed from hdmi1; that alias was removed, not repointed), present→absent runs tv off on the camwall TV. 🔑 Acts on TRANSITIONS ONLY and deliberately does NOT actuate on startup — so a daemon restart can’t power-cycle the TV and a human who switches it off manually isn’t fought back every cycle. 🔑 Whole-subnet arp-scan rather than fixed IPs so DHCP reassignment can’t silently break tracking, and active probing rather than passive ARP-cache reads because phones sleep their Wi-Fi radio and vanish from the cache while their owner is still in the house (same union-of-sources lesson as 2026-07-17-intruder-alarm). ✅ no longer inert — the TV was paired 2026-09-02 16:13 and actuation works; 🚧 the absent→present path has still never fired on a real arrival.

• 2026-09-02-tv-presence-wifi-union-daemon — ✅ Actuation is now real, the arrival path is not. The LG TV was paired 2026-09-02 16:13; tv on / tv off / tv camwall all verified against hardware, including a Wake-on-LAN wake from deep standby (so LG “Mobile TV On” is confirmed already enabled). 🚧 But the daemon’s absent→present transition has never fired on a real arrival — it is verified only by code inspection and by the identical subprocess mechanism working through tv-http. Untested-in-anger risk: the first post-wake tv camwall can TimeoutError because webOS accepts TCP before SSAP answers, and that retry loop has only been exercised by hand.

• 2026-09-04-starlink-wan-migration-dish-telemetry — 🔴 why the Starlink router must NEVER become the main router: it would destroy presence detection. A Starlink router has only two states — Bypass (WiFi off, pure L2 bridge) or main router (own NAT+DHCP) — there is no AP/bridge mode, and its mesh nodes have no Ethernet backhaul. Going non-Bypass would displace telep-router and take out br-cams (the 192.168.30.0/24 camera VLAN + telep-cc SSID), dnsmasq leases and .lan names, and critically the iwinfo assoclist on phy0-ap0/phy1-ap0, which both 2026-07-17-intruder-alarm and 2026-09-02-tv-presence-wifi-union-daemon union with arp-scan. Proven live: a sleeping iPhone (38:7f:8b:df:2a:79) is ABSENT from arp-scan but PRESENT in the assoclist — losing the assoclist drops both systems to arp-scan-only, i.e. the TV switches off with someone in front of it and the alarm arms with people home. Also: an unmanaged Starlink router upstream puts its WiFi clients outside the firewall, DNS, VLANs and both presence systems

Printing

• 2026-09-05-session-handover — 🖨️ bambuddy hardening summary: chamber-target bug fixed by setting PC/PC-FR to 0 in preheat_filament_targets (a per-item preheat_chamber_target_override beats the map) — ⚠️ PC-FR jobs must now pass preheat_chamber_target_override: 60 when queued or they get no chamber preheat, and ABS/ASA are still 45 so the trap returns if either is loaded alongside PLA. print-guard halts the queue with manual_start: true on HIGH→LOW temp transitions (the purge window). Backups verified (daily 03:00, bambuddy-backup-20260903-030031.zip, 292 MB). Per-filament humidity thresholds set — the PC alert firing at 37–42% RH is CORRECT, and the AMS physically cannot fix it (65 °C max vs PC’s 80 °C), so it means “dry externally”. Tapo P115 on 127.0.0.2:8117 is monitoring-only. 🔴 bambuddy’s API is UNAUTHENTICATED with an EMPTY users table (GET/PATCH on 127.0.0.2:8000 succeed with no credentials, on a machine that heats to 280 °C — the tailnet ACL is the only gate), and its API key leaked into a transcript and must be rotated

• telep-mainframe-handover — Bambuddy Bambu Lab 3D-print control plane (Docker bambuddy, network_mode host, UI :8000) on its own tailnet node bambuddy.taild4189d.ts.net; printer on LAN/Dev-Mode 192.168.1.202 (AMS detected); slicer = Bambu Studio on the Mac (not OrcaSlicer) printing through Bambuddy’s virtual printer over the tailnet. ✅ .202 reserved in DHCP 2026-08-26 (name='bambu', resolves bambu.lan).

• 2026-08-08-bambuddy-deploy-plan — the Bambuddy deploy plan + the host-network tailnet-node exception.

• 2026-08-26-router-reflash-mac-5ghz-dfs-and-bambu-offline — 🖨️ Bambu H2S offline after a router reflash (the H2S is dual-band — observed on 5 GHz telep1, NOT 2.4-only). Studio log_iotc.txt ErrCode -12 is a symptom; confirm with an SSDP M-SEARCH probe (ST: urn:bambulab-com:device:3dprinter:1 → 239.255.255.250:1990/:2021) + nc -z 192.168.1.202 8883 990 6000; recover with a router Wi-Fi restart. 💤 Standby ≠ offline: a sleeping printer still pings/ARPs but closes 8883/990/6000 and stops SSDP — “pings but all ports closed + no SSDP” = asleep (wake via touchscreen), not a network fault. Printer internal model O1S, SN 0938BJ641800310, DevName 3DP-093-31, MAC 50:31:23:c9:63:ba, LAN .202 (reserved as bambu.lan). Ports: 8883 MQTT/TLS, 990 FTPS, 6000 camera, 322. ⚠️ If Studio only connects when the Mac is on the same band → Problem 3 — Wi-Fi cross-band client isolation (driver-level) (wire over Ethernet or same-band; Tailscale won’t help on-LAN).

• 2026-08-10-session-handover — HP LaserJet M203dw won’t print because it’s OFF the network (no DHCP lease / ARP / mDNS; it drops WiFi) — CUPS is healthy, so power it on / put it on Ethernet and it just works. No software fix. Add a DHCP reservation so it stops dropping off

• 2026-08-12-airplay-mdns-fix-printer-migration — ⭐ the M203dw saga resolved: Ethernet NIC confirmed HARDWARE-DEAD from the surge (serial VNC3920651, USB 03f0:632a) → set up on USB-to-CUPS (queue HP_M203dw_USB; ⚠️ had to unload+blacklist usblp so the CUPS libusb backend could claim it; reused PPD HP_LaserJet_M203dw_B8AE8C.ppd) → then moved to WiFi via the HP Smart iOS app (M203dw has WiFi; Wi-Fi Direct pw 12345678, EWS 192.168.223.1). PENDING: once the WiFi IP is known → repoint CUPS to a network queue, add a DHCP reservation on telep-router, remove the USB queue + two stale dead-NIC queues (HP_LaserJet_M203dw_B8AE8C, …@NPIB8AE8C.local)

• 2026-08-10-top-kepek-drive-curation-print-pipeline — photo-print staging pipeline (distinct from the thermoprint label printer): pick good Frigate camera detections in the Top képek picker, crop freeform at native resolution (720p detect stream → fine for 4×6, extract from record/main for larger), and rclone them to Google Drive for printing on the HP LaserJet. Self-serve save-to-Drive buttons replace feeding timestamps to an agent

• 2026-07-31-thermoprint-appliance-spec — design spec: self-host [tomLadder/thermoprint] on telep-mainframe to drive a Marklife P15 BLE label printer 24/7 from the tailnet. Approach A: a Bun print-service owns all BLE (connect-per-job over Noble), serves the patched web editor same-origin, tailscale serve + landing page

• 2026-07-31-thermoprint-appliance-plan — task-by-task implementation plan (provision+BLE de-risk → print-service → editor patch → systemd/tailscale → e2e). SW fully staged at /home/levander/thermoprint (Bun 1.3.14, 672 pkgs incl sharp+@stoprocent/noble, setcap on bun)

• 2026-07-31-telep-mainframe-ax210-ble-scan-broken — ⛔ the blocker: the mainframe’s AX210 had no antenna → zero BLE devices seen → couldn’t discover the P15. Fix = plug the AX210 antenna (or USB BLE dongle fallback), then re-run bun run packages/cli/src/index.ts discover

• 2026-08-04-thermoprint-connect-clobber-and-2sided-labels — debugging reusables: (1) connect-clobber bug — connect overwrites editor label/paperType from the profile default in connect-flow.tsx + printer-store.ts; fix in core/.../profiles/p15.ts labelConfig; PR #25 fixed a DIFFERENT store-source bug; (2) L11 gap sensor is always physical — a die-cut gap mid-bitmap halts the head (out-of-paper), software continuous can’t override; (3) no built-in 2-sided — spanning composite fails at the fold gap → design 1 label WYSIWYG, print, rotate 180°, print, stick back-to-back; (4) P15 35×12.5mm, canvas rotates 90° pre-print, Konva rotates around top-left; (5) core resolved from src → only a packages/web rebuild needed, Vite hashes bundles → hard-refresh

• 2026-09-01-orcaslicer-tailnet-deploy — 🖨️ OrcaSlicer 3D-printing slicer desktop GUI, self-hosted on the same box as bambuddy + the Bambu printer (192.168.1.202) so slice→send works locally; the full desktop is streamed to the browser via KasmVNC, tailnet-only at https://orcaslicer.taild4189d.ts.net. GPU not passed through (software GL). Note the sibling Bambu Studio on the Mac path via Bambuddy still exists

• 2026-09-01-bambuddy-slicer-api-pipelines-mcp — 🖨️ Bambuddy full print control plane: server-side Bambu Studio slicer-api sidecar (bambuddy slices on the box, no desktop app needed), 4 material pipelines PLA/PETG/ABS/PC-FR on the H2S 0.4 nozzle + 0.20mm Standard @BBL H2S + Textured PEI (per-material chamber/bed/drying rules baked in; ⚠ never co-print PLA with ABS/PC-FR), a maintenance tracker (6 H2S tasks vs print-hours), and Telegram alerts. This replaced OrcaSlicer (now retired) with bambuddy’s built-in Slicer API

• 2026-09-03-bambuddy-preheat-chamber-target-bug-and-clog-rca — 🚨 ⭐⭐ THE bambuddy preheat bug: _derive_chamber_target() (print_scheduler.py ~L4042) returns the MAX chamber target over EVERY loaded AMS tray and never consults the print’s ams_mapping. One PC spool in AMS slot 1 ⇒ every PLA print inherited a 50 °C chamber target, and with no bed_temperature in the archive metadata the bed fell back to queue_keep_warm_bed_temp = 90 °C (PLA wants 35–45 °C) → over-adhesion, elephant’s foot, textured-PEI wear. 57 s ramp + 300 s soak ≈ 6 min lost per print (the FTP upload is 0.4 s — never the culprit). ✅ Fix = invert the default: PC: 0 / PC-FR: 0 in preheat_filament_targets, because a per-item preheat_chamber_target_override beats the filament map — the common case (PLA) is then correct with zero action. ⚠️ ABS and ASA are still 45 — the identical trap returns if either is loaded alongside PLA. ⚠️ PC-FR reports tray_type: "PC" (tray_sub_brands: "PC FR") and the derivation reads tray_type, so it resolves to PC’s 50 °C, 10 °C under PC-FR’s documented 60 °C. Also here: nozzle-clog RCA, material-sequencing rules, PC-FR checklist, maintenance/backup changes, config review

• 2026-09-07-session-handover — 🔑 per-material AMS humidity thresholds are structurally unworkable: ONE sensor serves all four slots, so whichever material has the lowest threshold alerts permanently regardless of what is loaded. AMS humidity alerts had become 32 of the last 40 notifications (hourly, forever) because PC’s threshold was 30 while the AMS runs at 37–42% and the AMS physically cannot dry to 30% (65 °C ceiling, PC needs 80 °C) — correct but unactionable. The per-material map from 2026-09-03-bambuddy-preheat-chamber-target-bug-and-clog-rca was replaced with {"default": 50} = “the AMS is wetter than it normally runs”, which is actionable; the PC-drying knowledge moved to the PC / PC-FR Dryness Check maintenance task (14 days) and the PC-FR pipeline description. Also: bambuddy alerts now go to Matrix (provider id=2; id=1 Telegram disabled, not deleted), and the Tapo P115 is wedged and needs a physical power-cycle

Runbooks & Incident Response

• 2026-09-02-freecad-mcp-rpc-refused-gui-not-running — 🔧 FreeCAD MCP tools all return Failed to get RPC status: [Errno 111] Connection refused → symptom → the nc false-positive trap → the accept-then-close discriminating probe → fix (“Start RPC Server” in the FreeCAD GUI on the cad host).

• 2026-09-05-freecad-mcp-502-dead-container-and-qt-event-loop-wedge — 🔧 FreeCAD MCP returns HTTP 502 on :8443 (both /mcp and /) → that is a third, distinct failure mode above the two 2026-09-02 ones (which gave Errno 111 / RemoteDisconnected on :9875). Triage: serve status (target is 127.0.0.2:9876) → docker ps -a → Exited → docker start freecad-mcp. Then, if execute_code still times out at 90 s with no dialog on screen, the Qt event loop is dead: verify list_documents() is [], move the FreeCAD_Doc_* recovery snapshots to a dated stash (never rm), then docker restart freecad. Includes a revised triage flowchart superseding the one in 2026-09-02-freecad-mcp-gui-dispatch-timeout-modal-dialog.

• runbooks-index — ⭐ action-first “when X breaks, do Y” index — symptom→runbook table + recovery access (router ssh root@100.69.112.32, mainframe ssh levander@100.115.209.87; mainframe .123 / Pi .200)

• no-wifi-or-dhcp — whole network down / no IP → #1 cause = dnsmasq refused to start on telep-router (duplicate dhcp-host or bad config kills DHCP+DNS site-wide); logread | grep 'dnsmasq\[', dnsmasq --test, delete the bad section, restart

• service-unreachable — a *.telep.lan (Caddy) or Tailscale service won’t load; DNS→Caddy→backend checks, curl -H Host for 502s, name→port map; kraken.telep.lan 502 = the Pi is offline

• caddy-boot-race — 🔴 EVERYTHING by-domain down after a reboot (all *.telep.lan and admin/id.levandor.io) but backends up = caddy.service lost a :443 bind-race against the LAN IP coming up. systemctl status caddy (bind: cannot assign requested address) → systemctl start caddy; permanent fix net.ipv4.ip_nonlocal_bind=1 + restart drop-in; also bring up oauth2-proxy if admin 502s. Incident: Caddy boot-race outage (2026-08-17)

• host-offline — mainframe/Pi dropped off the tailnet → usually a lost DHCP lease (fix DHCP, re-leases in ~1 min); recovery over Tailscale; BIOS auto-power-on

• airplay-telep-tv — AirPlay telep-tv broken (not appearing / won’t cast / no video / no audio); restart avahi-daemon uxplay, netifd AP-isolation patch, plughw:1,3 sink

• 2026-08-30-telep-tv-mdns-vanishes-after-uxplay-restart — telep-tv vanishes from mDNS after a uxplay restart while both uxplay + avahi-daemon stay active and avahi-browse -rt _airplay._tcp/_raop._tcp return nothing (but avahi still advertises _nut._tcp / the printer). Cause: uxplay 1.71’s avahi-compat-libdnssd layer doesn’t reliably re-register across a restart/D-Bus hiccup. FIX (order matters): restart avahi FIRST, sleep 2, then uxplay — bouncing uxplay alone hits the same race

• camwall-not-on-tv — camera wall black; camwall-x (Xorg :0) vs camwall (grid), /run/uxplay/casting, ONVIF cam reboot. New branch: ALL cameras fps=0 + both unreachable → check ip addr show br-cams on the router (wireless-only bridge down after a wifi reload; fix ifdown/ip link up/ifup cams) — incident 2026-08-15-camwall-dead-br-cams-bridge-down-after-wifi-reload. Variant added 2026-08-31: if ifstatus cams = DEVICE_CLAIM_FAILED (device up, no 192.168.30.1), ifup cams won’t clear it → /etc/init.d/network restart — incident 2026-08-31-cams-vlan-device-claim-failed-wifi-reload

• 2026-08-31-cams-vlan-device-claim-failed-wifi-reload — 🔴 wifi reload raced the cams VLAN into netifd DEVICE_CLAIM_FAILED → all 4 Frigate feeds down + camwall.service stuck activating (start-pre) (its ExecStartPre gates on Frigate /api/version). /etc/cams-guard.sh’s per-minute ifup cams CANNOT clear a stuck claim; only a full /etc/init.d/network restart does. Verify after any wireless commit: ifstatus cams | grep '"up"' + ip -4 addr show br-cams. Distinct from the 08-15 carrier-down outage — branch on the netifd state

• krakensdr-df — KrakenSDR DF; Overdrive = RF over-gain not PSU, ghost bearings = array/config mismatch (≤0.5 λ, ~428 MHz ceiling), DoA/PR both bind :8080

• admin-portal-login-broken — admin.levandor.io login 500 (unverified email) / invalid_request / PKCE / 502 (upstream loopback) / ssh actions fail (user: root + drop :ro) / Caddy :443 collision; symptom→fix table, chain Caddy→oauth2-proxy→OliveTin. Design: 2026-08-15-admin-portal-passkey-olivetin

• camera-alerts-not-firing — no Telegram person/car alerts (“Dezsi az őr” silent) despite live cameras; Already notified on this zone flood is a RED HERRING → docker logs … | grep WRN for server misbehaving (container DNS) vs 401/chat not found (token/chat); FIX = pin dns: on frigate-notify. Incident: 2026-08-15-camera-alerts-dead-tailscale-magicdns-docker-dns

• 2026-08-26-router-reflash-mac-5ghz-dfs-and-bambu-offline — 🌐 post router-reflash triage: (a) a Mac can’t see the 5 GHz SSID telep1-5G (phones can) = it’s on a DFS channel → move the 5 GHz radio to non-DFS (36/40/44/48 or 149/153/157/161); don’t trust macOS networksetup/system_profiler for association (they redact/lie — use ifconfig en0/ipconfig getifaddr en0/route -n get default/ping); (b) Bambu H2S offline → SSDP probe + nc -z .202 → router Wi-Fi restart; (c) router SSH host-key + authorized-keys reset → ssh-keygen -R 192.168.1.1, re-add key to /etc/dropbear/authorized_keys (or LuCI/console)

• 2026-08-31-nextcloud-drive-code-server-runbook — 📗 ops runbook for the Nextcloud “Drive” stack + code-server: architecture (per-service Tailscale sidecar), full loopback port map, the occ invocation form, restart commands per service, sidecar status/serve status checks, manual scan, .env locations, gotchas, a host-provable-vs-tailnet-only verification checklist, and every .bak + rollback

• yubikey-luks-lockout — disk won’t unlock / box powered itself off / swap-or-add a YubiKey. Powered-off = live-lock fired (key pulled) → reseat+power on; stuck at prompt = passphrase at the physical console (no dropbear); disarm 99-yk-lock.rules before any ykman config; header corrupt → restore the post-YK backup only (never the shredded pre-YK/TPM one). Design: 2026-08-15-yubikey-gated-luks-fde

• 2026-08-31-telep-mainframe-mce-hardware-fault — what to do after the next mainframe crash: copy telep-router:/tmp/netconsole/kmsg.log off first (tmpfs, lost on router reboot), grep for MCE broadcast, then pull the thermalwatch run-up from /var/log/thermalwatch/samples.log

• 2026-09-01-orcaslicer-tailnet-deploy — 🛠️ OrcaSlicer deploy runbook: browser-streamed slicer GUI at https://orcaslicer.taild4189d.ts.net (tailnet-only, no Funnel, LE cert via tailscale serve). First load ~13s (KasmVNC session spin-up), fast after. Backend loopback http://127.0.0.2:8570; restart via docker restart orcaslicer; sidecar tailscaled-orcaslicer.service; config at /home/levander/orcaslicer/config; portal tile under 🛠️ Eszközök & Média (backup services.yaml.bak-orca)

• 2026-09-03-bambuddy-preheat-chamber-target-bug-and-clog-rca — 🛠️ “a print is mysteriously slow / the printer looks unavailable” triage, in order: (1) read the preheat log lines FIRST, not the not available line — preheat starting - bed=… chamber_target=… (source=…) names the decision and its source; (2) check the derived chamber target against the print’s actual ams_mapping — if it corresponds to a filament the print doesn’t use, it’s the _derive_chamber_target max-over-all-trays bug; (3) check per-input connected state before blaming hardware. Corollary: the FTP upload is never the problem (2.2 MB in 0.4 s) — look at ramp + soak. Also contains the PC-FR print checklist (dry externally at 80–100 °C since the AMS can’t; close chamber + glue + ventilate; 🔴 pass preheat_chamber_target_override: 60 or preheat is SKIPPED ENTIRELY and the print starts cold; brim is not automatic; purge/cold-pull at PC temps before any PLA/PETG; leave the spool in the AMS — it’s the driest storage at 41 % RH)

• 2026-09-04-starlink-wan-migration-dish-telemetry — 🛰️ Starlink triage: reach the dish at http://192.168.100.1 from the telep1 SSID (the mobile app does not work in Bypass); if the dish is unreachable after a reboot re-add ip route add 192.168.100.1/32 dev wan and check /etc/hotplug.d/iface/99-starlink-dish still exists at 0755; never /etc/init.d/network reload to apply router config; confirm bypass is clean by checking 100.64.0.1 and 192.168.100.1 share one MAC; and if the telemetry logger is ever rebuilt, pass -emit-defaults and use port 9200. ⚠️ UNCONFIRMED early signal to re-check: in the dish’s first hour fraction_obstructed climbed 0.0012 → 0.0222 (2.2%) and avgProlongedObstructionIntervalS fell 21600 s → 2700 s — obstruction stats need ~12 h to settle so this is not yet evidence, but the direction was worsening; the obstruction map at http://192.168.100.1 shows the blocked sector and starlink.csv is now the before/after record for any reposition

SDR & Radio Direction Finding

• 2026-08-26-krakensdr-field-test-milestone — ⭐ WORKING milestone: mobile TETRA-uplink DF end-to-end, field-testing across the city (2026-08-26). Two deltas: (1) the official Kraken Pro app HAS an iOS build (earlier “Android-only” assumption CORRECTED) — iPhone joins hunter-ap (10.42.0.1), app LOCAL to server 10.42.0.1, uses the phone’s own GPS for the map ⇒ the whole USB-tether/gpsd/USB-GPS-puck plan is UNNECESSARY for the DF map (Apple NMEA lockdown irrelevant); config already had doa_data_format="Kraken App" + krakenpro_key=0ae4ca6b3. Diagnostic: iPhone Safari http://10.42.0.1:8080/doa confirms the net path before blaming the app. (2) POWER: Pi 5 + DAQ browns out on 5V/3A (generic PD-30W does NOT help) → needs 5V/5A (throttled=0x0); earlier “OOM crashes” were largely brownouts. Field technique: ANT-0 points in direction of travel (app uses GPS heading), bursty/trunked signal (bearings only while a handset TXes — drive across/around, many samples), array_offset=0=relative, keep gain 19.7 just under overdrive. Final config: 382.114 MHz uplink, 19.7 dB, 25 kHz VFO, UCA 0.20 m, ext-3 whips, MUSIC, decorrelation Off, short-bursts On

• krakenpi-carbox — ⭐ the 3D-printed car enclosure for this same rig (KrakenSDR + cased Pi 5 + 12 V power bay, Fusion 360, model complete + verified 2026-08-26, not yet printed). Manufacturer dimensions worth reusing are in krakensdr-pi5-reference-dimensions — Kraken 177.3 x 113.5 x 25.86 mm (+4.7 fan guard), fixed by its own 8x M3 case screws on a 162.6 x 88.8 pattern (needs M3x16, the stock M3x12 no longer reaches through an enclosure floor), and the 5-SMA bank is offset −1.7 mm from the body centreline so a centred cutout row is wrong

• 2026-08-12-krakensdr-doa-rig — ⭐ 5-channel KrakenSDR direction-finding (DoA) rig on a Raspberry Pi 5 (raspi/192.168.1.200, Tailscale kraken-rig) for mobile TETRA (380–385 MHz) DF — DoA autostarts on boot, full chain verified (420 MHz TX → clean MUSIC peak, bearing ~294°). Same physical Pi as wifi-hunter (the wlan0 “Hotspot” AP is wifi-hunter’s). HW: Kraken = 5× RTL2838 behind an internal USB hub, NO serial/CH340 (noise-source cal over tuner GPIO via the krakenrf librtlsdr fork), own USB-C power, “Power Level: Overdrive” = RF over-gain not a weak PSU. SW: all from source, conda env kraken/py3.9, librtlsdr krakenrf fork REQUIRED; krakenrf/krakensdr_pr is 404 (→ krakensdr_suite, no PR) so PR from mirror mfkiwl/krakensdr_pr; trixie pins incl. dash_bootstrap_components 0.13.1 + dash_devices (imported as dash); DVB drivers blacklisted; DoA & PR both bind :8080 (never both). 🔴 start scripts’ conda activate silently fails over SSH/systemd → system python → ModuleNotFoundError: dash_devices; FIX = source .../miniforge3/etc/profile.d/conda.sh && conda activate kraken first. Autostart via /usr/local/bin/kraken-mode {doa|pr|off} + krakensdr.service. Array: UCA 0.30 m radius, MUSIC, Compass; physical pentagon 30 cm radius / 5× ~20 cm whips, spacing ≤ 0.5 λ → ~428 MHz ambiguity ceiling. 🚫 don’t TX on 380–385 (emergency band); RTL-SDR shows a fake DC spike at center freq (offset-tune). UI at kraken-rig.taild4189d.ts.net:8080/doa — the :8080 is REQUIRED

• 2026-08-26-raspi-oom-zram-hunter-ap-433-retune — ⭐ retuning the DoA above the array’s ambiguity ceiling. The 0.30 m TETRA pentagon has a ~428 MHz ceiling (spacing = 1.176 × R must stay < λ/2) so a 433.92 MHz retune (center_freq/vfo_freq_* in krakensdr_doa/_share/settings.json, backup .bak-pre433) is aliased/ambiguous on it — for unambiguous 433 DF rebuild to ~0.25 m radius + ext-1 whips (~17 cm ≈ λ/4) (λ=69.1, λ/2=34.6 cm) and update the custom_array coords. Config-only change (SDR was unplugged), applies next krakensdr.service start. Keyfobs are bursty OOK → Optimize-Short-Bursts ON, drop squelch. Also on this session: zram OOM fix + hunter-ap bring-up on the same Pi

Tailscale / Tailnet

• telep-router — Tailscale hand-upgraded 1.80.3 → 1.98.9 (static arm64 build; opkg still records 1.80.3-r1 → a future opkg upgrade may clobber it, backup /overlay/ts-backup/), nftables mode, 100.69.112.32 (was 100.115.194.51 before the 2026-08-03 factory reset); the workaround for having no public IP

• Since last handover (2026-07-24) — router tailscale upgrade done out-of-band via a setsid-detached self-reverting swap (busybox has no nohup); persist a binary backup on /overlay + auto-revert so a bad swap can’t lock you out of the router

• Access — Tailscale Serve as the TLS front door (needs HTTPS certs enabled in the tailnet admin console); persists across reboots

• Exposure — one tailscale serve port per app on telep-mainframe: :443 Frigate, :8443 top4 picker, :8445 Filestash

• 2026-08-09-xiaomi-c302-tailnet-facetime-cam — the C302 go2rtc WebUI exposed interim via a shared-node tailscale serve --bg --https=8450 http://127.0.0.1:1984 on the telep-mainframe node → https://telep-mainframe.taild4189d.ts.net:8450 (winning stream page webrtc.html?src=c302). ⚠️ per tailnet-service-exposure-convention a NEW service should get its own tag:telep node (c302.taild4189d.ts.net) rather than a shared serve port — noted as a follow-up

• homelab — tailnet peer list and cross-project role

• 2026-08-15-camera-alerts-dead-tailscale-magicdns-docker-dns — ⚠️ MagicDNS is a trap for Docker containers on telep-mainframe: when the host /etc/resolv.conf becomes nameserver 100.100.100.100 (Tailscale MagicDNS, via --accept-dns/MagicDNS enable), tailscaled intercepts it for the HOST but Docker’s embedded resolver (127.0.0.11) forwarding to it fails for containers (server misbehaving, external lookups return http 000). Enabling MagicDNS on this host on 2026-07-29 silently killed frigate-notify’s api.telegram.org resolution for 17 days. NB telep-router deliberately runs --accept-dns=false for this class of reason; per-service dns: overrides are the container-side workaround

• 2026-08-12-krakensdr-doa-rig — kraken-rig = 100.87.1.1 / kraken-rig.taild4189d.ts.net, tag:telep — the KrakenSDR DoA rig’s tailnet identity for remote field monitoring. DoA UI at http://kraken-rig.taild4189d.ts.net:8080/doa — ⚠️ the :8080 is REQUIRED (without it → port 80 → refused, which masquerades as an ACL problem)

• 2026-07-23-frigate-viewer-alert — tailscale serve writes the tailnet client IP into the proxied request’s X-Forwarded-For → a 100.x XFF in an nginx access line is a real human viewer, - is internal automation; map the IP → device name with tailscale status. The basis for identifying who opened the Frigate UI

• 2026-07-24-knowledgebase — a separate tailnet identity on one host = a second tailscaled (--tun=userspace-networking, own --socket/--statedir) then up --hostname= + serve; gives knowledgebase.<tailnet>.ts.net as its own device without a container or GPU passthrough. serve --https needs --statedir not --state=<file> or it fails “no TailscaleVarRoot” (no var root for the LE cert; TLS 000 while TCP/ping succeed)

• 2026-07-24-global-dashboard — home.taild4189d.ts.net, a second tag:telep node using the same separate-tailscaled-identity pattern; the knowledgebase became its own tag:telep node too

• Tailscale exit node — exit-vpn / 100.98.208.98 advertises an exit node + 192.168.105.0/24 subnet route (two separate admin-console checkboxes); off by default, tailscale set --exit-node=exit-vpn to use; exit nodes do not carry RFC1918 traffic; full-tunnel route changes strand tailscaled offline → oneshot unit restarts it after the tunnel

• 2026-08-03-telep-router-factory-reset-recovery — ⚠️ A factory reset forces a fresh Tailscale node registration → a NEW tailnet IP. The router moved 100.115.194.51 → 100.69.112.32; MagicDNS telep-router.taild4189d.ts.net is stable across the change, so prefer it in scripts. Stale telep-router + telep-router-1 duplicate nodes were deleted; re-joined tag:telep --ssh --accept-dns=false with an API-minted auth key, LuCI re-served on :443

• 2026-08-11-ruview-esp32-csi-real-hardware — ruview.taild4189d.ts.net, a new dedicated node following tailnet-service-exposure-convention: tailscaled-ruview.service (userspace networking, --statedir=/var/lib/tailscale-ruview, --socket=/run/tailscale-ruview/tailscaled.sock, --port=0) then up --hostname=ruview --advertise-tags=tag:telep + serve --bg 3000. ⚠️ The tag is what prevents a silent disappearance — untagged user-owned nodes get key expiry and would drop off the tailnet in ~6 months; home and pdf show keyexpiry: None because they’re tagged. Verified took here (tags: ['tag:telep'], keyexpiry: None, owned by the tag) — so --advertise-tags can work, but the convention’s tagged auth key is still the preferred route because --advertise-tags validates against the authenticating user’s tagOwners and can be rejected when that user doesn’t own the tag. ⚠️ moving a service to a new tailnet hostname can 421 if the app does Host-header validation (RuView’s SENSING_ALLOWED_HOSTS)

• 2026-08-31-nextcloud-drive-code-server-runbook — 🔴 standing rule: NO FUNNEL, EVER on drive/cad/knowledgebase/code — absolute for code-server, which runs --auth none so tailnet membership IS the credential; every serve status mount must read (tailnet only). Also: identity from tailscale whois → Remote-User injected by a loopback tsauth-proxy (only serve can front it) → this is precisely why Drive has no LAN vhost — a LAN entry point carries no tailnet source IP and would bypass identity. ⚠️ node hostname ≠ socket dir for the KB node (knowledgebase ↔ /run/tailscale-kb/); ⚠️ Taildrive shares serve as root (no per-share uid map in TS 1.102.2) so writes land root:root; ⚠️ the host cannot hairpin to its own tailnet HTTPS — browser hops are only provable from an in-ACL device

• 2026-08-31-telep-kb-mcp-server — 🔴 tailscale serve forwards the ORIGINAL Host header, which trips host-allowlist defaults in the app behind it: FastMCP on mcp ≥1.29 enables DNS-rebinding protection with an empty allowed_hosts (only 127.0.0.1 passes), so every tailnet request to /mcp returned 421 Invalid Host header while loopback worked. The 421 names no layer — it reads as a proxy or auth failure, not an app setting. Fix = FastMCP(..., transport_security=TransportSecuritySettings(allowed_hosts=[...])), overridable via TELEP_KB_MCP_ALLOWED_HOSTS. When a 4xx has no matching proxy log line, suspect the app before the proxy

• 2026-09-02-freecad-mcp-rpc-refused-gui-not-running — 🔴 the userspace-networking raw forwarder makes port scans lie about service health: cad:9875 answers the TCP handshake with FreeCAD entirely absent, so nc -z reports OPEN. Companion to 2026-08-31-tailnet-plaintext-port-hardening (same forwarder, security angle there, diagnostic angle here). Also: cad (100.120.203.1) is online but not Tailscale-SSH-enabled — only telep-mainframe and telep-router are — so it cannot be repaired remotely.

• 2026-09-05-freecad-mcp-502-dead-container-and-qt-event-loop-wedge — 🔴 A 502 from a tailnet name means the serve mount is UP and its backend is not — a materially different fingerprint from the Errno 111 / accept-then-close signatures of the raw forwarder. serve status on the cad node read :8443 -> http://127.0.0.2:9876 (⚠️ .2, per 2026-08-31-tailnet-plaintext-port-hardening — 2026-08-26-freecad-cad-workstation still documented 127.0.0.1) with nothing listening on 9876. Two checks in order when a tailnet mount 502s: which loopback does serve target, and does the backend actually bind that one (same .1-vs-.2 split that silently 502’d Homepage’s :8450, see 2026-09-02-lg-tv-network-control-presence) — then docker ps -a for an Exited container.

• 2026-08-31-nvidia-drm-host-crash-embedding-pass — ⚠️ pkill -f '<pattern>' on a Tailscale SSH host kills your own session: -f matches full command strings including the tailscaled SSH wrapper’s own, which contains the command you just typed — so the pkill matches itself. Use a bracketed pattern ([o]bsidian_index) or match on comm (pkill -x python3). Especially nasty while chasing a host crash, where the dropped session is indistinguishable from the crash

• 2026-08-31-nextcloud-drive-code-server-runbook — the code node is tagged tag:telep (matching cad/knowledgebase/home). Re-authing reuses the node key, so device approval is not retriggered; ⚠ --hostname=code must be passed explicitly or tailscale up resets it to the OS hostname

• 2026-08-31-nextcloud-drive-code-server-runbook — tsauth-proxy tag identity: configured tags map to a Nextcloud user via TS_TAG_USERS in /etc/tsauth-proxy.env. Only explicitly-configured tags map, tag:telep maps to nothing, client-supplied Remote-User is stripped after resolution, fails closed. ⚠ Tagging a device replaces its personal Tailscale identity

• 2026-08-31-nextcloud-drive-code-server-runbook — the drive node now serves three ports: 443 Nextcloud · 8443 OnlyOffice · 8444 drive-mcp

• 2026-08-31-openchatcut-chatcut-deploy — chatcut sidecar node (100.85.231.40, tag:telep) via tailscaled-chatcut.service (userspace, socket /run/tailscale-chatcut/…, statedir /var/lib/tailscale-chatcut); serve https://chatcut.taild4189d.ts.net/ → http://127.0.0.2:5199 (127.0.0.2 per plaintext-port-hardening). Provisioned via Aperture Tailnet_provision_node auth-key flow, admin-approved + tagged in console. ⚠ no app-level auth → tailnet ACL for tag:telep on a multi-user tailnet is the ONLY gate for the web UI

• 2026-09-01-orcaslicer-tailnet-deploy — orcaslicer sidecar node (100.126.7.69, tag:telep) via tailscaled-orcaslicer.service (userspace, socket /run/tailscale-orcaslicer/tailscaled.sock, statedir /var/lib/tailscale-orcaslicer); serve https://orcaslicer.taild4189d.ts.net/ → http://127.0.0.2:8570 (127.0.0.2 per plaintext-port-hardening). Provisioned via Aperture Tailnet_provision_node one-time-key, admin-approved + tagged. ⚠ truncated authkey → NeedsLogin (not NeedsMachineAuth)

• 2026-09-01-bambuddy-slicer-api-pipelines-mcp — path-mount instead of a new node: exposed the bambuddy MCP by adding serve --https=443 --set-path=/mcp http://127.0.0.2:8091/mcp on the existing bambuddy node’s socket (--socket=/run/tailscale-bambuddy/tailscaled.sock) — serve status shows both / (UI, preserved) and /mcp. ⚠ on tailscale 1.102.2 the mount strips the prefix and joins the remainder onto the target, so external /mcp → backend /mcp. ⚠ the mainframe’s userspace tailscaled (TUN:false) can’t hairpin to its own tailnet HTTPS → prove reachability from another node. No app auth → tag:telep ACL is the only gate; no Funnel

• 2026-09-02-lg-tv-network-control-presence — tv-control on the tailnet: /opt/tv-control/tv-http (stdlib HTTP, no framework) binds 127.0.0.2:8102 per the plaintext-forwarder invariant (2026-08-31-tailnet-plaintext-port-hardening) and is published tailnet-only at https://telep-mainframe.taild4189d.ts.net:8451 — POST /api/{on,off,camwall,pair}, GET → 405 by design. 🔴 Found and fixed in passing: Homepage’s tailnet mount at :8450 was silently 502-ing because tailscale serve pointed at 127.0.0.1:3010 while next-server binds 127.0.0.2:3010 — same 127.0.0.1-vs-127.0.0.2 split as the port-hardening work. When a tailnet mount 502s on this host, check which loopback the backend actually binds before anything else.

• 2026-09-04-starlink-wan-migration-dish-telemetry — 🔴 Starlink CGNAT and Tailscale BOTH use 100.64.0.0/10, and neither is configurable. The collision appeared the instant the WAN swapped (the NE200’s 192.168.254.2/24 did not overlap); the router now carries a link-scope 100.64.0.0/10 dev wan route. ✅ Tailscale survives because it uses policy routing: ip rule priority 5270 → table 52 (per-peer /32s), consulted before the main table. Verified healthy: tailscale status --json → "Health": [], UDP: true, IPv4: 150.228.113.241:36509, MappingVariesByDestIP: false — Starlink’s CGNAT is well-behaved for NAT traversal. 🚩 Trap: ip route get for 100.84.98.18 (bambuddy) / 100.120.203.1 (cad) resolves to dev wan, not dev tailscale0 — this is NOT collision damage, those peers are simply absent from table 52 because the ACL doesn’t grant telep-router (tag:telep) access to those tagged-devices; they’d be unreachable regardless. The only real consequence of the collision: packets to ACL-denied peers leak toward the Starlink link instead of failing closed. 🚩 Also ping is not a valid reachability test here (ICMP filtered). ⚠️ CGNAT means no inbound ports — everything must stay tailnet/DERP-mediated

• 2026-09-07-session-handover — chat = a new tag:telep sidecar node (tailscaled-chat.service) fronting the Matrix homeserver at https://chat.taild4189d.ts.net, tailnet-only, never Funnel — the same separate-tailscaled-identity pattern as 2026-07-24-knowledgebase. 🔴 The tailnet ACL BLOCKS tag:telep → these sidecar nodes on TCP/443, so telep-mainframe and telep-router cannot fetch chat.taild4189d.ts.net at all — chatcut and drive fail identically; this masquerades as a broken service. Self-verification from the mainframe is impossible: test from a phone/laptop or against the loopback backend 127.0.0.2:8008. 🔑 server_name was set to the tailnet hostname deliberately — because the server genuinely is there, no .well-known delegation is needed and Tailscale supplies the TLS cert (and it is permanent, baked into the DB). 📦 tailscale serve imposes NO upload size cap — verified by source inspection of ipn/ipnlocal/serve.go at v1.102.2 (no MaxBytesReader/LimitReader/ContentLength gate, stock httputil.ReverseProxy), and proven with a real 400 MiB upload returning HTTP 200

VPN

• 2026-07-22-facekom-vpn-exit-node — FaceKom “VirtualOffice” OpenVPN on the exit-vpn Incus container: tcp-client to 92.119.122.32:1194, AES-256-CBC/SHA1, full tunnel via redirect-gateway def1, persistent through openvpn-client@farm (symlink into /etc/openvpn/client/ — the tt/ subdir isn’t scanned and openvpn.service is a /bin/true stub)
• Three root causes (all previously misdiagnosed) — the “bad CA” that stalled this for weeks was really a wrong remote IP; plus wrong ciphers and a label-corrupted password file
• SESSION-HANDOVER — one-screen current state in the exit-vpn container section

WiFi

• 2026-09-05-session-handover — ⭐ 2.4 GHz interferer hunt: radio1 moved ch11 → ch1, channel busy 92.9% → 25.8% (backup /etc/config/wireless.bak-1788552343); ch1 chosen as non-overlapping with the HP printer’s WiFi-Direct on ch6 at −20 dBm. 🔎 The survey proves a non-WiFi emitter the user OWNS, not neighbours: ch11 93% busy while our own traffic was only ~20%, noise floor −60…−76 dBm across ch8–13 vs −93…−98 dBm on ch1–7, and only 1–3 APs in the entire scan. 🆕 /home/levander/net-monitor/wifi_survey.py → wifi_survey.csv, cron daily 04:00 (a scan briefly interrupts clients, hence nightly); it only measures, never changes a wireless setting, and runs mainframe→router over SSH to 192.168.1.1 (the LAN IP — the tailnet ACL blocks port 22 mainframe→router). ⚠️ CSV caveats: the in-use row’s busy_pct is a LIFETIME average over monotonic counters (needs deltas between consecutive rows, handling active_time_ms decreasing on interface restart), and scanned rows are a single ~150 ms sample and very noisy (ch2 read 98.67% then 78.0% nineteen seconds apart) — active_time_ms is present precisely to separate high- from low-confidence rows. 🚩 Parser trap: Channel:\s+(\d+) on iwinfo scan also matches Primary Channel: in HT/VHT blocks and double-counts every AP — anchor on ^Mode:, key by frequency
• 2026-09-02-tv-presence-wifi-union-daemon — ⚠️ query ONLY phy0-ap0 (ESSID telep1, 5 GHz) and phy1-ap0 (telep1-2G) for presence — NOT phy1-ap1, whose ESSID is telep-cc, the camera VLAN (leaks cameras into presence). Also: iwinfo <bad-ap> assoclist prints “No such wireless device” and exits 1, so wrap it as for ap in ...; do iwinfo $ap assoclist || exit 1; done — a renamed AP must fail loudly, not return an empty set that reads as “nobody home”.
• telep-mainframe-handover — current WiFi state (2026-08-19): the telep1 SSID was SPLIT — telep1 is now 5 GHz only (radio0, ch36, HE40) and 2.4 GHz is a separate SSID telep1-2G (radio1, ch11) for range/through-walls; both network=lan. Cams SSID telep-cc (2.4 GHz, isolated cams VLAN). TX power pinned at the HU/ETSI ceiling (23/20 dBm). (The telep-router note predates the split — the handover is the current record.)
• telep-router — earlier state: SSID telep1 on both radios; radio0 5 GHz ch36 HE40 (narrowed from HE80 2026-08-15), radio1 2.4 GHz HE20 (now telep1-2G on ch11 — see the handover)
• [[telep-router#tp-link-tl-wa850re-added-as-a-wired-ap-ssid-bandi|TP-Link TL-WA850RE added as a wired AP (SSID bandi)]] — a TP-Link TL-WA850RE (2.4 GHz N300, single-band) reconfigured from repeater → Access Point, wired into br-lan, broadcasting SSID bandi (this is the bandi previously seen on 2.4 GHz ch2 in scans — own device, not a neighbour). BSSID ac:84:c6:1b:04:31, static reservation .101. Coverage AP, not fast-lane (100 Mb link + N300 ceiling). Gotcha: a repeater-mode extender has no router IP (default 192.168.0.254, reach via Tether app / temp 192.168.0.x); confirm it is truly wired via /sys/class/net/lanN/carrier + brctl showmacs br-lan before trusting the uplink
• 2026-08-15-camwall-dead-br-cams-bridge-down-after-wifi-reload — ⚠️ a wifi reload during WiFi tuning (radio1 channel 1→11) is what dropped the wireless-only cams bridge br-cams → the whole cams VLAN + camwall went down. Any wifi reload touching radio1 must be followed by the br-cams down/up recovery
• 2026-08-15-router-wifi-tuning-htmode-band-penetration — ⭐ weak signal / poor wall penetration: TX power is NOT a lever — both radios already at the HU/ETSI ceiling (5 GHz 23 dBm, 2.4 GHz 20 dBm; HW 30 dBm only on DFS ch 100–140). Mitigated by narrowing 5 GHz HE80→HE40 (uci set wireless.radio0.htmode='HE40'; uci commit wireless; wifi reload) → same power into half the bandwidth ≈ +3 dB range/penetration, peak 1200→600 Mbps, reversible. Survey: two 5 GHz clients at −83/−85 dBm dying through walls; Észak cam .139 @ −67 dBm (weak, drops RTSP) vs Dél .119 @ −42 dBm. Biggest un-applied win = band steering (dawn package) to push far clients to 2.4 GHz; real dead-zone fix = a 2nd AP/mesh with wired backhaul. Keep 2.4 GHz at 20 MHz (never 40)
• 2026-08-26-router-reflash-mac-5ghz-dfs-and-bambu-offline — ⭐ a Mac won’t list a 5 GHz SSID that’s on a DFS channel (52–144) while phones show it instantly (“it’s the Mac” — macOS passive-scans DFS). Fix router-side: 5 GHz radio to non-DFS (36/40/44/48 or 149/153/157/161). ⚠️ macOS networksetup -getairportnetwork / system_profiler SPAirPortDataType LIE about association (Location-Services CLI redaction) — verify with ifconfig en0 (status active + inet), ipconfig getifaddr en0, route -n get default, ping; and don’t setairportpower off/on (leaves en0 stuck, err -3900). Post-reflash the 5 GHz SSID is telep1-5G (Mac saved nets had only telep1-2G+telep1). Correction (2026-08-26): the DFS theory was wrong — the 5 GHz SSID is telep1 on ch36 (non-DFS).
• Problem 3 — Wi-Fi cross-band client isolation (driver-level) — ⭐⭐ DRIVER-level cross-band Wi-Fi client isolation: a client on telep1 (5 GHz, phy0-ap0) can’t reach a client on telep1-2G (2.4 GHz, phy1-ap0) even though EVERY config knob reads non-isolating (hostapd ap_isolate only on guest telep-cc; uci isolate='0'; bridge isolated=0 learning=1 *_flood=1; br-lan vlan_filtering=0) and wifi down; wifi up changes nothing. Decisive tell: wifi→ethernet works but wifi→wifi cross-band fails ⇒ isolation is in the radio driver/firmware, below all config. Fix: wire the device over Ethernet (reachable from any band) or same-band the client. Tailscale does NOT help an on-LAN client — the connected en0 route beats any subnet-route; subnet-router is only for reaching a device from OUTSIDE the LAN. ⚠️ This is NOT the AirPlay bug — telep-tv undiscoverable on Wi-Fi is the band-independent 10G-port multicast issue below
• 2026-08-30-telep-tv-airplay-10g-port-multicast-not-flooded — ⭐ telep-tv AirPlay undiscoverable on Wi-Fi = the box’s 10G switch port, NOT 5 GHz. Corrects the months-long misattribution to cross-band isolation. Band-independent (fails on 2.4 AND 5 GHz); box’s mDNS multicast is on the wire (tcpdump on enp5s0 → 224.0.0.251) but the switch won’t flood it from the 10G port (br-lan port 2) to Wi-Fi, while 1G-port devices (printer) do reach Wi-Fi. Unicast fine (dns-sd -L resolves, ping/ssh/connect work); only dns-sd -B browse fails. Ruled out multicast_to_unicast (tried both SSIDs → reverted), Mac cache flush, band. Fix = 1G port, or an mDNS reflector
• Camera VLAN (telep-cc) — hidden SSID, WPA2-PSK forced (Tapo has no WPA3-SAE), client isolation on; hidden-SSID rejoin tested and works
• 2026-08-12-airplay-mdns-fix-printer-migration — ⭐ AP client isolation on telep1 was breaking wired→WiFi mDNS/AirPlay even though uci said isolate=0 — OpenWrt auto-injects ap_isolate=1 at hostapd-config-gen time for bridged non-isolated APs with multicast_to_unicast on. Fixed by patching /lib/netifd/netifd-wireless.sh line 309 (telep1 de-isolated on both radios; the cams SSID telep-cc, which has an explicit isolate=1, STAYS isolated). Reapply after any OpenWrt sysupgrade
• Presence detection — AP map used for presence: telep1 = phy0-ap0 (5 GHz) + phy1-ap0 (2.4 GHz); telep-cc camera SSID = phy1-ap1 (excluded)
• 2026-07-22-wifi-usage-strip — same telep1 AP map for per-station byte counters (iw dev <ap> station dump); telep-cc excluded by construction, never referenced at all
• 2026-07-31-wifi-qr-code-sheet — printable scan-to-join Wi-Fi QR sheet on the Mac; payload WIFI:T:WPA;S:;P:;; (T:WPA also joins WPA3/SAE telep1); read ssid/key live from the router, passwords kept out of the vault
• 2026-08-12-krakensdr-doa-rig — the KrakenSDR Pi has a single WiFi radio hosting wifi-hunter’s NetworkManager “Hotspot” AP on wlan0 (10.42.0.0/24); AP and client can’t run at once, so the planned car deployment flips wlan0 AP → client on the driver’s phone hotspot for internet on the move (needs the SSID+password preloaded)
• 2026-08-26-raspi-oom-zram-hunter-ap-433-retune — 🔑 CLARIFICATION on the above: eth0 (wired) + wlan0 (AP) run SIMULTANEOUSLY. The single-radio limit is intra-wlan0 only (it can’t be AP and client at once) — on the bench the Pi keeps its wired uplink/tailnet AND serves hunter-ap at the same time; the AP↔client flip only matters in the field once eth0 is unplugged. Bring the AP up with sudo nmcli connection up Hotspot (SSID hunter-ap, 10.42.0.1/24; DoA UI http://10.42.0.1:8080/doa)
• 2026-07-31-camwall-lag-24ghz-wifi-bufferbloat — the camera radio phy1-ap1 (telep-cc, 2.4 GHz ch1 / 20 MHz) is 98.9% channel-busy — saturated by the cameras’ own uplink (4 continuous HD Frigate record streams), noise floor clean at -106 dBm → bufferbloat (350–420 ms ping, 0% loss) that lags the camera wall. Off-2.4 GHz (Ethernet/PoE or 5 GHz) is the real fix; channel change won’t help
• 2026-08-11-ruview-esp32-csi-real-hardware — WiFi as a sensor (CSI), not a link. An ESP32-S3 in promiscuous mode feeds channel-state information to RuView over UDP 5005. 📶 Placement finding, counter-intuitive: further from the AP is BETTER. Moving the node from RSSI −26 → −43/−44 dBm took confidence off a pinned 0.50 up to 0.58–0.61 — too close and the direct path dominates so a human body is only a small fractional perturbation of the channel. Target −40 to −60 dBm, torso height, rigidly mounted (a node that moves is indistinguishable from a room that moves), with the area of interest ON the AP↔node line — that link is the sensor
• 2026-09-04-starlink-wan-migration-dish-telemetry — ❌ do NOT “optimise” radio0 from HE40 to HE80 — it cannot improve internet speed. The wired line is ~250 Mbps and HE40 already yields a 573.5 Mbit/s PHY (ch36); HE80 would only help LAN-to-LAN transfers. Current radios: radio0 5 GHz ch36 HE40, radio1 2.4 GHz ch11 HE20; SSIDs telep1 (5G/lan), telep1-2G (2.4G/lan), telep-cc (2.4G, cams, isolate='1'). 🔴 The TL-WA850RE (192.168.1.101) caps at ~50 Mbps ≈ 20% of the line — but CORRECTED 2026-09-05: it is NOT a repeater. It is already a wired AP on br-lan (SSID bandi); the ceiling is its 100 Mb Fast-Ethernet port + single-band N300 radio + 2.4 GHz congestion. “Replace the repeater with a wired AP” is a no-op — that conversion already happened. A dual-band AP on a gigabit port is the actual upgrade. It also answers on a second factory IP 192.168.0.254 (same MAC), which explains a mystery ARP entry. 🔴 A Starlink router can never serve as an AP (Bypass = WiFi off; non-Bypass = it IS the router; mesh nodes have no Ethernet backhaul; reversing Bypass needs a factory reset)