Homelab Activity Log

For Agents

Reverse-chronological session log. Newest entries at top, grouped by date (## YYYY-MM-DD). Each bullet: one piece of work, short summary, wikilinks to docs touched. Updated by obsidian-documenter on every project doc write. Read by historian at bootstrap.

2026-09-07

• 🚨 ⭐ intruder-alarm RETIRED — archived, not deleted (2026-09-07-alert-source-inventory §6). Source 6 of the alert inventory is gone from live paths: /etc/systemd/system/intruder-alarm.service and /home/levander/intruder-alarm.py both confirmed removed, daemon-reload run, and systemctl status now returns “Unit intruder-alarm.service could not be found”. Archive at /root/retired/2026-09-07-intruder-alarm/ holding the unit, the script and a README — restore is one step: mv both back + systemctl daemon-reload. Safe because nothing depended on it, grep-verified — the only hit outside its own files was a comment in /home/levander/ruview/scripts/c6-presence-watcher.py (# so the operator never sees "intruder detected" framing) plus two coincidental word matches in unrelated ruview/examples/research-sota/ research files; evidence recorded so nobody re-derives it. It was never portable in the first place — 1175 lines of an interactive Telegram bot (send_photo, edit_message_text, inline keyboards, callback polling for MAC enrollment) and Matrix has no inline-keyboard equivalent. 🧹 Side effect: one source of journal noise is gone — its unit emitted Unknown key 'StartLimitIntervalSec' in section [Service] on every daemon-reload. ✅ No wider misconfiguration to chase: camwall.service, camwall-x.service and nvidia-cdi-refresh.service all carry that key correctly in [Unit]. 🔑 Consequence for the token block: intruder-alarm was one of the readers of frigate-notify/config.yml’s telegram: block; remaining readers are top_kep_remind and frigate-viewer-alert (both deliberately kept for rollback) plus the duplicate top-kep-remind.py — deletion is still unsafe, the count changed but not the verdict.
• 🔒 ⭐⭐ GOTCHA: Tailscale SSH logs the FULL command line to the journal (2026-09-07-alert-source-inventory). tailscaled writes the entire invocation of every Tailscale SSH command into the journal, which bites twice. (1) Debugging — your own grep command becomes a match. A command containing the literal StartLimitIntervalSec, run to check whether that warning had stopped, matched itself — making an already-fixed problem look unfixed. Rule: when grepping the journal for a string, remember your own command is in there too — filter on the systemd[1]: prefix, or grep for the shape of the message rather than a bare keyword. (2) 🔴 Security — never pass a secret as a command-line argument over Tailscale SSH; it lands in the journal in plaintext. ⚠️ Directly relevant to the pending Telegram token rotation (2026-09-07-session-handover open item 4): set the new token by editing files or via stdin, never as an argument. 🔴 And it compounds with the migration: now that top_kep_remind is on Matrix it is no longer the loud HTTPError 401 canary, so a missed rotation location fails completely silently — a careless rotation can therefore both leak the new token into the journal AND quietly leave a live one behind. Verify all three locations by hand. ↔️ Also recorded: retire by archiving, not deleting (mv + daemon-reload restores), and StartLimitIntervalSec belongs in [Unit], not [Service] — recurring Unknown key … noise is usually one bad unit, not a fleet-wide problem.
• 🔁 ⭐⭐ The four remaining Telegram alerters MIGRATED to the Matrix relay — done, verified, running (2026-09-07-alert-source-inventory). /usr/local/bin/frigate-fps-watchdog.py → [fps-watchdog], /etc/nut/nut-outage-handler.sh → [ups], /home/levander/top_kep_remind.py → [top-kep], /home/levander/frigate-viewer-alert/{notify,watch}.py → [viewer-alert], all posting to http://127.0.0.2:8118/<label>. Pattern: Telegram code kept INTACT but INERT, mirroring bambuddy’s id=1 disabled-not-deleted precedent — each swap is one line, and the exact rollback line is recorded per script (alert = send_matrix ← alert = lambda text: send_telegram(token, chatid, text) at fps-watchdog:169; send_matrix "$1" ← send_telegram "$1" at nut-outage-handler.sh:51 in send_alert(); send_matrix(text) ← send_telegram(text) at top_kep_remind.py:52 in send(); emit = send_matrix ← emit = lambda text: send(token, chatid, text) at watch.py:67). .bak backups beside all five edited files; 34 unit tests pass; alert semantics UNCHANGED — viewer-alert still fires on any tailnet request to Frigate, proven by a delivery triggered by GET / rather than /live/. 🛡️ The live room was never touched: all testing ran on a throwaway relay on :8119 with its own device and crypto store (since removed — device UrCaL98Ni3 logged out; @alerts now lists only the live uremzoqE3U | matrix-relay), the live relay PID 1564638 was never restarted, and journalctl -u matrix-relay | grep -c 'sent event' → 0 across the whole migration window. Independently re-verified afterwards: all four MATRIX_URL constants grep-confirmed as :8118, no stray 8119 anywhere, 8119 unbound, live relay listening on 127.0.0.2:8118 + 172.18.0.1:8118. 🟡 Still unproven — the :8118 production path was NEVER exercised end to end; everything was proven on :8119 and the only delta is the port digit, visually confirmed but not traffic-tested (testing it would have posted into the live room). The first real alert is the test — the 20:00 top-kep reminder, and viewer-alert the next time anyone opens Frigate. Also unproven: no real UPS event simulated (send proven with the three real message strings, script run end to end via the safe unmapped COMMBAD branch) and no unattended timer fire (weekly triggered manually). 🔴 top_kep_remind.py --daily deliberately NOT run for real — it advances a watermark and would have silently suppressed that evening’s 20:00 family reminder; --dry only, watermark still 1788717196. 🔑 Consequence for the credential rotation: rotating the Telegram bot token is now SAFE (all four alerters are on Matrix; the Telegram path is inert rollback only) — but deleting the telegram: block is STILL NOT, because load_telegram_creds() is deliberately still called for rollback, so deletion would still crash-loop frigate-viewer-alert. The world-readable router copy in /etc/casino-alert.sh is untouched and still needs rotating. Untouched: relay source, /etc/matrix-relay.env, config.yml, cameras.exclude, telep-router, intruder-alarm.py. Open item 12 in 2026-09-07-session-handover flipped from 🚧 to ✅.
• 🎯 ⭐⭐ GOTCHA: Continuwuity’s /_matrix/client/v3/rooms/{room_id}/event/{event_id} IGNORES the room in the path — it resolves purely by event ID, so /rooms/{scratch}/event/{id} cheerfully returns an event that lives in the live room, and vice versa (2026-09-07-alert-source-inventory). A naive “is this event in room X?” check therefore returns YES for any event that exists at all, and it fails in BOTH directions: it reports pollution that never happened and it hides a genuine live-room leak. Hit for real this session (a false positive on the first room check) and confirmed with a control — a known live-room event id, requested through the scratch-room path, returned successfully with room_id = the live room. ✅ Reliable checks: read the returned event’s own room_id field (authoritative — the URL is not); to audit a room, page /rooms/{live}/messages?dir=b instead of per-event lookups; or count at the source with journalctl -u matrix-relay --since '…' | grep -c 'sent event'. ⚠️ Directly undermines any per-event re-audit of the “Telep Cam9” test-pollution incident and the 41 redactions (2026-09-07-session-handover §5/§6.3) — that method would have given a wrong answer. Other reusable rules from the same work: migrating an alerter is a one-function URL swap — keep the old transport callable and record the exact rollback line; test a new transport against a scratch relay instance, never the live one (own device, own crypto store, own port, torn down after); and some “test” invocations have side effects that suppress future real alerts — a watermark, a debounce, a rate-limit window — check for state advancement before running an alerter for real.
• 🔔 ⭐⭐ Full alert-source inventory — six producers, four silent holes, and two corrections to the vault (2026-09-07-alert-source-inventory). Six alert producers exist across telep-mainframe + telep-router; two already on Matrix (bambuddy provider id=2, frigate-notify webhook → 172.18.0.1:8118/frigate), four still on Telegram — frigate-fps-watchdog (/usr/local/bin/frigate-fps-watchdog.py, creds /etc/nut/telegram.env; fires most — 12+ real alerts Sep 4: cam restarts, a breaker trip, recovery), the NUT outage handler (/etc/nut/nut-outage-handler.sh via upsmon NOTIFYCMD; real outages Aug 18/19/20/22), the top-kép reminders (~/top_kep_remind.py, 3 timers — a family voting nudge, not security, candidate for its own room) and frigate-viewer-alert (highest volume; ⚠️ fires on ANY tailnet request to Frigate, not just /live/, camera optional, 300 s per-IP gate; 🎣 its firing rate is UNPROVABLE from logs — notify.send() prints only on failure; evidence it works = stable PID 4d22h, 3 failures in 60 days, 98 tailnet-XFF hits in 20k log lines). 🚧 Their migration was IN PROGRESS when this entry was written — ✅ it completed and was verified later the same day; see the entry above. ⛔ Two are not migratable: router casino-alert is infra-blocked (relay binds 127.0.0.2 + 172.18.0.1, neither reachable from the router) and is running TWICE (two PIDs, each with its own tail -F ⇒ every hit double-sends; never observed firing, but state lives in /tmp and resets on boot ⇒ weak window, unproven not dead); intruder-alarm (confirmed still disabled/inactive) is 1175 lines of an INTERACTIVE bot — send_photo, edit_message_text, inline keyboards, callback polling for MAC enrollment — and Matrix has no inline-keyboard equivalent ⇒ leave dormant or retire; porting is a rewrite, not a migration. 🔴 HEADLINE — Telegram fails exactly when it is needed: 31 of 34 UPS alert sends FAILED, all clustered inside the Aug 7 (6) and Aug 18 (25) outage windows — the power was out ⇒ the WAN was down ⇒ the alert could not leave the building; successes landed only after power/network returned. Continuwuity runs locally, so a Matrix alert still delivers on battery ⇒ the UPS migration is a CORRECTNESS FIX, not tidiness — that alarm has never worked during the event it exists for. 🔴 Silently broken, nobody watching: smartd alerts go NOWHERE (/etc/smartd.conf -m root -M exec …/smartd-runner → 10mail, but NO MTA is installed at all — no sendmail/mail/mailx/msmtp/postfix/exim, /var/mail empty, nothing in the journal; given the earlier NVMe damage this is arguably the biggest hole on the box); netdata notifies nobody (all 28 SEND_* = "NO", a deliberate override of the stock YES); no OnFailure= on ANY of the ~70 custom units ⇒ nothing alerts when a service dies. 🔴 The leaked Telegram token is in THREE places, not one — ~/nvr/frigate-notify/config.yml, /etc/nut/telegram.env (0600), and hardcoded plaintext in /etc/casino-alert.sh on the router, mode 0755 — WORLD-READABLE; rotation must cover all three. ⚠️ Rotate ≠ delete: rotating leaves alerters silently posting to a dead token (silent for viewer-alert / fps-watchdog / nut-handler / casino-alert; LOUD only for top_kep_remind — urlopen raises HTTPError 401, no try/except, unit fails visibly), while deleting the telegram: block makes load_telegram_creds() raise ValueError ⇒ frigate-viewer-alert crash-loops on Restart=always ⇒ do not delete the block until the three readers are migrated. ✅ CORRECTION 1 — print-guard and tv-presence are NOT alert producers, contra open item 11 / 2026-09-05-session-handover item 8: print-guard (290 lines) calls only the bambuddy API on 127.0.0.2:8000 (GET queue/printers, PATCH …/{id} {"manual_start": true}) and tv-presence (241 lines) has NO HTTP client at all (SSH to phy0-ap0/phy1-ap0 + arp-scan, then drives /opt/tv-control/tv) — “never fired” is about their ACTION paths; neither has ever messaged anybody. ✅ CORRECTION 2 — the relay’s lack of filtering is now CONFIRMED by source, not inferred: source = parts.path.strip("/").split("/")[0] ⇒ the path is just a LABEL, render_payload falls back to title/message then raw text, decorate prefixes [source] — it accepts a POST on ANY path and posts whatever it receives. 🔴 So the relay is UNAUTHENTICATED and UNVALIDATED on 127.0.0.2:8118 + 172.18.0.1:8118 (ss -lntp: exactly those two binds, one pid; tailnet IP 100.115.209.87 not bound) ⇒ any container on the nvr_default docker bridge can post arbitrary content into the LIVE alert room. ↔️ Same property = migrating needs no relay change (one-function URL swap to http://127.0.0.2:8118/<name>). 🎣 Ruled out, do not re-check: internal-cam-alert.py (X11 banner on DISPLAY=:0), telep-selftest (api.telegram.org only as a reachability probe — it will keep showing up in Telegram greps), thermalwatch/dnsmon/camwall-watchdog/camwall-frigate-watch/wifi-usage/power-restore.sh, net-monitor probe.sh/starlink_status.py/wifi_survey.py (CSV only), continuwuity-backup.py (.err file), router cams-guard.sh (logger only), and Frigate itself (mqtt: enabled: false, no webhooks ⇒ no independent notification path). No ntfy, gotify, pushover, discord, slack, healthchecks.io, Home Assistant or Uptime Kuma on either host. 💡 New gotchas: curl -s without -f exits 0 on HTTP 4xx ⇒ scripts log “sent” while sending nothing · a “disabled but not deleted” rollback block can become load-bearing (three scripts now read frigate-notify’s disabled telegram: creds — grep before deleting) · an alerter sharing a failure domain with what it monitors is not an alarm · “never seen it fire” is only as strong as the state you kept
• 🖼️ ⭐⭐ Both Matrix alert defects FIXED and verified — and the recorded root cause was WRONG (2026-09-07-session-handover). (1) Frigate snapshots now render. ⚠️ The suspected incomplete info block was NOT the cause — the m.image event was spec-compliant all along (correct msgtype, non-empty body, complete info, well-formed encrypted file: v: "v2", A256CTR, iv, hashes.sha256, mxc://, no stray plaintext url), and no thumbnail_file is needed — Element renders encrypted images without one. 🔴 Real cause = a CONCURRENCY RACE: the relay’s _prepare() called add_changed_users() + keys_query() from the HTTP thread while nio’s sync_forever consumed the same shared users_for_key_query set — whichever won left the other calling keys_query() on an empty set, raising LocalProtocolError: No key query required., which kills the Matrix session and closes the aiohttp client BETWEEN the text send and the image send (sent event $kV3Y… → matrix session ended → image send failed: RuntimeError: Session is closed). Intermittent and timing-dependent, so aggregate stats looked healthy. Fixes: stop mutating nio’s shared key-query state (device freshness now from read-only device_store inspection; room_send already handles members_synced/keys_query), text+image in ONE coroutine, up to 3 attempts, with the text tracked so a retry never duplicates it, and a fallback chain snapshot.jpg → thumbnail.jpg → /api/<camera>/latest.jpg that no longer skips when has_snapshot is false (bogus id → both 404 → 106 KB frame recovered from latest.jpg). ✅ Zero session ended / Session is closed since the fix; live person detection on telep_cam3 at 03:11:22 decrypted as @phone = 162,321 bytes, valid JPEG 1280×720. 🔴 Why a cam3 alert exists at all when cam3 is excluded — and the generalisable footgun: cameras.exclude is a FRIGATE-NOTIFY setting enforced UPSTREAM of the relay, and the relay has NO camera filtering of its own — it posts whatever hits its webhook. The genuine cam3 detection was replayed straight at 127.0.0.2:8118, bypassing the exclusion because the exclusion never ran; cameras.exclude was genuinely untouched and cam3 genuinely still notifies nothing through the normal path — both facts are true. ⚠️ Inferred from the fixing agent’s report + the architecture, NOT verified on the box (nobody has grepped the relay source for a camera filter — that is the one-line check). Consequence: ANY direct POST — testing, replay, a future integration — reaches the LIVE alert room regardless of camera, which is exactly how the “Telep Cam9” test alert got in and why #relay-scratch now exists. If a camera must never appear in a room, the relay needs its own filter. (2) bambuddy finish photos now attached. Key is finish_photo_url, a RELATIVE path /api/v1/archives/{archive_id}/photos/finish_<ts>_<hash>.jpg, present on print_complete / print_failed / print_stopped. 🚩 Confirmed: /print-log/{id}/thumbnail AND /archives/{id}/thumbnail BOTH return the SLICER RENDER (512×512 PNG) — the camera capture exists only under /archives/{id}/photos/{filename}. Same resolve_media → Fetcher → encrypted upload → m.image path as frigate, plus PNG magic-byte/dimension support; proof from archive 94 decrypted as @phone = size: 424330, w: 1680, h: 1080, SHA256 byte-identical to source (b861624248f8550c…). finish_photo_url is now suppressed from the alert text. 🟡 STILL UNPROVEN: the live print_complete trigger has never fired with a photo — bambuddy’s provider test endpoint sends a generic payload with no finish_photo_url, so only a real print completion confirms the hook; print_failed/print_stopped are inferred from the same key and unfired, only car/person label emoji were exercised, and the latest.jpg fallback derives the camera slug from the display name (Telep Cam1 → telep_cam1) so a renamed camera would silently skip it. 🧹 Test-pollution remediated: all synthetic traffic ran in an isolated scratch instance (own @alerts device + crypto store, port 8119, room #relay-scratch:chat.taild4189d.ts.net), now stopped, logged out, files removed, port gone — ✅ but the scratch ROOM is retained as the designated place for test alerts, the direct answer to the “Telep Cam9” incident. ⚠️ ALL 41 relay-sent events were REDACTED from the live telep-ertesitesek room, including two real-sender verification alerts — irreversibly. There is NO alert history before this point; an empty room is expected, not a delivery failure. Pre-change relay backed up at /root/.mrelay/matrix-relay.bak3; regression checks all clean (binds 127.0.0.2:8118 + 172.18.0.1:8118 with 127.0.0.1 refused, Restart=always/RestartSec=10, env 0600 / store 0700 / token absent from journal, bambuddy id=2 on & id=1 disabled-not-deleted, frigate-notify telegram: false / webhook: true, cameras.exclude untouched, never-drop verified).
• 🗂️ ⭐⭐ Session handover written for the 2026-09-05 → 2026-09-07 run (2026-09-07-session-handover) — successor to 2026-09-05-session-handover. Headline: a self-hosted Matrix homeserver with all homelab alerts rerouted off Telegram into it, plus a restore-tested backup. ✅ Both alert defects are now FIXED and verified (see the entry above) — image attachments render and bambuddy finish photos are attached; ⚠️ the suspected incomplete info block was NOT the cause — it was a concurrency race that killed the Matrix session between the text and image sends. 🟡 The live print_complete trigger has still never fired with a photo. 🔴 Biggest standing risks: NO off-site backups (Matrix backups, bambuddy backups and live data all on the same 3.6 T LVM volume) and two unrotated leaked credentials.
• 💬 ⭐⭐ Matrix homeserver LIVE — Continuwuity v26.8.1 (forgejo.ellis.link/continuwuation/continuwuity:v26.8.1, digest sha256:fdf3cd0f…, pinned). Compose /home/levander/matrix/, config continuwuity.toml, RocksDB data /home/levander/matrix/data. Backend 127.0.0.2:8008 (127.0.0.1 refused); own sidecar node chat (tag:telep, tailscaled-chat.service) at https://chat.taild4189d.ts.net, tailnet-only, never Funnel. Federation OFF + open registration OFF, both proven with live 403s. Accounts @andras (admin), @phone, @mfalusi, @puliki, @alerts (bot) + built-in @conduit; passwords in /root/matrix-credentials.txt (0600). 🔴 server_name = chat.taild4189d.ts.net is PERMANENT — baked into the DB, changing it means wiping everything; chosen deliberately over a custom domain because the server genuinely is at that hostname, so no .well-known delegation and Tailscale supplies the cert. 📦 max_request_size = 536870912 (512 MiB, 25× default) because the point is uncompressed phone photos — proven end-to-end with a real 400 MiB upload returning HTTP 200, and tailscale serve’s reverse proxy verified CAPLESS by source inspection of ipn/ipnlocal/serve.go at v1.102.2 (no MaxBytesReader/LimitReader/ContentLength gate, stock httputil.ReverseProxy). 🚩 Gotchas: conduwuit is ARCHIVED (Continuwuity = maintained community continuation, Tuwunel = competing live fork — do not install conduwuit), but the rename is only skin deep — binary /sbin/conduwuit, data dir /var/lib/conduwuit, log modules conduwuit_*, admin bot @conduit:…, example config conduwuit-example.toml; the image is DISTROLESS (no shell, no curl ⇒ no docker exec debugging, no meaningful healthcheck); allow_announcements_check and allow_check_for_updates are ALIASES of one field — setting both is a fatal duplicate field that crashloops the container; /.well-known/matrix/client 404s unless [global.well_known] client is set explicitly — it is NOT derived from server_name. 📱 QR sign-in (MSC4108) is NOT implemented in v26.8.1 — /_matrix/client/v1/rendezvous and both unstable paths 404, and no rendezvous code exists in the source tree; the token half (MSC3882, get_login_token: true) and full OAuth 2.0 are implemented. Re-check after a version bump. ✅ OAuth browser sign-in DOES work and is the practical answer — auth_metadata advertises authorization_code/refresh_token/device_code at /_continuwuity/oauth2/*, and both Element Desktop and Element X on iOS handed off to a browser instead of an in-app password form (server log: Issuing OAuth authorization code client_name="Element"). 🔴 The Admin Room must stay UNENCRYPTED — the @conduit bot must read !admin in the clear and room encryption is one-way in Matrix, so enabling it would permanently break user management. 🔴 Self-verification from the mainframe is IMPOSSIBLE — the tailnet ACL blocks tag:telep → these sidecars on TCP/443, so telep-mainframe and telep-router cannot fetch https://chat.taild4189d.ts.net at all (chatcut and drive fail identically — it’s the ACL, not a broken service); test from a phone/laptop or against 127.0.0.2:8008. 👤 User management in the Admins room: !admin users create-user|reset-password|list-users|make-user-admin|deactivate|suspend|logout — ⚠️ the reply prints the password in PLAINTEXT into the room and it persists in the database; redact after copying.
• 💾 ⭐⭐ Matrix backups built AND restore-rehearsed — /usr/local/sbin/continuwuity-backup.py, root crontab 30 3 * * * (deliberately offset from bambuddy’s 03:00), dest /home/levander/backups/continuwuity/ (dir 0700, archives 0600), 7-day retention with pruning. Strategy = native online RocksDB backup, zero recurring downtime, triggered non-interactively by SIGUSR2 via admin_signal_execute = ["server backup-database"] — verified empirically (docker kill -s SIGUSR2 → Created database backup #1 … in 47 files, no admin room, no human). 🔴 Two findings that would have made a naive backup UNRESTORABLE: (1) backup-database emits a RocksDB BackupEngine store (meta/, private/, shared_checksum/), NOT an openable database, and there is no restore admin command — the script materialises it back into a plain db/ so recovery is tar xzf + cp with no tooling; (2) the native backup covers only the database — without media/ and archive/ the server refuses to boot with “Critical error starting server: Failed to verify media integrity.” Both now included. ✅ Restore was actually rehearsed: extracted → throwaway container → /_matrix/client/versions 200 → users list-users returned all five accounts. 🎣 The “76 MB” data dir is a du artifact — real size 1.4 MB; du counts RocksDB’s preallocated WAL and MANIFEST. Archives ~222 KB.
• 🔔 ⭐⭐ All homelab alerts rerouted Telegram → Matrix. Room telep-ertesitesek (#telep-ertesitesek:chat.taild4189d.ts.net / !gCuOI7uLN2JNqp5XR2voIsisfO0CXxBECMBh7CqynqQ), encrypted (m.megolm.v1.aes-sha2); @alerts/@andras/@phone joined, @mfalusi/@puliki invited only. Relay /opt/matrix-relay/matrix-relay + matrix-relay.service, matrix-nio 0.26.0 / vodozemac 0.10.0 (nio 0.26 dropped libolm for the Rust vodozemac backend — installed libolm3 is present but unused), key store /opt/matrix-relay/store (0700), config /etc/matrix-relay.env (0600). 🔌 Binds 127.0.0.2:8118 AND 172.18.0.1:8118 — the second bind is required because frigate-notify sits on the nvr_default docker bridge and cannot reach host loopback; verified NOT tailnet-exposed (node’s tailnet IP refuses 8118, no tailscale serve mount references it, 172.18.0.0/16 not among the advertised routes). 🐞 Two real bugs found and fixed: (1) undecryptable messages — nio only re-shares a megolm session when it EXPIRES, so a member joining later never received keys; fixed by refreshing device lists and rotating the session when the member-device set changes (ignore_unverified_devices=True is the correct API for 0.26); (2) crash loop on restart — with a persisted sync token the incremental sync returns no rooms, so the join check failed forever; fixed by forcing a full sync at startup. Final routing: bambuddy provider id=2 webhook enabled (15 events, mirroring id=1) with provider id=1 Telegram DISABLED but NOT deleted (rollback path); frigate-notify webhook enabled, telegram disabled; intruder-alarm.py untouched and still inactive. 🚩 frigate-notify’s native matrix backend was deliberately NOT used — that container has no persistent volume beyond config.yml, so its crypto store would be wiped on every restart ⇒ recurring “unable to decrypt”; one E2EE identity with one persistent store is the safer design. 🐛 Link-rewriting bug worth recording: payload URLs use the container-internal host http://frigate:5000 which the relay cannot resolve — they must be rebuilt against the public tailnet URL or the room fills with useless links.
• 🖨️ AMS humidity alert noise KILLED — and the insight generalises. AMS humidity alerts were 32 of the last 40 notifications, hourly forever: PC’s threshold was 30 while the AMS runs at 37–42%, and the AMS physically cannot dry to 30% (65 °C ceiling, PC needs 80 °C) — a correct and unactionable alert. 🔑 Per-material humidity thresholds do NOT work on a single-chamber AMS: there is ONE sensor for all four slots, so whichever material has the lowest threshold alerts permanently regardless of what is loaded. The whole per-material map (set in 2026-09-03-bambuddy-preheat-chamber-target-bug-and-clog-rca) was replaced with {"default": 50} = “the AMS is wetter than it normally runs”, which is actionable (exhausted desiccant, wet spool, humid weather). The PC-drying knowledge moved to where it belongs: the PC / PC-FR Dryness Check maintenance task (14 days) and the PC-FR pipeline description.
• 🔌 Tapo P115 (192.168.1.167) wedged at IP level with a perfect radio link — associated at −39 dBm / SNR 64 with a valid DHCP lease, but ping 100% loss, ARP STALE, port 80 closed. Classic wedged IoT network stack; needs a physical power-cycle. ✅ The bridge handled it correctly — fast 503s with backoff, NRestarts=0, no hang — and nothing depends on it (auto_on/auto_off both false).
• 📷 Two camera/alert findings. frigate-notify excludes telep_cam3 and telep_cam4 via cameras.exclude — a deliberate, user-confirmed setting (see 2026-07-28-frigate-notify-camera-exclude); do NOT “fix” it. Consequence, now documented: a genuine person was detected on cam3 at 02:27 and 02:37 (79% confidence, verified by viewing the snapshot — a real person by the railing at night) and produced no alert. ⚠️ A synthetic test alert (“Telep Cam9”, event id 9999999999.000000-nope) was posted into the LIVE alert room by a subagent testing the snapshot-failure path and confused the user into thinking a real detection had failed — lesson: test alerts belong in a scratch room, or must be redacted. Real detections DO carry images: 4 of 5 frigate posts in the sample had image=True (85,900 and 77,374 byte JPEGs); the only image-less one was the synthetic test.
• 📚 Homepage markdown-renderer research — nothing built, and the content to render was never specified. ✅ The existing renderer is the knowledgebase (2026-07-24-knowledgebase): Astro 5.14 + Starlight 0.36, built with Bun, project /home/levander/kb-astro/, canonical builder build_content.py — NOT convert.py, which is partial and produces a STALE site (independently re-confirmed by a second research pass); deploy kb_build.sh → atomic swap into ~/knowledgebase/site/ (1.6 GB built); served by knowledgebase.service (Flask + waitress) on 127.0.0.2:8092 → https://knowledgebase.taild4189d.ts.net. ⚠️ The knowledgebase is NOT containerised — plain systemd; the only related container is kb-qdrant, previously unrecorded. ⚠️ The sidecar unit/socket/statedir are named kb while the hostname is knowledgebase. History: mkdocs-material → Astro (Aug 2026), done specifically to preserve byte-identical directory URLs. ✅ CONFLICT RESOLVED: home.taild4189d.ts.net fronts gethomepage (→ 127.0.0.2:3010), not the old static home-portal — older vault notes claiming otherwise are STALE; homepage also answers on the LAN at http://home.telep.lan via Caddy on :80. 🟡 But the old static portal is STILL RUNNING and orphaned — python3 on 127.0.0.1:8093, pid 2716, serving a 3.8 KB August index.html nothing proxies to; worth killing, and note it sits on 127.0.0.1, the address the hardening sweep moved services off. 🚩 gethomepage cannot host arbitrary pages (fixed dashboard) — the proven mechanism for arbitrary UI is the iframe service widget (as used for the TV control buttons). 🚩 Three gethomepage gotchas: docker restart homepage does NOT apply config changes (use curl http://127.0.0.2:3010/api/revalidate); quote every description: — an unquoted colon-space blanks the whole dashboard; services.yaml has concurrent writers ⇒ surgical line insertion only, never a YAML round-trip.

2026-09-05

• 🗂️ ⭐⭐ Session handover written for the multi-day 2026-09-02 → 2026-09-05 run (2026-09-05-session-handover) — successor to 2026-09-02-session-handover. Covers, with cross-links rather than duplication: LG TV network control + presence (2026-09-02-lg-tv-network-control-presence, 2026-09-02-tv-presence-wifi-union-daemon), the bambuddy preheat chamber-target bug and clog RCA (2026-09-03-bambuddy-preheat-chamber-target-bug-and-clog-rca), the Starlink WAN migration and dish telemetry (2026-09-04-starlink-wan-migration-dish-telemetry), the reMarkable evaluation (2026-09-03-remarkable-paper-pure-claude-integration) and Frigate resource tuning (2026-09-02-frigate-resource-tuning). 🔴 Three headline open items: (1) the TV is physically powered off and unreachable (WiFi radio off, no ARP, no association, tv on times out on WoL) — needs a physical button press, and Auto Power Off has STILL never been disabled (third time the TV has gone dark); it cannot be set over the network because the settings surface 404s under aiowebostv’s permission manifest and a wider manifest means re-pairing, which means a trip to the TV anyway. (2) The PC-FR Starlink mount is printed but NOT installed while fraction_obstructed climbed 0.12% → 3.3% and the prolonged-outage estimate fell 6 h → 45 min; starlink.csv holds the before-baseline. (3) The north camera 192.168.30.139 is marginal from MULTIPATH, not congestion — −63 dBm / SNR 40 but only 8.6–11 Mbit/s, versus south at 114.7 Mbit/s / −47 dBm / SNR 57; channel tuning cannot fix it (needs a closer wired AP, a directional antenna, or repositioning — more TX power and wider channels make multipath WORSE).
• 📶 ⭐ 2.4 GHz interference hunt → radio1 moved ch11 → ch1; channel busy fell 92.9% → 25.8% (backup /etc/config/wireless.bak-1788552343). Camwall flashing was a watchdog re-roll loop driven by stalled camera streams. 🔎 The survey was decisive and points at a device the user OWNS, not neighbours: ch11 93% busy while our own traffic was only ~20%, and the noise floor across ch8–13 sat at −60…−76 dBm versus −93…−98 dBm on ch1–7, with only 1–3 APs in the whole scan — few APs + a raised noise floor confined to the upper band = a non-WiFi emitter. ch1 chosen as non-overlapping with the HP printer’s WiFi-Direct on ch6 at −20 dBm. South camera recovered instantly; north needed ~/tapo-ctl/reboot-cams-onvif.py 192.168.30.139, after which all four cams returned to ~5 fps and recording resumed. 🆕 New logger /home/levander/net-monitor/wifi_survey.py → wifi_survey.csv, cron daily 04:00 (a scan briefly interrupts clients, hence nightly); it only measures — never changes a wireless setting, and runs from the mainframe over SSH to the router at 192.168.1.1 (the LAN address — the tailnet ACL blocks port 22 mainframe→router). ⚠️ Two caveats for anyone reading the CSV: the in-use row’s busy_pct is a LIFETIME average over monotonic counters so it barely moves day-to-day (real congestion needs deltas between consecutive rows, handling active_time_ms decreasing on interface restart), and scanned rows are a single ~150 ms sample and very noisy (ch2 read 98.67% then 78.0% nineteen seconds apart) — active_time_ms is in the CSV precisely to tell high- from low-confidence rows. 🚩 Parser trap: a naive Channel:\s+(\d+) regex on iwinfo scan also matches Primary Channel: inside HT/VHT blocks and double-counts every AP — anchor on ^Mode: and key by frequency. ⚠️ Not yet in its own note — the handover is currently the only record.
• 🔌 Verified 127.0.0.2 port map recorded (a mid-session port-collision report was WRONG and is corrected): tv-http = 8102 (tailnet :8451), tapo-bridge = 8117, bambuddy = 8000 (UNAUTHENTICATED, users table EMPTY), bambuddy-mcp-bridge = 8091, bambu-studio-api = 3001, homepage = 3010 (tailnet :8450), nextcloud 11000 / tsauth-proxy 11001 / onlyoffice 11002. No conflict. All four custom services confirmed active: tv-http, tapo-bridge, print-guard, tv-presence. 🔴 Also open: the bambuddy API key leaked into a chat transcript and must be rotated (it is bambuddy-mcp-bridge’s credential). 🚩 Reusable gotchas consolidated in the handover: NEVER /etc/init.d/network reload|restart on telep-router (drops the site AND the tailnet SSH managing it — use live-then-persist); busybox lacks timeout, and nc -z / ping -M LIE — this produced FOUR false diagnoses in one session, test from the mainframe; Frigate stores recordings in UTC (local 07:23 lives in the 05/ hour dir); xrandr HDMI-1 ≠ the TV’s HDMI 1; grpcurl needs -emit-defaults; ps -eo pcpu is a lifetime average, not instantaneous CPU.
• 🧊 ⭐⭐ FreeCAD MCP outage root-caused to TWO stacked failures, and it CORRECTS the 2026-09-02 modal-dialog note (2026-09-05-freecad-mcp-502-dead-container-and-qt-event-loop-wedge). https://cad.taild4189d.ts.net:8443/mcp 502’d on both POST /mcp and GET / and Claude Code reported the freecad MCP server failed at startup. (1) 🔴 NEW FAILURE MODE — restart: unless-stopped does NOT guarantee a container comes back. serve status showed :8443 → http://127.0.0.2:9876 (⚠️ .2, not the 127.0.0.1 in 2026-08-26-freecad-cad-workstation) with nothing listening on 9876; docker ps -a (the -a is the whole gap) showed freecad-mcp Exited (128) for 3 days. docker inspect: Error="failed to create task for container: failed to create shim task: ttrpc: closed", OOMKilled=false, FinishedAt=2026-09-02T03:27:08Z, RestartCount=0 — the policy never fired, because the containerd shim died before the task was ever created, so there was no task for Docker to restart. Fix = docker start freecad-mcp. Reusable: whenever a tailnet service 502s, run docker ps -a early. (2) 🔴 CORRECTION — dismissing the modal dialog is NOT sufficient. With the proxy back, get_rpc_status reported {"rpc_server":"running","gui_dispatch":{"state":"healthy","task_id":0}} while every execute_code hit GUI dispatch timed out after 90s; direct XML-RPC to 127.0.0.1:9875 isolated it away from the proxy (ping()/list_documents() 0.0 s, execute_code("print(42)") 90 s timeout) ⇒ RPC thread fine, only GUI dispatch blocked. xwininfo -root -tree found the documented 0x400075 "Document Recovery" still standing since the Sep 2 crash — ⚠️ DISPLAY is :1, NOT :0 (a :0 socket exists but xwininfo can’t open it; the old note is wrong and is now corrected in place). 🔴 xdotool windowclose 4194421 DID remove the window (verified gone from the tree) and execute_code STILL timed out at 90 s — no dialog present, gui_dispatch still self-reporting healthy. After ~3 days behind that modal the Qt event loop stayed dead; killing the window freed nothing. ✅ What worked: MOVE the recovery snapshots aside, THEN restart — as abc, mv /config/.cache/FreeCAD/v1-1/Cache/FreeCAD_Doc_*/ (10 dirs, Aug 31 – Sep 2) → /config/.cache/FreeCAD/recovery-stash-2026-09-05/, then docker restart freecad → RPC back in ~10 s, execute_code 0.0 s. The old “do NOT restart” rule is right only because a bare restart re-raises the dialog while recovery files are present and re-wedges you; the stash is the load-bearing step, the restart just collects it. ⚠️ MOVE, never delete — the user’s only copy of unsaved work; list_documents() returning [] is the safety check that made the restart safe. 🟡 OPEN: 10 stashed recovery dirs sit at /config/.cache/FreeCAD/recovery-stash-2026-09-05/ awaiting the user’s restore-or-delete decision. ✅ Also corrected: cad is not Tailscale-SSH-enabled (only telep-mainframe/telep-router are) but that does NOT mean a person on site — every per-service tailscaled sidecar node lives on the one bare-metal host, so ssh 100.120.203.1 lands on telep-mainframe; the entire recovery was done remotely. 🚩 Minor trap: command -v a b c under the container’s dash only reports the FIRST argument, which made xdotool look absent when it was installed all along — use a loop.

2026-09-04

• 🛰️ ⭐⭐ The site’s WAN moved off the Telekom NE200 5G FWA CPE onto Starlink in Bypass mode — this CLOSES OUT the chronic episodic WAN degradation investigation. The NE200’s RF logger wrote its final entry at 2026-09-04T02:15 still degraded (RSRP -103 dBm, SINR 9, downlink QPSK) — exactly the fault that was diagnosed. After: WAN 100.87.12.221/10 gw 100.64.0.1, 18.3–35.3 ms at 0% loss, 235–279 Mbps wired (Cloudflare + Cachefly 50 MB), Cloudflare in 5 hops; the NE200 at 192.168.254.1 is 100% unreachable. ✅ Clean-bypass proof = one ARP comparison: 100.64.0.1 and the dish 192.168.100.1 share the same MAC 26:12:ac:1a:80:01 — the OpenWrt WAN port talks straight to the dish with no Starlink router routing in between. ⚠️ Still CGNAT (100.64.0.0/10) → no inbound ports, same as the old double-NAT, which is why the tailcat plan for the offline company Mac remains right. 🔴 NEW COLLISION: Starlink CGNAT and Tailscale BOTH use 100.64.0.0/10 and neither is configurable — appeared the instant the WAN swapped (the NE200 gave non-overlapping 192.168.254.2/24). Tailscale survives via policy routing: ip rule prio 5270 → table 52 (per-peer /32s) consulted before main; tailscale status --json shows "Health": [], UDP: true, MappingVariesByDestIP: false — Starlink’s CGNAT is well-behaved for NAT traversal. 🚩 Trap: ip route get for 100.84.98.18 (bambuddy) / 100.120.203.1 (cad) resolving to dev wan is NOT collision damage — those peers are absent from table 52 because the ACL doesn’t grant telep-router (tag:telep) access to those tagged-devices; unreachable regardless of Starlink. The only real consequence is that packets to ACL-denied peers leak toward the Starlink link instead of failing closed (link-scope /10) — cosmetic, not an outage. 🚩 ping is useless as a tailnet test here — 100.115.209.87 pings unreachable while routing fine over tailscale0 (ICMP filtered). ❌ Do NOT renumber the LAN — 192.168.1.0/24 collides with nothing; the overlap is the /10 and no local subnet change can fix it.
• 📡 Dish access + telemetry in Bypass mode (2026-09-04-starlink-wan-migration-dish-telemetry): the dish at 192.168.100.1 is not routed by default, so telep-router got a live ip route add 192.168.100.1/32 dev wan plus TWO independent boot mechanisms — a uci config route stanza in /etc/config/network (never verified, because the network was deliberately never reloaded) and /etc/hotplug.d/iface/99-starlink-dish (0755, ip route replace, idempotent) which WAS verified for real (route deleted → script run → route back; lan ifup and wan ifdown both correctly no-op). 🔴 /etc/init.d/network reload|restart was DELIBERATELY never run — it would drop the whole site AND the tailnet SSH used to manage the router; the live-then-persist pattern (apply with ip route, write uci without reloading) is the reusable safe approach. Any LAN client reaches the dish via wan-zone masquerade (firewall.@zone[1].masq='1') + default lan→wan forward; ports 80, 8080, 9200, 9201 verified open. ❌ The Starlink mobile app does NOT work in Bypass (it talks to the router, whose WiFi/services are off) — symptom was “unreachable” for the dish and “Not on account…” for a spare unregistered router; ✅ use http://192.168.100.1 from telep1, fully self-contained (serves its own script.js.gz, 172166 bytes). 🚩 Two red herrings: webapp.starlink.com is NXDOMAIN but is only a permissive CSP entry, and grpcurl fails on :9201 because that port is gRPC-WEB (a raw POST returns 200 with proper gRPC headers — healthy). 🆕 Telemetry logger live: /home/levander/net-monitor/starlink_status.py → starlink.csv, cron */5 as levander, mirroring the retired NE200 logger’s conventions (HDR constant, header only if CSV absent, %Y-%m-%dT%H:%M:%S%z, blanks for missing, never crashes a row); client is grpcurl v1.9.4 static in /usr/local/bin (chosen over Python grpc because Debian 13 enforces PEP 668 and reflection means no .protos), calling SpaceX.API.Device.Device/Handle {"get_status":{}} on :9200. 🔴🔴 THE highest-value gotcha: -emit-defaults is MANDATORY — grpcurl omits proto3 zero-values, silently dropping popPingDropRate, fractionObstructed, currentlyObstructed, timeObstructed exactly when they’re zero, i.e. when everything is healthy; a logger without it looks fine and records nothing useful. Firmware 2026.08.26.mr85524, hw rev4_pez_proto1: no state enum (deviceState carries only uptimeS → the CSV’s state column is derived: outage.cause → non-OKAY disablementCode → CONNECTED → NOT_READY) and no snr (only isSnrAboveNoiseFloor, so signalQuality is logged). 🚩 readyStates has a legacy cady key permanently false on rev4, so all(readyStates.values()) wrongly says NOT_READY on a healthy dish → fixed with an explicit core set ("scp","l1l2","xphy","aap","rf"). ⚠️ alerts reads noEthernetLink on EVERY row while ethSpeedMbps is 1000 — a Bypass firmware artifact, so the column is never empty and is useless as a naive alert trigger; avg_prolonged_obstruction_interval_s can be the literal string NaN; .err gets success rows too and at */5 grows ~4× faster with no rotation; grpcurl is pinned, root-owned, package-manager-untracked. ⚠️ UNCONFIRMED early signal: first hour fraction_obstructed 0.0012 → 0.0061 → 0.0208 → 0.0222 (2.2%) and avgProlongedObstructionIntervalS 21600 s → 2700 s — obstruction stats need ~12 h to settle and uptime was <1 h, but the direction was worsening, not settling; the obstruction map at http://192.168.100.1 shows the blocked sector and the CSV is now the before/after evidence for a reposition. 🗄️ NE200 logger retired: the */15 ne200_signal.py cron line removed (backup /root/levander.crontab.bak.1788486106), but the script and its 342 rows of rflog.csv were deliberately PRESERVED — that data is the evidence base that justified the switch; rflog.err had grown to ~30 KB of failures against a modem that no longer exists.
• 📶 WiFi/coverage findings + the “Starlink router as AP” verdict (2026-09-04-starlink-wan-migration-dish-telemetry): radios are radio0 5 GHz ch36 HE40 @ 573.5 Mbit/s PHY and radio1 2.4 GHz ch11 HE20; SSIDs telep1 (5G/lan), telep1-2G (2.4G/lan), telep-cc (2.4G, cams, isolate='1'). ❌ Widening to HE80 would NOT improve internet speed — the line is ~250 Mbps and HE40 already gives 573 Mbps PHY, so it only helps LAN transfers; recorded so nobody “optimises” it for no gain. 🔴 The TL-WA850RE extender (192.168.1.101) caps at ~50 Mbps ≈ 20% of the line — ⚠️ CORRECTED 2026-09-05: it is NOT a repeater. It was already converted to a wired AP on br-lan (SSID bandi); the ceiling is its 100 Mb uplink port + single-band N300 radio + 2.4 GHz congestion, not repeater halving. The original claim was inferred from the model name, never measured. It also answers on a second factory IP 192.168.0.254 (same MAC), explaining a mystery ARP entry. 🔴 VERDICT: a Starlink router can NEVER be an access point — it has exactly two states, Bypass (WiFi off, pure L2 bridge) and not-Bypass (it IS the main router, own NAT+DHCP); there is no AP/bridge mode that joins a third-party network, Starlink mesh nodes have no Ethernet backhaul and pair only to a Starlink main router (incompatible with Bypass by definition), and reversing Bypass requires a factory reset. ⚠️ Going non-Bypass would be actively HARMFUL: it would displace telep-router and take out br-cams (the 192.168.30.0/24 camera VLAN + telep-cc), dnsmasq leases/.lan names, and critically the iwinfo assoclist on phy0-ap0/phy1-ap0 — both the intruder alarm and the tv-presence daemon compute presence as the union of arp-scan + that assoclist, and it was proven live that a sleeping iPhone (38:7f:8b:df:2a:79) is ABSENT from arp-scan but PRESENT in the assoclist; losing it drops both to arp-scan-only — the exact configuration that switches the TV off with someone in front of it and arms the alarm with people home. Also, an unmanaged Starlink router upstream puts its WiFi clients outside the firewall, DNS, VLANs and both presence systems. 🔎 Incidental: /home/levander/tapo-ctl already has a venv with python-kasa 0.10.2 (plus a nightly 04:30 set-cam-time.py) — the same version the Tapo P115 bridge installed into /opt/tapo-bridge/venv; duplicate venvs, and TP-Link credentials likely already exist on the box, which may make populating /etc/tapo-bridge.env easy.

2026-09-03

• 🚨 ⭐⭐ Root-caused a 6-minute-per-print delay AND a destroyed nozzle assembly to ONE bambuddy bug: [[2026-09-03-bambuddy-preheat-chamber-target-bug-and-clog-rca|_derive_chamber_target maxes over every loaded AMS tray and never consults the print’s ams_mapping]] — one PC spool parked in AMS slot 1 forced a 50 °C chamber target onto every PLA print, and with no bed_temperature in the archive metadata the bed fell back to queue_keep_warm_bed_temp = 90 °C (guidance says 35–45 °C). Cost: 57 s ramp + 300 s soak ≈ 6 min on a 3 g keychain — the FTP upload everyone blames is 0.4 s. 🚩 Three wrong trails recorded: awaiting_plate_clear=True is a context dump, not a diagnosis (require_plate_clear was false, the gate couldn’t fire), state=FINISH is idle, and there were no stale printing queue rows. 🔧 Clog RCA: cold end = PLA heat-creep from the forced 50/90 °C; hot end = PC residue carbonising across relentless PC↔PLA alternation (PC needs 260–280 °C, PLA runs ~220 °C). ✅ Fix = invert the default (PC: 0, PC-FR: 0; a per-item preheat_chamber_target_override beats the map) — rejected pulling the spool, the AMS is the driest storage available (41 % RH). ⚠️ ABS/ASA are still 45 — same trap returns. ⚠️ bambuddy has NO material-transition logic: PLA→PC safe, PC→PC best, PC→PLA dangerous — use manual_start: true as the purge window. Also: 2 new maintenance types (Cold End Inspection 100 h, PC-FR Dryness 14 d), local backups enabled (were OFF with 61 prints unbacked), and a built-but-unstarted, UNTESTED Tapo P115 → rest bridge at 127.0.0.2:8117 (🔴 consumption_total is None for ALL Tapo devices in python-kasa — lifetime energy is synthesized from state.json)
• 🖊️ Evaluated live-integrating a reMarkable Paper Pure with Claude Code — RESEARCH ONLY, the device is NOT purchased. Verdict: easy, a mature MCP server already exists (remarkable-mcp, SamMorrowDrums, MIT, uvx remarkable-mcp, built on rmscene+PyMuPDF), and Paper Pure is supported (reMarkable groups Paper Pro / Pro Move / Pure together throughout). 🔴 THE operational landmine: enabling Developer Mode FACTORY-RESETS the tablet — unlike rM1/rM2, Paper Pro/Pro Move/Pure all need it before SSH works, so the SSH decision must be made BEFORE anything is put on the device and dev mode enabled on day one, out of the box; credentials then appear under Settings → General → Help → About → Copyrights and Licenses → “GPLv3 Compliance”. Four transports (local dir / USB web / cloud / SSH) compared in a table: only cloud needs a paid Connect subscription, only SSH exposes remarkable_author — the one tool that writes native ink (draw/add_page/create_document, byte-identical to the canvas Save button); everything else can only upload PDFs that look like imports. ⚠️ None of the modes are push — every transport is pull, no webhook, no watch; “live” means “current when queried”. OCR is the gate on handwriting being readable at all: Google Vision (good, API key, not offline) vs Tesseract (offline, poor at handwriting); typed text + PDF annotations extract natively. Self-hosted alternative rmfakecloud (ddvk) supports Paper Pure and kills the subscription, pairs with the existing Nextcloud Drive stack over WebDAV — but file sync is only tested to firmware 3.27.1, so a tablet auto-update is a real breakage risk, and v0.0.25 needs eu.tectonic.remarkable.com added to the tablet’s /etc/hosts. 🔒 Unresolved conflict with house convention: remarkable-mcp --http has no auth, rejects wildcard binds and documents 127.0.0.1 — but 2026-08-31-tailnet-plaintext-port-hardening requires 127.0.0.2; verify the bind check before deploying, and expect the same 421 trap as 2026-08-31-telep-kb-mcp-server. Recommendation: USB web for a 10-minute zero-risk proof, SSH for real capability (dev mode day one), rmfakecloud on telep-mainframe for the homelab-native path. Note ships an ordered “when the device arrives” checklist led by the irreversible dev-mode decision.

2026-09-02

• 📺 LG TV network control + presence automation on telep-mainframe — the TV showing the camwall can now (in principle) be powered on/off and switched to HDMI1 from the LAN. 🔴 Key finding: HDMI-CEC is IMPOSSIBLE on this box — /dev/cec* and /sys/class/cec/ do not exist and never will, because NVIDIA consumer GPUs do not implement HDMI-CEC (verified on driver 610.57.04); no driver option or kernel module adds it, so do not burn time on cec-client over the existing cable. Workarounds are a Pulse-Eight USB-CEC adapter (~EUR40, must be wired INLINE GPU→adapter-in→adapter-out→TV, else “set active source” picks the wrong port), an IR blaster, or the vendor network API — the network API was chosen. 🔍 Identifying the unknown display: xrandr LIES under the NVIDIA blob — it reported HDMI-1 as 1600mm x 900mm (implying 72–75”) when the panel is 43”; the proprietary driver never populates /sys/class/drm/*/edid and 1600x900mm is a round 16:9 placeholder. What worked was avahi-browse -rt _airplay._tcp, whose TXT record gave model+firmware+MAC in one shot (model=43UP75003LF, fv=p20.03.53.45, serialNumber=209MAAKHWU24_ac:5a:f0:8b:48:da) — mDNS TXT is the highest-yield way to name a smart TV on the LAN; SSDP M-SEARCH returned nothing and port scanning found ports but no identity. ⚠️ Red herring: telep-tv in mDNS is NOT the TV — it’s the mainframe’s own uxplay receiver (2026-08-04-telep-mainframe-airplay-receiver-uxplay). Device: LG 43UP75003LF, webOS 6, 192.168.1.171 / LGwebOSTV.lan / ac:5a:f0:8b:48:da, SSAP WebSocket 3000 plain (HTTP 101) / 3001 TLS (self-signed CN=LGE TV SSG → SSL verify must be OFF); also AirPlay 7000, Miracast 7250, HomeKit _hap._tcp. Built: /opt/tv-control/tv Python CLI (venv, aiowebostv 0.10.0, pair|status|on|off|hdmi1|input, key at /opt/tv-control/client-key.json 0600, hand-rolled WoL, venv mandatory because Debian 13 enforces PEP 668) + /opt/tv-control/presence & tv-presence.service (arp-scan on enp5s0 every 60 s, 900 s absence debounce, absent→present = on+hdmi1, present→absent = off; tracked 38:7f:8b:df:2a:79, a4:40:e1:02:01:e9) + OliveTin buttons TV be/TV ki/TV → HDMI1 + a TV vezérlés Homepage tile. Design: acts on TRANSITIONS ONLY and never actuates at startup (a restart can’t power-cycle the TV, and a human who kills the TV manually isn’t fought every cycle); whole-subnet arp-scan not fixed IPs (DHCP-proof); active probing because phones sleep their radio and drop out of the passive ARP cache. 🐛 aiowebostv 0.10.0 gotchas: no __version__ (use importlib.metadata.version), and connect() always calls _check_registration() → an unpaired call throws a pairing prompt ON SCREEN, so status does a bare TCP probe first — anything that merely checks state must not connect() while unpaired. 🔴 OliveTin 3000.19.0 removed /api/StartActionByGet/<id> — the API is Connect-RPC at /api/olivetin.api.v1.OliveTinApiService/<Method> and StartActionByGet is POST-only (GET → 405 Allow: POST, not no_side_effects); worse, action IDs are now bindingId UUIDs minted at config-load, so hardcoded trigger URLs rot on restart → the “one-click dashboard tile fires an OliveTin action” pattern is dead on 3000.x (a internal/webhooks/execOnWebhooks subsystem exists but its YAML schema is unconfirmed; tile just links to the panel for now). 🚧 NOT DONE — nothing has ever run against the real TV: (1) pairing has never happened, client-key.json absent, needs tv pair with a human at the set → buttons + daemon are inert; (2) WoL may be disabled on the TV (Settings → General → Mobile TV On → Turn on via Ethernet/Wi-Fi) — packet verified byte-for-byte but never transmitted; (3) Auto Power Off must be turned off or webOS kills the camwall every 4 hours (All Settings → General → Timers → Auto Power Off; possibly scriptable via luna://com.webos.settingsservice/setSystemSettings, which would need bscpylgtv not aiowebostv, unconfirmed); (4) unknown whether this 2021 UP75 reports power_state at all, so status may print power: unknown. Backups .bak-20260902-124934 for both YAMLs. — 2026-09-02-lg-tv-network-control-presence

• 📉🎥 Frigate resource tuning on telep-mainframe — birdseye was encoding a 3840×1080 canvas 24/7, and face_recognition had been burning 678 MiB VRAM since July with ZERO faces ever enrolled. 🔴 Read the measurement trap first: the alarming “Frigate at 149.81% CPU” came from ps -eo pcpu, which is a LIFETIME AVERAGE since process start, not instantaneous — every process was ~12 min old on a fresh boot so post-reboot startup churn was permanently baked in (load avg visibly settling 15.93 → 6.04 → 3.86). Use docker stats --no-stream sampled repeatedly or top -bn2 (second iteration), never a single ps %CPU; and 149.81% on a 24-core box is only ~6.2% of capacity — not pathological. Fix 1: birdseye.mode: continuous → objects + width: 3840 → 1920 — continuous composites and encodes even with zero viewers, spawning a permanent ffmpeg -f rawvideo -video_size 3840x1080 … -codec:v mpeg1video doing SOFTWARE mpeg1 encode of a 4.1 MP canvas (largest single ffmpeg CPU consumer in the stack); verified the 3840x1080 encoder is gone, replaced by 1920x1080 (half the pixels). ⚠️ GOTCHA / OPEN ITEM: with restream: true the birdseye encoder keeps running even under mode: objects because go2rtc holds a consumer attached — to make it truly idle also set restream: false (not done; check consumers first — the TV wall moved off birdseye in 2026-07-28-camwall-4-substream-composite). Fix 2: face_recognition.enabled: false — proven dead weight: /media/frigate/clips/faces/ EMPTY (created Jul 15, never populated), GET /api/faces → {}, 0 of the last 500 events had a sub_label. ⚠️ this did NOT remove frigate.embeddings_manager — in 0.17 that process also serves semantic search + LPR, so it halved 678 → 324 MiB rather than disappearing. Measured: GPU VRAM 2148 → 1804 MiB (−344), container RAM 3.887 → 3.164 GiB (−723). 🔴 CPU is INCONCLUSIVE — do NOT claim a CPU win: post-change samples were bursty (37.94 / 30.40 / 131.08%) and the 149.81% “before” was a post-boot artifact, so there is no fair baseline. Already fine, don’t touch: detect 1280×720 @5fps preset-nvidia + scale_cuda ~2.7% CPU/cam, detector onnx/yolo-generic 640×640 ~11–19% CPU + 394 MiB VRAM, record ffmpegs -c:v copy ~1.5% each. Remaining levers: (1) /tmp/cache is an ANONYMOUS DOCKER VOLUME, not tmpfs — 10 s segments from 4 cams churn /var/lib/docker on disk; fix with tmpfs: - /tmp/cache:size=1g in compose; (2) record.motion.days: 10 retains ALL motion segments on top of 14-day alerts/detections — /srv/frigate/recordings at 496 GB (disk 735 G/3.6 T, 22%) ⚠️ but that setting was added deliberately in 2026-08-22-frigate-recording-retention-config-not-jam, don’t just delete it. Config /home/levander/nvr/frigate/config.yml (mounted /config), backup config.yml.bak-<timestamp>; YAML validated inside the container, docker restart frigate → healthy in 20 s. ⚠️ docker logs frigate REPLAYS FULL HISTORY — errors dated 2026-07-27 surfaced and were NOT from this restart; always check timestamps / use --since. — 2026-09-02-frigate-resource-tuning

• 🧩 FreeCAD MCP, part 2 — execute_code times out while get_rpc_status says healthy. After docker restart freecad the RPC came up correctly (ss -ltn → 0.0.0.0:9875, status {"rpc_server":"running","gui_dispatch":{"state":"healthy"}}) but every execute_code failed; the MCP layer showed only “The operation timed out”, while the raw XML-RPC call gave the real error {'success': False, 'error': 'GUI dispatch timed out after 90s'}. 🔴 Key insight: get_rpc_status deliberately does NOT use the GUI thread, so healthy is no proof code can run (nor are ping()→True / list_documents()→[]) — execute_code is the only real probe. Root cause: FreeCAD had crashed earlier, so on relaunch it showed its Document Recovery modal dialog, which owns Qt’s main thread and starves the addon’s GUI-dispatch queue. Diagnose with docker exec -u abc -e DISPLAY=:0 freecad sh -c "xwininfo -root -tree" (listed 0x400075 "Document Recovery" next to 0x400062 "FreeCAD 1.1.3"); fix non-destructively with xdotool windowactivate <id> (Escape defers recovery, backups kept; window id changes each run; a BadWindow on the follow-up key call is harmless and confirms it’s gone). Decision rule: timeout + healthy ⇒ list X windows, do NOT restart the container. ⚠️ Also learned: you cannot launch FreeCAD via docker exec — /opt/freecad/AppRun line 13 hardcodes export QT_QPA_PLATFORM=xcb, so -e QT_QPA_PLATFORM=wayland (and XDG_RUNTIME_DIR/WAYLAND_DISPLAY variants) are silently overridden → “no Qt platform plugin could be initialized”; the only supported start is docker restart freecad, letting s6 (/defaults/autostart = AppRun via startwm_wayland.sh) supply the session env (RPC up in 4 s). Probe convention: curl http://127.0.0.1:9875/ returning HTTP 501 is the CORRECT healthy answer (XML-RPC is POST-only); accept-then-close = docker-proxy bound but nothing listening inside. Env facts: settings at /config/.FreeCAD/ and /config/.local/share/FreeCAD/freecad_mcp_settings.json (both remote_enabled:true, auto_start_rpc:true, allowed_ips incl. 172.16.0.0/12), network freecad_default = 172.22.0.0/16 (freecad .3, freecad-mcp .5), and remote_enabled:true is REQUIRED — false binds the container’s 127.0.0.1 while docker-proxy forwards to eth0, so the published port can never reach it. Exports go to /exports = host /home/levander/freecad/exports — 2026-09-02-freecad-mcp-gui-dispatch-timeout-modal-dialog

• 🔧 FreeCAD MCP dead — Failed to get RPC status: [Errno 111] Connection refused on both the direct https://cad.taild4189d.ts.net:8443/mcp server and the aperture mcp__aperture__freecad_* route (same backend). Errno 111 = Linux ECONNREFUSED (macOS would be 61) → fault is remote, not the Mac. 🔴 nc -z cad…9875 says OPEN and it is a lie: the userspace-networking raw forwarder holds the port and drops the connection when 127.0.0.1:9875 is empty (see 2026-08-31-tailnet-plaintext-port-hardening). Discriminating probe = speak a protocol: XML-RPC/http, XML-RPC/https and a raw TLS handshake all return RemoteDisconnected / SSLEOFError: UNEXPECTED_EOF_WHILE_READING (accept-then-close, zero bytes), while 8443 completes a real TLS 1.3 handshake → MCP server is up, only the FreeCAD RPC behind it is dead. cad (100.120.203.1) is online but not Tailscale-SSH-enabled (only telep-mainframe / telep-router are) → no remote fix. Fix = open FreeCAD on the host and hit “Start RPC Server” in the freecad-mcp workbench (RPC runs on the GUI thread → no GUI, no RPC) — 2026-09-02-freecad-mcp-rpc-refused-gui-not-running

• 📡 tv-presence daemon rebuilt on telep-mainframe — presence is now the UNION of the router WiFi association table and arp-scan, and the arp-only version was deleted. /opt/tv-control/presence (stdlib-only python3) + /etc/systemd/system/tv-presence.service, enabled and running: 60 s poll, 900 s presence window, drives /opt/tv-control/tv on|off|hdmi1 on transitions only. 🔴 Core lesson, confirmed live today: arp-scan alone is provably broken here — iPhone 38:7f:8b:df:2a:79 (spider-web) was PRESENT in iwinfo phy0-ap0 assoclist while ABSENT from arp-scan --interface=enp5s0 --localnet; a sleeping iOS device stays WiFi-associated but stops answering ARP. The daemon logged it oscillating seen=router (13:24:07) → seen=both (13:25:11) across consecutive polls, i.e. an arp-only design would have reported a false “everyone left” and powered the TV off with the owner in the room (same finding as 2026-07-17-intruder-alarm, now independently confirmed twice). ⚠️ AP gotcha: query ONLY phy0-ap0 (telep1) and phy1-ap0 (telep1-2G) — NOT phy1-ap1, whose ESSID is telep-cc, the camera VLAN, which leaks cameras into presence. ⚠️ Silent-empty gotcha: iwinfo <bad-ap> assoclist prints “No such wireless device” and exits 1, so the remote command is for ap in ...; do iwinfo $ap assoclist || exit 1; done — a renamed AP fails the source loudly instead of returning an empty set that reads as “nobody home”. ⚠️ Router host key rotated — dropbear at 192.168.1.1 tripped REMOTE HOST IDENTIFICATION HAS CHANGED from root; fixed with ssh-keygen -f /root/.ssh/known_hosts -R 192.168.1.1 + reconnect with -o StrictHostKeyChecking=accept-new; newly pinned ED25519 SHA256:5WXvfnjJYyOyZa3h/iVwdsYrOb17SLqVFfXV6CLXugY, router = telep-router, Linux 6.6.73 aarch64, key /home/levander/.ssh/router_alarm (shared with the intruder alarm). Poll-failure rule: a failed poll is DISCARDED entirely — no last-seen update, no evaluation, no aging out; one succeeding source is still a valid poll. Safety invariants: presence starts None and the first successful poll is adopted without actuation (a restart never power-cycles the TV); transitions only, never re-assert (a human switching the TV off manually is not fought back on); a failed tv on/tv off does not latch and is retried next cycle. 🚧 Still unvalidated: the LG TV is unpaired (/opt/tv-control/client-key.json absent) so no tv on/off/hdmi1 has ever run against real hardware. 🐍 Side lesson from the test harness: extension-less python files cannot be loaded with importlib.util.spec_from_file_location (returns a spec with loader=None) — use importlib.machinery.SourceFileLoader + spec_from_loader. → 2026-09-02-tv-presence-wifi-union-daemon, updates 2026-09-02-lg-tv-network-control-presence

• 📺✅ LG TV control COMPLETE — paired, verified end to end, and the headline gotcha was that the TV input number has nothing to do with the GPU output name. ⭐🔴 HDMI-1 in xrandr is the GPU’s port index; the mainframe is physically in the TELEVISION’s HDMI_2. The whole build assumed they matched, so switchInput HDMI_1 moved the set to an empty socket and it showed “no signal” — while xrandr, the framebuffer and the camwall were provably healthy (4 live panes, per-quadrant signalstats YAVG 115–128, correct 3840x2160 mode, grabbed with DISPLAY=:0 ffmpeg -f x11grab). A perfect false trail: both ends look fine and the screen is blank. The TV tells you the truth directly — WebOsTvState.inputs carries per-input connection state (com.webos.app.hdmi1 id=HDMI_1 connected=False / com.webos.app.hdmi2 id=HDMI_2 connected=True). 📏 Diagnostic rule: when an LG shows “no signal” but the source looks healthy, check connected per input BEFORE touching modes, cables or Deep Colour. Fix: single TV_INPUT = "HDMI_2" constant and the subcommand renamed hdmi1 → camwall across tv, tv-http and presence; the hdmi1 alias was deliberately REMOVED rather than repointed (a command named hdmi1 that switches to HDMI_2 is a trap) — tv hdmi1 now exits 2 unknown command. 🔌 Active Standby defeats a WoL guard: webOS has two standby depths and in Active Standby SSAP port 3000 stays OPEN, so if not await ssap_port_is_open(): send_magic_packet() skipped WoL entirely and fell back on SSAP power_on(), which LG sets honour unreliably — that was the user-reported “flaky”. Fix: send the magic packet unconditionally (harmless UDP broadcast if already on). ⏱️ webOS accepts TCP before SSAP is ready: after a real wake from deep standby, port 3000 opens seconds before the service answers — observed live, WoL woke the TV and then tv on, tv status and the input switch all failed with TimeoutError within ~9 s while the TV was booting fine. Commands reported failure while succeeding. Fix: CONNECT_ATTEMPTS = 6, CONNECT_RETRY_DELAY = 3, explicit connect_timeout=10 (it IS a supported WebOsClient ctor param). ⚠️ Honest nuance — don’t over-correct: CONNECT_TIMEOUT = 2 (aiowebostv default) is fine in steady state, measured 0/10 failures at 0.29–0.41 s over ten runs; it’s only short in the seconds after a wake. Do NOT “fix” steady-state timeouts from post-wake symptoms. 🤝 The aiowebostv pairing window is 10 SECONDS: 0.10.0 module globals in webos_client.py are CONNECT_TIMEOUT=2, RECEIVE_TIMEOUT=10, REQUEST_TIMEOUT=20, HEARTBEAT=5; _check_registration does ws.receive_json(timeout=RECEIVE_TIMEOUT) after the on-screen prompt, so a human has 10 s to grab the remote and press Accept. RECEIVE_TIMEOUT is not exposed via WebOsClient.__init__ (only connect_timeout is), so the only lever is monkeypatching the module global — which works because it’s read at call time: webos_client.RECEIVE_TIMEOUT = PAIR_RECEIVE_TIMEOUT; PAIR_RECEIVE_TIMEOUT = 180 is set in cmd_pair only, leaving normal commands responsive at 10 s. Layered-timeout rule: the outer timeout must EXCEED the inner one or it fires first and masks the real error — tv-http PAIR_TIMEOUT = 210 > tv PAIR_RECEIVE_TIMEOUT = 180. 🐍 The error-reporting bug that hid all of this: f"{command} failed: {err or type(err).__name__}" tests the truthiness of the exception object, which is always True, so the fallback never fires and it interpolates str(err) — empty for exceptions raised with no message. Every failure printed pair failed: with nothing after it for several rounds. Correct form: str(err) or type(err).__name__. ✅ Verified state: /opt/tv-control/tv (venv, aiowebostv 0.10.0) pair|status|on|off|camwall|input <id>, key /opt/tv-control/client-key.json 0600, paired 2026-09-02 16:13; tv status returns real data power: Active / on: True / input: HDMI_2 (⚠️ input: unknown while in standby is normal — current_app_id is empty, not a fault); /opt/tv-control/tv-http stdlib HTTP on 127.0.0.2:8102 (the 127.0.0.2 bind is the tailnet plaintext-forwarder invariant, 2026-08-31-tailnet-plaintext-port-hardening) published tailnet-only at https://telep-mainframe.taild4189d.ts.net:8451, POST /api/{on,off,camwall,pair}, GET → 405, embedded in Homepage via the gethomepage iframe service widget; tv-presence.service unchanged in design, now calls camwall. Wake-on-LAN is CONFIRMED ENABLED — a magic packet woke the set from deep standby with 3000 closed, so “Mobile TV On” is already on (remove any note saying that still needs doing). 🚧 STILL NOT DONE: Auto Power Off must be disabled at the TV (All Settings → General → Timers → Auto Power Off) or webOS kills the camwall after 4 idle hours. 🚧 The presence daemon’s absent→present path has still NEVER fired for real — verified by code inspection and by the identical subprocess mechanism working through tv-http, but no actual arrival has been observed. ❓ UNRESOLVED, record as disputed not fact: whether OliveTin 3000.19.0 supports a GET-triggered action route — one investigation tested StartActionByGet and got 405 Allow: POST, another found both StartActionByGet and StartActionByGetAndWait registered in the binary but did not test them. The TV buttons moved off OliveTin because the user wanted them on Homepage directly, not because of this. 🎛️ Two side gotchas found in passing: (1) docker restart homepage does NOT apply config changes — a prerendered /app/.next/server/pages/en.html survives restarts; curl http://127.0.0.2:3010/api/revalidate is what actually applies them (likely explains past “dashboard edits not taking”); (2) Homepage’s tailnet mount at :8450 was silently broken (502) — tailscale serve pointed at 127.0.0.1:3010 but next-server binds 127.0.0.2:3010; fixed in passing, same 127.0.0.1-vs-127.0.0.2 split as the port-hardening work. Also 2xl:h-48 in the Homepage widget config was a dead class Tailwind never compiled. — 2026-09-02-lg-tv-network-control-presence, 2026-09-02-tv-presence-wifi-union-daemon

2026-09-01

• 🖨️ Bambuddy full buildout on telep-mainframe (bambuddy maziggy/bambuddy, host-net, UI 127.0.0.2:8000, node bambuddy.taild4189d.ts.net): (1) server-side Slicer API sidecar bambu-studio-api (ghcr.io/maziggy/bambu-studio-api, bridge 127.0.0.2:3001:3000 — 127.0.0.2 because the ESP32 owns 127.0.0.1:3001; bambuddy reserves 3000/3002 for its virtual printer), wired via PATCH /settings/ use_slicer_api=true bambu_studio_api_url=http://127.0.0.2:3001; (2) 4 material pipelines (ids 1-4, POST /slicer-pipelines/) PLA/PETG/ABS/PC-FR on printer Bambu Lab H2S 0.4 nozzle + process 0.20mm Standard @BBL H2S + Textured PEI Plate, with per-material chamber/bed/drying guidance baked in (⚠ don’t co-print PLA with ABS/PC-FR — highest chamber temp wins → PLA heat-creep); (3) maintenance tracker — 6 H2S tasks assigned to printer id 1 (3DP-093-310), Lubricate Linear Rails DUE; Steel/Carbon-Rod types correctly 400 (A1/P1-only); (4) Telegram provider (POST /notifications/, type telegram) → “Telephely biztonsági riasztások” supergroup, 9 events, test sent (token/chat-id in .env, not logged); (5) bambuddy-mcp on the tailnet for Aperture — stdio-only MCP (731 endpoints via meta-tools) wrapped in an stdio→streamable-HTTP bridge (bambuddy-mcp-bridge, mcp-proxy --host 127.0.0.2 --port 8091, ⚠ pin mcp==1.29.1 in both envs — SDK 2.x breaks proxy + bambuddy-mcp), path-mounted onto the existing bambuddy node (serve --https=443 --set-path=/mcp), endpoint https://bambuddy.taild4189d.ts.net/mcp, verified from personal-mac (mainframe can’t self-hairpin). Security: /mcp has no auth of its own — tailnet + tag:telep ACL is the only gate; no Funnel. — 2026-09-01-bambuddy-slicer-api-pipelines-mcp
• ♻️ Retired the standalone OrcaSlicer KasmVNC sidecar — container/image/on-host config AND the dedicated orcaslicer tailscale node + home-portal tile all removed; replaced by bambuddy’s built-in Slicer API (above). Marked the old note status: outdated with a Superseded callout. — 2026-09-01-orcaslicer-tailnet-deploy
• Deployed OrcaSlicer (3D slicer desktop GUI) on telep-mainframe, streamed to the browser via the linuxserver.io KasmVNC image, behind a dedicated Tailscale sidecar node orcaslicer (100.126.7.69, tag:telep) per the per-service convention — https://orcaslicer.taild4189d.ts.net (tailnet-only, no Funnel). Container -p 127.0.0.2:8570:3000 --shm-size=1gb, config at /home/levander/orcaslicer/config; GPU not passed through (software GL). Portal tile added under 🛠️ Eszközök & Média. Gotcha: a truncated --authkey on tailscale up silently yields NeedsLogin (not NeedsMachineAuth) — NeedsLogin = bad/incomplete key, NeedsMachineAuth = key fine, awaiting console approval. — 2026-09-01-orcaslicer-tailnet-deploy

2026-08-31

• 🔀 OpenChatCut editor trust-model fix behind tailscale serve: the web editor showed “Could not load the MCP connection token” + 403 invalid request origin because the app enforces a hardcoded local-device trust model (loopback socket + loopback Host + same-origin + Sec-Fetch-Site). Fixed in two layers — moved the container to --network host binding Vite to 127.0.0.2 (so the loopback-socket check passes; Docker bridge showed the gateway 172.17.0.1), and patched loopbackHost() to also accept OPENCHATCUT_EDITOR_URL’s host (env-derived, baked via patch-trust.mjs in the Dockerfile) for the loopback-Host check. Bounded, tailnet-only relaxation of a DNS-rebinding defense; same-origin/Sec-Fetch CSRF guards stay intact. Reusable: local-first apps behind serve fail in 3 Host/origin layers (Vite allowedHosts, loopback-socket, loopback-Host). — 2026-08-31-openchatcut-chatcut-deploy
• 📡🛠️ NE200 RF logger BUILT & DEPLOYED — the 71014 login blocker is SOLVED, and live per-cell readings give real-time confirmation the radio is the root cause (QPSK downlink). The NE200 “GDPR encrypt” web login is fully reverse-engineered: POST /cgi_gdpr?9 (⚠ ?9 suffix REQUIRED — its absence WAS the 71014), HTTP/0.9 body sign=<hex>\r\ndata=<b64>; RSA-512 pubkey from POST /cgi/getParm (ee=010001, seq); sign = RSA raw nopadding of key=<K16>&iv=<IV16>&h=<md5(name+pwd)>&s=<seq+len(datab64)> (login) / h=…&s=… (subsequent); AES-128-CBC/PKCS7, 16-ASCII-digit key+iv per session; username is user NOT admin (adminType="user"), hash md5("user"+pwd); login plaintext needs operation:"cgi" (missing → 71011) and no trailing CRLF, success decrypts $.ret=0;; then GET / scrape a 30-hex TokenID header (missing → 406); data queries operation:"gl" WITH trailing \r\n (missing → 406); signal OID DEV2_LTE_SERVING_CELL_INFO (per-cell RSRP/RSRQ/SINR/CQI/RSSI/mod), DEV2_ADT_WAN=byte counters, DEV2_CELL_INTF=IMEI. ⚠ NE200 allows only ONE web session — logger logs out after each run. Deployed: /home/levander/net-monitor/ne200_signal.py on telep-mainframe (pure stdlib + curl + openssl, no pip deps), cred in .ne200_cred (chmod 600), cron */15 → rflog.csv (cols timestamp_iso,nr_rsrp,nr_rsrq,nr_sinr,nr_cqi,nr_dl_mod,nr_ul_mod,lte_rsrp,lte_rsrq,lte_snr,lte_rssi,lte_cqi,lte_dl_mod), correlate vs netlog.csv by timestamp. ⚠ Caveat: nr_sinr raw units unclear (raw 20-70 vs UI 5-9.5 dB) — treat as relative; the unambiguous indicators are nr_dl_mod (QPSK vs 256-QAM), nr_rsrp/nr_rsrq, nr_cqi. ✅ Confirming evidence ~12:43-12:45 (33%-degraded day): serving N78 5G RSRP swung -100→-106→-108 dBm in ~2 min, RSRQ -12/-13, CQI 10-12, downlink QPSK on EVERY sample (uplink 256→64-QAM once); LTE B3 anchor RSRP -95…-98, CQI 6, QPSK; web UI SS-SINR dropped 9.5→5 dB, data now 784.5 GB. QPSK = ~4x throughput loss vs 256-QAM; the 6-8 dB RSRP swings in 2 min explain the episodic collapses — direct real-time proof the radio, not DNS/LAN/router, is the cause. — 2026-08-31-episodic-wan-degradation
• 📡📶 RF EVIDENCE captured from the NE200 web UI (192.168.254.1) CONFIRMS the episodic-WAN root cause is the cellular radio — and reframes the link: the NE200 is a 5G NR CELLULAR FWA CPE on a Telekom HU SIM, NOT a WISP point-to-point radio. Baseline read ~mid-day on a 33%-degraded day: Internet Connected, ISP Telekom HU NET, SIM prepared, NR5G on bands B3/B3/B8/N78 (LTE anchors B3 1800 ×2 CA + B8 900, plus 5G N78 3.5 GHz), Signal 75%, SS-RSRP -102 dBm (POOR for NR, -100…-110 band), SS-RSRQ -12 dB (FAIR), SS-SINR 9.5 dB (FAIR, low end — the throughput-limiting metric), 777.405 GB total used, PS session 11 h 19 m (short-lived), WAN CGNAT 10.182.14.228 + public v6 2a00:1110:138:1496::/64, gw 10.182.14.229, carrier DNS 84.2.46.1/84.2.44.1 + 2001:4c48:2::1/2001:4c48:1::1. ✅ Confirmed: at SINR ~9.5 dB the modem can’t sustain 256-QAM → forced to 64/16-QAM → throughput cut several-fold, collapsing to the 3-15 Mbit in netlog when SINR dips further (good 5G wants SINR >15-20 dB; RSRP -102 on N78 3.5 GHz is genuinely weak). ⚠ Competing hypothesis for the MULTI-day blocks: the Aug 19-22 (100%×4d) → Aug 23-25 (clean) shape fits a carrier DATA-CAP / fair-use throttle (777 GB used) better than pure RF jitter — check the Telekom plan’s cap/reset date BEFORE investing in antenna aiming, since a cap won’t be fixed by aiming. 🎯 Next actions (revised, RF-pull now DONE): (1) [biggest lever] physically aim/adjust the NE200 antenna to raise RSRP/SINR; (2) check Telekom plan for a monthly cap; (3) extend 2026-07-28-net-monitor to scrape RSRP/RSRQ/SINR each run — the measurement that disambiguates RF vs carrier — ⚠ blocked: NE200 login is an AES-128-CBC + RSA-512 “GDPR encrypt” scheme, a scripted login returned error 71014 (signature rejected) and is UNSOLVED, so the scraper needs a solved login or headless browser; (4) netlog CSV still strong evidence for a carrier complaint if no cap. 5G NR reference bands recorded for future triage (RSRP/RSRQ/SINR excellent→none). Also corrected the “fixed-wireless/WISP” mischaracterization in telep-router’s WAN + Double-NAT sections. — 2026-08-31-episodic-wan-degradation, telep-router, 2026-07-28-net-monitor
• 📡🐌 “Terrible internet speed” is NOT DNS — it’s EPISODIC WAN DEGRADATION on the NE200 fixed-wireless uplink, upstream of the LAN. 35 days of netlog.csv on telep-mainframe classified by day (samples < 100 Mbit = degraded, normal 180-265 Mbit) show multi-day episodes separated by clean periods: Aug 8 52%, Aug 19 55% → Aug 20 100% → Aug 21 100% → Aug 22 82% (worst episode, 4 days), Aug 26 52%, Aug 31 33%; Aug 23-25 + Aug 27-30 clean at 0-9%. Worst samples 3.0 / 3.1 / 4.6 / 9.1 / 11.2 / 14.1 / 14.2 / 17.2 Mbit; Aug 26 logged three 100%-loss samples. Latency stays healthy ~25-40 ms even during throughput collapse ⇒ capacity/loss fault, NOT a latency fault. ❌ RULED OUT with evidence — do not re-investigate: DNS (30 ms from all three resolvers — Tailscale 100.100.100.100, router 192.168.1.1, 1.1.1.1; curl -w showed time_namelookup ~0.03 s while time_connect was slow, the inverse of a DNS problem); IPv6/Happy Eyeballs (the 07-28 root cause — fix still holding: en8 has only link-local fe80::, ndp -rn shows no RA on en8, curl -6 fails in 2-32 ms instead of stalling 2-7 s); LAN (Mac on telep-router port lan3 @ 1000baseT full-duplex, 0 interface errors either side, 0% loss to gw @ 0.72 ms over 150 pkts, counter-delta proved 111 MB forwarded to lan3 cleanly); bufferbloat (loaded latency only 26.4 → 27.4 ms — SQM working). ⭐ A Mac-specific cause was suspected and RETRACTED — one window had the mainframe at 177 Mbit vs the Mac at 39-55, but later samples had the Mac at 80-99 Mbit while the mainframe showed 5-7 s TCP SYN-retransmit stalls; that gap was sampling noise. Lesson: in an episodic-fault environment never conclude from a single paired comparison. Live capture caught the burstiness directly: one window = 26.7% ICMP loss from the Mac + TCP handshakes of 5.1 / 5.1 / 7.1 s from the mainframe to the same host; minutes later 150-pkt runs from both machines and hop-by-hop from the router (NE200 192.168.254.1, ISP hops 10.153.240.158 / 84.1.85.225, 1.1.1.1) all showed 0% loss. 🔁 Reusable signature: DNS fast + latency normal + bufferbloat fine + 0% loss on most samples, but throughput collapsing 5-50x intermittently and TCP connects occasionally taking 1/2/4/5/7 s (SYN retransmit backoff); discriminate from the IPv6 fault by whether curl -6 stalls (IPv6) or fails instantly (not IPv6). ⚠ SECONDARY, UNRELATED FAULT FOUND: router port 10g-sfp is badly faulted — 3,792 carrier_changes vs 26 on next-worst lan2 over 5 d uptime, 252 rx_crc_errors (the ONLY port on the router with any), operstate=down + speed=65535 (invalid), only 414 KB rx in 5 days, flapping on a ~1 s cycle in bursts; it IS a br-lan member but br-lan STP is disabled (stp_state=0, topology_change=0) so it is NOT causing bridge-wide topology churn — almost certainly a faulty/empty oscillating SFP module, spams syslog, remove or replace. Also noted: port lan5 linked at only 100 Mbit with 1 GB rx — unidentified device worth finding. Environment confirmed: double-NAT intact (router WAN 192.168.254.2 behind NE200 192.168.254.1), NE200 exposes HTTP :80 + telnet :23 at 192.168.254.1 (RF signal/SNR not yet retrieved — needs credentials), router uptime 5.04 d but WAN uptime only 6.8 h (wan carrier_changes=9). ⚠ net-monitor’s 20 MB sample is small enough that TCP slow-start inflates variance — trust day-level aggregates, not single rows. NEXT (not done): (1) scrape NE200 RF/SNR/modulation from 192.168.254.1 during a degraded episode — the missing evidence for RF-link vs ISP; (2) extend probe.sh to log NE200 RF stats every run so the next episode is retroactively diagnosable; (3) pull the faulty SFP; (4) the netlog CSV is strong evidence for an ISP complaint — Aug 20-21 were 100% degraded all day. — 2026-08-31-episodic-wan-degradation, 2026-07-28-net-monitor, 2026-07-28-ipv6-slow-internet, telep-router, telep-mainframe-handover
• 💀🧠 PROVEN: telep-mainframe’s crashes are a CPU HARDWARE FAULT — mce: CPUs not responding to MCE broadcast: 8-9 → Kernel panic … Not all CPUs entered broadcast exception handler. CPUs 8-9 = the two SMT threads of ONE physical P-core (core_id=16) on the i9-12900K. Board Gigabyte Z690 AORUS MASTER, BIOS F29 (2024-09-27), kernel 6.12.100+deb13-amd64; panic captured 2026-08-31 02:06:30 UTC after 8 days uptime. TRIGGER, not cause: obsidian_sync.sh → obsidian_index.py (kb-vectors embedding pipeline, torch 2.13.0+cpu + sentence-transformers, PID 82973 named in the panic) — a sustained all-core load. The sync had been failing to start since 08-26, so the box was idle-stable for 8 days; new vault notes on the night of 08-30/31 gave it a real batch and the machine died within seconds. obsidian-sync.timer + telep-kb-obsidian-sync.timer now disabled (removes the trigger, not the fault). 🔴 THE TRAP — NVIDIA IS A RED HERRING, RULED OUT: the repeated WARNING … nv_drm_revoke_modeset_permission [nvidia_drm] (DKMS 550.163.01, Comm: vo = mpv/camwall) appear before AND after the panic and on healthy boots; two separate agents independently blamed nvidia and both retracted — 2026-08-31-nvidia-drm-host-crash-embedding-pass is now marked superseded. 🔴 Why every earlier crash was forensically invisible: efi_pstore_write runs in <#MC> context where the FPU is unavailable → kernel_fpu_begin_mask trips a WARNING and the EFI variable write fails → /sys/fs/pstore always empty; pstore will KEEP failing for MCE panics, netconsole is the only method that works. Thermal evidence is against a simple thermal story: continuous sampling puts core_id=16 at avg 38.1 °C, mid-pack, other cores peak higher — the initial one-shot 54 °C was a transient artifact. ✅ Mitigation confirmed working: 43+ minutes stable with both timers disabled (vs a baseline of 4 crashes in ~20 min) — but this is mitigation, not a fix; the faulty core is untouched and any sustained all-core load is expected to provoke it again, so long uptime only means nothing has loaded core_id=16 hard. Instrumentation now live (all units confirmed enabled): netconsole-target.service 192.168.1.123:6665 → telep-router 192.168.1.1:6666, log /tmp/netconsole/kmsg.log (⚠ RAM-backed — copy off after a crash); thermalwatch.service → /var/log/thermalwatch/samples.log (10 s, fsync’d per line, ~10 days, tags the suspect core SUSPECT_core16_cpu8_9=); kernel.printk = 5 4 1 7 because the box boots quiet at console_loglevel 4 which was silently dropping EVERY KERN_WARNING including soft/hard-lockup and hung-task. Next steps (hardware): BIOS defaults → F34a (Jul 2026, notes “Fixed VccSA option visibility for 12th Gen”) → isolate CPUs 8-9 → CPU warranty. ⚠ F31/F32+ enable Secure Boot + pre-boot DMA BY DEFAULT → the unsigned NVIDIA DKMS module won’t load → camwall dies; disable Secure Boot after flashing. ⚠ Prefer Q-Flash Plus (board controller, no CPU/OS involved) precisely because a core is faulty. ⚠ NOT a microcode story — Debian intel-microcode already loads 0x3d, newer than any BIOS. — 2026-08-31-telep-mainframe-mce-hardware-fault, telep-mainframe, telep-router, 2026-08-31-nvidia-drm-host-crash-embedding-pass, runbooks-index
• 🪟📗 Drive runbook brought up to date: read-only /windows external storage, UUID fstab, a scan timer that no longer walks 123k NTFS entries, full file-action routing, drive-mcp, and tag identity. New /windows storage — id 2, Drive /windows, read-only at FOUR independent layers: kernel ntfs3 ro → docker bind :ro → Nextcloud readonly: "1" via a PermissionsMask storage wrapper → code-server files.readonlyInclude. (id 1 = /CAD, datadir /mnt/cad, rw — its bind carries no :ro.) ⭐ The two consumers bind the Windows partition DIFFERENTLY, and it matters: Nextcloud binds /mnt/win:/mnt/windows:ro — the whole C: drive — and narrows to Users at the Nextcloud layer via datadir: /mnt/windows/Users; code-server binds /mnt/win/Users:/home/coder/windows:ro, narrowing in the bind itself. Consequence worth knowing: widening Drive to the full C: drive is ONE command, no compose edit, no restart — occ files_external:config 2 datadir /mnt/windows — whereas changing code-server’s view does need a compose edit. ⚠ host path is win, Nextcloud’s container path is windows. ⚠ code-server binds Drive-home per user (…/nextcloud/data/<user>/files:/home/coder/drive/<user>:ro), not as one tree — a new Nextcloud account is invisible in the IDE until its line is added; quote any path containing @. filebrowser retired — it was the only other door onto the same data (container stopped, volumes preserved, its tailscale serve :8445 removed). 🔴 /etc/fstab now mounts /mnt/win by UUID because the NVMe device nodes SWAP across boots (nvme0n1p2 ↔ nvme1n1p2) — a device-node mount would eventually target the wrong disk. 🔴 Scan timer restructured: naive files:scan --all with the windows mount exceeded 10 minutes and never completed on a 3-minute timer; now files:scan --all --home-only + a targeted /CAD scan → 0.83 s measured. File actions: .stl→O3DV, .csv/.docx→OnlyOffice (native defFormats config, not a custom action), code files→VS Code, .md/.txt→Nextcloud Text, everything else→VS Code as fallback, Download always last, “Open with” submenu on every file (API details stay in 2026-08-31-nextcloud-34-custom-file-action-registration). code-server mounts: /home/coder/cad rw, /home/coder/windows ro, /home/coder/drive/<user> ro deliberately — it reuses existing 0644 bits instead of loosening Nextcloud’s data-dir permissions; reversal is one compose line per account. The code node is now tagged tag:telep (matching cad/knowledgebase/home); re-authing reuses the node key so device approval is not retriggered, and ⚠ --hostname=code must be passed explicitly or tailscale up resets it to the OS hostname. New service drive-mcp.service — WebDAV-backed MCP at https://drive.taild4189d.ts.net:8444/mcp, loopback 9100, served from the existing drive sidecar (not a new node), a single shared identity, secrets in /etc/drive-mcp.env (0600 levander:levander — ⚠ in /etc, not next to a compose dir like the nextcloud/code-server .envs); ⚠ FastMCP needs allowed_hosts or it 421s behind tailscale serve (same trap as 2026-08-31-telep-kb-mcp-server). tsauth-proxy tag identity: configured tags map to a Nextcloud user via TS_TAG_USERS in /etc/tsauth-proxy.env — only explicitly-configured tags map, tag:telep maps to nothing, client-supplied Remote-User is stripped after resolution, fails closed; ⚠ caveat: tagging a device REPLACES its personal Tailscale identity. Language: default_language=hu, default_locale=hu_HU, defaultapp=files, force_language deliberately unset so per-user switching survives. Port map: 9100 added; drive now serves 443 + 8443 + 8444. ⚠ Recorded that cadviewer is now a MISNOMER — it handles all file actions, not just CAD — and was deliberately not renamed because a Nextcloud app id is baked into install paths, enabled-apps config and asset URLs. Crash-capture + thermalwatch units added to the maintenance section, linked to the new MCE note. — 2026-08-31-nextcloud-drive-code-server-runbook, 2026-08-31-nextcloud-34-custom-file-action-registration, 2026-08-31-telep-mainframe-mce-hardware-fault, filestash, runbooks-index
• 🔌🤖 The knowledgebase now speaks MCP — telep-kb-mcp.service on 127.0.0.1:9099, own tailnet node, Aperture connector telebkb. FastMCP streamable-http server at /home/levander/telep-kb-mcp/server.py wrapping knowledgebase.service (:8092) over kb-qdrant; vector modules reached via app.py:8 sys.path.insert(0, "/home/levander/kb-vectors"); embeddings BAAI/bge-large-en-v1.5 1024-dim cosine, CPU-only torch. Exposed per tailnet-service-exposure-convention at https://knowledgebase.taild4189d.ts.net:8443/mcp and fronted by the Aperture connector https://ai.taild4189d.ts.net/v1/connectors/telebkb/ (307→upstream). Tool surface reshaped to mirror the historian subagent’s “named sources, explicit selection” pattern: get_collections() / search(collection, query, limit, folder) / get_note(path) / list_topics() — an unknown collection returns {error, available} rather than throwing, so the model self-corrects from the error. Two collections: manuals 7816 pts (hybrid BM25+semantic) and notes (Obsidian, semantic ONLY) — notes can’t be hybrid because hybrid.py keeps its BM25 index in module globals (_bm25/_docs/_facets) → exactly one collection per process; per-collection instances deliberately deferred. 🔴 GOTCHA: 421 Invalid Host header on every tailnet request while loopback worked — mcp ≥1.29 enables DNS-rebinding protection with an empty allowed_hosts (only 127.0.0.1 passes) and tailscale serve forwards the original Host header; the 421 names no layer so it reads as a proxy/auth failure, not an app setting. Fix = FastMCP(..., transport_security=TransportSecuritySettings(allowed_hosts=[...])), overridable via TELEP_KB_MCP_ALLOWED_HOSTS. — 2026-08-31-telep-kb-mcp-server, kb-agent-api, 2026-07-24-knowledgebase
• 📚🧹 Vault→Qdrant indexing: selection is PER-PROJECT and opt-OUT, and the generated index pages are dropped as noise. kb-vectors/obsidian_index.py. Unit of selection = a project (each dir under projects/, each top-level dir, and vault-root .md files as one pseudo-unit); a unit is excluded entirely if ANY note in it carries the frontmatter tag personal — ⚠ one private note silently removes its whole project from search, grep for the tag before debugging “missing” results. Within an included unit LOG.md + TOPICS.md are skipped as scaffolding — near-pure wikilink lists (homelab’s pair alone = 159 chunks at ~0.9 link density) that embed to noise and displace real answers, and whose content is derived restatement of already-indexed notes; index.md, moc-tagged overviews and Agent Landing.md are KEPT (measured 0.09–0.34, genuine prose). ⭐ A link-density heuristic was tried and REJECTED — it split identical file roles incoherently (tatabanya TOPICS.md 0.42 kept vs esp32 TOPICS.md 0.67 dropped) because it measures project size, not page quality; exclude by role when the role is already knowable. Scope: 28 units excluded / 16 included / 218 notes / ~979 chunks. Indexer is incremental + resumable: per-note sha256 content_hash in the payload, unchanged notes skipped, changed notes re-embedded and upserted immediately (not batched to the end), vanished/newly-excluded notes deleted, no delete_collection. Verified live: run1 embedded 2 → run2 re-embedded 0 → run3 re-embedded only the edited note → run4 purged the deleted note. — 2026-08-31-obsidian-vault-qdrant-index-selection, 2026-08-31-telep-kb-mcp-server
• 💥🖥️ The full embedding pass HARD-CRASHES telep-mainframe — nv_drm_revoke_modeset_permission — plus two traps that make diagnosis worse than the bug. Kernel trace nv_drm_revoke_modeset_permission+0x327/0x340 [nvidia_drm] via drm_file_free/drm_release; driver 550.163.01, RTX 3080, nvidia_drm modeset=Y, Xorg + mpv holding DRM fds. The identical trace appears on 2026-08-22, before this work → the driver bug is PRE-EXISTING, but the embedding job coincided with 3 crashes in 3 attempts and the last logged no kernel output at all. The job is CPU-only torch and never opens a DRM fd → it is a trigger, not the direct caller. Real fix = driver update off 550.163.01, or drop nvidia-drm.modeset=1 — both costly (550 is load-bearing for Frigate/OCR CUDA 12.4; modeset=1 is mandatory or the 3080 exposes zero display connectors → no TV wall). Consequence: the notes index build is incomplete — 27 points across 5 of 218 notes; searches against notes look empty and that is not a retrieval bug. 🔴 TRAP 1: a systemd timer with OnBootSec= pointed at a host-crashing job is a BOOT LOOP (host returns → timer fires → host dies); disable the timer FIRST when diagnosing — telep-kb-obsidian-sync.timer is currently disabled for this reason. 🔴 TRAP 2: pkill -f '<pattern>' on a Tailscale SSH host matches the tailscaled SSH wrapper’s own command string and kills your own session — indistinguishable from the crash you’re chasing; use a bracketed pattern ([o]bsidian_index) or match on comm. — 2026-08-31-nvidia-drm-host-crash-embedding-pass, 2026-08-31-obsidian-vault-qdrant-index-selection, telep-mainframe
• 📗🔐 Operational runbook for the Nextcloud “Drive” stack + the new code-server — closes out the Nextcloud plan. Captures the whole build as one actionable page: the per-service Tailscale sidecar pattern (own userspace tailscaled, own /run/tailscale-<svc>/tailscaled.sock + /var/lib/tailscale-<svc> statedir, own tailnet hostname, serve fronting a loopback-only port) across nodes drive/cad/knowledgebase/code; a full port map (NC 11000, tsauth-proxy 11001, OnlyOffice DS 11002, Stirling 8080, o3dv 8087, KB 8092, telep-kb MCP 9099, homepage 3010, code-server 8888 + the adjacent cad mounts 3080/8085/9876) with which tailnet hostname fronts each. 🔴 NO FUNNEL, EVER — standing rule, absolute for code-server which runs --auth none (tailnet membership IS the credential); every serve status mount must read (tailnet only). Identity: Tailscale whois → Remote-User injected by tsauth-proxy → user_saml in environment-variable mode (provider id 1, HTTP_REMOTE_USER), loopback-safe because only tailscale serve fronts :11001 and the proxy strips client-supplied headers — which is exactly why there is NO LAN vhost (a LAN entry point has no tailnet source IP → bypasses identity; deliberately skipped, don’t “helpfully” add one later). nextcloud-scan.timer (3 min) = three ExecStartPre normalization steps then occ files:scan --all; the two shares differ on purpose — data dir gets chown 33:33 (owner+group), CAD exports get chgrp www-data + g+rw with owner preserved because the FreeCAD pipeline writes there as levander; ⚠ ≤3-min lag before a Taildrive-written file is editable from the web UI. Gotchas: Taildrive writes land root:root (no per-share uid map in TS 1.102.2 — the timer is the mitigation); always QUOTE description: in services.yaml (an unquoted colon-space blanks the entire dashboard), tiles use loopback siteMonitor because homepage is host-networked, no tile uses icon:; OnlyOffice needs matching JWT secret + jwt_header=Authorization + internal http://onlyoffice/ http://nextcloud/ URLs + allow_local_remote_servers; ⚠ loopback 8090 is an unrelated python listener — O3DV’s 8090 is the cad node’s serve port → loopback 8087; KB’s socket dir is tailscale-kb while its hostname is knowledgebase. Verification checklist splits honestly into host-provable (loopback curls, occ status/setupchecks, onlyoffice:documentserver --check, files_external:verify 1, timer Result=success, serve status, code-server loopback-only proof 192.168.1.123:8888/100.115.209.87:8888 → 000) vs in-ACL-tailnet-device-only (browser loads of drive/cad/code, OnlyOffice edit+save, .stl click after one hard-refresh, Finder Taildrive mount, dashboard tiles) — the host cannot hairpin to its own tailnet HTTPS. Plus every .bak the work created and what reverting each undoes. — 2026-08-31-nextcloud-drive-code-server-runbook, 2026-08-31-nextcloud-drive-tailscale-plan, 2026-08-31-nextcloud-drive-tailscale-spec, 2026-08-31-nextcloud-34-custom-file-action-registration
• 🧊🐛 Clicking a .stl in Nextcloud downloaded instead of opening Online3DViewer — the custom file action was registered on a DEAD global. Minimal custom app cadviewer at /var/www/html/custom_apps/cadviewer (inside the persistent app named volume → no compose edit), JS injected via \OCP\Util::addScript from a LoadAdditionalScriptsEvent listener. First attempt failed silently — no console error, no nextcloud.log line, action simply never appeared; only symptom was the built-in download firing (XHR HEAD at downloadAction.ts:69). 🔴 ROOT CAUSE: NC 34 ships @nextcloud/files 3.x — the registry is window._nc_files_scope.v4_0.fileActions (a Map, written by registerFileAction, register:action event dispatched); the literal _nc_fileactions appears in ZERO served bundles — it’s a dead global from an older major, so registering there is a no-op. 🔴 Second, independently-fatal defect: NC 34 calls enabled/exec with a single context object {nodes, view, folder, contents}, not positional (nodes, view) — so Array.isArray(ctx) guards fail and the action is filtered out before defaultFileAction selection (which tests a.default !== undefined, presence not value). Working shape: set into the Map, read context.nodes, default: 'default' + order: -100 to beat built-in download (order: 30); match by file extension, NOT mimetype (NC serves .stl/.step/.3mf as application/octet-stream); bump occ config:app:set theming cachebuster so the ?v= changes or clients keep a stale Files bundle (one hard-refresh still needed). ⭐ Lessons: a silent no-op is a signal — grep the actually-served bundles for the identifier before trusting any doc/tutorial/prior code; and with no browser reachable (host can’t reach its own tailnet-only URL), the decisive technique was extracting the real selection predicate from the bundles’ source-map sourcesContent (FileEntryMixin.ts, downloadAction.ts) and driving it in a Node isolation harness with a fake node → genuine PASS/FAIL. O3DV deep link https://cad.taild4189d.ts.net:8090/o3dv/#model=/exports/<enc-basename> resolves only for CAD-folder files (O3DV mounts /home/levander/freecad/exports RO) — documented v1 limitation. — 2026-08-31-nextcloud-34-custom-file-action-registration, 2026-08-31-nextcloud-drive-tailscale-plan, 2026-08-26-freecad-cad-workstation
• 📷🔌 wifi reload raced the cams VLAN into netifd DEVICE_CLAIM_FAILED — Frigate lost all 4 feeds, camwall stuck in start-pre; only network restart cleared it. During Wi-Fi tuning on telep-router (multicast_to_unicast, then dtim_period/uapsd on telep1, each + wifi reload), the cams interface (gw 192.168.30.1 on br-cams, sole member = cams AP phy1-ap1) stuck at ifstatus cams "up": false + errors: ["DEVICE_CLAIM_FAILED"]. Cameras (.119 DÉL, .139, telep_cam1–4) stayed Wi-Fi-associated but L3-unreachable → Frigate lost all 4 feeds, and camwall.service hung in activating (start-pre) (its ExecStartPre loops until curl -sf http://127.0.0.1:5000/api/version). The per-minute /etc/cams-guard.sh couldn’t self-heal — ifup cams cannot clear DEVICE_CLAIM_FAILED (nor ip link set br-cams down; network reload; ifup cams). FIX = /etc/init.d/network restart → cams back "up": true w/ 192.168.30.1/24, camwall active, all 4 feeds ~5fps. Distinct from the 08-15 carrier-down outage (branch on state: br-cams DOWN qdisc noop vs ifstatus cams DEVICE_CLAIM_FAILED). Follow-up: cams-guard being hardened to escalate to network restart after 3 failed ifup cams. Op rule: after ANY wifi reload, verify ifstatus cams | grep '"up"' + ip -4 addr show br-cams. Cross-linked telep-router, camwall-not-on-tv. — 2026-08-31-cams-vlan-device-claim-failed-wifi-reload, telep-router, camwall-not-on-tv
• 🎬🐳 OpenChatCut (conversational AI video editor) deployed as a self-hosted web service + MCP behind a dedicated chatcut Tailscale sidecar. https://chatcut.taild4189d.ts.net (tailnet-only, no Funnel, LE cert); MCP at /api/external-mcp/mcp (Streamable HTTP, Bearer, registered in Claude Code user scope, Connected). Hand-written Dockerfile (node:24-bookworm, system chromium for Remotion), container openchatcut --cpus=8 -p 127.0.0.2:5199:5199, data at /home/levander/openchatcut/data. 🔴 Gotchas: Vite-8 allowedHosts 403s the tailscale-serve host for UI+MCP (no --allowed-hosts flag; patch config/vite.config.ts) — the Vite equivalent of the FastMCP 421 trap; Vite ignores CRA HOST (pass --host 0.0.0.0); ONNXRUNTIME_NODE_INSTALL=skip; buildkit needs --network=host for MagicDNS. ⚠ MCE-fault host + Remotion rendering → --cpus=8 + OPENCHATCUT_RENDER_CONCURRENCY=1/MAX_ACTIVE_EXPORTS=1, avoid batch exports; no app-level auth → tailnet ACL for tag:telep is the only gate. — 2026-08-31-openchatcut-chatcut-deploy

2026-08-30

• 📺🔌 telep-tv AirPlay undiscoverable on Wi-Fi — ROOT CAUSE is the box’s 10G switch port, NOT 5 GHz. Months of blaming the router’s “5 GHz cross-band client isolation” was WRONG for AirPlay discovery. Proven 2026-08-30: the box (telep-mainframe, 192.168.1.123, MAC d8:5e:d3:a7:05:d6, single NIC enp5s0 @ 10000 Mbps) advertises _airplay._tcp/_raop._tcp correctly ON THE WIRE (tcpdump → 224.0.0.251, TXT model=AppleTV3,2 features=0x527FFEE6 srcvers=220.68). From the Mac (192.168.1.113, Wi-Fi): dns-sd -L RESOLVES + ping/ssh work (unicast fine), but dns-sd -B (browse, what Control Center uses) NEVER shows telep-tv. Mac’s multicast RX is healthy (fresh _ipp._tcp finds the wired HP printer; fresh _nut._tcp finds NOTHING from the box) → the box’s multicast specifically never arrives. Fails on BOTH 2.4 + 5 GHz ⇒ not band. Router br-lan fdb: box on port 2 = 10G, printer on port 5 = 1G, Mac on port 11 = phy0-ap0; multicast_snooping=0 (floods), no flow_offloading/packet_steering. ROOT CAUSE: the switch doesn’t flood multicast arriving on the 10G port out to Wi-Fi; 1G-port devices do reach Wi-Fi — hence only multicast-based DISCOVERY fails. RULED OUT: multicast_to_unicast=1 on both SSIDs (no change → reverted to 0), Mac mDNS cache flush / Wi-Fi toggle. FIX options (none applied): (1) move cable to a 1G port; (2) keep 10G + mDNS reflector re-advertising telep-tv at box IP:port; (3) dig into router DSA switch multicast-to-CPU on the 10G port. Corrected the old 5 GHz-isolation assumption in airplay-telep-tv (new symptom 2b), telep-router, and telep-mainframe-handover. — 2026-08-30-telep-tv-airplay-10g-port-multicast-not-flooded, airplay-telep-tv, telep-router, telep-mainframe-handover
• 📺 telep-tv AirPlay vanishes from mDNS after a uxplay restart. Devices show ⚠ telep-tv nem látszik mDNS-en while uxplay.service + avahi-daemon are both active and avahi-browse -rt _airplay._tcp/_raop._tcp return nothing (avahi still advertises _nut._tcp / the HP printer, so avahi is fine). Root cause: uxplay 1.71 publishes its AirPlay/RAOP records via avahi-compat-libdnssd → avahi over D-Bus, and across a uxplay restart/D-Bus hiccup the compat layer doesn’t reliably re-register — uxplay keeps running with its adverts silently gone (observed after a 21:56 restart). FIX (order matters): systemctl restart avahi-daemon → sleep 2 → systemctl restart uxplay (bouncing uxplay alone hits the same race). Added the case to the airplay-telep-tv runbook under symptom (a). — 2026-08-30-telep-tv-mdns-vanishes-after-uxplay-restart, airplay-telep-tv

2026-08-29

• 📋 Session handover for the 2026-08-22 → 29 session — one concise, scannable entry point cross-linking the detailed dated notes (not re-documenting): control-plane OAuth fix, Frigate retention + cat alerts, router 5 GHz cross-band client isolation + Bambu H2S, the FreeCAD CAD studio + Online3DViewer + cad-designer agent + export pipeline, KrakenSDR mobile-DF field-working, Filestash remount. Also refreshed telep-mainframe-handover: added status rows (CAD export + Online3DViewer, KrakenSDR mobile DF field-working, Filestash remounted), the top open-items checklist + §5 detail with the four next-steps (KrakenSDR array_offset calibration; KrakenSDR car 5V/5A power blocker; Filestash reboot-safe systemd mount unit; unresolved 5 GHz cross-band isolation) + Bambu maintenance-due, and updated the standing KrakenSDR item to the field-working state. — 2026-08-29-session-handover, telep-mainframe-handover

2026-08-26

• 🧊 🔒 Online3DViewer (cad:8090): added a native file picker + frame-ancestors hardening. (1) Landing now has an “Open file…” button + click-to-browse drop zone firing a hidden <input type=file multiple accept=".step,.stl,.3mf,.gltf,.bin,.obj,.mtl,…">; both picker and drag-drop share one openInViewer(files) path (File → /o3dv/ iframe #open_file → change → LoadModelFromFileList), multi-file passes together, no upload. 🔴 Bug found+fixed: the picker’s input.value='' reset empties the live FileList → openInViewer must Array.from(files) up front or the first picked file silently fails. (2) nginx now sends X-Frame-Options: SAMEORIGIN + Content-Security-Policy: frame-ancestors 'self' (both always, server-level) — blocks cross-origin framing, still permits the same-origin landing→/o3dv/ embed (the JS frame-buster stays; header is the proper control). All 5 checks pass headless: picker STEP+STL render, drag-drop still works, /exports click-to-view works, curl -I shows both headers, same-origin iframe still renders with headers active. Zero console errors. — 2026-08-26-freecad-cad-workstation
• 🧊 🖱️ Added drag-and-drop to the Online3DViewer landing page (cad:8090). Drop a local file anywhere on the landing → overlay opens an iframe of /o3dv/ and the dropped File objects are fed client-side to its #open_file input (input.files=<DataTransfer> + change → LoadModelFromFileList); no server upload (a local File can’t ride a URL hash). Full toolbar (orbit/measure/section) since it’s the real website. 🔴 Blocker cleared: O3DV’s if(window.self!==window.top) anti-embed guard blanked the iframe — Dockerfile now sed-patches source/website/index.js (→ false, esbuild dead-code-eliminates it). Verified headless: local STEP (600v/588t, OCCT translator) + STL (1716v/572t) render on drop, click-to-view list stays intact, zero console errors. — 2026-08-26-freecad-cad-workstation
• 🧊 🖥️ Deployed a self-hosted Online3DViewer (3dviewer.net engine, kovacsv/Online3DViewer v0.19.0) on telep-mainframe for the FreeCAD CAD exports → https://cad.taild4189d.ts.net:8090/. Compose service o3dv (multi-stage node:20 build → nginx:alpine, host 127.0.0.1:8087), one origin serves / landing + /o3dv/ viewer + /exports/ models (no CORS). Landing is pure-JS off nginx autoindex_format json → new exports show up on reload, click-to-view launches /o3dv/#model=/exports/<file>. STEP imports client-side via occt-import-js WASM — self-hosted (Dockerfile sed-patches importerutils.js off jsdelivr → local /o3dv/libs/). Routed via tailscale serve --bg --https=8090 on the cad node (persisted, reboot-safe). Verified headless: plate.step renders through OCCT (600 verts / 588 tris, 60×40×4 mm), zero console errors, all wasm 200. — 2026-08-26-freecad-cad-workstation
• 📡 ✅ MILESTONE — mobile TETRA-uplink DF is WORKING end-to-end; field-testing across the city. Two deltas made it work: (1) the official Kraken Pro app HAS an iOS build (earlier “Android-only” assumption CORRECTED) — iPhone joins hunter-ap (SSID hunter-ap/huntme123, Pi 10.42.0.1), app connects LOCAL to server 10.42.0.1, uses the phone’s own GPS for the map (config already had doa_data_format="Kraken App" + krakenpro_key=0ae4ca6b3; verified connected). ⇒ the whole USB-tether/gpsd/USB-GPS-puck effort is UNNECESSARY for the DF map (Apple NMEA lockdown irrelevant); gpsd/tether only if you want the Pi itself online (optional). Diagnostic: from iPhone Safari http://10.42.0.1:8080/doa confirms the network path before blaming the app. (2) POWER root cause of today’s reboots — the Pi 5 + DAQ browns out and resets on 5V/3A (generic PD-30W only gives 5V/3A → does NOT help); needs a real 5V/5A (throttled=0x0 under load, usb_max_current_enable auto→1); earlier “OOM crashes” were largely these brownouts. Car: 12V→5V/5A buck + usb_max_current_enable=1, Kraken+phone on their OWN feeds (Kraken self-powered via its own USB-C). Field technique: ANT-0 points in direction of travel (app uses GPS heading), signal bursty/trunked (bearings only while a handset TXes — many samples, drive across/around), array_offset=0=relative, keep gain (19.7) just under overdrive. Final working config: 382.114 MHz uplink, 19.7 dB, 25 kHz VFO, UCA 0.20 m, ext-3 whips, MUSIC, decorrelation Off, short-bursts On, hunter-ap on wlan0, zram reboot-safe. — 2026-08-26-krakensdr-field-test-milestone, 2026-08-12-krakensdr-doa-rig, krakensdr-df
• 📡 ✏️ CORRECTION + FINAL CONFIG for the kraken-rig retune — the same-day 433 draft below is SUPERSEDED. The DoA is now LIVE on the TETRA mobile UPLINK at 382.114 MHz (10 MHz below the 392.114 base downlink), SDR connected, verified (DoA UI HTTP 200). settings.json: center_freq/vfo_freq_*=382114000, uniform_gain=19.7 dB (down from 38.6 — overdrive = ADC clipping that corrupts DoA phase), vfo_bw_0=25000 (TETRA channel), ant_arrangement=UCA ant_spacing_meters=0.2, doa_method=MUSIC doa_decorrelation_method=Off en_optimize_short_bursts=True, array_offset=0 (uncalibrated, bearings relative), location_source=gpsd. 🔧 ARRAY CORRECTED: radius is 200 mm (printed pentagon spacer hole; holes 100/150/200/250 mm) = KrakenSDR default s≈0.33 (r = sλ/1.176 for n=5), unambiguous across 382–434 MHz — NOT the earlier “25 cm rebuild”; the 0.30 m array’s ~428 MHz ceiling stands as a fact about the old geometry only. 📶 WHIP CORRECTED: ext-1 (366–950 MHz) covers BOTH 382 and 433 → no whip swap between bands (382/392/433 all in range). 🛠️ zram made REBOOT-SAFE via drop-in /etc/systemd/system/zramswap.service.d/reset.conf (ExecStartPre swapoff /dev/zram0 + echo 1 > /sys/block/zram0/reset) fixing the boot-time “Device or resource busy” fail; 1 GB zstd zram confirmed active. 📍 gpsd+gpsd-clients installed for the phone/USB-GPS path. 🚗 FIELD PLAN: USB-tether an Android phone → data on usb0, wlan0 stays the AP (no AP↔client flip); iPad = in-car display only (http://10.42.0.1:8080/doa); best GPS = a USB GNSS puck (u-blox VK-172/VK-162); ESP32 is NOT a NIC, iOS/iPad is a poor GPS source. — 2026-08-26-raspi-oom-zram-hunter-ap-433-retune, 2026-08-12-krakensdr-doa-rig, krakensdr-df
• 📡 🛠️ Three changes to the Pi 5 raspi/kraken-rig (KrakenSDR DoA + wifi-hunter host) after it OOM-crashed twice then wouldn’t boot. BOOT: the non-boot was just the SD card physically pulled — reinserting fixed it; card NOT corrupted (bootfs FAT clean, ext4 root rw, no EXT4 errors) → reseat before assuming corruption. (1) zram swap (OOM fix on the 2 GB Pi): zram-tools, /etc/default/zramswap ALGO=zstd PERCENT=50 PRIORITY=100, zramswap.service enabled → /dev/zram0 ~1 GB. 🔴 GOTCHA: systemctl restart zramswap reports “failed” while the device is already active (can’t re-init a live zram) but swap IS working (/proc/swaps shows zram0); clean reset = swapoff /dev/zram0; zramswap stop; systemctl restart zramswap; fresh boot starts clean on its own. (2) hunter-ap AP up: sudo nmcli connection up Hotspot → wlan0 AP 10.42.0.1/24, SSID hunter-ap/huntme123, DoA UI http://10.42.0.1:8080/doa. 🔑 CLARIFICATION: eth0 (wired) + wlan0 (AP) run SIMULTANEOUSLY — the single-radio limit is intra-wlan0 only (AP vs client); bench keeps wired uplink AND the AP; the field flip is only when eth0 is unplugged. (3) 433.92 MHz DoA retune (garage keyfob DF): center_freq=433.92 + vfo_freq_*=433920000 in krakensdr_doa/_share/settings.json (backup settings.json.bak-pre433), config-only (SDR unplugged), applies next start. ⚠️ RF GEM: the 0.30 m TETRA pentagon has a ~428 MHz ambiguity ceiling (spacing=1.176×R must stay < λ/2) → 433.92 is ABOVE it = aliased bearings; rebuild to ~0.25 m radius + ext-1 whips (~17 cm) for unambiguous 433 (λ=69.1, λ/4=17.3, λ/2=34.6 cm), then match the DoA custom_array coords. Keep Optimize-Short-Bursts ON (bursty OOK). — 2026-08-26-raspi-oom-zram-hunter-ap-433-retune
• 🤖 🖨️ Added a FreeCAD export pipeline + a dedicated on-prem CAD-designer Claude agent on telep-mainframe. (1) Export helper /exports/cad_export.py — export_all(obj, name) writes STEP+STL+3MF (mm, LinearDeflection 0.05 / AngularDeflection 0.5) in one call; verified via MCP execute_code. (2) Task A — browse exports: nginx:alpine exports-http container (autoindex, ./exports:/exports:ro, host 127.0.0.1:8085) routed through the cad node’s tailscale serve --https=8080 → https://cad.taild4189d.ts.net:8080/ (no collision with noVNC :443/ or MCP :8443/mcp); verified plate.3mf downloads from the Mac. (3) Task C — auto-pull to Mac: launchd com.levander.cad-exports-sync.plist (90 s) rsyncs …/freecad/exports/ → ~/cad-exports/ (LAN-first, tailnet fallback, quiet/idempotent); verified files land. (4) CAD agent cad-designer.service — a SECOND always-on claude remote-control (workspace /home/levander/cad-agent, persona CLAUDE.md, FreeCAD MCP wired at 127.0.0.1:9876/mcp → ✔ Connected), modeled on the control-plane operator agent; appears in the Claude app Code tab as session cad-designer-telep. 🔴 GOTCHA: a Type=forking tmux unit on the control-plane’s SHARED default tmux server never persists (no new daemon for systemd to track → thrash loop) — gave it its OWN socket -L cad-designer → active, NRestarts=0, enabled at boot. — 2026-08-26-cad-designer-agent, 2026-08-26-freecad-cad-workstation
• 🛠️ 🖥️ Stood up a headless FreeCAD CAD workstation on telep-mainframe with browser preview + an MCP server, on its OWN Tailscale node cad (cad.taild4189d.ts.net / 100.120.203.1; dedicated tailscaled-cad.service, statedir /var/lib/tailscale-cad, authed off /etc/agent-tsauthkey). noVNC live GUI at https://cad.taild4189d.ts.net/, MCP (Streamable HTTP, 15 tools, server FreeCADMCP) at https://cad.taild4189d.ts.net:8443/mcp. Stack = LSIO FreeCAD GUI + Selkies noVNC + the neka-nat freecad-mcp addon auto-started ON the GUI thread (so MCP-created parts render live in the browser) + a mcp-proxy→freecad-mcp(stdio)→XML-RPC:9875 bridge container. Verified end-to-end: MCP initialize/tools/list over the tailnet, a 40×20×5 mm plate w/ 6 mm hole built via execute_code, rendered in the live viewport, and exported to STL + glTF in /home/levander/freecad/exports/. Reboot-safe (compose unless-stopped + enabled units + persisted serve). Key gotchas: proximile’s container runs RPC in a separate offscreen FreeCAD (parts wouldn’t show in noVNC) so we used the GUI-thread addon instead; pin mcp<2 or mcp-proxy crashes on request_ctx; ruview owns 3000/3001 so noVNC is on host 3080/3081. — 2026-08-26-freecad-cad-workstation
• 📡 🧰 Cross-linked the KrakenSDR rig to its new mechanical half. The 3D-printed car enclosure for this exact rig (Kraken + cased Pi 5 + 12 V power bay) is now its own project, krakenpi-carbox — model complete and geometrically verified 2026-08-26, not yet printed. Added the pointer to the rig note’s agent callout, its car-deployment open item, and its Related list. Reusable facts recorded there: Kraken 177.3 x 113.5 x 25.86 mm (+4.7 mm fan finger guard) from drawing KH-ASSEMBLYC1, mounted via its own 8x M3 case-assembly screws on a 162.6 x 88.8 pattern — which means M3x16, not the stock M3x12 — and the 5-SMA bank sits −1.7 mm off the body centreline. — 2026-08-12-krakensdr-doa-rig, krakenpi-carbox
• ✏️ 🖨️ CORRECTION: the Bambu H2S is DUAL-BAND, not 2.4-only. Observed associated to the 5 GHz SSID telep1 (phy0-ap0, ch36, −40 dBm, 150 Mbit) — so the earlier “Bambu printers are 2.4 GHz-only” claim was wrong. Implication: the cross-band Wi-Fi isolation blocker only bites when the printer is on 2.4 (telep1-2G) while the Mac is on 5 GHz; both on telep1 (5 GHz) just works (Ethernet still most robust). Also captured a reusable standby gotcha: a Bambu printer in STANDBY/SLEEP keeps its network up (pings/ARP, stays associated) but SHUTS DOWN app services — 8883/990/6000 closed + no SSDP — so “pings but all ports closed + no SSDP” = asleep, not a network fault (wake via touchscreen). Corrected 2026-08-26-router-reflash-mac-5ghz-dfs-and-bambu-offline, telep-mainframe-handover, TOPICS. — 2026-08-26-router-reflash-mac-5ghz-dfs-and-bambu-offline
• 🐱 📡 frigate-notify: cat detections weren’t pinging Telegram — cat was tracked in Frigate but missing from the notify alerts.labels.allow list; added it. Alerting stack = container frigate-notify (ghcr.io/0x2142/frigate-notify v0.5.4) on telep-mainframe, config /home/levander/nvr/frigate-notify/config.yml, polls Frigate’s WEB API (frigate.webapi.enabled:true, 15s; MQTT off) at http://frigate:5000, Telegram provider (title “Telep riasztás”, send_clip:false). Filter model = two independent lists: frigate.cameras.exclude (telep_cam3+telep_cam4 EXCLUDED → only telep_cam1/2 notify) and alerts.labels.allow (was person,car). cat tracking was enabled in Frigate on 08-22 but NOT added to labels.allow, so cats never notified for 4 days. RULE: a new tracked object must be added to BOTH Frigate objects.track AND frigate-notify alerts.labels.allow. ✅ FIX: added cat → person,car,cat (backup config.yml.bak-add-cat) + docker restart frigate-notify (came up clean: Successfully connected / Config file validated / App ready). Two reusable gotchas: (1) after a Frigate restart frigate-notify logs ERR Cannot get reviews ... /api/review error=500 — Frigate API briefly 500s during its own restart, recovers on its own (restart frigate-notify to resync cursor); (2) it can get STUCK re-processing the SAME review_id for an EXCLUDED camera every 15s (Processing review / Event dropped - Camera Excluded / Review dropped - No events eligible spam on an active excluded review) → docker restart frigate-notify clears it. Verified: last good Alert sent · provider=Telegram 2026-08-25 21:19; Frigate healthy (64 person + 36 car alerts/24h, 0 cat events yet). — 2026-08-26-frigate-notify-cat-alerts
• 🖨️ 📡 Bambu Studio can’t reach the H2S printer when the Mac is on 5 GHz — traced to DRIVER-level cross-band Wi-Fi client isolation on telep-router (OpenWrt 24.10), plus the .202 DHCP reservation done. Mac on telep1 (5 GHz, phy0-ap0) can’t ping/reach the printer 192.168.1.202 on telep1-2G (2.4 GHz, phy1-ap0) — 8883/990/6000 blocked, SSDP no reply; same band works. Isolated it via ssh root@100.69.112.32 (Tailscale SSH — worked when key/pw auth was locked out post-reflash): router reaches both clients (hub-and-spoke), but every config knob reads non-isolating (hostapd ap_isolate only on guest telep-cc/phy1-ap1, uci isolate='0', bridge isolated=0 learning=1 *_flood=1 same as ethernet, br-lan vlan_filtering=0, no br_netfilter); wifi down; wifi up changed nothing. Decisive test: wifi→ethernet works (reaches wired .123/.200) but wifi→wifi cross-band fails (.202) ⇒ isolation is in the radio driver/firmware, below all config layers. Fixes: proper = wire the printer over Ethernet (wifi→ethernet forwards, reachable from any band); instant = put the client on the same band. Tailscale does NOT help an on-LAN client — the Mac’s connected en0 route to .202 beats any tailnet subnet-route, traffic never enters Tailscale; subnet-router is only for reaching the printer from OUTSIDE the LAN. Correction: the earlier DFS theory was wrong — the 5 GHz SSID is telep1 on ch36 (non-DFS). Also added an OpenWrt DHCP static host name='bambu' mac='50:31:23:c9:63:ba' ip='192.168.1.202' dns='1' → resolves bambu.lan (convenience only, doesn’t change the route/fix cross-band); marks the long-standing .202 reservation open item DONE. — 2026-08-26-router-reflash-mac-5ghz-dfs-and-bambu-offline
• 🌐 🖨️ Post-reflash diagnosis: Mac couldn’t see 5 GHz SSID (DFS channel) + Bambu H2S offline — both traced to telep-router being reflashed/reset. (1) 🍎 The Mac’s Wi-Fi was FINE throughout (connected, valid IP, internet+DNS OK); networksetup -getairportnetwork falsely said “not associated” and system_profiler SPAirPortDataType redacted SSIDs — that’s the macOS Location-Services privacy redaction on CLI tools, NOT a disconnect. Verify association with ifconfig en0 (status active + inet), ipconfig getifaddr en0, route -n get default, ping — NOT networksetup/system_profiler. Toggling setairportpower off/on left en0 unable to re-associate (err -3900) — avoid. Root cause the SSID was missing from the menu (phones saw it): telep1-5G on a DFS channel (52–144) which macOS passive-scans and often won’t list — “it’s the Mac.” Fix is router-side: set 5 GHz radio to a non-DFS channel (36/40/44/48 or 149/153/157/161) — uci set wireless.radioX.channel='149'; uci commit wireless; wifi reload (radioX via iwinfo); may still be open pending the channel. Mac saved nets have telep1-2G+telep1 but no telep1-5G (naming split). (2) 🖨️ Bambu Lab H2S (SSDP internal model O1S, DevName 3DP-093-31, SN 0938BJ641800310, LAN .202) offline — the reflash disrupted Wi-Fi and the 2.4 GHz-only printer (on telep1-2G, unrelated to the 5 GHz/DFS issue) lost its link. Bambu Studio’s log_iotc.txt IOTC_Check_Session_Status Error: Not Initialized! ErrCode -12 is a symptom not a cause; confirmed offline via SSDP M-SEARCH probe (ST: urn:bambulab-com:device:3dprinter:1 → 239.255.255.250:1990/:2021, no reply) + no ping at .202 + no service ports. A router Wi-Fi restart recovered it: SSDP replies from .202, LAN-mode ports open (8883 MQTT/TLS, 990 FTPS, 6000 camera, 322), Studio LAN mode reconnects. Reusable check: nc -z 192.168.1.202 8883 990 6000. Follow-ups: reserve .202 in DHCP (still pending), wire the H2S over Ethernet to end Wi-Fi churn. Also: the reflash reset the router’s SSH host key + authorized keys — re-add your pubkey to /etc/dropbear/authorized_keys (or LuCI/console); clear stale known_hosts with ssh-keygen -R 192.168.1.1. — 2026-08-26-router-reflash-mac-5ghz-dfs-and-bambu-offline

2026-08-22

• 🔴 ⭐ FIXED — control-plane agent OAuth error on telep-mainframe root-caused to a ZEROED ~/.claude/.credentials.json. control-plane.service (claude remote-control) showed active/running but the agent pane churned Reconnected after Ns and the Claude app threw an OAuth error. ROOT CAUSE: the creds file (rewritten 07:48 that morning) had every token empty — claudeAiOauth.accessToken/refreshToken len 0, expiresAt=0, and every mcpOAuth.*.accessToken (Supabase/Gmail/Drive/GitHub) empty (only subscriptionType=max metadata survived). With an empty refreshToken silent refresh was impossible → full re-login required. ✅ FIX: interactive claude login on the box in a REAL terminal (ssh -t levander@telep-mainframe '~/.local/bin/claude login', open URL, approve, paste code). The agent shares the same per-user creds file, so after login the service (already on Restart=always) picked up the new tokens automatically — no restart needed. Verified: access/refresh len 108, expiresAt ~8 h out, service active(running), pane ✔︎ Connected · obsidian · main. 🔴 GOTCHAS: (1) claude login needs a TTY — fails via Claude Code’s ! prefix or plain ssh (OAuth session expired and could not be refreshed); (2) restarting the service on empty creds just re-loops — login is the only fix; (3) off-LAN, 192.168.1.123/hostname unreachable, only the relayed Tailscale 100.115.209.87 works and it’s SLOW (~800 ms RTT via DERP) → SSH needs ConnectTimeout 30 s+ or it times out during banner exchange. — 2026-08-22-control-plane-oauth-zeroed-creds
• 🚨 CRITICAL finding — Frigate RECORDING is broken (0 clips written) — logged in telep-mainframe-handover. Cameras stream fine (camera_fps ~5 telep_cam1..4) but no .mp4 segments land in /srv/frigate/recordings; degraded for DAYS (daily sizes collapsed: 08-20=68 MB, 08-21=98 MB vs expected GBs). A docker restart frigate did NOT fix it; NOT disk (root LVM 6% used) and NO errors in docker logs frigate. Suspected: repeated radio1 wifi reloads / br-cams drops / telep1 SSID split wedged the Tapo RTSP into go2rtc and the record pipeline jammed silently (fps-watchdog only heals feed/detect jams, not a record-only stall). Consequence: overnight footage LOST — night of 08-21→22 recorded ~nothing in the dark hours; a “last night” (cat escaped) review couldn’t be fulfilled. Exported surviving footage to /srv/frigate/exports/. Recorded gotchas: Frigate stores recordings in UTC (dirs off by +2h vs CEST), export API POST /api/export/{cam}/start/{epoch}/end/{epoch}, mount /srv/frigate→/media/frigate, retain 14 d. Also corrected the handover’s stale “as of 2026-08-19” date → 2026-08-22 (verified host clock). Added URGENT item to the top of the open-items checklists. — telep-mainframe-handover
• ✅ ⭐ RESOLVED — the Frigate “recording broken” scare above was WRONG: it was a RETENTION-CONFIG issue, NOT a pipeline jam. The wifi-reload / br-cams-drop / go2rtc-wedge hypothesis is closed/corrected. ROOT CAUSE (found 2026-08-22): the resolved record config had continuous.days=0 AND motion.days=0; only alerts+detections retention (14 d each, mode motion) were set, which only keep segments overlapping a tracked object (person/car). During any window with no person/car, ffmpeg still wrote 10s segments into the /tmp/cache tmpfs but Frigate’s maintainer DISCARDED them instead of moving them to /srv/frigate/recordings → the “68 MB/day collapse” was just low person/car activity, not a stall. EVIDENCE everything was healthy: detect camera_fps=5.0 all 4 cams; record ffmpeg running + writing fresh ~1.5 MB segments to /tmp/cache; disk fine (root LVM 6%); the only anomaly = 0 files moved to recordings/ in a 3-min window while cache held segments. 🩺 DIAGNOSTIC ORDER (runbook): /api/stats per-cam camera_fps (detect health) → docker exec frigate ls -la /tmp/cache (ffmpeg producing?) → /api/config resolved record block; cache-has-segments + recordings/-empty + continuous.days==0 + motion.days==0 ⇒ retention config, not a jam. ✅ FIX: edited /home/levander/nvr/frigate/config.yml (mounted /config, NOT /srv/frigate/config; backup config.yml.bak-20260822-catmotion) — added record.motion.days: 10 (user chose motion + 10-day retention over ~150 GB/day continuous) and added cat to global objects.track → [person, car, cat] (coco-80 model, cat valid). docker restart frigate; verified /api/config shows the new track list + record.motion.days=10, and 13 segments/camera landed in /srv/frigate/recordings within 3 min. Cameras = 2 physical Tapo dual-lens on the cams VLAN (.119=cam1/2, .139=cam3/4); detect from camN_sub, record from camN_main via go2rtc restream rtsp://127.0.0.1:8554. The “cat outside at 03:00” clip never existed (cat untracked + no continuous/motion retention); going forward motion + cat tracking captures such events. Benign loose end: /home/levander/nvr/frigate/go2rtc_homekit.yml is a 0-byte extra go2rtc config path (source of stray c302/c302_h264 streams) — harmless, removal candidate. Closed the URGENT/danger item in telep-mainframe-handover. — 2026-08-22-frigate-recording-retention-config-not-jam

2026-08-19

• 📋 Consolidated the telep-mainframe-handover to current state (2026-08-19) — folded the 2026-08-15→19 work into an accurate status table + a top-of-doc open-items checklist. Added current-state records for services with no dedicated note yet: dnsmon (FastAPI+SQLite DNS-monitor + gambling watchlist, dnsmon.service 127.0.0.1:8099, ssh-tails the router dnsmasq log, 48 h retention, own tailnet node dnsmon.taild4189d.ts.net); the router casino-alert.sh procd alerter now emitting device-name + context (real fires: bet365 from iPhone .146, adpool.bet); Bambuddy 3D-print control plane (own node bambuddy.taild4189d.ts.net, printer LAN .202, slicer = Bambu Studio not OrcaSlicer); home.telep.lan dashboard; WiFi SSID split (telep1=5 GHz / telep1-2G=2.4 GHz, cams telep-cc); new wired AP bandi (.101); the /etc/cams-guard.sh minute-cron restoring br-cams; the Caddy :443 boot-race hardening; telep-selftest as the OliveTin status backend (+ Go-template {{ }} gotcha, router-via-LAN-.1 gotcha, ONVIF-needs-venv-python gotcha); and the YubiKey-FDE current state (serial 32875420, live-lock, SPOF, ~/telep/*postyk.img backups). New open items: backup YubiKey, encrypt sda+Windows NVMe, watch.levandor.io passkey, unify watchlist, reserve .202, per-device (not LAN-wide) Private-Relay handling (LAN-wide block was reverted). Cross-linked the separate aposemati project. — telep-mainframe-handover

2026-08-17

• 🔴 ⭐ CRITICAL outage found + fixed — Caddy :443 BOOT-RACE took ALL domain access down for ~1.6 days on telep-mainframe. Discovered while wiring the new home dashboard into Caddy: caddy.service had been failed since 2026-08-15 18:44 (the YubiKey-FDE reboots) with listening on 192.168.1.123:443: bind: cannot assign requested address. ROOT CAUSE: Caddy starts at boot before the LAN IP 192.168.1.123 is assigned to enp5s0, fails to bind :443, and does NOT retry → stayed dead. IMPACT: every *.telep.lan service AND both levandor.io portals (admin/id) were unreachable-by-domain the whole time (unnoticed because the YubiKey work was all console/ssh). The bind 192.168.1.123 on the :443 blocks (needed to avoid colliding with Tailscale’s 100.115.209.87:443) is what makes it sensitive to the IP not being up yet. ✅ FIX: (a) net.ipv4.ip_nonlocal_bind=1 persisted in /etc/sysctl.d/99-caddy-nonlocal-bind.conf (the real fix — lets Caddy bind an address before it’s assigned); (b) systemd drop-in /etc/systemd/system/caddy.service.d/resilience.conf with Restart=on-failure, RestartSec=5, Wants=/After=network-online.target (belt-and-suspenders retry). After systemctl start caddy: home.telep.lan→200, admin.levandor.io→302. Also restarted oauth2-proxy (admin.levandor.io was 502 — that container was down too). Documented in Caddy boot-race outage (2026-08-17) + banner in Caddy :443 boot-race + new runbook caddy-boot-race (added to runbooks-index + cross-linked from service-unreachable). — caddy-boot-race
• ⭐ NEW home.telep.lan “where is what” live-status service dashboard — one URL to find every service (and its fallback address) during an outage. Tech: gethomepage/homepage (Next.js) in Docker at /home/levander/homepage/ (docker-compose.yml + config/{settings,services,widgets,bookmarks}.yaml), network_mode: host + HOSTNAME=127.0.0.1 PORT=3010 (binds 127.0.0.1:3010; 3000 was taken by ruview). Host networking REQUIRED so siteMonitor: health checks reach the 127.0.0.1:PORT + LAN/VLAN IPs directly. Must set HOMEPAGE_ALLOWED_HOSTS (recent Homepage rejects unknown Host headers). REACHABLE TWO WAYS (redundant on purpose): (a) http://home.telep.lan via the :80 Caddy — replaced the old static landing block with reverse_proxy 127.0.0.1:3010 (backup /etc/caddy/Caddyfile.bak-home-dash; router already resolves *.telep.lan→.123); (b) https://telep-mainframe.taild4189d.ts.net:8450 via tailscale serve --https=8450 — deliberately Caddy-independent so the board survives a Caddy outage (which is exactly what it surfaced). LAYOUT: 18 tiles / 6 Hungarian groups (Kamerák & NVR, Admin & Hozzáférés, Tárolás & Tudás, Eszközök & Média, SDR/Kraken, Hálózati eszközök); health = siteMonitor: per tile except cameras (ping:); Jellyfin OMITTED (backend dead on :8096). OUTAGE-SAFE: no icon: fields (no CDN dependency), only local widgets (datetime+resources), fallback IPs/ports in each tile’s description + a ”⚡ Kiesés esetén” bookmarks group (tailnet ports, raw IPs, ssh targets). 🐛 GOTCHA: Homepage services.yaml is YAML — a description: value with a colon-SPACE (e.g. Windows C: (RO)) breaks parsing (bad indentation of a mapping entry) → QUOTE all description values. Added to Other services on this box. — 2026-08-17-home-dashboard

2026-08-15

• 📶 NEW device — TP-Link TL-WA850RE (2.4 GHz N300, single-band) reconfigured from repeater → ACCESS POINT, wired into a telep-router br-lan LAN port. Broadcasts SSID bandi (resolves the earlier mystery: bandi on 2.4 GHz ch2 in scans was THIS device, not a neighbour). WiFi BSSID ac:84:c6:1b:04:31, eth/mgmt MAC ac:84:c6:1b:04:30; static DHCP reservation dhcp.@host[3] name=TL-WA850RE ip=192.168.1.101 dns=1, dashboard http://192.168.1.101. 100 Mb Fast-Ethernet link + N300 ceiling = coverage AP, not a fast-lane AP. GOTCHAS: (1) in repeater mode it had NO router IP (transparent default 192.168.0.254) so the dashboard was unreachable from a 192.168.1.x client — reach via Tether app or a temporary 192.168.0.x static IP → http://192.168.0.254; (2) first cable “on port 4” showed carrier=0 and the MAC wasn’t learned on any wired port (silently still on wireless backhaul) — a different cable fixed it, then it pulled the lease → verify wired with /sys/class/net/lanN/carrier + brctl showmacs br-lan | grep <mac>; (3) reservation added with a uci show dhcp | grep <mac/ip> pre-check + dnsmasq verified running afterwards, per the duplicate-reservation outage lesson. Updated [[telep-router#tp-link-tl-wa850re-added-as-a-wired-ap-ssid-bandi|TP-Link TL-WA850RE added as a wired AP (SSID bandi)]] + DHCP reservations note. — telep-router
• 🔒 ⭐ MAJOR security change — root LUKS FDE converted from clevis/TPM2 auto-unlock to a YubiKey-gated unlock on telep-mainframe. GOAL (user): “when I unplug my YubiKey nothing can be read.” FINAL MODEL: YubiKey 5C NFC (fw 5.7.4, serial 32875420, USB id 1050:0407 after enabling OTP) lives plugged in → box auto-unlocks unattended (survives mains cuts, no prompt); pull the key → udev syncs + systemctl poweroff; boot without it → passphrase prompt. 🔴 WHY NOT FIDO2: Debian initramfs-tools ignores systemd-cryptenroll FIDO2/TPM2 tokens (same reason clevis was used for TPM) → unlock is HMAC-SHA1 challenge-response (OTP slot 2, no-touch, randomly-generated secret stays in the key) via a custom initramfs keyscript /usr/local/sbin/yk-keyscript.sh (reads /etc/ykluks/root.challenge, modprobe usbhid, retries ykchalresp -2 -x up to 10s → 40 hex to stdout; nothing if key absent → passphrase fallback; logs /dev/kmsg) + hook /etc/initramfs-tools/hooks/yk-unlock (copy_exec ykchalresp + keyscript, manual_add_modules usbhid, copies challenge in) + crypttab ,keyscript=… (backup /etc/crypttab.bak-preyk) + update-initramfs -u. VOLUMES (resolve by UUID — NVMe nodes drift): root LUKS2 /dev/nvme1n1p3 UUID a3a8e37d-79fa-484b-bc3f-40c56df95337 (mapper nvme1n1p3_crypt, VG telep-mainframe-vg/LV root); cam-archive /dev/sdb1 UUID 7af64460-…. ROOT KEYSLOTS NOW: 0 = passphrase (break-glass), 2 = YubiKey; NO slot 1, NO TPM token. 🔴 ENROLL GOTCHA: piping the new key to cryptsetup luksAddKey DEV - FAILS (No key available with this passphrase) — the pipe occupies stdin so cryptsetup reads the piped key as the EXISTING passphrase → use a /dev/shm keyfile so the terminal stays free for the passphrase prompt (root enroll REQUIRED a human at a real TTY; agent can’t). DE-RISK METHOD (reusable): before removing TPM, ran an init-premount probe running ykchalresp in the REAL initramfs logging to /dev/kmsg (dmesg|grep YK-PROBE-RESULT → OK len=40 after 0s) while clevis still unlocked, + verified the keyscript output opens root via cryptsetup luksOpen --test-passphrase --key-file - → only then clevis luks unbind -d /dev/nvme1n1p3 -s 1 -f (harmless “Nothing to read on input”, verify slot1 gone/Tokens empty). LIVE-LOCK: /etc/udev/rules.d/99-yk-lock.rules (ACTION=="remove"…ENV{PRODUCT}=="1050/407/*" RUN+=/usr/local/sbin/yk-removed-lock.sh = logger+sync+systemctl --no-block poweroff; clean poweroff NOT a hard sysrq cut — avoids the NVMe-damage-from-power-loss issue), verified with a temporary log-only test rule (no poweroff needed). ⚠️ CAVEATS: (1) ykman config re-enumerates the key = a udev remove = poweroff → disarm 99-yk-lock.rules first; (2) SINGLE POINT OF FAILURE — only ONE key enrolled (OPEN: add a backup); (3) whole-box-theft still boots WITH the key (deliberate — user chose possession-only auto-boot over a PIN); (4) ykman needs pcscd+root on this key; (5) no dropbear → lockout recovery = passphrase at physical console or restore the post-YK header backup (~/telep/root-nvme1n1p3-20260815-postyk.img / camarchive-sdb1-…-postyk.img on the Mac, 600). 🔴 The pre-YK header backup CONTAINED the clevis TPM token → SHREDDED from Mac+box (restoring it would re-enable TPM bypass). Rebooted → YK-KEYSCRIPT: root key from YubiKey ok (0s), zero clevis tokens. Superseded 2026-07-24-luks-tpm-autounlock (banner + status/outdated); updated Boot & reboot safety, telep-mainframe-handover, new runbook yubikey-luks-lockout in runbooks-index. — 2026-08-15-yubikey-gated-luks-fde
• 🔴 ⭐ CAMWALL DEAD / all 4 cameras fps=0 root-caused to the cams-VLAN bridge br-cams stuck DOWN on telep-router after a wifi reload — NOT the cameras. TRIGGER: during WiFi tuning a 2.4 GHz channel change uci set wireless.radio1.channel=11; uci commit; wifi reload bounced the cams AP phy1-ap1 (SSID telep-cc, network=cams, isolate=1, hidden=1). 🔴 ROOT CAUSE: br-cams’ ONLY member is that wireless AP, so a runtime wifi reload dropped the wireless-only bridge and OpenWrt left it state DOWN, qdisc noop with NO inet — the static L3 config (network.cams proto static 192.168.30.1/24 on br-cams) was NOT reapplied and does not auto-recover. SYMPTOM CHAIN: cams gateway/DHCP 192.168.30.1 gone → both Tapo cams (DÉL .119=18:69:45:a9:01:25=cam1/2; ÉSZAK .139=c0:3a:55:5c:8b:33=cam3/4) stayed WiFi-ASSOCIATED (in iwinfo assoclist, good signal) but L3-UNREACHABLE (no ping even from the router, no ARP/neigh, .119 had no lease) → go2rtc producers=None, Frigate camera_fps=0 on all 4, the fps-watchdog auto-restarted Frigate (~18min uptime) but that did NOT help (break is network-side), camwall (4× mpv on HDMI) froze. 🎣 MISLEADING: ONVIF reboot ~/tapo-ctl/reboot-cams-onvif.py ConnectTimeout’d port 2020 on BOTH cams — only because they were L3-unreachable, NOT bad cameras → do NOT chase cameras, check ip addr show br-cams FIRST. DIAGNOSIS THAT WORKED: docker exec frigate curl 127.0.0.1:1984/api/streams (producers None) → .../5000/api/stats (camera_fps=0) → ping cams from router (down) → grep 192.168.30 /tmp/dhcp.leases (.119 missing) → ip neigh | grep 192.168.30 (empty) → ip addr show br-cams (state DOWN qdisc noop, no inet = ROOT CAUSE). ✅ FIX: ifup cams alone did NOT work, nor /etc/init.d/network reload; wifi up; what worked = forced down/up cycle on the router ifdown cams; sleep 2; ip link set dev br-cams up; ifup cams; sleep 5 → br-cams UP with inet 192.168.30.1/24, phy1-ap1 master br-cams state UP, cams pingable; then mainframe sudo systemctl restart camwall → camera_fps ~5.0 on all 4, mpv tiles re-pulled, wall live. ⚠️ SAME fps=0 SYMPTOM as the 2026-08-03 wedged-RTSP gotcha but DIFFERENT layer — that was camera-side (fix=ONVIF reboot+camwall restart, cams pingable); THIS is router-side bridge-down (fix=ifdown/ip link up/ifup cams, cams NOT pingable) → runbook must branch on ip addr show br-cams. DURABLE GUARDS (recorded, NOT applied, user to decide): (a) hotplug hook auto-ifup cams after wifi events (recommended); (b) persistent dummy member on br-cams so a wireless-only bridge never goes carrier-down; (c) rule: never wifi reload radio1 without the cams down/up cycle. 🐛 SEPARATE BUG FIXED: OliveTin action “Kamerák újraindítása (ONVIF)” (+ raw script) ran system python3 which lacks the onvif module (lives only in ~/tapo-ctl/venv) → fixed /home/levander/admin-portal/olivetin/config/config.yaml to call ~/tapo-ctl/venv/bin/python ~/tapo-ctl/reboot-cams-onvif.py; RULE = all ~/tapo-ctl/ scripts run via the venv python, not system python3. Updated telep-router (new br-cams outage section + open item), 2026-08-15-router-wifi-tuning-htmode-band-penetration, 2026-08-03-cam-stall-recovery-and-casino-alert, 2026-08-15-admin-portal-passkey-olivetin, runbook camwall-not-on-tv (new fps=0/br-cams branch). — 2026-08-15-camwall-dead-br-cams-bridge-down-after-wifi-reload
• ⭐ OliveTin status actions REFACTORED to a single telep-selftest script — logic pulled OUT of the YAML, fixes the “Szolgáltatások” action, adds a MagicDNS/alerts canary. 🔴 ROOT PROBLEM: OliveTin runs every action’s shell: string through Go text/template BEFORE executing, so any literal {{ }} (e.g. docker ps --format "{{.Names}} {{.Status}}") is read as an OliveTin template var → Error executing template ... can't evaluate field Names in type *tpl.actionTemplateContext (this is what broke “Szolgáltatások”). GOTCHA: avoid {{ }} in OliveTin shell: commands (or escape as Go-template literals), OR keep logic out of YAML entirely. ✅ FIX/REDESIGN: one script at /usr/local/bin/telep-selftest on telep-mainframe (192.168.1.123) printing clean ✓/✗/⚠ colored output, sections dns|airplay|camwall|services|cameras|health|alerts|all, usage telep-selftest [section|all]; OliveTin status actions now just ssh 192.168.1.123 'telep-selftest <section>' (DRY, no quoting/template traps). Added actions “Teljes önteszt” (all) + “Riasztások (Telegram)“. The alerts section is a CANARY for the MagicDNS→container-DNS outage: checks frigate + frigate-notify up, api.telegram.org REACHABLE FROM INSIDE the frigate container (docker exec frigate curl https://api.telegram.org), last “Alert sent” ts, and counts misbehaving DNS errors in the last 30 min. 🔴 GOTCHA: the script runs ON the mainframe so its router DNS check must ssh the router via LAN IP 192.168.1.1 NOT tailscale 100.69.112.32 — the mainframe cannot reach the router’s tailnet IP (ping 100.69.112.32 times out) but 192.168.1.1 works; asymmetric with OliveTin’s OWN container which DOES reach the router on 100.69.112.32 for its fix/reboot actions. Config at /home/levander/admin-portal/olivetin/config/config.yaml; container already user: root with writable /root/.ssh. Updated 2026-08-15-admin-portal-passkey-olivetin. — 2026-08-15-admin-portal-passkey-olivetin
• ⭐ Router WiFi tuning — TX power is MAXED (not a lever), narrowed 5 GHz HE80→HE40 for range/wall penetration. Complaint: weak signal, poor wall penetration; UI shows “max”. CONFIRMED both radios already at the HU/ETSI regulatory ceiling: 5 GHz radio0 (ch36, was HE80) = 23 dBm, 2.4 GHz radio1 (ch1 HE20) = 20 dBm; HW max 30 dBm but only on DFS ch 100–140 → TX power can’t go higher. CLIENT SURVEY: 5 clients on 5 GHz, two at −83/−85 dBm (dying through walls); 2.4 GHz nearly empty except the two Tapo cams (Dél .119 @ −42 dBm good; Észak .139 @ −67 dBm — the weak link that drops its RTSP). telep1 is dual-band (same SSID both radios → band roaming possible); telep-cc is the 2.4 GHz cam SSID. ✅ CHANGE: uci set wireless.radio0.htmode='HE40' (was HE80) + uci commit wireless; wifi reload → concentrates same power into half the bandwidth ≈ +3 dB range/penetration, peak 1200→600 Mbps, reversible (set back to HE80). RECS (not applied): (a) band steering via dawn package so far devices move to 2.4 GHz = biggest win (5 GHz can’t penetrate walls); (b) optionally HE20 on 5 GHz for max range (~300 Mbps); (c) keep 2.4 GHz at 20 MHz (never 40); (d) real dead-zone fix = 2nd AP/mesh node with wired backhaul — no single-router setting beats RF physics. Updated WiFi. — 2026-08-15-router-wifi-tuning-htmode-band-penetration
• 🔴 ⭐ POST-MORTEM: Frigate camera Telegram alerts were SILENTLY DEAD ~17 days (last Alert sent 2026-07-29 13:01:15 → fixed 2026-08-15) — Tailscale MagicDNS broke Docker container external DNS on telep-mainframe. 🎣 RED HERRING: frigate-notify logs flooded with Event dropped - Already notified on this zone (empty zones=) + Review dropped - No events eligible — looks like a dedup/zones bug but is NOT. frigate-notify (events/reviews.go+filters.go, otter cache in events/cache.go keyed by event ID, 1 h TTL) caches an event during filtering BEFORE the Telegram send; a FAILED send leaves it cached so every 15 s poll re-logs “Already notified” → a failed send masquerades as dedup. 🔴 REAL ROOT CAUSE: host /etc/resolv.conf = nameserver 100.100.100.100 (Tailscale MagicDNS) + search taild4189d.ts.net local lan; Docker’s embedded DNS (127.0.0.11) forwards container external lookups to that upstream and MagicDNS misbehaves for containers → frigate-notify couldn’t resolve api.telegram.org (buried WRN only via docker logs --tail 4000 frigate-notify | grep WRN: ... server misbehaving). PROOF: docker exec frigate curl https://api.telegram.org=http 000 while the SAME curl from the host = ok (tailscaled intercepts .100.100 for the host, a container forwarding to it fails). Something enabled --accept-dns/MagicDNS on 2026-07-29 (matches the outage start). Everything else HEALTHY: bot getMe/getChat from host ok (still in group -1004475187307), snapshots ?bbox=1=200, labels/zones config fine. ✅ FIX: added dns: [192.168.1.1 (router dnsmasq), 1.1.1.1] to the frigate-notify service in /home/levander/nvr/docker-compose.yml (backup docker-compose.yml.bak-dns) + docker compose up -d --force-recreate frigate-notify → INF Alert sent provider=Telegram fired instantly for cam1+cam2, zero server misbehaving since. Internal frigate service-name resolution unaffected. GOTCHAS: (1) ANY container needing EXTERNAL DNS on this host silently fails while the resolver is MagicDNS — frigate-notify bit us; other containers mostly need only internal/service-name resolution; host systemd services (intruder-alarm.py) use the host resolver and are fine; (2) durable fix "dns":["192.168.1.1","1.1.1.1"] in /etc/docker/daemon.json DEFERRED (restarts ALL containers); (3) “Already notified” flood + zero “Alert sent” ⇒ grep WRN for server misbehaving, don’t chase zones. SCOPE: alerts wanted only on DÉL .119 (telep_cam1/2); ÉSZAK .139 (telep_cam3/4) excluded on purpose (2026-07-28-frigate-notify-camera-exclude); Xiaomi C302 .168 REMOVED (dead IP — corrected 2026-08-09-xiaomi-c302-tailnet-facetime-cam to status/outdated; it was the only cam ever on the separate go2rtc-cam path, Tapos go direct RTSP into Frigate). MINOR: OliveTin “Kamerák” action was pinging dead .168 → corrected to .119/.139. Updated Alerting; new runbook camera-alerts-not-firing added to runbooks-index. — 2026-08-15-camera-alerts-dead-tailscale-magicdns-docker-dns
• ⭐ Admin portal admin.levandor.io NOW WORKING — iPhone-only passkey-gated control panel documented (architecture + the 2 gotchas that ate the most time). All on telep-mainframe (192.168.1.123), docker-compose stack at /home/levander/admin-portal/. Chain: Caddy (host, TLS via Cloudflare DNS-01) → oauth2-proxy (OIDC, PROXY mode) → OliveTin (Hungarian YAML actions that ssh into hosts). IdP = Pocket-ID v2.13 at id.levandor.io (passkey OIDC, SQLite pocket-id/data/pocket-id.db, client “Mainframe fallback” fbd09167-4f21-4277-80e7-b635f35c2861). 🔴 GOTCHA 1 — login HTTP 500 email in id_token isn't verified: a self-hosted Pocket-ID account has email_verified=0 and oauth2-proxy rejects unverified emails by default → FIX OAUTH2_PROXY_INSECURE_OIDC_ALLOW_UNVERIFIED_EMAIL=true + recreate. 🔴 GOTCHA 2 — OliveTin ssh actions Host key verification failed/Permission denied (LOOKED like a stale host key): jamesread/olivetin image runs as USER olivetin (HOME /home/olivetin) but key+known_hosts mounted at /root/.ssh → ssh found no identity/known_hosts → FIX add user: root to the service AND drop :ro on the ssh mount (so accept-new persists). Other load-bearing config: oauth2-proxy upstream MUST be compose service name http://olivetin:1337 (NOT 127.0.0.1:1337 = its own loopback → 502); OAUTH2_PROXY_PROMPT=login (empty→force→invalid_request); OAUTH2_PROXY_CODE_CHALLENGE_METHOD=S256 (Pocket-ID requires PKCE). Caddy admin.levandor.io block MUST bind 192.168.1.123 (avoids :443 collision with Tailscale 100.115.209.87:443), multi-line tls { dns cloudflare {env.CF_API_TOKEN} } (single-line won’t parse), CF token in /etc/caddy/cloudflare.env via systemd drop-in, systemctl restart (not reload) after cert/env changes. Actions ssh to mainframe .123, router root@100.69.112.32, pi/kraken .200; pubkey olivetin/ssh/id_ed25519.pub already authorized on mainframe+router. OPEN: authorize pubkey on the Pi when the Kraken rig is back (Pi offline, No route to host); oauth2-proxy --trusted-proxy-ip not set (trusts X-Forwarded-* from any IP, low risk — only Caddy reaches it); remote/off-LAN access deferred (Caddy binds LAN IP only). New runbook admin-portal-login-broken added to runbooks-index. — 2026-08-15-admin-portal-passkey-olivetin
• 📓 NEW operational RUNBOOKS set — action-first “when X breaks, do Y” guides in projects/homelab/runbooks/. Six symptom→fix runbooks (symptoms → numbered copy-paste diagnosis/fix → deeper-causes links back to the detailed incident notes; don’t duplicate their depth): no-wifi-or-dhcp (whole network down → #1 cause dnsmasq refused to start on telep-router from a duplicate dhcp-host), service-unreachable (*.telep.lan/Caddy + Tailscale paths, name→port map, curl -H Host for 502s), host-offline (mainframe/Pi dropped off → usually a lost DHCP lease; recovery over Tailscale), airplay-telep-tv (telep-tv not appearing/won’t cast/no video/no audio; netifd AP-isolation patch, plughw:1,3), camwall-not-on-tv (camwall-x Xorg :0 vs camwall grid, /run/uxplay/casting, ONVIF cam reboot), krakensdr-df (Overdrive=RF over-gain not PSU, ghost bearings=array mismatch, DoA/PR both bind :8080). Recovery access baked into the index runbooks-index (router ssh root@100.69.112.32, mainframe ssh levander@100.115.209.87; mainframe .123/enp5s0, Pi .200, router .1, cameras 192.168.30.x). Linked from telep-mainframe-handover + TOPICS (new “Runbooks & Incident Response” topic). — runbooks-index
• 🔴 ⭐ POST-MORTEM: total site-wide DHCP+DNS outage (~8.5 h, mainframe + ~a dozen tailnet devices offline) caused by a DUPLICATE dhcp-host for IP 192.168.1.123 on telep-router. While adding a reservation for the AirPlay receiver, a second named section dhcp.telep_tv was created with the SAME mac (d8:5e:d3:a7:05:d6) + SAME ip (.123) as the already-existing canonical dhcp.@host[2] (telep-mainframe). 🔴 dnsmasq does NOT ignore a duplicate dhcp-host IP — it’s a FATAL error (duplicate dhcp-host IP address 192.168.1.123 … FAILED to start up) so dnsmasq refused to start at all → DHCP(:67) AND DNS(:53) dead network-wide, no client could get/renew a lease, the mainframe lost .123 and dropped off Tailscale. FIX: uci delete dhcp.telep_tv; uci commit dhcp; /etc/init.d/dnsmasq restart → back in <1 min. ⚠️ LESSON: before adding any dhcp-host, uci show dhcp | grep <ip-or-mac> first; the telep-mainframe reservation already covers that MAC/IP so a separate telep-tv reservation is MOOT (and is exactly what caused this). Recovery access when LAN is down: router ssh root@100.69.112.32, mainframe ssh levander@100.115.209.87; mainframe LAN=.123/enp5s0, Pi=.200 (don’t confuse). — 2026-08-15-dhcp-outage-duplicate-reservation-postmortem
• ⭐ NEW *.telep.lan split-horizon local access — internet-independent path to every homelab service (LAN wildcard DNS + Caddy reverse proxy), Tailscale kept for remote. Motivation: Tailscale-only exposure vanishes with no internet (and some services were bound 127.0.0.1+tailscale-serve only). LOCAL DNS on telep-router: uci add_list dhcp.@dnsmasq[0].address='/telep.lan/192.168.1.123' (committed) → every *.telep.lan resolves to the mainframe; LAN clients get it automatically (router = their DHCP DNS). AAAA is NXDOMAIN (cosmetic; A works). REVERSE PROXY on telep-mainframe: Caddy v2.11.4 from the official apt repo (⚠️ cloudsmith config.deb.txt produced a broken any-distro/any-version URL on trixie → corrected to .../deb/debian any-version main), /etc/caddy/Caddyfile, listens *:80 with auto_https off (never tries LE for the non-public .lan); tailscale serve (9 proxies) + all 7 docker containers left untouched. Name→backend (HTTP :80, all 200 unless noted): telep.lan/home→inline HTML, files→:8334, frigate→https :8971 (tls_insecure_skip_verify), kraken→192.168.1.200:8080 (502 while Pi offline, entry correct), top→:8090, print→:8095, whiteboard→:8790, ruview→:3000 (needed header_up Host {upstream_hostport}, else 421), go2rtc→:1984, jellyfin→:8096 (guessed name, backend down — flagged for removal/rename). Deliberately NOT proxied: stirling-pdf 8080, qdrant 6333/6334, knowledgebase 8092, home-portal 8093, ruview-alt 3001, frigate-unauth 5000. — 2026-08-15-telep-lan-split-horizon-caddy

2026-08-12

• ⭐ AirPlay “broken” open item RESOLVED — root-caused to OpenWrt auto-injecting ap_isolate=1 into the generated hostapd conf, fixed by patching netifd (the old uci isolate=0 recipe was WRONG). Plus printer USB→WiFi and wifi-hunter AP killed. 🔴 THE GOTCHA: on telep-router (OpenWrt 24.10.0) the telep1 client BSSes (5 GHz telep1_5/radio0 + 2.4 GHz telep1_2/radio1) had ap_isolate=1 in the RUNNING hostapd conf blocking wired→WiFi mDNS (AirPlay discovery of telep-tv) — yet uci show wireless said isolate=0 and ubus call network.wireless status said isolate:false. ⚠️ The parked handover fix (uci set wireless.telep1_{5,2}.isolate='0' + wifi reload) is TRIED-AND-INSUFFICIENT — it does NOT clear it; neither does wifi down; wifi up nor network reload (/var/run/hostapd-phy*.conf stays byte-identical with ap_isolate=1). ROOT CAUSE: /lib/netifd/netifd-wireless.sh fn _wireless_set_brsnoop_isolation (~line 309: [ ${multicast_to_unicast:-1} -gt 0 -o ${proxy_arp:-0} -gt 0 ] && json_add_boolean isolate 1) auto-injects isolate 1 for any BRIDGED, NON-ISOLATED AP when multicast_to_unicast is on — and multicast_to_unicast is promoted to the interface top-level as 1 (effective default), so isolate=1 gets injected transiently at config-gen time → hostapd.sh writes ap_isolate=1. The wifi-iface multicast_to_unicast=0 uci option does NOT propagate to the top-level value for_each_interface reads (line 327, before json_select config); changing the line-309 default :-1→:-0 ALSO didn’t help (value is explicitly promoted as 1, not merely unset). ✅ FIX THAT WORKED: commented out line 309 (sed -i '309s/^/#DISABLED-airplay-fix#/' …, backup /lib/netifd/netifd-wireless.sh.bak) + wifi down; wifi up → telep1 both bands now have NO ap_isolate line (mDNS/AirPlay flows), while telepcc (cams, SSID telep-cc) KEEPS ap_isolate=1 because its explicit isolate=1 early-returns at line 308 before reaching the disabled 309. 🔴 CRITICAL CAVEAT: this patches a /lib/netifd/ system file → an OpenWrt sysupgrade REVERTS it and AirPlay breaks again; reapply by grepping json_add_boolean isolate 1 (line numbers shift), re-sed, wifi down; wifi up. Also committed (harmless, not sufficient alone): uci set wireless.telep1_{5,2}.isolate=0 + .multicast_to_unicast=0. 🖨️ PRINTER (HP LaserJet M203dw): Ethernet NIC confirmed HARDWARE-DEAD from the surge (serial VNC3920651, USB 03f0:632a) → set up on USB-to-CUPS (queue HP_M203dw_USB, system default, device usb://HP/LaserJet%20M203-M206?serial=VNC3920651, reused PPD /etc/cups/ppd/HP_LaserJet_M203dw_B8AE8C.ppd; had to unload+blacklist usblp via /etc/modprobe.d/blacklist-usblp.conf so the CUPS libusb backend could claim it; installed hplip — hp-wificonfig gone, hp-probe needs root) → then user needed the USB cable back so MOVED TO WIFI (M203dw has WiFi) via the HP Smart iOS app (Wi-Fi Direct pw 12345678, EWS 192.168.223.1 no admin pw). STILL PENDING: once WiFi IP known → repoint CUPS to a network queue, add a DHCP reservation on telep-router, remove the USB queue + two stale dead-NIC queues (HP_LaserJet_M203dw_B8AE8C and …@NPIB8AE8C.local). 📶 WIFI-HUNTER: on raspi brought down the wlan0 NetworkManager “Hotspot” AP (sudo nmcli connection down Hotspot + sudo nmcli con mod Hotspot connection.autoconnect no) → wlan0 now free for the future car-deploy client link; note this stops wifi-hunter’s AP. — 2026-08-12-airplay-mdns-fix-printer-migration
• ⭐ New KrakenSDR direction-finding (DoA) rig built end-to-end on a Raspberry Pi 5 (raspi/192.168.1.200, Tailscale node kraken-rig, tag:telep) for mobile TETRA (380–385 MHz) DF — DoA autostarts on boot and the full chain is verified with a real transmitter (420 MHz TX → clean MUSIC peak, bearing ~294°). Same physical Pi as wifi-hunter (the wlan0 NetworkManager “Hotspot” AP is wifi-hunter’s, not a firewall problem). HARDWARE: KrakenSDR = exactly 5× RTL2838 behind an internal USB hub, NO separate serial/CH340 (noise-source cal is over tuner GPIO via the krakenrf librtlsdr fork); needs its own USB-C power; “Power Level: Overdrive” = RF over-gain/ADC clipping, NOT a weak PSU (a weak PSU shows as USB dropouts in dmesg). SOFTWARE (all built from source under /home/levander, Miniforge conda env kraken/py3.9): librtlsdr krakenrf fork REQUIRED (not distro), Ne10, heimdall_daq_fw, krakensdr_doa, krakensdr_pr; ⚠️ krakenrf/krakensdr_pr is 404 (org moved to krakensdr_suite, DoA+spectrum only, no PR) → PR installed from mirror mfkiwl/krakensdr_pr; trixie/py3.9/aarch64 pins (flask 2.0.3, werkzeug 2.0.2, dash 1.20.0, dash_bootstrap_components 0.13.1 not the doc’s 1.1.0, dash_devices the ws dash fork imported AS dash); DVB drivers blacklisted in /etc/modprobe.d/blacklist-rtlsdr.conf; ports DoA/PR UI :8080, php data-out :8081, node backend :8042 (DoA & PR both bind :8080 — never both at once); miniserve uninstallable (distro rustc too old, optional). 🔴 CRITICAL GOTCHA: start scripts do conda activate kraken which SILENTLY FAILS over non-interactive SSH/systemd → app.py runs under system python → ModuleNotFoundError: dash_devices; FIX = always source /home/levander/miniforge3/etc/profile.d/conda.sh && conda activate kraken first; also clean stale /dev/shm decimator/delay_sync buffers if the DAQ won’t sync (“Shared memory not exist”). AUTOSTART: /usr/local/bin/kraken-mode {doa|pr|off} (sources conda, stops both then starts one) + systemd krakensdr.service (oneshot, RemainAfterExit, User=levander, ExecStart=kraken-mode doa) enabled+active, verified HTTP 200 on :8080; exit 255 over interactive SSH is a harmless TTY artifact. ARRAY/DoA: UCA, radius 0.30 m (0.45 λ @382 MHz), MUSIC, Compass, Optimize Short Bursts ON; physical recipe = regular pentagon, 30 cm radius (~35 cm spacing), 5× ~20 cm ¼-wave whips, equal coax, element #0 = reference; ⚠️ spacing ≤ 0.5 λ or ghost bearings → 35 cm gives an ambiguity ceiling ~428 MHz (don’t test above it); software config must MATCH the physical build (cal fixes electronics phase, not geometry); bearings relative to element #0 until Array Offset is calibrated. OPS: 380–385 = TETRA bursty uplink, 🚫 do NOT TX on 380–385 (emergency band) — test below 428 MHz and retune (whip-length mismatch is common-mode, doesn’t hurt bearings); RTL-SDR shows a fake DC spike at center freq → offset-tune. REMOTE: UI at http://kraken-rig.taild4189d.ts.net:8080/doa — the :8080 is REQUIRED (without it → port 80 → refused; looked like an ACL problem, wasn’t); Pi net = eth0 home LAN + wlan0 wifi-hunter AP (single radio). OPEN: build the physical array + calibrate Array Offset; car deploy (wlan0 AP→client on the driver’s hotspot, needs SSID+password preloaded); GPS deferred (bearing-only). — 2026-08-12-krakensdr-doa-rig
• **Session handover for 2026-08-12 — single entry point tying together: top-kép voting LIVE (replaces the Drive workflow; mobile-first UI, tap-to-zoom lightbox, Person/Car filter, on-demand HD button, verified in a real browser), Frigate now recording the 2304×1296 MAIN streams (detect still 720p subs; ⚠️ 4 RTSP pulls/cam), camwall hardened (per-pane freeze watchdog on all 4 mpv sockets + HDMI 4K mode-enforce + resolution auto-heal), cameras rebooted/healthy over ONVIF (pytapo creds dead) with OSD re-sync, and two PARKED needs-you items: AirPlay (approve clearing ap_isolate on the telep1 Wi-Fi via uci set wireless.telep1_{5,2}.isolate='0' on the router — ~5–10s Wi-Fi bounce) and the printer (M203dw Ethernet NIC dead post-outage — link up, zero frames across cable/port/power-cycle swaps → plan is USB-to-CUPS). Doesn’t re-derive; summarises + links the dated notes. — 2026-08-12-session-handover

2026-08-11

• ⭐ RuView UNBLOCKED — real ESP32-S3 WiFi CSI flowing end-to-end into the ruview container, on its own tailnet node; supersedes the “STOPPED, blocked on hardware” state. 🔴 THE BLOCKER was never the hardware alone: UDP 5005 was only EXPOSEd, never published, so no ESP32 could ever have reached the CSI ingest port. Ports can’t be added to a running container → full recreate (safe: it had NO volumes, only a regenerable 16K /app/data/session-secret) with -p 5005:5005/udp, CSI_SOURCE=esp32 (was simulated; prefer esp32 over auto — it fails loud, issue #937 removed the silent synthetic fallback), a persistent -v /home/levander/ruview-data:/app/data, and --model /app/data/models/model.rvf appended (the entrypoint PREPENDS its defaults --source/--tick-ms 100/--ui-path/--http-port 3000/--ws-port 3001/--bind-addr 0.0.0.0 whenever arg1 starts with -, so appending preserves ports/bind). ⚠️ HTTP 421 on every new hostname — DNS-rebinding Host-header validation; any new name must go in SENSING_ALLOWED_HOSTS (bit us twice: :8448, then the tailnet name). 🌐 Moved to its own node https://ruview.taild4189d.ts.net per tailnet-service-exposure-convention (second userspace tailscaled, tailscaled-ruview.service, statedir /var/lib/tailscale-ruview, --port=0, serve --bg 3000, --advertise-tags=tag:telep) — the tag matters because untagged user-owned nodes get key expiry and would silently drop off in ~6 months; VERIFIED took: tags: ['tag:telep'], keyexpiry: None, owned by the tag not by ledererandras2004@. Note tailnet-service-exposure-convention prescribes a tagged auth key instead and warns --advertise-tags can be rejected (it validates against the authenticating user’s tagOwners) — it passed here because that user owns tag:telep; still prefer the auth-key route for future nodes since it doesn’t depend on who authenticates. 🔴 FIRMWARE TRAP (the biggest one): do NOT flash the prebuilt release_bins/ on a display-less S3 (WROOM-1/DevKitC-1) — the ADR-045 runtime panel probe false-positives with no TCA9554 + floating QSPI pins (SH8601 init “succeeds”, display_is_active() true), so main.c skips the RuView#893 MGMT+DATA promiscuous upgrade and CSI yield collapses to 0 pps with no visible cause; prebuilts are also 2 versions stale (0.6.7 vs 0.8.4). FIX = build with the one-line sdkconfig.defaults.devkitc overlay (# CONFIG_DISPLAY_ENABLE is not set) → has_display constant-false; Docker espressif/idf:v5.4 is the only reliable build path; verified yield=33–37 pps. 🔌 FLASHING: board has TWO USB-C ports — the UART one (CH343 1a86:55d3, /dev/ttyUSB*/ttyACM*) flashes reliably (bridge drives DTR/RTS); native USB/OTG (303a:*) FAILED No serial data received on both --before default_reset and usb_reset; flash from Docker with --device /dev/ttyACM0 (sidesteps levander being in plugdev but NOT dialout). provision.py writes NVS so WiFi changes need no reflash (--ssid/--password/--target-ip 192.168.1.123/--target-port 5005/--node-id, plus --tdm-slot/--tdm-total; firmware already runs ESP-NOW leader election for time sync). 📶 PLACEMENT finding: moving the node from RSSI −26 → −43/−44 took confidence off a pinned 0.50 to 0.58–0.61 — too close to the AP and the direct path dominates so a human is a small fractional perturbation; target −40..−60 dBm, torso height, rigid, area-of-interest ON the AP↔node line. 🧠 MODEL ruvnet/wifi-densepose-pretrained (⚠️ container creates the models dir root-owned → chown first or curl silently writes nothing, http=200 size=0); BE HONEST: 48 KB weights / 2048 LoRA params / 12 minutes of training, and the “82.3%” headline is temporal-triplet representation accuracy NOT presence accuracy — the authors retracted an earlier “100% presence” figure as having been measured on a single-class recording; shipped node-1/2.json adapters encode the AUTHOR’s environment. ❗ NOT WORKING / open: presence/motion/RSSI/variance are real and responsive, but estimated_persons is not trustworthy (read 1 with 2 people present, later 2 and 3, ~0.5 confidence throughout); outputs can be internally contradictory (motion_level: absent + presence: true + estimated_persons: 3 in one sample); a malformed sample every ~20–40 readings has confidence ABOVE 1.0 (4.79, 12.96) and an off RSSI — likely the priority/vital-signs channel through the same parser, and it would false-trigger any alarm built on this feed; with ONE node there is NO localisation (the 3D blob is feature magnitudes, not position — needs multiple nodes + --node-positions). 🎯 --calibrate (empty room) has NEVER been run and variance never drops below ~200 in any condition — the signature of no empty-room baseline (everything above threshold → everything reads as presence); this is the prime suspect and the recommended next step. All three walk-tests were contaminated (hardware being plugged in; a second person moving during the “still” controls) → no valid controlled test exists; recommended alternative is an overnight scripts/record-csi-udp.py capture looking for diurnal structure, which needs nobody’s cooperation. 🛠 scripts/synth-csi-udp.py emits the exact 0xC511_0001 wire format and proved the whole ingest path before hardware arrived (also record-csi-udp.py, csi-udp-relay.py, collect-training-data.py, train-count.py, ruview_occ_dataset.py; server flags --calibrate, --node-positions, --train, --embed, --build-index, --convert-model). STATUS: parked (next project is servo control) — node 1 flashed/provisioned/deployed, node 2 flash FAILED (wrong USB port) and unprovisioned. Updates the RuView block in 2026-08-05-power-root-cause-nvme-damage-ups-kb-handover. — 2026-08-11-ruview-esp32-csi-real-hardware
• ⭐ Top-kép voting system build pass 2 — mobile-first UI + tap-to-zoom lightbox, person/car filter, and REAL HD (2304×1296) pics via a Frigate recording reconfig; all DONE + verified live in a headless real browser (Playwright) at phone width. 🖼️ MOBILE UI: render_grid/templates in ~/top4-web.py reworked — 16:9 cards, full-width Nap/Hét/Hónap tabs, 44px+ touch targets, responsive repeat(auto-fill,minmax(min(100%,340px),1fr)); tapping an image opens a full-screen lightbox (#lb) with the full-res snapshot (/pic/<id>?full=1), a “Nagyítás” zoom toggle (native-size scrollable + mobile pinch-zoom), a ❤ vote button, and “Bezár”. 🚗 PERSON/CAR FILTER: _frigate_events/day_candidates/votes_for_window now take a label param, routes read ?label=person|car (default person, validated vs LABELS=("person","car")); a label TEXT column added to the pics table (ALTER-TABLE migration in init_db), stored on persist + threaded through toggle_vote/_vote; a Személy/Autó bar persists the label across tiers (verified today person=189, car=36; a car vote stored pics.label='car'). 🔴 HD RECORDING (the real enabler): previously EVERY camera both detected AND recorded the 720p SUB stream so NO HD frame ever existed for past events — now each telep_camN detects on camN_sub (720p, role:detect) and RECORDS a new camN_main go2rtc stream (role:record) at 2304×1296. Lens map (still-verified): cam1 DÉL fix=.119/stream1, cam2 DÉL PTZ=.119/stream6, cam3 ÉSZAK fix=.139/stream1, cam4 ÉSZAK PTZ=.139/stream6. Config backup ~/nvr/frigate/config.yml.bak-hqrec-1786445779; detect fps unaffected (~5 on subs); storage ~3–4× recordings on a disk 6% used (3.2T free). ⚠️ CAVEAT: each Tapo now serves 4 concurrent RTSP pulls (2 lenses × sub+main) + ONVIF — watch for camera-side connection limits. ✨ “Magasabb minőség” (HD) BUTTON in the lightbox → /pic/<id>?hq=1 fetches the main-stream recording-snapshot at the event’s ts (/api/<camera>/recordings/<start_time>/snapshot.jpg, now HD), caches to /srv/top-kep/hq/<id>.jpg, serves 2304×1296; fallback chain recording-snapshot → event-clip ffmpeg frame → 720p detect snapshot (never 500s); event_id charset-validated (^[0-9.]+-[A-Za-z0-9]+$) before any subprocess/URL/path use (also closes the earlier deferred “validate event_id charset”). HD works ONLY for events captured AFTER the reconfig; older pics stay 720p. INVARIANTS held: all render_grid interpolations html.escaped (stored-XSS fix stays); GRID_HEAD/GRID_CARD .format()ed (doubled braces) but GRID_TAIL raw-concatenated (single braces) → served <script> must have ZERO {{ (the prior critical bug). Backups: top4-web.py.bak-ui-*/.bak-hq-*/.bak-lbl-*/.bak-hq2-*, top_kep_store.py.bak-lbl-*. VERIFIED live (Tailscale identity via tailscale serve): filter toggles (car page=36 cards, data-label=car), lightbox full-res, “Magasabb minőség” upgrades 1280→2304×1296 (button→“HD ✓”), vote→persist→un-vote→reclaim path. — 2026-08-10-top-kep-community-voting-system
• ⭐ Camwall “stuck cameras” root-caused to MISSING per-pane freeze detection + FIXED, plus the camera-reboot tooling was silently dead and is now working over ONVIF, plus post-reboot OSD-clock re-sync. A quadrant on the TV camwall sat FROZEN on a static image with a stopped OSD clock for a long time while Frigate ingest stayed healthy (camera_fps ~5) — so neither the Frigate FPS watchdog nor the fps side saw anything wrong. 🔴 ROOT CAUSE 1 (the real “stuck”): /usr/local/bin/camwall-mpv.sh runs FOUR independent mpv panes (cam1_sub/cam3_sub/cam4_sub/cam2_sub go2rtc substreams) but only the cam2 pane had --input-ipc-server=/run/camwall.sock, and camwall-watchdog.py checked time-pos on that ONE socket → a stalled cam1/cam3/cam4 pane (stream stops → static image, frozen clock) was UNDETECTED and never re-rolled. This is exactly the “watchdog covers only the IPC-socket-owning pane, security-relevant” regression left open in 2026-07-28-camwall-4-substream-composite. FIX: all 4 panes now get own socket /run/camwall-cam{1,2,3,4}.sock; watchdog polls time-pos on ALL FOUR with per-socket stall counters, re-rolls whole wall (systemctl restart camwall) if ANY pane hits STALL_LIMIT; HDMI-reconnect preserved (backups *.bak-1786433100). VERIFIED: pausing the cam3 pane → rerolling camwall: pane /run/camwall-cam3.sock stalled ~33s later + re-roll. Catches STALLED-STREAM freezes (no new frames), NOT repeated-frozen-frame-at-full-fps (user confirmed these are the static kind, so no content-hash needed). 🔴 ROOT CAUSE 2: ~/tapo-ctl/reboot-cam.py (pytapo, hardcoded nvr42vhy1 creds) now FAILS Invalid authentication data on BOTH cams (Tapo KLAP auth breakage) → pytapo path dead. FIX: new ~/tapo-ctl/reboot-cams-onvif.py reboots via ONVIF SystemReboot() port 2020 using FRIGATE RTSP creds (same resolution as set-cam-time.py), waits for ping-back, then re-syncs OSD clock; dead reboot-cams.py removed, old hardcoded .139-only scripts left but dead. 🔴 ROOT CAUSE 3: RTC-less Tapos lose their clock on reboot; hourly cam-timesync.timer self-heals but leaves a ~40-min wrong-OSD window → reboot-cams-onvif.py now calls set-cam-time.py on come-back, closing the gap (both cams verified correct time). This session: both cams manually rebooted (healthy after, ~5fps), camwall re-rolled. — 2026-08-11-camwall-freeze-and-cam-reboot-fixes
• ⭐ Camwall black/dead after a power outage root-caused to Xorg falling back to 640x480 on the HDMI output — self-heal added + verified by fault injection. After a power blip the TV wall “didn’t really come back” though the HOST never rebooted (on the UPS, uptime 3d17h); the power event only restarted services (Frigate, camwall-x/Xorg, camwall). 🔴 ROOT CAUSE: after the HDMI renegotiation Xorg brought HDMI-1 (DRM card1-HDMI-A-3) up at 640x480 with NO mode set, even though preferred 3840x2160@60 was available → the four 1920x1080 mpv panes tiled across the 3840x2160 screen couldn’t map, all 4 black (mpv time-pos=null). Nothing auto-corrected: the camwall-watchdog only watched HDMI connect/disconnect (stayed “connected”, wrong res), and it SKIPS null time-pos (unreadable≠stall) so the black panes never tripped the stall path; every re-roll (incl. camwall-frigate-watch on Frigate restart) came back on the same 640x480 screen. Clincher: xrandr = Screen 0: current 640 x 480 + HDMI-1 connected no geometry, while go2rtc was healthy (ffprobe cam1_sub=h264 1280x720, all producers connected) and the RTX 3080 was fine → display-mode problem, not stream/GPU. RECOVERY: DISPLAY=:0 xrandr --output HDMI-1 --primary --mode 3840x2160 --pos 0x0 --fb 3840x2160 + systemctl restart camwall. DURABLE FIX (deployed+verified): (1) /usr/local/bin/camwall-mpv.sh now runs that xrandr near the top (after xset, before panes) so EVERY start/re-roll self-heals the mode — verified force-640x480→restart→4K; (2) /usr/local/bin/camwall-watchdog.py poll loop now parses xrandr (env DISPLAY=:0, no XAUTHORITY needed) for Screen 0: current WxH and calls the existing reroll("resolution fallback WxH") if ≠3840x2160 — verified force-640x480 with NO manual action auto-restored to 4K in ~22s (rerolling camwall: resolution fallback 640x480), POLL=10s so ~10–20s worst case. Backups camwall-mpv.sh.bak-mode-1786483068, camwall-watchdog.py.bak-mode-1786483068. Note: host has NO ffprobe; host ffmpeg=/usr/bin/ffmpeg (not the Frigate container’s /usr/lib/ffmpeg/7.0/...); wall screenshot via DISPLAY=:0 /usr/bin/ffmpeg -f x11grab -video_size 3840x2160 -i :0 -frames:v 1. — 2026-08-11-camwall-hdmi-mode-fallback-power-recovery

2026-08-10

• Two-lens adversarial review pass (correctness/concurrency + security/data-integrity) over the top-kép voting system — findings fixed + verified on telep-mainframe. 🔴 CRITICAL pre-existing bug missed by earlier reviews (they curl’d the JSON endpoint, never rendered the page): GRID_TAIL was concatenated RAW into the HTML but written with doubled {{/}} braces (as if .format()ed like GRID_HEAD/GRID_CARD) → the braces leaked literally into the <script> making it invalid JS, so vote() never defined → client-side ❤ voting had NEVER worked; fixed to single braces, verified served /nap JS passes node --check. Lesson: verify web UIs by rendering the page / checking the served JS, not only curling the JSON. Confirmed SAFE: identity spoofing impossible (top4-web binds 127.0.0.1, tailscale serve overwrites Tailscale-User-Login from the verified peer), bot token can’t leak via exceptions (str(e) omits URL; web proc never touches token). Fixed+verified: stored XSS (client camera rendered unescaped → render_grid now html.escapes event_id/camera/title, quote=True), concurrent toggle race (double-tap → two POSTs → IntegrityError/HTTP500; toggle_vote rewritten atomic DELETE…rowcount + INSERT OR IGNORE), disk reclaim (un-vote to 0 votes now deletes jpg + pics row), client error handling (vote() treated non-403 errors as success → added if(!r.ok) guard), CSRF (/vote 403s on Sec-Fetch-Site: cross-site), DB perms (/srv/top-kep/votes.db 644→600, exposed voter emails). OPEN product decision: paired physical cams (telep_cam1/cam2=south, cam3/cam4=north) are different Frigate NAMES so the same moment can appear twice in /nap (dedup keys on camera-name+second) — dedup by physical group or keep both angles? Deferred minors: derive camera server-side from Frigate, validate event_id charset, per-handler sqlite conn close (benign). — 2026-08-10-top-kep-community-voting-system
• ⭐ Top kép community voting system LIVE — a tailnet-only Hungarian voting flow that REPLACES the Google-Drive top-pics workflow. People vote favourite Frigate person snapshots per day (GET /nap) → rolling up into weekly (/het) → monthly (/honap) rounds; voted pics saved LOCALLY on the mainframe (NOT Drive), surviving Frigate’s ~14-day purge. Identity via the Tailscale-User-Login header (one vote per tailnet identity; POST /vote → 403 without it). Built on telep-mainframe into the existing ~/top4-web.py — a stdlib http.server app, NOT Flask (correcting older vault text) (top4-web.service, 127.0.0.1:8090 → https://telep-mainframe.taild4189d.ts.net:8443): new routes /nap /het /honap, GET /pic/<event_id> (local jpg else Frigate proxy), POST /vote (form event_id/camera/captured_ts → JSON {votes}). Data/logic in new ~/top_kep_store.py (Python 3 stdlib only): SQLite /srv/top-kep/votes.db — pics(event_id PK,…,saved_path,first_voted_ts), votes(event_id,voter,PK(event_id,voter)), reminder_state(kind PK,last_event_ts). Candidates = Frigate label=person events for the day deduped to one per (camera, whole-second) keeping highest top_score; on a pic’s FIRST vote its clean full-res snapshot (bbox=0&quality=100) is fetched + saved to /srv/top-kep/img/YYYY-MM-DD/<event_id>.jpg (idempotent). Week=ISO Mon–Sun, Month=calendar, all Europe/Budapest. Reminder CLI ~/top_kep_remind.py (symlink ~/top-kep-remind.py): --daily|--weekly|--monthly [--dry], reads bot token at runtime from ~/nvr/frigate-notify/config.yml (no secret in source), sends via Bot API to group -1004475187307; daily sends only if newest candidate > watermark (advanced only after a confirmed non-dry send), --dry side-effect-free. Timers (User=levander): top-kep-remind-daily.timer (20:00), -weekly (Sun 19:30), -monthly (daily 21:00, CLI-guarded to fire only when tomorrow is the 1st — covers 28/30/31), all Persistent=true. Drive: new flow writes NOTHING to Drive; weekly top4-export.timer DISABLED; old picker’s Drive buttons/endpoints (/save-full-to-drive, /save-crop-to-drive, ~/top-kep-drive.py, ~/top-kep-id.py) + 4-slot camwall flow left INTACT (full rip-out deferred). Home portal (2026-07-24-global-dashboard) “Top képek” tile repointed to …:8443/nap. Tests ~/test_top_kep.py 12/12 pass. Follow-ups: ❤ button doesn’t pre-highlight already-voted pics after reload (tally still correct); deferred = full Drive rip-out + a print/export path off the local store + >720p pics needing the record/main stream. Supersedes 2026-08-10-top-kepek-drive-curation-print-pipeline; spec 2026-08-10-top-kep-community-voting-spec, plan 2026-08-10-top-kep-community-voting-plan. — 2026-08-10-top-kep-community-voting-system
• Session handover for the 2026-08-09/10 overnight — single entry point tying together AirPlay fix, Frigate FPS watchdog, UPS shed/restore hardening, KB Qdrant restart, camera OSD clock DST fix, .139 weak-WiFi frame drops, camwall birdseye re-roll, offline printer, the master control-plane agent, Stirling-PDF, the C302 tailnet FaceTime cam, the Top képek→Drive pipeline, and the telep-router DHCP/VLAN/SSH changes. Per-subsystem status table + a front-and-centre “needs you” list (printer power-on/Ethernet, DHCP reservations for printer + C302, Ethernet to the north camera, go2rtc-cam WebUI → own tag:telep node, Stirling-PDF doc gap). Doesn’t re-derive — summarises + links the individual dated notes. Living runbook telep-mainframe-handover refreshed alongside (status table, open-items split into session/standing, new-services pointer block, C302 + AirPlay lines in the cameras section). — 2026-08-10-session-handover
• ⭐ “Top képek → Drive” curation/print pipeline — browse Frigate detections, crop at native res, push to Google Drive for printing, all self-serve in the picker (no more feeding timestamps to an agent). Extends the top4-web.py picker (top4-web.service, User=levander, 127.0.0.1:8090 → https://telep-mainframe.taild4189d.ts.net:8443, the Top képek tile) with four things: (1) batched infinite scroll — was a hard bug (dumped ALL ~1500 events into the DOM at once, each thumbnail a live Frigate fetch → jank + hammering); now renders 48/batch via an IntersectionObserver on a #sentinel; (2) jump-to-date/time — Ugrás/Most text box, flexible input (2026-08-04 17:13 or Aug 4 2026, 5:13 PM) parsed client-side with new Date(), fetches /events?from=<t-1800>&to=<t+7200> (30 min before→2 h after); server /events now takes optional from/to unix params (falls back to last-7-days); (3) save-to-Drive — per-card ⬆ chip POST /save-full-to-drive?id= (clean full snapshot bbox=0&quality=100) + crop-modal “Kivágás → Drive” POST /save-crop-to-drive?id= (freeform box exported at NATIVE resolution, NOT the 880×495 camwall letterbox — for print quality); both land in gdrive:top_képek/. Bulk helpers on the box (as levander): ~/top-kep-drive.py "<YYYY-MM-DD HH:MM>" … (ALL person detections in t-45s→t+75s, deduped >3s/cam, FLAT into gdrive:top_képek/archive/) + ~/top-kep-id.py <eventid> … (exact frames by id → top). Drive reorg: top_képek/=curated flat, top_képek/archive/=raw flat pile (no month subfolders, user’s explicit call), ready-to-print folder referenced BY ID gdrive,root_folder_id=1wPBPJ3avEAXeIbKCu2clVtGULB_I3C5M: (rclone connection-string folder-by-id). Frigate: snapshots enabled, http://127.0.0.1:5000 (/api/events?after=&before=&has_snapshot=1, /api/events/<id>/snapshot.jpg?bbox=0&quality=100, /api/events/<id>); detect stream 1280×720 → 720p snapshots (fine for 4×6, extract from record/main for larger — future upgrade). 🔴 GOTCHAS: (1) the _0.95 “score” in filenames is Frigate confidence (top_score), NOT a unique id — ties share a score → picking collisions, TIME is the unique key (a NNN_ unique-numbering pass over archive started); (2) Drive rate-limits HARD after bulk moves (~18–60s/file) — run big rclone reorg/rename/flatten detached (setsid nohup … </dev/null &, poll log); a timeout-wrapped SSH is killed mid-batch (rclone rmdirs --leave-root / deletefile for cleanup); (3) rclone folder-by-id = gdrive,root_folder_id=<id>: reusing the gdrive remote + --config /home/levander/.config/rclone/rclone.conf; (4) Frigate’s React UI can’t take a native “add to top képek” button (would need userscript/injecting proxy) → resolved via the picker’s jump-to-time + Drive buttons. Supersedes/extends the prior top4 spec that lived only in SESSION-HANDOVER.md. — 2026-08-10-top-kepek-drive-curation-print-pipeline

2026-08-09

• ⭐ Xiaomi C302 (indoor pan-tilt, China-cloud) turned into a low-latency FaceTime/Zoom/Meet cam over the tailnet via go2rtc + WebRTC — WITHOUT adding it to Frigate, and while re-air-gapping the Tapo NVR cams. New go2rtc-cam container at /home/levander/go2rtc-cam/ on telep-mainframe (alexxit/go2rtc, network_mode: host, api :1984 / rtsp :8555 / webrtc :8556 — Frigate’s own go2rtc holds :8554, hence the second instance). Streams: c302: xiaomi://<account>:de@192.168.30.168?did=1183965641&model=xiaomi.camera.c302n (H265, LAZY = 0 CPU idle) + c302_h264 ffmpeg transcode fallback. 🔴 WebRTC needs explicit candidates: [192.168.1.123:8556, 100.115.209.87:8556] because host-network binds all docker/incus bridges too. Exposed interim via tailscale serve --bg --https=8450 → https://telep-mainframe.taild4189d.ts.net:8450 (follow-up: give it its own tag:telep node per tailnet-service-exposure-convention). Client = OBS macOS Window Capture of a chromeless Chrome app-window rendering webrtc.html?src=c302 → OBS Virtual Camera → FaceTime (sub-second). Privacy: C302 can’t be network-air-gapped (P2P needs a cloud-brokered key) so it’s on a hardware relay + a red BELSŐ KAMERA AKTÍV banner (internal-cam-alert.service, pings .168 q3s, draws on camwall :0). Firewall re-scoped on telep-router from zone-wide cams→wan/cams→lan to per-IP rules (c302-wan/c302-lan, src_ip=192.168.30.168) → Tapo cams .119/.139 fully air-gapped again. THE GOTCHA GAUNTLET: (1) login 70016 登录验证失败 = wrong region (must be de), fails BEFORE captcha/2FA which go2rtc otherwise handles; (2) did/model/region via PiotrMachowski Xiaomi-cloud-tokens-extractor at ~/xiaomi-tokens/ — its console output isn’t saved, | tee; (3) the xiaomi source must log in (stores V1:***), can’t be fed pre-obtained tokens = inherently not air-gapped; (4) P2P across VLAN needs cams→lan or you get read udp i/o timeout + 0-byte frames; (5) C302 is H265 so OBS RTSP shows black → transcode, but WebRTC decodes it natively; (6) OBS Media Source black even when VLC works → uncheck “Use hardware decoding” on macOS; (7) OBS built-in browser (CEF) renders WebRTC gray (known limitation) → real Chrome app-window + Window Capture; (8) latency floor is TRANSPORT not the cam (WebRTC sub-second, MSE ~1s, RTSP+transcode worst); (9) no local PTZ — no miio endpoint (UDP 54321 unreachable), motor control is Mi cloud only, auto-framing = an OBS face-track plugin; (10) telep-cc SSID temporarily unhidden for Mi Home onboarding then re-hidden, and router SSH host key changed again post-factory-reset. Secrets (Mi pw, V1: blob, cam token) kept OUT of the vault. — 2026-08-09-xiaomi-c302-tailnet-facetime-cam

2026-08-08

• ⭐ Camera OSD clocks were wrong on BOTH devices — ÉSZAK (camwall RIGHT column) by 109 DAYS, DÉL (left) by exactly 1 h — root-caused + FIXED with an hourly ONVIF time push. .139 (ÉSZAK, telep_cam3+telep_cam4) self-reported DateTimeType=NTP while stuck at 2026-04-21 04:32; authenticated GetNTP showed FromDHCP=true, IPv4Address=0.0.0.0 → the camera never had an NTP server address, so the cams-VLAN NTP DNAT documented in Camera VLAN (telep-cc) DNATs traffic that is never generated (DHCP option 42 is not served on that VLAN). .119 (DÉL) held local CEST in its UTCDateTime field with TZ=GMT-01:00 on top → +1 h. 🔴 GOTCHA: Tapo firmware SILENTLY IGNORES the ONVIF TimeZone + DaylightSavings fields — SetSystemDateAndTime returns a clean HTTP 200 and keeps TZ=GMT-01:00 anyway (tried a full POSIX TZ string and the device’s own format); only the UTCDateTime numbers are writable, and DateTimeType=NTP is aspiration NOT sync status. Also: the ONVIF TZ string is POSIX-signed, so GMT-01:00 means UTC +1. FIX: /usr/local/bin/cam-timesync.py + cam-timesync.timer (hourly, OnBootSec=3min, Persistent=true, enabled) pushes host time pre-compensated by the camera’s own live-read TZ offset, no-ops when skew ≤60s — self-corrects across the Oct 25 DST changeover because the offset is derived, not hardcoded. Scope note: OSD pixels only, Frigate stamps recordings/events/alerts from HOST time so those were always right. Verified both panes read 23:49:48 against host 23:49:48. Open follow-up: uci set dhcp.cams.dhcp_option='42,192.168.30.1' on telep-router (root cause), but keep the timer regardless or the +1 h DST error returns. — 2026-08-08-camera-osd-clock-drift-onvif-timesync
• ⭐ Master control-plane agent LIVE on the HOST telep-mainframe — always-on, phone-drivable Claude, mirrors the Incus container control-planes + adds systemd autostart. control-plane.service (User=levander, Type=forking, Restart=always, enabled) runs claude remote-control --name control-plane-telep-mainframe --spawn same-dir in a tmux session; came up headless, shows Connected · obsidian · main in the Claude app Code tab, VERIFIED working from the phone. ⚠️ the app displays the auto-SPAWNED session name (e.g. …-velvety-star), NOT the device --name. Vault cloned to /home/levander/obsidian (deploy key id_ed25519, repo-local core.sshCommand, git identity telep-mainframe/ledererandras2004@gmail.com; user added it as a WRITE deploy key on private wowjeeez/obsidian). obsidian-sync.timer (enabled, OnBootSec=5min/OnUnitActiveSec=15min) → /usr/local/bin/obsidian-sync.sh does bidirectional pull --rebase --autostash+commit+push. Harness parity: path-rewritten historian+obsidian-documenter at /home/levander/.claude/agents/ + host ~/.claude/CLAUDE.md (Mac-only sources — claude-mem, auto-memory, obsidian CLI — absent on the box but degrade gracefully to file I/O + git). Worklog projects/homelab/telep-master-worklog.md. 🔴 SECURITY POSTURE (deliberate, mirrors containers): trust dialogs PRE-SEEDED (remoteDialogSeen/hasTrustDialogAccepted=true in ~/.claude.json, backups .bak-*) so it launches headless — but --dangerously-skip-permissions is NOT used, tool permissions stay ACTIVE → destructive ops surface approval prompts in the app = human-in-the-loop guardrail. CAVEAT: box has passwordless sudo → an approved sudo action from the phone = instant root; the guardrail is APPROVAL, not a sandbox (cf. the 2026-08-03-telep-router-factory-reset-recovery blast-radius lesson). Design/spec at 2026-08-08-telep-master-agent-design. — 2026-08-08-telep-master-agent-impl
• ⭐ UPS shed/restore HARDENED — a missed NUT ONLINE can no longer leave services shed (root-caused a knowledgebase.service crash-loop of 2360×). Overnight failure: ONBATT→shed (KB, Qdrant, camwall, AirPlay, jobs; kb-qdrant explicitly docker stopped)→LOWBATT→graceful poweroff→mains returned WHILE THE BOX WAS OFF→BIOS auto-power-on→boot, but the NUT ONLINE/restore event NEVER FIRED (upsmon wasn’t running to observe OB→OL) → kb-qdrant (no boot recovery) stayed down → KB crash-looped against a missing Qdrant at 127.0.0.1:6333. Core flaw: restore was coupled ONLY to the ONLINE event, which is unobservable if the box is off across mains-return (the EXPECTED path when an outage outlasts the battery). FIX (all on the box): extracted restore into shared idempotent /usr/local/bin/power-restore.sh ([RECONCILE]-tagged logs) called by BOTH the NUT ONLINE handler AND a NEW boot-time oneshot power-restore-reconcile.service (enabled, After docker+network-online) → box self-heals shed state on every boot; kb-qdrant → --restart unless-stopped; knowledgebase.service.d/wait-qdrant.conf drop-in polls 6333/readyz (~60s) before starting instead of crash-looping. Handler backup /etc/nut/nut-outage-handler.sh.bak-*. 🔴 GOTCHA: sudo tee SILENTLY DROPPED a script’s #!/usr/bin/env bash shebang → direct exec fell back to dash and choked on a bash array → transfer shebang-bearing files as base64. — 2026-08-08-ups-shed-restore-hardening
• Frigate per-camera FPS watchdog (new) — auto-restarts the frigate container on BOTH a frozen feed AND a detection jam, with a circuit breaker vs thrash. /usr/local/bin/frigate-fps-watchdog.py + /etc/systemd/system/frigate-fps-watchdog.service (active, enabled, User=root, Restart=always). Polls http://127.0.0.1:5000/api/stats every 20s; trips on (a) frozen feed camera_fps<1.5 (normal ~5) or (b) detection jam process_fps<1.0 while camera_fps healthy → docker restart frigate. Bulletproofing: 3-consecutive-poll debounce (~60s), 120s post-restart cooldown, and a CIRCUIT BREAKER (max 3 restarts/hour; 4th suppressed → Telegram alert-only, re-alert every 30min) so a genuinely-offline camera can’t cause restart thrash. Telegram via /etc/nut/telegram.env; all tunables are constants at the top of the script. Context: Frigate jammed TWICE today (whole-pipeline detection jam per 2026-08-06-frigate-detect-record-jam-cpu-starvation, then cam3 frozen at camera_fps=0.3) each needing a manual docker restart frigate; the camwall watchdogs don’t watch per-camera fps, hence this. Note: a breaker trip is the signal to check for a WEDGED camera needing an ONVIF reboot (2026-08-03-cam-stall-recovery-and-casino-alert) — a restart won’t fix that. — 2026-08-08-frigate-fps-watchdog
• Home portal tile swap — Szemantika → Képosztályozás. /home/levander/home-portal/index.html (static, python -m http.server 8093, own tailnet node home.taild4189d.ts.net): replaced the Szemantika tile (→ knowledgebase.../semantics) with a Képosztályozás tile (→ knowledgebase.../image-review) because semantic search is being merged into main KB search. Backup index.html.bak-* left. — 2026-07-24-global-dashboard
• ⭐ telep-mainframe AirPlay (telep-tv) dead after the 2026-08-08 UPS switch + reboot — root-caused + FIXED, verified (real cast streamed 14:55). TWO compounding issues: (1) avahi was publishing telep-tv on ALL interfaces (allow-interfaces commented out → default all-ifaces) so telep-tv.local resolved to bogus 172.17.0.1 (docker0) / 172.18.0.1 + 172.19.0.1 (incus/veth) / 127.0.0.1 alongside the real LAN 192.168.1.123 → Apple clients latched an unreachable record = “appears but casting does nothing”; (2) DHCP IP flap on boot — enp5s0’s .123 is a DYNAMIC lease (noprefixroute), briefly withdrawn+re-added on this boot; avahi logged Leaving mDNS multicast group ... Interface no longer relevant 14:47:22→rejoined 14:47:58 = ~35s where telep-tv wasn’t announced at all = “doesn’t appear”. FIX (durable, survives reboot): set allow-interfaces=enp5s0 under [server] in /etc/avahi/avahi-daemon.conf (backup avahi-daemon.conf.bak-<ts>), systemctl restart avahi-daemon uxplay → avahi-browse -rtp _airplay._tcp now shows ONLY the enp5s0/192.168.1.123 record, junk gone. VERIFIED: Accepted IPv6 client→Begin streaming to GStreamer video pipeline, camwall handoff cycled (grid stopped during cast, restored after). ⚠️ GOTCHA: tcpdump was NOT installed on telep-mainframe — mDNS/net debugging silently produced empty output until apt-get installed (now present); verify tcpdump exists FIRST for future net debugging on this host. Casting state file = /run/uxplay/casting (0=idle), /run/uxplay/ root-only. 🔴 OPEN FOLLOW-UP: the boot flap RECURS unless .123 gets a DHCP reservation on the router (MAC d8:5e:d3:a7:05:d6) — the 2026-08-03 factory reset (2026-08-03-telep-router-factory-reset-recovery) likely dropped the prior reservation. — 2026-08-04-telep-mainframe-airplay-receiver-uxplay

2026-08-06

• ⭐ KB HEADLINE: the “orphaned procedure-fragment” pages are a marker CHUNKER/SPLITTER bug, NOT OCR-legibility — recoverable deterministically without re-OCR. Pages with a generic H1 (# Removal and installation, # Inspection) and no component name were created because marker emitted a page-break BETWEEN a component header and its procedure, and the KB’s # H1 splitter stranded the component name. The true header survives verbatim in marker’s RAW output ~/ocr/marker_out/<manual>/<manual>.md (e.g. # Idle Air Control (IAC) valve) → recover by text lookup, no GPU. OCR BAKE-OFF (GOT-OCR2 stepfun-ai/GOT-OCR-2.0-hf, ~3.6GB VRAM, ~10–24s/page): recovers dropped headers + rebuilds catastrophically-mangled spec tables (e.g. kickfix-docs/specsgeotracker1994/004-power-teams) BUT regresses clean prose (char noise), emits LaTeX not GFM, fragile seams, useless on wiring line-art. DECISION: patch (deterministic header re-attach), NOT full re-OCR; targeted micro-re-OCR only for destroyed spec-table + code-block pages. SOTA OCR (Aug 2026) noted for later: GLM-OCR, DeepSeek-OCR 2, Dolphin (structure-first), PaddleOCR-VL. GOTCHAS: JustVugg/colibri is an MoE-LLM inference engine NOT OCR; DeepSeek-OCR won’t install (needs flash_attn + transformers 4.4x vs installed 5.x). — 2026-08-06-kb-marker-chunker-bug-and-ocr-bakeoff
• KB orphan-fix subsystem — deterministic heading recovery + gated re-embed/re-tag (~/knowledgebase/kb-vectors/). orphans.py→orphans.json (235 found), pagesrc.py (locate page body in marker RAW → nearest non-generic heading), orphanfix.py (recover + confidence GATE + re-embed/re-tag), orphan_apply.py (batch), pageflow.py (Inspect/Fix-heading/Re-OCR/Apply/Rebuild). RESULT: 89 applied / 146 staged-uncertain (gated — a WRONG component name is worse than a generic one), 149 Qdrant points re-embedded+re-tagged, e.g. point 259 IAC valve Cooling→Fuel. Re-embed = new text→bge embed_passages→update vector+text payload IN PLACE (same ID, preserve other keys); re-tag via kb-vectors/tagger.py. Reversible: .bak-orphanfix + staging/orphan-changelog.json. — 2026-08-06-kb-orphan-heading-recovery-subsystem
• KB per-page Tools UI + persistent job queue (Flask app.py @ 127.0.0.1:8092). app.py injects a floating “Tools” button on every page + /orphans, /api/orphans, /api/page-flow. Job queue: jobstore.py = persistent JSONL kb-vectors/jobs.jsonl (survives restart/power-blip — matters on this box), GET /api/jobs + a /jobs auto-refresh dashboard; every apply/re-OCR/fix/rebuild registers a TYPED job. — 2026-08-06-kb-tools-ui-and-job-queue
• KB section-code extraction — FSM codes into Qdrant payload. FSM codes (8A-9, 6E-92) parsed from contents tables + text → payload section_codes/section_names/section_prefixes on 490 pages + kb-vectors/section_index.json (1254 codes). Prefix→system map: 8A/8B/8C→Electrical, 6E/6C/6J/6K→Fuel&Emissions, 6A→Engine, 6B→Cooling, 7A–7F→Drivetrain, 3x→Steering&Suspension, 1A/1B→HVAC, 9J→Body, 0x→Diagnostics. ~61% agreement with the embedding text system-tags → complementary (hard structural signal vs soft semantic), not redundant. — 2026-08-06-kb-section-code-extraction
• KB two embedding-derived classifiers. (a) TEXT system-tags: bge doc-vectors vs 10 PROTOTYPE vectors, cosine-assigned → Qdrant primary_tag/tags on 7820 points (kb-vectors/tagger.py); GOTCHA — the Fuel prototype lacked idle-air-control terms so IAC mis-scored Cooling → augment the prototype (same fix as point 259). (b) IMAGE types: SigLIP google/siglip-so400m-patch14-384 zero-shot on wiring-gallery imgs (~2GB VRAM, ~26ms/img) → 8 types → public/wiring-gallery/image_types.json → gallery filter pills; gallery renamed “Wiring & Schematics” (URL /wiring-gallery/ KEPT). KEY FINDING: corpus is almost all scanned B&W LINE ART — of 480 imgs: line-drawing 222 / wiring 149 / flowchart 55 / connector 33 / other 17 / spec-table 3 / photo 1 / exploded 0 (photos/spec-tables/exploded ~absent); SigLIP softmax low on look-alike classes → gate ≈0.35/0.10 (0.55 too strict). — 2026-08-06-kb-embedding-classifiers-system-tags-and-image-types
• KB BUILD GOTCHA — backup dirs leaked into the live site + TSB diagram restore. build_content.py auto-discovered ALL dirs under manuals-src/docs INCLUDING *.bak* (e.g. kick-fix.bak-linkrewrite) → ~1646 stale pages (of 4566) served live. FIX: a list_manuals() helper excluding any name containing .bak. LESSON: content-tree backup dirs (.bak-orphanfix, .bak-linkrewrite) must be excluded from discovery. ALSO: FSM diagram images restored to 20 kick-fix/tsb legend pages by rendering source PDFs (pdftoppm) — the crops were never captured on ingest. — 2026-08-06-kb-build-backup-dir-leak-and-tsb-diagram-restore
• ⭐ telep-mainframe POWER ROOT CAUSE FOUND + FIXED — an unplugged GPU 12V PCIe connector, NOT marginal mains. The RTX 3080 is fed by 3 separate (non-daisy-chained) PSU PCIe cables and one bank was disconnected → starved GPU power delivery under load → the recurring under-load hard-offs. Reconnected during reassembly. VALIDATION: a graduated CPU+GPU stress test (CPU PL2 160→200→241→250W, GPU cap 320→360→400→450W, ~3 min/stage) FULLY PASSED, zero crashes — uptime climbed straight through, kernel journal clean (no thermal/throttle/undervolt/MCE/hardware-error). Peak GPU ~411W, peak combined ~536W; GPU 86–87°C no throttle, CPU 76–82°C. Corsair 1000W PSU was adequate (NOT the fault). This supersedes the §6c “heavy CPU triggers brownouts on marginal mains” theory in 2026-08-05-power-root-cause-nvme-damage-ups-kb-handover for the load-triggered cuts (a UPS+NUT still worthwhile for genuine mains + the deferred fsck/NVMe self-test still apply). GOTCHAS: CPU stayed ~125W every stage because PL1=min(125,PL2) is enforced under sustained load (PL2 only in the brief boost window) — raise PL1 to stress the CPU rail; GPU FP32 matmul is compute-bound below the 450W cap (never reached it). ⚠️ TODO: the test’s caps are NOT persistent — reset on reboot (mobo aggressive/uncapped CPU default + GPU 320W stock); author a persistent sane profile. — 2026-08-06-power-root-cause-gpu-12v-connector-stress-test-pass
• Frigate detect/record pipeline JAMS under CPU starvation and does NOT self-recover — needs sudo docker restart frigate. The 20-min power stress test’s all-core CPU load starved Frigate’s detect+record processes on telep-mainframe; frame queues + recording-segment cache jammed and stayed jammed even after load dropped (container back to ~10% CPU, box load normal). SYMPTOM: main-page live tiles black, latest.jpg frozen/identical across cameras. DIAGNOSIS via curl http://127.0.0.1:5000/api/stats: camera_fps normal (~5) but process_fps 0.1, skipped_fps ≈ camera_fps, and global detection_fps=0.0; log spam frigate.record.maintainer WARNING: Too many unprocessed recording segments in cache.... FIX: sudo docker restart frigate (process_fps→5, detection_fps→113, warnings stop). per-camera detect_fps=None is a display quirk — trust the GLOBAL detection_fps. Distinct from the wedged-RTSP camera-side failure in 2026-08-03-cam-stall-recovery-and-casino-alert (where a Frigate restart does NOT help). Arch: 2 cams (.119/.139, stream2/stream7) → go2rtc restream (rtsp://127.0.0.1:8554/camN_sub) → Frigate detect+record both pull from restream; image ghcr.io/blakeblackshear/frigate:stable-tensorrt, ports 5000/8971 on 127.0.0.1, ONNX ~15ms. PREVENTION: nice/ionice heavy jobs, re-check /api/stats after. — 2026-08-06-frigate-detect-record-jam-cpu-starvation
• KB wiring-gallery low-res FIXED (marker-native crops) + Astro cache-control headers. The /wiring-gallery/ scans were low-res (480 imgs, ~7MB, ~12–20KB each) NOT because of OCR but because /home/levander/wiring-collection/gen_thumbs.py downscaled marker’s native-res figure crops to WIDTH=320, quality=72 and the gallery served the thumbnails. FIX: copy the marker-native originals verbatim (the image_abs path in gallery_items.json) into manuals-src/docs/assets/wiring-gallery/ and kb-astro/public/wiring-gallery/img/ → 2.5× sharper (320px→822px). CEILING: source scans are only 150 dpi so ~822px is the real limit (300dpi re-render = pointless upscaling); marker *_meta.json has NO per-figure bboxes so you can’t re-crop without re-running marker. build_content.py doesn’t touch the gallery dirs (durable). ALSO: Flask app.py now sets Cache-Control: immutable for fingerprinted /_astro/* and no-cache for everything else, stopping stale-HTML/missing-image after redeploys. Follows 2026-07-30-kb-wiring-gallery-pivot-complete. — 2026-08-06-kb-wiring-gallery-native-res-fix-and-astro-caching
• ⭐ KB fixkick expansion LIVE + new-page search indexing PARKED + brownout root-cause upgraded (handover updated for 2026-08-07 resume). KB (Astro+Starlight, knowledgebase.taild4189d.ts.net) expanded from fixkick.com (live successor to archived kick-fix.com): 135 → 388 EN pages — +~253 pages incl. 23 TSB bulletins (kick-fix/tsb/), Wayback-recovered dead pages (e.g. schematics-run 500s live but had a 2017 snapshot), new sections general/power-elect/tsb. 920 internal .html cross-links rewritten (0 broken remain), 344 dropped; linkmap kickfix_linkmap.json + ADDED_MANIFEST.json; backup manuals-src/docs/kick-fix.bak-linkrewrite. Single-child folders flattened (folder URL serves the one page, child→folder redirect; flatten_redirects.json wired into astro.config) via ~/kb-astro/build_content.py. Deterministic sidebar-focus deployed (astro.config head script scrolls .sidebar-pane to active on page-load/after-swap + double-rAF + timeout + MutationObserver to beat Starlight’s scroll-restore race) — ⚠ live Playwright click-through proof NOT yet run (box down). Glossary/acronym query-expansion LIVE: hybrid.py from kb-vectors/glossary.json = 325 bidirectional entries (154 Vitara manual abbrevs + 171 fixkick JARGON); exact matches still rank first. Qdrant/BM25 target ≈ 3166 + new chunks. 🔴 BUILD-TOOL GOTCHA: canonical builder = build_content.py (full regen: section indexes + folder auto-discovery); convert.py is a PARTIAL builder — do NOT use for real builds. Deploy = atomic swap dist→~/knowledgebase/site; rollback site.astro-prev, deeper site.mkdocs-bak. PARKED: only ~90/253 new pages embedded into Qdrant manuals when the box died — pages render but aren’t searchable (hybrid.py builds BM25 from Qdrant, so searchable only after embed). RESUME: CPU-capped resume-safe embed (systemd-run --unit=kbembed -p CPUQuota=400% embed_launch.sh; index_new.py deterministic-ID skip-already-done; relaunch each reboot until 0), then restart knowledgebase.service, verify /api/search returns a new /tsb//power-elect//general/ page + run deferred Playwright proof; new-page images not yet downloaded (light follow-up, build guard drops missing refs). 🔴 POWER upgraded: full 24-core embedding REBOOTS the box (brownout) on this marginal mains — CPU-cap ≤500%/≤5 cores stayed stable → keep ALL heavy CPU/GPU capped until UPS. Overnight 2026-08-06 outage: stable 08-05 21:36 → 03:21 first cut → ~12 hard reboots 03:21–04:56 → dead 04:56–09:00 (~4h) → back 09:00; NVMe unsafe_shutdowns climbing (was 241, damage ongoing); UPS due Fri 2026-08-07 + NUT = the fix. 🔴 PROCESS gotcha: an empty TaskList does NOT mean a long bg agent finished — a crawl agent ran ~83 min across reboots with no visible tasks → launched 3-way overlapping KB writes (converged, dedup OK, new G16A page unique); rely on completion notifications, never run two site-rebuild/deploy agents at once. — 2026-08-05-power-root-cause-nvme-damage-ups-kb-handover

2026-08-05

• ⭐ telep-mainframe power root cause CONFIRMED via NVMe SMART — and the hard cuts are now DAMAGING the SSD. Box hard-cut ~6× in ~50 min (boots as short as 90s); last shows every boot back to Jul 31 ended in crash. Ruled out thermal (CPU 49°C / GPU 55°C, zero throttling) and hardware (no MCE/EDAC/panic); crash signature = normal logs then INSTANT silence = external mains loss. PROOF = nvme smart-log: 240 unsafe_shutdowns, 1440 power_cycles, 1617 media_errors, 4% used, critical_warning 0 — the power-loss-during-write is actively corrupting the drive (not failing yet, but bleeding errors). The recurring nv_drm_revoke_modeset_permission WARNING is KNOWN-BENIGN cosmetic nvidia — do NOT chase it. FIX/DECISION: UPS arriving ~2026-08-07 — pure-sine (mandatory for active-PFC PSU) + AVR + ~1500VA/900-1000W + USB/NUT auto-shutdown (CyberPower CP1500PFCLCD or APC Smart-UPS 1500); install NUT for clean auto-shutdown = what stops the media-error bleed. TODO once stable: fsck + NVMe self-test; consider electrician for site wiring. GOTCHA: don’t run heavy/write-heavy builds until UPS is in. ALSO parked & staged for 2026-08-07 resume: KB hybrid search DONE+LIVE (rank_bm25 + Qdrant via RRF k=60, wired into /api/search, lunr /browse retired) and KB Astro 5 + Starlight 0.36 rebuild STAGED at ~/kb-astro/ (~2203 pages, 6 folders, 68 landing pages, HU locale, directory-URL byte-identical, image opt resumable from webp cache; wiring diagrams DROPPED) — resume/cutover/rollback steps documented; RuView container STOPPED, blocked on buying an ESP32-S3/C6 for real WiFi CSI. — 2026-08-05-power-root-cause-nvme-damage-ups-kb-handover

2026-08-04

• FaceKom exit-vpn “funky” recurrence — traffic egressing as raw WAN 37.76.13.180 + npm.facekom.net 403. Tailscale layer was HEALTHY; real fault was OpenVPN tunnel DOWN (openvpn-client@farm inactive, no tun0, journal AUTH_FAILED, user tt_lederer_andras is already active). ROOT CAUSE: FaceKom VPN allows only ONE concurrent session per account — a laptop/phone/stale session steals it and exit-vpn falls back to raw WAN → 403s from IP-allowlisted hosts. FIX: clear the other session, sudo incus exec exit-vpn -- systemctl restart openvpn-client@farm → egress flips to 92.119.122.32, tun0 up, npm 200, tailscaled stayed healthy. Diag one-liners via host: systemctl is-active openvpn-client@farm, curl -s https://api.ipify.org (37.76.x = tunnel down), journalctl -u openvpn-client@farm -n 10. Backup gap: exit-vpn’s OpenVPN/tailscale config is NOT in telep-infra (container-only). — 2026-07-22-facekom-vpn-exit-node
• ⭐ Self-hosted real-time collaborative tldraw whiteboard (“Miro-type” board) on telep-mainframe with PDF annotation, plus a TV kiosk “board mode” that toggles with the camera wall. App dir /home/levander/tldraw-board: a SINGLE Bun process serves the built Vite/React tldraw client AND the WebSocket sync + asset endpoints on port 8790 (binds 0.0.0.0). systemd unit tldraw-board.service (runs as levander, Restart=always, enabled). Data: data/rooms/<roomId>.json (persisted snapshots) + data/assets/ (uploaded images + PDF page PNGs). URLs: tailnet http://telep-mainframe.taild4189d.ts.net:8447/ (via tailscale serve --http 8447 → 8790), LAN http://192.168.1.123:8790/; rooms /r/<roomId> or ?room=<id>, default room main at /. STACK: tldraw 5.2.5; client uses useSync({uri, assets}) from @tldraw/sync; server keeps ONE TLSocketRoom (@tldraw/sync-core) per room over Bun native WebSocket (handleSocketConnect/Message/Close); JSON snapshot persistence (initialSnapshot + debounced onDataChange→getCurrentSnapshot); assets via PUT/GET /uploads/:id (SHARED uploads, not data-URLs); PDF import = pdf.js renders each page→PNG→uploaded asset→locked stacked image shapes (tldraw pdf-editor example approach). ⭐ CRITICAL GOTCHA: tldraw 5.2.5 renders a BLANK canvas over HTTPS without a paid license — its LicenseManager.isDevelopment is only true when protocol is NOT https, OR host is loopback, OR NODE_ENV != production. FIX = serve the tailnet endpoint over plain HTTP (tailscale serve --http), still WireGuard-encrypted over the tailnet; leaves a small “get a license” watermark link. Proper HTTPS needs a purchased tldraw license (licenseKey prop, sales@tldraw.com) — THIS is why the kiosk + tailnet URLs are HTTP not HTTPS. ⭐ TV KIOSK “board mode” toggles with the camera wall on the 4K HDMI: /usr/local/bin/tv wrapper (tv board = stop camwall.service + start board-kiosk.service; tv cams = reverse; tv status; board/cams need sudo — sudo -n passwordless on this box); /usr/local/bin/board-kiosk.sh starts matchbox WM then chromium --kiosk --app=http://192.168.1.123:8790/r/main on DISPLAY=:0 (flags incl. --no-sandbox MANDATORY as chromium runs as root, --force-device-scale-factor=2 for a legible 4K TV, --user-data-dir=/var/lib/board-kiosk); /etc/systemd/system/board-kiosk.service (root, DISPLAY=:0, Restart=always, Requires/After=camwall-x.service, Conflicts=camwall.service) — NOT enabled at boot (camwall is default); /etc/systemd/system/camwall.service.d/conflict.conf adds Conflicts=board-kiosk.service (mutual exclusion BOTH directions). ⭐ ARCHITECTURE (confirms 2026-08-04-telep-mainframe-airplay-receiver-uxplay): the Xorg :0 server is owned by a SEPARATE camwall-x.service (xinit, -nolisten tcp) — camwall.service is ONLY the mpv grid client, so stopping camwall does NOT kill X; kiosk/uxplay reuse the running :0; root connects with just DISPLAY=:0, no XAUTHORITY. ⚠️ matchbox is the WM that fullscreens kiosk windows and it is a CHILD of camwall’s session — a kiosk that stops camwall must launch its OWN matchbox or windows won’t fullscreen (chromium came up as a floating half-window until fixed). Existing tailscale serve entries preserved (8443, 8445, 8446, root→8971); only 8447 added. Cross-linked from telep-mainframe, 2026-08-04-telep-mainframe-airplay-receiver-uxplay — 2026-08-04-telep-mainframe-tldraw-whiteboard-board-kiosk
• ⭐ telep-mainframe is now an AirPlay receiver (telep-tv, uxplay 1.71 from apt) — enabled by splitting Xorg :0 out of camwall.service. THE BIG GOTCHA: the original camwall.service ran xinit directly (ExecStart=/usr/local/bin/camwall-x.sh → xinit camwall-session.sh -- :0 vt1), so it OWNED Xorg :0 — stopping it killed X entirely (verified), making the “pause the wall while casting” handoff impossible (uxplay needs :0 alive). FIX = 3-service split: NEW /usr/local/bin/camwall-xserver.sh (exec xinit /usr/local/bin/camwall-xsession.sh -- :0 vt1 -nolisten tcp -keeptty) + NEW camwall-xsession.sh (xset dpms/screensaver off → exec sleep infinity, owns X, NO WM) + NEW camwall-x.service (persistent Xorg :0, Restart=always, enabled); camwall.service MODIFIED into a client (Environment=DISPLAY=:0, ExecStart=/usr/local/bin/camwall-session.sh = matchbox + pane-runner.sh grid, Requires+BindsTo+After camwall-x.service, keeps the Frigate-wait ExecStartPre curl 127.0.0.1:5000/api/version). Now stopping/starting camwall.service toggles ONLY the grid; Xorg :0 persists. Backups camwall.service.bak.<ts> + camwall-x.sh.bak.<ts>. uxplay.service runs /usr/local/bin/uxplay-cast.sh as ROOT (DISPLAY=:0, XDG_RUNTIME_DIR=/run/uxplay, Requires+After camwall-x.service + avahi, enabled —now); wrapper runs stdbuf -oL -eL uxplay -n telep-tv -vs xvimagesink -as "alsasink device=plughw:1,3" and parses stdout → toggles the grid via a /run/uxplay/casting state file, with an EXIT trap that restores camwall on crash. ⭐ REAL uxplay 1.71 log markers (verified from a live macOS mirror — the GUESSED ones were WRONG): CONNECT (stop grid) = Accepted <ip> client on socket; STREAM START (fullscreen) = Begin streaming to GStreamer video pipeline; DISCONNECT/END (restart grid) = Connection closed for socket OR raop_rtp_mirror->running is no longer true. uxplay 1.71 does NOT print Open connections: N/TEARDOWN on a macOS mirror stop at default verbosity (the initial guess used those → camwall failed to return). ⭐ NO HW H.264 decode on this box: Debian gstreamer1.0-plugins-bad ships NO nvcodec (no nvh264dec/nvdec) and no VAAPI (vainfo empty) → decode is SOFTWARE avdec_h264 (gstreamer1.0-libav); scaling still GPU via the sink; OK since AirPlay isn’t true 4K. ⭐ FULLSCREEN gotcha: uxplay -fs did NOT fill the 4K panel — matchbox made the outer window 3840x2160 but glimagesink rendered ~1:1 top-left; FIX = xvimagesink (Xv rescales every frame) + no WM + wrapper force-resizes uxplay’s window (title telep-tv@telep-mainframe) to 3840x2160+0+0 via xdotool on the Begin streaming marker. Audio = NVIDIA-HDMI ALSA card 1 dev 3 “LG TV SSCR2” = alsasink device=plughw:1,3 (works ALSA-direct despite user PipeWire — camwall has no audio). Display = RTX 3080 (driver 550.163.01), HDMI-1 3840x2160@60. Firewall: nftables INPUT policy ACCEPT (only tailscale+docker chains, no ufw) → AirPlay on LAN needs NO change. Packages: uxplay, gstreamer1.0-tools, gstreamer1.0-alsa, xdotool (+ libavahi-compat-libdnssd1). VERIFIED: uxplay.service active+enabled on :0; avahi advertises telep-tv as _airplay._tcp+_raop._tcp; camwall split works (2×2 grid screenshot); grid stop/start keeps Xorg alive; a real macOS connect triggered the handoff + stopped the grid; EXIT trap restored camwall. STILL TO CONFIRM (live device): xvimagesink+xdotool fills the 4K panel edge-to-edge; stopping a cast now restores camwall (corrected-trigger fix); iPhone disconnect strings may need a one-time tweak (all markers verified on macOS). ⚠️ Discrepancy: TV wall (HDMI) still says mpv straight to DRM/KMS with no X server — now stale (the wall runs under Xorg :0); flagged inline there. Cross-linked from telep-mainframe — 2026-08-04-telep-mainframe-airplay-receiver-uxplay
• thermoprint debugging session — connect-clobber bug, L11 paper sensor, and the 2-sided cable-flag conclusion (Marklife P15, /home/levander/thermoprint on telep-mainframe). (1) Connect-clobber bug — “the layout changes when I connect the printer”: on connect, BOTH packages/web/src/editor/connect-flow/connect-flow.tsx (~L46-66) AND packages/web/src/store/printer-store.ts (~L49-56) overwrite the editor store’s label (size) + paperType with the device profile’s labelConfig.defaultSize/defaultPaperType, resetting the user’s chosen size the moment the printer connects. The profile is the single source of truth (packages/core/src/device/profiles/p15.ts → labelConfig); to make a label stick, set defaultSize/defaultPaperType THERE. Upstream PR #25 “fix/ignored-settings” (in HEAD) fixed a RELATED but different bug — print() read density/dither/threshold/paperType from printer-store instead of editor-store — it does NOT fix the connect clobber. (2) L11 paper handling (packages/core/src/protocol/l11/protocol.ts buildPrintSequence): after printBitmap, gap mode sends positionToGap(), continuous sends feedDots(100); STATUS_CODES has 0x01 out_of_paper. KEY HW FACT: the printer’s physical gap/die-cut sensor is ALWAYS active regardless of software paperType — a die-cut gap falling partway through one bitmap halts the head mid-print (reads out-of-paper, “only printed one side”). Software continuous does NOT override the physical sensor. (3) 2-sided conclusion: thermoprint has NO built-in two-sided/fold/cable-flag feature; auto-composing two mirrored halves into one spanning bitmap fails because the fold often coincides with a die-cut gap that halts the print. Adopted workaround (the intended usage): design ONE label WYSIWYG, print (side 1), rotate the element 180°, print again (side 2), stick back-to-back — the app natively supports per-element rotation + WYSIWYG single-label. (4) Geometry: Marklife P15 label 35mm long × 12.5mm across-head; in profile gapSizes widthMm=feed/length, heightMm=across-head width (head ~12mm); editor rotates the canvas 90° before printing (documented); Konva rotates around top-left origin (no offsetX/offsetY) so a 180° box renders up-and-left of its (x,y). (5) Build/deploy: web resolves @thermoprint/core from src (core main=src/index.ts, no build) so editing core .ts only needs cd packages/web && ~/.bun/bin/bun run build; editor served static from packages/web/dist; Vite hashes bundle names → hard-refresh/incognito to pick up new builds. FINAL STATE: stock gap mode, label 35×12.5, empty canvas, WYSIWYG single-label + manual 2-print rotation workflow. Cross-linked from 2026-07-31-thermoprint-appliance-spec, 2026-07-31-thermoprint-appliance-plan, telep-mainframe — 2026-08-04-thermoprint-connect-clobber-and-2sided-labels

2026-08-03

• Two follow-ups after the router rebuild: a wedged-camera recovery + the casino DETECTOR’s alert half. (1) After isolating the cams (removed cams→wan), the ÉSZAK/.139 dual-lens camera’s Frigate feeds telep_cam3/telep_cam4 dropped to ~0.7 fps (frozen on the TV wall) while .119 DÉL (telep_cam1/telep_cam2) stayed at ~5 fps. NOT the internet isolation — re-adding cams→wan made it 0 fps; a fresh direct ffprobe on .139 was fine (h264 720p). ROOT CAUSE: go2rtc’s existing streams were wedged because the dnsmasq/firewall restarts during the casino-block attempts dropped the camera’s weak WiFi mid-stream and stuck its RTSP session slots (Tapo allows only a few concurrent RTSP conns; stale ones don’t close). A full Frigate restart did NOT recover it. FIX: reboot the camera over ONVIF (ONVIFCamera(host,2020,'nvr42vhy1',pw).create_devicemgmt_service().SystemReboot() — pytapo CAN’T, needs TP-Link cloud creds; the nvr42vhy1 account is ONVIF/RTSP-only); after ~90 s cam3/cam4 recovered to 5.4 fps. mpv still showed frozen panes (holds last frame of a stalled RTSP, no auto-reconnect) → systemctl restart camwall.service. SIDE EFFECT: the reboot reset the camera’s clock (OSD 2026-04-21) and, isolated, it can’t NTP-sync → pushed time via ONVIF SetSystemDateAndTime (Manual, TZ CET-1CEST,M3.5.0,M10.5.0/3) + added a daily mainframe cron 30 4 * * * → /home/levander/tapo-ctl/set-cam-time.py. Scripts: reboot-onvif.py, set-cam-time.py. LESSON: go2rtc/mpv don’t auto-recover a wedged RTSP → camera reboot + camwall restart is the recovery; and rebooting an isolated Tapo cam loses its clock → keep a periodic ONVIF time-sync cron. (2) Deployed the casino/gambling detector’s ALERT half on the router (root@100.69.112.32): /etc/casino-alert.sh (busybox-ash) tail -F /tmp/dnsmasq-queries.log | while read line matching a gambling regex inside the loop (busybox grep has NO --line-buffered), 1 h per-(client|domain) cooldown via /tmp/casino-alert-state, Telegram alert via uclient-fetch to the frigate-notify bot (chat_id -1004475187307; token from ~/nvr/frigate-notify/config.yml, kept OUT of vault); procd service /etc/init.d/casino-alert (respawn, enabled); verified end-to-end (injected line fired, Telegram ok:true). BLOCKING NOT deployed: /etc/casino-block.conf (gTLD wildcards .casino/.poker/.bingo/.bet + ~75 brands, conf-file include) passed dnsmasq --test but dnsmasq didn’t answer the blocked query after restart → auto-rolled-back (verify now tests a LOCAL blocked query, not upstream, to avoid false rollback on WAN-lease flap); also every dnsmasq restart risks re-stalling the weak .139 cam. TODO: debug blocking via addnhosts+SIGHUP (no restart) or a maintenance window. Cross-linked from 2026-08-03-telep-router-factory-reset-recovery, telep-router, telep-mainframe — 2026-08-03-cam-stall-recovery-and-casino-alert
• ⭐ telep-router FACTORY-RESET + rebuilt from scratch after a casino-blocker deploy took DNS down site-wide. ROOT CAUSE: a subagent deploying a casino/gambling DNS blocklist (dnsmasq address=/…/ sinks + conf-file include) restarted dnsmasq with a broken config, then hit an API error mid-deploy and died → dnsmasq failed to restart → DNS down for the whole network → user had to factory-reset the only gateway. No config backup existed. LESSON: never blind-restart dnsmasq on the live router with a new/large config — dnsmasq --test -C /etc/dnsmasq.conf FIRST, keep a backup, apply atomically so a mid-run crash rolls back instead of bricking resolution. Rebuilt over SSH via the telep-mainframe jump host (mainframe LAN 192.168.1.123 → router LAN 192.168.1.1): WiFi telep1 (WPA3/sae, both radios, net lan) + telep-cc (WPA2/psk2, hidden, isolate → net cams), country HU; cams network = static 192.168.30.1/24 on br-cams + DHCP + reservations (telep_cam1 18:69:45:a9:01:25→.119, telep_cam2 c0:3a:55:5c:8b:33→.139); new cams firewall zone + forwardings cams→wan (cameras get internet — REVERSES the old no-internet posture) and lan→cams; DNS query logging restored (logqueries=1, /tmp/dnsmasq-queries.log, /etc/dns-log-rotate.sh now truncates in place at 20MB — no dnsmasq restart on rotate, hourly cron); IPv6 LAN-disable re-applied (wan6.disabled=1, dhcp.lan.ra/dhcpv6/ndp=disabled — a fresh OpenWRT re-triggers 2026-07-28-ipv6-slow-internet); Tailscale reinstalled (opkg v1.80.3 + kmod-tun), re-joined tag:telep --ssh --accept-dns=false via an API-minted auth key, LuCI re-served --https=443 http://127.0.0.1:80, mainframe ed25519 pubkey added to router authorized keys. ⚠️ Router tailnet IP CHANGED 100.115.194.51 → 100.69.112.32 (fresh node registration; MagicDNS telep-router.taild4189d.ts.net stable); stale telep-router + telep-router-1 dupes deleted from the tailnet; updated the IP in homelab, telep-router, TOPICS, SESSION-HANDOVER and 2026-07-31-wifi-qr-code-sheet. Config backup now EXISTS: mainframe:/home/levander/telep-router-config-backup-20260803.tar.gz (13KB, sysupgrade -b); restore via LuCI → System → Backup/Flash Firmware → Restore. Verified after rebuild: WiFi APs up, both cameras reachable + all 4 feeds into Frigate (go2rtc), frigate-notify + camwall active, internet OK, DNS log growing, Tailscale reachable. NOT restored (deliberately): the casino blocker — redo only with offline config validation. Cross-linked from telep-router, telep-mainframe, homelab — 2026-08-03-telep-router-factory-reset-recovery

2026-07-31

• ⭐ thermoprint BLE blocker ROOT-CAUSED: the mainframe’s onboard Intel AX210 (hci0, USB 8087:0032, BD 7C:50:79:07:A9:53) saw ZERO BLE devices — not even stray phones — because there is NO ANTENNA plugged into the AX210 M.2 card (WiFi+BT share the u.FL connector; no antenna = no RF receive). The controller powers on and reports UP RUNNING / le (chip init needs no antenna) but hcitool -i hci0 lescan → Set scan parameters failed: Input/output error and btmgmt find -l finds nothing. The “zero devices incl. strays” symptom is the antenna tell. A detour of OS-side fixes was tried and was INEFFECTIVE (couldn’t fix a physical fault): controller power-cycle, modprobe -r/+ btusb firmware reload, USB unbind/rebind of 1-14, firmware-currency check (firmware-iwlwifi 20250410-2, BT fw ibt-0041-0041.sfi build 81864 — latest), kernel 6.12.95→6.12.100, full reboot. FIX = plug the antenna into the AX210 u.FL connector (USB BLE dongle → hci1 only as fallback), then cd /home/levander/thermoprint && ~/.bun/bin/bun run packages/cli/src/index.ts discover (P15 powered/awake/≤10m) and continue the plan. Lesson: check physical RF (antenna) BEFORE firmware/kernel/reboot when a scan sees zero devices incl. strays. thermoprint SW fully staged (Bun 1.3.14, repo cloned, bun install 672 pkgs incl sharp+@stoprocent/noble, setcap on bun) — only the RF path is missing. Cross-linked from telep-mainframe, 2026-07-31-thermoprint-appliance-spec, 2026-07-31-thermoprint-appliance-plan — 2026-07-31-telep-mainframe-ax210-ble-scan-broken
• Recorded telep-mainframe reboot-safety facts (verified 2026-07-31, kernel 6.12.95→6.12.100): root LUKS (nvme0n1p3) auto-unlocks via clevis + TPM2 with NO pcr_ids binding ({"hash":"sha256","key":"ecc"}) → kernel/firmware/bootloader updates do NOT break auto-unlock; clevis IS in the initramfs; there is NO dropbear remote-unlock fallback (auto-unlock is the only unattended path — keep the binding intact). NVIDIA is DKMS (nvidia-current 550.163.01), rebuilt for 6.12.100 (still needs Secure Boot off). The reboot recovered all services cleanly: camwall, knowledgebase, frigate, kb-qdrant. Added as a new “Boot & reboot safety” section on telep-mainframe — telep-mainframe
• ⭐ Camwall lag ROOT-CAUSED to saturated 2.4 GHz camera WiFi (bufferbloat), NOT an mpv/go2rtc problem — supersedes the earlier player-tuning assumption. The “insanely delayed and laggy” camera-wall panes (up to ~12 s drift, worst on DÉL/South) survived an mpv restart AND a full rebuild to a go2rtc MSE/chromium wall → the bottleneck is upstream of the player. Evidence chain (verified on telep-mainframe + OpenWrt root@100.115.194.51): ping from mainframe to both Tapo cameras (DÉL 192.168.30.119, ÉSZAK 192.168.30.139) = 350–420 ms avg / ~700 ms peaks / mdev 130–190 ms / 0% loss (local should be <5 ms; high-latency+0%-loss = bufferbloat, queueing under load, not a lossy link) → channel survey: camera SSID telep-cc is phy1-ap1 on 2.4 GHz ch1 / 20 MHz, iw survey busy/active = 26,960,546/27,263,522 = 98.9% busy, 86% of active is receive, noise floor -106 dBm (clean) → so it’s self-congestion from the cameras’ own uplink, NOT external interference (.119 signal -45 dBm good, .139 -68 dBm weak w/ tx retries) → Frigate config (~/nvr/frigate/config.yml): each of 4 lens-streams runs sub→detect (5fps) + main→record full-HD continuous 24/7 = 4 continuous HD record streams over one saturated channel. Ranked fixes: (1) get cameras off 2.4 GHz — wire Ethernet/PoE (best, kills the bottleneck) or 5 GHz; (2) cut record bitrate/res (camera-side changes hard remotely: pytapo has no encoder/GOP setter, ONVIF flaky/partial-auth on these Tapos); (3) changing WiFi channel WON’T help (self-congestion, clean noise floor). Separate additive ~2 s substream keyframe/GOP floor. ⚠️ Discrepancy surfaced: Recording still says record.retain.days:0 (events-only) but config now records 24/7 — reconcile. Cross-linked from telep-mainframe, 2026-07-28-camwall-4-substream-composite, SESSION-HANDOVER — 2026-07-31-camwall-lag-24ghz-wifi-bufferbloat
• Printable Wi-Fi QR code sheets on the Mac — reusable how-to. Pipeline: read ssid/key live from telep-router (uci show wireless; main = telep1 SAE/WPA3, cameras = telep-cc psk2 with a leaked PSK to rotate first) → build an A4 HTML card with an inline SVG QR via segno.helpers.make_wifi(security="WPA") → render to PDF with headless Google Chrome (--headless --print-to-pdf, no wkhtmltopdf/qrencode on this box) → verify+scan once → print. Payload format WIFI:T:WPA;S:<ssid>;P:<pw>;; (escape \ ; , : "); T:WPA also joins WPA3/SAE. Three gotchas: PEP 668 python3.14 is externally-managed so use a throwaway venv (NOT --break-system-packages); segno’s SVG writer emits BYTES so use io.BytesIO().getvalue().decode() not StringIO; T:WPA covers WPA3. Passwords kept OUT of the vault — 2026-07-31-wifi-qr-code-sheet

2026-07-30

• OBD2 ELM327 bridge — “phone can’t connect / RFCOMM open fails” runbook. Root-caused the recurring failure of the Mac’s ELM327-over-TCP bridge (~/obd-bridge/obdbridge, launchd com.levander.obdbridge, TCP 100.83.222.120:35000) to a stale macOS Bluetooth SDP cache after the bus-powered dongle power-cycles (car ignition off). bridge.err loops openRFCOMMChannelSync failed: ret=0x-1ffffd44 (= kIOReturnError 0x2bc) because system_profiler SPBluetoothDataType shows OBDII advertising Services: 0x802000 < Braille ACL > instead of Serial Port (SPP) → the bridge’s SDP lookup finds no channel, falls back to hardcoded RFCOMM channel 1, which fails. Key diagnosis: a listening TCP 35000 (lsof) does NOT mean the serial link is up — check bridge.err; and it’s NOT a /dev/tty.*//dev/cu.* claimant problem. Fix (device-scoped, keeps BT keyboard/mouse alive): blueutil --disconnect 00-1d-a5-68-98-8b → wait 3s → blueutil --connect … → launchctl kickstart -k gui/$(id -u)/com.levander.obdbridge, then tail -f bridge.err until RFCOMM connected→client connected. Restarting the bridge alone does NOT fix it (re-runs the same failing SDP query). Gotcha: blueutil --connect can BLOCK to a 2-min timeout on a slow re-establish — poll with non-blocking --is-connected, don’t re-run --connect. blueutil at /opt/homebrew/bin/blueutil v2.13.0. Recurs on every ignition-off — 2026-07-30-obd2-elm327-bridge-rfcomm-fix
• KB Hungarian translation — prioritized corpus COMPLETE. The KB manual markdown is now translated to Hungarian for the full prioritized set: 820/820 prioritized pages (956 total .hu.md incl. consolidated). Manuals done: chevy-tracker/geo-tracker-repair 434, suzuki-vitara/workshop-1988-1998 293, 5door-supplement 61, supplement-61a40 23, suzuki-sidekick/wiring-1996 9, all kick-fix/* how-tos, consolidated. Deliberately skipped (low value): parts-catalogue-89-98, body-measurements, wiring-diagrams booklet, owners-1995, kickfix-docs/* — addable later. Served read-only at /hu/<page_url> via mkdocs-static-i18n suffix mode (English fallback for untranslated); NOT indexed for search (bge-large is English-only). Method: per-page .hu.md siblings written by session-model FORK subagents over disjoint page ranges, each page flushed to disk the INSTANT it’s translated (heredoc/ssh-stdin/scp piping to survive quotes/$/backticks) + grep-verified — crash-safe: the first bulk attempt lost everything to a mid-run API drop (nothing flushed); after flush-per-page, 3 API “connection closed” drops AND a hard “session usage limit” cost ZERO completed work (skip-if-.hu.md-exists resumes). ~6–12k tokens/page. Faithfulness contract: translate prose+headings, keep VERBATIM all numbers/units/torque/clearance/resistance/voltage/part-numbers/DTC/wire-colors/mileage/URLs + every image ref; tables labels translated + numeric cells verbatim; don’t translate garbled OCR tables; lookup/paint-mark CODES stay English; banner > AI-forditas... on every page. Two open judgment calls: page 198 bearing paint-mark colors translated (Green→Zöld) — revert if codes; page 202 one garbled OCR TOC cell dropped — 2026-07-30-kb-hungarian-translation-complete
• Wiring pivot: netlist extraction RETIRED → consolidated scan gallery + zoom. User deemed the AI netlist extraction too inaccurate for agents. RETIRED it reversibly (nothing deleted, backups kept): hooks._append_wiring no longer called (on-page netlist tables + Cytoscape graphs gone), 65 .wires.txt sidecars moved to /home/levander/wiring-retired/, wiring-charts.md + write_charts_index disabled, the 2 wiring manuals reindexed so Qdrant//api/search returns no extracted netlist text (marker count 0). REPLACED with the accurate thing — a consolidated scan library at /wiring-gallery/: 480 curated genuine wiring-diagram images (auto-included the 5 dedicated wiring manuals + vision-verified the ambiguous ones, dropping 147 photos/icons/exploded-views), 320px thumbs grouped vehicle→manual, each thumbnail links to the full source scan; home-page nav link + cited in kb-agent-api as cluzter’s tracing-underlay source. Enabled mkdocs-glightbox (click-to-zoom, full-screen pan/zoom) — fixed the “diagrams look cut off” issue (they were being shrunk into the ~60em content column). selection.json + thumbs at /home/levander/wiring-collection/. Distinction: retirement kills the AI netlists, NOT the OCR’d scans (which remain good, cluzter’s underlay source) — 2026-07-30-kb-wiring-gallery-pivot-complete
• Updated SESSION-HANDOVER (new 2026-07-30 delta: HU corpus complete + wiring gallery pivot) and TOPICS
• Clarified the scope of the wiring-extraction retirement in kb-agent-api — it retires the AI-extracted netlists (.wiring.json, wires.txt sidecars, generated connection tables, Mermaid circuit graphs), not the OCR’d scanned diagram pages, which remain good and are now the authoritative scan source for cluzter’s tracing underlays. The blanket “ignore any wiring content” read as “the wiring scans are worthless”, which would have thrown away the useful half — kb-agent-api, 2026-07-29-kb-wiring-extraction-v1-complete
• New consumer of the Vitara/Sidekick scan library: cluzter, a browser wiring-diagram editor for the 1995 Geo Tracker (G16B). It loads a scanned manual page as an underlay layer, applies a two-point calibration to it (world-mm-per-paper-mm ratio), and traces over it by hand — deliberately instead of importing an extracted netlist. Its design cites this project’s own finding that scan resolution is the ceiling (1632×808 → wire-colour stripe letters like B/R vs B/Bl fail ~40%) as part of the reason a human traces rather than a pipeline — cluzter, cluzter-wiring-gotchas

2026-07-29

• KB semantic-search frontend v1 COMPLETE — a server-side semantic search page is now the KB’s front door, replacing mkdocs’ slow client-side lunr search. Runs IN the existing knowledgebase Flask app (no new service); installed CPU-torch 2.13.0+cpu (NO CUDA) + sentence-transformers 5.6.1 + qdrant-client 1.18.0 + transformers 5.14.1 + numpy 2.5.1 INTO the app venv so kb-vectors/search.py imports in-process (~1.63 GB RSS). app.py: sys.path.insert(0,"/home/levander/kb-vectors"), module global QCLIENT=QdrantClient(127.0.0.1:6333), main() warms the bge-large embedder once + torch.set_num_threads(4) (lazy singleton so import app stays cheap). New routes: GET /api/search?q=&folder=&manual=&limit= → JSON {took_ms,count,hits:[{heading,manual_id,vehicle,page_url,snippet,score,type}]} (dedupe-by-page + snippet/highlight + classify + vehicle label); /+/search → search-first UI (dark, autofocus, debounced fetch, type badges, vehicle filter, ↑/↓/Enter nav, i18n); /browse → mkdocs index (kept); catch-all still serves site/ (site() no longer @app.route("/")). New searchui.py (dedupe_by_page/make_snippet/highlight [XSS-safe]/classify/vehicle_label) + tests; i18n hu+en parity. Backups app.py.bak-search-fe/app.py.bak-search-ui. Verified live: warm ~54 ms; real semantic wins (“why won’t the fuel pump prime”→“Pump is dead, now what” 0.728, “fuel cut controller”→FUEL CUT SYSTEM 0.726, “stop light switch”→Stop Light Switch Adjustment 0.681); folder filter works; wiring deep-links to kinyert-huzalozasi-adatok-ai; 141 tests pass; non-destructive. v1 = retrieval only, NOT RAG synthesis — 2026-07-29-kb-semantic-search-frontend-complete
• ⭐ Gotcha — community mkdocs-material’s client-side lunr search is fundamentally unfixable at scale. The 5.4 MB / 6683-section index is built in-browser on every visit → hangs on “Initializing search”; prebuild_index is Insiders-only (removed from community); dropping the HU search lang either doubles the index (reconfigure_search:false→13366 docs) or removes the HU site — no config path exists. The ONLY real fix was replacing it with a server backend. The semantic backend already existed (2026-07-24-kb-vectorize-complete); installing CPU-torch INTO the app venv let it run in-process (no new service) — 2026-07-29-kb-semantic-search-frontend-complete
• ⭐ Graceful degradation verified — Qdrant stopped → /api/search 503 but home + manual pages still 200 and the service didn’t crash (NRestarts=0); restored → search back. The mkdocs site is independent of the search backend. Phase-2 candidates: answer synthesis via claude (cited), hybrid semantic+BM25 fusion, retire the lunr search on /browse entirely — 2026-07-29-kb-semantic-search-frontend-complete
• Updated 2026-07-29-kb-semantic-search-frontend-complete (new v1-complete note), SESSION-HANDOVER (new KB search-frontend delta) and TOPICS
• Wiring extraction SCALED to real content (v1.1) — 5door-supplement 55/55 diagram pages. Beyond the thin wiring-diagrams booklet (10/10), the full suzuki-vitara/5door-supplement manual is now 55/55 diagram pages extracted, rendered on-page, and vectorized/searchable (Qdrant ~377 points for this manual; 55 pages carry the “Diagram data (extracted)” marker). Finding confirmed at scale: clean single-system circuits (stop/tail/interior lights, wipers, defogger, washers, DTC sensor circuits 037–043) trace well at high/med confidence — clearly beating the ~60% dense-booklet ceiling; bundled harness/operation diagrams (central-locking, power-window, 060 body-electrical multi-market) stay honestly conservative (med/low, power-path + representative edges); routing/flowchart/block/mechanical are labels-only — NO fabrication. Honest gap: page 044’s crank-angle-sensor “circuit” is only a flowchart+pinout in the scan (not drawn) → left labels-only. Reusable box tooling: cvtrace.py/legend.py/diagram_index.py/emit.py/run.py + per-page drivers emit_driver.py/mech_emit.py/build_page.py/specs/; vision+fusion subagent-driven. Next targets: workshop-manual electrical sections + Sidekick wiring-1996 — 2026-07-29-kb-wiring-extraction-v1-complete
• Added an on-page render hook + a NEW Mermaid graphical circuit renderer + fixed a folder-root 404. hooks.py on_page_markdown (merged with the kickfix deeplink rewriter + lazy-image) appends a per-page ”## Kinyert huzalozási adatok (AI)” section from the .wiring.json — an HTML connections table (Honnan|Huzalszín|Hová|Bizt.) + components + systems + AI-provenance note (non-destructive; source md untouched). NEW Mermaid renderer: enabled pymdownx.superfences + a mermaid custom fence in mkdocs.yml (material 9.7.7, client-side); the hook also emits a per-page ”### Áramköri gráf (AI)” graph LR (nodes=components, edges labeled by decoded wire color, LOW-confidence edges dashed -.->, subgraph per subsystem, ~30-edge cap) across all 62+ processed pages (render-layer, no re-extraction). gen_index.py extended to generate a landing index.md for every folder + manual dir (idempotent <!-- generated-index --> marker) so /suzuki-vitara/5door-supplement/ etc. no longer 404 under use_directory_urls (backup gen_index.py.bak-404) — 2026-07-29-kb-wiring-extraction-v1-complete
• ⚠️ Mermaid render markup-verified but NOT live-browser-confirmed — verified the emitted markup + Material’s mermaid loader bundle, but not a live render (Chrome extension not connected; Material lazy-loads mermaid from CDN → needs browser internet). Open follow-up. Also: empty pages (divider/cover/TOC e.g. 017-wiring-diagram, geo-tracker cover) are OCR artifacts, not bugs — the thin booklet’s real schematics were concentrated on page 019 — 2026-07-29-kb-wiring-extraction-v1-complete
• Fixed the camwall’s ~10–13 s latency → ~1–2 s. Isolated to mpv’s single-composite movie= lavfi buffering (camera + go2rtc were each ~1 s). Rearchitected /usr/local/bin/camwall-mpv.sh into 4 tiled single-input low-latency mpv instances (per-quadrant --geometry, --profile=low-latency --cache=no, per-pane drawtext labels) → ~1–2 s per pane, stable. Backup camwall-mpv.sh.bak-latency. ⚠️ Two known regressions (open): (a) WiFi strip now confined to the BL pane (was full-width) — cosmetic; (b) watchdog freeze-detection now only covers the one IPC-socket-owning pane (cam2/BL), not all 4 — security-relevant, fix via per-pane IPC sockets or framebuffer-based per-quadrant staleness — 2026-07-28-camwall-4-substream-composite
• Updated 2026-07-29-kb-wiring-extraction-v1-complete (new v1.1 scale-out + render-layer section), 2026-07-28-camwall-4-substream-composite (latency-fix section), SESSION-HANDOVER (new 2026-07-29 scale/render + camwall-latency deltas) and TOPICS
• KB wiring-diagram extraction v1 COMPLETE — hybrid classical-CV + vision, proven end-to-end on 1 pilot page. New module /home/levander/wiring-extract/ on telep-mainframe (house-rules-clean): legend.py+legend.json (13 Suzuki wire colors transcribed from scan _page_8_Picture_7.jpeg, decode_color("B/Y")->"Black/Yellow"), diagram_index.py (list_diagram_pages() — 10/21 Vitara wiring pages are diagrams / 38 images), cvtrace.py the connectivity engine under ocr/venv (cv2 4.11 + scikit-image 0.26 [installed this session] + surya; Otsu→despeckle→TEXT mask via surya DetectionPredictor→SYMBOL mask via vision boxes→skeletonize→graph→dot-vs-crossing classifier→union-find nets), emit.py (.wiring.json+<page>.wires.txt+.lines), run.py (status/reindex/search/run_page). kb-vectors chunker.py PATCHED (backup chunker.py.bak-wiring) to merge .wires.txt onto the same-page_url chunks (never its own page); 51 kb-vectors tests pass. Non-destructive (source md/scans/clusters.json md5-unchanged). Coverage 1/10 Vitara diagram pages; remaining is pure scale-out — 2026-07-29-kb-wiring-extraction-v1-complete
• ⭐ Gotcha — naive CV path-tracing is a NO-GO on 1632px scans; masked CV is a GO. Skeletonizing the raw binary → 161 false junctions / 502 segments because TEXT (titles, labels, fuse ratings) and COMPONENT SYMBOLS (fuse coils, switch boxes) skeletonize into dense false-junction clusters — line-following worked, connectivity didn’t. Masking text (surya auto-detect ~35 boxes) + component symbols (vision-supplied boxes) BEFORE skeletonizing → junctions 161→37 (-77%), conductors intact. Symbols were the bigger contaminant (161→106 text-only→37 with symbols). Division of labor: vision locates components/terminals/labels + reads colors → boxes passed to cvtrace as suppress_boxes → cvtrace returns nets → vision fuses semantics + reconnects sub-pixel gaps, never fabricating untraceable connections — 2026-07-29-kb-wiring-extraction-v1-complete
• ⭐ Gotcha — scan resolution is the ceiling, not the model; and the search win is real but scoped. 1632×808 @ ~9 systems/page = few px/label; wire-color stripe letters (B/R vs B/Bl) fail (~60%); 300–600 DPI rescans or single-circuit pages → ~90%+ (future lever). Pilot page 019-light-head-… went from unfindable (0 text) to #1 within its manual for “fuel cut controller”/“IC regulator” via the indexed sidecar — NOT global top-50 (a terse structured list can’t out-rank prose). Vision+fusion is subagent-driven (session model, not box claude -p/not a local VLM) at ~13 min + ~110k tokens PER PAGE → scaling all pages is the token-heavy remaining step — 2026-07-29-kb-wiring-extraction-v1-complete
• Updated SESSION-HANDOVER (new wiring-extraction-v1 delta section: built, 1/10 coverage, scale-out pending, the naive-vs-masked CV finding) and TOPICS (new CV & Diagram Understanding topic + Knowledge Tooling link)

2026-07-28

• Root-caused the Mac’s intermittent “slow internet” to broken IPv6 advertised on a v6-dead link (FIXED). telep-router advertised a global IPv6 prefix (2a00:1110:210:7653::/64) + ULA (fd83:5e9d:98ca::/64) via RA/DHCPv6, but IPv6 had NO working upstream (fixed-wireless + double-NAT). macOS prefers v6 (RFC 6724 / Happy Eyeballs) → stalled 2–7 s per dual-stack site → IPv4 fallback. Signature: curl -w time_connect 2–7 s while time_namelookup instant; curl -4 instant, curl -6/ping6 dead. Fix: uci disable dhcp.lan.ra/dhcpv6/ndp + network.wan6, commit + odhcpd restart + network reload → connect 2–7 s → 0.02–0.06 s. Reversible (backups dhcp.bak-ipv6fix/network.bak-ipv6fix) — 2026-07-28-ipv6-slow-internet
• ⭐ Gotcha — advertising IPv6 to a LAN with no working v6 upstream makes macOS “slow”, not the bandwidth. Fixed-wireless + double-NAT almost never carries IPv6; don’t advertise it. Diagnose by splitting DNS from connect in curl -w and comparing curl -4 vs curl -6 — 2026-07-28-ipv6-slow-internet
• Installed the net-monitor link-quality logger — /home/levander/net-monitor/probe.sh + user crontab (every 5 min) → netlog.csv (timestamp_iso,rtt_avg_ms,rtt_max_ms,loss_pct,dl_mbps,loaded_rtt_ms); pings 1.1.1.1 each run + a 20 MB throughput/bufferbloat sample near the top of the hour (~480 MB/day). Purpose: characterize suspected fixed-wireless peak-hour degradation. Review column -s, -t netlog.csv | tail — 2026-07-28-net-monitor
• Added a second camera (ÉSZAK) and moved the camwall OFF birdseye to a 4-substream 2×2 composite. New Tapo TC47 ÉSZAK (192.168.30.139, telep_cam3 fix / telep_cam4 ptz) added to Frigate alongside DÉL (telep_cam1/telep_cam2). camwall-mpv.sh rewritten to composite the 4 go2rtc substreams into a grouped 2×2 (DÉL left, ÉSZAK right) via mpv --vf lavfi (cam1_sub primary + cam2/3/4 via movie=), 4 drawtext labels, TOP KÉPEK dropped, wifi-usage IPC strip kept. Keep --hwdec=no. Backup camwall-mpv.sh.bak-2x2 — 2026-07-28-camwall-4-substream-composite
• ⭐ Gotcha — Frigate birdseye is UNRELIABLE as a fixed >2-cam wall source (flip-flops between showing 2 and 4 cameras); composite the substreams directly instead. mpv movie= rtsp URLs need colons escaped (rtsp\://...\:8554/...), can’t take rtsp_transport (benign 461), and a movie= pane can freeze on go2rtc restart (watchdog only covers the primary input) — 2026-07-28-camwall-4-substream-composite
• Excluded telep_cam3 (indoor-aimed) from frigate-notify alerts — correct key is frigate.cameras.exclude: [telep_cam3] (nested under the frigate: block, by camera id), NOT alerts.cameras.block / not top-level cameras:. frigate-notify v0.5.4, config /home/levander/nvr/frigate-notify/config.yml, backup config.yml.bak-cam3excl — 2026-07-28-frigate-notify-camera-exclude
• ⭐ Gotcha — frigate-notify’s koanf SILENTLY IGNORES misplaced/unknown keys; “Config validated! / App ready!” does NOT prove a filter works. A top-level cameras.exclude logged “validated” and still sent the alert; only under frigate: did it log Event dropped - Camera Excluded. Always verify suppression against a real event, not config validation — 2026-07-28-frigate-notify-camera-exclude
• Updated telep-router (new IPv6-disabled section + double-NAT-undo open item), telep-mainframe (second camera, camwall-off-birdseye, per-camera alert exclude), SESSION-HANDOVER (new “Since last handover (2026-07-28)” delta block, camwall/Frigate inline updates, 3 new key gotchas, KB HU-toggle in OPEN/next) and TOPICS
• Noted the KB Hungarian/English toggle is MID-BRAINSTORM (paused). User chose Approach A: mkdocs-static-i18n + Material language selector for the static site + a shared lang cookie for the Flask pages; HU/EN over UI chrome + cluster labels + generated consolidated articles; source manuals stay English. No spec doc written yet — resume via the brainstorming → writing-plans flow — Since last handover (2026-07-28)

2026-07-25

• KB consolidated generation phase 3 COMPLETE — full knowledge system now done end-to-end. On-demand consolidated repair-article generation from a cross-source cluster via claude -p --model opus, in a draft → human review → publish workflow. Proven live. Flow: /semantics cluster “Generál” → POST /generate/<id> → serialized background worker gathers member section .md + images → rewrites image refs to source-unique names → never-invent prompt → claude -p --model opus → DRAFT (drafts/<slug>/ = draft.md + meta.json + images) + speccheck → review at /drafts/<slug> (AI-provenance banner + speccheck warnings + Közzététel/Elvetés) → publish → docs/consolidated/<slug>/ (build_site atomic swap, browsable, vectorizable) or discard. Code /home/levander/knowledgebase/ (genprompt.py/speccheck.py/gen.py/kbgen.py + tests) — co-located in knowledgebase/ NOT kb-vectors/ to avoid cross-venv imports; routes on the existing KB Flask app; NO new service. Claude natively installed /home/levander/.local/bin/claude, logged in as levander (subscription), --model opus explicit + fail-loud. Two live articles: engine-cooling PUBLISHED at knowledgebase.taild4189d.ts.net/consolidated/engine-cooling/ (3 real source tags, 49 images, AI header); parking-brake DRAFT (40 citations, verbatim specs 20 to 25 kg (44 to 55 lbs)/7 to 9 notches, nothing invented, flagged a both-ways disagreement + a source that contributed nothing). Read-only held. ⚠️ parking-brake draft still PENDING REVIEW — publish/discard from /drafts/<slug> — 2026-07-25-kb-consolidated-gen-phase3-complete
• ⭐ Gotcha — image basename collision across manuals silently swaps in the WRONG diagram. Every marker-OCR’d manual names images the same (_page_62_Figure_16.jpeg), so a consolidated article pulling from multiple manuals would copy the wrong manual’s figure (a real accuracy failure). Fix: before generation rewrite each source’s image refs to SOURCE-UNIQUE names (safe_slug(manual_id) + "__" + basename) + an image_map, so copies resolve 1:1 to the correct source image (verified 4→4) — 2026-07-25-kb-consolidated-gen-phase3-complete
• ⭐ Gotcha — a number-token spec-check’s “false flags” are mostly REAL verbatim-deviations; keep it ADVISORY, never a publish gate. speccheck flags draft numbers absent from sources; these fire mainly when the model reformats a number (1,000→1000, 40→40.0) which the never-invent prompt forbids — so a “false flag” is usually a real deviation. Also harmlessly flags year-digits from citation slugs. Human review is the backstop. Also: start_worker must be idempotent (else concurrent workers break serialization); safe_slug guarantees a non-empty single path component so a degenerate label can’t rmtree the whole consolidated/; claude -p on the box = native installer → ~/.local/bin + claude login (subscription) + headless claude -p --model opus on stdin — 2026-07-25-kb-consolidated-gen-phase3-complete
• Updated SESSION-HANDOVER (new phase-3 “Generál” generation/drafts/consolidated section, flipped phase-2’s “phase 3 NOT started” note to DONE, the two ⭐ gotchas in Key gotchas, full-system-complete + pending parking-brake draft in OPEN/next) and TOPICS

2026-07-24

• KB semantics phase 2 COMPLETE — fully-local semantic SECTION clustering over the phase-1 vectors + a read-only web explorer (“Szemantika”), NO LLM. Pipeline: scroll all 3252 chunk vectors from Qdrant manuals (READ-ONLY) → group by (manual_id,page) → L2-normalized mean-pool → ~1248 SECTION vectors → HDBSCAN(min_cluster_size=2, min_samples=1, euclidean≈cosine) → representative-heading labels → clusters.json. 194 clusters (post boilerplate filter), 24 cross-SOURCE, 631 noise singletons (noise EXPECTED for a dedup goal). Top clusters are real cross-make overlaps (WINDOW REGULATOR spanning chevy-tracker + suzuki-vitara/5door + workshop; ENGINE COOLING, PARKING BRAKE, THROTTLE BODY). Code /home/levander/kb-vectors/ (sectionvecs.py/labels.py/cluster.py/kbclust.py + tests, CLI build/list/show); explorer /semantics + /api/clusters folded into the EXISTING knowledgebase Flask app (NO new service) at knowledgebase.taild4189d.ts.net/semantics; 🧭 Szemantika dashboard tile. Ranked by cross-SOURCE span so dedup targets surface first; labels pluggable behind make_label for a future claude -p relabel. Phase 3 (claude -p generation) still NOT started — clusters.json is its retrieval bundle — 2026-07-24-kb-semantics-phase2-complete
• ⭐ Gotcha — HDBSCAN’s min_samples defaults to min_cluster_size, so leaving it default made ALL sections noise (0 clusters). Must set min_samples=1 explicitly for min_cluster_size=2 to form clusters — looked like “no structure in the data” but was purely the two knobs being coupled by default — 2026-07-24-kb-semantics-phase2-complete
• ⭐ Gotcha — cross-source span was inflated because manual_id is <folder>/<manual> and the kick-fix archive is ONE source split into ~25 topic-subfolders (each counted as a distinct “manual”). Fix: define span by distinct TOP-LEVEL source folder (source_of = manual_id.split("/",1)[0]) so kick-fix collapses to 1 source while genuinely-separate manuals still count; keep source_manuals for display, rank by (span=distinct sources, #manuals, size). Added an is_boilerplate() label stoplist (22 clusters dropped — cover pages “SUZUKI”, “GENERAL DESCRIPTION”/“DIAGNOSIS”). 51% noise is EXPECTED/correct for dedup. Whole pipeline is READ-ONLY scroll → local clusters.json, never upsert/delete — 2026-07-24-kb-semantics-phase2-complete
• Updated SESSION-HANDOVER (new Szemantika/semantics service section, phase-2-DONE + phase-3-still-open, the two starred min_samples + span-inflation gotchas in Key gotchas, OPEN/next KB-deferred) and TOPICS
• Live state ~14:00: box UP (booted 13:55 after a physical power-on following a morning mains outage); clevis TPM auto-unlock worked again unattended; all services active (knowledgebase, home-portal, wifi-usage, intruder-alarm, camwall, both tailnet nodes, kb-qdrant + frigate Up); Qdrant manuals at 2839 points — SESSION-HANDOVER
• Upgraded telep-router Tailscale 1.80.3 → 1.98.9 — hand-installed a static arm64 build (the opkg feed only carries 1.80.3) via a setsid-detached self-reverting swap script that rolls back to /overlay/ts-backup/ if the new binary isn’t online in 45s. Router online, 8 peers, verified over SSH served by the new binary. ⚠️ opkg still RECORDS 1.80.3-r1 → a future opkg upgrade could clobber the new binary. Gotcha: on OpenWrt/busybox use setsid not nohup (absent) + a persisted /overlay backup + auto-revert so a bad swap can’t lock you out of your only remote foothold — Since last handover (2026-07-24)
• ⚠️ New observation — a daily ~06:00 mains die-off pattern. The box (and the router with it) appears to lose power around 06:00 most mornings (Jul 19 ~06:59, Jul 23 ~06:36, Jul 24 morning). The router dies too = MAINS-level, distinct from the PSU-load resets (2026-07-24-psu-load-resets). Suspect a scheduled utility relay (Hungarian vezérelt/GEO off-peak tariff) or a timed high-draw appliance browning the shared circuit — NOT confirmed; confirm via journalctl --list-boots vs router-reboot regularity — Since last handover (2026-07-24)
• Remote-wake (WOL) is NOT possible as configured — after a mains outage the box needs a physical power-button press; WOL never enabled (BIOS+NIC), the router has no etherwake/wakeonlan, and the box’s MAC is unknown after the DHCP lease expires (no static reservation). Better fix than WOL: BIOS Restore on AC Power Loss → Power On so the box auto-recovers from the daily blip like the router (one physical BIOS visit) — Since last handover (2026-07-24)
• In-flight (check next session): ~/ocr/resume_extras.sh (setsid-detached, sentinels .done/.failed) — body-measurements already OCR’d+seeded into suzuki-vitara; this OCRs the Sidekick parts-catalogue → seeds suzuki-sidekick/parts-catalogue-89-98 → rebuilds the site → vectorizes BOTH new manuals into Qdrant (thread-capped OMP_NUM_THREADS=4 + nice). Re-run bash ~/ocr/resume_extras.sh if it died — Since last handover (2026-07-24)
• Updated SESSION-HANDOVER with a “Since last handover” delta block (router tailscale upgrade, daily-06:00 mains pattern, WOL, in-flight OCR job) + a live-state callout, added the ⭐ setsid-not-nohup gotcha, the daily-mains and opkg-clobber warnings, and carried the power-hardware + KB-deferred items into OPEN/next
• Expanded the knowledgebase manual library — many OCR’d manuals now under docs/<folder>/<manual>/: suzuki-vitara (5door-supplement, supplement-61a40/99501, workshop-1988-1998 835pg, wiring-diagrams/99512, body-measurements), chevy-tracker (owners-1995, geo-tracker-repair), suzuki-sidekick (wiring-1996, parts-catalogue-89-98). OCR pipeline marker (~/ocr/venv, marker_single --force_ocr, TORCH_DEVICE=cuda) → seed_import.py → build; manuals queued as CHAINED nohup scripts (each waits on the prior’s .done sentinel) to serialize the single GPU. Dedup discipline: md5-dedup downloads BEFORE OCR — many were exact dupes / same-content-different-name (workshop ×2, 99501 ×2, parts-catalogue ×2); zips sometimes contain the PDF (sidekick wiring was a zip of wiring2.pdf) — Library expansion (2026-07-24)
• Ingested the kick-fix.com archive — a 112 MB offline archive of the defunct kick-fix.com Suzuki Sidekick/Tracker repair site into docs/kick-fix/ (25 topics, 127 articles, 741 images, 8 text-layer PDFs; 27 image-only PDFs deferred for OCR). Dead-site archive → no ToS/robots issue. Script /home/levander/knowledgebase/kickfix_ingest.py (KB venv + markdownify + beautifulsoup4 + pypdfium2) — 2026-07-24-kickfix-ingest
• ⭐ Gotcha — markdownify SILENTLY DROPS <img> inside <table> cells. Old sites use tables for LAYOUT so nearly all figures were copied to disk but never referenced (11 refs for 122 pages). FIX: before markdownify, .unwrap() the layout containers (table/tbody/tr/td/th/center/font) so text+images flow linearly, rewrite <img src> to the copied basename in place, wrap imgs in own <p> (→ 528 refs); keep a trailing ## Figures only as a safety net — The two that cost real time
• ⭐ Gotcha — “See photo here” TEXT links (<a href="X.jpg">) are separate from inline <img>, 365/664 broken (targets never copied). FIX: also walk <a href>; if the URL-decoded, ../-resolved href is an existing image, copy under basename + rewrite href (keep as text link, don’t inline); leave external http(s)/mailto; drop dead archive-local hrefs (→ 365→0 broken, images 449→741). src/href need URL-decode (%20) + cross-topic ../ resolution — The two that cost real time
• Vectorization phase 1 COMPLETE — built /home/levander/kb-vectors/ (chunker/embed/index/search/kbvec CLI): local BAAI/bge-large-en-v1.5 CPU embeddings + Qdrant container kb-qdrant (127.0.0.1:6333, collection manuals, 1024-dim cosine), idempotent per-manual (delete-by-filter + uuid5 ids). 2839 chunks indexed across 33 manual/kick-fix collections; cross-source semantic search blends official manuals + kick-fix community guides. Chose Qdrant over embedded stores for payload filtering + UI + growth; suzuki-forums crawler DROPPED (Tollbit-gated). Foundation for phase 2 (semantic dedup) + 3 (claude -p generation) — 2026-07-24-kb-vectorize-complete
• ⭐ Root-caused a NEW, distinct power problem: PSU-can’t-sustain-peak-load resets (separate from the overnight mains outages). The box crash-rebooted ~3× in 30 min (04:11/04:20/04:32) under many heavy jobs at once (GPU OCR + CPU embedding + kick-fix conversion + Frigate). NOT thermal (CPU 39°C/GPU 47°C, no MCE), NOT mains — the router stayed up 14h+ straight through (the router-uptime cross-check is the discriminator; mains = router dies too). Dead-stop journal signature. The default 16-thread CPU embedding maxing all cores reset the box in 2–6 min every time. MITIGATION: cap to 4 threads (OMP_NUM_THREADS=4/torch.set_num_threads(4)) + nice + serialize heavy jobs → box stayed up 45+ min, finished the index. A UPS does NOT fix this; real fix = a bigger/better PSU — 2026-07-24-psu-load-resets
• clevis LUKS TPM2 auto-unlock VALIDATED IN PRODUCTION — the PSU reboot storm above doubled as the reboot test: the box self-recovered the encrypted root through all 3 unattended reboots (systemd-cryptsetup@nvme1n1p3_crypt on boot, root mounted, no passphrase). All services auto-recovered (knowledgebase, kb-qdrant, home/knowledgebase tailscale nodes, Frigate, camwall, wifi-usage, intruder-alarm). Reboot-test status: pending → PASSED in production — Validated in production (2026-07-24 reboot storm)
• Added a global dashboard home.taild4189d.ts.net (own tag:telep node, Hungarian “Elérhető rendszerek”, tiles Kamerák/Tudásbázis/Top képek/Router); the knowledgebase also became its own tag:telep node. Same reusable pattern: a dedicated second tailscaled per identity (--tun=userspace-networking, own --socket/--statedir) — 2026-07-24-global-dashboard
• Updated SESSION-HANDOVER (KB expansion + kick-fix + kb-vectors/Qdrant + global-dashboard sections, PSU-load power bullet, LUKS reboot-test PASSED) and added the starred markdownify-table-images, text-link-images, PSU-vs-mains and md5-dedup gotchas
• Set up TPM2 auto-unlock for telep-mainframe’s root LUKS — the follow-up to the mains-power work so an unattended reboot no longer stalls at the LUKS passphrase prompt (unreachable over SSH, no dropbear-initramfs). Root is LUKS2-on-LVM (/dev/nvme0n1p3, mapper nvme1n1p3_crypt, UUID a3a8e37d-79fa-484b-bc3f-40c56df95337); TPM2 /dev/tpmrm0 (MSFT0101), Secure Boot OFF. Bound the TPM with clevis (clevis-initramfs+clevis-tpm2, clevis luks bind -d /dev/nvme0n1p3 tpm2 '{}', empty PCRs) → keyslot 1 = TPM, keyslot 0 = passphrase kept as fallback; update-initramfs -u installs the clevis unlock hook so the disk auto-unseals at boot. Header backed up first at ~/telep/luks-header-nvme0n1p3-20260723.img — 2026-07-24-luks-tpm-autounlock
• Gotcha — Debian 13 stock initramfs (initramfs-tools) does NOT consume systemd-cryptenroll TPM2 tokens. Tried systemd-cryptenroll --tpm2-device=auto --tpm2-pcrs= first: it enrolls a valid TPM keyslot, but update-initramfs warns ignoring unknown option 'tpm2-device' and the box still prompts at boot (the classic cryptsetup hook can’t read a systemd-tpm2 token). Abandoned that path (wiped the slot with --wipe-slot=tpm2, no passphrase needed) and used clevis instead — [[2026-07-24-luks-tpm-autounlock#why-not-systemd-cryptenroll|Why not systemd-cryptenroll]]
• Gotcha — adding any new LUKS keyslot needs the existing passphrase (an agent can’t do it for the user) and is genuinely slow (argon2 KDF + TPM seal, tens of seconds); an impatient Ctrl-C orphans a systemd-crypt* process that SURVIVES holding the LUKS header lock AND /dev/tpmrm0 and re-broadcasts a systemd-ask-password — silently blocking the later clevis luks bind. Fix: SIGKILL the orphan, confirm fuser/lsof clear + passphrase slot intact, retry — Hard-won gotchas
• Honest incomplete state: reboot test still PENDING (needs console access — a failed unlock hangs with no SSH); auto-unlock only helps once the box BOOTS, and BIOS Restore on AC Power Loss is still stay-off so a mains cut still needs a button press (2026-07-23-mains-power-shutdowns); empty-PCR + Secure-Boot-off is an accepted tradeoff — protects against bare-drive theft, not whole-box theft. A passphrase-echo hygiene issue occurred → passphrase rotation suggested (no values in the vault) — complementary (honest state)
• Added a “Disk encryption / auto-unlock” note to the drives section and the two starred TPM/keyslot gotchas to SESSION-HANDOVER; cross-linked the follow-up from 2026-07-23-mains-power-shutdowns
• Built the knowledgebase — a tailnet-only manual library + OCR ingest portal on telep-mainframe, live at https://knowledgebase.taild4189d.ts.net as its own tailnet node (not a port on the mainframe). Renders OCR’d PDF manuals as an mkdocs-material static site (folder-tree browse, full-text search, dark mode, phone-responsive, all images loading="lazy") plus a Flask+waitress /upload portal that runs marker OCR on the GPU, splits the markdown into per-section pages, copies images and atomically rebuilds the site. Two enabled systemd units (knowledgebase.service app on 127.0.0.1:8092 as levander, tailscaled-knowledgebase.service the dedicated node). Code /home/levander/knowledgebase/ (26 tests). Seeded with the Suzuki Vitara 5-door supplement (59 pages/1248 imgs) + Chevy Tracker 1995 owners manual (5 pages/352 imgs) — 2026-07-24-knowledgebase
• Core pattern — a separate tailnet identity on one host = a second tailscaled with --tun=userspace-networking, its own --socket/--statedir, then tailscale --socket=… up --hostname=knowledgebase (one interactive auth) + tailscale --socket=… serve. Gives knowledgebase.<tailnet>.ts.net as its own device without a container or GPU passthrough — the app + OCR stay on the host where the GPU + ~/ocr marker venv live — The two that cost thought (separate-node + TLS)
• TLS gotcha — tailscale serve --https fails with “no TailscaleVarRoot” when the second tailscaled is started with only --state=<file>: a TLS handshake error where curl gets 000 even with -k though TCP/ping succeed. tailscaled has no var root to store the LE cert. Fix: start it with --statedir=<dir> instead (the state file lives inside the statedir, so the node stays authenticated), then tailscale --socket=… cert <name> provisions the cert — The two that cost thought (separate-node + TLS)
• Chunked loading = split each manual by its # h1 headings into many small per-section HTML pages (a 795 KB markdown would be one huge page) + a mkdocs on_post_page hook injecting loading="lazy" into every <img> (verified 1600/1600 lazy) — this fixed “Ulysses chokes on 1248 images”. Browse is the folder tree under manuals-src/docs/ → mkdocs auto-nav (no hand-maintained nav:); one service serves both the static site and the /upload portal; mkdocs builds to a temp dir then os.rename-swaps site/ so a broken source never takes down the live site — How it works (the reusable parts)
• Ingest reuses the existing ~/ocr/venv marker pipeline as a subprocess (marker_single, TORCH_DEVICE=cuda) — no duplicate torch/model install; single background worker, GPU-serialised (shared with Frigate); job states queued→ocr→splitting→building→done/failed. Path-traversal guard (safe_slug + realpath containment) before any write, %PDF magic-byte check, 500 MB cap. Re-ingesting the same folder+name must rmtree the destination first or stale orphan pages/images linger as ghost pages in nav/search — Security & robustness
• Added the Knowledgebase section and the two starred tailnet-node/TLS gotchas to SESSION-HANDOVER

2026-07-23

• Built the Frigate viewer alert — frigate-viewer-alert.service on telep-mainframe (python3 stdlib, /home/levander/frigate-viewer-alert/, 34 tests, runs as levander not root): follows docker logs -f --since 0m frigate, parses each nginx access line, reads the real client IP from X-Forwarded-For (set by tailscale serve), maps 100.x → tailscale device name and Telegrams 👁 <device> (100.x) opened Frigate — HH:MM[ · live: <cam>] on a new UI session. Reuses the frigate-notify bot+chat (token/chatid read at runtime from ~/nvr/frigate-notify/config.yml, never copied) — viewer pings land beside the person/car alerts — 2026-07-23-frigate-viewer-alert
• Core detection technique: tailscale serve writes the tailnet client IP into the nginx X-Forwarded-For field. A real viewer → 100.x XFF; internal automation (frigate-notify polling, self-polls) → XFF -, excluded by construction with no allowlist. Both live (/live/jsmpeg/… 101) and plain UI (/api/config, /login) requests carry it, so every session is visible off one log stream. “Every UI session without spam” via a per-source-IP 5-min cooldown — an open tab polls constantly so it edge-triggers (first request after a quiet gap = new session; later requests just refresh the timer) — How detection works (the reusable part)
• Security gotcha: X-Forwarded-For is the LAST quoted field before request_time=, not the first. Leftmost-match (re.search) is spoofable — a client injecting a literal " into its User-Agent/Referer forges an earlier "…" request_time= and plants the captured viewer IP. Default nginx escapes quotes so it’s safe in practice, but findall(...)[-1] (the field truly adjacent to request_time=) removes the dependency on the log-escape setting. Reusable for any nginx-log parsing that trusts a quoted field — [[2026-07-23-frigate-viewer-alert#1-x-forwarded-for-is-the-last-quoted-field-before-request_time|1. X-Forwarded-For is the LAST quoted field before request_time=]]
• Ops gotcha: never spawn tailscale status per log line. An early version refreshed the name cache on every unknown-IP line; if tailscale status is momentarily failing or the IP isn’t listed, that’s one blocking ~10s subprocess PER line from an active viewer — a self-DoS. Rate-limit miss-triggered refreshes (≤ once/60s) on top of the periodic refresh. Also: reset the docker logs -f reconnect backoff only after a line is actually read, or an immediate-EOF crashloop never escalates; --since 0m so restarting the daemon doesn’t replay stale alerts — [[2026-07-23-frigate-viewer-alert#2-never-spawn-tailscale-status-per-log-line|2. Never spawn tailscale status per log line]]
• Added the Frigate viewer alert bullet under Frigate NVR and gotchas 1+2 to SESSION-HANDOVER
• Root-caused telep-mainframe’s recurring “randomly turns off overnight” to MAINS POWER loss, not the PC. Recurring unclean shutdowns in last -x (Jul 23 06:36, 19 06:59, 17 21:28, 16, 14); the box stays dark until manually powered on. Verdict conclusive — the telep-router (separate hardware/PSU) died at the same instant (mainframe journal dead 06:36:10 CEST, router booted 06:36:27, +17s POST), which only a shared-mains event can cause — 2026-07-23-mains-power-shutdowns
• Reusable method — the dead-stop signature: journalctl -b -1 ends mid-normal-operation (a routine arp-scan presence check) with NO Stopping sequence, NO panic/stack trace, NO thermal/MCE/Xid line, NO OOM. That absence is the diagnosis: software shutdown always logs Stopping, a panic always logs a trace → a clean dead-stop = power physically removed. Confirmed cool (acpitz 27°C, GPU 47°C/128W), no poweroff/suspend timers, atq/wakealarm empty, sleep+suspend masked — Evidence chain (the reusable method)
• Clincher method — cross-check an independent always-on device: derive the router’s boot epoch from date +%s minus /proc/uptime and compare to the instant the PC’s journal died. Same-timestamp death = mains; only-the-PC = its own PSU. Turns “probably power” into “definitely mains” — Evidence chain (the reusable method)
• Explained why only the PC “stays off”: BIOS Restore on AC Power Loss = stay-off (common default) so the desktop sits dark until someone presses power, while the router auto-powers-on (embedded, no button) so its recovery is invisible. Every last -x crash → a boot many hours later, never an auto-restart (inconsistent with panic=reboot) — Why only the PC appears to “stay off”
• Recommendations to the user, priority order: (1) ~600–900 VA UPS + NUT — the real fix for unstable mains, logs every sag/transfer and triggers graceful shutdown on long outages; (2) free stopgap — flip BIOS AC-loss to Power On (physical BIOS access only, not over SSH); (3) rule out an overloaded shared circuit (i9-12900K + RTX 3080 ~500W + 4K TV). ext4 clean and all services recovered so far, but repeated hard cuts risk eventual corruption. Noted frigate.service reads inactive only because Frigate is docker compose, not a systemd unit (containers healthy) — Recommendations (priority order)
• Added the mains-power bullet under Hardware/host and the three reusable lessons to SESSION-HANDOVER

2026-07-22

• Built the WiFi device usage strip on the camwall — wifi-usage.service on telep-mainframe (python3 stdlib, /home/levander/wifi-usage/, 44 tests): every 5s it polls the router over one persistent SSH connection, diffs per-station byte counters into live Mbps, resolves MACs to names and pushes an ASS osd-overlay into the running mpv. Strip along the bottom of the cctv pane (y 1880–2140) showing every telep1 device’s ↓/↑ plus a 5G WAN total — 2026-07-22-wifi-usage-strip
• Core discovery: mpv destroys osd-overlay entries when the IPC client disconnects. Connect→send→close silently does nothing while still replying {"error":"success"}; holding the same socket open with an identical payload made the strip appear. So the IPC connection must be persistent, mpv’s event stream must be drained, and reconnect-on-failure is what gives self-healing (strip returned ~23s after a camwall restart) — 1. The mpv IPC connection must be persistent
• iwinfo assoclist has no byte counters; iw dev <ap> station dump does (rx/tx bytes + signal, iw already on the router) → per-device bandwidth with no extra package, no nftables accounting, no conntrack parsing, no nlbwmon. And the counters are AP-perspective: station tx bytes = the device’s DOWNLOAD, rx bytes = its upload; the WAN interface is not inverted (verified live: device 35.4 ↑ vs WAN 35.1 ↑) — [[2026-07-22-wifi-usage-strip#2-bytes-come-from-iw-not-iwinfo|2. Bytes come from iw, not iwinfo]]
• Chose osd-overlay over camwall-reload: camwall-reload’s loadfile restarts video playback (fine weekly for TOP KÉPEK, would stutter the live cameras every 5s) and goes through the lavfi chain that previously killed the wall. Also: /run/camwall.sock is srw------- root root → the daemon must run as root — [[2026-07-22-wifi-usage-strip#4-osd-overlay-never-camwall-reload|4. osd-overlay, never camwall-reload]]
• Security work on an untrusted input path (DHCP hostnames are attacker-supplied): subprocess.run(text=True) decodes strictly and UnicodeDecodeError is a ValueError not caught by except (SubprocessError, OSError) → errors="replace"; ASS escaping stops tag injection (validated vs libass 0.17.3) but names still need a 22-char cap or a 400-char hostname paints across the 4K frame; camera VLAN excluded by construction (phy1-ap1 never queried) — Untrusted input: LAN device names
• Gotcha: SSH ControlMaster masks credential loss — removing the router key showed no degradation because the multiplexed ControlPersist connection was already authenticated and the key file is never re-read; you must also drop /run/wifi-usage/router.sock to test it — SSH ControlMaster hides credential loss
• Added the WiFi usage strip section and five new gotchas to SESSION-HANDOVER
• Unstalled the FaceKom “VirtualOffice” OpenVPN on the exit-vpn container and wired it as an on-demand Tailscale exit node (exit-vpn / 100.98.208.98). The CA was never broken — real causes were the wrong gateway IP (.185 vs 92.119.122.32, both running OpenVPN on tcp/1194 → foreign CA), wrong crypto (AES-256-CBC/SHA1), and an auth file still carrying the Hungarian jelszó:<TAB> label inside the password. Full tunnel via redirect-gateway def1, persistent via openvpn-client@farm — 2026-07-22-facekom-vpn-exit-node
• Discovered FaceKom “VPN-only” hosts (npm.facekom.net) are public IPs behind an nginx source-IP ACL, not a DNS issue — 403 from home WAN, 200 via VPN, same resolution from every resolver. Also: route-up scripts are impossible under the hardened openvpn-client@.service (fork errno=11 kills the tunnel) → separate oneshot unit restarts tailscaled instead — Gotcha: tailscaled goes offline after VPN route changes
• Corrected the now-wrong “exit-vpn — STALLED” section and added 6 new gotchas to SESSION-HANDOVER
• Diagnosed an unplayable 1.6 GB SplitCam recording (20251025075725.mp4) pulled off the mainframe’s Windows drive via filestash: moov atom not found, mdat size 0. Verdict = unrecoverable — SplitCam crashed and NTFS-preallocated clusters were never flushed, so the mdat is stale deleted-file residue (7.997 bits/byte entropy, zero valid NAL chains, tail all zeros). Only ~69 KB of real H.264 hit disk — mp4-missing-moov-forensics
• Reusable trick: xattr -p com.apple.metadata:kMDItemWhereFroms <file> | xxd -r -p | strings recovered the Filestash origin URL, which let the source at /mnt/win/Users/kisapus69/odi/data/SplitCam/ be chunk-md5-compared against the local copy (identical — source equally dead, no other copies) — Verifying the local copy wasn’t the problem

2026-07-21

• Remounted the Windows NTFS drive at /mnt/win after a reboot and restarted filebrowser; browsing verified working at https://telep-mainframe.taild4189d.ts.net:8445 — Mount procedure (verified 2026-07-21)
• Correction: the NVMe device path is NOT stable across reboots. The docs said /dev/nvme1n1p2; the Windows NTFS partition is now /dev/nvme0n1p2 (1.8T nvme0n1), and nvme1n1 is the 3.6T LUKS+LVM OS disk. Always lsblk -f and pick the 1.8T ntfs partition — and any fstab/systemd entry must use UUID=, not a device node — The Windows drive
• Gotcha: filebrowser needs a restart after the mount. The container binds /mnt/win → /srv RO; started against an empty unmounted dir it holds the stale pre-mount view and /srv looks empty inside the container while the host mount is fine. docker restart filebrowser. Every reboot = mount, then restart, in that order — The Windows drive
• Strengthens the open item on persisting /mnt/win: a systemd mount unit on UUID= ordered Before=docker.service would eliminate both the reboot gap and the container-restart dance — SESSION-HANDOVER
• Drive inventory update: sdb is now present — 115.5G UDF labelled HBCD_PE_x64 (Hiren’s BootCD PE rescue USB stick); sda (447.1G) remains unused (single 16M partition, no filesystem) — Other services on this box

2026-07-18

• Diagnosed & fixed the TV camwall “blue glare”: blocky blue/purple/green chroma on the two live panes (clean on the static JPEG panes). Root cause = a wedged NVENC encoder session on the RTX 3080 corrupting Frigate’s birdseye restream (-c:v h264_nvenc, inherited from global hwaccel_args: preset-nvidia) even at GPU idle — the encode-side sibling of the old NVDEC magenta bug — 2026-07-18-birdseye-nvenc-blue-glare
• Reusable isolation method: grab a still at each pipeline layer (TV framebuffer via x11grab → birdseye RTSP → source cam RTSP → /api/<cam>/latest.jpg → ps aux | grep birdseye). source clean + latest.jpg clean + birdseye corrupt = the h264_nvenc encode is the sole corrupting stage — Isolation method (reusable — this is the valuable part)
• Fix: cd ~/nvr && docker compose restart frigate cleared the wedged session (still on NVENC). Durable option if it recurs: blank hwaccel_args → birdseye encodes libx264 on the 12900K (YOLO stays on GPU; CUDA ≠ NVENC/NVDEC) — Durable fix if it recurs (not yet applied)
• OCR’d a 564-page scanned Suzuki Vitara manual on the mainframe’s RTX 3080 via marker-pdf (~22.5 min, 794 KB markdown, 1,248 images) — set up a reusable venv at ~/ocr/venv. Core gotcha: pip install marker-pdf pulls a cu130 torch that driver 550.163.01 can’t init, so torch.cuda.is_available() is False and marker silently falls back to CPU (hours instead of minutes). Fix = pip uninstall -y torch then reinstall from the cu126 index (minor-version compat); the reinstall is a silent no-op without the uninstall. Driver deliberately NOT upgraded — 550 is load-bearing for Frigate — Running it on a CUDA GPU box (telep-mainframe), [[telep-mainframe#gpu-ocr-venv-ocrvenv|GPU OCR venv (~/ocr/venv)]]
• Generic ops trap surfaced twice this session: pgrep -f / pkill -f match your own command line. The OCR completion monitor reported RUNNING for 2.5 h after the job finished; a pkill -f openvpn... over SSH killed its own shell (exit 143) and silently skipped the remaining steps — shell-gotchas-pgrep-self-match

2026-07-17

• Built and documented the presence-based intruder alarm (intruder-alarm.service, /home/levander/intruder-alarm.py, python3 stdlib-only) on telep-mainframe: auto-arms when no trusted phone is on the network, escalates person/car detections to Telegram, sirens the LG TV — 2026-07-17-intruder-alarm
• Presence = union of router WiFi association table (SSH iwinfo assoclist on telep1 APs) + arp-scan on enp5s0. KEY INSIGHT: sleeping iOS phones stay WiFi-associated but stop answering ARP, so arp-scan alone false-arms overnight; the assoc table sees them. Failed scans are skipped, never treated as “everyone left” — Presence detection
• Gave the mainframe router access it never had (router was Tailscale-SSH-only, ACL-blocked, no port-22 daemon): enabled a LAN-only key-only dropbear (uci Interface=lan, PasswordAuth off) + dedicated key /home/levander/.ssh/router_alarm. GOTCHA: this rotated the router’s SSH host key (breaks existing known_hosts) — Prerequisite router access
• Camera-VLAN gotcha: SSID telep-cc is AP phy1-ap1; querying it made the Tapo camera appear as an enrollable device. Fix: only query telep1 APs + exclude infra MACs (mainframe/printer/router/camera) — Camera-VLAN gotcha — telep-cc leaked an enrollable device
• Enrollment via Telegram Trust/Ignore inline buttons (same “Dezsi az őr” bot as frigate-notify, token read from ~/nvr/frigate-notify/config.yml); device names resolved mDNS avahi-resolve → dnsmasq reverse DNS → eszköz-<last4>, hardened against leaking dig errors — Device naming & enrollment
• State machine: dormant until ≥1 trusted device, arms after ALL trusted absent 10 min, disarms instantly on return, manual /arm /disarm /auto; intruder escalation 🚨 BETŐRŐ! + snapshot (1×day/2×night); TV siren (2-beep, aplay -D plughw:1,3, NVIDIA HDMI = ALSA card 1 dev 3) on every detection armed or not — State machine

2026-07-16

• Deployed Filestash on telep-mainframe to browse the box’s Windows NTFS drive in-browser (previews for images/video/PDF/code/markdown/archives); container bound to 127.0.0.1:8334, config in filestash_data volume — filestash
• Mounted Windows C: (/dev/nvme1n1p2, NTFS) read-only at /mnt/win via in-kernel ntfs3 (RO because the live Windows install’s fast-startup/hibernation would corrupt a RW mount); passed into the container at /mnt/data. Gap: mount is manual, not in fstab — won’t survive reboot — The Windows drive
• Exposed it at https://telep-mainframe.taild4189d.ts.net:8445 via tailscale serve --https=8445, same pattern as top4 picker (:8443) and Frigate (:443); port map now :443/:8443/:8445 — Exposure
• First-run wizard (admin password + “Local” backend at /mnt/data) left for the user to finish; OnlyOffice for inline Office docs noted as optional/not-added — First-run setup (unfinished)

2026-07-14

• Built and documented the Frigate NVR end-to-end on bare-metal telep-mainframe (i9-12900K / RTX 3080 / Debian 13): Frigate 0.17.2 + frigate-notify, ONNX YOLOv9-s detection (~12ms, 6% GPU), events-only recording to /srv/frigate — telep-mainframe
• Exposed the Frigate UI at https://telep-mainframe.taild4189d.ts.net via Tailscale Serve (real LE cert, tailnet-only, no LAN exposure) — Access
• Built the HDMI camera wall: birdseye 3840x1080 → mpv straight to DRM/KMS with no X server; required nvidia-drm modeset=1 or the 3080 exposes zero display connectors — TV wall (HDMI)
• Wired Telegram alerting (bot “Dezsi az őr” → supergroup) on person + car via frigate-notify polling the HTTP API every 15s — Alerting
• Big gotcha: the dual-lens Tapo’s /cam1/ and /cam2/ RTSP paths are ALIASES FOR THE SAME LENS (proved by PSNR ≈36 vs ≈3.9); caused duplicate events + duplicate alerts and an invisible PTZ lens. Correct map: fixed=/stream1, PTZ=/stream6 — are the SAME lens
• Recorded eight more gotchas: Secure Boot rejecting the DKMS-signed NVIDIA module, TensorRT detector dead on amd64 in 0.17 (use onnx), Frigate’s YOLOv9 export broken by torch ≥2.6, Tapo silently truncating long passwords, Frigate’s DB wiped unless the whole /config dir is mounted, birdseye not drawing camera names, birdseye grid geometry, iOS Private Relay breaking MagicDNS — Gotchas
• Added the isolated telep-cc camera VLAN to the router doc (192.168.30.0/24, hidden SSID, WPA2 forced by Tapo, no cams → wan, NTP DNAT’d back) and recorded the router’s outstanding security gaps — Camera VLAN (telep-cc), Security Gaps

2026-07-13

• Initialized the homelab project (physical/on-prem infra, separate from cloud-only levandor-infra) — homelab
• Surveyed the OpenWrt router end-to-end: RT-AX89X on OpenWrt 24.10.0, LAN/WAN/firewall/DHCP, WPA3-SAE WiFi, Tailscale 1.80.3, 174 stock packages — telep-router
• Fixed hostname drift: device still identified as OpenWrt (kernel, uname -n, UCI) while telep-router was only a Tailscale MagicDNS name. Set uci system.@system[0].hostname='telep-router', committed, reloaded — Changes Made (2026-07-13)
• Recorded three gotchas: wifi-ifaces declaring network='lan wan' (inert but a footgun), 8-char WiFi PSK, and double-NAT behind 192.168.254.1 — Gotchas