Living operations doc for the homelab: self-hosted car-repair knowledge base + NVR + DNS-monitoring + 3D-print + power-resilience stack.
Host telep-mainframe (Debian 13) · i9-12900K · RTX 3080 · Corsair 1000W · on a CyberPower PR1500 UPS. Tailnet taild4189d.ts.net.
Access: /usr/bin/ssh levander@telep-mainframe. When the LAN/DNS is down (MagicDNS may not resolve), reach the mainframe directly over Tailscale at ssh levander@100.115.209.87. LAN IP 192.168.1.123 (enp5s0); the Pi is 192.168.1.200 — don’t confuse them.
Routertelep-router = LAN 192.168.1.1, tailnet 100.69.112.32. ⚠️ From the mainframe, reach the router over LAN ssh root@192.168.1.1, NOT its tailnet IP — the mainframe cannot reach 100.69.112.32 (ping times out); only 192.168.1.1 works. (OliveTin’s own container CAN reach the router at 100.69.112.32; the asymmetry bites scripts that run ON the mainframe — e.g. telep-selftest.)
Mid-incident? Use the action-first runbooks
For “when X breaks, do Y” — no WiFi/DHCP, service down, host offline, AirPlay, camwall, KrakenSDR — see the Homelab Runbooks index. Symptom → runbook table, copy-paste fixes, links back to the detailed notes below.
✅ armed + hardened (full power-off self-recovers on boot)
Disk encryption (root FDE)
✅ NEW — YubiKey-gated LUKS (key present → unattended auto-unlock; pull key → box powers off; boot w/o key → passphrase). ⚠️ TPM/clevis REMOVED; only ONE key enrolled (add a backup)
Cameras / Frigate
✅ RESOLVED 2026-08-22 — “recording broken” was a retention-config issue (not a jam): continuous.days=0+motion.days=0 meant footage was discarded unless a person/car overlapped. Fixed via record.motion.days: 10 + tracking cat. See 2026-08-22-frigate-recording-retention-config-not-jam
Camera OSD clocks
✅ correct — DST-aware ONVIF push
Camwall / cams VLAN
✅ healthy — br-cams guard cron restores the gateway after wifi reloads
AirPlay receiver (telep-tv)
✅ fixed — avahi locked to enp5s0
Master control-plane agent
✅ LIVE on host — phone-drivable, permissions ACTIVE
FreeCAD CAD workstation
✅ NEW — headless FreeCAD GUI + noVNC + MCP on own cad tailnet node; export pipeline (browse cad:8080, auto-sync to Mac) — 2026-08-26-freecad-cad-workstation
CAD-designer agent
✅ NEW — 2nd always-on claude remote-control (cad-designer.service), FreeCAD MCP wired, app session cad-designer-telep — 2026-08-26-cad-designer-agent
CAD export pipeline + Online3DViewer
✅ NEW — export_all → STEP/STL/3MF into /exports, browse cad:8080, auto-syncs to Mac ~/cad-exports; Online3DViewer at cad:8090 (click-to-view + drag-drop + Finder picker; frame-ancestors 'self') — 2026-08-26-freecad-cad-workstation
KrakenSDR mobile DF
✅ field-working — TETRA uplink 382.114 MHz, UCA 200 mm, ext-3 whips; Kraken Pro iOS app over hunter-ap → 10.42.0.1 using the phone’s GPS. ⚠️ bearings RELATIVE (array_offset=0); car power blocker — 2026-08-26-krakensdr-field-test-milestone
Filestash (Windows drive browser)
✅ remounted — reboot had dropped /mnt/win; re-mounted RO + docker restart filebrowser. ⚠️ still NOT reboot-safe — filestash
Stirling-PDF
✅ on own tag:telep node pdf.taild4189d.ts.net (⚠️ no note yet)
C302 tailnet FaceTime cam
✅ working (go2rtc P2P + WebRTC → OBS)
Top képek → Drive pipeline
✅ live (self-serve crop + Drive push in the picker)
Local access (*.telep.lan)
✅ LAN wildcard DNS + Caddy reverse proxy; internet-independent
Caddy reverse proxy
✅ hardened after a ~1.6-day :443 boot-race outage (nonlocal-bind + restart drop-in)
Service dashboard (home.telep.lan)
✅ NEW — Homepage “where-is-what” live-status board (also on :8450 tailnet, Caddy-independent)
dnsmon DNS monitor + gambling watchlist
✅ NEW — FastAPI panel on own tailnet node dnsmon.taild4189d.ts.net
Casino/gambling DNS alerter
✅ live on the router — Telegram alerts now include device name + context
Bambuddy (Bambu Lab 3D-print)
✅ NEW — control plane on own node bambuddy.taild4189d.ts.net; printer LAN 192.168.1.202
WiFi
✅ SSID split — telep1=5 GHz, telep1-2G=2.4 GHz (both LAN); telep-cc=cams VLAN; new wired AP bandi
Admin portal (admin.levandor.io)
✅ iPhone passkey-gated OliveTin; actions call telep-selftest (clean ✓/✗); LAN-only for now
Disk encryption (root FDE)
✅ YubiKey-gated (serial 32875420) — see danger callout; ⚠️ only ONE key enrolled (SPOF)
Network / DHCP+DNS
✅ stable — recovered from a duplicate-dhcp-host outage 2026-08-15
KrakenSDR — array_offset calibration. Bearings are still RELATIVE (array_offset=0); calibrate so headings are absolute. See 2026-08-26-krakensdr-field-test-milestone.
KrakenSDR — car power blocker. Pi 5 + DAQ brown out on 5 V/3 A (a generic PD-30W only gives 5V/3A → no help); needs a real 5 V/5 A. On-hand XL4015 is set to 6 V + marginal at 5 A / 38 V ceiling (load-dump risk); buy = MINI560-5V-5A (HESTORE 100.423.75) or a proper 12V→5V/5A(/8A) buck + usb_max_current_enable=1; Kraken + phone on their own feeds. See 2026-08-26-krakensdr-field-test-milestone.
Filestash /mnt/win is NOT reboot-safe — fix = a systemd mount unit keyed on the NTFS partition UUID, ordered Before=docker.service, so it auto-mounts RO on boot. (Windows drive still needs data migration → then LUKS-encrypt; sda 447 G still empty/unencrypted.) See filestash.
Enroll a BACKUP YubiKey — only one enrolled (SPOF); disarm the live-lock udev rule during enrollment.
Encrypt sda (447 G, empty) + the Windows NTFS disk (nvme0n1) after the user migrates data off + wipes it (Windows boot is expendable).
Passkey on watch.levandor.io for the dnsmon panel off-tailnet (needs a Pocket-ID OIDC client + dedicated oauth2-proxy; tailnet access already works).
Unify the watchlist — the dnsmon panel watchlist and the router casino-alert.sh built-in list are currently separate.
✅ DONE 2026-08-26 — reserved 192.168.1.202 (Bambu printer) in DHCP on telep-router: static host name='bambu', mac='50:31:23:c9:63:ba', resolves as bambu.lan. See Problem 2 — Bambu Lab H2S offline.
Kraken/Pi (192.168.1.200) — authorize the OliveTin ssh pubkey when the rig is back; kraken.telep.lan 502s while offline.
Optional: nlbwmon on the router for per-device WAN accounting + history; QoS to prevent uplink saturation.
Private Relay / DoH LAN-wide block was fully reverted — if revisited, do it per-device, not LAN-wide (evasive devices e.g. iPhone on iCloud Private Relay .146 stay invisible to DNS monitoring).
✅ RESOLVED 2026-08-22 — Frigate recording was a RETENTION-CONFIG issue, not a jam
The earlier hypothesis below (wifi reloads / br-cams drops / telep1 SSID split wedging RTSP into go2rtc and jamming the record pipeline) was WRONG. Full writeup: 2026-08-22-frigate-recording-retention-config-not-jam.
Actual root cause: the resolved record config had continuous.days = 0 AND motion.days = 0; only alerts + detections retention were set (14 days each, mode: motion), which only keep segments overlapping a tracked object (person/car). During any window with no person/car, ffmpeg still wrote 10s segments into the /tmp/cache tmpfs but Frigate’s maintainer DISCARDED them instead of moving them to /srv/frigate/recordings. The “68 MB/day collapse” was just low person/car activity, not a stall.
Everything downstream was healthy: detect camera_fps=5.0 on all 4 cams; record ffmpeg running and writing fresh ~1.5 MB segments to /tmp/cache; disk fine (root LVM 6%). The only anomaly was 0 files moved to recordings/ in a 3-min window while cache held segments — the tell-tale of retention config, not a jam.
Fix (2026-08-22): edited /home/levander/nvr/frigate/config.yml (backup config.yml.bak-20260822-catmotion) — added record.motion.days: 10 (user chose motion, 10-day retention, over ~150 GB/day continuous) and added cat to global objects.track (now [person, car, cat]; coco-80 model). docker restart frigate. Verified: /api/config shows the new track list + record.motion.days=10, and 13 segments/camera landed in /srv/frigate/recordings within 3 min. Recording works.
Diagnostic order (runbook):/api/stats per-cam camera_fps (detect health) → docker exec frigate ls -la /tmp/cache (ffmpeg producing?) → /api/config resolved record block. Cache has segments + recordings/ empty + continuous.days==0 + motion.days==0 ⇒ retention config, not a jam.
The “cat outside at ~03:00” clip never existed (cat untracked AND no continuous/motion retention). Going forward, motion recording + cat tracking will capture such events.
Latest session
The 2026-08-22 → 29 work (control-plane OAuth fix, Frigate retention + cat alerts, router 5 GHz cross-band isolation + Bambu H2S, the FreeCAD CAD studio + Online3DViewer + cad-designer agent + export pipeline, KrakenSDR mobile-DF field-working, Filestash remount) is tied together in 2026-08-29-session-handover. The 2026-08-09/10 overnight work (AirPlay, Frigate watchdog, UPS hardening, master agent, C302 cam, Drive pipeline, router changes) is in 2026-08-10-session-handover. The 2026-08-15 → 19 work (YubiKey FDE, admin portal + telep-selftest, *.telep.lan + Caddy hardening, home.telep.lan dashboard, dnsmon + casino alerter, Bambuddy, WiFi split, br-cams guard) is consolidated below.
1. Knowledge Base
Astro 5 + Starlight static site, served by a Flask app that also does hybrid search. Built with Bun (no Node runtime).
Persistent JSONL (kb-vectors/jobs.jsonl), dashboard /jobs, list /api/jobs.
Image review
app.py + image_types.json
/image-review — click-to-classify the flagged gallery images.
OCR decision
The “orphaned procedure” pages were a marker chunker bug (it read the headers fine, the splitter dropped them). Verdict from the bake-off: patch (recover headers from marker raw output) rather than full re-OCR. GOT-OCR2 targeted re-OCR only for catastrophically-mangled spec tables — full re-OCR regresses clean prose + costs 8–12 GPU-hr. See 2026-08-06-kb-marker-chunker-bug-and-ocr-bakeoff.
Build gotcha
build_content.py excludes *.bak* dirs (a list_manuals() helper) — they were leaking ~1,646 stale pages into the live build. Don’t remove that filter; backup dirs live inside the content tree. See 2026-08-06-kb-build-backup-dir-leak-and-tsb-diagram-restore.
Recurring hard-reboots are solved. Root cause: an unplugged GPU 12V connector (fixed) + weak mains that browns out under CPU load spikes. Stress test held 536 W. Now on a UPS with full automation.
📱 Telegram · 🔴 red · then upsmon graceful systemctl poweroff
Restored
📱 Telegram · 🟢 green blip → normal · restart shed services
Kept alive always
🎥 Frigate/CCTV + networking (router is also on the UPS → WiFi survives)
RGB beacon — /usr/local/bin/kb-rgb.sh <red|normal|green>. Kraken Z LCD (liquidctl) is the reliable red surface; ITE motherboard best-effort; Lian Li Uni Fan SL abandoned (garbled/unreliable on Linux). Idle color/brightness tunable at the top of kb-rgb.sh.
Telegram creds reused from frigate-notify → /etc/nut/telegram.env (600). Channel “Telephely biztonsági riasztások”.
NUT services enabled on boot; upsmon runs least-privilege via the sudoers drop-in.
Live-test
Briefly unplug the UPS mains input → red screen + Telegram + load-shed; plug back → green→normal + restart. Only reaches auto-shutdown if left on battery to LOW.
Shed/restore hardened after a real outage (2026-08-08)
An overnight outage exhausted the battery → graceful poweroff → mains returned while the box was off → BIOS auto-power-on booted it, but the NUT ONLINE/restore event never fired, so kb-qdrant (shed via docker stop) stayed down and knowledgebase.service crash-looped 2360×. Fixed: a shared idempotent /usr/local/bin/power-restore.sh is now called by both the NUT handler and a boot-time power-restore-reconcile.service; kb-qdrant is --restart unless-stopped; KB waits on Qdrant 6333/readyz before starting. See 2026-08-08-ups-shed-restore-hardening.
FPS watchdog (2026-08-08):frigate-fps-watchdog.service now auto-docker restart frigate on both a frozen feed (camera_fps<1.5) and a detection jam (process_fps<1.0), polling /api/stats every 20s, with a 3-restart/hour circuit breaker (4th → Telegram alert only) so an offline camera can’t cause restart thrash. A breaker trip = check for a wedged camera needing an ONVIF reboot. See 2026-08-08-frigate-fps-watchdog. ⚠️ The watchdog does NOT catch a record-only stall (feed + detect stay healthy) — though note the 2026-08-22 “recording broken” scare turned out to be retention config, not a record stall: see 2026-08-22-frigate-recording-retention-config-not-jam.
Frigate stores recordings in UTC. Dir paths /media/frigate/recordings/YYYY-MM-DD/HH are UTC, while the host is Europe/Budapest (CEST, UTC+2). Eyeballing the HH dirs to find a local-time window is off by +2 h (caused a wrong first export → “No recordings found”). The export API takes unix epoch, so date -d "LOCAL time" +%s is correct — trust epoch, not the dir names.
Export API:POST http://127.0.0.1:5000/api/export/{camera}/start/{epoch}/end/{epoch} with body {"playback":"realtime","name":"..."} → stitches the available segments into /srv/frigate/exports/ (viewable via Frigate → Exports). Fails with “No recordings found for time range” if the window has no clips.
Mount: host /srv/frigate → container /media/frigate. Config lives at /home/levander/nvr/frigate/config.yml (mounted /config), NOT/srv/frigate/config. Retention (as of 2026-08-22): record.motion.days: 10, plus alerts/detections 14 days each; continuous.days: 0. See 2026-08-22-frigate-recording-retention-config-not-jam.
Camera wall — camwall-x X server on HDMI-1 (4K) + 4× mpv grid; crash-prone → has camwall-watchdog + camwall-frigate-watch. If the monitor goes dark: sudo systemctl restart camwall-x camwall.
Camera clocks — ~/tapo-ctl/set-cam-time.py (cron 04:30) now syncs both cams, resolving each one’s login from Frigate’s env (the old script only did .139, so .119 drifted). Both currently correct (CEST).
Encrypted archive — /mnt/cam-archive (916 G LUKS2, /dev/sdb1 UUID 7af64460-…; now also has a YubiKey keyslot — 2026-08-15-yubikey-gated-luks-fde). Set up but the offload cron is NOT wired yet.
C302 FaceTime cam (2026-08-09) — the indoor Xiaomi C302 (192.168.30.168) is a low-latency FaceTime/Zoom cam over the tailnet via a SECOND go2rtc instance (go2rtc-cam) + WebRTC → OBS — deliberately NOT in Frigate. Hardware relay + red BELSŐ KAMERA AKTÍV banner. Firewall scoped to just .168 so the Tapo cams stay air-gapped. See 2026-08-09-xiaomi-c302-tailnet-facetime-cam.
AirPlay OSD note:telep-tv broke after the UPS switch (avahi broadcasting on docker/veth ifaces + a DHCP flap) — fixed by locking avahi to enp5s0; a DHCP reservation on the router stops the flap. See 2026-08-04-telep-mainframe-airplay-receiver-uxplay.
3b. New services & tooling (2026-08-08 → 10)
Master control-plane agent — always-on, phone-drivable claude remote-control on the host (systemd control-plane.service), vault cloned to /home/levander/obsidian with obsidian-sync.timer. Permissions stay ACTIVE (no --dangerously-skip-permissions) → destructive ops prompt in the app. See 2026-08-08-telep-master-agent-design + 2026-08-08-telep-master-agent-impl. 🔴 If the app shows an OAuth error / the pane churns Reconnected after Ns while the service looks active → the shared ~/.claude/.credentials.json tokens are likely zeroed; fix = interactive claude login on the box in a REAL terminal (ssh -t levander@telep-mainframe '~/.local/bin/claude login'), no restart needed (auto-restart picks up new creds). See 2026-08-22-control-plane-oauth-zeroed-creds.
Stirling-PDF — self-hosted on its own tag:telep node pdf.taild4189d.ts.net (login disabled, Hungarian default), home-portal tile added. ⚠️ No dedicated vault note yet (doc gap).
Top képek → Drive pipeline — the top4-web.py picker now has batched infinite scroll, jump-to-date/time, and per-card save-to-Drive (full + native-res crop → gdrive:top_képek/). Bulk helpers ~/top-kep-drive.py / ~/top-kep-id.py. See 2026-08-10-top-kepek-drive-curation-print-pipeline.
*.telep.lan split-horizon local access (2026-08-15) — internet-independent LAN path to every service: a *.telep.lan wildcard DNS record on telep-router resolves every name to the mainframe, fronted by a Caddy v2.11.4 reverse proxy here (/etc/caddy/Caddyfile, :80, auto_https off) mapping files/frigate/kraken/top/print/whiteboard/ruview/go2rtc.telep.lan etc. → each backend. Tailscale (9 serve proxies) + all 7 docker containers left untouched. See 2026-08-15-telep-lan-split-horizon-caddy.
admin.levandor.io passkey admin portal (2026-08-15) — iPhone-only, passkey-gated tap-to-run control panel. Docker-compose stack at /home/levander/admin-portal/: Caddy (host, TLS via Cloudflare DNS-01, ⚠️ its admin.levandor.io block MUST bind 192.168.1.123 to avoid a :443 collision with Tailscale, and needs systemctl restart not reload) → oauth2-proxy (OIDC PROXY mode; upstream is the compose service name http://olivetin:1337, PKCE S256, PROMPT=login, allow unverified email) → OliveTin (user: root, writable /root/.ssh, Hungarian actions that ssh into mainframe/router/pi) → IdP Pocket-ID v2.13 at id.levandor.io. Config /home/levander/admin-portal/olivetin/config/config.yaml. Two gotchas (login-500 unverified-email + OliveTin ssh HOME/:ro) written up in 2026-08-15-admin-portal-passkey-olivetin; runbook admin-portal-login-broken. Remote/off-LAN access deferred (Caddy binds the LAN IP only).
Status actions run telep-selftest (2026-08-15) — OliveTin’s status actions now just ssh 192.168.1.123 'telep-selftest <section>', calling /usr/local/bin/telep-selftest [dns|airplay|camwall|services|cameras|alerts|health|all] on the mainframe → clean colored ✓/✗/⚠ output. 🔴 GOTCHA — OliveTin runs every shell: string through Go text/template before executing, so literal {{ }} (e.g. docker ps --format "{{.Names}}") breaks with a template error → keep logic in telep-selftest, not in the YAML. The alerts section is a canary for the MagicDNS→container-DNS outage (checks api.telegram.org reachable from inside the frigate container). 🔴 The dns section must ssh the router at LAN 192.168.1.1 (the mainframe can’t reach 100.69.112.32). 🔴 The ONVIF camera-reboot action must use ~/tapo-ctl/venv/bin/python — system python3 lacks the onvif module. See 2026-08-15-admin-portal-passkey-olivetin.
3c. DNS monitoring, service dashboard & 3D print (2026-08-17 → 19)
home.telep.lan — “where-is-what” service dashboard.gethomepage/homepage (Next.js) in Docker at /home/levander/homepage/, network_mode: host, binds 127.0.0.1:3010 (3000 was taken by ruview), HOMEPAGE_ALLOWED_HOSTS set. Two redundant paths on purpose: http://home.telep.lan via the :80 Caddy, andhttps://telep-mainframe.taild4189d.ts.net:8450 via tailscale serve (Caddy-independent, so the board survives a Caddy outage — which is exactly what it surfaced). 18 tiles / 6 Hungarian groups, per-tile live health + fallback IPs. 🐛 services.yamldescription: values with a colon-space break YAML — quote them. See 2026-08-17-home-dashboard.
dnsmon — DNS monitoring + gambling-watchlist panel. FastAPI + SQLite systemd service dnsmon.service at 127.0.0.1:8099 (/home/levander/dnsmon/, venv). Ingests the router’s dnsmasq query log by ssh-tailing root@192.168.1.1 (/tmp/dnsmasq-queries.log); 48 h retention (flagged rows kept indefinitely); resolves device names via mDNS (avahi-browse/avahi-resolve on the mainframe) + DHCP leases + MAC-OUI, with an honest “randomized MAC” label. UI: live monitor, /devices, /flagged, /watchlists, /context (view a flagged hit in surrounding-query context). Search q= matches domain OR ip OR hostname. watchlist.json holds gambling_keywords + watch + allow. Exposed on its own dedicated tailnet nodehttps://dnsmon.taild4189d.ts.net (systemd tailscaled-dnsmon, userspace, --statedir=/var/lib/tailscale-dnsmon, tag:telep, serve :443→:8099), per tailnet-service-exposure-convention. ⚠️ No dedicated dated note yet — this handover is the current record.
Casino/gambling DNS alerter (on the ROUTER)./etc/casino-alert.sh runs as a procd service — single instance (beware duplicate instances). Tails /tmp/dnsmasq-queries.log, broad gambling regex, Telegram alerts now include the device name (resolved from DHCP leases) + surrounding-domain context. Real fires seen: bet365.com from an iPhone (.146), adpool.bet/adnetwork.bet ad-CDN lookups. Related: 2026-08-03-cam-stall-recovery-and-casino-alert. ⚠️ Its built-in list is separate from the dnsmon panel watchlist — unify them (open item).
Bambuddy — Bambu Lab 3D-print control plane. Docker bambuddy (ghcr.io/maziggy/bambuddy), network_mode: host, UI :8000, on its own tailnet nodehttps://bambuddy.taild4189d.ts.net (systemd tailscaled-bambuddy, --statedir=/var/lib/tailscale-bambuddy, tag:telep, serve :443→:8000) — the host-network exception in Exception — host-network services (e.g. Bambuddy). The Bambu printer is connected via LAN / Dev-Mode at 192.168.1.202 (AMS detected). Slicer = Bambu Studio on the Mac (NOT OrcaSlicer — upstream bugs); it prints through Bambuddy’s virtual printer over the tailnet. Deploy background: 2026-08-08-bambuddy-deploy-plan. ✅ .202reserved in DHCP 2026-08-26 (name='bambu', mac='50:31:23:c9:63:ba', resolves bambu.lan). 🩺 If the printer goes offline (Studio IOTC … ErrCode -12), it’s usually a Wi-Fi/router event — the H2S is dual-band (2.4 + 5 GHz; observed 2026-08-26 on 5 GHz telep1, ch36, −40 dBm), NOT 2.4-only; diagnose with an SSDP probe + nc -z 192.168.1.202 8883 990 6000 and recover with a router Wi-Fi restart: Problem 2 — Bambu Lab H2S offline (printer internal model O1S, SN 0938BJ641800310). 💤 Standby ≠ offline: a printer in STANDBY/SLEEP still pings/ARPs but closes 8883/990/6000 and stops SSDP — “pings but all ports closed + no SSDP” = asleep, wake it via the touchscreen. ⚠️ If Studio can reach the printer only when the Mac is on the SAME band (fails only when they’re on different bands — printer on 2.4 GHz telep1-2G while Mac is on 5 GHz telep1): that’s driver-level cross-band Wi-Fi client isolation, not an outage — put both on the same band (telep1 5 GHz works) or wire the printer over Ethernet. Tailscale won’t help an on-LAN client. See Problem 3 — Wi-Fi cross-band client isolation (driver-level).
FreeCAD CAD workstation + CAD-designer agent + export pipeline (2026-08-26). Headless FreeCAD GUI + noVNC + a FreeCAD MCP in Docker (/home/levander/freecad/) on its own tailnet node cad (https://cad.taild4189d.ts.net/ noVNC, :8443/mcp MCP). Feeds the Bambuddy print flow above: model 3MF → Bambu Studio slice → Bambuddy virtual printer. Export pipeline: the export_all helper (/exports/cad_export.py) writes STEP+STL+3MF to /home/levander/freecad/exports/; browse them at https://cad.taild4189d.ts.net:8080/ (nginx exports-http container + tailscale serve --https=8080); they auto-sync to the Mac~/cad-exports/ (launchd com.levander.cad-exports-sync.plist, 90 s rsync). A 2nd always-on Claude agentcad-designer.service (app session cad-designer-telep, FreeCAD MCP wired at 127.0.0.1:9876/mcp) designs parts from the phone — same active-permissions posture as the control-plane operator. 🔴 Its tmux unit needs its OWN socket -L cad-designer (sharing the control-plane’s default socket thrashes). See 2026-08-26-freecad-cad-workstation + 2026-08-26-cad-designer-agent.
3d. Network & Caddy state (2026-08-15 → 19)
WiFi SSID split.telep1 is now 5 GHz only (radio0, ch36, HE40); 2.4 GHz is a SEPARATE SSID telep1-2G (radio1, ch11) for range / through-walls. Both are network=lan (same LAN, bridged into br-lan). The cams SSID is telep-cc (2.4 GHz, isolated cams VLAN 192.168.30.0/24). TX power is pinned at the HU/ETSI regulatory ceiling (5 GHz 23 dBm / 2.4 GHz 20 dBm — “max” is real, not a lever). See 2026-08-15-router-wifi-tuning-htmode-band-penetration. ⚠️ After a 2026-08-26 router reflash the 5 GHz SSID is now telep1-5G; if a Mac can’t see it in the Wi-Fi menu (phones can), it’s on a DFS channel — move the 5 GHz radio to a non-DFS channel (36/40/44/48 or 149/153/157/161). Also: don’t trust macOS networksetup/system_profiler for association state (they redact/lie). Runbook: 2026-08-26-router-reflash-mac-5ghz-dfs-and-bambu-offline.
New wired AP — TP-Link TL-WA850RE (“bandi”). Access-Point mode, wired into a br-lan port, static reservation 192.168.1.101. A weak 2.4-only N300 — coverage only, not video-grade. See [[telep-router#tp-link-tl-wa850re-added-as-a-wired-ap-ssid-bandi|TP-Link TL-WA850RE added as a wired AP (SSID bandi)]].
cams-VLAN guard (applied). A wifi reload on radio1 can leave the wireless-only bridge br-cams stuck state DOWN → kills the 192.168.30.1 gateway → cameras unreachable → camwall freezes. GUARD:/etc/cams-guard.sh runs every minute via router cron (crond running) and restores br-cams (ifup cams) if 192.168.30.1 is missing. Still, proactively restore br-cams after any manual radio1 wifi reload. Incident + manual recovery: 2026-08-15-camwall-dead-br-cams-bridge-down-after-wifi-reload; runbook camwall-not-on-tv.
Caddy hardened after a boot-race outage. Caddy was found DEAD ~1.6 days (2026-08-15→17): it binds 192.168.1.123:443 at boot before the LAN IP is up, fails, and does not retry → every *.telep.lan service AND both levandor.io portals were unreachable-by-domain. FIX:/etc/sysctl.d/99-caddy-nonlocal-bind.conf (net.ipv4.ip_nonlocal_bind=1, the real fix) + systemd drop-in /etc/systemd/system/caddy.service.d/resilience.conf (Restart=on-failure, After/Wants=network-online.target). If everything .telep.lan/levandor.io is down after a reboot → systemctl status caddy for the :443 bind race. See Caddy boot-race outage (2026-08-17); runbook caddy-boot-race.
4. Operations cheat-sheet
# rebuild + deploy the KB (niced to protect Frigate)cd ~/kb-astro && nice -n 19 ionice -c3 bash kb_build.shcd ~/knowledgebase && rm -rf site.new && cp -a ~/kb-astro/dist site.new \ && rm -rf site.old && mv site site.old && mv site.new site && rm -rf site.old \ && sudo systemctl restart knowledgebase.service# fix Frigate if the live view stallssudo docker restart frigate# bring the camera-wall monitor back (HDMI dead)sudo systemctl restart camwall-x camwall# re-sync both camera clocks nowcd ~/tapo-ctl && ./venv/bin/python set-cam-time.py# UPS status / runtime / loadupsc cyberpower# RGB manually/usr/local/bin/kb-rgb.sh red # or: normal / green# graceful shutdown (does NOT increment unsafe-shutdowns)sudo systemctl poweroff
BIOS "Restore on AC Power Loss" is ON
The box auto-powers-on after any mains event. To keep it truly off during work, flip the PSU switch / unplug, or change that BIOS setting.
Disk is YubiKey-gated (2026-08-15) — pulling the key powers the box off
Root LUKS now unlocks via a YubiKey 5C NFC (serial 32875420, OTP slot 2 = HMAC-SHA1 challenge-response, no-touch) kept plugged in — NOT clevis/TPM anymore (2026-08-15-yubikey-gated-luks-fde). Key present → auto-boots unattended (survives mains cuts); key absent → won’t unlock. initramfs keyscript /usr/local/sbin/yk-keyscript.sh + challenge /etc/ykluks/root.challenge. Live-lock:/etc/udev/rules.d/99-yk-lock.rules → pull the key → sync + poweroff. Never run ykman config … (or anything that re-enumerates the key) without first disarming99-yk-lock.rules (sudo mv /etc/udev/rules.d/99-yk-lock.rules /root/… ; sudo udevadm control --reload-rules). Break-glass = the passphrase at the physical console (slot 0; no dropbear). ⚠️ SINGLE POINT OF FAILURE — only ONE key enrolled (open item). Header backups (YubiKey-only state) on the Mac at ~/telep/*postyk.img; the old clevis-containing backup was shredded. Recovery: yubikey-luks-lockout.
sudo cryptsetup luksDump /dev/nvme1n1p3 # slots 0 (passphrase) + 2 (YubiKey); Tokens empty, no clevissudo ykman info # needs pcscd + root; USB id 1050:0407
5. Open items — needs you
🚨 From the 2026-08-22 session:
✅ RESOLVED — Frigate recording. Was NOT a pipeline jam: record.continuous.days=0+motion.days=0 meant footage was discarded unless a person/car overlapped. Fixed by adding record.motion.days: 10 + tracking cat in /home/levander/nvr/frigate/config.yml. Verified segments now land in /srv/frigate/recordings. Full writeup: 2026-08-22-frigate-recording-retention-config-not-jam.
Optional cleanup: /home/levander/nvr/frigate/go2rtc_homekit.yml is a 0-byte extra go2rtc config (source of stray c302/c302_h264 streams) — harmless, candidate for removal from the config path.
KrakenSDR — car power is the blocker. The Pi 5 + DAQ brown out and reset on 5 V/3 A (a generic PD-30W only supplies 5V/3A → does NOT help; earlier “OOM crashes” were largely these brownouts). Needs a real 5 V/5 A source. On hand: an XL4015 buck, but it’s currently set to 6 V (servo/lightaim use) and marginal at 5 A with a 38 V input ceiling (car load-dump risk). Recommended buy = MINI560-5V-5A (HESTORE 100.423.75) or a proper 12V→5V/5A(/8A) buck + usb_max_current_enable=1; run the Kraken and the phone on their OWN feeds (Kraken self-powers via its own USB-C).
Filestash /mnt/win mount is NOT reboot-safe. A reboot dropped the NTFS mount (remounted RO this session + docker restart filebrowser). Proper fix = a systemd mount unit keyed on the NTFS partition UUID, ordered Before=docker.service, so it auto-mounts RO on boot and filebrowser sees it (kills the whole restart-the-container-after-mount class of problem). The Windows drive still needs data migration → then LUKS-encrypt per the older FDE plan; sda (447 G) is also still empty/unencrypted. See filestash.
5 GHz cross-band client isolation on telep-router — driver/firmware-level (wifi→ethernet forwards, wifi↔wifi cross-band fails; all config knobs clean, survives a wifi restart). This is the Bambu-printer wifi↔wifi unicast issue. Workaround in place: put clients on the same band or wire them over Ethernet. Optional: dig into the radio chipset for an inter-VAP isolation override. See 2026-08-26-router-reflash-mac-5ghz-dfs-and-bambu-offline. ⚠️ AirPlay is a DIFFERENT bug — telep-tv undiscoverable on Wi-Fi was long misattributed here but is actually the box’s 10G switch port not flooding multicast to Wi-Fi (band-independent; unicast fine). See 2026-08-30-telep-tv-airplay-10g-port-multicast-not-flooded.
telep-tv AirPlay undiscoverable on Wi-Fi — 10G-port multicast bug on telep-router — the box (192.168.1.123, enp5s0 @ 10000 Mbps, br-lan port 2 = 10G) emits mDNS multicast correctly on the wire but the switch doesn’t flood it to Wi-Fi; 1G-port devices (printer) do reach Wi-Fi. Unicast (ssh/ping/resolve/AirPlay-connect) unaffected. Quick fix = move cable to a 1G port; keep-10G fix = mDNS reflector. See 2026-08-30-telep-tv-airplay-10g-port-multicast-not-flooded.
Bambu H2S maintenance is DUE per the delivered Hungarian schedule PDF (X-axis rail monthly; quarterly items).
Never re-add a telep-tv DHCP reservation (2026-08-15 outage)
On 2026-08-15 a duplicate dhcp-host for 192.168.1.123 (a dhcp.telep_tv section with the same MAC/IP as the existing telep-mainframe reservation) made dnsmasq fail to start entirely → DHCP and DNS dead site-wide for ~8.5 h. The AirPlay telep-tv receiver is the mainframe, so its telep-mainframe reservation (dhcp.@host[2], .123, d8:5e:d3:a7:05:d6) already covers it. Before adding ANY reservation: uci show dhcp | grep <ip-or-mac>. Full post-mortem: 2026-08-15-dhcp-outage-duplicate-reservation-postmortem.
Site-wide DHCP/DNS outage — ✅ RESOLVED. Duplicate dhcp-host deleted (uci delete dhcp.telep_tv; uci commit dhcp; /etc/init.d/dnsmasq restart); mainframe re-leased .123 and returned to the tailnet.
jellyfin.telep.lan — a guessed name with the backend down; remove or rename the Caddy entry once the real service state is known. See 2026-08-15-telep-lan-split-horizon-caddy.
kraken.telep.lan — returns 502 while the Pi (192.168.1.200) is offline; the Caddy entry is correct, no action unless the Pi stays down.
Admin portal — authorize OliveTin’s ssh pubkey on the Pi (192.168.1.200) when the Kraken rig is back online (currently No route to host). Append /home/levander/admin-portal/olivetin/ssh/id_ed25519.pub to the Pi’s authorized_keys; mainframe + router are already authorized. Also outstanding: oauth2-proxy --trusted-proxy-ip not set (low risk, only Caddy reaches it) and remote/off-LAN access is deferred. See 2026-08-15-admin-portal-passkey-olivetin.
From the 2026-08-17 → 19 session (new services + network state, §§3c–3d above):
Passkey on watch.levandor.io for the dnsmon panel off-tailnet — needs a Pocket-ID OIDC client (user creates it) + a dedicated oauth2-proxy, same pattern as admin.levandor.io. Tailnet access already works via dnsmon.taild4189d.ts.net.
Unify the watchlist — the dnsmon panel watchlist (watchlist.json) and the router casino-alert.sh built-in list are currently separate; a change in one doesn’t affect the other.
Reserve 192.168.1.202 (Bambu printer) in DHCP on telep-router — offered, not yet done (pre-check with uci show dhcp | grep <ip-or-mac> per the duplicate-reservation lesson). Still pending after the 2026-08-26 reflash knocked the printer offline (recovered via router Wi-Fi restart): 2026-08-26-router-reflash-mac-5ghz-dfs-and-bambu-offline. Consider wiring the H2S over Ethernet to end the Wi-Fi churn.
Private Relay / DoH block was ATTEMPTED then FULLY REVERTED — it was classifier-flagged, caused a scare, and broke a Bambu connection during the dnsmasq restart. Evasive devices (e.g. an iPhone on iCloud Private Relay, .146) stay invisible to DNS monitoring. If revisited, do it PER-DEVICE, not LAN-wide.
Optional router accounting — nlbwmon for per-device WAN usage/history; QoS to prevent uplink saturation.
AirPlay (telep-tv) — ✅ RESOLVED 2026-08-12. Root cause: OpenWrt auto-injects ap_isolate=1 into the generated hostapd conf for bridged non-isolated APs (_wireless_set_brsnoop_isolation, /lib/netifd/netifd-wireless.sh line 309) whenever multicast_to_unicast is on — so uci/ubus showed isolate:false while /var/run/hostapd-phy*.conf still had ap_isolate=1. ⚠️ The old parked recipe here (uci set wireless.telep1_{5,2}.isolate='0' + wifi reload) was TRIED and is INSUFFICIENT — it does not touch the generated conf. FIX: commented out line 309 (#DISABLED-airplay-fix#, backup .bak) + wifi down; wifi up → telep1 no longer isolated (mDNS flows), telepcc/cams stays isolated. 🔴 Must be reapplied after any OpenWrt sysupgrade (patches a /lib/netifd/ file). Full write-up + reapply command: telep-router mDNS isolation — RESOLVED.
Printer (HP LaserJet M203dw) — Ethernet NIC confirmed hardware-dead (surge; serial VNC3920651, USB 03f0:632a). Set up on USB-to-CUPS (queue HP_M203dw_USB, needed usblp unloaded+blacklisted), then moved to WiFi via the HP Smart iOS app (M203dw has WiFi; Wi-Fi Direct pw 12345678, EWS 192.168.223.1). STILL PENDING: once the WiFi IP is known → repoint CUPS to a network queue, add a DHCP reservation on telep-router, and remove the USB queue + the two stale dead-NIC queues (HP_LaserJet_M203dw_B8AE8C and …@NPIB8AE8C.local). Details: 2. Printer (HP LaserJet M203dw) — Ethernet dead → USB → WiFi.
Enroll a BACKUP YubiKey (disk FDE) — only ONE key is enrolled; if lost/broken, boot needs the console passphrase. Repeat the root enroll for a 2nd key, disarm the live-lock during enrollment. See Caveats, gotchas, open items / yubikey-luks-lockout.
Encrypt sda (447 G) + the Windows NTFS disk (nvme0n1) — deferred until the user migrates data off + wipes them (Windows boot is expendable). Same YubiKey model.
KrakenSDR DoA rig (2026-08-12-krakensdr-doa-rig, separate host raspi/kraken-rig, not this box) — mobile DF now WORKING + field-testing (TETRA uplink 382.114 MHz, UCA 200 mm, Kraken Pro iOS app over hunter-ap → 10.42.0.1 using the phone’s own GPS — no tether/gpsd/USB-puck needed; zram reboot-safe). Remaining: array_offset calibration (bearings still relative) and a real 5 V/5 A car power source (Pi 5 + DAQ brown out on 5 V/3 A) — see the 2026-08-26→29 open items above. Notes: 2026-08-26-krakensdr-field-test-milestone · 2026-08-26-raspi-oom-zram-hunter-ap-433-retune.
65 flagged gallery images — classify at /image-review (progress saved; hit “Apply to gallery” when done).
Camera 24h format — times are correct; if a feed still shows AM/PM, the 12/24h toggle is a Tapo OSD setting (likely app-side; ONVIF can’t set it).
Camera-recording offload → /mnt/cam-archive — encrypted HDD ready but no offload cron exists yet.
141 unrecoverable orphan pages — staged in staging/orphan-fixes/; no marker source or genuinely generic. Manual only.
Nice-to-haves — merge cel/002+cel/010; AI-upscale the 150-dpi figures; harden the crash-prone camwall-x.
Hardware (your call) — 3rd GPU PCIe cable; a dedicated circuit for the space (mains is weak under load).
6. Conventions & recovery
Backups: every mutating pass leaves <file>.bak-<tag> / <dir>.bak-<tag> in place; site rollback = ~/knowledgebase/site.astro-prev; change logs under ~/knowledgebase/staging/.
Re-embed caveat: if a page’s Qdrant point is ever re-embedded/re-upserted, it drops chapter/summary — re-apply from chapter_index.json + summaries.jsonl.
Docs: full session knowledge is in this vault under projects/homelab/ (see LOG / TOPICS).
Secrets (camera / Telegram / Tailscale) live in existing configs/env only — never in this vault or git.
7. Related (separate projects)
Not homelab infra
projects/phone-camera-clone/ (“Aposemati”) — a DIY Continuity/Camo clone research effort, handed to another agent. Not part of this homelab stack. Key finding relevant to anyone hoping for a cross-platform receiver: Apple Wi-Fi Aware is effectively iOS-only, which narrows the cross-platform receiver options. See aposemati.