Levandor

telep-router Factory-Reset Recovery (casino-blocker DNS outage)

5 min read

A DNS-blocklist deploy on telep-router restarted dnsmasq with a broken config, then the deploying agent died mid-run — taking DNS down for the whole site. With no config backup, the router was factory-reset and rebuilt from scratch over SSH through telep-mainframe as a jump host. This note records the root cause, the full rebuild, the changed tailnet IP (100.115.194.51 → 100.69.112.32), the new config backup, and the lessons.

Router tailnet IP CHANGED — 100.115.194.51 → 100.69.112.32

The factory reset forced a fresh Tailscale node registration, so the router got a new tailnet IP. Any script/doc using root@100.115.194.51 must move to root@100.69.112.32 or, better, the stable MagicDNS name root@telep-router.taild4189d.ts.net (Tailscale SSH). The stale old telep-router node and a telep-router-1 duplicate were deleted from the tailnet.

For Agents

Access to the rebuilt router: /usr/bin/ssh root@telep-router.taild4189d.ts.net (Tailscale SSH) or root@100.69.112.32, or via the LAN 192.168.1.1. The mainframe’s ed25519 pubkey (levander@telep-mainframe) is in the router’s authorized SSH keys, so mainframe → router SSH works as a jump path. Config backup now exists: mainframe:/home/levander/telep-router-config-backup-20260803.tar.gz (13 KB, sysupgrade -b). Restore via LuCI → System → Backup/Flash Firmware → Restore. Deliberately NOT restored: the casino/gambling DNS blocker — it caused this outage. Redo only with offline config validation.

What happened (root cause)

A subagent was deploying a casino/gambling DNS blocker on the OpenWRT router: a dnsmasq blocklist of address=/domain/ sink entries plus a conf-file include. To apply it, the subagent restarted dnsmasq with the new/large config in place — then hit an API error mid-deploy and died, leaving dnsmasq in a broken state.

Result: dnsmasq failed to come back → DNS was down for the entire network. The site lost name resolution (and with it, effectively, connectivity). With no way to remotely recover a dead resolver on the only gateway, the user had to factory-reset the router.

Never blind-restart dnsmasq on the live gateway with a new/large config

dnsmasq is the site’s only resolver and runs on the only gateway. Restarting it with an unvalidated config, in a flow where a mid-run failure leaves it broken, takes the whole network down with no remote recovery path. See Lessons.

The router

PropertyValue
DeviceAsus RT-AX89X
FirmwareOpenWRT 24.10.0
Targetqualcommax/ipq807x, aarch64
radio05 GHz, ch 36, HE80
radio12.4 GHz, ch 1, HE20
LAN192.168.1.1
Tailnet100.69.112.32 (was 100.115.194.51) / telep-router.taild4189d.ts.net

Full survey: telep-router.

Rebuild from scratch (no backup existed)

The rebuild ran over SSH through telep-mainframe as a jump host (mainframe LAN 192.168.1.123 → router LAN 192.168.1.1), since the fresh router had only its default config and no tailnet identity yet.

WiFi

  • SSID telep1 on both radios — WPA3-SAE (encryption='sae'), network lan. Key restored into the router config (kept OUT of this vault).
  • SSID telep-cc on radio1 — WPA2-PSK (psk2), hidden, client isolation on → network cams.
  • Country HU.

cams camera network

  • Static 192.168.30.1/24 on bridge br-cams; DHCP pool enabled.
  • Camera DHCP reservations:
HostMACIP
telep_cam118:69:45:a9:01:25192.168.30.119
telep_cam2c0:3a:55:5c:8b:33192.168.30.139

(.119 is also set camera-side static.)

Firewall

  • New cams zone: input ACCEPT, forward REJECT, output ACCEPT.
  • Forwardings: cams → wan (cameras reach internet) and lan → cams (so the mainframe / Frigate reaches the cameras).

Difference from the pre-reset firewall posture

The rebuilt cams → wan allows camera internet, whereas the original Camera VLAN (telep-cc) deliberately rejected cams → wan (no internet, NTP DNAT’d back). If the no-internet-for-cameras posture is still wanted, re-tighten cams → wan back to REJECT + restore the NTP DNAT.

DNS query logging (restored)

  • dhcp.@dnsmasq[0].logqueries=1
  • logfacility=/tmp/dnsmasq-queries.log
  • /etc/dns-log-rotate.sh + an hourly cron.

Rotate script rewritten to be outage-safe

The old rotate script did mv + dnsmasq restart — itself a DNS-outage risk on every rotation. The new /etc/dns-log-rotate.sh truncates the log in place at 20 MB (no dnsmasq restart), so log rotation can never drop DNS.

IPv6 fix (restored)

A fresh OpenWRT would re-trigger the ISP bad-prefix slow-internet issue (2026-07-28-ipv6-slow-internet), so the LAN was made IPv4-only again:

  • network.wan6.disabled=1
  • dhcp.lan.ra=disabled
  • dhcp.lan.dhcpv6=disabled
  • dhcp.lan.ndp=disabled

Tailscale (fresh node registration)

  • opkg install tailscale → v1.80.3 + kmod-tun.
  • Re-joined as node telep-router, tag:telep, --ssh, --accept-dns=false, using an auth key minted via the Tailscale API.
  • LuCI re-served: tailscale serve --https=443 http://127.0.0.1:80 → https://telep-router.taild4189d.ts.net.
  • OS hostname set to telep-router.
  • New tailnet IP 100.69.112.32 (see the callout at top). Stale telep-router + telep-router-1 duplicate nodes deleted.

SSH access

The mainframe’s ed25519 pubkey (~/.ssh/id_ed25519.pub, levander@telep-mainframe) was added to the router’s authorized SSH keys, so mainframe → router SSH works.

Config backup (now exists)

Backup location + restore path

mainframe:/home/levander/telep-router-config-backup-20260803.tar.gz (13 KB, made with sysupgrade -b). Restore: LuCI → System → Backup/Flash Firmware → Restore. Recommendation: make periodic backups so a future reset is a restore, not a rebuild.

Verified working after rebuild

  • WiFi APs up on both radios.
  • Both cameras reachable; all 4 feeds streaming into Frigate (go2rtc producers).
  • frigate-notify + camwall active.
  • Internet OK; DNS query log growing.
  • Tailscale reachable directly at the new IP / MagicDNS name.

Not restored (deliberately)

The casino/gambling blocker/detector was NOT put back — it caused the outage. Redo only with offline config validation and a safe deploy pattern (below).

Lessons

1. Validate dnsmasq config BEFORE restarting it

Always dnsmasq --test -C /etc/dnsmasq.conf (test-parse) before applying a new/large config on the live router. A config error must be caught before the resolver is bounced, not after it is already dead.

2. Keep a router config backup

No backup existed → the only recovery was a full factory reset + hours of rebuild. A sysupgrade -b backup now exists; make them periodically. A reset should cost a restore, not a rebuild.

3. Safe deploy pattern for network-critical router changes

Never run a change on the only gateway in a way that a mid-run failure leaves DNS/routing broken. Stage + validate the config offline, keep the current config backed up, and apply atomically so a crash rolls back rather than bricking resolution. Don’t restart dnsmasq/network blind.

Graph View

Backlinks