The site’s WAN moved off the Telekom NE200 5G FWA CPE onto Starlink in Bypass mode, closing out the chronic episodic WAN degradation investigation. This note records the cutover evidence, the proof of a clean bypass, the Starlink CGNAT ↔ Tailscale 100.64.0.0/10 address collision, how to reach the dish from the LAN, the new grpcurl-based dish telemetry logger, and the verdict that a Starlink router can never be an AP here without breaking presence detection.
For Agents — quick facts
WAN: Starlink Bypass. Router WAN 100.87.12.221/10, gateway 100.64.0.1. Still CGNAT — no inbound ports, ever.
Clean-bypass proof: 100.64.0.1 and 192.168.100.1 share one MAC26:12:ac:1a:80:01.
Telemetry: /home/levander/net-monitor/starlink_status.py → starlink.csv, cron */5 as user levander on telep-mainframe.
🔴 The single highest-value gotcha: grpcurl needs -emit-defaults or all the healthy-state zero values silently vanish.
NE200 logger retired (cron line removed); ne200_signal.py + 342 rows of rflog.csvdeliberately preserved as the evidence base.
🔴 Never run /etc/init.d/network reload|restart on telep-router — it drops the site and the tailnet SSH used to manage it.
❌ Do NOT renumber the LAN.192.168.1.0/24 collides with nothing.
1. The migration — before and after
The NE200’s own RF logger wrote its final entry at 2026-09-04T02:15, still degraded right to the end — exactly the condition 2026-08-31-episodic-wan-degradation identified:
RSRP -103 dBm, SINR 9, downlink QPSK
After the swap to Starlink:
Metric
NE200 (before)
Starlink (after)
WAN address
192.168.254.2/24 (double-NAT behind CPE)
100.87.12.221/10, gw 100.64.0.1
Latency
~25-40 ms, episodic collapse
18.3-35.3 ms, 0% loss (8 pings)
Wired throughput
180-265 Mbit normal, 3-17 Mbit in episodes
235-279 Mbps (Cloudflare + Cachefly 50 MB)
Path
QPSK-capped cellular N78
traceroute reaches Cloudflare in 5 hops via Starlink backbone
The NE200 at 192.168.254.1 is now 100% unreachable — it is fully out of the path.
Topology finding — the dish and the gateway are the same device
Same MAC ⇒ same device. The OpenWrt WAN port talks straight to the dish / bypass bridge with no Starlink router doing any routing in between. This ARP comparison is the diagnostic that proves a clean Bypass — reuse it if bypass is ever in doubt.
Still CGNAT — inbound ports remain impossible
100.64.0.0/10 is Starlink’s CGNAT range. There is no public IPv4 and no inbound port forwarding, exactly as with the NE200’s double-NAT. Everything tailnet/DERP-based is unaffected. This is why the tailcat plan for the offline company Mac remains the right approach — outbound-only, tailnet-mediated.
2. 🔴 The Tailscale ↔ Starlink 100.64.0.0/10 collision
Starlink CGNAT and Tailscale both use 100.64.0.0/10. Neither is configurable. This appeared the moment the WAN was swapped — the NE200 handed out 192.168.254.2/24, which did not overlap. The router now carries a link-scope route 100.64.0.0/10 dev wan.
Tailscale survives this because it uses policy routing, not the main table:
ip rule → priority 5270 → table 52
Table 52 holds per-peer /32 routes and is consulted before the main table, so peer traffic never sees the /10.
Verified healthy state from tailscale status --json:
Empty health, UDP working, and endpoint-independent mapping ⇒ Starlink’s CGNAT is well-behaved for NAT traversal.
The trap: ip route get showing dev wan for a peer is NOT collision damage
ip route get for 100.84.98.18 (bambuddy) and 100.120.203.1 (cad) resolves to dev wan, not dev tailscale0. This looks like the collision eating tailnet routes. It is not. Those peers are simply absent from table 52 because the tailnet ACL does not grant telep-router (tag:telep) access to those tagged-devices. They would be unreachable regardless of Starlink.
The only real consequence of the collision: packets to ACL-denied tailnet peers leak toward the Starlink link instead of failing closed, because the /10 route is link-scope. Cosmetic, not an outage.
ping is useless as a tailnet reachability test here
100.115.209.87 reports unreachable via ping while routing correctly over tailscale0 — ICMP is filtered. Test with an actual TCP service (ssh/http), never ping.
Do NOT renumber the LAN
192.168.1.0/24 collides with nothing — the dish is 192.168.100.1 and the Starlink WAN is 100.x. The real overlap is 100.64.0.0/10, and no change to any local subnet can fix it. Renumbering would be pure churn (and would break every reservation, .lan name, and reference in this vault).
3. Dish access in Bypass mode
In Bypass mode the dish at 192.168.100.1 is not routed by default. Two things were added on telep-router:
a) Live route (non-disruptive):
ip route add 192.168.100.1/32 dev wan
b) Boot persistence — TWO independent mechanisms (belt and braces, because only one of them could be tested):
uci stanza committed to /etc/config/network — applies via netifd at boot:
⚠️ Never verified, because the network was deliberately never reloaded (see the safety note below).
/etc/hotplug.d/iface/99-starlink-dish (mode 0755) — fires on wan ifup, uses ip route replace so it is idempotent. ✅ This one WAS verified for real: the route was deleted, the script run, and the route came back. Wrong-interface (lan ifup) and wrong-action (wan ifdown) both correctly no-op.
/etc/init.d/network reload (or restart) was deliberately NEVER run on this router
✅ Reuse the live-then-persist pattern: apply with ip route (takes effect instantly, disrupts nothing), then write the uci config without reloading, and add a hotplug script you can test in isolation.
Any LAN client reaches the dish with the route in place — masquerade on the wan zone (firewall.@zone[1].masq='1') plus the default lan→wan forward handle it. Verified open from a LAN host: ports 80, 8080, 9200, 9201.
❌ The Starlink mobile app does NOT work in Bypass mode
The app is built to talk to the Starlink router, whose WiFi and services are off in Bypass. Symptom observed: connecting a phone to a spare Starlink router’s WiFi gave “unreachable” for the dish and “Not on account, this router doesn’t seem to be associated with your Starlink” for the router itself (a second-hand / unregistered unit).
✅ Use the dish’s own web UI: http://192.168.100.1 from a client on the OpenWrt WiFi (telep1). It is fully self-contained — verified it serves its own /static/js/script.js.gz (172166 bytes) and /static/css/style.css locally.
Two red herrings while debugging the dish UI
webapp.starlink.com is NXDOMAIN. It appears only as a permissive CSP entry and is not required — the UI loads entirely from the dish.
grpcurl fails against port 9201. That port is gRPC-web (different framing), not plain gRPC. A raw POST returns HTTP 200 with proper gRPC headers, so the port is healthy. Use :9200 for grpcurl.
The dish’s own CSP header is what reveals the UI’s real dependency:
connect-src http://192.168.100.1:9201/SpaceX.API.Device.Device/Handle
It deliberately mirrors the retired NE200 logger’s conventions (see 2026-07-28-net-monitor and 2026-08-31-episodic-wan-degradation): stdlib Python driving an external binary, an HDR constant, header written only if the CSV is absent, time.strftime("%Y-%m-%dT%H:%M:%S%z") timestamps, missing values become empty strings, and it never crashes a row.
Client: grpcurl, not Python grpc
grpcurl v1.9.4, static binary in /usr/local/bin (root:root, 0755). Chosen because Debian 13 enforces PEP 668 — grpcio would mean a venv plus a compiled toolchain for a single RPC. Server reflection means no .proto files are needed.
SpaceX.API.Device.Device/Handle {"get_status":{}} port 9200
🔴 THE CRITICAL FLAG — -emit-defaults
By default grpcurl omits proto3 zero-values, which silently drops popPingDropRate, fractionObstructed, currentlyObstructed and timeObstructedexactly when they are zero — i.e. exactly when everything is healthy.
A logger built without this flag looks fine and records nothing useful. If this logger is ever rebuilt, ported, or reimplemented in another language, this is the first thing to get right.
state — no state enum exists; deviceState carries only uptimeS. The CSV’s state column is therefore derived, not dish-reported:
outage.cause if outage is non-null, else
disablementCode if it is not OKAY, else
CONNECTED if the core ready states are all true, else
NOT_READY
snr — gone. Only the boolean isSnrAboveNoiseFloor survives, so signalQuality is logged instead.
The cady ready-state trap
readyStates contains a legacy cady key that is permanently false on rev4 hardware, so a naive all(readyStates.values()) yields NOT_READY on a perfectly healthy dish. Fixed with an explicit core set: ("scp", "l1l2", "xphy", "aap", "rf").
Other quirks baked into the data
⚠️ alerts reads noEthernetLink on EVERY row while ethSpeedMbps is 1000. This is a Bypass-mode firmware artifact, not a fault. Consequence: the alerts column is never empty, so it is useless as a naive “any alert” trigger. Any alerting must exclude noEthernetLink.
⚠️ avg_prolonged_obstruction_interval_s can be the literal string NaN until enough data accumulates. Anything parsing the CSV must handle it.
.err receives success rows too (mirroring the NE200 convention). At */5 it grows ~4× faster than the old */15 logger did. No rotation exists — this will need attention.
grpcurl is pinned at whatever was latest on install day, root-owned, and not tracked by any package manager. Nothing will update it.
5. ⚠️ Early observation — obstruction climbing (UNCONFIRMED)
Over the dish’s first hour:
Sample
fraction_obstructed
1
0.0012
2
0.0061
3
0.0208
4
0.0222 (2.2%)
and avgProlongedObstructionIntervalS fell from 21600 s (6 h) → 2700 s (45 min).
Treat as early and NOT yet trustworthy
Obstruction statistics need roughly 12 hours to settle, and dish uptime was under an hour. These numbers are not yet evidence. But the direction was worsening, not settling — worth re-checking once a full day of starlink.csv exists.
The obstruction map at http://192.168.100.1 shows which sky sector is blocked. The CSV now provides the before/after evidence for any reposition — capture a baseline before moving the dish.
6. NE200 logger retired
The */15 cron line for ne200_signal.py was removed from levander’s crontab. Backup: /root/levander.crontab.bak.1788486106.
✅ The script and its 342 rows of rflog.csv history were deliberately preserved — that data is the evidence base that justified the Starlink switch and is referenced by 2026-08-31-episodic-wan-degradation. Do not delete it.
rflog.err had grown to ~30 KB of failures against a modem that no longer exists, which is what prompted the retirement.
Do NOT "optimise" to HE80 — there is no internet gain
The wired line is ~250 Mbps and HE40 already delivers a 573 Mbit/s PHY rate. Widening to HE80 would only help LAN-to-LAN transfers, not internet speed. Recorded so nobody spends a maintenance window on it. (Cf. 2026-08-15-router-wifi-tuning-htmode-band-penetration.)
The TL-WA850RE extender caps at ~50 Mbps
192.168.1.101, eth/mgmt MAC ac:84:c6:1b:04:30, WiFi BSSID ac:84:c6:1b:04:31. Measured ~50 Mbps ≈ 20% of the line.
CORRECTED 2026-09-05 — it is NOT a repeater
This section originally attributed the ~50 Mbps ceiling to repeater-mode halving. That was wrong, and the claim was an inference from the model name rather than a measurement.
[[telep-router#tp-link-tl-wa850re-added-as-a-wired-ap-ssid-bandi|TP-Link TL-WA850RE added as a wired AP (SSID bandi)]] documents this device as already reconfigured repeater → Access Point, wired into br-lan, broadcasting SSID bandi. It is not repeating anything.
The real ceiling is structural: a 100 Mb Fast-Ethernet uplink port plus a single-band 2.4 GHz N300 radio. The telep-router note already calls it a “coverage AP, not a fast-lane AP” for exactly this reason. Add the 2.4 GHz congestion measured on 2026-09-04 (ch11 at 93% busy) and ~50 Mbps is the expected result.
✅ Fix = a dual-band AP on a gigabit port, if fast-lane coverage is wanted. Swapping “repeater for wired AP” is a no-op — that conversion already happened.
⚠️ bandi-vip is a different device. An AP with that SSID was seen on ch11 at −73/−77 dBm with BSSID 5E:6D:F7:27:10:E3 — locally-administered and not TP-Link’s ac:84:c6 OUI, so it is not this extender. Unidentified.
🔎 It also answers on a second factory IP 192.168.0.254 (same MAC) — which explains a previously mysterious ARP entry.
8. 🔴 Verdict: Starlink routers CANNOT be access points
Verified against current documentation. A Starlink router has exactly two states, and neither is an AP:
Mode
Behaviour
Bypass
WiFi off entirely, pure L2 bridge
Not bypass
It is the main router, doing its own NAT and DHCP
There is no AP/bridge mode that joins a third-party network. Starlink’s own mesh nodes have no Ethernet backhaul (WiFi only) and pair only to a Starlink router acting as main router — so they are incompatible with Bypass by definition. Reversing Bypass requires a factory reset.
Why going non-Bypass would be actively harmful here — presence detection would break
Making the Starlink router the main router would displace telep-router and take out:
br-cams — the 192.168.30.0/24 camera VLAN and the telep-cc SSID
dnsmasq leases and .lan names
critically, the iwinfo assoclist on phy0-ap0 / phy1-ap0
Both the intruder alarm and the tv-presence daemon compute presence as the union of arp-scan and that assoclist. It was proven live that a sleeping iPhone (38:7f:8b:df:2a:79) is ABSENT from arp-scan while PRESENT in the assoclist.
Losing the assoclist drops both systems to arp-scan only — the exact configuration that switches the TV off with someone standing in front of it and arms the alarm with people home.
Additionally: an unmanaged Starlink router plugged upstream of the OpenWrt box puts its WiFi clients outside the firewall, DNS, VLANs, and both presence systems entirely.
/home/levander/tapo-ctlalready contains a venv with python-kasa 0.10.2 (plus a set-cam-time.py run nightly at 04:30) — the same version the separately-built Tapo P115 bridge installed into /opt/tapo-bridge/venv. Two venvs, same library.
Useful consequence: TP-Link credentials likely already exist on the box, which may make populating the P115 bridge’s /etc/tapo-bridge.env much easier than re-deriving them. (Cf. 2026-08-08-camera-osd-clock-drift-onvif-timesync for the nightly camera time job.)
10. Triage / runbook
Reach the dish
Connect to the OpenWrt WiFi telep1 and open http://192.168.100.1 in a browser. The obstruction map and full status live there. The Starlink mobile app will not work — Bypass mode disables the router services it talks to.
Dish unreachable after a router reboot? Re-add the route
ip route add 192.168.100.1/32 dev wan # or: ip route replace ...
Then check that /etc/hotplug.d/iface/99-starlink-dish still exists and is 0755 — that is the mechanism that was actually verified to restore it on wan ifup.
Never reload the router network to "apply" config
Do not run /etc/init.d/network reload or restart on telep-router. It drops the site and the tailnet SSH session you are using. Apply changes live with ip route, then persist to uci without reloading.
Confirm Bypass is still clean
Compare the two ARP entries — same MAC means the OpenWrt WAN port is talking directly to the dish:
If the telemetry logger is ever rebuilt: -emit-defaults
grpcurl omits proto3 zero-values by default, so popPingDropRate, fractionObstructed, currentlyObstructed and timeObstructeddisappear precisely when the dish is healthy. Always pass -emit-defaults. Also: use port 9200 (:9201 is gRPC-web and grpcurl cannot speak it), and exclude the permanent noEthernetLink alert from any trigger logic.
Testing tailnet reachability
Do not use ping — ICMP is filtered and healthy peers report unreachable. Use a real TCP service. And remember that a peer resolving to dev wan usually means an ACL denial, not a routing bug.
Related
2026-08-31-episodic-wan-degradation — ⭐ the NE200 5G root-cause investigation this migration closes out; its rflog.csv evidence is preserved
2026-07-28-net-monitor — the original net-monitor logger whose conventions starlink_status.py mirrors
telep-router — the OpenWrt box holding the WAN, the dish route, the hotplug script, and the assoclist
telep-mainframe — host running the telemetry cron, grpcurl, and the retired NE200 logger