Levandor

Every Tailscale sidecar leaked a plaintext port onto the tailnet — the 127.0.0.2 hardening sweep

5 min read

Every userspace-networking Tailscale sidecar on telep-mainframe was forwarding its backend loopback port onto the node’s own tailnet IP as plain HTTP, sitting beside the tailscale serve TLS front door rather than behind it. This exposed an unauthenticated code-server shell and an SSO-bypassing raw Nextcloud to the whole tailnet in cleartext. The fix is to bind each backend to 127.0.0.2 instead of 127.0.0.1. All sidecars were swept; one remote-code surface (cad:9875) is left open and flagged for follow-up.

The systemic finding — tailscale serve does NOT close the raw forwarder

tailscaled --tun=userspace-networking unconditionally forwards inbound TCP on the node’s tailnet IP to 127.0.0.1 on the same port. Adding a tailscale serve --https=<p> http://127.0.0.1:<backend> mount puts a TLS door in front of the backend, but it does not disable that raw forwarder. So every sidecar answered on two doors:

  • https://<svc>.taild4189d.ts.net (:443) — the intended TLS front door.
  • http://<node-tailnet-IP>:<backend> — plain HTTP, no TLS, no identity, returning the same app.

The second door bypassed everything the first one added. For drive that meant raw Nextcloud on :11000 with the SSO/identity layer skipped entirely; for code it meant an unauthenticated browser VS Code + shell (--auth none) in cleartext.

The fix — bind the backend to 127.0.0.2

The forwarder only ever targets 127.0.0.1. Move the backend’s listener to 127.0.0.2 (still loopback, still host-only) and re-point tailscale serve at 127.0.0.2 too. The raw forwarder then finds nothing on 127.0.0.1:<port> and the plaintext door goes dead, while the TLS front door keeps working because serve was moved with it.

BEFORE:  serve :443 ──► 127.0.0.1:P ◄── raw forwarder (tailnet IP:P, plaintext)  ← LEAK
AFTER:   serve :443 ──► 127.0.0.2:P     raw forwarder (tailnet IP:P) ──► 127.0.0.1:P = nothing  ← dead

Per service the change is one of:

  • Container — flip the compose port map 127.0.0.1:P:... → 127.0.0.2:P:... and docker compose up -d <svc>.
  • Host process — change the app’s bind (systemd unit env / ExecStart flag, or a source bind line) and restart the unit.
  • Then tailscale --socket=/run/tailscale-<svc>/tailscaled.sock serve --bg --https=<p> http://127.0.0.2:P.

Test closure by IPv4, NOT by hostname

A MagicDNS hostname resolves IPv6 first, and the userspace forwarder does not forward IPv6 — so curl http://<svc>.taild4189d.ts.net:P returns a false “connection refused” and makes an open port look closed. Always probe the bare 100.x IPv4 of the node; that is the authoritative test. Every before/after below was verified by IPv4 from an owner tailnet peer.

House rule — ANY future sidecar must bind 127.0.0.2 from the start

This is a property of the whole per-service sidecar pattern (see tailnet-service-exposure-convention), not of any one service. A new sidecar that binds its backend to 127.0.0.1 reintroduces the plaintext door the moment it comes up. Bind 127.0.0.2, serve 127.0.0.2, and point the homepage siteMonitor at 127.0.0.2 too.

What was swept (IPv4-verified before → after)

All moved cleanly; every backend now REFUSED on the node’s tailnet IPv4, every TLS front door still serves.

ServiceNode:portBeforeAfterHow moved
drive Nextcloud100.110.208.117:11000200 (SSO-bypass!)refusednextcloud compose port map
drive tsauth-proxy:11001302refusedhardcoded Go const in main.go → 127.0.0.2, rebuilt, atomic-swapped binary
drive OnlyOffice DS:11002302refusednextcloud compose
drive drive-mcp:9100404refusedunit env DRIVE_MCP_HOST
code code-server100.96.41.118:8888302 (no-auth shell!)refusedcompose port map
monitor netdata100.97.52.21:19999200refusednetdata.conf [web] bind to
cad exports-http100.120.203.1:8085200refusedfreecad compose (nginx)
cad freecad GUI (noVNC):3080200refusedfreecad compose
cad freecad-mcp:9876200refusedfreecad compose
cad o3dv (serves Drive data):8087200refusedfreecad compose (Drive mount preserved)
cad freecad-open shim:8094400refusedserver.py LISTEN_HOST
transmute100.73.152.121:3313200refusedcompose
pdf/stirling100.101.222.61:8080200refusedcompose
dnsmon100.69.80.116:8099200refusedunit --host
knowledgebase kb-mcp100.116.0.83:9099404refusedunit env TELEP_KB_MCP_HOST
knowledgebase portal backend:8092200refusedapp.py serve(host=)
bambuddy100.84.98.18:8000200 (was 0.0.0.0!)refusedcompose env HOST
home/homepage100.93.133.62:3010400 (responds)refusedhomepage compose HOSTNAME

kb:8092 was a false negative in round 1

The first pass mislabeled the knowledgebase portal backend :8092 as “refused/safe”. It was actually open — a false negative from an IPv6-first hostname probe. Round 2 caught and closed it. This is exactly why the IPv4 rule above exists.

Two regressions the sweep created and fixed

  • drive-mcp lost its Nextcloud backend. DRIVE_MCP_NC_BASE still pointed at 127.0.0.1:11000, which went dead when Nextcloud moved to 127.0.0.2 in round 1 — drive-mcp’s WebDAV had been silently broken. Repointed to 127.0.0.2:11000.
  • Healthchecks went unhealthy. netdata, homepage and bambuddy image healthchecks probe 127.0.0.1/localhost, which is now empty. Added 127.0.0.2 healthcheck overrides (and 127.0.0.2:3010 to HOMEPAGE_ALLOWED_HOSTS). All back to healthy. This is a standard side-effect of the move: anything that self-probes on 127.0.0.1 must be repointed to 127.0.0.2.

Still open — flagged for follow-up

cad:9875 is FreeCAD's XML-RPC — a remote-code surface, still plaintext on the tailnet

cad:9875 (OPEN, HTTP 501) is FreeCAD’s XML-RPC server — the upstream that the freecad-open shim drives via execute_code. Left plaintext-open on the tailnet because closing it needs another FreeCAD-GUI container recreate plus a shim RPC_URL change, which could not be done safely under active-agent contention. cad:3081 (400) is the FreeCAD container’s other secondary port, also not serve-fronted. These are the one remaining exposure — remote FreeCAD scripting reachable in cleartext by any tailnet peer, including any tag:telep-user collaborator. Close them the same way (move the bind to 127.0.0.2, re-point the shim’s RPC_URL) when the box is stable.

Everything else visible on the tailnet is closed. No Funnel anywhere (verified — every mount is (tailnet only)).

Backups

Every edited file has a .bak-portclose (round 1) or .bak-portclose2 (round 2) sibling on the host: the netdata / code-server / nextcloud / freecad / transmute / stirling-pdf / bambuddy / homepage composes, the dnsmon / drive-mcp / telep-kb-mcp units, tsauth-proxy/main.go, freecad-open/server.py, and knowledgebase/app.py.

The host crash-looped during the sweep

The box rebooted at 06:20 mid-run — the known CPU fault, not the sweep. Because every change is on-disk (compose / unit / persisted serve state), all containers auto-restarted on 127.0.0.2 and every post-reboot verification passed.

Graph View

Backlinks