Levandor

Runbook: Camera wall is black / not showing on the TV

5 min read

Mid-incident? Try this first 

ssh levander@100.115.209.87        # mainframe
systemctl is-active camwall-x camwall
cat /run/uxplay/casting            # 1 = an AirPlay cast stopped the grid
sudo systemctl restart camwall     # bounce the 2x2 grid (X server stays up)

Two services: camwall-x = the persistent Xorg :0 on HDMI-1; camwall = the 4× mpv 2×2 grid that runs on it.

Symptoms

  • The TV shows black / no camera grid.
  • One or more panes are frozen or black while others are live.
  • The grid vanished after someone AirPlayed to the TV.

Diagnose → Fix

  1. Are both services up? 
    systemctl is-active camwall-x camwall
    • camwall-x inactive → the Xorg :0 server died: sudo systemctl restart camwall-x camwall.
    • camwall inactive → only the grid is down (see step 2).
  2. Did an AirPlay cast stop the grid and not restart it? 
    cat /run/uxplay/casting            # 1 = mid-cast; 0 = idle
sudo systemctl start camwall
    If it reads 1 but nobody is casting, the uxplay handoff wrapper missed the disconnect string — start camwall manually.
  3. Are the cameras reachable? They live on the telep-cc WiFi at 192.168.30.x — if they drop, the panes go black: 
    ping -c1 192.168.30.139            # ÉSZAK fix (cam3/cam4)
ping -c1 192.168.30.119            # DÉL
curl -s -o /dev/null -w '%{http_code}\n' http://127.0.0.1:1984   # go2rtc UI (:1984)
    go2rtc restreams RTSP on :8554 and its UI is on :1984.
    • ALL 4 cameras fps=0 and BOTH cams unreachable (not just one pane) → suspect the cams-VLAN bridge is DOWN on the router, not the cameras. Jump to step 3a.
  4. Reconnect panes to live streams — a movie=/per-pane mpv can freeze on its last frame after a go2rtc restart: 
    sudo systemctl restart camwall
    3a. camwall frozen / ALL cameras fps=0 → check the cams-VLAN bridge on the router. If every camera is fps=0 and both .119/.139 are unreachable (cameras still show in iwinfo assoclist — associated but L3-dead), the router’s wireless-only bridge br-cams likely dropped after a wifi reload (e.g. a radio1 channel change during WiFi tuning). On telep-router (ssh root@100.69.112.32): 
    ip addr show br-cams            # state DOWN, qdisc noop, NO inet = ROOT CAUSE
    If DOWN / no inet 192.168.30.1/24, a forced down/up cycle is needed — ifup cams alone does NOT work: 
    ifdown cams; sleep 2; ip link set dev br-cams up; ifup cams; sleep 5
ip addr show br-cams            # now UP with inet 192.168.30.1/24
    Then back on the mainframe: sudo systemctl restart camwall. Full incident + why: 2026-08-15-camwall-dead-br-cams-bridge-down-after-wifi-reload.

    Don't chase the cameras when br-cams is down br-cams down the cameras are L3-unreachable, so an ONVIF reboot will ConnectTimeout on port 2020 — a misleading symptom. Fix the bridge first, THEN treat any still-wedged pane as a single-camera issue (step 5). [!warning] If br-cams shows an address but ifstatus cams says DEVICE_CLAIM_FAILED → ifup cams won't fix it A DIFFERENT wifi reload failure: ip addr show br-cams may look present but ifstatus cams = "up": false + errors: ["DEVICE_CLAIM_FAILED"] and there's no 192.168.30.1. ifup cams (and even ip link set br-cams down; /etc/init.d/network reload; ifup cams) do NOT clear it — this is also why cams-guard's per-minute ifup cams fails to self-heal. Only a full /etc/init.d/network restart works. Verify: ifstatus cams | grep '"up"' = true, ip -4 addr show br-cams shows 192.168.30.1. Full incident: 2026-08-31-cams-vlan-device-claim-failed-wifi-reload. [!info] camwall stuck in activating (start-pre)? That's the same root cause camwall.service gates on Frigate (ExecStartPre = until curl -sf http://127.0.0.1:5000/api/version). If the cams VLAN is down, Frigate has no feeds and camwall waits forever — fix the cams gateway on the router first, camwall then goes active on its own.

    With

  5. A single camera is wedged (its pane stays frozen/black after a bounce, but br-cams is UP and the camera IS pingable) → ONVIF reboot it: 
    ~/tapo-ctl/venv/bin/python ~/tapo-ctl/reboot-cams-onvif.py

    Use ONVIF, not pytapo — the pytapo/Tapo-cloud creds are dead (cameras are internet-isolated). ONVIF is the working reboot path. python3 The onvif module lives only in ~/tapo-ctl/venv. A bare python3 ~/tapo-ctl/reboot-cams-onvif.py fails ModuleNotFoundError: onvif. Always use ~/tapo-ctl/venv/bin/python.

    [!bug] Run tapo-ctl scripts via the venv python, not system

  6. No display at all (TV dark, not even black panes) → check the Xorg :0 output: HDMI-1 should be 3840x2160 on the NVIDIA output. sudo systemctl restart camwall-x camwall.

Deeper causes / links

Graph View

Backlinks