Levandor

Camera OSD clocks drift — Tapo NTP never syncs, fixed with an hourly ONVIF time push

5 min read

The burned-in OSD timestamps on the telep-mainframe cameras were wrong on both camera devices — the ÉSZAK pair (right column of the camwall) by 109 days, the DÉL pair (left column) by exactly 1 hour. Root cause: the cameras sit on the isolated cams VLAN with no working NTP, and the Tapo firmware ignores the ONVIF TimeZone and DaylightSavings fields. Fixed with cam-timesync.timer on the host, which pushes host time over ONVIF hourly with a DST-aware offset compensation.

For Agents

  • Scope: OSD only. This is the timestamp painted into the video by the camera. Frigate stamps its own recordings/events from host time, so recording timelines, event times and alerts were always correct. Only the pixels lied.
  • The tell: curl -s -o /tmp/x.jpg 'http://127.0.0.1:5000/api/telep_cam3/latest.jpg' and read the top-left corner. Compare to date.
  • The fix is installed and automatic: cam-timesync.timer (hourly, Persistent=true, OnBootSec=3min). Manual run: sudo /usr/local/bin/cam-timesync.py (add --force to push even when skew ≤ 60s).
  • Do NOT try to fix this by setting the ONVIF timezone. Tapo firmware accepts the SOAP call, returns HTTP 200, and silently keeps its own TZ. Verified twice.

Symptoms

Camwall paneCameraDeviceOSD readActualError
top/bottom lefttelep_cam1 / telep_cam2 (DÉL)192.168.30.1192026-08-09 00:472026-08-08 23:47+1 h
top/bottom righttelep_cam3 / telep_cam4 (ÉSZAK)192.168.30.1392026-04-21 04:322026-08-08 23:47−109 d

Pane→camera mapping per 2026-07-28-camwall-4-substream-composite (DÉL = left column, ÉSZAK = right column). Both physical devices are dual-lens Tapos, so one broken device = two broken panes.

Diagnosis

Unauthenticated GetSystemDateAndTime (ONVIF allows this one without WS-Security) on port 2020 tells you everything:

curl -s -X POST "http://192.168.30.139:2020/onvif/device_service" \
  -H 'Content-Type: application/soap+xml; charset=utf-8' \
  -d '<?xml version="1.0"?><s:Envelope xmlns:s="http://www.w3.org/2003/05/soap-envelope"><s:Body><GetSystemDateAndTime xmlns="http://www.onvif.org/ver10/device/wsdl"/></s:Body></s:Envelope>'
  • .139 (ÉSZAK): DateTimeType=NTP, DaylightSavings=true, clock stuck at 2026-04-21. Authenticated GetNTP returned FromDHCP=true, IPv4Address=0.0.0.0 → the camera has no NTP server address at all. It claims NTP mode and syncs against nothing, so its RTC has been free-running since ~21 April.
  • .119 (DÉL): DateTimeType=Manual, DaylightSavings=false, and its UTCDateTime field held local CEST, not UTC. With TZ=GMT-01:00 on top, the OSD landed an hour ahead.

Host time itself was fine (timedatectl: synchronized, Europe/Budapest), as was the Frigate container’s (/etc/localtime bind-mounted, TZ=Europe/Budapest).

Why NTP never worked

Camera VLAN (telep-cc) documents the design as “NTP is DNAT’d back to the router, so a camera with no internet still keeps its clock.” The DNAT rule exists, but DHCP option 42 is not being served on the cams VLAN, so the cameras never learned an NTP server IP to send packets to in the first place. A DNAT of traffic that is never generated does nothing. (192.168.30.1:123 also did not answer an NTP probe from the LAN side, though that may be interface-scoped and is not conclusive.)

Gotchas

Tapo silently ignores the ONVIF TimeZone and DaylightSavings fields

SetSystemDateAndTime returns HTTP 200 with a clean SetSystemDateAndTimeResponse and then keeps TZ=GMT-01:00 regardless of what you sent — tried both a full POSIX string (CET-1CEST,M3.5.0/2,M10.5.0/3) and the device’s own GMT-02:00 format. DaylightSavings=true is likewise not applied to the local-time computation. Only the date/time numbers in UTCDateTime are writable. A 200 here means “message parsed”, not “setting applied” — always read back with GetSystemDateAndTime and compare LocalDateTime to date.

The ONVIF TZ string is POSIX-signed, so GMT-01:00 means UTC +1

Confirmed empirically on this hardware: with TZ=GMT-01:00 the device reports LocalDateTime = UTCDateTime + 1 h. Do not read that string as “one hour behind GMT”. cam-timesync.py parses the sign and derives the real offset rather than assuming.

DateTimeType=NTP does not mean the clock is synced

.139 self-reported NTP mode the entire time it was 109 days out. The mode field is aspiration, not status. Always check GetNTP for an actual server address — 0.0.0.0 + FromDHCP=true is the “never syncs” signature.

The fix — cam-timesync

Because the timezone is unwritable and NTP is unreachable, the host pushes pre-compensated time: read the camera’s own TZ offset, subtract it from host local time, and write the result into UTCDateTime so the camera’s (wrong but fixed) TZ arithmetic lands on the correct local time.

Files installed on telep-mainframe:

PathRole
/usr/local/bin/cam-timesync.pyONVIF WS-Security digest client; reads state, pushes compensated time, reads back to verify
/etc/cam-timesync.envFRIGATE_CAM{,2}_{USER,PASSWORD}, mode 0600
/etc/systemd/system/cam-timesync.serviceoneshot, After=time-sync.target
/etc/systemd/system/cam-timesync.timerOnBootSec=3min, OnUnitActiveSec=1h, Persistent=true, enabled

Behaviour: skips any camera already within 60 s (so the hourly run is a cheap no-op), otherwise pushes and re-reads, logging corrected -3601s -> 2026-08-08 23:49:13 (skew -0s). Non-zero exit if a camera can’t be reached or won’t take the correction.

It survives the DST changeover on its own

The compensation is recomputed every run from host local offset vs camera TZ offset, both read live. On 25 Oct 2026 the host drops to UTC+1, the camera stays GMT-01:00, the compensation goes to zero, and the OSD stays correct with no intervention. This is why the offset is derived rather than hardcoded to 1 h.

.139 was moved from DateTimeType=NTP to Manual as part of this — deliberate, since its NTP was decorative and Manual is what the push maintains.

Verified: fresh latest.jpg from both telep_cam1 and telep_cam3 read 2026-08-08 23:49:48 against a host clock of 23:49:48.

Open follow-up — the root cause is router-side

The host-side push fully fixes the symptom and is robust to reboots, drift and DST, but the actual gap is that the cams VLAN never hands out an NTP server. The clean fix on telep-router is to serve DHCP option 42:

uci set dhcp.cams.dhcp_option='42,192.168.30.1'
uci commit dhcp && /etc/init.d/dnsmasq restart

…and confirm OpenWrt’s NTP server is actually listening on the cams interface. Not done here — router changes carry real blast radius on this box (2026-08-03-telep-router-factory-reset-recovery) and the timer already covers it. If option 42 is fixed later, cam-timesync can stay as a harmless belt-and-braces check (it no-ops when skew ≤ 60 s), though the Tapo TZ bug means the +1 h DST error would come back under pure NTP — so keep the timer regardless.

Graph View

Backlinks