Levandor

Network.framework, QUIC and AWDL Gotchas

13 min read

Traps in Apple’s Network.framework found by execution during the 1 build and the Phase 2 live-preview build. Several of these overturned things this vault previously recorded as settled.

For Agents

All of these are measured on macOS 26.x / iOS 26.x with the 26.2 SDK. Every one of them looks healthy while broken — that is what they have in common.

1. NWBrowser.Result.interfaces is the only public source of an AWDL interface

The single most load-bearing API fact in the project

NWPathMonitor never reports awdl0, under any configuration. The only public place an AWDL NWInterface object appears is NWBrowser.Result.interfaces, which is public let ... [NWInterface].

Take it from the browse result and pin it with NWParameters.requiredInterface. NWParameters takes interface objects, not names — which is exactly why there is no other way in.

Confirmed empirically before it was relied on: a browse returned the service on en13, awdl0, lo0, en0, en10; pinning requiredInterface to awdl0 succeeded; the accepted connection came from fe80::b894:7bff:fece:a351%awdl0.53904 — a genuine AWDL link-local address.

Never filter by interface type

awdl0 reports type == .wifi. So prohibitedInterfaceTypes([.wifi]) — the intuitive way to say “not the router” — kills the peer-to-peer path you are trying to select. Interface identity is the discriminator, not interface type.

Corollaries the build settled:

  • Classify by name, with a prefix-plus-ASCII-digits rule (awdl0, llw0 → peer-to-peer; en0, en13 → infrastructure; lo0 → loopback; everything else → other). hasPrefix alone lets a hypothetical llwhatever read as peer-to-peer, which is the expensive direction.
  • Consumers must test == .peerToPeer positively, never != .infrastructure. The mutation kind != .infrastructure && kind != .loopback makes a VPN tunnel utun4 report as “direct” — a false direct-link claim is the exact failure this product exists to catch.
  • Availability is not carriage. Reading currentPath.availableInterfaces and preferring an awdl* name can render ”— direct” from an interface that carried nothing. Derive the interface name from the busiest path in PendingDataTransferReport.pathReports, and report nothing before bytes have moved.

2. QUIC does not support TLS-PSK on this platform

Measured on plain 127.0.0.1 — no Bonjour, no service type, no radio

ConfigurationResult
QUIC + PSK, as specified-9858 errSSLHandshakeFail
QUIC + PSK + client selection block-9858
QUIC + PSK + selection blocks on both sides-9858
QUIC + PSK + server identity hint-9858
QUIC + PSK + min TLS 1.3-9858
TLS over TCP + the identical PSKhandshake OK, 1024 bytes delivered
TLS/TCP + a deliberately mismatched key-9846 bad MAC

The mismatched-key control is what makes this proof rather than anecdote: the TCP success is genuine PSK authentication, not a silent fallback to something unauthenticated.

Supporting SDK evidence: -9858 is errSSLHandshakeFail (SecBase.h:879) — a local failure to build a handshake, not errSSLPeerHandshakeFail (-9824), which is what a peer’s alert produces. The modern tls_ciphersuite_t enum in SecProtocolTypes.h contains no PSK ciphersuites at all; TLS 1.2 PSK suites survive only in the deprecated SSLCipherSuite enum. QUIC.TLS exposes only localIdentity, certificateValidator, peerAuthentication and cipherSuites.

This overturned the design mid-build. The spec had chosen PSK explicitly because “a pre-shared key is just bytes from a pairing code and needs no SecIdentity or keychain at all”. That rationale collapsed, and pairing moved to a self-signed in-memory identity — see aposemati-pairing-security-model.

3. .ready proves nothing on a QUIC listener

A QUIC NWListener reaches .ready with a broken — or entirely absent — security configuration.

QUIC defers handshake failure to handshake time, so a naive spike looks like it works right up until a client connects. The same is true of a UDP listener: it reaches .ready whether or not anything can reach it.

The only proof is data arriving on the receiving side. Design smoke tests to assert on delivered bytes, never on connection state.

Related operator trap: a QUIC/PSK handshake failure is legible only on the sending terminal, while the results doc trains the operator to read the listener.

4. A QUIC listener hands you TWO connections per peer — and cancelling either resets both

This is the root cause of an "intermittent" POSIX 57 that went unexplained for the whole build

For one incoming QUIC connection an NWListener delivers two NWConnection objects:

streamIdentifierwhat it is
1the connection the listener was handed
0the peer’s own, real stream

The obvious-looking code accepts one and cancels the sibling. That is fatal: cancelling any one stream resets the entire QUIC connection. The peer sees POSIXErrorCode 57 (socket is not connected) — and it looks intermittent, because whether the sibling is cancelled before or after the peer’s stream arrives is a race.

Diagnostic signature, from 400 instrumented pairings:

  • the error is always raised in the receive path, never in the send path
  • connection.state == .ready at the exact moment Network.framework reports ENOTCONN
  • every immediate retry succeeded
  • healthy pairings log guard-rejected stream=1 then accept-ok stream=0; failing ones show the rejection and no accept-ok ever arrives

The tolerance is a hard cliff at ~250 ms, not a probability

Sweeping the delay between the listener receiving its connection and the peer’s stream settling: 0 ms → 7/8 pairings succeed; 250 ms and beyond → 0/8. It fails totally, not intermittently, past the edge. An apparent ~1.8% failure rate was that cliff being grazed by jitter.

Loopback baseline for the gap, 299 pairings: min=0 median=0 p90=0 p99=2 max=4 ms — roughly 60× inside the tolerance, which is why nobody noticed. Nobody has measured it on a duty-cycling radio.

The fix that works: a reference lifetime. Park the sibling keyed by its 4-tuple, release it when the peer’s stream settles, and reap orphans after handshakeTimeout + pairingTimeout (derive the reap window from your own timeouts, never from a loopback measurement). Verified over 1800 pairings, zero failures.

The tempting one-liner is worse than the bug

Simply returning nil for the sibling instead of cancelling it trades a 1.8% instant-failure rate for a 0.56% rate of thirty-second hangs, plus ~2× the throughput cost. A fast failure that retries beats a hang.

Full context and the margin sweep: ⭐ POSIX 57 is solved — the branch’s one unexplained defect.

5. Keepalive vs idle timeout — two correct fixes that combined into a hang

A quiet QUIC link dies after ~30 s. Measured: traffic every 10 s holds it indefinitely; 40 s of silence gives POSIXErrorCode 60: Operation timed out on both ends. The ordinary state of a camera remote between shutter presses is silence.

There is no keepaliveInterval on NWProtocolQUIC.Options (verified: does not compile). Use nw_quic_set_keepalive_interval at the C layer, or an application-level heartbeat. QUIC PING lives at the transport layer, so it does not compete with application sends.

Adding the keepalive exposed a latent deadlock

The pairing exchange had an uncancellable withCheckedContinuation inside a withTaskGroup. The QUIC idle timer had been accidentally rescuing it. Once PING frames kept the idle timer from ever firing, the rescue vanished and the exchange hung forever.

Measured against a mute peer with a 3 s handshake timeout: keepAlive nil + idle 4s → clean failure at 4.012 s; keepAlive 1s + idle 4s → still blocked at 120 s; production defaults keepAlive 10s + idle 30s → still blocked past 200 s.

Full write-up: swift-uncancellable-continuation-trap.

6. Streams are not self-describing

NWProtocolQUIC.Metadata exposes streamIdentifier but no streamType, though nw_quic_get_stream_type exists in C. The accepting side learns only a stream ID — so the wire protocol must open every stream with a channel-identifier preamble.

7. Datagrams are never fragmented

nw_quic_get_stream_usable_datagram_frame_size returns a uint16_t, and per RFC 9221 a DATAGRAM frame must fit in one QUIC packet. On the measured awdl0 MTU of 1500 the usable payload is ~1200–1350 bytes, so a 1080p HEVC frame is 10–100+ datagrams and losing one corrupts the frame. Read usable_datagram_frame_size at runtime and drive encoder slice size from it — never hardcode 1200.

Also: QUIC DATAGRAM frames are not flow-controlled. Nothing pushes back on a datagram sender the way the 32 MB stream window pushes back on a stream sender — so a parked, unconfirmed datagram flow has no transport-level bound at all, and any bound has to be yours. See hardware checklist item 4.

8. Local Network permission denial is silent and recoverable

Denial produces no error: NWBrowser simply returns nothing forever, indistinguishable from “no peer nearby”. It parks the browser in .waiting with -65555: NoAuth.

Model .waiting as a fourth state, not as .failed

.waiting is recoverable the instant permission is granted, so finishing the peers stream on it manufactures a false “transport failed” verdict. .failed is terminal and does finish the stream. A consumer awaiting only peers hangs forever under a permission denial.

Do not match the permission banner on the DNS-SD signature alone — POSIXErrorCode(1): Operation not permitted is a canonical case that a signature match misses, and EHOSTUNREACH on connect reads as a radio fault at range.

9. QUIC datagrams ARE reachable from Swift — but read the size off the datagram flow

Established by a kill-criterion spike (Phase 2 Task 0, reproduced independently by a reviewer with identical output bar ephemeral ports). Nothing contradicted it before; it was simply unverified, and the whole phase was gated on it.

usableDatagramFrameSize on the parent connection returns 0

Read it off the datagram flow’s own connection, not the parent. The parent returns 0, which — used as a clamp — silently disables everything.

An oversize datagram is accepted with NO error and silently dropped

Sending usable + 1 bytes returns no error at all and never arrives. There is no failure to observe and nothing to log unless you clamp and count it yourself. The application’s own clamp is the only thing standing between a slightly-too-large frame and a viewfinder that vanishes with nothing to debug from.

Verified by mutation: deleting the oversize guard makes its test fail; the raw oversize send returns success and the receiver never sees it.

The two ends disagree, so the clamp must use the local read

Measured in this codebase: the sender reads 9156 and the receiver advertises 9164 — an 8-byte gap on loopback. Each end must clamp with its own local read. Nothing in a fragmenter’s unit tests can catch a caller feeding the peer’s advertised size instead; it has to be checked where the send path is wired.

Corollary for anything built on top: fragment count is a parameter the rig chooses for you. Loopback’s 9156 B makes a 283,729 B frame 32 fragments; awdl0’s estimated 1200 B makes the same frame 240. NWConnection.send costs one continuation per fragment, so per-send cost scales with that count — measured 3.3–4× slower at 408 fragments than at 53. Write the datagram size into the caption beside any number you take.

10. NWConnectionGroup + NWMultiplexGroup on QUIC — hard, possibly not viable

Every extracted connection failed with POSIXErrorCode 50: Network is down

Across many variants. The group reported .ready while its path was down — the same “state proves nothing” trap as §3. Two independent agents failed at it, and the reviewer failed differently and earlier than the implementer (its group stuck in .setup and never reached .ready), which is what establishes that the first attempt did not simply give up early.

This records “hard, possibly not viable” — never “impossible”. Two independent failures cannot establish a negative, and committing a probe to prove one is scope no phase owes. The spec carries the hedge explicitly, and states that the shipped route was chosen because it was measured working, not because the group route was disproved.

What shipped instead: a second NWConnection to a second NWListener on a separate port. It is better than the design it replaced — preview is isolated from stills rather than sharing a congestion window with control — but the durable spec still records the architecture that was planned, not the one that shipped.

Setting BOTH newConnectionGroupHandler and newConnectionHandler on one QUIC NWListener fails with POSIXErrorCode 22 (EINVAL)

Either one alone is fine. This is the trap that makes an incremental migration to groups look broken for the wrong reason — you add the group handler beside the existing one and the listener dies at setup.

11. A datagram send into a dead peer SUCCEEDS over awdl0 and FAILS on loopback

The sender learns nothing, forever — and your harness cannot show you that

Over awdl0, a QUIC datagram send() into a peer that has gone away completes successfully. Over loopback the identical send fails immediately with POSIXErrorCode(rawValue: 57): Socket is not connected.

The two media disagree about whether an error exists, so a test exercises a path production never reaches. A test written for a send-into-nothing defect passes against unmodified code, because loopback hands the sender the very error the radio withholds.

Therefore: never make send failure your peer-death detector. Install a stateUpdateHandler on every NWConnection and tear down on .failed / .cancelled. Connection state is a local transport fact; a send result over the radio is not evidence of anything.

connection.stateUpdateHandler = { [weak self] state in
    switch state {
    case .failed, .cancelled: self?.shutDown()
    default: break
    }
}

Two details that are easy to get wrong:

  • [weak self] is load-bearing — the handler is retained by the connection, which the object retains.
  • Nil the handler inside shutDown() before calling cancel(), because cancel() re-enters the handler with .cancelled.

The shape to grep for: a correct pattern applied to only one of two sibling connections

In this codebase NetworkTransport.begin() had the right handler for the control connection from the beginning. The preview connection, added later beside it, was simply never given the equivalent — and because the phone only ever sends on that connection, it had no path to teardown at all. The Mac survived only incidentally, because it is the end that reads.

Whole story, log output and hand-run mutation evidence: aposemati-loopback-blind-spot.

Timing, for planning purposes: on loopback the peer’s cancel() propagated to the other end’s stateUpdateHandler in ~76 ms. Over awdl0 this is unmeasured — the QUIC CONNECTION_CLOSE may be lost, in which case the survivor falls back to the 30 s idle timeout.

12. Smaller notes worth keeping

  • Multipeer Connectivity: not deprecated in the 26.x SDKs (zero markers, compiles clean under -Wdeprecated-declarations), but Apple’s docs mark every class deprecatedAt: 27.0 and TN3213 states Xcode 27 deprecates the framework. Accurate phrasing is “not deprecated in 26.x; deprecated as of the 27.0 SDK”. Raw AWDL measures 758 Mbit/s peer-reviewed (MobiCom ‘18) against Multipeer’s 0.1–2 MB/s — same radio, ~50–90× gap. The bottleneck was the framework, not the link.
  • includePeerToPeer does use AWDL (confirmed by Apple DTS, June 2026, and by the EC’s DMA decision), but there is no preference knob — hard interface pinning from a browse result is the mechanism that works.
  • iOS 26’s BGContinuedProcessingTask does allow background CPU and network access, which corrects the earlier “iOS will not transfer in the background”. It is a real path for the transfer leg. Capture remains prohibited in the background (VideoDeviceNotAvailableInBackground).
  • usbmuxd is unconditionally denied to sandboxed apps — application.sb:751-752 denies network-outbound to the socket with no entitlement gate, and a files exception cannot fix it. Confirms the Developer-ID-not-App-Store verdict.

Graph View

Backlinks