Voice Agent Topics

For Agents

Living index of themes for this project. Each H2 is a topic; bullets are wikilinks to related notes. Updated by obsidian-documenter when documenting work. Read by historian at bootstrap. Topics kept alphabetical.

Architecture Decisions

• voice-agent — the three decisions that fell out of the 2026-07-21 research round: cascaded (not S2S) · Rust (not Pipecat/LiveKit) · Twilio (despite being most expensive)
• voice-agent-speech-to-speech-disqualification-2026-07-21 — S2S rejected; three documented failure modes, plus the Sierra counter-evidence (they rebuilt a cascade’s control plane around the S2S core, including replacing its built-in turn detection)
• voice-agent-rust-ecosystem-status-2026-07-21 — no framework has a Rust agent runtime, so the runtime is greenfield; flowcat = watch, don’t bet

Benchmarks & Their Limits

• voice-agent-stt-vendor-landscape-2026-07-21 — Artificial Analysis’ streaming STT index contains no narrowband telephony audio at all (AA-AgentTalk / VoxPopuli / Earnings22, all wideband); nobody publishes per-domain telephony WER
• voice-agent-tts-vendor-landscape-2026-07-21 — Speech Arena top-4 spans 24 ELO with ±16 CIs = statistically noise; Coval numbers reach us via Gradium’s blog and Gradium tops that benchmark
• voice-agent-telephony-carriers-2026-07-21 — carrier RTT table is published on Telnyx’s own site and Telnyx wins it
• voice-agent-rust-ecosystem-status-2026-07-21 — flowcat’s “p99 0.61ms” measures frame-routing overhead, not voice-to-voice latency

Cost & Build-vs-Buy

• voice-agent-cost-model-2026-07-21 — $0.0326/min,$0.098 per 3-min call; the LLM is ~10% of cost — optimize TTS, not the model; break-even vs Vapi ≈ 600k minutes ≈ 200k calls
• voice-agent-managed-platform-control-surfaces-2026-07-21 — the managed premium (+161% Vapi, +207–299% ElevenLabs Agents) buys a smaller control surface
• voice-agent-telephony-carriers-2026-07-21 — carrier price spread is $6vs$14/month at SMB volume = noise; Telnyx’s headline $0.0032 is not what you pay
• voice-agent-speech-to-speech-disqualification-2026-07-21 — Gemini Live ~$0.023/min reaches cascade parity, so cost no longer disqualifies S2S generally — only OpenAI’s, which scales with conversation length

Latency & Turn-Taking

• voice-agent-latency-budget-2026-07-21 — the headline: ~480ms of a measured 1,048ms loop is unaccounted orchestration overhead; perception ladder; every platform triples p50→p95; TTS connection warmth ≈ 250ms
• voice-agent-stt-vendor-landscape-2026-07-21 — Flux fused EOT ~260ms p50 (p95 1.5s); EagerEndOfTurn speculative dispatch hides LLM TTFT; AssemblyAI’s 1280ms max_turn_silence
• voice-agent-managed-platform-control-surfaces-2026-07-21 — onNumberSeconds exists only on Vapi and Telnyx; Vapi’s waitFunction; Pipecat’s fixed 1s aggregation_timeout; LiveKit interruption bugs 3685
• voice-agent-telephony-carriers-2026-07-21 — co-locating in us-east-1 beats an 18ms carrier RTT delta

Rust Implementation

• voice-agent-rust-ecosystem-status-2026-07-21 — ecosystem status, flowcat assessment, and the concrete list of what the runtime must own
• voice-agent-stt-vendor-landscape-2026-07-21 — Deepgram’s crate v0.10.0 is the only first-party Rust SDK in the field (verify it covers /v2/listen)
• voice-agent-latency-budget-2026-07-21 — the 480ms residual is the design thesis every implementation choice is judged against

Telephony & Audio Format

• voice-agent-telephony-carriers-2026-07-21 — Twilio Media Streams: μ-law 8k only, ONE bidi stream per call, DTMF inbound-only; ConversationRelay is text-only → AVOID
• voice-agent-stt-vendor-landscape-2026-07-21 — Flux takes native encoding=mulaw&sample_rate=8000 with zero transcoding; Nova-2 phonecall is the only narrowband-special-cased control model
• voice-agent-tts-vendor-landscape-2026-07-21 — μ-law support is a hard vendor filter; Rime’s μ-law 8k is the open blocker
• voice-agent-speech-to-speech-disqualification-2026-07-21 — Gemini Live is PCM16 16k/24k only (no μ-law); 8kHz is out-of-distribution for realtime models

Vendor Risk & Compliance

• voice-agent-telephony-carriers-2026-07-21 — account-suspension risk is the real carrier differentiator when carrying other businesses’ numbers: Telnyx HIGH (~$4.5MproposedFCCfine;suspensionover$0.06) · Plivo MEDIUM ticket-only · Twilio LOWEST
• voice-agent-managed-platform-control-surfaces-2026-07-21 — Vapi’s ~19+ hours of call-affecting degradation in 10 weeks + weekly release channel
• voice-agent-tts-vendor-landscape-2026-07-21 — PlayHT/Play.ai is dead (Meta acqui-hire, API shut 2025-12-31) — its presence in any comparison is a staleness signal

Vendor Selection — STT

• voice-agent-stt-vendor-landscape-2026-07-21 — Deepgram Flux (flux-general-en, wss://api.deepgram.com/v2/listen — note /v2); AssemblyAI rejected on 1280ms turn-commit + session-duration billing

Vendor Selection — TTS

• voice-agent-tts-vendor-landscape-2026-07-21 — concurrency is the binding constraint, not price or quality: ElevenLabs 15 concurrent at $990/mo(sameas$299) and Cartesia 15 → both disqualified; Deepgram 45/60 self-serve wins. IQR > P50. ElevenLabs v3 is batch-only so its expressive tags are unreachable; Rime is the only spontaneous-speech vendor but μ-law is unverified