Turn latency is the gap your caller feels: the milliseconds between the end of the caller’s speech and the start of the agent’s reply. When it goes past ~700 ms, the call stops feeling conversational. The caller talks over the agent, repeats the question, or hangs up. The good news: turn latency is almost always the sum of a small number of hops, and each hop has a specific fix. Work from the top down. First, measure where the time goes. Then attack the biggest hop. Do not guess. The dominant cost is rarely the one you assume.
This page is about the turn budget: the round-trip from silence to the first audio back. If the audio is delayed but steady (a fixed offset, or choppy or robotic playback), that is a media-plane problem. See One-way / choppy audio and MOS, jitter & loss.

​
Measure before you tune

Turn latency is a chain, and the system attributes every hop for you. Open the per-turn latency breakdown for a recent call. Read the hop-by-hop split before you change anything. A typical cascaded turn has these hops: A speech-to-speech (realtime) agent collapses ASR + LLM + TTS into one provider round-trip, so its breakdown is much shorter. Find your longest hop in the breakdown. Then go to the matching row below.

​
Symptom → cause → fix

1

Read the breakdown and find the dominant hop

Open the latency breakdown for a slow turn. The largest hop is your target. The rest is noise until you fix it.
2

Match it to a cause below

Use the table to map the dominant hop to a root cause and a concrete fix.
3

Change one thing, re-measure

Apply one fix. Place a fresh test call. Read the breakdown again. Turn latency fixes compound, so check each fix before you add the next.

​
Cascade vs speech-to-speech

This is the biggest lever. A cascaded agent (ASR → LLM → TTS) pays three serialized provider round-trips per turn. The transcript must finalize before the LLM starts. The LLM must emit text before TTS can synthesize. A speech-to-speech (realtime) agent streams audio straight into one model that emits audio back. This collapses all three hops into a single bidirectional stream. The difference is often a ~1.2 s turn versus a sub-500 ms turn.

Switch to speech-to-speech

When to use a realtime model, and how to wire one up. Options include OpenAI Realtime, Gemini Live, xAI Grok Voice, ElevenLabs Conversational AI, and a self-hosted realtime model.

Keep a cascade, make it fast

If you need the flexibility of separate ASR/LLM/TTS, the fixes below keep a cascade competitive.
Use a cascade when you need per-stage control: a specific ASR for an accent or language, a domain-tuned LLM, or a particular TTS voice. If you do not need that control, speech-to-speech is the faster default.

​
Provider round-trip

If one hop dominates and it is an individual provider, the latency comes from that provider’s own service, not from us. There are three fixes, in order of impact:
1

Use streaming clients, not batch

Streaming ASR emits partials and finalizes fast. Streaming TTS returns time-to-first-audio in a few frames instead of after the whole utterance. A non-streaming HTTP LLM is the worst offender. It returns nothing until it generates the full response, so the caller waits for the entire completion. Prefer a streaming or realtime model for any latency-sensitive agent.
2

Move the provider region closer

Every provider hop crosses the network twice. If your agent runs in one region and the provider’s endpoint is in another region, that geography is part of every turn. Point the provider at its region nearest to your media plane.
3

Pick a faster model

A smaller or faster-tier model at each stage directly cuts time-to-first-token and time-to-first-audio. Latency and quality trade off here. Tune per agent.
A non-streaming LLM also breaks clean barge-in. The runtime can silence playback when the caller interrupts, but the provider still completes the full response and bills for it. See Turn detection & barge-in for why.

​
Region and relay distance

Media rides QUIC / media-over-QUIC to the relay at relay.clutchcall.dev, then continues to your agent runtime and providers. Two distances matter, and they add up:
  • Caller → relay. Callers connect to the nearest edge. Browser and app legs auto-reconnect with capped backoff if a path degrades. This hop is usually small.
  • Runtime ↔ provider. This is the larger, more variable cost. If your agent runtime and your ASR/LLM/TTS providers sit in different regions, a cascade pays that crossing three times on every turn.
Check that you are on QUIC. QUIC / media-over-QUIC is the first-class transport. WebSocket and WebRTC are fallbacks for Safari, UDP-blocked networks, and TCP-only proxies. A fallback leg can roughly double turn latency and blow out the tail. In a WAN prototype at 50 ms link RTT with ~1% loss, turns ran p50 ~91 ms over QUIC vs ~193 ms over TCP+TLS, with a far cleaner tail. These are prototype-rig numbers, not a production SLA. If your tail latency is spiky, check that the leg did not fall back to TCP.
For colocated agents, keep the runtime and the providers it calls in the same region. The runtime ↔ provider hop then stays local.

​
Cold models

The first turn of a call can be slow while everything downstream warms up. The runtime already fights this for you:
  • Session prewarm. For realtime agents, the runtime opens and warms the provider session during the inbound INVITE / SDP exchange, before the caller completes the connection. The first turn does not pay connection setup.
  • Continuous prefill (self-hosted). A warm self-hosted realtime model with continuous prefill can return time-to-first-audio in roughly 0–15 ms after the session is live.
The failure mode is a cold self-hosted pod. If you run your own inference and the GPU pod scaled to zero, the first call after idle pays the full model cold-start. The result is a slow first turn, then fast turns after it. Keep a warm floor of capacity for latency-sensitive traffic.

Self-hosted inference

How the on-prem inference control plane routes to your model pods, and how to keep capacity warm.

​
End-of-turn detection

Sometimes the providers are fast and the agent still feels late. The cause: the runtime waits out silence before it decides the caller is done. That wait is silence_duration_ms (default 500). Every millisecond here adds to the front of every turn.
  • Lower it (~300 ms) for faster replies with terse callers.
  • Raise it (~800 ms) for callers who pause mid-thought, if the agent continues to cut them off.
This is a direct latency-vs-interruption trade, not a bug. Tune it against real calls.

Turn detection & barge-in

The full set of end-of-turn and barge-in timings, including silence_duration_ms, prefix_padding_ms, and hold-and-confirm barge-in.

​
Codec transcode

Phone legs arrive as G.711 (µ-law / A-law). The agent bus runs on narrowband PCM16. Browser and app legs use Opus. The runtime transcodes and resamples between these in-process. In practice this cost is sub-frame and not your latency problem. Time spent on it is almost never worth it. Codec choice does matter in one place: bridging. When you keep the same codec end to end — for example, when you pass Twilio Media Streams µ-law straight through to a PCMU trunk — you skip transcode on that path. See Codecs and Migrate from Twilio for the passthrough fast path.

Latency breakdown

The hop-by-hop attribution to read first — where every millisecond of a turn goes.

Choose a runtime

Cascade vs speech-to-speech vs bring-your-own, and the latency of each.

Barge-in not working

The other half of turn timing — when the agent does not yield to the caller.

MOS, jitter & loss

When the problem is audio quality rather than turn speed.