Turn Detection & Barge-in

Turn detection makes a call feel human. Your agent should stop the instant the caller starts to talk (barge-in). It should reply the instant the caller finishes (end-of-turn). It should not cut off a caller who only paused to think. It should not cancel itself when the caller mutters “mhm” to keep it going. To get this, pick a detection mode per agent and tune a small set of timings. The interrupt signal is wired end to end for you across every transport: PSTN, browser, and the WebRTC fallback leg. The same configuration behaves the same way on a phone call and in a browser tab.

Three ways a turn gets detected

Every agent uses exactly one of these. The right choice follows from the shape of your pipeline.

Mode	Who decides the turn	Use it when
On-device VAD (default)	The runtime, on the caller’s audio frames	Cascaded ASR → LLM → TTS agents (for example Deepgram → your LLM → ElevenLabs). Works on any transport.
`server_vad`	The realtime provider’s own endpointing	Realtime speech-to-speech models that already emit speech-start / speech-stop events (OpenAI Realtime, Gemini Live).
Client-commit	Your app / the runtime’s local detector	Self-hosted speech-to-speech models that expose no server-side VAD — turn boundaries are committed explicitly.

On-device VAD is the default because most agents are cascaded. They need a local trigger to know when to hand the utterance to the ASR stage. When an agent’s entry node is a realtime provider and you do not set a mode explicitly, the runtime auto-promotes the default to server_vad. The realtime session owns the audio path end to end, so a local detector would never actually run.

Two on-device backends exist: a lightweight built-in energy gate (type: "energy") and a higher-accuracy on-device VAD model. You get the VAD model by default when you omit type. Both feed the identical turn-taking and barge-in logic below. A backend switch never changes your timing knobs.

Configure turn detection per agent

Turn detection lives in your agent config under turn_detection. Every field is optional. Omit the block, and you get the phone-safe defaults.

{
  "entry_node": "ASR",
  "pipeline": { "ASR": { }, "LLM": { }, "TTS": { } },
  "turn_detection": {
    "prefix_padding_ms": 300,
    "silence_duration_ms": 500,
    "params": {
      "barge_confirm_ms": 300,
      "response_min_speech_ms": 600,
      "answer_min_speech_ms": 150
    }
  }
}

{
  "entry_node": "REALTIME",
  "pipeline": { "REALTIME": { } }
  // turn_detection omitted → auto-promoted to { "type": "server_vad" }
}

You can always override the auto-promotion. Set "type": "energy" on a realtime agent to force local detection instead. An explicit type wins over the default in every case. Set this on the agent from your workspace console at agent.clutchcall.dev, or push it through the control-plane API. See Runtime configuration for the full agent-config shape.

Tune when the agent replies

Start with the defaults, then use these fields. Lower values give faster replies. Higher values help callers who pause to gather their thoughts.

Field	Default	What it does	When to change
`silence_duration_ms`	500	Trailing silence before end-of-turn fires	Lower (~300) for faster replies; raise (~800) for deliberate callers
`prefix_padding_ms`	300	Pre-speech audio kept ahead of the speech-start marker	Raise if the ASR clips first syllables
`threshold`	0.5	Speech-probability gate for the on-device VAD model	Raise on noisy trunks to reject background chatter
`params.answer_min_speech_ms`	150	Shortest burst that counts as a real answer after the agent yields	Keep low so terse replies (“yes”, a digit) still commit
`params.response_min_speech_ms`	600	Shortest burst that counts as a real interruption over the agent	Raise to suppress more backchannels; lower to barge more eagerly
`params.silence_floor`	5	Accumulator level treated as “quiet again”	Rarely touched; lower on very quiet lines

The split between answer_min_speech_ms and response_min_speech_ms is deliberate. A short burst means opposite things at different moments. In the answering window, the agent stopped and waits. There, even a one-syllable “yes” is a real turn. Over the agent while it talks, that same short burst is almost always a continuer (“mhm”, “ok”, “right”). The over-agent floor is therefore higher, and backchannels never cancel the response.

Hold-and-confirm barge-in

Barge-in is armed automatically whenever the agent speaks. The mechanism is hold-and-confirm, controlled by params.barge_confirm_ms (default 300):

Onset arms a pending barge

The instant the caller starts to speak over the agent, the runtime arms a pending barge. It does not cancel the agent yet.

Sustained speech confirms it

The runtime fires the interrupt and cancels the in-flight response only if that speech continues past barge_confirm_ms.

Backchannels self-cancel

A short “mhm” or “ok” ends before the window elapses. The runtime drops the pending barge, and the agent continues to talk. The only cost of a real interruption is the ~barge_confirm_ms of extra latency before the cut.

Set barge_confirm_ms: 0 to disable the hold. The interrupt then fires on the very first frame of over-talk. Use 0 only on transports with client-side echo cancellation (the browser path). There, a fast cut is wanted, and the agent’s own audio cannot bleed back into the mic.

There is no acoustic echo cancellation in the telephony media path. On PSTN, the agent’s own TTS would otherwise leak back through the caller’s mic and self-trigger a barge. To prevent this, the runtime raises the during-speech mic gate (params.margin_while_tts) to match the quiet-line gate (params.margin) on phone calls. The agent’s voice can then never cancel the agent. On the browser path, where the client cancels echo, you can lower margin_while_tts for a faster barge.

Backchannel suppression here is acoustic only. It is based on how long the caller speaks, not on what they say. Lexical gating (to treat “stop” differently from “uh-huh”) and provider-side semantic endpointing are not built today. If you need word-aware interrupts, handle them in your own tool logic.

Where barge-in actually works

Barge-in quality depends on whether the runtime can cancel the downstream provider mid-flight. The on-device VAD always detects the interruption. The honest question is whether the agent’s audio actually stops cleanly. A “partial” row is a known limitation, not a misconfiguration.

Provider path	Mid-flight cancel	What happens on barge
OpenAI Realtime / Gemini Live (`server_vad`)	Clean	Runtime sends the provider’s interrupt/truncate; the session stops generating
Streaming TTS (ElevenLabs, Deepgram, and similar)	Clean	The runtime flushes the outbound audio buffer within a frame; playback stops immediately
Streaming ASR	Clean	Cancel propagates; the runtime discards the partial transcript
HTTP (non-streaming) LLM	Partial	The runtime silences playback locally, but the request runs to completion server-side, and the provider still bills it

And by transport leg, where the caller’s audio actually comes from:

Transport	Interrupt source	Status
SIP / RTP (PSTN trunk)	On-device VAD on the caller’s RTP → runtime cancel	End to end
Browser (WebTransport / media-over-QUIC)	Caller mic decoded → on-device VAD	Same path as PSTN
WebRTC fallback leg	Server-side WebRTC termination → on-device VAD	Same path as PSTN
External vendor bridge (LiveKit / Vapi / Twilio)	Depends on the vendor’s own interrupt event	Varies by adapter

When you route media through an external agent-orchestration vendor, not the built-in pipeline, the barge signal is only as good as that vendor’s adapter. For the tightest interrupt budgets, keep the media on the native SIP/RTP, browser, or WebRTC-fallback path.

Idle watchdog

The flip side of barge-in is silence. When both the caller and the agent go quiet, the idle watchdog can re-engage or end the call. Set it under turn_detection.idle:

soft_prompt_s — after this many idle seconds, the runtime nudges the model to speak (soft_prompt_text, or a bare re-engage on a realtime model).
hangup_s — after this many idle seconds, the runtime drops the AI leg.

Both default to 0 (off) in the raw schema. But the runtime ships sensible ceilings: a re-engage prompt, a hangup, a capped response length, and a hard maximum session duration. A stuck agent therefore cannot burn a call forever. See Session lifecycle for those guardrails.

Common gotchas

silence_duration_ms too low (< 300 ms). End-of-turn fires on the natural pauses between words, so the agent replies mid-sentence. Raise the value before you blame the ASR.

server_vad on a cascaded agent. Only a realtime provider emits server-side VAD events. A Deepgram → LLM → ElevenLabs agent set to server_vad will never see a turn boundary and will sit silent. Use on-device VAD (omit type) instead.

Delayed barge on an HTTP LLM. Non-streaming LLM calls run to completion server-side. The runtime silences playback the moment the caller barges. But the provider still finishes, and bills, the full response. For tight barge budgets, use a streaming or realtime model.

Runtime configuration

See the full agent-config shape, including where turn_detection lives.

Handoffs

Learn what a barge-in and end-of-turn feed into when you transfer to a human.

Bring your own speech-to-speech

Use the client-commit path for self-hosted realtime models with no server VAD.

Debug barge-in

Trace why an interrupt did or did not fire on a live call.

​Three ways a turn gets detected

​Configure turn detection per agent

​Tune when the agent replies

​Hold-and-confirm barge-in

​Where barge-in actually works

​Idle watchdog

​Common gotchas

​Related

Runtime configuration

Handoffs

Bring your own speech-to-speech

Debug barge-in

Three ways a turn gets detected

Configure turn detection per agent

Tune when the agent replies

Hold-and-confirm barge-in

Where barge-in actually works

Idle watchdog

Common gotchas

Related