Turn Detection & Barge-In

Tune when your voice agent listens, talks, and yields the floor. Turn detection makes a call feel natural. The agent stops the instant the caller starts to talk (barge-in). The agent replies the instant the caller finishes (end-of-utterance). If the tuning is wrong, the call feels broken: the agent talks over people, or it stays silent for awkward seconds while it waits for “real” silence. You control this with two settings: a detection mode per agent, and a small set of tuning parameters. Pick a mode. Set the timings. The interrupt signal is then wired end-to-end for you.

The two detection modes

┌─────────────────────────────────────────────────────────────────┐
│  TurnDetection {                                                │
│    type: "silero" | "server_vad",                               │
│    silence_threshold_ms?, min_speech_ms?, prefix_padding_ms?    │
│  }                                                              │
└─────────────────────────────────────────────────────────────────┘

`type`	Who runs VAD	When to use
`silero`	Gateway, locally (on-device VAD)	Cascaded ASR → LLM → TTS. Default.
`server_vad`	The AI provider (e.g. OpenAI Realtime)	Realtime models that already emit `speech_started`/`speech_stopped`.

silero is the default because most pipelines are cascaded (for example, Deepgram → Anthropic → ElevenLabs) and need a local trigger. The runtime auto-promotes to server_vad only when the agent’s entry node is REALTIME AND the operator did not supply an explicit type.

Configure turn detection per agent

Put a TurnDetection block in your agent config:

# agent-config.yml
entry_node: ASR
turn_detection:
  type: silero
  silence_threshold_ms: 500    # min silence before EOU fires
  min_speech_ms: 300           # filter out clicks / noise
  prefix_padding_ms: 200       # include this much pre-speech audio
nodes:
  ASR: { provider: deepgram, ... }
  LLM: { provider: anthropic, ... }
  TTS: { provider: elevenlabs, ... }

For a realtime entry node, omit the whole block. The runtime then defaults to server_vad:

entry_node: REALTIME
nodes:
  REALTIME: { provider: openai-realtime, model: gpt-4o-realtime-preview }
# turn_detection auto-promotes to {type: server_vad} based on entry_node

The runtime always honors an override. If you want gateway-side detection with a realtime entry node, set type: silero explicitly.

Tune when the agent replies

These are the timings that you will change most often. Start with the defaults. Lower them for faster replies. Raise them for callers who pause to think.

Field	Default	What it does	When to change
`silence_threshold_ms`	500	Min trailing silence before EOU fires	Lower (300) for snappier replies; higher (800) for thinkers
`min_speech_ms`	300	Discard candidate utterances shorter than this	Raise to 500 if line noise causes false triggers
`prefix_padding_ms`	200	Audio kept before the speech-start marker	Raise if first syllables are getting clipped on the ASR
`silence_floor_dbfs`	-45	Signal level below which is treated as silence	Quieter trunks need lower (-50); noisy ones higher (-40)

Know where barge-in actually works

This is an honest scorecard. Where it says “no”, the agent will continue to talk over the user. Treat this as a known limitation, not a configuration error.

Agent LLM/ASR/TTS providers

Provider	Cancel mid-flight?	Barge trigger
OpenAI Realtime	✅	Local VAD or `speech_started` → `truncate`
OpenAI HTTP / Anthropic / Gemini / Ollama	❌ (timeout-only)	Gateway silences playback; LLM completes server-side
Deepgram ASR streaming	✅ (forward cancel)	Cancel propagates; partial transcript discarded
Deepgram TTS / ElevenLabs TTS	✅ (clear buffer)	Outbound PCM buffer cleared at PushAudio node

WebRTC / streaming vendor adapters

Vendor	Path	Interrupt signal	Status
SIP/RTP	Native PSTN trunk	on-device VAD → bridge control	✅ End-to-end
Twilio	Media Streams	`clear` event	⚠️ Adapter doesn’t yet emit it on barge
Vapi	WebSocket transport	`stop` message	⚠️ Adapter doesn’t yet emit it on barge
LiveKit	Room participant	None (no clean media-plane interrupt)	❌ Client-side action required
Daily	Room participant	None	❌ Client-side action required
Chime	Meeting attendee	None	❌ Client-side action required
Browser	WebTransport	Caller mic → server VAD	✅ Same as SIP path

For ❌ vendors, the AI’s TTS continues to stream. ClutchCall silences the playback locally, but the AI is unaware. For latency-sensitive UX, prefer SIP/RTP or Browser. After the adapter work lands, Twilio/Vapi will also be good options.

Common gotchas

Realtime entry node + explicit type: silero: this works, but you do VAD twice. The OpenAI server-side VAD still fires. It can produce speech_started events that the runtime ignores. Pick one mode.
silence_threshold_ms too low (< 300): end-of-utterance fires on inter-word pauses. The agent interrupts the user mid-sentence.
Cascaded pipeline with server_vad: only the realtime model can produce server-VAD events. A Deepgram → Anthropic → ElevenLabs pipeline with server_vad will never emit a turn boundary. This is a misconfiguration. Use silero instead.
HTTP LLM “barge” feels delayed: Anthropic/Gemini/OpenAI HTTP run to completion server-side. ClutchCall silences the audio playback, but the server still bills you for the full response. For tight barge-in budgets, use a Realtime model.

How it works

Barge-in feels instant because the interrupt travels one control path end to end. You configure the mode. The plumbing is already wired. When local VAD trips, this chain fires:

caller starts speaking
       │
       ▼
VAD source: on-device VAD on the gateway (SIP/RTP path)
            OR vendor server-VAD via the agent runtime (direct-media path)
       │
       │  control frame: [0xFFFFFFFF sentinel][type=BARGE_IN][session_id]
       ▼
agent_runtime_bridge → runtime
       │
       ▼
DagExecutor::on_cancel(CancelEvent{source: user_vad})
       │
       ├──▶ ASR / LLM / TTS nodes: forward cancel
       ├──▶ PushAudio node: clear outbound buffer
       └──▶ RealtimeNode: emit `conversation.item.truncate` to OpenAI,
                          close session

The same chain fires on source: server_vad when the AI provider reports that the user spoke first (e.g. OpenAI Realtime’s input_audio_buffer.speech_started). Both sources land in the same handler. Node code is thus detection-mode-agnostic.

Where to dig further

get_page rpc/audio-frames — wire format for the PCM/PCMU stream that feeds the gateway VAD.
get_page rpc/method-ids — Barge method ID for operator-initiated interrupts.
get_page admin/agent-dags — full TurnDetection schema in the agent config.

Getting Started

Concepts

Raw RPC Format

Telephony Reference

Platform

Admin API (raw RPC, no SDK)

Turn Detection & Barge-In

The two detection modes

Configure turn detection per agent

Tune when the agent replies

Know where barge-in actually works

Agent LLM/ASR/TTS providers

WebRTC / streaming vendor adapters

Common gotchas

How it works

Where to dig further

​The two detection modes

​Configure turn detection per agent

​Tune when the agent replies

​Know where barge-in actually works

​Agent LLM/ASR/TTS providers

​WebRTC / streaming vendor adapters

​Common gotchas

​How it works

​Where to dig further

The two detection modes

Configure turn detection per agent

Tune when the agent replies

Know where barge-in actually works

Agent LLM/ASR/TTS providers

WebRTC / streaming vendor adapters

Common gotchas

How it works

Where to dig further