OpenAI Realtime

​

What you get

• Duplex audio. ClutchCall Voice opens the Realtime WebSocket on the first inbound audio frame. It forwards each PCM chunk. It streams the model’s spoken reply back onto the call. The runtime does the sample-rate conversion to and from the model’s expected rate. Thus the same agent works on a G.711 phone leg and an Opus web leg without changes.
• Server-side VAD. OpenAI does its own endpointing. It emits speech-started / speech-stopped signals. A REALTIME agent uses these signals directly. You do not add a local turn detector.
• Instant barge-in via truncate. When the caller talks over the agent, the runtime truncates the in-flight assistant turn at the model. It also drops the buffered audio that is already on its way to the caller. Details are below.
• Function calling and a spoken greeting. Both are on by default.

​

Configure the agent

entry_node: REALTIME
pipeline:
  REALTIME:
    node_type: REALTIME
    provider: openai
    model: gpt-realtime        # GA model; sent via the dial URL, not session.update
    voice: alloy               # alloy | ash | ballad | coral | echo | sage | shimmer | verse
    instructions: |
      You are a friendly support agent for Acme. Keep replies to one or
      two sentences and never read out account numbers.

await client.agents.upsert({
  id: "support-realtime",
  entryNode: "REALTIME",
  pipeline: {
    REALTIME: {
      nodeType: "REALTIME",
      provider: "openai",
      model: "gpt-realtime",
      voice: "alloy",
      instructions:
        "You are a friendly support agent for Acme. Keep replies short.",
    },
  },
});

​

How the session is bootstrapped

• session.type is realtime.
• The runtime sets turn detection under audio.input.turn_detection as server_vad. You can tune threshold, prefix_padding_ms, and silence_duration_ms.
• The runtime sets the voice under audio.output.voice.
• instructions carries your prompt.
• The runtime caps max_output_tokens for each assistant turn. OpenAI defaults this value to “inf”. A voice turn is short. The cap stops a runaway response that generates minutes of speech and burns tokens.

​

Barge-in via truncate

1. The runtime sends a conversation.item.truncate for the assistant item that currently speaks. This tells OpenAI to stop generation of the rest of that turn. Without the truncate, the model continues to produce audio that you decided not to play. That wastes tokens.
2. The runtime emits a cancel downstream. This flushes the outbound audio buffer. Without the cancel, the caller would continue to hear the tail of the cancelled turn from the jitter buffer, even after the model stopped.

​

Greeting on inbound calls

​

Tools mid-conversation

1. The model emits a function call.
2. The runtime runs the tool.
3. The runtime returns the result as a function_call_output.
4. The runtime requests the next response, and the model speaks the answer.

​

Session budgets

• Connect timeout. The handshake gets a few seconds. After that, the runtime gives up and falls back to a silent passthrough.
• Per-turn token cap. The session bootstrap bounds it, as described above.
• Max session. The runtime closes sessions at a 15-minute ceiling, even if OpenAI did not close them. Tune lifetime, idle re-engage, and hangup budgets on Session Lifecycle.

​

Next steps