Inference — Recipes

These longer, end-to-end examples combine agent attach, turn-detection tuning, the audio bridge, and call status into realistic mini-apps. All of them use the Voice surface.

Recipe 1 — Outbound speech-to-speech agent on a PSTN call

Dial a number. Bridge a speech-to-speech model at the moment the callee answers. Let ClutchCall own turn-taking for the whole call. This is the canonical inference app. There is no ASR/LLM/TTS pipeline to wire: one model, one connection.

Create the client

One tenant token authorizes the call control and the agent leg.

import { Voice } from "@clutchcall/sdk/voice";

const voice = new Voice({ baseUrl: BASE_URL, apiKey: KEY, orgId: ORG });

Originate with the agent inline

Pass agent to originate. This bridges the model when the callee answers, with no second round-trip.

const call = await voice.calls.originate({
  to:      "+15551234567",
  from:    "+15558675309",
  trunkId: "trunk_main",
  agent: {
    mode:          "speech_to_speech",
    modelCodec:    "pcm16",       // model ingests/emits raw PCM16
    inputRateHz:   16000,
    outputRateHz:  24000,
    turnDetection: {
      responseMinSpeechMs: 600,   // drop backchannels
      silenceThresholdMs:  500,   // end-of-turn after 500ms silence
      bargeConfirmMs:      300,   // hold-and-confirm barge-in
    },
  },
});

Follow the call to completion

The model drives the conversation. You only watch status.

call.onStatus((s) => {
  console.log("status:", s);
  if (s === "completed" || s === "failed" || s === "no_answer") {
    console.log("call ended:", s);
  }
});

That is the whole app. The PSTN leg is µ-law. ClutchCall transcodes to PCM16 for the model and back. It resamples 8 kHz ↔ 16/24 kHz at the bridge. The caller hears the agent’s first words within tens of milliseconds after the caller’s own sentence ends, because the model heard the utterance as it streamed.

Recipe 2 — A barge-in profile tuned per transport

Barge-in behavior that feels good on a browser leg (fast, cut-on-first-frame) triggers itself on a raw PSTN leg. On that leg, the agent reads its own echo as caller speech. Select the turn-detection profile from the leg type at attach time.

import { Voice } from "@clutchcall/sdk/voice";

const voice = new Voice({ baseUrl: BASE_URL, apiKey: KEY, orgId: ORG });

// Two profiles: one for echo-cancelled legs, one for raw audio.
const profiles = {
  browser: {            // client-side AEC: cut instantly, low guard
    bargeConfirmMs: 0,
    ttsGuardMs:     150,
    responseMinSpeechMs: 500,
  },
  pstn: {               // no AEC: confirm before cutting, high guard
    bargeConfirmMs: 350,
    ttsGuardMs:     400,
    responseMinSpeechMs: 700,
  },
} as const;

async function attachAgent(callSid: string, leg: "browser" | "pstn") {
  await voice.agents.attach(callSid, {
    mode:          "speech_to_speech",
    modelCodec:    leg === "browser" ? "opus" : "pcm16",
    turnDetection: profiles[leg],
  });
}

Two settings change with the transport. bargeConfirmMs sets how long speech must continue before the agent is cancelled. ttsGuardMs sets how long the mic gate stays raised after the agent sends audio. On a no-AEC leg, raise both so that the agent’s own voice cannot trigger barge-in. On an AEC leg, lower both for a fast interrupt.

Use it from your call setup:

const browserCall = await voice.calls.originate({ to: "+1...", agent: undefined });
await attachAgent(browserCall.sid, "browser");

const pstnCall = await voice.calls.originate({ to: "+1...", trunkId: "trunk_main" });
await attachAgent(pstnCall.sid, "pstn");

Recipe 3 — Measure turn latency on a live call

Turn latency is the time from end-of-user-speech to the first agent audio-out frame. It is the number that defines “responsiveness.” Tap the audio bridge beside the agent. Mark the last inbound frame. Measure the time to the first outbound frame.

Attach the agent and a passive bridge

The agent drives the conversation. The bridge is a read-only tap on both legs.

const call = await voice.calls.originate({ to: "+15551234567", trunkId: "trunk_main" });

await voice.agents.attach(call.sid, {
  mode: "speech_to_speech",
  turnDetection: { silenceThresholdMs: 500, responseMinSpeechMs: 600 },
});

const bridge = await voice.audioBridge.attach(call.sid, {
  codec:    "pcm16",
  onUplink: (frame, tsUs) => onUplink(frame, tsUs),
});

Detect end-of-speech locally

Track when caller frames stop for longer than the silence threshold. That approximates the turn detector’s commit point.

let lastVoiceUs = 0n;
let turnOpen    = false;
const SILENCE_US = 500_000n;   // mirror silenceThresholdMs

function onUplink(frame: Uint8Array, tsUs?: bigint) {
  const ts = tsUs ?? BigInt(Date.now()) * 1000n;
  if (rms(frame) > VOICE_FLOOR) { lastVoiceUs = ts; turnOpen = true; }
}

Time to the first agent frame

The first downlink frame after a closed turn marks start-of-agent-speech.

const samples: number[] = [];
bridge.onDownlink((frame, tsUs) => {
  const now = tsUs ?? BigInt(Date.now()) * 1000n;
  if (turnOpen && now - lastVoiceUs > SILENCE_US && rms(frame) > VOICE_FLOOR) {
    const turnLatencyMs = Number(now - (lastVoiceUs + SILENCE_US)) / 1000;
    samples.push(turnLatencyMs);
    turnOpen = false;
  }
});

Report the tail, not the mean

p95 / p99 is the product story. A good mean with a bad tail still feels broken.

function percentile(xs: number[], p: number) {
  const s = [...xs].sort((a, b) => a - b);
  return s[Math.floor((s.length - 1) * p)];
}
call.onStatus((st) => {
  if (st === "completed") {
    console.log("turn latency p50 ms:", percentile(samples, 0.50));
    console.log("turn latency p95 ms:", percentile(samples, 0.95));
    console.log("turn latency p99 ms:", percentile(samples, 0.99));
  }
});

Over a WAN-emulated link (50 ms RTT, ~1% loss), the QUIC substrate keeps steady-state first-audio latency at approximately half that of TCP+TLS, with a much cleaner p99. A dropped packet on one stream does not stall the audio on another. If your p99 grows under load, look at the caller leg’s loss first, not the model.

Recipe 4 — Hand off from the model to a human

Run the speech-to-speech agent first. Then transfer to a live agent when the caller asks. The model leg detaches. The call continues on the new leg.

import { Voice } from "@clutchcall/sdk/voice";

const voice = new Voice({ baseUrl: BASE_URL, apiKey: KEY, orgId: ORG });

const call = await voice.calls.originate({
  to: "+15551234567", trunkId: "trunk_main", agent: "support-s2s",
});

// Your own logic decides when a human is needed (a tool call, a keyword, a
// supervisor request). When it does, transfer the live audio to a human leg.
async function escalate() {
  await call.transfer({ to: "+15558675309", trunkId: "trunk_main" });
  // The model is no longer driving; the new leg owns the audio.
}

transfer performs a SIP REFER (or an HTTP fallback) to hand off the live audio. The original sid stays in history. The new leg gets its own sid. The speech-to-speech agent stops driving the conversation at the moment the transfer completes.

Recipe 5 — Speech-to-speech agent with a tool call

Give the agent a tool, and the conversation gains actions. In this example, the model handles a support call. When the caller asks about an order, the model calls an HTTP lookup_order tool. It gets the status back and speaks the answer, all inside one natural conversation. ClutchCall runs the HTTP call and feeds the result back to the model. You write zero glue.

Create the client

One tenant token authorizes call control, the agent leg, and the tool call.

import { Voice } from "@clutchcall/sdk/voice";

const voice = new Voice({ baseUrl: BASE_URL, apiKey: KEY, orgId: ORG });

Define the agent with an HTTP tool

Declare the model leg, the turn-detection policy, and a lookup_order tool. The tool’s description and parameters JSON-schema are what the model sees. The url substitutes {{order_id}} from the arguments that the model fills in.

const agent = {
  mode:          "speech_to_speech",
  modelCodec:    "pcm16",
  inputRateHz:   16000,
  outputRateHz:  24000,
  turnDetection: { responseMinSpeechMs: 600, bargeConfirmMs: 300 },
  tools: [
    {
      kind: "http",
      name: "lookup_order",
      description: "Look up an order by its ID and return its status and ETA.",
      silent: false,   // model may say "let me check that for you"
      spec: {
        method: "GET",
        url: "https://api.example.com/orders/{{order_id}}",
        headers: { "X-Tenant": "{{tenant_id}}" },
        auth: { type: "bearer", token: process.env.ORDERS_TOKEN! },
        parameters: {
          type: "object",
          properties: {
            order_id:  { type: "string", description: "Order id to look up." },
            tenant_id: { type: "string", description: "Caller's tenant." },
          },
          required: ["order_id"],
        },
        timeoutMs: 2000,   // tight — the invoke runs inside the model turn
      },
    },
  ],
};

Originate the call with the agent inline

Pass agent to originate. This bridges the model, and its tool, at the moment the callee answers.

const call = await voice.calls.originate({
  to:      "+15551234567",
  from:    "+15558675309",
  trunkId: "trunk_main",
  agent,
});

The model calls the tool mid-conversation

There is nothing else to wire. When the caller says “where’s my order A-123?”, the model decides to call lookup_order. ClutchCall sends the HTTP request. The structured result comes back as a tool_result, and the model speaks it: “Order A-123 shipped this morning, it’ll arrive Thursday.” You only watch status.

call.onStatus((s) => {
  console.log("status:", s);
  if (s === "completed" || s === "failed" || s === "no_answer") {
    console.log("call ended:", s);
  }
});

The tool invoke is synchronous within the model turn. While the HTTP call is in flight, the model cannot speak. Keep the endpoint fast, so that the caller never hears dead air. The timeoutMs: 2000 above is a ceiling, not a target.

On a failed lookup, return a non-2xx status with a structured body (for example, { "error": "order_not_found" }). The model receives that as the tool result. It recovers instead of stalling: “I couldn’t find that order, can you re-read the number?” See the cookbook for the error pattern. See Agent DAGs for the full operator tool spec.

Inference — Details — the commit gate, codecs, and turn-latency metric
Inference — Cookbook — short single-task snippets
Voice — Recipes — call control patterns this builds on

​Recipe 1 — Outbound speech-to-speech agent on a PSTN call

​Recipe 2 — A barge-in profile tuned per transport

​Recipe 3 — Measure turn latency on a live call

​Recipe 4 — Hand off from the model to a human

​Recipe 5 — Speech-to-speech agent with a tool call

​Related

Recipe 1 — Outbound speech-to-speech agent on a PSTN call

Recipe 2 — A barge-in profile tuned per transport

Recipe 3 — Measure turn latency on a live call

Recipe 4 — Hand off from the model to a human

Recipe 5 — Speech-to-speech agent with a tool call

Related