Inference — SDK Methods

The inference modality has no standalone typed client. It uses the Voice surface. You attach a speech-to-speech agent to a call with voice.agents.attach. You configure turn detection and the model codec. Optionally, you tap the audio bridge. This page is the inference-relevant slice of the Voice SDK plus the turn-detection knobs.

A dedicated typed Inference client is Preview. See the bottom of this page. For shipping code today, use the Voice surface documented here.

Import

The agent path lives on the voice subpath:

TypeScript
Python

import { Voice } from "@clutchcall/sdk/voice";

const voice = new Voice({ baseUrl: BASE_URL, apiKey: KEY, orgId: ORG });

from clutchcall.voice import Voice

voice = Voice(base_url=BASE_URL, api_key=KEY, org_id=ORG)

Go, Rust, Java, and .NET bindings expose the same shapes (snake_case in Python, PascalCase methods in Go/.NET).

`voice.agents.attach`

Bind a speech-to-speech agent to a live call or a just-originated call. The runtime bridges the call’s uplink / downlink audio tracks to the model. It owns turn-taking for the duration of the call.

TypeScript
Python

await voice.agents.attach(callSid, agent);

voice.agents.attach(call_sid, agent)

string

required

The call sid to bind the agent to, from originate() or an inbound answer.

string | AgentSpec

required

Either an agent id (string) registered in the control plane, or an inline agent spec. The spec selects the speech-to-speech model and its turn-detection config.

Returns Promise<void>. It resolves after the agent is bridged. Audio starts to flow immediately. Turn detection gates the first agent utterance.

Attaching at originate time

originate accepts an agent directly, so the common case is one call:

TypeScript
Python

const call = await voice.calls.originate({
  to:      "+15551234567",
  from:    "+15558675309",
  trunkId: "trunk_main",
  agent:   "support-s2s",     // attach the speech-to-speech agent inline
});

call = voice.calls.originate(
    to="+15551234567",
    from_="+15558675309",
    trunk_id="trunk_main",
    agent="support-s2s",
)

The agent spec

An inline AgentSpec selects the model leg and the turn-detection policy. The shape mirrors the agent config that the control plane stores.

type AgentSpec = {
  // The model leg the runtime serves on the call's audio — a single
  // speech-to-speech model configured in your registry.
  mode: "speech_to_speech";
  modelCodec?: AudioCodec;        // what the model ingests/emits (default "pcm16")
  inputRateHz?: number;           // model input rate (default 16000)
  outputRateHz?: number;          // model output rate (default 24000)
  turnDetection?: TurnDetection;  // turn-taking + barge-in policy (below)
  tools?: ToolSpec[];             // function-calling tools the model can invoke (below)
};

type AudioCodec = "opus" | "pcm16" | "g711_ulaw" | "g711_alaw";

ClutchCall resamples between the caller leg rate (commonly 8 kHz on PSTN) and the model’s inputRateHz / outputRateHz. You set the model’s rates. The bridge handles the caller leg.

Turn detection

The TurnDetection block is the heart of this modality. It owns the client-commit gate, backchannel suppression, and hold-and-confirm barge-in. Put it in the agent spec, or set it server-side in the agent config.

type TurnDetection = {
  responseMinSpeechMs?: number;   // bursts shorter than this = backchannel, dropped
  commitMinIntervalMs?: number;   // cooldown between turn commits
  cancelMinIntervalMs?: number;   // cooldown between barge-ins
  bargeConfirmMs?: number;        // sustain speech this long before cancelling the agent
  ttsGuardMs?: number;            // raise the mic gate this long after agent audio
  silenceThresholdMs?: number;    // trailing silence before end-of-turn fires
  minSpeechMs?: number;           // discard candidate utterances shorter than this
  prefixPaddingMs?: number;       // audio kept before the speech-start marker
};

Field	Default	What it does	When to change
`responseMinSpeechMs`	600	Speech shorter than this is a backchannel. It never commits a turn	Lower (400) to let very short utterances answer; raise to ignore more continuers
`commitMinIntervalMs`	1500	Minimum gap between two turn commits	Raise if the agent over-answers rapid speech
`cancelMinIntervalMs`	400	Minimum gap between barge-ins	Raise on noisy trunks that false-trigger
`bargeConfirmMs`	300	Speech must continue this long over the agent before the agent is cancelled	`0` = cancel on first frame (browser/AEC legs); raise on noisy legs
`ttsGuardMs`	200	Mic gate is raised for this long after each agent chunk	Raise on no-AEC SIP/PSTN legs where TTS bleeds into the mic
`silenceThresholdMs`	500	Trailing silence before end-of-turn fires	Lower (300) for faster replies; higher (800) for callers who pause to think
`minSpeechMs`	300	Discard candidate utterances shorter than this	Raise to 500 if line noise causes false triggers
`prefixPaddingMs`	200	Audio kept before the speech-start marker	Raise if first syllables get clipped

silenceThresholdMs below ~300 fires end-of-turn on inter-word pauses. The agent will interrupt the caller mid-sentence. Keep it at or above 300.

Tools

Declare function-calling tools on the agent. The model can then invoke them mid-conversation. ClutchCall runs the call and feeds the structured result back to the model. Put a tools array in the agent spec. The shape mirrors the operator tool config that the control plane stores. See Agent DAGs for the full operator spec.

type ToolSpec = {
  kind: "http" | "mcp" | "client";  // http + mcp shipped; client is scaffolded
  name: string;                     // tool name the model sees (unique per agent)
  description: string;              // what the tool does — the model reads this
  silent?: boolean;                 // false = model may narrate the call (default false)
  spec: ToolHttpSpec;               // the call to make (http/mcp); see below
};

type ToolHttpSpec = {
  method: string;                   // "GET" | "POST" | ...
  url: string;                      // supports {{argument}} substitution
  headers?: Record<string, string>; // values support {{argument}} substitution
  auth?: ToolAuth;                  // bearer / basic / custom header
  parameters: JsonSchema;           // JSON-schema for the tool arguments
  timeoutMs?: number;               // per-call timeout (default 10000)
};

type ToolAuth =
  | { type: "bearer"; token: string }
  | { type: "basic"; user: string; pass: string }
  | { type: "header"; name: string; value: string };

"http" | "mcp" | "client"

required

http calls an arbitrary HTTP(S) endpoint. mcp sends a JSON-RPC 2.0 tools/call to a remote MCP server. client is scaffolded. The tool is advertised and the call is parsed, but invoke returns not_yet_plumbed today.

string

required

The tool name that the model sees. It must be unique per agent. On a name collision, it overrides a built-in telephony tool of the same name.

string

required

Plain-language description of what the tool does and when to use it. This is what the model reasons over when it decides to call the tool.

boolean

default:"false"

When false, the model can narrate the call (“let me look that up”). When true, it stays quiet until it has the result.

string

required

HTTP method, for example GET or POST.

string

required

Endpoint URL. Supports {{argument}} substitution from the model’s tool arguments, for example https://api.example.com/orders/{{order_id}}.

Record<string, string>

Request headers. Values support the same {{argument}} substitution as the URL.

ToolAuth

{ type: "bearer", token }, { type: "basic", user, pass }, or a custom header { type: "header", name, value }.

JsonSchema

required

A JSON-schema object (type / properties / required) that describes the tool’s arguments. The model fills these in. They drive the {{argument}} substitution.

number

default:"10000"

Per-call timeout in milliseconds. Invokes run synchronously within the model turn, so keep this tight.

Tool invokes are synchronous within the model turn. A slow upstream eats the turn budget, and the caller hears dead air. Keep individual tools fast. Or split the work across multiple turns instead of cascaded tool calls inside one turn.

Every agent also advertises a set of built-in telephony tools (transfer, hold, DTMF, and more) with no configuration. Tools that you declare here are operator tools. They override a built-in tool on a name collision.

Tapping the audio bridge (optional)

You usually do not touch raw frames. The runtime bridges the model leg for you. When you need to observe or inject audio (a recording sidecar, custom DSP), attach the Voice AudioBridge to the same call.

TypeScript
Python

const bridge = await voice.audioBridge.attach(call.sid, {
  codec: "pcm16",
  onUplink: (frame, tsUs) => recorder.feed(frame),   // caller audio (also fed to the model)
});

bridge = voice.audio_bridge.attach(
    call.sid,
    codec="pcm16",
    on_uplink=lambda frame, ts_us: recorder.feed(frame),
)

AudioBridge methods relevant here:

Method	Signature	Notes
`publishDownlink`	`(frame: Uint8Array, timestampUs?: bigint)`	Inject audio toward the caller (for example, a pre-roll prompt before the model speaks).
`publishUplink`	`(frame: Uint8Array, timestampUs?: bigint)`	Inject audio toward the model (browser caller path).
`close`	`()`	Detach the tap. The agent stays attached.

See Voice — SDK Methods for the full AudioBridge surface.

Events

The agent leg shows turn-boundary and barge events through the call’s status stream. Subscribe with the Voice call handle:

TypeScript
Python

call.onStatus((s) => {
  // dialing → ringing → in_progress → completed | failed | no_answer
  if (s === "in_progress") console.log("agent bridged, listening");
});

call.on_status(lambda s: print("status:", s))

Event	Fires when
turn committed	The turn detector decided that a real turn ended. The model generates its reply.
barge-in confirmed	Sustained speech over the agent passed `bargeConfirmMs`. The in-flight reply is cancelled.
backchannel dropped	A short continuer was suppressed. The agent kept the floor (diagnostic).

Preview: a dedicated typed Inference client

Preview. A standalone Inference client is in design. It is a typed handle that wraps agent attach, the commit gate, and audio bridging behind one object. The shape below is forward-looking and can change. For shipping code, attach via voice.agents.attach as shown above.

// PREVIEW — not yet stable
import { Inference } from "@clutchcall/sdk/inference";

const session = await Inference.attach(call.sid, {
  model: "speech-to-speech",
  codec: "pcm16",
  turnDetection: { responseMinSpeechMs: 600, bargeConfirmMs: 300 },
  onTurn:    (t) => console.log("turn latency ms:", t.latencyMs),
  onBargeIn: ()  => console.log("caller interrupted"),
});
await session.close();

Inference — Details — wire model, the commit gate, turn-latency metric
Inference — Cookbook — short task snippets
Voice — SDK Methods — the full call + audio-bridge surface

​Import

​voice.agents.attach

​Attaching at originate time

​The agent spec

​Turn detection

​Tools

​Tapping the audio bridge (optional)

​Events

​Preview: a dedicated typed Inference client

​Related

Import

`voice.agents.attach`

Attaching at originate time

The agent spec

Turn detection

Tools

Tapping the audio bridge (optional)

Events

Preview: a dedicated typed Inference client

Related