Agents

An agent is the AI voice pipeline that answers a call. The pipeline defines the mode (cascade or duplex), the provider for each stage, when the agent yields the floor, and the tools the agent can call. You author the pipeline once as an AgentSpec. You then address the agent by id. Use the Agents API to bind an agent to a live call. One call to agents.attach(sid, agentId) connects the audio bridge end to end. This is the “drop-in voice agent” path. You do not open an AudioBridge. You do not move audio frames yourself. You give the runtime a call_sid and an agent id. The runtime then controls the call.

The SDK attach call takes an agent id string, not an inline spec object. The AgentSpec stays on the server. Author the spec in the console at agent.clutchcall.dev or push it through the control-plane API. Then reference the spec by id from every call. This keeps provider keys and prompts out of your client code. You can also re-tune an agent without a client release. This page documents the spec fields in summary. Runtime configuration documents the fields in full.

Attach an agent to a call

Bind an agent to a call_sid. You get the call_sid from voice.calls.originate, from an inbound-call webhook, or from voice.calls.get. The call must be live (ringing or in progress).

TypeScript
Python

import { Voice } from "@clutchcall/sdk";

const voice = new Voice({
  baseUrl: "https://engine.clutchcall.dev",
  apiKey:  process.env.CLUTCHCALL_API_KEY!,
  orgId:   "<workspace-id>",
});

// Bind the server-side agent "support-agent" to a live call.
await voice.agents.attach(callSid, "support-agent");

from clutchcall import Voice

voice = Voice(
    base_url="https://engine.clutchcall.dev",
    api_key=os.environ["CLUTCHCALL_API_KEY"],
    org_id="<workspace-id>",
)

# Bind the server-side agent "support-agent" to a live call.
voice.agents.attach(call_sid, "support-agent")

attach resolves the control-plane route voice.agents.attach with your orgId, the sid, and the agent id. On success, the engine hydrated the AgentSpec, prewarmed the providers, and connected the caller’s audio to the pipeline. The same signature exists in the Go and Rust SDKs (Agents.Attach(callSid, agent) / agents.attach(call_sid, agent)). See the SDK reference.

Parameters

string

required

The call_sid of a live call. This is the only identifier that you need to address a call. The SIP dialog, the media session, the agent session, and the telemetry all key off this id.

string

required

The id of a server-side agent (its AgentSpec). Author the agent in the console at agent.clutchcall.dev or through the control-plane API. Do this before you reference the agent.

Attach at originate time

For outbound calls, you rarely need a separate attach. Pass the agent id to originate. The engine binds the agent automatically on answer. The agent speaks as soon as the callee answers.

TypeScript
Python

const call = await voice.calls.originate({
  to:      "+15551234567",
  from:    "+15559876543",
  trunkId: "<trunk-id>",
  agent:   "support-agent",   // attached on answer — no second call
});

call = voice.calls.originate(
    to="+15551234567",
    from_="+15559876543",
    trunk_id="<trunk-id>",
    agent="support-agent",   # attached on answer — no second call
)

For inbound calls, your dialplan or trunk routing selects the agent (a DID or a skill maps to an agent). No client attach is necessary. Use agents.attach when you want to bind or swap the agent from your own code. Examples: after you screen a call, on an IVR selection, or in a webhook handler.

Re-attach to a different agent

A swap of agents mid-call is a transfer to an agent, not a second attach. The transfer stops the current pipeline cleanly before the new pipeline takes over. Use Call.transfer with an agent (see Calls):

await call.transfer({ agent: "billing-agent" });

A handoff of a live call from an AI agent to a human is a different operation. See AI ↔ human handoffs.

The AgentSpec

The AgentSpec is the single document that defines an agent. attach binds a call to the spec by id. This section lists the fields that the spec resolves. Every field has a default. A minimal spec is a name, a prompt, and the providers that you want to override. This section is a summary of the top-level shape. Runtime configuration is the full field-by-field reference, with every timing knob and budget.

string

required

The human-readable agent name. The console shows this name. The engine stamps this name onto call records, so you can tell agents apart in the dashboards.

string

default:"cascaded"

The agent mode. cascaded chains three provider sessions (speech-to-text → language model → text-to-speech). This mode gives per-stage control and the widest model choice. realtime gives the full turn to one duplex speech-to-speech model. This mode gives the lowest turn latency. See BYO ASR/LLM/TTS and BYO speech-to-speech.

string

The agent’s persona and task (instructions on a realtime node). Keep replies short and spoken-friendly. Short sentences work better than paragraphs on a phone line.

Model & provider

The provider fields that you set depend on the mode. Provider keys are per-tenant. The control plane resolves the keys at call setup. The spec never contains the keys. If a selected provider has no usable key, the engine refuses the agent at session creation with a clean error. The agent does not fail mid-call.

string

Cascade only. The streaming speech-to-text vendor and model — for example, deepgram / nova-2. The engine feeds the vendor at the runtime’s 8 kHz PCM16 contract. The engine resamples for providers that natively want 16/24 kHz.

string

Cascade only. The language-model vendor and model — openai, anthropic, gemini, or self-hosted ollama, with a model id such as gpt-4o-mini or claude-sonnet-4-5. Tool calling runs at this stage. Set llm_fallback_provider for a one-turn resilience path.

string

Cascade only. The text-to-speech vendor and voice id — for example, elevenlabs, cartesia, deepgram, or openai, with the provider’s voice id.

string

Realtime only. On a realtime entry node, you set one duplex provider instead of three cascade stages. Examples: openai / gpt-realtime / shimmer, or gemini with a Kore voice. One model listens and speaks on the same bidirectional session.

Shipped duplex providers include OpenAI Realtime, Gemini Live, and xAI Grok Voice. Support to serve your own open-weights duplex model on your GPUs is in preview. See self-hosted inference.

turn_detection

This block controls how the agent decides that the caller stopped talking. It also controls how the agent yields the floor when the caller barges in. Omit the block for phone-safe defaults. Turn Detection & Barge-in documents the full behavior.

string

default:"on-device VAD"

This field sets which component runs voice-activity detection. The default (omit type) runs an on-device VAD on the gateway. The default is the correct choice for cascaded agents. Set "energy" for the lightweight built-in energy gate. "server_vad" defers end-of-turn to a cloud realtime provider that already emits speech-start/stop events. The runtime auto-selects "server_vad" for a realtime entry node unless you set type explicitly. A cascade with server_vad never sees a turn boundary. Use the on-device default for a cascade.

integer

default:"500"

The trailing silence before end-of-utterance fires. Set a lower value (~300) for faster replies. Set a higher value (~800) for callers who pause to think.

integer

default:"300"

The hold-and-confirm barge-in window. A speech onset arms a pending barge. The pending barge cancels the agent only if speech continues past this window. As a result, a backchannel (“mhm”, “ok”) never cuts the agent off. Set 0 for legacy immediate barge-in. Use 0 only on echo-cancelled (browser) transports.

object

The idle watchdog. soft_prompt_s re-engages a silent caller. hangup_s then ends the call. The watchdog is on by default, with sensible ceilings, so a stuck agent cannot hold a line open forever.

tools

The tools that the agent can call mid-conversation: an HTTP endpoint, a remote MCP server, or a handler in your own SDK. The Agent DAG uses the same tool model. An llm (or realtime) node advertises tools by name. The model decides when to call a tool. The runtime validates the arguments against the tool’s JSON Schema before dispatch. Each session also unions your tools with a built-in telephony toolset (transfer, hold, DTMF, disconnect, supervisor request). On a name collision, your tool wins. Tool calling is the full reference.

array

A list of tool definitions. Each entry shares one envelope: kind, name, description, silent, and spec. The kind-specific configuration goes inside spec.

string

required

http (shipped) calls an endpoint that you own. mcp (shipped) runs a JSON-RPC tools/call against a remote MCP server over Streamable HTTP. client (scaffolded) pushes the call to your connected SDK, which fulfils the call.

string

required

The function name that the model sees and calls, and the human-readable purpose that the model reads. Write the description for the model. The description is the primary signal for when the model calls the tool.

object

A standard JSON Schema for the tool’s arguments. On an http tool, you can omit the schema. The runtime then synthesizes a schema from the {{placeholder}} tokens in your URL and body.

Example AgentSpec

This example shows a cascaded support agent with one HTTP tool. Author the spec in the console, or push the spec through the control-plane API. Then reference the spec by id (support-agent) from agents.attach.

AgentSpec

{
  "name": "support-agent",
  "pipeline_mode": "cascaded",
  "system_prompt": "You are a friendly support agent for Acme. Keep replies short and spoken-friendly.",
  "asr_provider": "deepgram",
  "asr_model": "nova-2",
  "llm_provider": "openai",
  "llm_model": "gpt-4o-mini",
  "llm_fallback_provider": "anthropic",
  "tts_provider": "elevenlabs",
  "tts_voice": "<voice-id>",
  "turn_detection": {
    "silence_threshold_ms": 500,
    "barge_confirm_ms": 300
  },
  "tools": [
    {
      "kind": "http",
      "name": "lookup_order",
      "description": "Look up an order by its ID and return its status.",
      "spec": {
        "method": "GET",
        "url": "https://api.example.com/orders/{{order_id}}",
        "parameters": {
          "type": "object",
          "properties": { "order_id": { "type": "string" } },
          "required": ["order_id"]
        }
      }
    }
  ]
}

Runtime configuration

The full AgentSpec field reference, with every provider, timing, and budget knob.

Calls

Originate, transfer (re-attach to a different agent), and hang up.

Tool calling

Give the agent HTTP, MCP, and client tools that it can call mid-conversation.

Turn detection & barge-in

The behavior behind the turn_detection block.

Sessions & audio bridge

The lower-level path, for when you want to move audio frames yourself.

AI ↔ human handoffs

Hand a live call from the agent to a human.

Overview

Quickstart

Core Concepts

Build Guides

Agent Runtime

Transport to Web & Apps

Transport to Telephony

CTI (CSTA)

High Availability

Comparison & Benchmarks

Migration

Integrations

Observability

SDK Reference

API Reference

Troubleshooting

Cookbooks

Recipes

Attach an agent to a call

Parameters

Attach at originate time

Re-attach to a different agent

The AgentSpec

Model & provider

turn_detection

tools

Example AgentSpec

Runtime configuration

Calls

Tool calling

Turn detection & barge-in

Sessions & audio bridge

AI ↔ human handoffs

​Attach an agent to a call

​Parameters

​Attach at originate time

​Re-attach to a different agent

​The AgentSpec

​Model & provider

​turn_detection

​tools

​Example AgentSpec

​Related

Runtime configuration

Calls

Tool calling

Turn detection & barge-in

Sessions & audio bridge

AI ↔ human handoffs

Attach an agent to a call

Parameters

Attach at originate time

Re-attach to a different agent

The AgentSpec

Model & provider

turn_detection

tools

Example AgentSpec

Related