voice.on("...") event stream. You register a small set of callbacks:
one for each direction of audio, and one for transport session state.
Human-agent softphones also get a real push channel. This page lists every
callback. It also shows where the SDK gives you a stream and where you must
poll.
The data plane uses our media-over-QUIC (MoQT) transport and is fully
callback-driven. The call control plane is request/response — there is
no server-push socket for call state. Only the human-agent control channel
(
Agent) is a live EventEmitter. Design your integration around that split.The three surfaces at a glance
Audio frame callbacks
onUplink / onDownlink fire once per encoded 20 ms frame, in order,
with a microsecond timestamp. This is your real-time media loop.Transport session state
The client reports
ConnectionState transitions (Connecting → Connected →
Reconnecting → Closed / Failed). Auto-reconnect runs underneath.Human-agent control
Agent, the softphone control channel, is a real push
EventEmitter (callOffered, stateChanged, forcedLogout, …).Audio frame callbacks (data plane)
AnAudioBridge gives you one callback per direction. Both deliver encoded
frames (for example, a 20 ms Opus packet or a PCM16 chunk) exactly as they
arrive from the media plane. The SDK does not decode, resample, or buffer them
for you.
(frame: Uint8Array, timestampUs: bigint) => void
Inbound caller audio — what the caller says. You register this callback
when you
attach() the bridge to a call_sid. It subscribes to the
voice/<sid>/uplink track.(frame: Uint8Array, timestampUs: bigint) => void
Inbound cloud→caller audio — the audio that plays back to the caller.
You register this callback when you call
attachCaller() (the browser-caller
mirror of attach). It subscribes to the voice/<sid>/downlink track.timestampUs is the media timestamp in microseconds. Use it to sequence
frames, measure inter-frame gaps, or align frames with your own clock. The SDK
delivers frames in order, one at a time. There is no batching.
- TypeScript
- Python
codec accepts opus (default), pcm16, g711_ulaw, or g711_alaw. This is
the same AudioCodec set in every language SDK. Voice is audio-only. There are
no video frame callbacks.
Transport session state & auto-reconnect
The media plane keeps itself alive. The underlying MoQT transport client reconnects transparently. After a drop, it re-opens the session, re-announces every publication, and re-subscribes every subscription. YouronUplink / onDownlink callbacks resume without new wiring.
You can observe those transitions through the transport client’s onState
callback, which reports a ConnectionState:
enum
Connecting (0) · Connected (1) · Reconnecting (2) · Closed (3) ·
Failed (4). The callback signature is (state, reason?) => void.1
Connecting
The client reports this state immediately when you open it, before the
first session is up.
2
Connected
The session is established. On a re-connect (not the first connect), the
client replays your publications and subscriptions automatically.
3
Reconnecting
A live session dropped. The client retries with capped exponential
backoff. The backoff starts at 250 ms, doubles to a 5 s ceiling, and
resets on success.
4
Failed
The first connect never succeeded (bad relay host, bad auth, or blocked
UDP). This state is terminal for that client and rejects the connect
promise. It is different from
Reconnecting, which occurs only after at
least one successful connect.5
Closed
You called
close(), or the retry loop received a stop. This state is
terminal and clean.Known gap: the high-level
AudioBridge (attach / attachCaller) opens the
transport for you, but it does not currently forward an onState callback.
It reconnects silently. To observe session state today, you have two options.
Drive the MoQT transport client directly (the primitive under the bridge) and
layer the bridge’s publish/subscribe calls on top. Or infer a stall when your
frame callbacks go quiet. onState support in AudioBridge is planned, not
shipped.Reconnect leaves a gap, and the SDK tells you
A transparent reconnect is not gap-free. Objects published while the session was down are lost. (True gap-free replay needs a durable origin, which is not the default.) The transport does not hide this gap — it reports it. Each subscription exposes two optional lifecycle hooks on the MoQT primitive:(reason: string) => void
The track ended (the publisher unpublished, or the subscription was torn
down).
(info: DropInfo) => void
A delivery gap occurred. After a reconnect, you receive one
DropInfo with
reason: "reconnect" and sentinel fromGroup/fromObject/toGroup/toObject
values of -1. Use it, for example, to flush a jitter buffer or re-prime a
decoder.AudioBridge
wrapper wires only the frame callback. If you need onEnded / onDropped,
work with the MoQT subscription directly. See
Realtime tracks for
the track/namespace model these hooks belong to.
There is no call-state event stream
This is the most important point on this page. Call control is request/response over the control-plane API (tRPC to the control-plane host). The SDK does not open an event socket for the call lifecycle. There is nocall.on("answered"), no onRinging, and no server-push CDR event in the
Voice SDK.
A Call carries a status snapshot from the moment you fetched it
(dialing | ringing | in_progress | completed | failed | no_answer). To observe
a transition, re-fetch the call:
- TypeScript
- Python
Human-agent control channel (the one real event stream)
The human-agent softphone control channel is the exception to everything above. When you build a browser softphone for a live human agent, theAgent class is a real EventEmitter over a persistent connection (a
control-plane WebSocket at /agent/control). It pushes these events:
{ deviceId }
The gateway accepted the connection and assigned a device id.
{ state, ts }
The gateway acknowledged a
setState transition.{ state, auxReason?, ts }
The agent presence/aux state changed (server-driven, for example forced to
acw).CallOffer
The gateway offers a call to this agent. Confirm or reject before
ringUntilMs.{ callId, missedCount, forcedRona }
The ring window ended with no confirm.
forcedRona is true when the missed
counter crossed the RONA threshold and the gateway forced aux:rona.{ callId }
The gateway accepted your confirm. This is a one-shot event, different from
stateChanged.{ code, msg }
A protocol or authorization error occurred on the channel.
{ reason }
The gateway evicted this device (for example, a single-device kick).
{ clean, reason? }
The channel closed.
clean shows whether the close was graceful or a drop.This channel runs over the control-plane WebSocket today.
Agent accepts a
preferTransport: 'wt' | 'wss' | 'auto' hint, but WebTransport carriage of
/agent/control is not yet shipped. 'auto' resolves to WSS.Callback inventory
Related
TypeScript SDK
Full
Voice / Calls / AudioBridge / Agents reference.Python SDK
The same callback shapes in Python (
on_uplink, publish_downlink).Sessions, calls & tracks
The
voice/<sid>/{uplink,downlink} track model behind these callbacks.Transport API
Connection states, reconnect, and the MoQT client surface in depth.

