SIP gateway (B2BUA)
The gateway terminates carrier trunks and agent phones. It owns call
setup, auth, and teardown.
RTP media plane
The media plane moves the audio. It decodes and encodes, handles jitter,
DTMF, recording, and quality.
Dialplan & routing
The dialplan decides where each call goes: IVR, skill queues, and agent
selection.
Agent runtime
The runtime runs the AI agent: ASR/LLM/TTS or speech-to-speech, tools, and
turn-taking.
MoQT transport
The transport carries browser and app audio as media-over-QUIC on a single
:443 plane.
Relay mesh
The mesh fans a published track out to subscribers across cores and
across hosts.
Inference
An optional self-hosted GPU control plane runs on-prem speech-to-speech
models.
SIP gateway (B2BUA)
The SIP gateway is the front door for telephony. It terminates carrier trunks and registrations from agent phones. It runs as a back-to-back user agent (B2BUA). Every call is two legs (caller and agent) that the gateway bridges. The gateway can then transcode, transfer, and record without either side seeing the other’s signalling. It owns the full SIP lifecycle:INVITE / 200 / ACK / BYE, REFER-based
transfers, digest authentication, source-IP ACLs, and a spam filter. An
internal registrar is on by default. Software and desk phones can
REGISTER directly against your workspace’s SIP domain, and you run no
separate registrar. The gateway matches trunks by source IP and resolves them
to a tenant. Phone registrations land in a short-lived location store keyed by
address-of-record.
The gateway is a native SIP stack, not a fork of Asterisk or FreeSWITCH. If
you migrate from one of those, you re-point trunks and agents. You do not
bridge two systems. See
Migrate from Asterisk / FreeSWITCH.
RTP media plane
The gateway handles signalling. The media plane handles the audio itself. It runs RTP/RTCP send and receive in the engine, on the same core that owns the call. There is no hop between where the packet lands and where the engine processes it. Its jobs:- Codec bridging — the media plane decodes carrier G.711 (PCMU/PCMA) to the runtime’s internal 8 kHz PCM16. It encodes agent audio back out. Opus and G.711 legs meet here.
- Jitter handling and pacing — a per-session jitter buffer smooths arrival. Per-session pacers clock outbound frames on a 20 ms cadence.
- DTMF — the media plane detects and injects RFC 2833 telephone-event digits.
- Recording tap — a recording sink can capture both legs (caller and agent as separate stereo channels) without effect on the live path.
- Quality signals — RTCP feedback yields jitter and packet-loss. The engine attributes per-call byte counts to each tenant for the traffic dashboard.
Jitter and packet-loss are available today. A full MOS score per call is
still rolling out. See what is live on
MOS, jitter & loss.
Dialplan & routing
The dialplan decides what happens to a call after the engine answers it: play an IVR prompt, send it directly to an AI agent, or queue it for a human. It was the first routing role built into the engine. It is where you express “this number, from this trunk, at this time, goes here.” For contact-center flows, a skill-queue routing brain sits on top. It processes an inbound call in this sequence:- It classifies the call (IVR / VDN).
- It matches the call to a skill queue.
- It offers the call to an eligible agent, AI or human, with ring strategies, glare protection, and re-offer on no-answer.
Agent runtime
The agent runtime makes a call conversational. It embeds the AI agent directly in the engine, next to the media plane. Audio does not leave the box to reach the model layer. Each turn runs through a small pipeline (a DAG). The pipeline is either a staged ASR → LLM → TTS chain or a single speech-to-speech provider. You bring your own providers, with credentials scoped per tenant:- Cloud realtime endpoints (OpenAI Realtime, Gemini Live).
- Separate ASR/LLM/TTS vendors (Deepgram, ElevenLabs, and others).
- Your own realtime endpoint.
Runtime overview
Learn about providers, pipelines, and how the runtime attaches to a call.
Turn detection & barge-in
Learn when the agent listens, speaks, and yields the floor.
MoQT transport
Telephony reaches you over SIP/RTP. Browsers and apps reach you over QUIC. The transport component carries their audio as media-over-QUIC (MoQT) over WebTransport on a single:443 plane. You open no separate media port.
A browser publishes its microphone as a MoQT track. It subscribes to the
agent’s reply as another track. The engine publishes the caller’s audio the
same way, so a human agent’s browser can subscribe to a live PSTN caller. The
SDK uses the browser’s WebRTC stack only as an encoder tap: it extracts
encoded Opus frames and sends them over QUIC. ICE/DTLS/SRTP are never on the
wire for these clients.
QUIC/MoQT is first-class. WebSocket (control and data) and WebRTC
(media) are supported fallbacks for browsers and networks where QUIC or
WebTransport is unavailable. See
Browser compatibility.
Relay mesh
A published track usually has more than one subscriber: the agent, a supervisor who listens in, a recorder. These subscribers are not always on the same core or even the same host. The relay mesh does the fan-out. It takes one published copy of a track and delivers it to every subscriber, across cores within a host and across hosts within a region. You do not call the relay directly. Publish and subscribe operations route through it automatically. The mesh lets a call scale past one core and one machine, and the publisher pays no per-subscriber cost. The architecture page shows where the mesh sits in the data path. Sessions, calls, tracks & streams covers the track model that the mesh fans out.Inference
For most deployments, the model layer is a cloud provider that you configure in the runtime. In that case, there is no separate inference component to think about. The inference component matters when you run models on your own GPUs, typically on-prem, for data-residency or cost reasons. In that mode, a control plane discovers model pods, scrapes their load, and routes requests to them over QUIC. It presents a single speech-to-speech endpoint. The agent runtime dials that endpoint like any other provider.The self-hosted inference control plane is shipped. End-to-end
speech-to-speech on it is still being hardened. Validate latency against
your workload before you cut production traffic over. Details and current
status are in
Self-hosted inference.
Next steps
Architecture
See these components wired together into the end-to-end data path.
Sessions, calls, tracks & streams
Learn the object model that the components operate on.
Deployment models
Compare managed cloud, on-prem, and hybrid, and see where each component
runs.
Handoffs
Learn how AI and human agents trade a live call without loss of audio.

