The four codecs
Every SDK exposes the sameAudioCodec type. Set it when you attach an audio
bridge or bind an agent:
¹
pcm16 frames are 8 kHz mono on the call bus by default; set sampleRate
to resample. See Audio shape.
- TypeScript
- Python
- Rust
opus.
Choosing a codec
Work from the consumer of the audio, not from the carrier. The carrier leg is the bridge’s problem, not yours:1
Talking to a browser or app? Use opus.
Browser softphones capture and encode Opus natively. The encoder comes
straight from the browser’s audio pipeline. Keep the leg on
opus
end-to-end. There is then no re-encode, and you get the best quality for
the bitrate. This is why opus is the default.2
Feeding a realtime AI model? Use pcm16.
Most speech-to-speech and ASR models want uncompressed PCM on their input.
Ask for
pcm16. The bridge hands your code decoded 16-bit samples with no
codec in the middle. This is the same raw format that the agent runtime
uses internally.3
Bridging straight to a PSTN trunk? Match the trunk's G.711.
You can relay a call leg to a SIP/PSTN trunk without a need to inspect or
transform the audio. In that case, ask for the trunk’s own G.711 variant:
g711_ulaw for North America and Japan, g711_alaw for the rest of the
world. The frames pass through untouched (see
passthrough).µ-law vs A-law is a regional wire convention, not a quality choice. North
American and Japanese carriers use µ-law (PCMU, payload type 0). Nearly everyone
else uses A-law (PCMA, payload type 8). Both are 8 kHz 8-bit companded G.711
with effectively identical fidelity. Pick the one that your carrier negotiates.
If you get it wrong, you hear distortion. When two G.711 legs of different
flavours meet, the bridge transcodes between the two anyway.
Transcoding at the bridge
Carrier legs almost always arrive as G.711 at 8 kHz (PCMU or PCMA). Browser legs arrive as Opus at 48 kHz. The bridge sits between the wire and your code. It converts in both directions: it decodes to the runtime’s internal PCM representation, and it re-encodes to the format that each side needs:- The runtime bus is 8 kHz PCM16 end-to-end. A 20 ms tick is 160 samples at 8 kHz. That is the fidelity ceiling for anything routed through the agent path. This is telephone-band audio — exactly what PSTN delivers, and what speech models are trained on.
- Opus keeps a 48 kHz RTP clock even though the audio is narrowband. The
Opus rtpmap stays
opus/48000/2. Each 20 ms frame advances the timestamp by 960, per the codec’s spec. The encoder’s internal rate is driven to narrowband to match the bus. You do not manage any of this — it is the fix that makes browser Opus play back clean. But it is why you see48000in an SDP, even for an 8 kHz call.
Passthrough fast paths (zero transcode)
Transcoding is cheap, but the fastest audio is audio that you never touch. Two paths skip the codec entirely:1
G.711 in, same G.711 trunk out.
Bridge a PSTN leg to another PSTN/SIP trunk of the same G.711 flavour,
and ask for that flavour. The bridge then relays the frames verbatim: no
decode, no re-encode. This is the norm for straight call-forwarding and
carrier hand-off.
2
External µ-law WebSocket audio to a µ-law trunk.
Are you migrating from a Twilio Media Streams–style source? Its 8 kHz µ-law
WebSocket frames map straight onto a PCMU SIP trunk with zero
transcode. The payloads are already in the trunk’s format. See
Migrate from Twilio for the
wiring.
Audio shape: sampleRate, channels, frameMs
The codec choice sets the format. Three more knobs set the shape. All have sensible defaults, so you usually set onlycodec:
"opus" | "pcm16" | "g711_ulaw" | "g711_alaw"
default:"opus"
The wire format for this leg, per the table above.
number
default:"48000"
Samples per second.
opus runs at 48000. G.711 is fixed at 8000. For
pcm16, set this to the rate that your model wants (e.g. 16000 or
24000). The bridge resamples for you. The stateful resampler avoids the
“chipmunk” artefacts of a naive double-resample.number
default:"1"
Voice is mono. Recordings mix the two legs into stereo downstream (caller
left, agent right). That is a recording concern, not a codec concern.
number
default:"20"
Milliseconds of audio per frame/object. 20 ms is the RTP and Opus standard
(160 samples at 8 kHz, 960 at 48 kHz). Every pacer and jitter buffer on the
path assumes it. Change it only if a downstream consumer demands it.
Related
Sessions, calls, tracks & streams
Where the codec’s capability tag (
voice/opus, voice/pcm16) fits the track model.SIP trunking
How carrier legs negotiate G.711 on the wire before the bridge sees them.
Browser audio capture
The Opus path: the browser’s own encoder onto QUIC/MoQT.
Migrate from Twilio
The µ-law WebSocket → PCMU trunk passthrough on-ramp.

