# Streams — Details

> One-to-many live broadcast and VOD over MoQT. RTMP/SRT/WHIP/TUS2 ingest, CMAF packaging, signed playback over WebTransport.

<img src="/images/diagrams/streams.png" alt="streams data flow" />

The **streams** modality is one-to-many live broadcast plus video-on-demand
(VOD). Content arrives over a standard ingest protocol, the engine packages it
as CMAF/fMP4, and the relay mesh fans it out to any number of viewers over
**MoQT** (Media over QUIC Transport). Browsers play it natively through
WebTransport; native SDKs subscribe the same relay namespace. A short-lived,
signed playback URL is the only thing a viewer needs — the relay enforces the
token for you.

Two surfaces make up the modality:

- A **control plane** (`Streams`) that manages live inputs, signing keys, API
  keys, assets, and analytics. It is plain HTTPS — stateless request/response.
- A **data plane** (`BroadcastPublisher` + `BroadcastViewer`) that moves
  CMAF segments over the persistent MoQT connection.

  - **Live broadcast** — Push once, fan out to thousands. Sub-second glass-to-glass on a warm path.
  - **VOD** — Resumable uploads via TUS2, packaged to CMAF, played from the same catalog
    the live path uses.
  - **Four ingest protocols** — RTMP, SRT, WHIP (WebRTC-HTTP ingest), and TUS2 — pick what your encoder or
    browser already speaks.
  - **Signed playback** — Mint a JWT-bearing playback URL; the relay verifies it inside the subscribe
    handler. The URL *is* the auth.

## When to use it

Reach for streams when one producer feeds many consumers and you want adaptive,
buffered media rather than a tight interactive loop:

- Live events, sports, town halls, product launches, IRL streaming.
- Low-latency "watch parties" where every viewer needs the same timeline.
- VOD libraries built from the same packaging pipeline as your live streams.
- Embedding playback in a browser without a separate plugin — WebTransport is
  the playback path.

If you need a bidirectional, per-participant audio loop (a call), use
[Voice](/modalities/voice/details). If you need typed pub/sub of small
messages, use [Data](/modalities/data/details).

## Ingest

The publish side accepts four ingest protocols. Each live input declares its
`ingest` kind (`rtmp`, `srt`, `whip`, or `fmp4`); VOD uses TUS2.

  <Tab title="RTMP">
    Classic encoder ingest (OBS, hardware encoders, mobile broadcasters). Point
    the encoder at the RTMP endpoint and authenticate with the per-input stream
    key. The engine ingests, transcodes, and CMAF-packages on the fly.
  </Tab>
  <Tab title="SRT">
    Secure Reliable Transport for lossy/long-haul contribution links. Same
    stream-key auth; preferred when the contribution path crosses the public
    internet and RTMP-over-TCP head-of-line blocking hurts.
  </Tab>
  <Tab title="WHIP">
    WebRTC-HTTP Ingestion Protocol — the browser-native, sub-second contribution
    path. A page captures camera/screen and WHIP-publishes with a single HTTP
    POST of the SDP offer; the engine answers and pulls the WebRTC media,
    transcoding it into the same CMAF pipeline. Ideal for "go live from the
    browser" without an external encoder.
  </Tab>
  <Tab title="TUS2 (VOD)">
    Resumable upload (the TUS2 protocol) for VOD. Large files upload in chunks
    and survive network drops — a paused upload resumes from the last confirmed
    offset. When the upload completes the engine packages the file to CMAF and
    publishes an asset with a playback id you can stream immediately.
  </Tab>

> **NOTE:**
> The browser `BroadcastPublisher` in the SDK is a WHIP-style alternative: it
> fragments a `MediaStream` to CMAF in the page and pushes each fragment as a MoQT
> object. For server-class contribution, prefer RTMP/SRT/WHIP into the engine and
> let the transcoder do the packaging.

## Wire model

The SDK connects over one of two **URL paths**, both of which resolve to the
same MoQT **namespace** on the relay:

```
URL path (client → relay):
  /publish/<input_id>?sk=<stream_key>   broadcaster ingest session
  /playback/<input_id>?tok=<jwt>        viewer session (carries a playback token)

MoQT namespace (what the relay fans out, live + VOD uniform):
  stream/<org>/<input_id>               live  (input_id = lv_…)
  stream/<org>/<asset_id>               VOD   (asset_id  = as_…, playback id pb_…)

Tracks within the namespace:
  .catalog        in-band catalog track (draft-ietf-moq-catalog)
  <rendition>     one media track per ABR rendition (e.g. video/720p, audio)
```

`/publish/…` and `/playback/…` are URL paths, not namespaces — the relay's
stream resolver maps them to `stream/<org>/<id>`. VOD reuses the same
`stream/<org>/…` namespace so live and on-demand playback are uniform.

The broadcast capability tag is `media.broadcast`. The relay writes a
per-broadcast analytics row keyed on that tag.

### The catalog

Every stream carries a **catalog** (a subset of `draft-ietf-moq-catalog`) that
declares its tracks — name, RFC 6381 codec string, and (for VOD) the CMAF init
segment. A **live** input publishes the catalog *in-band* on the `.catalog`
track, so it can update mid-stream (a rendition added, a codec switched). **VOD**
ships the catalog as `catalog.json` over HTTPS. The viewer parses the catalog,
opens a `MediaSource` buffer per track, and feeds CMAF segments into it.

```jsonc
{
  "version": 1,
  "live": true,
  "tracks": [
    { "name": "video/720p", "kind": "video", "codec": "avc1.42E01F",
      "width": 1280, "height": 720, "bitrate": 2500000 },
    { "name": "audio",      "kind": "audio", "codec": "opus",
      "sampleRate": 48000, "channels": 2 }
  ]
}
```

## Lanes & QoS

Streams ride MoQT's group/object model. Each CMAF fragment is one MoQT object;
an init/keyframe fragment opens a new **group** (so a late joiner can start at a
group boundary). Objects carry a **priority**: the init segment is highest
priority (it must arrive before media decodes), media segments follow. Under
loss, the relay drops by priority and group age rather than blocking the whole
stream — buffered, adaptive delivery, not a frame-perfect interactive lane.

| Property | Streams behavior |
| -------- | ---------------- |
| Delivery | Per-group, priority-ordered objects over MoQT |
| Late join | Starts at the most recent group boundary (keyframe) |
| ABR | One media track per rendition; the catalog enumerates them |
| Fallback | LL-HLS over HTTPS where WebTransport/MoQT is unavailable |

## Codecs & packaging

- **Video:** H.264/AVC (`avc1.*`) on the broad-compatibility path; the codec
  string is declared per track in the catalog.
- **Audio:** Opus or AAC (`mp4a.40.2`).
- **Container:** CMAF / fragmented MP4 (fMP4) — `ftyp+moov` init segment, then
  `moof+mdat` media fragments, fed straight into the browser's `MediaSource`.
- **Fallback:** **LL-HLS** (Low-Latency HLS) over HTTPS for environments where
  WebTransport isn't available. The CMAF segments are the same bytes; only the
  delivery framing changes.

## Signed playback (JWT)

Playback is authorized by a **short-lived JWT** minted from one of the org's
**signing keys** (Ed25519 by default, RS256 optional). The control plane returns
a playback URL with the token already attached as `?tok=`. The relay verifies
the JWT inside the SUBSCRIBE handler; a bad or expired token closes the session
with `auth_failed`. Tokens are server-clamped (30 s – 24 h, default 1 h) — re-mint
and re-open before expiry to keep long viewers alive.

> **WARNING:**
> The cleartext **stream key** for a live input is returned **once**, at
> `create()` / `rotateStreamKey()`. The control plane stores only a hash and will
> never return it again — capture and persist it the moment it's returned.

## Architecture

```
encoder/browser          engine (shard-per-core)              edge relays
─────────────            ─────────────────────────            ───────────
RTMP / SRT  ─┐           ┌─ ingest + transcode ─┐             ┌─ POP A ─┐
WHIP        ─┼─ ingest ─▶│  CMAF / fMP4 packager │─ MoQT ────▶│  fan-out │─▶ viewers
TUS2 (VOD)  ─┘           │  in-band catalog      │  (QUIC)    └─ POP B ─┘   (browser /
                         └───────────────────────┘                          native SDK)
```

The data plane runs on a **shard-per-core** reactor with an **AF_XDP**
kernel-bypass NIC fast path, **io_uring** for async I/O, and **lock-free
mcache / dcache rings** to move packets between cores without contention. One
QUIC connection multiplexes every track; the relay mesh fans out each group to
subscribers, so a million-viewer fan-out costs the publisher one upload, not one
copy per viewer.

## Related

- [Realtime Tracks](/concepts/realtime-tracks) — the MoQT publish/subscribe
  primitive underneath every modality.
- [Streams — SDK Methods](/modalities/streams/sdk-methods)
- [Streams — Cookbook](/modalities/streams/cookbook)
- [Streams — Recipes](/modalities/streams/recipes)