open-standards/wai/extensions/live-ingest.md

# WAI Extension: Live Ingest and Egress

> Status: Draft. Binds the contribution and playback protocols of standard encoders and
> players to the tracks of a room ([live-rooms](live-rooms.md)): WHIP (RFC 9725) into a
> room's tracks, WHEP (`draft-ietf-wish-whep-05`) out of them, and, as mappings only in
> this revision, Enhanced RTMP v2 multitrack and SRT. An ingest gateway admits a
> publisher by its room grant, takes its media frame by frame, packages each frame as one
> WAI envelope on a MoQ track ([moq-streaming-format](moq-streaming-format.md)), maps
> its media time onto the room clock with a stated bound, and signs every group's
> receipts as the first hop ([jwp-receipts](jwp-receipts.md)). An egress gateway verifies
> every object against its receipt before it hands the frame to a player. No new
> transport and no new token format: WAI defines none by charter. The reference
> implementation of WHIP and WHEP is `joule-moq-ingest` in
> `joule-code/joule-code-rs/crates/`, on the `joule-moq` codec and the `joule-moq-relay`
> relay.
> Keywords **MUST**, **MUST NOT**, **SHOULD**, **MAY** are RFC 2119/8174.
>
> Texts read for this revision: RFC 9725 (WHIP); `draft-ietf-wish-whep-05` (WHEP), an
> Internet-Draft not past Working Group Last Call; RFC 6750 §2.1 and §3; RFC 8840 (the
> trickle ICE fragment); RFC 9110 §8.8.3 and §13.1.1; RFC 6585 §3; RFC 9457; RFC 6184
> (H.264 over RTP); RFC 7741 (VP8); RFC 9628 (VP9); the AV1 RTP payload format v1.0.0;
> RFC 7587 and RFC 6716 (Opus); `draft-ietf-moq-transport-22` (MOQT) and
> `draft-ietf-moq-msf-01` (MSF); Enhanced RTMP v2, document version
> `v2-2026-01-31-r2` (E-RTMP); and SRT 1.5.7, its stream ID guidelines and its rejection
> codes. A section number of one of those texts here is that revision's.

## 1. Scope

In scope: how a WHIP request is admitted under a room grant (§3); the tracks an ingest
publishes and its catalog (§4); one frame as one object, the payload framing of each
codec, and how frames form groups (§5); the `live` member of an object's manifest (§6);
the mapping from RTP time to the room clock and its bound (§7); the receipts the gateway
signs (§8); WHEP egress and what it verifies (§9); the bounds a gateway holds a session to
(§10); trickle ICE and ICE restarts (§11); the mappings of E-RTMP v2 multitrack (§12) and
SRT (§13); the refusals and their statuses (§14); and how an ingest relates to capture
binding and to metering (§15, §16).

Out of scope: a transport (WebRTC, MoQ, RTMP and SRT are the transports); how a principal
obtains a grant (live-rooms §1); transcoding: a gateway carries the frames the publisher
sent, as the publisher encoded them; slots and their takeover and failover (live-rooms
§7), which an ingest into a slot follows and this revision does not map; and simulcast or
SVC within one WHIP session (RFC 9725 §4.8).

**Why an extension of its own.** [moq-streaming-format](moq-streaming-format.md) says
what a WAI track is on MoQ. This text binds four other protocols onto those tracks, under
the rooms and grants of the `wai.live` family; written into that extension it would make
a MoQ format describe RTMP and SRT. It sits beside [live-rooms](live-rooms.md),
[live-events](live-events.md) and [capture-binding](capture-binding.md), and changes none
of them.

Relationship to the core spec: no record and no capability is registered here. The
payload capabilities a live track carries are SPEC §5's, among them `wai.video.vp8` and
`wai.audio.opus.packet`, registered for this binding. The manifest member `live` (§6) is
defined here; a reader that does not know it ignores it (SPEC §3).

## 2. Terms

- **Room**, **room clock**, **principal**, **grant**, **room authority**: as in
  live-events §2 and live-rooms §2.
- **Gateway**: an endpoint that speaks one of the protocols of this text on one side and
  MOQT to a relay on the other. An **ingest** gateway takes media from a **publisher**; an
  **egress** gateway hands media to a **player**. One process may be both.
- **First hop**: the gateway an ingest's media reaches first. It is the MoQ publisher of
  the tracks (MOQT §7.6) and the signer of their receipts (§8).
- **Owner**: the principal whose tracks they are: the grant's `principal` (live-rooms
  §6.1).
- **Frame**: one access unit of video, or one packet of audio, as the RTP payload format
  of its codec delivers it whole (§5.1).
- µs: microseconds. Integers are little-endian where bytes are written.

## 3. Admission

### 3.1 The bearer token

A WHIP or WHEP request is authenticated by an HTTP `Authorization` header with a bearer
token (RFC 9725 §4.7.1; WHEP §4.9.1; RFC 6750 §2.1). The token **is a room grant**: the
`WRG1` record of live-rooms §4.1, whole, in the base64url alphabet without padding (RFC
4648 §5). The room it names is the room the request is for. WAI defines no second token
format; a deployment that issues tokens of another format maps them onto a grant as
live-rooms §10 says, and admits by the grant.

A grant is a public record, so presenting it proves nothing about who presents it. Every
request that carries a grant also carries a **proof of possession** of the grant's key
(live-rooms §10.5): a `WRP1` record, in base64url, in the `WAI-Room-Proof` field, over a
challenge the gateway issued and the request's context (§3.2 step 7). The gateway issues
single-use challenges in the `WAI-Room-Challenge` field: on a `401` that refuses for want of
a good proof, and on every response to a request that carried a grant, the one a `201`
carries being scoped to the session it opened (live-rooms §10.5.3).

### 3.2 The checks

A gateway holds the signed descriptor of every room it serves. It admits a token by these
checks, in order, and refuses at the first that fails with the reason it names:

1. The token decodes, and the room its grant names (bytes 5 to 36) is a room the gateway
   holds: otherwise `room_unknown`, or the reading refusal of live-rooms §4.3 step 1
   (`truncated`, `malformed`, `unknown_version`).
2. live-rooms §4.3 steps 1, 2 and 4 to 8, against that room's descriptor: the record
   reads; its signature verifies under the room's authority; it names the room; its
   fields keep §4.3's rules; it is at most fourteen days long; it is within its role and
   within the room's ceiling. Each refusal is §4.3's reason. Step 3, an exchanged grant's
   link to the grant it replaces, needs the earlier grant, which a token does not bring:
   the gateway checks instead that the room clock has reached the exchanged grant's
   `effective` tick.
3. The grant is good at the gateway's clock (live-rooms §4.3): otherwise
   `outside_grant_window`.
4. Neither its digest nor its viewer's tag is among the revocations the gateway holds for
   the room's authority's key log (live-rooms §5): otherwise `grant_revoked`.
5. Where the grant lists `origins`, the gateway's own origin is one of them: otherwise
   `origin_not_granted`.
6. Where the grant's `once` is not zero, no session of this gateway was opened under it:
   otherwise `grant_reused`. The nonce is spent when a session opens.
7. The request's proof passes live-rooms §10.5.4 for the grant, as this gateway reads it
   (its digest and its principal), with the challenge it holds for the request's scope (the
   endpoint, or the session its URL names), the challenges it consumed before, and the
   context `METHOD //authority/path` of the request it received: otherwise that section's
   reason (`pop_missing`, `pop_malformed`, `pop_wrong_grant`, `pop_wrong_key`,
   `pop_bad_signature`, `pop_wrong_challenge`, `pop_replayed`, `pop_wrong_context`). A
   proof that passes consumes its challenge: it is good for one request.

A token missing or not a bearer token is answered `401` with `WWW-Authenticate: Bearer
realm=…` and no error (RFC 6750 §3.1). A token refused by these checks is answered `401`
with `error="invalid_token"` and the reason as `error_description`.

### 3.3 The powers

The grant must give what the request asks for (live-rooms §3.3). A grant that passes §3.2
and lacks it is answered **`403`** with `WWW-Authenticate: Bearer
error="insufficient_scope"` (RFC 6750 §3.1):

- **WHIP**: the grant's `publish` bit for each media kind the offer carries, audio (1)
  and video (2): otherwise `publish_not_permitted`. A screen is published as media kind
  `screen` (3) only through a mapping that names it, which this revision does not.
- **WHEP**: `subscribe`, and a `max_layer` at or above the layer asked for:
  otherwise `subscribe_not_permitted` or `layer_not_permitted`. The room in the resource's
  path is the grant's: otherwise `wrong_room`.

Where the grant names a publish profile for a media kind (live-rooms §4.1 `profiles`),
the session accepts only the codecs of that profile's `codecs` that the gateway carries,
by RFC 6381 family (`avc1`, `vp8`, `vp09`, `av01`, `opus`), and states the profile's
layer-0 `max_kbps` as the media section's bandwidth (§10).

### 3.4 The session's resource

Every request to a session's resource (`PATCH`, `DELETE`, `GET`, `HEAD`) carries the token
the session was opened with, and a proof of possession over a challenge scoped to that
session (§3.2 step 7); another token, however good, is answered `403`
(`not_the_session_token`). A session's URL carries at least 128 random bits (RFC 9725
§5).

## 4. Tracks and the catalog

### 4.1 Names

A WHIP session publishes, for each media kind its offer carries, one **media track** and
its **receipts track**, and, for the owner, one **catalog track** and its receipts track:

| track | Track Namespace (live-rooms §6.1) | Track Name |
|---|---|---|
| media | (room id, owner, `audio` or `video`) | `0` |
| its receipts | the media track's | `0_receipts` |
| catalog | (room id, owner) | `catalog` |
| its receipts | the catalog's | `catalog_receipts` |

A WHIP session sends one encoding of each kind, so its media tracks are layer 0
(live-rooms §6.3, `single`). A receipts track is named as live-rooms §6.1 names
one, the media track's name and `_receipts`, and is of packaging `wai-receipts`. The
catalog is the owner's MSF catalog (MSF §5): the namespace prefix (room id, owner) reaches
it and every track it declares.

### 4.2 Publication

The gateway publishes every track at the relay with PUBLISH (MOQT §9.8) before it answers
the WHIP request, so a subscriber can subscribe from the first group. The tracks' default
publisher priority is 0 for audio and the catalog and 1 for video (MOQT §5.1.1): a
subscriber that falls behind keeps the sound. A receipts track's priority is its media
track's (moq-streaming-format §4.1). No track carries a delivery timeout.

When the session ends the gateway ends each open group, publishes its receipts, waits for
every stream to be acknowledged, and sends PUBLISH_DONE with status `TRACK_ENDED` on each
track (MOQT §9.9). The relay then ends its subscriptions: a subscriber sees the track end.

### 4.3 The catalog

The catalog is RFC 8785 canonical JSON, `{"tracks": [...], "version": 1}`, one object a
group. Each media track's entry carries MSF's `namespace` (the MOQT §8.8 rendering of its
namespace, live-rooms §6.2), `name`, `packaging` `"wai"`, `isLive` `true`, `role`,
`renderGroup` 1, and, where the bitstream states them, `codec` (RFC 6381), `width` and
`height`, or `samplerate` and `channelConfig`; and moq-streaming-format §4's
`wai_capabilities` (the one capability its objects dispatch to), `wai_receipts` `true`,
`wai_receipt_alias` (§8.2) and `wai_receipts_track`. Each is followed by its receipts
track's entry.

The gateway publishes the catalog once every media track has carried its first frame, or a
second after the first did, whichever comes first, and publishes it again, whole, in a new
group, when a track's codec or picture size changes. Its group IDs follow §5.3's rule. The
catalog is signed as the media is (§8): each catalog group is one object, receipted on
`catalog_receipts`. A relay that checks every track of a room at its door forwards it, and
a player verifies what the catalog declares before it relies on it (§9.1).

## 5. Objects and groups

### 5.1 One frame, one object

Each frame is one MOQT object, and each object is exactly one `WAI1` envelope
(moq-streaming-format §3). The frame is what the codec's RTP payload format reassembles,
whole:

| codec | RTP payload format | capability (SPEC §5) | the object's payload |
|---|---|---|---|
| H.264 | RFC 6184 | `wai.video.avc` | one access unit, Annex B byte stream: each NAL unit after a four-byte start code |
| VP8 | RFC 7741 | `wai.video.vp8` | one frame in the IVF frame container, FourCC `VP80`: the 32-byte file header (the frame's size, a time base of 1/90 000, one frame), the 12-byte frame header (its length, its time in 90 kHz ticks), the frame |
| VP9 | RFC 9628 | `wai.video.vp9` | as VP8, FourCC `VP90` |
| AV1 | AV1 RTP payload format v1.0.0 | `wai.video.av1` | one temporal unit in SPEC §5's AV1 framing: `n` = 1, the largest frame's height and width from the sequence header, a frame rate of 90 000 over the ticks since the track's previous frame, then the temporal unit's length and the unit: a temporal delimiter OBU (which RTP does not carry, and which the gateway restores), then every OBU with its size field |
| Opus | RFC 7587 | `wai.audio.opus.packet` | one Opus packet (RFC 6716 §3) |

A frame of a codec this table does not list is not carried. A frame larger than the
gateway's bound is dropped and counted (§10).

**A live track's objects are one stream.** A frame is a unit of its codec's stream, not a
picture or a sound decodable alone. The objects of a media track, in object order from a
group's first object and in group order, **are** consecutive units of one stream of the
codec: a sink decodes them in that order with one decoder state, from a group's first
object, and resets that state where a group follows one it did not receive. The capability
of every object says how its payload is read; this rule says how consecutive payloads
relate. A sink that joins a track joins at a group's first object.

### 5.2 Groups

- **Video**: a group begins at each keyframe (an IDR access unit for H.264; a key frame
  for VP8 and VP9; a temporal unit with a sequence header for AV1) and holds it and every
  frame up to the next. A frame that arrives while no group is open (before the first
  keyframe) is dropped, counted, and answered with a keyframe request (a PLI, RFC 4585
  §6.3.1). A group that runs longer than `max_video_group_ms` (2 000 by default) on the
  room clock makes the gateway ask for a keyframe, at most once a
  `keyframe_request_interval_ms`; the group ends when the keyframe comes.
- **Audio**: a group holds the frames whose room-clock time (§7) falls in one window of
  `audio_group_ms` (200 by default). The windows are laid end to end from the track's
  first frame; after a pause longer than a window the next window starts at the next
  frame.

A group's objects travel on one subgroup stream (MOQT §10.3, Subgroup ID 0), written as
the frames arrive, and the stream ends with the group: its END_OF_GROUP bit is set and its
FIN follows the group's last object.

### 5.3 Group IDs

A track's group IDs ascend by one. The first group's ID is the room-clock time, in µs, of
its first frame (§7). A publisher that reconnects, or a new session of the same owner,
therefore continues above every ID the track used before, so a relay never takes a new
group for a copy of an old one (MOQT §7.3). A catalog group's ID follows the same rule,
from the room-clock time it is published.

## 6. The `live` manifest member

An object's manifest carries SPEC §3's members and the member `live`, an object:

| member | type | meaning |
|---|---|---|
| `t` | integer | the frame's room-clock time, µs since the room's `t0` (§7) |
| `rtp` | integer | the frame's RTP timestamp, extended past its 32 bits by the WebRTC stack |
| `rate` | integer | its RTP clock rate: 90 000 for video, 48 000 for Opus |
| `key` | boolean | video only: whether the frame is a keyframe, the first of its group |
| `grant` | object | on the first object of each group only: the grant reference of live-rooms §4.5, `grant_hash`, `capability` (`wai.live.room`), `joule_ceiling_micro` and `funds_ceiling` (`null` for none) |

The receipt's content hash covers the envelope's bytes (jwp-receipts §2.1), so the group
receipt that covers a group's first object signs the grant reference for the whole group:
this is how every object of an ingest names the grant it was carried under (live-rooms
§4.5), at the cost of one reference a group. The manifest's members are written in a fixed
order with no insignificant whitespace (SPEC §2).

A sink that reads `t` reads the gateway's statement of the frame's time, bounded as §7
says; it is not the publisher's.

## 7. Time

### 7.1 The mapping

The gateway's clock gives Unix time; the room clock is that time less the room's `t0`
(live-rooms §3.1). For each media track, the first frame **anchors** the mapping: its
room-clock time is its arrival at the gateway, `A`, and its extended RTP timestamp is
`R0`. A later frame whose timestamp is `R` maps to

    t = A + ⌊(R − R0) · 10^6 / rate⌋ µs.

A frame whose mapped time lies more than `max_skew_ms` (250 by default) from its own
arrival, or whose timestamp is below `R0`, **re-anchors** the mapping at itself: its time
is its arrival, and the anchor is (its arrival, its timestamp). Mapped times never
decrease on a track: a frame that would map before the last one takes the last one's time.

### 7.2 The bound

So **every frame's `t` lies within `max_skew_ms` of its arrival at the gateway.** A frame
arrives after it was captured, by the path's one-way delay, so `t` is no earlier than its
capture less `max_skew_ms`, and no later than its capture plus its one-way delay plus
`max_skew_ms`, both on the gateway's clock. Two frames of one session, of either track, are
placed relative to each other within their arrivals' difference plus twice `max_skew_ms`.
Between re-anchors, the publisher's media clock is followed exactly: `t` advances as the
RTP timestamps do.

The bound is to the gateway's clock. Its error against the room clock, as the room's clock
key keeps it, is the gateway host's clock error, which this revision neither measures nor
states. A gateway counts its re-anchors in its session record (§16).

### 7.3 Across the room

A marker or a command set on the room clock lands on an ingest's media within §7.2's bound.
An event anchored to an object of an ingest track (live-events §3.1) names the object, and
the object's `live.t` gives that object's time; a sink that maps an anchored event takes
`t` from the object, not from the event.

## 8. Receipts

### 8.1 What the gateway signs

When a group ends, the gateway signs its group receipt (jwp-receipts §2.4.1) over every
object of the group and publishes it with the object receipts as group *g* of the track's
receipts track (moq-streaming-format §4.1). Every object receipt is labelled
(jwp-receipts §2.4.2): its figure is the unmetered marker, `0` with class tag 0
([energy-measurement](energy-measurement.md) §2), since no meter backs the gateway; it is
never a figure of zero joules. Its `origin` label is the gateway's. The group receipt is
signed in the labelled form, its class composed from its objects', and chains to the
track's previous group by `parent_receipt_hash`. Its `signer_id` is the gateway's
(jwp-receipts §7.1): origin-bound where the gateway's origin publishes its key log, its
key's `did:key` otherwise.

What the receipts state is the first hop's: that the gateway received these bytes, under
this grant, at these room-clock times. A WebRTC publisher signs nothing, so they are not
the publisher's statement. A publisher that speaks WAI publishes its tracks over MoQ itself
and signs its own receipts.

### 8.2 The chain

A track's `wai_receipt_alias` names its receipt chain (moq-streaming-format §4): the first
eight bytes, little-endian, of BLAKE3 over `wai:ingest-receipt-alias` (ASCII), 0x01, the
session's 16-byte id, the rendered namespace, one zero byte, and the Track Name, reduced
below 2^53. Each session's tracks so sign chains of their own: a publisher that reconnects
starts a new chain, and never forks the old one. The catalog that follows the reconnection
names the new alias.

### 8.3 When the receipts are published

A group's receipts go out after its last object, since the group receipt's root covers
every object of the group. This differs from moq-streaming-format §4.1's rule for a stage
set, where a live group holds one object and its receipt precedes it: an ingest's groups
hold many frames, and a frame is published as it arrives. A subscriber that does not verify
receives each frame at once; one that verifies holds a group until its receipts arrive, at
the group's end. A relay with a door on the room's namespace (jwp-receipts §4.1) holds each
object until its group's receipts arrive, so its `hold` must exceed the longest group,
`max_video_group_ms` plus the time a keyframe takes to come.

## 9. WHEP egress

### 9.1 A session

A player POSTs its offer to the resource of one owner's tracks,
`…/whep/<room id>/<owner>[?layer=<n>]`, the room id and the owner as 64 lowercase
hexadecimal digits each, and `n` the layer, 0 by default (WHEP §4.3). Admission is §3's.
The gateway:

1. subscribes at the relay to the owner's catalog and its receipts track, and reads the
   latest catalog whose receipt verifies (§9.2) under a key it trusts as a first hop.
   Where none arrives within its wait, the request is answered **`409`** with
   `Retry-After` (WHEP §4.3.9): the owner is not live. Where the catalog declares no track
   of the layer, `404` (`layer_unknown`);
2. answers the offer with the codecs of the declared tracks only, so a section of a kind
   the owner does not publish, or of no common codec, is rejected (WHEP §4.3.4). An offer
   none of whose sections is accepted is `422` (`no_common_codec`);
3. subscribes to each accepted kind's media track and its receipts track from the start of
   the latest group: the live part from the Next Object, and a fill of the latest group
   from its first object (MOQT §3.5, §9.20.9). A player so starts at a group's first
   object, a keyframe for video.

A track's end at the relay (PUBLISH_DONE) ends the session, after every frame that verified
is handed on.

### 9.2 Verification

For each group of each track, before any of its frames reaches the player:

1. its receipts object reads as a bundle: otherwise `refused_bundle`;
2. its receipts name this MOQT group, each object receipt its Object ID, and the track's
   `wai_receipt_alias` (moq-streaming-format §4.1, `moqt_mismatch`): otherwise
   `refused_moqt`. The catalog's own chain is not declared anywhere, so a catalog receipt
   is held to its group and Object ID alone;
3. its group receipt verifies under a trusted first hop's key (jwp-receipts §3):
   otherwise `refused_signature`;
4. where the session holds the receipt of the group before it, its `parent_receipt_hash` is
   that receipt's link hash: otherwise `refused_chain`;
5. each object matches its object receipt: its content hash, and its Merkle proof to the
   signed root (jwp-receipts §3): otherwise `refused_inclusion`;
6. each object opens as a `WAI1` envelope of the track's capability with a `live` member,
   and its payload as one frame of its framing (§5.1): otherwise `unopenable`.

What fails is dropped and counted, and never reaches the player. A group whose receipts do
not arrive within `hold_ms` (5 000 by default) of its last object is dropped and counted
(`receipt_missing`); an object a verified group's receipts name that did not arrive is
counted as receipted and not delivered (`object_missing`). Groups are handed on in group
order, their frames in object order. A frame's RTP time is its `live.t` at the codec's
clock rate, and its wallclock is placed from the session's first frame by `t`, so the
tracks of a session keep the timing §7 gave them.

### 9.3 What a WHEP player cannot see

WebRTC carries media, not receipts: a WHEP player receives frames and nothing that proves
where they came from or that they are whole. The egress gateway is the verifier on its
behalf, and its session record (§16) is its account of what it verified, dropped and why. A
**WAI-aware player** gets the receipts itself: it subscribes to the same media tracks and
their receipts tracks over MoQ, through the relay's WebTransport or WebSocket paths, and
verifies each object as §9.2 does, with no gateway between. Both read the same tracks; only
the second holds the proof.

### 9.4 Latency

A frame reaches a WHEP player when its group's receipts have verified, at the group's end:
up to a video group's length after it reached the relay, plus the relay's and the
network's delay. A deployment that wants a shorter path shortens its groups (§5.2), at the
cost of keyframes, or plays over MoQ unverified and verifies afterwards.

## 10. Bounds

A gateway states each bound in its configuration, and refuses what passes it:

| bound | refused with |
|---|---|
| WHIP sessions at once; WHEP sessions at once | `503` with `Retry-After` (RFC 9725 §4.5), `session_limit` |
| media sections in one offer (`max_tracks_per_session`, 2 by default); more than one of a kind | `422` (RFC 9725 §4.4.2), `too_many_tracks`, `one_track_per_kind` |
| the media a WHIP session sends, in kilobits per second over a window | stated per media section in the answer as `b=AS` (RFC 8866 §5.8); a session that passes it over `bitrate_window_ms` is ended, `bitrate_exceeded` |
| the largest frame | dropped and counted |
| the bytes waiting for the relay | the session is ended, `relay_backpressure` |
| proof-of-possession challenges held at once, issued and consumed each (`max_challenges`), and how long one stays good (`challenge_ttl_s`) | the oldest is dropped; a proof over a challenge past its time is refused `pop_wrong_challenge` |

## 11. ICE

A session supports trickle ICE and ICE restarts (RFC 9725 §4.3; WHEP §4.5), with
entity-tags: the `201` carries the ICE session's `ETag`, and a `PATCH` names the ICE
session it is for in `If-Match`. A `PATCH` without `If-Match` is `428`; one whose tag is not
the current one's, or `*`, is `412` (RFC 9110 §13.1.1; RFC 6585 §3).

- **Trickle**: a fragment whose credentials are the current ones, or absent, adds its
  candidates; one the gateway cannot use is discarded. `204`, no `ETag`.
- **Restart**: a fragment with new `ice-ufrag` and `ice-pwd` restarts ICE. The gateway
  applies to its WebRTC stack the client's offer with those credentials and the fragment's
  candidates in place of the old ones, which is the offer an ICE restart would have sent
  (RFC 8839 §4.4.1.1.1), and answers `200` with an `application/trickle-ice-sdpfrag` body:
  its `ice-lite` and `ice-options` attributes, the BUNDLE group, the first media section's
  `m=` line and `mid`, its new credentials, its candidates, `a=end-of-candidates`; and a new
  `ETag`. A restart it cannot apply keeps the ICE session it has and is `422`
  (`ice_restart_refused`, RFC 9725 §4.3.3).

The gateway is an ICE lite agent where every client reaches its address (RFC 9725 §4.4.5),
and a full agent otherwise. A session whose ICE stays disconnected past a grace period is
ended (`ice_disconnected`).

## 12. E-RTMP v2 multitrack (mapping)

This section maps Enhanced RTMP v2 (`v2-2026-01-31-r2`) onto room tracks. This revision
implements none of it.

- **Connection.** The gateway is the RTMP server. In its `connect` `_result` it declares
  `capsEx` with `Reconnect` (0x01) and `Multitrack` (0x02), and `ModEx` (0x04) and
  `TimestampNanoOffset` (0x08) where it reads them, and the codecs of the table below in
  `videoFourCcInfoMap` and `audioFourCcInfoMap` (E-RTMP *Enhancing NetConnection connect
  Command*).
- **Admission.** The `publish` command's stream name is the bearer token of §3.1, and is
  checked as §3.2 and §3.3 check it. A refusal is an `onStatus` with `level` `error`, code
  `NetStream.Publish.BadName`, and the reason of §14 as `description`; then the stream is
  closed.
- **Tracks.** Each (media type, `trackId`) of a multitrack stream is a candidate room track.
  `trackId`s are identifiers that imply no order (E-RTMP *Multitrack Media Message
  Guidelines*), so a video track's layer is the layer of the grant's publish profile whose
  width, height and frame rate the track's `onMetaData` entry states, matched exactly, and
  layer 0 where the profile has one layer or the grant names none. A track that matches no
  layer, a second track of one layer (another codec of the same rendition, as
  `ManyTracksManyCodecs` sends), and an audio track other than `trackId` 0, are not carried
  in this revision, and are counted. A layer is a room track named as §4.1 names layer 0,
  with the layer as its Track Name (live-rooms §6.1, §6.3).
- **Codec FourCCs to the catalog.**

  | FourCC | capability | a frame | the object's payload |
  |---|---|---|---|
  | `avc1` | `wai.video.avc` | a `CodedFrames` body: length-prefixed NAL units | Annex B: each NAL unit after a start code, the SPS and PPS of the `AVCDecoderConfigurationRecord` before each IDR access unit, so a group decodes from its first object |
  | `hvc1` | `wai.video.hevc` | as `avc1` | Annex B, the `HEVCDecoderConfigurationRecord`'s parameter sets before each IRAP access unit |
  | `vvc1` | `wai.video.vvc` | as `avc1` | Annex B, the `VVCDecoderConfigurationRecord`'s parameter sets before each IRAP access unit |
  | `av01` | `wai.video.av1` | one temporal unit of OBUs | §5.1's AV1 framing, the `AV1CodecConfigurationRecord`'s sequence header before each key temporal unit that lacks one |
  | `vp09` | `wai.video.vp9` | one frame | §5.1's IVF framing |
  | `vp08` | `wai.video.vp8` | one frame | §5.1's IVF framing |
  | `Opus` | `wai.audio.opus.packet` | one Opus packet, for one stream (`N` = 1 in the ID header) | the packet; a multistream body (`N` > 1) is not carried |
  | `fLaC` | `wai.audio.flac` | one or more FLAC frames | a FLAC stream: `fLaC`, the sequence start's `STREAMINFO`, the frames |
  | `mp4a` | `wai.audio.aac` | one raw AAC frame | one ADTS frame, its header built from the `AudioSpecificConfig` |

  A FourCC this table does not list (`ac-3`, `ec-3`, `.mp3`) is not carried. The catalog's
  `codec` is written from the configuration record (`avc1.PPCCLL` from the
  `AVCDecoderConfigurationRecord`'s profile, compatibility and level; `av01` from the
  `AV1CodecConfigurationRecord`; `vp09` from the `VPCodecConfigurationRecord`), and its
  `width`, `height`, `samplerate` and `channelConfig` from the configuration record or the
  track's `onMetaData`.
- **Time.** A message's timestamp is milliseconds on the publisher's clock; a video frame's
  presentation time adds its `compositionTimeOffset`, and a `TimestampOffsetNano` ModEx
  adds its nanoseconds. §7's mapping applies with these as the media time and a rate of
  10^6 per µs, anchored at the first message's arrival; a timestamp that wraps its 32 bits
  is extended past them.
- **Groups.** A video group begins at a `KeyFrame` of its track (§5.2); audio windows as
  §5.2.
- **The reconnect request.** Before a gateway shuts down, or to move a publisher to
  another gateway, it sends `onStatus` with code `NetConnection.Connect.ReconnectRequest`,
  `level` `status`, and the new gateway's `tcUrl`, to a client that declared `Reconnect`
  in `capsEx` (E-RTMP *Reconnect Request*). The client reconnects at its next keyframe.
  The new session's first group ID is its first frame's room-clock time (§5.3), above every
  ID the old session used; its receipt chain is new (§8.2), and its catalog names it. A
  gateway that receives a MOQT GOAWAY from its relay (MOQT §9.2) sends the reconnect
  request to its publishers.
- **Latency.** RTMP runs over TCP, so no frame is lost and none reordered; a stall shows
  as frames that arrive late together, which §7 bounds by re-anchoring.

## 13. SRT (mapping)

This section maps SRT 1.5.7 caller-listener connections onto room tracks. This revision
implements none of it.

- **The stream ID and the grant.** A stream ID holds at most 512 characters (SRT stream ID
  guidelines). A grant's fixed fields alone take 315 bytes, 420 characters in base64url,
  and with the room, the owner and the mode beside them a stream ID would pass 512 before
  the grant names a profile, an origin or an attribute. A caller therefore first obtains a
  session ID: it POSTs to the gateway's
  `…/srt` resource with the bearer token of §3.1, admitted as §3 admits WHIP and WHEP, and
  receives a one-time session ID, valid for 60 seconds and invalidated by its first use,
  and the passphrase for the connection. Its stream ID is then
  `#!::r=<room id>/<owner>,m=publish,s=<session ID>` to publish and `m=request` to play,
  in the guidelines' key-value syntax with their standard keys (`r` the resource, `m` the
  mode, `s` the session ID, which the guidelines define for exactly this case: details
  negotiated over a separate connection first). The gateway's listener callback finds the
  session, checks that its grant gives the mode (`m=publish` the publish power, `m=request`
  the subscribe power) for that room and owner, and sets the connection's passphrase, so
  the SRT handshake fails for a caller without it.
- **Refusals.** The listener callback rejects with the SRT rejection codes of the HTTP
  statuses §14 gives: `1401` (`SRT_REJX_UNAUTHORIZED`) for a session or grant that is not
  good, `1403` (`SRT_REJX_FORBIDDEN`) for a power the grant does not give, `1404`
  (`SRT_REJX_NOTFOUND`) for a room the gateway does not hold, `1405`
  (`SRT_REJX_BAD_MODE`) for a mode other than `publish` and `request`, `1406`
  (`SRT_REJX_UNACCEPTABLE`) for a payload it cannot read, `1409` (`SRT_REJX_CONFLICT`) for
  an owner already publishing that kind through another session, and `1402`
  (`SRT_REJX_OVERLOAD`) past a bound of §10.
- **Payload.** SRT carries any payload; in live mode a contribution encoder sends an MPEG
  transport stream (ISO/IEC 13818-1), seven 188-byte packets a datagram. The gateway
  demultiplexes it by its program map table: stream type `0x1B` is `wai.video.avc` (each
  PES packet's access unit, already Annex B), `0x24` `wai.video.hevc`, `0x0F`
  `wai.audio.aac` (one ADTS frame an object). Other stream types are not carried in this
  revision. Each carried elementary stream of a kind is a room track as §4.1 names it; a
  second of one kind is not carried.
- **Time.** A PES packet's presentation time stamp, 33 bits at 90 kHz, extended past its
  33 bits, is the media time of §7's mapping, with a rate of 90 000. SRT delivers each
  packet at its sender's time plus the connection's latency (its timestamp-based packet
  delivery), so frames arrive with that latency added and evenly; §7's bound applies to the
  variation.
- **Latency.** The connection's latency is the larger of the caller's and the listener's
  (`SRTO_LATENCY`; 120 ms unless set). It bounds how long SRT retransmits a lost packet,
  and a retransmission takes at least one round trip, so a gateway sets its listener's
  latency to several round-trip times of the paths it serves. The gateway states its
  latency in its configuration; a grant's profile states none.
- **Groups and receipts** are §5 and §8's.

## 14. Reasons

The refusals of this extension, each with its HTTP status and where it is raised. A reader
of the WHIP and WHEP statuses keeps RFC 9725 §4.1's rule: a client falls back to a
status's generic class for a reason it does not know. Each refusal's body is a problem
details object (RFC 9457) whose `reason` member is the code.

| reason | status | raised at |
|---|---|---|
| `token_missing` | 401 | §3.2 |
| `token_malformed` | 401 | §3.2 step 1 |
| `room_unknown` | 401 | §3.2 step 1 |
| `truncated`, `malformed`, `unknown_version`, `bad_key`, `bad_signature`, `wrong_room`, `grant_invalid`, `grant_too_long`, `caps_exceed_role`, `grant_exceeds_room` | 401 | §3.2 step 2 (live-rooms §4.3, §11) |
| `outside_grant_window` | 401 | §3.2 steps 2 and 3 |
| `grant_revoked` | 401 | §3.2 step 4 |
| `origin_not_granted` | 401 | §3.2 step 5 |
| `grant_reused` | 401 | §3.2 step 6 |
| `pop_missing`, `pop_malformed`, `pop_wrong_grant`, `pop_wrong_key`, `pop_bad_signature`, `pop_wrong_challenge`, `pop_replayed`, `pop_wrong_context` | 401 | §3.2 step 7 (live-rooms §10.5.4, §11) |
| `publish_not_permitted` | 403 | §3.3 |
| `subscribe_not_permitted`, `layer_not_permitted` | 403 | §3.3 |
| `wrong_room` | 403 | §3.3, a WHEP resource of another room |
| `not_the_session_token` | 403 | §3.4 |
| `not_sdp`, `not_sdpfrag` | 415 | an offer that is not `application/sdp`; a `PATCH` that is not a fragment |
| `offer_malformed`, `sdpfrag_malformed` | 400 | the body does not read |
| `section_unsupported`, `direction`, `no_media`, `too_many_tracks`, `one_track_per_kind`, `one_media_stream` | 422 | §10; RFC 9725 §4.2, §4.4.2; WHEP §4.3.6 |
| `no_common_codec`, `offer_refused` | 422 | §3.3, §9.1; RFC 9725 §4.4.3 |
| `if_match_missing` | 428 | §11 |
| `if_match_stale` | 412 | §11 |
| `ice_restart_refused` | 422 | §11 |
| `not_live` | 409, with `Retry-After` | §9.1 |
| `layer_unknown`, `owner_unknown`, `session_unknown` | 404 | §9.1; a session that has ended |
| `body_too_large` | 413 | a request body past the gateway's bound |
| `body_timeout` | 408 | a request body that does not arrive in time |
| `not_found` | 404 | a path that is no resource of the gateway |
| `session_limit` | 503, with `Retry-After` | §10 |
| `relay_unreachable`, `relay_refused`, `media_socket` | 503 | the gateway cannot reach its relay, or bind a socket |

A session that ends states why in its record: `deleted`, `connect_timeout`, `idle`,
`ice_disconnected`, `bitrate_exceeded`, `relay_backpressure`, `relay_lost`, or, for WHEP,
`track_ended`.

## 15. Ingest and capture binding

An ingest is not a capture binding. A WHIP session's tracks are the **live** tier: what the
publisher's encoder sent, receipted by the first hop on arrival. A take
([capture-binding](capture-binding.md)) is the **master** tier: the device's own recording,
committed by the device's key into the room's events while it is captured. The two relate
through the room clock (capture-binding §10): the ingest's objects carry their room-clock
times (§6, §7), and the take's commitments carry theirs.

A WHIP publisher's take can still be committed by the device, if the device speaks
capture binding: it joins the room under a grant that binds it to the principal it captures
for (capture-binding §3.1), records its master, and sends its `take_commit` events on its
own events track over MoQ, beside its WHIP session. Its WHIP media and its master then
share the room clock: a marker set on the live call at room time *t* lands at *t* in the
master (capture-binding §10), within the master's bound there and this extension's §7.2
here. A device that speaks WHIP alone has a live tier and no take; nothing in this
extension makes its live media a take, and a `single-take` claim is never made of an
ingest's tracks.

## 16. Metering

A gateway is a hop of every room session it carries, in the role of an ingest gateway.
When each of its sessions ends it signs a record of what the session carried: per track
and direction the objects and bytes, and the drops by reason; the subscriber sessions
served and their time by layer class; the grant the session ran under, with its ceilings
and the bytes in and out under it; and its own energy, the unmetered marker. These are the
counts of the per-hop session record that WAI's live metering defines, kept in that
record's shape. Until that record is defined, the reference gateway signs them as JSON
(`joule-moq-ingest/session/1`); a later revision has the gateway sign the per-hop record
itself, under the key-log authority live metering gives an ingest gateway's key.

## 17. Status and conformance

| mapping | status | where |
|---|---|---|
| WHIP (§3 to §11) | implemented | `joule-moq-ingest`: admission, the tracks and catalog, framing for H.264, VP8, VP9, AV1 and Opus, groups, time, receipts, ICE, bounds |
| WHEP (§9) | implemented | `joule-moq-ingest`: the catalog read and verified, joining at a group, per-object verification, egress |
| E-RTMP v2 multitrack (§12) | specified | — |
| SRT (§13) | specified | — |

The reference implementation's tests run on loopback: a WHIP publisher of H.264 and Opus
whose frames a relay subscriber receives whole, every receipt verified (by the subscriber
and by the relay's door); a WHEP player that gets the same frames from the group it
joined; VP8, VP9 and AV1 each through WHIP; every refusal of §3 with its status; trickle
ICE and an ICE restart with media across it; and DELETE ending the tracks at the relay. Its
reading of room descriptors and grants is held to the corpus of live-rooms
(`live-conformance/rooms/`): every room id and grant digest there, the registered
profiles' records, and every refusal of a descriptor or a grant but those of an exchange's
link (§3.2 step 2) and of the key log (§3.2 step 4), which admission does not make.

Not implemented in this revision, beside §12 and §13: a WHEP counter-offer (WHEP §4.3.2); a
WHEP player's keyframe request carried upstream (MOQT's `NEW_GROUP_REQUEST`); publishing
into a slot (live-rooms §7); simulcast within one WHIP session; STUN and TURN servers in
`Link` headers (RFC 9725 §4.6); and reading the key log itself (§3.2 step 4 holds the
revocations a deployment gives it).

## 18. Security and privacy considerations

- A token is a bearer capability (live-rooms §13): whoever holds it may publish as its
  principal through a gateway. Its window is fourteen days at most, a single-use grant opens
  one session, and a gateway's TLS front keeps it off the wire in clear (RFC 9725 §5).
- The first hop's receipts attest what the gateway received, not what the publisher's
  camera saw: a gateway that lies signs its lie with its own key, and a player that trusts
  its receipts trusts that key. A deployment lists the gateways it trusts as first hops,
  and a player that needs more verifies a take (§15).
- The receipts are signed after a group ends: a relay that withholds a group's receipts
  holds back its frames from every verifying sink, which notices the gap by the chain (§9.2
  step 4), not by a missing frame alone.
- An egress gateway that verifies hands a player only frames that verified, so a tampered
  relay cannot reach a WHEP player with altered media; it can still withhold media, which
  the session record counts.
- A session's URL is unguessable (§3.4), and its resource answers only its own token, so a
  third party cannot end or restart another's session.
- Every bound of §10 limits what one session, and one grant, can make a gateway hold.