open-standards/wai/extensions/live-ingest.md

WAI Extension: Live Ingest and Egress

Status: Draft. Binds the contribution and playback protocols of standard encoders and players to the tracks of a room (live-rooms): WHIP (RFC 9725) into a room's tracks, WHEP (draft-ietf-wish-whep-05) out of them, and, as mappings only in this revision, Enhanced RTMP v2 multitrack and SRT. An ingest gateway admits a publisher by its room grant, takes its media frame by frame, packages each frame as one WAI envelope on a MoQ track (moq-streaming-format), maps its media time onto the room clock with a stated bound, and signs every group's receipts as the first hop (jwp-receipts). An egress gateway verifies every object against its receipt before it hands the frame to a player. No new transport and no new token format: WAI defines none by charter. The reference implementation of WHIP and WHEP is joule-moq-ingest in joule-code/joule-code-rs/crates/, on the joule-moq codec and the joule-moq-relay relay. Keywords MUST, MUST NOT, SHOULD, MAY are RFC 2119/8174.

Texts read for this revision: RFC 9725 (WHIP); draft-ietf-wish-whep-05 (WHEP), an Internet-Draft not past Working Group Last Call; RFC 6750 §2.1 and §3; RFC 8840 (the trickle ICE fragment); RFC 9110 §8.8.3 and §13.1.1; RFC 6585 §3; RFC 9457; RFC 6184 (H.264 over RTP); RFC 7741 (VP8); RFC 9628 (VP9); the AV1 RTP payload format v1.0.0; RFC 7587 and RFC 6716 (Opus); draft-ietf-moq-transport-22 (MOQT) and draft-ietf-moq-msf-01 (MSF); Enhanced RTMP v2, document version v2-2026-01-31-r2 (E-RTMP); and SRT 1.5.7, its stream ID guidelines and its rejection codes. A section number of one of those texts here is that revision's.

1. Scope

In scope: how a WHIP request is admitted under a room grant (§3); the tracks an ingest publishes and its catalog (§4); one frame as one object, the payload framing of each codec, and how frames form groups (§5); the live member of an object's manifest (§6); the mapping from RTP time to the room clock and its bound (§7); the receipts the gateway signs (§8); WHEP egress and what it verifies (§9); the bounds a gateway holds a session to (§10); trickle ICE and ICE restarts (§11); the mappings of E-RTMP v2 multitrack (§12) and SRT (§13); the refusals and their statuses (§14); and how an ingest relates to capture binding and to metering (§15, §16).

Out of scope: a transport (WebRTC, MoQ, RTMP and SRT are the transports); how a principal obtains a grant (live-rooms §1); transcoding: a gateway carries the frames the publisher sent, as the publisher encoded them; slots and their takeover and failover (live-rooms §7), which an ingest into a slot follows and this revision does not map; and simulcast or SVC within one WHIP session (RFC 9725 §4.8).

Why an extension of its own. moq-streaming-format says what a WAI track is on MoQ. This text binds four other protocols onto those tracks, under the rooms and grants of the wai.live family; written into that extension it would make a MoQ format describe RTMP and SRT. It sits beside live-rooms, live-events and capture-binding, and changes none of them.

Relationship to the core spec: no record and no capability is registered here. The payload capabilities a live track carries are SPEC §5's, among them wai.video.vp8 and wai.audio.opus.packet, registered for this binding. The manifest member live (§6) is defined here; a reader that does not know it ignores it (SPEC §3).

2. Terms

• Room, room clock, principal, grant, room authority: as in live-events §2 and live-rooms §2.
• Gateway: an endpoint that speaks one of the protocols of this text on one side and MOQT to a relay on the other. An ingest gateway takes media from a publisher; an egress gateway hands media to a player. One process may be both.
• First hop: the gateway an ingest's media reaches first. It is the MoQ publisher of the tracks (MOQT §7.6) and the signer of their receipts (§8).
• Owner: the principal whose tracks they are: the grant's principal (live-rooms §6.1).
• Frame: one access unit of video, or one packet of audio, as the RTP payload format of its codec delivers it whole (§5.1).
• µs: microseconds. Integers are little-endian where bytes are written.

3. Admission

3.1 The bearer token

A WHIP or WHEP request is authenticated by an HTTP Authorization header with a bearer token (RFC 9725 §4.7.1; WHEP §4.9.1; RFC 6750 §2.1). The token is a room grant: the WRG1 record of live-rooms §4.1, whole, in the base64url alphabet without padding (RFC 4648 §5). The room it names is the room the request is for. WAI defines no second token format; a deployment that issues tokens of another format maps them onto a grant as live-rooms §10 says, and admits by the grant.

A grant is a public record, so presenting it proves nothing about who presents it. Every request that carries a grant also carries a proof of possession of the grant's key (live-rooms §10.5): a WRP1 record, in base64url, in the WAI-Room-Proof field, over a challenge the gateway issued and the request's context (§3.2 step 7). The gateway issues single-use challenges in the WAI-Room-Challenge field: on a 401 that refuses for want of a good proof, and on every response to a request that carried a grant, the one a 201 carries being scoped to the session it opened (live-rooms §10.5.3).

3.2 The checks

A gateway holds the signed descriptor of every room it serves. It admits a token by these checks, in order, and refuses at the first that fails with the reason it names:

1. The token decodes, and the room its grant names (bytes 5 to 36) is a room the gateway holds: otherwise room_unknown, or the reading refusal of live-rooms §4.3 step 1 (truncated, malformed, unknown_version).
2. live-rooms §4.3 steps 1, 2 and 4 to 8, against that room's descriptor: the record reads; its signature verifies under the room's authority; it names the room; its fields keep §4.3's rules; it is at most fourteen days long; it is within its role and within the room's ceiling. Each refusal is §4.3's reason. Step 3, an exchanged grant's link to the grant it replaces, needs the earlier grant, which a token does not bring: the gateway checks instead that the room clock has reached the exchanged grant's effective tick.
3. The grant is good at the gateway's clock (live-rooms §4.3): otherwise outside_grant_window.
4. Neither its digest nor its viewer's tag is among the revocations the gateway holds for the room's authority's key log (live-rooms §5): otherwise grant_revoked.
5. Where the grant lists origins, the gateway's own origin is one of them: otherwise origin_not_granted.
6. Where the grant's once is not zero, no session of this gateway was opened under it: otherwise grant_reused. The nonce is spent when a session opens.
7. The request's proof passes live-rooms §10.5.4 for the grant, as this gateway reads it (its digest and its principal), with the challenge it holds for the request's scope (the endpoint, or the session its URL names), the challenges it consumed before, and the context METHOD //authority/path of the request it received: otherwise that section's reason (pop_missing, pop_malformed, pop_wrong_grant, pop_wrong_key, pop_bad_signature, pop_wrong_challenge, pop_replayed, pop_wrong_context). A proof that passes consumes its challenge: it is good for one request.

A token missing or not a bearer token is answered 401 with WWW-Authenticate: Bearer realm=… and no error (RFC 6750 §3.1). A token refused by these checks is answered 401 with error="invalid_token" and the reason as error_description.

3.3 The powers

The grant must give what the request asks for (live-rooms §3.3). A grant that passes §3.2 and lacks it is answered 403 with WWW-Authenticate: Bearer error="insufficient_scope" (RFC 6750 §3.1):

• WHIP: the grant's publish bit for each media kind the offer carries, audio (1) and video (2): otherwise publish_not_permitted. A screen is published as media kind screen (3) only through a mapping that names it, which this revision does not.
• WHEP: subscribe, and a max_layer at or above the layer asked for: otherwise subscribe_not_permitted or layer_not_permitted. The room in the resource's path is the grant's: otherwise wrong_room.

Where the grant names a publish profile for a media kind (live-rooms §4.1 profiles), the session accepts only the codecs of that profile's codecs that the gateway carries, by RFC 6381 family (avc1, vp8, vp09, av01, opus), and states the profile's layer-0 max_kbps as the media section's bandwidth (§10).

3.4 The session's resource

Every request to a session's resource (PATCH, DELETE, GET, HEAD) carries the token the session was opened with, and a proof of possession over a challenge scoped to that session (§3.2 step 7); another token, however good, is answered 403 (not_the_session_token). A session's URL carries at least 128 random bits (RFC 9725 §5).

4. Tracks and the catalog

4.1 Names

A WHIP session publishes, for each media kind its offer carries, one media track and its receipts track, and, for the owner, one catalog track and its receipts track:

track	Track Namespace (live-rooms §6.1)	Track Name
media	(room id, owner, audio or video)	0
its receipts	the media track's	0_receipts
catalog	(room id, owner)	catalog
its receipts	the catalog's	catalog_receipts

A WHIP session sends one encoding of each kind, so its media tracks are layer 0 (live-rooms §6.3, single). A receipts track is named as live-rooms §6.1 names one, the media track's name and _receipts, and is of packaging wai-receipts. The catalog is the owner's MSF catalog (MSF §5): the namespace prefix (room id, owner) reaches it and every track it declares.

4.2 Publication

The gateway publishes every track at the relay with PUBLISH (MOQT §9.8) before it answers the WHIP request, so a subscriber can subscribe from the first group. The tracks' default publisher priority is 0 for audio and the catalog and 1 for video (MOQT §5.1.1): a subscriber that falls behind keeps the sound. A receipts track's priority is its media track's (moq-streaming-format §4.1). No track carries a delivery timeout.

When the session ends the gateway ends each open group, publishes its receipts, waits for every stream to be acknowledged, and sends PUBLISH_DONE with status TRACK_ENDED on each track (MOQT §9.9). The relay then ends its subscriptions: a subscriber sees the track end.

4.3 The catalog

The catalog is RFC 8785 canonical JSON, {"tracks": [...], "version": 1}, one object a group. Each media track's entry carries MSF's namespace (the MOQT §8.8 rendering of its namespace, live-rooms §6.2), name, packaging "wai", isLive true, role, renderGroup 1, and, where the bitstream states them, codec (RFC 6381), width and height, or samplerate and channelConfig; and moq-streaming-format §4's wai_capabilities (the one capability its objects dispatch to), wai_receipts true, wai_receipt_alias (§8.2) and wai_receipts_track. Each is followed by its receipts track's entry.

The gateway publishes the catalog once every media track has carried its first frame, or a second after the first did, whichever comes first, and publishes it again, whole, in a new group, when a track's codec or picture size changes. Its group IDs follow §5.3's rule. The catalog is signed as the media is (§8): each catalog group is one object, receipted on catalog_receipts. A relay that checks every track of a room at its door forwards it, and a player verifies what the catalog declares before it relies on it (§9.1).

5. Objects and groups

5.1 One frame, one object

Each frame is one MOQT object, and each object is exactly one WAI1 envelope (moq-streaming-format §3). The frame is what the codec's RTP payload format reassembles, whole:

codec	RTP payload format	capability (SPEC §5)	the object's payload
H.264	RFC 6184	wai.video.avc	one access unit, Annex B byte stream: each NAL unit after a four-byte start code
VP8	RFC 7741	wai.video.vp8	one frame in the IVF frame container, FourCC VP80: the 32-byte file header (the frame's size, a time base of 1/90 000, one frame), the 12-byte frame header (its length, its time in 90 kHz ticks), the frame
VP9	RFC 9628	wai.video.vp9	as VP8, FourCC VP90
AV1	AV1 RTP payload format v1.0.0	wai.video.av1	one temporal unit in SPEC §5's AV1 framing: n = 1, the largest frame's height and width from the sequence header, a frame rate of 90 000 over the ticks since the track's previous frame, then the temporal unit's length and the unit: a temporal delimiter OBU (which RTP does not carry, and which the gateway restores), then every OBU with its size field
Opus	RFC 7587	wai.audio.opus.packet	one Opus packet (RFC 6716 §3)

A frame of a codec this table does not list is not carried. A frame larger than the gateway's bound is dropped and counted (§10).

A live track's objects are one stream. A frame is a unit of its codec's stream, not a picture or a sound decodable alone. The objects of a media track, in object order from a group's first object and in group order, are consecutive units of one stream of the codec: a sink decodes them in that order with one decoder state, from a group's first object, and resets that state where a group follows one it did not receive. The capability of every object says how its payload is read; this rule says how consecutive payloads relate. A sink that joins a track joins at a group's first object.

5.2 Groups

• Video: a group begins at each keyframe (an IDR access unit for H.264; a key frame for VP8 and VP9; a temporal unit with a sequence header for AV1) and holds it and every frame up to the next. A frame that arrives while no group is open (before the first keyframe) is dropped, counted, and answered with a keyframe request (a PLI, RFC 4585 §6.3.1). A group that runs longer than max_video_group_ms (2 000 by default) on the room clock makes the gateway ask for a keyframe, at most once a keyframe_request_interval_ms; the group ends when the keyframe comes.
• Audio: a group holds the frames whose room-clock time (§7) falls in one window of audio_group_ms (200 by default). The windows are laid end to end from the track's first frame; after a pause longer than a window the next window starts at the next frame.

A group's objects travel on one subgroup stream (MOQT §10.3, Subgroup ID 0), written as the frames arrive, and the stream ends with the group: its END_OF_GROUP bit is set and its FIN follows the group's last object.

5.3 Group IDs

A track's group IDs ascend by one. The first group's ID is the room-clock time, in µs, of its first frame (§7). A publisher that reconnects, or a new session of the same owner, therefore continues above every ID the track used before, so a relay never takes a new group for a copy of an old one (MOQT §7.3). A catalog group's ID follows the same rule, from the room-clock time it is published.

6. The live manifest member

An object's manifest carries SPEC §3's members and the member live, an object:

member	type	meaning
t	integer	the frame's room-clock time, µs since the room's t0 (§7)
rtp	integer	the frame's RTP timestamp, extended past its 32 bits by the WebRTC stack
rate	integer	its RTP clock rate: 90 000 for video, 48 000 for Opus
key	boolean	video only: whether the frame is a keyframe, the first of its group
grant	object	on the first object of each group only: the grant reference of live-rooms §4.5, grant_hash, capability (wai.live.room), joule_ceiling_micro and funds_ceiling (null for none)

The receipt's content hash covers the envelope's bytes (jwp-receipts §2.1), so the group receipt that covers a group's first object signs the grant reference for the whole group: this is how every object of an ingest names the grant it was carried under (live-rooms §4.5), at the cost of one reference a group. The manifest's members are written in a fixed order with no insignificant whitespace (SPEC §2).

A sink that reads t reads the gateway's statement of the frame's time, bounded as §7 says; it is not the publisher's.

7. Time

7.1 The mapping

The gateway's clock gives Unix time; the room clock is that time less the room's t0 (live-rooms §3.1). For each media track, the first frame anchors the mapping: its room-clock time is its arrival at the gateway, A, and its extended RTP timestamp is R0. A later frame whose timestamp is R maps to

t = A + ⌊(R − R0) · 10^6 / rate⌋ µs.

A frame whose mapped time lies more than max_skew_ms (250 by default) from its own arrival, or whose timestamp is below R0, re-anchors the mapping at itself: its time is its arrival, and the anchor is (its arrival, its timestamp). Mapped times never decrease on a track: a frame that would map before the last one takes the last one's time.

7.2 The bound

So every frame's t lies within max_skew_ms of its arrival at the gateway. A frame arrives after it was captured, by the path's one-way delay, so t is no earlier than its capture less max_skew_ms, and no later than its capture plus its one-way delay plus max_skew_ms, both on the gateway's clock. Two frames of one session, of either track, are placed relative to each other within their arrivals' difference plus twice max_skew_ms. Between re-anchors, the publisher's media clock is followed exactly: t advances as the RTP timestamps do.

The bound is to the gateway's clock. Its error against the room clock, as the room's clock key keeps it, is the gateway host's clock error, which this revision neither measures nor states. A gateway counts its re-anchors in its session record (§16).

7.3 Across the room

A marker or a command set on the room clock lands on an ingest's media within §7.2's bound. An event anchored to an object of an ingest track (live-events §3.1) names the object, and the object's live.t gives that object's time; a sink that maps an anchored event takes t from the object, not from the event.

8. Receipts

8.1 What the gateway signs

When a group ends, the gateway signs its group receipt (jwp-receipts §2.4.1) over every object of the group and publishes it with the object receipts as group g of the track's receipts track (moq-streaming-format §4.1). Every object receipt is labelled (jwp-receipts §2.4.2): its figure is the unmetered marker, 0 with class tag 0 (energy-measurement §2), since no meter backs the gateway; it is never a figure of zero joules. Its origin label is the gateway's. The group receipt is signed in the labelled form, its class composed from its objects', and chains to the track's previous group by parent_receipt_hash. Its signer_id is the gateway's (jwp-receipts §7.1): origin-bound where the gateway's origin publishes its key log, its key's did:key otherwise.

What the receipts state is the first hop's: that the gateway received these bytes, under this grant, at these room-clock times. A WebRTC publisher signs nothing, so they are not the publisher's statement. A publisher that speaks WAI publishes its tracks over MoQ itself and signs its own receipts.

8.2 The chain

A track's wai_receipt_alias names its receipt chain (moq-streaming-format §4): the first eight bytes, little-endian, of BLAKE3 over wai:ingest-receipt-alias (ASCII), 0x01, the session's 16-byte id, the rendered namespace, one zero byte, and the Track Name, reduced below 2^53. Each session's tracks so sign chains of their own: a publisher that reconnects starts a new chain, and never forks the old one. The catalog that follows the reconnection names the new alias.

8.3 When the receipts are published

A group's receipts go out after its last object, since the group receipt's root covers every object of the group. This differs from moq-streaming-format §4.1's rule for a stage set, where a live group holds one object and its receipt precedes it: an ingest's groups hold many frames, and a frame is published as it arrives. A subscriber that does not verify receives each frame at once; one that verifies holds a group until its receipts arrive, at the group's end. A relay with a door on the room's namespace (jwp-receipts §4.1) holds each object until its group's receipts arrive, so its hold must exceed the longest group, max_video_group_ms plus the time a keyframe takes to come.

9. WHEP egress

9.1 A session

A player POSTs its offer to the resource of one owner's tracks, …/whep/<room id>/<owner>[?layer=<n>], the room id and the owner as 64 lowercase hexadecimal digits each, and n the layer, 0 by default (WHEP §4.3). Admission is §3's. The gateway:

1. subscribes at the relay to the owner's catalog and its receipts track, and reads the latest catalog whose receipt verifies (§9.2) under a key it trusts as a first hop. Where none arrives within its wait, the request is answered 409 with Retry-After (WHEP §4.3.9): the owner is not live. Where the catalog declares no track of the layer, 404 (layer_unknown);
2. answers the offer with the codecs of the declared tracks only, so a section of a kind the owner does not publish, or of no common codec, is rejected (WHEP §4.3.4). An offer none of whose sections is accepted is 422 (no_common_codec);
3. subscribes to each accepted kind's media track and its receipts track from the start of the latest group: the live part from the Next Object, and a fill of the latest group from its first object (MOQT §3.5, §9.20.9). A player so starts at a group's first object, a keyframe for video.

A track's end at the relay (PUBLISH_DONE) ends the session, after every frame that verified is handed on.

9.2 Verification

For each group of each track, before any of its frames reaches the player:

1. its receipts object reads as a bundle: otherwise refused_bundle;
2. its receipts name this MOQT group, each object receipt its Object ID, and the track's wai_receipt_alias (moq-streaming-format §4.1, moqt_mismatch): otherwise refused_moqt. The catalog's own chain is not declared anywhere, so a catalog receipt is held to its group and Object ID alone;
3. its group receipt verifies under a trusted first hop's key (jwp-receipts §3): otherwise refused_signature;
4. where the session holds the receipt of the group before it, its parent_receipt_hash is that receipt's link hash: otherwise refused_chain;
5. each object matches its object receipt: its content hash, and its Merkle proof to the signed root (jwp-receipts §3): otherwise refused_inclusion;
6. each object opens as a WAI1 envelope of the track's capability with a live member, and its payload as one frame of its framing (§5.1): otherwise unopenable.

What fails is dropped and counted, and never reaches the player. A group whose receipts do not arrive within hold_ms (5 000 by default) of its last object is dropped and counted (receipt_missing); an object a verified group's receipts name that did not arrive is counted as receipted and not delivered (object_missing). Groups are handed on in group order, their frames in object order. A frame's RTP time is its live.t at the codec's clock rate, and its wallclock is placed from the session's first frame by t, so the tracks of a session keep the timing §7 gave them.

9.3 What a WHEP player cannot see

WebRTC carries media, not receipts: a WHEP player receives frames and nothing that proves where they came from or that they are whole. The egress gateway is the verifier on its behalf, and its session record (§16) is its account of what it verified, dropped and why. A WAI-aware player gets the receipts itself: it subscribes to the same media tracks and their receipts tracks over MoQ, through the relay's WebTransport or WebSocket paths, and verifies each object as §9.2 does, with no gateway between. Both read the same tracks; only the second holds the proof.

9.4 Latency

A frame reaches a WHEP player when its group's receipts have verified, at the group's end: up to a video group's length after it reached the relay, plus the relay's and the network's delay. A deployment that wants a shorter path shortens its groups (§5.2), at the cost of keyframes, or plays over MoQ unverified and verifies afterwards.

10. Bounds

A gateway states each bound in its configuration, and refuses what passes it:

bound	refused with
WHIP sessions at once; WHEP sessions at once	503 with Retry-After (RFC 9725 §4.5), session_limit
media sections in one offer (max_tracks_per_session, 2 by default); more than one of a kind	422 (RFC 9725 §4.4.2), too_many_tracks, one_track_per_kind
the media a WHIP session sends, in kilobits per second over a window	stated per media section in the answer as b=AS (RFC 8866 §5.8); a session that passes it over bitrate_window_ms is ended, bitrate_exceeded
the largest frame	dropped and counted
the bytes waiting for the relay	the session is ended, relay_backpressure
proof-of-possession challenges held at once, issued and consumed each (max_challenges), and how long one stays good (challenge_ttl_s)	the oldest is dropped; a proof over a challenge past its time is refused pop_wrong_challenge

11. ICE

A session supports trickle ICE and ICE restarts (RFC 9725 §4.3; WHEP §4.5), with entity-tags: the 201 carries the ICE session's ETag, and a PATCH names the ICE session it is for in If-Match. A PATCH without If-Match is 428; one whose tag is not the current one's, or *, is 412 (RFC 9110 §13.1.1; RFC 6585 §3).

• Trickle: a fragment whose credentials are the current ones, or absent, adds its candidates; one the gateway cannot use is discarded. 204, no ETag.
• Restart: a fragment with new ice-ufrag and ice-pwd restarts ICE. The gateway applies to its WebRTC stack the client's offer with those credentials and the fragment's candidates in place of the old ones, which is the offer an ICE restart would have sent (RFC 8839 §4.4.1.1.1), and answers 200 with an application/trickle-ice-sdpfrag body: its ice-lite and ice-options attributes, the BUNDLE group, the first media section's m= line and mid, its new credentials, its candidates, a=end-of-candidates; and a new ETag. A restart it cannot apply keeps the ICE session it has and is 422 (ice_restart_refused, RFC 9725 §4.3.3).

The gateway is an ICE lite agent where every client reaches its address (RFC 9725 §4.4.5), and a full agent otherwise. A session whose ICE stays disconnected past a grace period is ended (ice_disconnected).

12. E-RTMP v2 multitrack (mapping)

This section maps Enhanced RTMP v2 (v2-2026-01-31-r2) onto room tracks. This revision implements none of it.

• Connection. The gateway is the RTMP server. In its connect _result it declares capsEx with Reconnect (0x01) and Multitrack (0x02), and ModEx (0x04) and TimestampNanoOffset (0x08) where it reads them, and the codecs of the table below in videoFourCcInfoMap and audioFourCcInfoMap (E-RTMP Enhancing NetConnection connect Command).

• Admission. The publish command's stream name is the bearer token of §3.1, and is checked as §3.2 and §3.3 check it. A refusal is an onStatus with level error, code NetStream.Publish.BadName, and the reason of §14 as description; then the stream is closed.

• Tracks. Each (media type, trackId) of a multitrack stream is a candidate room track. trackIds are identifiers that imply no order (E-RTMP Multitrack Media Message Guidelines), so a video track's layer is the layer of the grant's publish profile whose width, height and frame rate the track's onMetaData entry states, matched exactly, and layer 0 where the profile has one layer or the grant names none. A track that matches no layer, a second track of one layer (another codec of the same rendition, as ManyTracksManyCodecs sends), and an audio track other than trackId 0, are not carried in this revision, and are counted. A layer is a room track named as §4.1 names layer 0, with the layer as its Track Name (live-rooms §6.1, §6.3).

• Codec FourCCs to the catalog.

  FourCC	capability	a frame	the object's payload
  avc1	wai.video.avc	a CodedFrames body: length-prefixed NAL units	Annex B: each NAL unit after a start code, the SPS and PPS of the AVCDecoderConfigurationRecord before each IDR access unit, so a group decodes from its first object
  hvc1	wai.video.hevc	as avc1	Annex B, the HEVCDecoderConfigurationRecord's parameter sets before each IRAP access unit
  vvc1	wai.video.vvc	as avc1	Annex B, the VVCDecoderConfigurationRecord's parameter sets before each IRAP access unit
  av01	wai.video.av1	one temporal unit of OBUs	§5.1's AV1 framing, the AV1CodecConfigurationRecord's sequence header before each key temporal unit that lacks one
  vp09	wai.video.vp9	one frame	§5.1's IVF framing
  vp08	wai.video.vp8	one frame	§5.1's IVF framing
  Opus	wai.audio.opus.packet	one Opus packet, for one stream (N = 1 in the ID header)	the packet; a multistream body (N > 1) is not carried
  fLaC	wai.audio.flac	one or more FLAC frames	a FLAC stream: fLaC, the sequence start's STREAMINFO, the frames
  mp4a	wai.audio.aac	one raw AAC frame	one ADTS frame, its header built from the AudioSpecificConfig

  A FourCC this table does not list (ac-3, ec-3, .mp3) is not carried. The catalog's codec is written from the configuration record (avc1.PPCCLL from the AVCDecoderConfigurationRecord's profile, compatibility and level; av01 from the AV1CodecConfigurationRecord; vp09 from the VPCodecConfigurationRecord), and its width, height, samplerate and channelConfig from the configuration record or the track's onMetaData.

• Time. A message's timestamp is milliseconds on the publisher's clock; a video frame's presentation time adds its compositionTimeOffset, and a TimestampOffsetNano ModEx adds its nanoseconds. §7's mapping applies with these as the media time and a rate of 10^6 per µs, anchored at the first message's arrival; a timestamp that wraps its 32 bits is extended past them.

• Groups. A video group begins at a KeyFrame of its track (§5.2); audio windows as §5.2.

• The reconnect request. Before a gateway shuts down, or to move a publisher to another gateway, it sends onStatus with code NetConnection.Connect.ReconnectRequest, level status, and the new gateway's tcUrl, to a client that declared Reconnect in capsEx (E-RTMP Reconnect Request). The client reconnects at its next keyframe. The new session's first group ID is its first frame's room-clock time (§5.3), above every ID the old session used; its receipt chain is new (§8.2), and its catalog names it. A gateway that receives a MOQT GOAWAY from its relay (MOQT §9.2) sends the reconnect request to its publishers.

• Latency. RTMP runs over TCP, so no frame is lost and none reordered; a stall shows as frames that arrive late together, which §7 bounds by re-anchoring.

13. SRT (mapping)

This section maps SRT 1.5.7 caller-listener connections onto room tracks. This revision implements none of it.

• The stream ID and the grant. A stream ID holds at most 512 characters (SRT stream ID guidelines). A grant's fixed fields alone take 315 bytes, 420 characters in base64url, and with the room, the owner and the mode beside them a stream ID would pass 512 before the grant names a profile, an origin or an attribute. A caller therefore first obtains a session ID: it POSTs to the gateway's …/srt resource with the bearer token of §3.1, admitted as §3 admits WHIP and WHEP, and receives a one-time session ID, valid for 60 seconds and invalidated by its first use, and the passphrase for the connection. Its stream ID is then #!::r=<room id>/<owner>,m=publish,s=<session ID> to publish and m=request to play, in the guidelines' key-value syntax with their standard keys (r the resource, m the mode, s the session ID, which the guidelines define for exactly this case: details negotiated over a separate connection first). The gateway's listener callback finds the session, checks that its grant gives the mode (m=publish the publish power, m=request the subscribe power) for that room and owner, and sets the connection's passphrase, so the SRT handshake fails for a caller without it.
• Refusals. The listener callback rejects with the SRT rejection codes of the HTTP statuses §14 gives: 1401 (SRT_REJX_UNAUTHORIZED) for a session or grant that is not good, 1403 (SRT_REJX_FORBIDDEN) for a power the grant does not give, 1404 (SRT_REJX_NOTFOUND) for a room the gateway does not hold, 1405 (SRT_REJX_BAD_MODE) for a mode other than publish and request, 1406 (SRT_REJX_UNACCEPTABLE) for a payload it cannot read, 1409 (SRT_REJX_CONFLICT) for an owner already publishing that kind through another session, and 1402 (SRT_REJX_OVERLOAD) past a bound of §10.
• Payload. SRT carries any payload; in live mode a contribution encoder sends an MPEG transport stream (ISO/IEC 13818-1), seven 188-byte packets a datagram. The gateway demultiplexes it by its program map table: stream type 0x1B is wai.video.avc (each PES packet's access unit, already Annex B), 0x24 wai.video.hevc, 0x0F wai.audio.aac (one ADTS frame an object). Other stream types are not carried in this revision. Each carried elementary stream of a kind is a room track as §4.1 names it; a second of one kind is not carried.
• Time. A PES packet's presentation time stamp, 33 bits at 90 kHz, extended past its 33 bits, is the media time of §7's mapping, with a rate of 90 000. SRT delivers each packet at its sender's time plus the connection's latency (its timestamp-based packet delivery), so frames arrive with that latency added and evenly; §7's bound applies to the variation.
• Latency. The connection's latency is the larger of the caller's and the listener's (SRTO_LATENCY; 120 ms unless set). It bounds how long SRT retransmits a lost packet, and a retransmission takes at least one round trip, so a gateway sets its listener's latency to several round-trip times of the paths it serves. The gateway states its latency in its configuration; a grant's profile states none.
• Groups and receipts are §5 and §8's.

14. Reasons

The refusals of this extension, each with its HTTP status and where it is raised. A reader of the WHIP and WHEP statuses keeps RFC 9725 §4.1's rule: a client falls back to a status's generic class for a reason it does not know. Each refusal's body is a problem details object (RFC 9457) whose reason member is the code.

reason	status	raised at
token_missing	401	§3.2
token_malformed	401	§3.2 step 1
room_unknown	401	§3.2 step 1
truncated, malformed, unknown_version, bad_key, bad_signature, wrong_room, grant_invalid, grant_too_long, caps_exceed_role, grant_exceeds_room	401	§3.2 step 2 (live-rooms §4.3, §11)
outside_grant_window	401	§3.2 steps 2 and 3
grant_revoked	401	§3.2 step 4
origin_not_granted	401	§3.2 step 5
grant_reused	401	§3.2 step 6
pop_missing, pop_malformed, pop_wrong_grant, pop_wrong_key, pop_bad_signature, pop_wrong_challenge, pop_replayed, pop_wrong_context	401	§3.2 step 7 (live-rooms §10.5.4, §11)
publish_not_permitted	403	§3.3
subscribe_not_permitted, layer_not_permitted	403	§3.3
wrong_room	403	§3.3, a WHEP resource of another room
not_the_session_token	403	§3.4
not_sdp, not_sdpfrag	415	an offer that is not application/sdp; a PATCH that is not a fragment
offer_malformed, sdpfrag_malformed	400	the body does not read
section_unsupported, direction, no_media, too_many_tracks, one_track_per_kind, one_media_stream	422	§10; RFC 9725 §4.2, §4.4.2; WHEP §4.3.6
no_common_codec, offer_refused	422	§3.3, §9.1; RFC 9725 §4.4.3
if_match_missing	428	§11
if_match_stale	412	§11
ice_restart_refused	422	§11
not_live	409, with Retry-After	§9.1
layer_unknown, owner_unknown, session_unknown	404	§9.1; a session that has ended
body_too_large	413	a request body past the gateway's bound
body_timeout	408	a request body that does not arrive in time
not_found	404	a path that is no resource of the gateway
session_limit	503, with Retry-After	§10
relay_unreachable, relay_refused, media_socket	503	the gateway cannot reach its relay, or bind a socket

A session that ends states why in its record: deleted, connect_timeout, idle, ice_disconnected, bitrate_exceeded, relay_backpressure, relay_lost, or, for WHEP, track_ended.

15. Ingest and capture binding

An ingest is not a capture binding. A WHIP session's tracks are the live tier: what the publisher's encoder sent, receipted by the first hop on arrival. A take (capture-binding) is the master tier: the device's own recording, committed by the device's key into the room's events while it is captured. The two relate through the room clock (capture-binding §10): the ingest's objects carry their room-clock times (§6, §7), and the take's commitments carry theirs.

A WHIP publisher's take can still be committed by the device, if the device speaks capture binding: it joins the room under a grant that binds it to the principal it captures for (capture-binding §3.1), records its master, and sends its take_commit events on its own events track over MoQ, beside its WHIP session. Its WHIP media and its master then share the room clock: a marker set on the live call at room time t lands at t in the master (capture-binding §10), within the master's bound there and this extension's §7.2 here. A device that speaks WHIP alone has a live tier and no take; nothing in this extension makes its live media a take, and a single-take claim is never made of an ingest's tracks.

16. Metering

A gateway is a hop of every room session it carries, in the role of an ingest gateway. When each of its sessions ends it signs a record of what the session carried: per track and direction the objects and bytes, and the drops by reason; the subscriber sessions served and their time by layer class; the grant the session ran under, with its ceilings and the bytes in and out under it; and its own energy, the unmetered marker. These are the counts of the per-hop session record that WAI's live metering defines, kept in that record's shape. Until that record is defined, the reference gateway signs them as JSON (joule-moq-ingest/session/1); a later revision has the gateway sign the per-hop record itself, under the key-log authority live metering gives an ingest gateway's key.

17. Status and conformance

mapping	status	where
WHIP (§3 to §11)	implemented	joule-moq-ingest: admission, the tracks and catalog, framing for H.264, VP8, VP9, AV1 and Opus, groups, time, receipts, ICE, bounds
WHEP (§9)	implemented	joule-moq-ingest: the catalog read and verified, joining at a group, per-object verification, egress
E-RTMP v2 multitrack (§12)	specified	—
SRT (§13)	specified	—

The reference implementation's tests run on loopback: a WHIP publisher of H.264 and Opus whose frames a relay subscriber receives whole, every receipt verified (by the subscriber and by the relay's door); a WHEP player that gets the same frames from the group it joined; VP8, VP9 and AV1 each through WHIP; every refusal of §3 with its status; trickle ICE and an ICE restart with media across it; and DELETE ending the tracks at the relay. Its reading of room descriptors and grants is held to the corpus of live-rooms (live-conformance/rooms/): every room id and grant digest there, the registered profiles' records, and every refusal of a descriptor or a grant but those of an exchange's link (§3.2 step 2) and of the key log (§3.2 step 4), which admission does not make.

Not implemented in this revision, beside §12 and §13: a WHEP counter-offer (WHEP §4.3.2); a WHEP player's keyframe request carried upstream (MOQT's NEW_GROUP_REQUEST); publishing into a slot (live-rooms §7); simulcast within one WHIP session; STUN and TURN servers in Link headers (RFC 9725 §4.6); and reading the key log itself (§3.2 step 4 holds the revocations a deployment gives it).

18. Security and privacy considerations

• A token is a bearer capability (live-rooms §13): whoever holds it may publish as its principal through a gateway. Its window is fourteen days at most, a single-use grant opens one session, and a gateway's TLS front keeps it off the wire in clear (RFC 9725 §5).
• The first hop's receipts attest what the gateway received, not what the publisher's camera saw: a gateway that lies signs its lie with its own key, and a player that trusts its receipts trusts that key. A deployment lists the gateways it trusts as first hops, and a player that needs more verifies a take (§15).
• The receipts are signed after a group ends: a relay that withholds a group's receipts holds back its frames from every verifying sink, which notices the gap by the chain (§9.2 step 4), not by a missing frame alone.
• An egress gateway that verifies hands a player only frames that verified, so a tampered relay cannot reach a WHEP player with altered media; it can still withhold media, which the session record counts.
• A session's URL is unguessable (§3.4), and its resource answers only its own token, so a third party cannot end or restart another's session.
• Every bound of §10 limits what one session, and one grant, can make a gateway hold.