# joule-moq-ingest A gateway between WebRTC and the MoQ tracks of a WAI room. A standard WebRTC encoder publishes into a room over **WHIP** (RFC 9725); the gateway becomes its tracks' MoQ publisher at the relay and signs every group's receipts as the first hop. A standard WebRTC player watches a room's tracks over **WHEP** (`draft-ietf-wish-whep-05`); the gateway subscribes at the relay and verifies every object's receipt before it hands the frame on. The mapping it implements is WAI's `extensions/live-ingest.md`, which also maps Enhanced RTMP v2 multitrack and SRT (specified, not implemented here). It is a library (`GatewayServer`, `Gateway`) and a binary, `joule-moq-ingest`. The relay is `joule-moq-relay`; the MOQT codec is `joule-moq`'s, `draft-ietf-moq-transport-22`. ## Resources | resource | methods | |---|---| | `/whip` | `POST` an `application/sdp` offer: `201`, the answer, `Location`, `ETag`. `OPTIONS` (CORS, `Accept-Post`), `GET`, `HEAD` | | `/whip/s/` | `PATCH` an `application/trickle-ice-sdpfrag`: trickle ICE (`204`) or an ICE restart (`200`, a fragment, a new `ETag`); `DELETE` (`200`) ends the session and its tracks | | `/whep//[?layer=]` | `POST` an offer for the owner's tracks: `201`, or `409` with `Retry-After` while the owner is not live. `HEAD` answers with `Link: ; rel="profile"` (WHEP §4.2) | | `/whep/s/` | `PATCH`, `DELETE`, as for WHIP | Every request but `OPTIONS` carries `Authorization: Bearer `. **The token is a room grant**: a WAI `WRG1` record (`live-rooms.md` §4), base64url without padding. A grant is public, so every request that carries one also carries a proof of possession of its key, `WAI-Room-Proof` (`live-rooms.md` §10.5.3), over a single-use challenge the gateway gave in `WAI-Room-Challenge`: on a `401` and on each response to a request with a grant. Refusals are RFC 9457 problem details whose `reason` member is `live-ingest.md` §14's code. ## Admission The gateway holds each room's signed descriptor (`WRD1`) and checks a token against the room it names, as the relay checks a token against its grants, here by the room's own signed records: the grant reads and its signature verifies under the room's authority (`live-rooms.md` §4.3, its field rules, its fourteen-day bound, its role and the room's ceiling), it is good now, it is not among the room's revocations, its origins include the gateway's, a single-use grant has opened no session here, and the request's proof passes `live-rooms.md` §10.5.4 over a challenge held for its endpoint or session (`joule_moq::grant_proof::check_proof`, through `gateway::check_http_proof`). What fails is `401` with `WWW-Authenticate: Bearer error="invalid_token"`. A good grant without the power the request needs is **`403`** with `error="insufficient_scope"` (RFC 6750 §3.1): the publish power for each media kind a WHIP offer carries, the subscribe power up to the layer a WHEP request asks for. A session's resource answers only the token that opened it. `room.rs` reads descriptors and grants from the extension's text; `tests/room_corpus.rs` holds it to WAI's corpus `wai/live-conformance/rooms/`: every room id and grant digest, the registered profiles' records, and every refusal of a descriptor or a grant there, but an exchanged grant's link and the key log, which a bearer token does not bring. It also runs the corpus's six HTTP proofs through `check_http_proof`, each with its stated outcome, and presents each accepted proof again as a replay. ## WHIP into the room - **Tracks.** Per media kind of the offer, `(room, owner, audio|video)` / `0` and its receipts track `0_receipts`; and the owner's catalog, `(room, owner)` / `catalog`, with `catalog_receipts`. All are published (PUBLISH) before the answer goes back. Audio and the catalog at publisher priority 0, video at 1. - **Frames.** str0m depacketises RTP into frames: H.264 (RFC 6184), VP8 (RFC 7741), VP9 (RFC 9628), AV1 (its RTP payload format v1.0.0) and Opus (RFC 7587). Each frame is one object, one `WAI1` envelope: | codec | capability | payload | |---|---|---| | H.264 | `wai.video.avc` | one access unit, Annex B | | VP8 | `wai.video.vp8` | one frame in IVF (`VP80`), its size from the keyframe header | | VP9 | `wai.video.vp9` | one frame in IVF (`VP90`), its size from the uncompressed header | | AV1 | `wai.video.av1` | one temporal unit in SPEC §5's AV1 framing, the temporal delimiter restored, the size from the sequence header | | Opus | `wai.audio.opus.packet` | one packet | - **Groups.** Video: one group per keyframe (a PLI goes to the publisher before the first one, and when a group passes `max_video_group_ms`). Audio: one group per fixed window of `audio_group_ms` on the room clock. One subgroup stream per group, written as the frames arrive. A track's first group ID is its first frame's room-clock time in microseconds. - **Time.** The manifest's `live` member carries the frame's room-clock time `t`, its RTP time and clock rate, whether it is a keyframe, and on a group's first object the grant reference (`live-rooms.md` §4.5). The mapping is anchored at a track's first frame's arrival, `t = A + (R − R0) · 10^6 / rate`, and re-anchored at any frame it would place more than `max_skew_ms` from its arrival: so **every frame's `t` lies within `max_skew_ms` of its arrival at the gateway**. - **Receipts.** When a group ends, the gateway signs its group receipt over every object and publishes it with the object receipts on the receipts track, chained to the group before. Every figure is the unmetered marker: no meter backs the gateway. The chain's alias is derived per session, so a publisher that reconnects starts a new chain. - **The catalog** is MSF JSON with WAI's `wai_*` fields, published once every track has begun (or a second after the first did) and again whenever a codec or picture size changes, and signed like the media, so a relay that checks every track of the room at its door forwards it. - **The end.** DELETE (or ICE lost, idle, a bound passed) ends each open group, publishes its receipts, waits for the relay to acknowledge every stream, and sends PUBLISH_DONE `TRACK_ENDED` on every track: the relay's subscribers see the tracks end. ## WHEP out of the room The gateway reads the owner's latest catalog once its receipt verifies, answers the player's offer with the codecs the tracks carry, and subscribes to each accepted track and its receipts track from the start of the latest group (the live part from the Next Object, the rest of that group by a fill), so the player starts at a keyframe. Before a frame reaches the player, its group's receipts must name its MOQT group and Object ID and the track's chain, verify under a trusted first hop's key (`whep.trust`; this gateway's own key by default), chain to the group before, and prove the object's inclusion; the object must open as an envelope of the track's codec. What fails is dropped and counted in the session's record; a group whose receipts never come is dropped after `hold_ms`. **What a WHEP player cannot see:** the receipts. WebRTC carries media only. The gateway verifies on the player's behalf and records what it dropped and why; a WAI-aware player subscribes to the same tracks and their receipts tracks over MoQ, through the relay's WebTransport or WebSocket paths, and verifies every object itself. A verified group is released when its receipts arrive, at the group's end, so a WHEP player is up to a video group behind the publisher. ## Bounds Each is a field of `[limits]`, `[groups]`, `[timeline]` or `[whep]` in the configuration: | field | default | past it | |---|---|---| | `max_whip_sessions`, `max_whep_sessions` | 64, 256 | `503` with `Retry-After` | | `max_tracks_per_session` | 2 | `422`; and one track of each kind at most | | `max_bitrate_kbps` over `bitrate_window_ms` | 8 000 over 2 000 ms | stated per section as `b=AS` in the answer (a grant's publish profile lowers it); a session past it is ended | | `max_frame_bytes` | 4 MiB | the frame is dropped and counted | | `max_queue_bytes` | 32 MiB | the session is ended (`relay_backpressure`) | | `audio_group_ms`, `max_video_group_ms` | 200, 2 000 | — | | `max_skew_ms` | 250 | the track is re-anchored | | `whep.hold_ms` | 5 000 | the group is dropped (`receipt_missing`) | `gateway.example.toml` shows every field. It holds no secret: the relay token and the signing key are files, and a room is its public, signed descriptor. ```text joule-moq-ingest --config gateway.toml # JOULE_MOQ_INGEST_LOG sets the log level ``` Every listener is loopback by default: put a TLS front before the HTTP resources (RFC 9725 §5) and set `media.bind_ip` to an address clients reach. ## The WebRTC stack [str0m](https://crates.io/crates/str0m) 0.24.1, pinned, MIT OR Apache-2.0, with `default-features = false` and its `rust-crypto` backend: DTLS by str0m's own `dimpl`, SRTP and the rest on RustCrypto, so no C library is linked. It is sans-IO: each session here owns a UDP socket and drives its `Rtc` from a tokio task, feeding it datagrams and the time and draining its output after every change. It depacketises and packetises H.264, VP8, VP9, AV1 and Opus at the frame level (AV1 is a hidden codec in its public API; this crate's tests take AV1 through it end to end), supports ICE lite and full ICE, ICE restarts, NACK and keyframe requests. Every crate new to the lockfile is MIT, Apache-2.0 or ISC. ## The session record, and live metering Each session ends with a record of what it carried, signed by the gateway's key (`record.rs`, format `joule-moq-ingest/session/1`): per track and direction the objects and bytes, the drops by reason, subscriber sessions and their time by layer class, the grant with its ceilings and the bytes under it, and the gateway's energy as the unmetered marker. `Gateway::records` keeps them and `summary.out_path` appends them as JSON lines. These are the counts of the per-hop session record `WMH1` of WAI's live metering, kept in its shape, role 3 (ingest gateway). Once that record lands, a `record::HopSink` that writes `WMH1` turns each record into its track entries, subscriber time and ledger entry and signs it with the same key, which the room's key log lists under `live/ingest`; nothing else in the gateway changes. ## Tests All on loopback, every wait bounded, the relay checking every receipt at its door: | test | what it shows | |---|---| | `whip_publish` | A WHIP publisher (str0m as the client) sends three seconds of H.264 and Opus. A subscriber at the relay receives every group of both tracks, verifies every receipt under the gateway's key and every chain link, and gets the frames' bytes as sent; one group per keyframe, every audio group within its window; DELETE ends the tracks with PUBLISH_DONE; the session's record verifies. | | `whep_play` | Before the publisher is live a WHEP POST is `409` with `Retry-After`. Then a WHEP player (str0m) gets the publisher's frames, byte for byte, from the keyframe of the group it joined to the last group closed; the WHEP session ends with the tracks, nothing refused. | | `auth_and_ice` | No token `401`; a token that is not a grant `401 invalid_token`; a viewer's grant `403 insufficient_scope`; a grant past its window, one of another room; `415`; `422`; WHEP without subscribe or above `max_layer` `403`; discovery and CORS. `PATCH` without `If-Match` `428`, stale `412`, trickle `204`, another principal's grant, its own key proved, `403`; a `PATCH` or `DELETE` presenting the grant without a proof `401 pop_missing`; an ICE restart (`If-Match: *`) `200` with new credentials and a new `ETag`, ICE connecting again, and every frame before and after it at the relay. | | `codecs` | VP8, VP9 and AV1 each through WHIP: every frame whole, one group per keyframe, the size each framing states. | | `room_corpus` | The room reader against `wai/live-conformance/rooms/`, and its six HTTP proofs of possession through the gateway's proof step, each accepted proof also refused as a replay. | ## Not implemented E-RTMP v2 and SRT ingest (`live-ingest.md` §12, §13: mapped only); a WHEP counter-offer; a player's keyframe request carried upstream; publishing into a slot; simulcast within one WHIP session; STUN and TURN in `Link` headers; reading the key log itself (revocations are the list a deployment configures). ## Licence Apache-2.0.