open-standards/joule-code/joule-code-rs/crates/joule-moq-ingest

joule-moq-ingest

A gateway between WebRTC and the MoQ tracks of a WAI room. A standard WebRTC encoder publishes into a room over WHIP (RFC 9725); the gateway becomes its tracks' MoQ publisher at the relay and signs every group's receipts as the first hop. A standard WebRTC player watches a room's tracks over WHEP (draft-ietf-wish-whep-05); the gateway subscribes at the relay and verifies every object's receipt before it hands the frame on. The mapping it implements is WAI's extensions/live-ingest.md, which also maps Enhanced RTMP v2 multitrack and SRT (specified, not implemented here).

It is a library (GatewayServer, Gateway) and a binary, joule-moq-ingest. The relay is joule-moq-relay; the MOQT codec is joule-moq's, draft-ietf-moq-transport-22.

Resources

resource	methods
/whip	POST an application/sdp offer: 201, the answer, Location, ETag. OPTIONS (CORS, Accept-Post), GET, HEAD
/whip/s/<id>	PATCH an application/trickle-ice-sdpfrag: trickle ICE (204) or an ICE restart (200, a fragment, a new ETag); DELETE (200) ends the session and its tracks
/whep/<room>/<owner>[?layer=<n>]	POST an offer for the owner's tracks: 201, or 409 with Retry-After while the owner is not live. HEAD answers with Link: <urn:ietf:params:whep>; rel="profile" (WHEP §4.2)
/whep/s/<id>	PATCH, DELETE, as for WHIP

Every request but OPTIONS carries Authorization: Bearer <token>. The token is a room grant: a WAI WRG1 record (live-rooms.md §4), base64url without padding. A grant is public, so every request that carries one also carries a proof of possession of its key, WAI-Room-Proof (live-rooms.md §10.5.3), over a single-use challenge the gateway gave in WAI-Room-Challenge: on a 401 and on each response to a request with a grant. Refusals are RFC 9457 problem details whose reason member is live-ingest.md §14's code.

Admission

The gateway holds each room's signed descriptor (WRD1) and checks a token against the room it names, as the relay checks a token against its grants, here by the room's own signed records: the grant reads and its signature verifies under the room's authority (live-rooms.md §4.3, its field rules, its fourteen-day bound, its role and the room's ceiling), it is good now, it is not among the room's revocations, its origins include the gateway's, a single-use grant has opened no session here, and the request's proof passes live-rooms.md §10.5.4 over a challenge held for its endpoint or session (joule_moq::grant_proof::check_proof, through gateway::check_http_proof). What fails is 401 with WWW-Authenticate: Bearer error="invalid_token". A good grant without the power the request needs is 403 with error="insufficient_scope" (RFC 6750 §3.1): the publish power for each media kind a WHIP offer carries, the subscribe power up to the layer a WHEP request asks for. A session's resource answers only the token that opened it.

room.rs reads descriptors and grants from the extension's text; tests/room_corpus.rs holds it to WAI's corpus wai/live-conformance/rooms/: every room id and grant digest, the registered profiles' records, and every refusal of a descriptor or a grant there, but an exchanged grant's link and the key log, which a bearer token does not bring. It also runs the corpus's six HTTP proofs through check_http_proof, each with its stated outcome, and presents each accepted proof again as a replay.

WHIP into the room

• Tracks. Per media kind of the offer, (room, owner, audio|video) / 0 and its receipts track 0_receipts; and the owner's catalog, (room, owner) / catalog, with catalog_receipts. All are published (PUBLISH) before the answer goes back. Audio and the catalog at publisher priority 0, video at 1.

• Frames. str0m depacketises RTP into frames: H.264 (RFC 6184), VP8 (RFC 7741), VP9 (RFC 9628), AV1 (its RTP payload format v1.0.0) and Opus (RFC 7587). Each frame is one object, one WAI1 envelope:

  codec	capability	payload
  H.264	wai.video.avc	one access unit, Annex B
  VP8	wai.video.vp8	one frame in IVF (VP80), its size from the keyframe header
  VP9	wai.video.vp9	one frame in IVF (VP90), its size from the uncompressed header
  AV1	wai.video.av1	one temporal unit in SPEC §5's AV1 framing, the temporal delimiter restored, the size from the sequence header
  Opus	wai.audio.opus.packet	one packet

• Groups. Video: one group per keyframe (a PLI goes to the publisher before the first one, and when a group passes max_video_group_ms). Audio: one group per fixed window of audio_group_ms on the room clock. One subgroup stream per group, written as the frames arrive. A track's first group ID is its first frame's room-clock time in microseconds.

• Time. The manifest's live member carries the frame's room-clock time t, its RTP time and clock rate, whether it is a keyframe, and on a group's first object the grant reference (live-rooms.md §4.5). The mapping is anchored at a track's first frame's arrival, t = A + (R − R0) · 10^6 / rate, and re-anchored at any frame it would place more than max_skew_ms from its arrival: so every frame's t lies within max_skew_ms of its arrival at the gateway.

• Receipts. When a group ends, the gateway signs its group receipt over every object and publishes it with the object receipts on the receipts track, chained to the group before. Every figure is the unmetered marker: no meter backs the gateway. The chain's alias is derived per session, so a publisher that reconnects starts a new chain.

• The catalog is MSF JSON with WAI's wai_* fields, published once every track has begun (or a second after the first did) and again whenever a codec or picture size changes, and signed like the media, so a relay that checks every track of the room at its door forwards it.

• The end. DELETE (or ICE lost, idle, a bound passed) ends each open group, publishes its receipts, waits for the relay to acknowledge every stream, and sends PUBLISH_DONE TRACK_ENDED on every track: the relay's subscribers see the tracks end.

WHEP out of the room

The gateway reads the owner's latest catalog once its receipt verifies, answers the player's offer with the codecs the tracks carry, and subscribes to each accepted track and its receipts track from the start of the latest group (the live part from the Next Object, the rest of that group by a fill), so the player starts at a keyframe. Before a frame reaches the player, its group's receipts must name its MOQT group and Object ID and the track's chain, verify under a trusted first hop's key (whep.trust; this gateway's own key by default), chain to the group before, and prove the object's inclusion; the object must open as an envelope of the track's codec. What fails is dropped and counted in the session's record; a group whose receipts never come is dropped after hold_ms.

What a WHEP player cannot see: the receipts. WebRTC carries media only. The gateway verifies on the player's behalf and records what it dropped and why; a WAI-aware player subscribes to the same tracks and their receipts tracks over MoQ, through the relay's WebTransport or WebSocket paths, and verifies every object itself.

A verified group is released when its receipts arrive, at the group's end, so a WHEP player is up to a video group behind the publisher.

Bounds

Each is a field of [limits], [groups], [timeline] or [whep] in the configuration:

field	default	past it
max_whip_sessions, max_whep_sessions	64, 256	503 with Retry-After
max_tracks_per_session	2	422; and one track of each kind at most
max_bitrate_kbps over bitrate_window_ms	8 000 over 2 000 ms	stated per section as b=AS in the answer (a grant's publish profile lowers it); a session past it is ended
max_frame_bytes	4 MiB	the frame is dropped and counted
max_queue_bytes	32 MiB	the session is ended (relay_backpressure)
audio_group_ms, max_video_group_ms	200, 2 000	—
max_skew_ms	250	the track is re-anchored
whep.hold_ms	5 000	the group is dropped (receipt_missing)

gateway.example.toml shows every field. It holds no secret: the relay token and the signing key are files, and a room is its public, signed descriptor.

joule-moq-ingest --config gateway.toml      # JOULE_MOQ_INGEST_LOG sets the log level

Every listener is loopback by default: put a TLS front before the HTTP resources (RFC 9725 §5) and set media.bind_ip to an address clients reach.

The WebRTC stack

str0m 0.24.1, pinned, MIT OR Apache-2.0, with default-features = false and its rust-crypto backend: DTLS by str0m's own dimpl, SRTP and the rest on RustCrypto, so no C library is linked. It is sans-IO: each session here owns a UDP socket and drives its Rtc from a tokio task, feeding it datagrams and the time and draining its output after every change. It depacketises and packetises H.264, VP8, VP9, AV1 and Opus at the frame level (AV1 is a hidden codec in its public API; this crate's tests take AV1 through it end to end), supports ICE lite and full ICE, ICE restarts, NACK and keyframe requests. Every crate new to the lockfile is MIT, Apache-2.0 or ISC.

The session record, and live metering

Each session ends with a record of what it carried, signed by the gateway's key (record.rs, format joule-moq-ingest/session/1): per track and direction the objects and bytes, the drops by reason, subscriber sessions and their time by layer class, the grant with its ceilings and the bytes under it, and the gateway's energy as the unmetered marker. Gateway::records keeps them and summary.out_path appends them as JSON lines.

These are the counts of the per-hop session record WMH1 of WAI's live metering, kept in its shape, role 3 (ingest gateway). Once that record lands, a record::HopSink that writes WMH1 turns each record into its track entries, subscriber time and ledger entry and signs it with the same key, which the room's key log lists under live/ingest; nothing else in the gateway changes.

Tests

All on loopback, every wait bounded, the relay checking every receipt at its door:

test	what it shows
whip_publish	A WHIP publisher (str0m as the client) sends three seconds of H.264 and Opus. A subscriber at the relay receives every group of both tracks, verifies every receipt under the gateway's key and every chain link, and gets the frames' bytes as sent; one group per keyframe, every audio group within its window; DELETE ends the tracks with PUBLISH_DONE; the session's record verifies.
whep_play	Before the publisher is live a WHEP POST is 409 with Retry-After. Then a WHEP player (str0m) gets the publisher's frames, byte for byte, from the keyframe of the group it joined to the last group closed; the WHEP session ends with the tracks, nothing refused.
auth_and_ice	No token 401; a token that is not a grant 401 invalid_token; a viewer's grant 403 insufficient_scope; a grant past its window, one of another room; 415; 422; WHEP without subscribe or above max_layer 403; discovery and CORS. PATCH without If-Match 428, stale 412, trickle 204, another principal's grant, its own key proved, 403; a PATCH or DELETE presenting the grant without a proof 401 pop_missing; an ICE restart (If-Match: *) 200 with new credentials and a new ETag, ICE connecting again, and every frame before and after it at the relay.
codecs	VP8, VP9 and AV1 each through WHIP: every frame whole, one group per keyframe, the size each framing states.
room_corpus	The room reader against wai/live-conformance/rooms/, and its six HTTP proofs of possession through the gateway's proof step, each accepted proof also refused as a replay.

Not implemented

E-RTMP v2 and SRT ingest (live-ingest.md §12, §13: mapped only); a WHEP counter-offer; a player's keyframe request carried upstream; publishing into a slot; simulcast within one WHIP session; STUN and TURN in Link headers; reading the key log itself (revocations are the list a deployment configures).

Licence

Apache-2.0.