open-standards/joule-code/joule-code-rs/crates/joule-moq-ingest

joule-moq-ingest

A gateway between contribution protocols and the MoQ tracks of a WAI room. A standard WebRTC encoder publishes into a room over WHIP (RFC 9725), and an RTMP encoder over Enhanced RTMP v2 (v2-2026-01-31-r2), single-track or multitrack, over TCP or TLS; the gateway becomes their tracks' MoQ publisher at the relay and signs every group's receipts as the first hop. A standard WebRTC player watches a room's tracks over WHEP (draft-ietf-wish-whep-05); the gateway subscribes at the relay and verifies every object's receipt before it hands the frame on. A broadcast automation system drives the gateway's SCTE 104 injector, which translates its requests to SCTE 35 and signs them as the room's cues. It also takes contribution over SRT and RIST into the same tracks (extensions/live-contribution.md, below), and signs each such link's record of loss, recovery and latency. The mappings it implements are WAI's extensions/live-ingest.md, extensions/cue-injection.md and extensions/live-contribution.md.

It is a library (GatewayServer, Gateway) and a binary, wai-ingest (the crate keeps its name). The relay is joule-moq-relay (binary wai-relay); the MOQT codec is joule-moq's, draft-ietf-moq-transport-22.

Resources

resource	methods
/whip	POST an application/sdp offer: 201, the answer, Location, ETag. OPTIONS (CORS, Accept-Post), GET, HEAD
/whip/s/<id>	PATCH an application/trickle-ice-sdpfrag: trickle ICE (204) or an ICE restart (200, a fragment, a new ETag); DELETE (200) ends the session and its tracks
/whep/<room>/<owner>[?layer=<n>]	POST an offer for the owner's tracks: 201, or 409 with Retry-After while the owner is not live. HEAD answers with Link: <urn:ietf:params:whep>; rel="profile" (WHEP §4.2)
/whep/s/<id>	PATCH, DELETE, as for WHIP
/rtmp	POST: 201, {"expires_in", "stream_key", "url"}: a one-time stream key for an Enhanced RTMP publish under the grant

Every request but OPTIONS carries Authorization: Bearer <token>. The token is a room grant: a WAI WRG1 record (live-rooms.md §4), base64url without padding. A grant is public, so every request that carries one also carries a proof of possession of its key, WAI-Room-Proof (live-rooms.md §10.5.3), over a single-use challenge the gateway gave in WAI-Room-Challenge: on a 401 and on each response to a request with a grant. Refusals are RFC 9457 problem details whose reason member is live-ingest.md §14's code.

Admission

The gateway holds each room's signed descriptor (WRD1) and checks a token against the room it names, as the relay checks a token against its grants, here by the room's own signed records: the grant reads and its signature verifies under the room's authority (live-rooms.md §4.3, its field rules, its fourteen-day bound, its role and the room's ceiling), it is good now, it is not among the room's revocations, its origins include the gateway's, a single-use grant has opened no session here, and the request's proof passes live-rooms.md §10.5.4 over a challenge held for its endpoint or session (joule_moq::grant_proof::check_proof, through gateway::check_http_proof). What fails is 401 with WWW-Authenticate: Bearer error="invalid_token". A good grant without the power the request needs is 403 with error="insufficient_scope" (RFC 6750 §3.1): the publish power for each media kind a WHIP offer carries, the subscribe power up to the layer a WHEP request asks for. A session's resource answers only the token that opened it.

room.rs reads descriptors and grants from the extension's text; tests/room_corpus.rs holds it to WAI's corpus wai/live-conformance/rooms/: every room id and grant digest, the registered profiles' records, and every refusal of a descriptor or a grant there, but an exchanged grant's link and the key log, which a bearer token does not bring. It also runs the corpus's six HTTP proofs through check_http_proof, each with its stated outcome, and presents each accepted proof again as a replay.

WHIP into the room

• Tracks. Per media kind of the offer, (room, owner, audio|video) / 0 and its receipts track 0_receipts; and the owner's catalog, (room, owner) / catalog, with catalog_receipts. All are published (PUBLISH) before the answer goes back. Audio and the catalog at publisher priority 0, video at 1.

• Frames. str0m depacketises RTP into frames: H.264 (RFC 6184), VP8 (RFC 7741), VP9 (RFC 9628), AV1 (its RTP payload format v1.0.0) and Opus (RFC 7587). Each frame is one object, one WAI1 envelope:

  codec	capability	payload
  H.264	wai.video.avc	one access unit, Annex B
  VP8	wai.video.vp8	one frame in IVF (VP80), its size from the keyframe header
  VP9	wai.video.vp9	one frame in IVF (VP90), its size from the uncompressed header
  AV1	wai.video.av1	one temporal unit in SPEC §5's AV1 framing, the temporal delimiter restored, the size from the sequence header
  Opus	wai.audio.opus.packet	one packet

• Groups. Video: one group per keyframe (a PLI goes to the publisher before the first one, and when a group passes max_video_group_ms). Audio: one group per fixed window of audio_group_ms on the room clock. One subgroup stream per group, written as the frames arrive. A track's first group ID is its first frame's room-clock time in microseconds.

• Time. The manifest's live member carries the frame's room-clock time t, its RTP time and clock rate, whether it is a keyframe, and on a group's first object the grant reference (live-rooms.md §4.5). The mapping is anchored at a track's first frame's arrival, t = A + (R − R0) · 10^6 / rate, and re-anchored at any frame it would place more than max_skew_ms from its arrival: so every frame's t lies within max_skew_ms of its arrival at the gateway.

• Receipts. When a group ends, the gateway signs its group receipt over every object and publishes it with the object receipts on the receipts track, chained to the group before. Every figure is the unmetered marker: no meter backs the gateway. The chain's alias is derived per session, so a publisher that reconnects starts a new chain.

• The catalog is MSF JSON with WAI's wai_* fields, published once every track has begun (or a second after the first did) and again whenever a codec or picture size changes, and signed like the media, so a relay that checks every track of the room at its door forwards it.

• The end. DELETE (or ICE lost, idle, a bound passed) ends each open group, publishes its receipts, waits for the relay to acknowledge every stream, and sends PUBLISH_DONE TRACK_ENDED on every track: the relay's subscribers see the tracks end.

WHEP out of the room

The gateway reads the owner's latest catalog once its receipt verifies, answers the player's offer with the codecs the tracks carry, and subscribes to each accepted track and its receipts track from the start of the latest group (the live part from the Next Object, the rest of that group by a fill), so the player starts at a keyframe. Before a frame reaches the player, its group's receipts must name its MOQT group and Object ID and the track's chain, verify under a trusted first hop's key (whep.trust; this gateway's own key by default), chain to the group before, and prove the object's inclusion; the object must open as an envelope of the track's codec. What fails is dropped and counted in the session's record; a group whose receipts never come is dropped after hold_ms.

What a WHEP player cannot see: the receipts. WebRTC carries media only. The gateway verifies on the player's behalf and records what it dropped and why; a WAI-aware player subscribes to the same tracks and their receipts tracks over MoQ, through the relay's WebTransport or WebSocket paths, and verifies every object itself.

A verified group is released when its receipts arrive, at the group's end, so a WHEP player is up to a video group behind the publisher.

Enhanced RTMP into the room

• Admission. An encoder has a server URL and a stream key, and cannot answer a challenge, so the publisher first POSTs to /rtmp with its grant and a proof of possession, admitted as a WHIP request is, and gets a stream key: 128 random bits, good for one publish within stream_key_ttl_s (60 s). The publish command's stream name is the key; the grant is admitted again when it is used. A refusal is onStatus NetStream.Publish.BadName with the reason as its description.
• The connection (rtmp::chunk, rtmp::amf0, rtmp::session): the simple handshake, the chunk stream with its extended timestamps, Set Chunk Size, Abort, acknowledgements and pings, and the commands. The connect result declares capsEx 0x0F and the FourCCs it carries.
• Tracks (rtmp::tracks, rtmp::flv): legacy AVC and AAC, and avc1, hvc1, av01, vp09, vp08, Opus, fLaC and mp4a, through ModEx and multitrack bodies and aggregate messages. Each RTMP track is bound to a room track at its first packet: under a grant with a layered video profile, the layer its onMetaData entry's size and frame rate match; audio track 0 only. Each frame is one object: Annex B with the record's parameter sets before a key access unit that carries none, the AV1 framing with the sequence header, IVF for VP8 and VP9, ADTS for AAC, a FLAC stream. live.rtp is the decode time in µs (rate 1 000 000) and live.cto the composition offset; §7's mapping gives t.
• RTMPS on rtmp.tls_listen with a PEM chain and PKCS #8 key (tokio-rustls on the workspace's rustls and ring).
• Shutdown. A publisher that declared Reconnect is sent NetConnection.Connect.ReconnectRequest (with rtmp.reconnect_url as its tcUrl, if set), its key is good once more, and the session ends when it leaves or after reconnect_grace_ms.
• Records. Each publish ends with a session record (protocol ertmp) whose detail counts what was read and not carried, by reason.

SCTE 104 into the room's cues

scte listens on injector.listen (SCTE 104's port is 5167) for one automation system at a time: init_request, alive_request, and multiple_operation_message()s of splice, time-signal and splice-null requests with avail, DTMF, segmentation, time, tier and private descriptor requests. Each is answered (inject_response, then inject_complete_response once its cues are signed) with SCTE 104's result codes. scte::scte104::translate builds each SCTE 35 section (scte::scte35) on the room's 90 kHz clock and the cues it yields; scte::Engine holds each cue until its splice instant, applies cancels and early returns, drops what live-ads' break rules would refuse, and signs the cue as a cue event of the injector's principal after its injection record (WIJ1, a meta event carrying the message, its arrival and the translation's parameters). The events go out on the principal's events track at the relay, published from the injector's start. The injector's grant must name its key and permit cue and meta.

A relay whose door checks receipts names the room's media kinds in its rules ([room, "*", "audio"], [room, "*", "video"]): an events track carries no receipts.

Bounds

Each is a field of [limits], [groups], [timeline] or [whep] in the configuration:

field	default	past it
max_whip_sessions, max_whep_sessions	64, 256	503 with Retry-After
max_tracks_per_session	2	422; and one track of each kind at most
max_bitrate_kbps over bitrate_window_ms	8 000 over 2 000 ms	stated per section as b=AS in the answer (a grant's publish profile lowers it); a session past it is ended
max_frame_bytes	4 MiB	the frame is dropped and counted
max_queue_bytes	32 MiB	the session is ended (relay_backpressure)
audio_group_ms, max_video_group_ms	200, 2 000	—
max_skew_ms	250	the track is re-anchored
whep.hold_ms	5 000	the group is dropped (receipt_missing)
rtmp.max_sessions	64	one more connection is closed at accept
rtmp.max_message_bytes	8 MiB	the connection ends (message_too_large)
rtmp.max_bitrate_kbps over bitrate_window_ms	40 000 over 2 000 ms	the publish ends (bitrate_exceeded)
rtmp.connect_timeout_ms, rtmp.idle_timeout_ms	15 000, 20 000	the connection ends
rtmp.stream_key_ttl_s	60	the key is refused (stream_key_expired)
injector.max_pending	256	a message is answered 124

gateway.example.toml shows every field. It holds no secret: the relay token and the signing key are files, and a room is its public, signed descriptor.

wai-ingest --config gateway.toml      # JOULE_MOQ_INGEST_LOG sets the log level

Every listener is loopback by default: put a TLS front before the HTTP resources (RFC 9725 §5) and set media.bind_ip to an address clients reach.

The WebRTC stack

str0m 0.24.1, pinned, MIT OR Apache-2.0, with default-features = false and its rust-crypto backend: DTLS by str0m's own dimpl, SRTP and the rest on RustCrypto, so no C library is linked. It is sans-IO: each session here owns a UDP socket and drives its Rtc from a tokio task, feeding it datagrams and the time and draining its output after every change. It depacketises and packetises H.264, VP8, VP9, AV1 and Opus at the frame level (AV1 is a hidden codec in its public API; this crate's tests take AV1 through it end to end), supports ICE lite and full ICE, ICE restarts, NACK and keyframe requests. Every crate new to the lockfile is MIT, Apache-2.0 or ISC.

SRT and RIST

resource	methods
/srt	POST with the grant and its proof: 201, a one-time session as JSON: its stream_id, passphrase, pbkeylen, the listener's latency_ms, host and port. With {"call": "ip:port", "passphrase": …}, the gateway calls the publisher's SRT listener, where [contrib.srt] call_allow lists it
/rist	POST with the grant and its proof, `{"profile": "simple"
/srt/s/<id>, /rist/s/<id>	DELETE ends the session, or one not yet connected; GET, HEAD

The SRT listener ([contrib.srt] listen) checks each caller's stream ID (#!::r=<room>/<owner>,m=publish,s=<session>) against the sessions it gave, and refuses with the codes of live-ingest §13 (1401, 1402, 1403, 1404, 1405, 1409) and the draft's (1010 for a wrong passphrase, 1011 for none). A RIST session's port takes the first flow whose RTCP CNAME is the session's. The link's transport stream is demultiplexed: the first H.264 or HEVC stream and the first AAC stream go to the video and audio tracks as WHIP's frames do, one access unit (one ADTS frame) an object, the PTS on the room clock; a PES missing a packet is dropped whole. The catalog declares the owner's link track.

The link record (linkrec, WCL1): every [contrib] record_interval_ms and when the session ends, the gateway signs a part: the sequence numbers passed, each delivered at the first try, recovered, dropped or outstanding, the loss runs, requests, retransmissions, round trips and recovery times, chained to the part before and naming each media track's receipt chain head. Each part goes out on the link track with its receipts, is kept (Gateway::link_records) and is appended to [summary] link_out_path. The corpus wai/live-conformance/links/ is written by joule-moq-ingest-link-vectors.

RIST is built only with the feature rist (joule-contrib's README, "Patents").

The session record, and live metering

Each session ends with a record of what it carried, signed by the gateway's key (record.rs, format joule-moq-ingest/session/1): per track and direction the objects and bytes, the drops by reason, subscriber sessions and their time by layer class, the grant with its ceilings and the bytes under it, and the gateway's energy as the unmetered marker. Gateway::records keeps them and summary.out_path appends them as JSON lines.

These are the counts of the per-hop session record WMH1 of WAI's live metering, kept in its shape, role 3 (ingest gateway). Once that record lands, a record::HopSink that writes WMH1 turns each record into its track entries, subscriber time and ledger entry and signs it with the same key, which the room's key log lists under live/ingest; nothing else in the gateway changes.

Tests

All on loopback, every wait bounded, the relay checking every receipt at its door:

test	what it shows
whip_publish	A WHIP publisher (str0m as the client) sends three seconds of H.264 and Opus. A subscriber at the relay receives every group of both tracks, verifies every receipt under the gateway's key and every chain link, and gets the frames' bytes as sent; one group per keyframe, every audio group within its window; DELETE ends the tracks with PUBLISH_DONE; the session's record verifies.
whep_play	Before the publisher is live a WHEP POST is 409 with Retry-After. Then a WHEP player (str0m) gets the publisher's frames, byte for byte, from the keyframe of the group it joined to the last group closed; the WHEP session ends with the tracks, nothing refused.
auth_and_ice	No token 401; a token that is not a grant 401 invalid_token; a viewer's grant 403 insufficient_scope; a grant past its window, one of another room; 415; 422; WHEP without subscribe or above max_layer 403; discovery and CORS. PATCH without If-Match 428, stale 412, trickle 204, another principal's grant, its own key proved, 403; a PATCH or DELETE presenting the grant without a proof 401 pop_missing; an ICE restart (If-Match: *) 200 with new credentials and a new ETag, ICE connecting again, and every frame before and after it at the relay.
codecs	VP8, VP9 and AV1 each through WHIP: every frame whole, one group per keyframe, the size each framing states.
room_corpus	The room reader against wai/live-conformance/rooms/, and its six HTTP proofs of possession through the gateway's proof step, each accepted proof also refused as a replay.
ertmp	A publisher of our own over RTMP and RTMPS: legacy AVC and AAC; enhanced HEVC with composition offsets and Opus; AV1 with FLAC and VP9 with AAC, a ModEx nanosecond offset; multitrack layers bound by the grant's profile; every refusal of a stream key; the reconnect request at shutdown. Every frame reaches a relay subscriber in its payload framing, at its mapped time, every receipt verified.
scte104_injection	An automation system of our own over TCP: init, alive, a second connection refused 110, splices with and without pre-roll, a cancel, time signals, refusals with their result codes; the chain verifies and every cue is its message's translation; a relay subscriber reads the chain from the events track.
conformance_corpus	wai/live-conformance/ertmp/ and wai/live-conformance/injection/ regenerate byte for byte; the texts' reason tables list the codes reported.
ertmp_ffmpeg (ignored; --ignored where a local encoder is present)	A local encoder as the test client: AVC with B-frames and AAC, HEVC and AVC on the platform's media engine, HEVC with Opus, AV1 with FLAC, VP9 with Opus; every frame decoded from what the relay carries.
contrib_srt	An SRT publisher, through a path that drops datagrams each way, into the gateway: a relay subscriber gets every H.264 access unit and AAC frame, every receipt verified; the link record's parts verify as a series, show the losses recovered and name each track's receipt-chain head as the subscriber found it, and are on the link track. Every listener refusal with its code; the gateway calling an encoder's listener with HEVC; 20 Mbit/s with groups of megabytes through the relay's door.
contrib_rist (feature rist)	A RIST Simple Profile flow and a Main Profile tunnel under PSK into the gateway, received whole; another CNAME opens no session.
link_corpus	Every vector of wai/live-conformance/links/ with its stated outcome.
contrib_interop (ignored)	The reference SRT caller and RIST sender into the gateway, with an encoder's H.264, HEVC and AAC through lossy paths (tools named by WAI_SRT_LIVE_TRANSMIT, WAI_RISTSENDER, WAI_FFMPEG, WAI_FFPROBE).
contrib_bench (ignored)	The gateway's throughput and energy on an SRT link, the sender in a process of its own.

Not implemented

vvc1 over Enhanced RTMP; a stream key another gateway accepts after a reconnect request; SCTE 104 in a video signal's ancillary data (cue-injection.md §10); SRT and RIST egress; a WHEP counter-offer; a player's keyframe request carried upstream; publishing into a slot; simulcast within one WHIP session; STUN and TURN in Link headers; reading the key log itself (revocations are the list a deployment configures).

Licence

Apache-2.0.