joule-moq-ingest: WHIP into a room's MoQ tracks and WHEP out of them, receipts signed at the first hop; live-ingest maps E-RTMP v2 and SRT (L6a) #325
Loading…
Reference in a new issue
No description provided.
Delete branch "joule-moq/ingest"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
L6a: WHIP ingest and WHEP egress for rooms
A new crate,
joule-moq-ingest(library and binary), lets a standard WebRTC encoder publish into a room's MoQ tracks over WHIP (RFC 9725):A standard WebRTC player watches a room track over WHEP (
draft-ietf-wish-whep-05). Every object's receipt is verified before the player gets the frame.The mapping is a new WAI extension,
wai/extensions/live-ingest.md. It specifies E-RTMP v2 multitrack (v2-2026-01-31-r2) and SRT 1.5.7 as mappings only. Two payload capabilities are registered so WebRTC media can be packaged at all:wai.video.vp8andwai.audio.opus.packet.What is implemented and what is only specified
Location+ETag, PATCH trickle ICE and ICE restart (If-Match,428/412), DELETE, OPTIONS/CORSWRG1, base64url), checked against the room's signed descriptor; publish power per media kind, else403 insufficient_scope(RFC 6750 §3.1)WAI1envelope; one group per keyframe for video, per fixed 200 ms window for audioDesign decisions worth reviewing
The spec is its own extension,
live-ingest.md, not a section of moq-streaming-format.wai.livefamily's rooms and grants.The token is the grant itself. The WHIP/WHEP bearer token is the
WRG1record, base64url without padding. The gateway checks it the way live-rooms §4.3 checks a grant, then checks:A refused grant gets
401 invalid_token; a good grant without the needed power gets403 insufficient_scope.The gateway's room reader is an independent implementation written from the text.
tests/room_corpus.rsholds it towai/live-conformance/rooms/:Two capabilities are registered (SPEC §5, broadcast/streaming table, sink-supplied, neither a default):
wai.video.vp8: one-frame IVF, FourCCVP80, like VP9.StandardUnclassified, because RFC 6386's conformance text was not read.wai.audio.opus.packet: one Opus packet with no container. Sending an Ogg Opus stream per 20 ms packet would be wrong twice over: it adds three Ogg pages to every packet, and its "whole stream" semantics would reset decoder state each object.The catalog is signed too, on
catalog/receipts. A relay with a door on the room's namespace (jwp-receipts §4.1) refuses objects of any track whose receipts never come, and that includes a catalog. This turned up in the e2e tests. WHEP reads a catalog only after its receipt verifies.Receipts are published after a group's last object. The group receipt's root covers every object, and frames stream as they arrive.
max_video_group_ms. §9.4 states this.Group IDs. A track's first group ID is its first frame's room-clock time in µs, and IDs then ascend by one. A reconnecting publisher continues above the old IDs, so the relay never dedups a new group as a copy (MOQT §7.3).
The timeline and its bound.
t = A + (R − R0)·10^6/rate.max_skew_ms(250) from its arrival.tnever decreases.tlies withinmax_skew_msof its arrival at the gateway.max_skew_ms.trides in the manifest's newlivemember, with the RTP time, the rate and the keyframe flag. On a group's first object it also carries the live-rooms §4.5 grant reference, so every object names its grant through the signed group receipt.What a WHEP player cannot see: the receipts. The gateway verifies on the player's behalf:
Failures are dropped and counted in the session's signed record. A WAI-aware player subscribes to the same tracks and receipts tracks over MoQ (the relay's WebTransport or WebSocket paths) and verifies each object itself (§9.3).
Ingest and capture binding (§15). An ingest is the live tier and is never a
single-take. A device that speaks L4 still commits its take withtake_commitevents on its own events track over MoQ, beside its WHIP session. The two share the room clock, so a marker set on the live call lands in the master within both bounds.The WebRTC stack: str0m 0.24.1, pinned
default-features = false, features = ["rust-crypto"]: DTLS by its owndimpl, SRTP on RustCrypto, no C library linked.Rtcfrom a tokio task.codecstest takes AV1 through it end to end.Every crate new to the lockfile is MIT, Apache-2.0 or ISC: str0m, str0m-proto, str0m-rust-crypto, dimpl, is, sctp-proto, hyper 1.11.1 (pinned), hyper-util 0.1.20 (pinned), http-body-util 0.1.5 (pinned), http-body, httpdate, atomic-waker, ccm, crc, crc-catalog, der_derive, fastrand, flagset, nom, untrusted, x509-cert. The lockfile check passes.
The seam for
WMH1(#317)The PR does not depend on #317. Each WHIP and WHEP session ends with a
SessionRecord(record.rs, formatjoule-moq-ingest/session/1), signed by the gateway's key. It is kept in exactly the shape of #317's per-hop record:WMH1TrackEntrysessions,subscriber_us[9]LedgerEntryenergyroleRecords go to a
record::HopSinktrait. The default sink keeps them (Gateway::records) and appends JSON lines.Once #317 lands, a
Wmh1Sink: HopSinkaggregates a room session's records into oneWMH1and signs it with the same key. The key is listed underlive/ingestin the room's key log. Nothing else in the gateway changes. live-ingest §16 states this without citing the unmerged text.rb results (build host, under memguard)
/Users/dcharlot/data-share/vibe-coding/_wt-moq-ingest)rb --pull <wt> -- 'cd joule-code/joule-code-rs && cargo check -p joule-moq-ingest'(dependency resolution, lockfile)rb <wt> -- 'cd joule-code/joule-code-rs && cargo test -p joule-moq-ingest --tests --no-fail-fast'rb <wt> -- 'cd joule-code/joule-code-rs && cargo clippy -p joule-moq-ingest --all-targets'rb <wt> -- '… cargo test -p joule-moq-ingest --tests --no-run …; copy whip_publish whep_play auth_and_ice codecs; for i in 1..32: run all four binaries × 4 concurrent copies (16 processes at once), wait, count'rb <wt> -- './scripts/check-lockfiles.sh joule-code/joule-code-rs wai/wai-rs; cd wai/wai-rs && cargo run --no-default-features --bin wai -- registry export --check; cargo test --lib --no-default-features codecs'rb <wt> -- 'cd joule-code/joule-code-rs && cargo test --locked -p joule-moq-ingest --all-targets; cargo check --locked --workspace --all-targets'(the new CI job's command, and the workspace with the new member)The registry commits were made in a separate worktree (
_wt-moq-ingest-reg) and cherry-picked here. They passed:wai registry exportwith--pull, then--check;cargo test --lib --no-default-features codecs: 54 passed;All exited 0.
Local scripts, all exit 0:
check-tests-in-ci.py(no entry needed: the job runs every target);check-wai-section-refs.py;check-spec-refs.py;check-no-vanity-metrics.sh;check-comparative-prose.sh;check-workflow-concurrency.sh.check-public-names.shneeds the CI secret and was not run locally. The addedwai/text was reviewed by hand and names only protocols, standards and open-source projects.The forge's CI was not waited for.
Tests (all loopback, every wait bounded, the relay checking every receipt at its door)
whip_publish: a WHIP client (str0m) sends 3 s of synthetic H.264 and Opus. A relay subscriber:It also checks:
200, and the relay sends PUBLISH_DONE on both tracks;404;whep_play:409withRetry-After;auth_and_ice:403 insufficient_scope;401;415; a data-channel-only offer gets422;max_layer, gets403;If-Matchgets428, a stale tag412, trickle204with no ETag, another principal's token403;If-Match: *) returns200with new credentials and a new ETag, ICE reconnects, the old ETag becomes stale, and every frame before and after the restart reaches the relay.codecs: VP8, VP9 and AV1 each go through WHIP whole, one group per keyframe, with each framing stating the right picture size.room_corpus: the room reader against the live-rooms corpus.CI
The new job
moq-ingestinjoule-code-conformance.ymlrunscargo test --locked -p joule-moq-ingest --all-targetson linux-arm64. This follows how #293 addedmoq-relay:wai/live-conformance/rooms/**, which the corpus test reads.Not implemented (also listed in the README and live-ingest §17)
NEW_GROUP_REQUEST);Linkheaders;Merge notes
wai/live-metering) toucheswai/CHANGELOG.md,SPEC.md,REGISTRY.md,registry/capabilities.json,vocab/*andcodecs/registry.rs. Whichever lands second needs a merge, then awai registry exportrerun.joule-moq/web) touches the joule-code workspaceCargo.tomlandCargo.lock, the conformance workflow and the joule-code CHANGELOG.Proof of possession (round 2, head
7181b8a0): stacked on #327This PR is now stacked on #327 (L5 hardening, which carries #324). It merges
joule-moq/hardenat its current head, which carries main with live-metering (#317). Merging #327 first leaves this PR's own diff.A room grant is public, so presenting it as the Bearer token proved nothing about who presents it. Following live-rooms §10.5.3 (from #327), the gateway now does the following:
POST /whipandPOST /whep, plusPATCH,DELETE,GETandHEADon a session's URL. The proof isWAI-Room-Proof, aWRP1record in base64url, over a challenge the gateway holds for the request's scope (the endpoint, or the session) and the contextMETHOD //authority/path. The check isgateway::check_http_proof, the proof step ofauthorize()written as a function, onjoule_moq::grant_proof::check_proof, with the gateway's own reading of the grant.WAI-Room-Challenge, on a401and on every response to a request with a grant; a201's is scoped to the new session. A proof that passes consumes its challenge, and the consumed ones are kept, so a proof presented again ispop_replayed. New bounds:http.challenge_ttl_s(300) andhttp.max_challenges(65536).WAI-Room-Proofand exposesWAI-Room-Challenge.0_receiptsandcatalog_receipts(live-rooms §6.1).room_corpusruns the six HTTP proofs ofwai/live-conformance/rooms/vectors.jsonthroughcheck_http_proof, each with its stated outcome (two accepted;pop_missing,pop_replayed, andpop_wrong_contexttwice). It presents each accepted proof a second time and requirespop_replayed. The gateway's grant reader andjoule-moq's agree on every grant's digest and principal.auth_and_icepresents grants with their proofs through the testAuthhelper. It shows a session'sPATCHandDELETErefusing a grant presented without a proof (401 pop_missing), and the session staying up.Docs: live-ingest §3.1, §3.2 step 7 (the proof), §3.4, §4.1 (
_receipts), §10 (the challenge bounds) and §14 (the eightpop_*reasons); the crate README; both CHANGELOGs.Runs on the build host (rb, under memguard):
cargo test --locked -p joule-moq-ingest --all-targetsroom_corpus3,auth_and_ice2,codecs,whip_publish,whep_play)cargo clippy --locked -p joule-moq-ingest --all-targetscargo check --locked --workspace --all-targetswai registry export --check;--lib codecs::registrywhip_publish,whep_play,codecs,auth_and_iceandroom_corpus, load average 57–89🤖 Generated with Claude Code
dcharlot referenced this pull request2026-10-19 06:16:02 -04:00