open-standards/joule-code/joule-code-rs/crates/joule-moq-web

joule-moq-web

A browser client for MOQT, draft-ietf-moq-transport-22, and the wasm build of joule-moq it runs on. A page publishes its camera and microphone to joule-moq-relay with WebCodecs, over WebTransport or the relay's WebSocket binding, subscribes to the other participants of a room, verifies every object against its receipt, shows the room's events and captions, and composes the participants with WAI's integer layout and mixer (wai/wai-live-web, the other wasm build the page loads).

The room page runs two ways: against a test room, every key from a fixture, on loopback; or against a room service (joule-moq-room), as the public room at wai.transaction.science/room/ does, where the page makes its own key and the service's clock authors the room's lifecycle.

The two wasm builds

Build	From	What the page uses it for
joule_moq_web	this crate, over joule-moq with default-features = false	SETUP, every request a client sends, framed control messages and data streams read as their bytes arrive, subgroup streams written; each group signed into its receipt (TrackSigner) and each object verified against it (TrackVerifier)
wai_live_web	wai/wai-live-web, over wai-rs with live_compose and timed_text	the room descriptor and grants, the lifecycle replay and the composition at an instant, the layout and the mixer, signed events and captions, the MSF event-timeline binding, the WAI1 envelope
wai_live_web_decode	wai/wai-live-web with its feature int_video	everything wai_live_web does, and the decoder on the browser sink's wasm route (WAI extensions/browser-sink.md §7.2): a wai.video.int_motion object verified against its group and object receipts, decoded by the integer decoder, and every frame bound. The binding page loads it; the room's own tracks route to the user agent's decoders and the room page does not
joule_moq_web_e2ee	this crate with its feature e2ee, over joule-moq-mls (MLS, RFC 9420) and joule-moq's secure-objects	everything joule_moq_web does, and the member of an end-to-end encrypted room: the room's confidentiality record checked, its MLS group joined and followed, each frame sealed and each object opened (WAI extensions/confidentiality.md). A page loads it in place of joule_moq_web for such a room alone

joule-moq without its default feature runtime is its codec, its carriage of objects and receipts, its receipt builder and verifiers and its session's setup rules, with no tokio and no native dependency. The relay and every other dependent take the default features and build what they built before.

Sizes. Built as e2e/run.sh builds them, joule_moq_web is about 780 KB (260 KB compressed); joule_moq_web_e2ee, under the workspace's wasm profile (link-time optimisation, one codegen unit, optimised for size) and with no name section, about 2.0 MB (650 KB compressed). The difference is MLS and its crypto provider, which a page in any other room never loads.

Signatures. Receipts are verified in wasm by joule-moq's ReceiptVerifier (its ReceiptChains for the chain), the same code the relay's door runs. WebCrypto's Ed25519 is not used: a receipt's verification also needs BLAKE3 and the Merkle walk, which WebCrypto does not have, and one implementation keeps the page's verdicts the relay's.

The page

web/room/ is the room client; web/moq/ its transport, session and media framing; web/compose/ runs live-conformance/compose in the page; web/binding/ runs the browser sink (WAI extensions/browser-sink.md) in the page; web/ads/ is the reference producer of WAI ad-presentation claims (a span recorder over a video element and the claim's writer and signer), which no page of this crate loads yet: the room page plays no ads. Beside the room page, web/room/messages.html is a member of a conversation (WAI extensions/live-messaging.md, a room service with [messaging]): it joins with a proof of its own key, keys through the service's log with joule_moq_web_e2ee's member (any member commits, anyMemberCommits), seals each message under its epoch and signs it into its receipt chain before posting it, opens and verifies every message of the log (wai_live_web's Messages), shows threads, reactions, edited and retracted messages, and acknowledges what it read; its keys live in its memory alone, so a page closed and opened again is a new member. web/room/call.html is a call's inbox and flow ([call]): every invitation checked with its call before it rings (callCheck), place, ring, accept, decline, cancel, a missed call shown; the call's media is the room page in a frame, entering with the grant the call came with (roomTest.enter(role, { seed, answer })).

Step	What it does
Enter (room service)	?service=<url> or data-service on #room-app: the page fetches the room (GET /room), turns the camera and microphone on for a publisher, makes an Ed25519 key with the browser's random generator, and asks for a grant for it (POST /grant, a publisher's or a viewer's; a full room offers watching). It connects over the WebSocket binding at /moq on its own origin (ws names another), WebTransport not tried, and presents the grant with a proof by its key. The lifecycle is the service's clock's: a join under a grant the page does not hold has it read the room's grants again (Room.setGrants). A viewer publishes nothing. Leave closes the session; the relay ends the tracks and the clock records it.
Connect	WebTransport to the relay's URL with protocols: ["moqt-22"] (WT-Available-Protocols) and, for a development certificate, serverCertificateHashes; when the browser has no WebTransport or the session does not open, the WebSocket binding (subprotocol moqt-22). transport=websocket forces the binding.
Session	SETUP with the receipt extension and the participant's room grant: the page waits for the relay's SETUP, which carries the session's challenge, then sends its own with the grant as its authorization token and a proof of the principal's key over that challenge and the URL's //authority/path, signed in wasm (WAI live-rooms §10.5.2); requests on bidirectional streams with even Request IDs; every data stream read by a task of its own as it arrives (the relay's README, "A subscriber that does not read"); GOAWAY: no new request, a new session to the URI it names, the tracks published and the subscriptions made again there, the old session closed by the page before the timeout.
Publish	getUserMedia (the profile's processing as its constraints) into VideoEncoder and AudioEncoder, the codec the first of the room profile's codecs the encoder supports. Each encoded frame is one group of one object (moq-streaming-format §4.1): a WAI1 envelope whose payload is in SPEC §5's framing for its capability (VP9 in IVF, AV1 in its five-field framing, Opus as an Ogg Opus stream). Its receipt is signed into a group receipt chained to the group before; the receipts-track object, in WAI's binary form (jwp-receipts §2.7.3), goes first, then the media object with its WAI_RECEIPT property. A catalog (MSF) names each track's codec, wai_receipt_alias and wai_receipts_track.
Subscribe	Per participant: its events track, its catalog, then each media track's receipts track (no delivery timeout) and the media track, audio at a higher subscriber priority than video. Each receipts-track object is admitted to the track's chain under the participant's key; each object is verified against the receipt of its position before it is decoded; an object whose receipts arrived and which did not is accounted for by inclusion. A late joiner FETCHes every earlier group of each track and its receipts track from group 0.
Route and bind	Before a track is decoded, the page routes it (browser-sink §4): it builds a capability report from the catalog's codec string, dimensions and wai_determinism, puts to the user agent only the codec strings the WebCodecs Codec Registry lists (isConfigSupported, then Media Capabilities for its powerEfficient), and decodes the track only if its route is the user agent's decoder. Each object verified against its receipt gives the page its content hash and its group receipt's link hash; every frame a video decoder outputs is bound to them in a side table keyed by decoder and timestamp (web/room/binding.js, §9.2), and clone() carries the binding. With ?debug=binding the page shows, for each tile, the state of the frame it last drew: its binding, the receipt and chain it verified under, its route and its energy entry (§10).
Stream termination	For each track it receives, the page keeps a stream log (WAI extensions/stream-termination.md, web/room/termination.js), its records written by wai-live-web's StreamLog under a key the page makes for its session: the open record (how the stream started, the viewer's autoplay setting, autoplay=off or this page's storage, beside the user agent's answer, the policy, the panel and the ladder from the catalog); a segment receipt for each 2 s of objects it verified, with the rendition's match against the panel and each hop as the unmetered marker (a page has no meter); and the observations of the document's visibility, the canvas leaving the viewport (an IntersectionObserver) and the viewer's activations. The panel is declared from screen times devicePixelRatio, colorDepth and the dynamic-range media queries, its refresh rate measured over 60 animation frames, and declared again when it changes. A video stream hidden or out of view for unseen_grace_ms (10 s by default) is terminated: the termination is signed, the subscription cancelled (its request stream reset), anything still in flight discarded unreceipted, and the audio kept; seen again, the video is subscribed under a new log. roomTest.terminationLogs() gives every log with the page's own verdict.
Compose	The room's lifecycle (the clock's join, publish, leave events) replayed by wai-live-web gives the participant set; the WCD1 descriptor and the set give the tiles, drawn on a canvas, re-laid out on each lifecycle event; the voices are mixed by the integer mixer, in blocks of 20 ms, into an AudioWorklet that plays them.
Events	Every record verified (signature, chain, the grant's policy, through the room verifier); caption events shown with their origin (human, machine, machine-reviewed), the other kinds as a timeline.
Admission by age	When the service's room admits by age (GET /room's age, WAI age-assurance §5.6), its grant request is answered 401 with Privacy Pass challenges (RFC 9577): the page picks the least threshold at or above the room's age the challenges offer (or age_threshold), checks the challenge (RFC 9577 §2.1.3), obtains a token from the issuer by blind issuance (RFC 9578 §6, web/room/privacy-pass.js), at the request URL of the issuer's directory (RFC 9578 §4; issuer_directory names where the directories are), and asks again with it. A publisher carries the room's rating on every track it publishes (WAI_RATING, §5.7), and with rate, rate_scheme and rate_value its own rating of its video, signed in wasm (ratingClaim). A request the relay refuses below_threshold is shown as withheld, not as an error.
Watching together	When the service's room has a co-watch policy (GET /room's watch, WAI extensions/co-watch.md), the page fetches the room's source and checks its bytes by the content hash the policy names, estimates the room clock against the service's, and enters the room's co-watch log by a proof of its key over the service's challenge for the request it makes, before it subscribes to the clock (web/room/cowatch.js). It follows the log, checks each control as §5 has it before it applies it at its instant, keeps the source at the controlled position by the catch-up rule (§9), signs a presentation statement twice a slot and posts it, and shows its own offset and the verdicts of the clock's last skew statements beside shared controls (play, pause, a seek, a rate). Its events in such a room are its co-watch statements alone.
End-to-end encryption	When the service's room has a confidentiality record (GET /room's e2ee), the page says so before anyone enters, offers no watching and no seeking back, checks the record against the descriptor (its room, authority and recording mode), and loads joule_moq_web_e2ee. It leaves a KeyPackage with the service before it publishes, then reads the room's log (web/room/e2ee.js): each lifecycle record checked as the clock's, each MLS message checked and applied by the wasm member; it creates the group or commits when the room calls on it, and applies what it made only when the log returns it. Each encoded frame is sealed under the page's epoch before its receipt is signed over the sealed bytes; nothing is sent before the page holds an epoch. Each object received is verified against its receipt, then opened through the receiving checks, and only then decoded; one sealed in an epoch the page has not reached waits, and one that does not open is counted with its reason. The page shows its epoch and the members of the group. Every key is in the wasm member's memory and nowhere else.
Co-performance	When the service's room has a co-performance policy (GET /room's co), the page verifies its audience's grants with its publishers', follows every present principal's events, and shows (web/room/co.js): the turn schedule and who holds the floor, the marks the clock signs, each participant's intents as upcoming lines and what became of them, the selection window open now with an input box for a viewer whose grant carries the audience-input power, each selection with its credits, and the clock's fidelity statement. A lead (the direct power) asks the room's clock for turns, a handover, a vote, prompts and a fidelity statement (POST <service>/co/..., its grant with a proof over the service's challenge, live-rooms §10.5.3, signed in wasm: grantProofHttp); the turn's holder may hand it over. A performer states its media's timestamps on the room clock (an audio object's from the inputs its encoder took) and sends media only while it holds the floor, from a guard after its turn starts to a guard before it ends. ai=scripted makes the page a demo AI participant: a grant marking it one, a drawn picture and a tone for its media, and a scripted line announced ahead of each it says as a machine's caption.

What a browser needs

Path	Needs
WebTransport	WebTransport with serverCertificateHashes (for a development certificate: ECDSA P-256, valid at most fourteen days), protocols (where it is missing the relay takes the session as -22), and a secure context (https: or loopback).
WebSocket binding	WebSocket with a subprotocol. Nothing else.
Publishing	getUserMedia, MediaStreamTrackProcessor, VideoEncoder for one of the profile's codecs (VP9 or AV1 here), AudioEncoder for Opus at 48 kHz, OffscreenCanvas where the camera's size is not the profile's.
Subscribing	VideoDecoder, AudioDecoder, a 2D canvas, AudioWorklet.
Both	WebAssembly with BigInt at the boundary (64-bit integers cross as BigInt).

What the WebSocket binding costs. Every stream rides one TCP connection, so a lost packet holds up every stream behind it, which QUIC's independent streams do not; there are no datagrams in a useful sense (a DATAGRAM capsule is reliable and ordered); stream data is framed again in capsules (a few bytes for each 16 KiB chunk and for each stream's open and FIN); stream-count credit is granted by capsule; and there is no data flow control below TCP's, so a receiver that stops reading is bounded by the relay's buffer and then closed. Delivery timeouts still apply at the relay, before a write.

Tests

• cargo test -p joule-moq-web: the surface's pure-Rust half on the host; with --features e2ee, the member's surface too: two members and a clock key through one ordered log as pages do, seal, sign, read the stream back, verify and open, and every refusal an object can meet.

• node --test web/moq/capsule.test.mjs web/moq/media.test.mjs web/room/binding.test.mjs: the binding's capsules against the bytes the relay's README states, the payload framings, and the page's side table against the sequences of wai/live-conformance/browser.

• node --test web/moq/session.test.mjs: how long a request waits for its answer, the clock mocked: a SUBSCRIBE that asks for a rendezvous takes the relay's refusal when the rendezvous ends, rather than giving up as the relay sends it.

• node --test web/room/privacy-pass.test.mjs: the page's Privacy Pass client against RFC 9578 Appendix A.2's first issuance vector, from WAI's age-assurance corpus: from the vector's key, challenge, nonce, blind and salt it writes the vector's TokenRequest, and from the issuer's TokenResponse the vector's token; a response changed by a bit is refused; RFC 9577's header fields read and write.

• node --test web/room/cowatch.test.mjs: the page's co-watch module (web/room/cowatch.js, WAI extensions/co-watch.md) against wai/live-conformance/cowatch/: every catch-up case its correction, each accepted room's controlled position, and its controls and presentation statements written and signed byte for byte.

• node --test web/room/endpoint.test.mjs: the check of the endpoints the room page takes from its query string and from a relay's GOAWAY (web/room/endpoint.js): the page's own origin alone (https://<host>, wss://<host> for a relay), and on a loopback page loopback addresses alone; another origin refused in every form a link can write it (another host or port, scheme-relative, the page's host as user information, javascript:, a plain scheme on a secure page).

• node --test web/room/markup.test.mjs: the room page's markup, every element it opens closed in order, and each mode's section holding its own controls: the Log, the binding table and the co-watch section never inside the co-performance section, which a room without it hides.

• node --test web/room/termination.test.mjs: the page's stream-termination rules (web/room/termination.js): the panel it declares from what a page reads, the refresh rate it measures, which reason is due under its policy as observations arrive, how verified objects gather into segments that never overlap, and which input events are the viewer's.

• node --test web/ads/presentation.test.mjs: the reference producer of WAI ad-presentation claims (web/ads/presentation.js, WAI extensions/ad-presentation.md §10): observation sequences recorded into spans, and claims written and signed from them, byte for byte as wai/live-conformance/presentation/ holds them.

• e2e/run.sh: everything built (the relay, both wasm builds, the host's half of wai-live-web), the browser-automation package from e2e/package-lock.json with install scripts off, then e2e/room.e2e.mjs in the host's installed browser (not a bundled one) with a fake camera and microphone, the relay on loopback under a test configuration (grants required, the door on every media namespace):

  Scenario	Shows
  compose	every layout, mix, descriptor and composition case of live-conformance/compose through the wasm build gives the corpus's outcome, and wai-rs's corpus check holds in the browser
  webtransport	two participants publish and receive each other: frames decode, every received object verifies against its receipt and the relay's door refuses none, the canvas's tiles equal the host's layout for the same descriptor and set and show video, the mix reaches the AudioWorklet; a machine caption and a human caption show with their labels; a late joiner gets the earlier groups by FETCH, each verified, its chains admitting them, group 0 among them, as history, and every page re-lays out for three; the receipts objects ride in the binary form, their sizes reported beside the JSON form's; a grant presented without its proof is refused the session
  websocket	the same over the binding, forced
  goaway	the relay is stopped with GOAWAY naming a second relay; every page moves, media is verified again there, and the first relay closes no session at its timeout
  service	a room service keeps the room, as in production: the relay (WebSocket only, its door holding each publisher to its own key, trusting the service's authority) and the service behind a same-origin proxy; pages enter from their buttons with keys of their own. Two publish and see each other, every object verifies, the layout equals the host's, a machine caption and a typed human caption show their labels, the clock (not a page) authors the lifecycle, a late joiner gets earlier groups by FETCH, a participant seeks back eight seconds into the other's tracks from the relay's DVR window (the relay keeps one and a cache far smaller than the span; the dvr scenario's checks: the control enables, the window's video plays in its own view from its first key frame at the pace it was captured, every object verifies, nothing is unknown, the page returns to live and live video goes on decoding, the relay served it from its window), a viewer watches, a grant with no proof or with a proof by another key is refused the session, a publisher past the room's places is told it is full, and a leave is recorded
  age	a room that admits by age, its service configured with the age corpus's test issuers and rating the room 16, and a test issuer on loopback (e2e/issuer.mjs, over the corpus's published keys, signing blindly whatever it is sent): each page answers the challenge with a token it obtains by blind issuance and enters at its threshold; a performer at 18 rates its own video 18; a viewer at 16 is served the audio, verified and decoded, and refused the video (below_threshold), which the page shows withheld; a viewer at 18 is served both; a viewer with a token of 13 is refused by the service; a token presented a second time is refused (token_replayed); the service's admission receipt verifies under the room's clock and suppresses every count under 50
  co	a room with co-performance kept by a room service, behind a relay that holds the floor and the audience's inputs: a lead, an AI participant (the scripted demo) and three audience members; turns started from the lead's controls, every page showing who holds the floor; a performer out of turn marked by the clock, and under a later schedule that refuses held back by the relay; the AI participant's lines shown ahead, one kept and one diverged; a late object marked; a vote and prompts from the page's input box, each input the rules refuse withheld by the relay with its reason and counted in its summaries; each window committed to and selected from, credited; the fidelity statement; and the room with the clock's log of objects verified (co-performance §9) the same in the page and on the host (wai_live_web co-verify)
  ageco	a room that admits by age (rated 16, the age scenario's issuers) and keeps co-performance (the relay holding the floor): the lead at 18 rates its video 18, an AI participant at 18; the room's clock proves the room's keys in its SETUP and the relay takes its session for the room's own service (WAI age-assurance §5.5), so the clock logs every track of both performers, the video rated 18 among them, and marks an object played out of turn; a viewer at 16 is still served the audio and refused the video (below_threshold)
  origin	the endpoints a link may name (web/room/endpoint.js): a link naming a relay on another origin (?ws=) is refused when the visitor presses Join, and nothing connects (no WebSocket, no request, no grant, no camera); a link naming a room service on another origin, written scheme-relative, behind the page's host as user information or as javascript:, is refused and Join stays off; a relay whose GOAWAY names another origin is refused, the session ends and the page says so. The service scenario checks besides that every request and WebSocket of every page stays on the page's own origin, as the production page's connect-src requires
  binding	the browser sink: web/binding/ runs live-conformance/browser's routes, constraints, bindings, frame states and side-table sequences through the page's wasm and JavaScript; decodes a wai.video.int_motion object on the wasm route after verifying it against its receipts, every frame bound and its SHA-256 the published one, a changed object refused and one with no receipt left unbound; carries the binding through clone(), close(), new VideoFrame(frame), transfer to a worker and back, copyTo() and re-encode; binds the user agent's own VP9 decoder's frames through the side table and refuses a timestamp given twice; reports what the user agent keeps of an unregistered VideoFrame metadata member; builds a capability report from the user agent's answers, the interfaces' own record showing no codec string outside the registry put to them; and, where the host has a per-process energy counter (e2e/energy.py), reports a figure per decode measured around bursts of decodes beside powerEfficient. Then a two-party room with ?debug=binding: each page's tile of the other shows a frame bound to an object that page verified, under its group receipt, as verified
  termination	stream termination (WAI extensions/stream-termination.md, capability-constraints §4): live-conformance/termination through the page's wasm build; then a room whose receiving page runs in a browser driven over the DevTools protocol alone (e2e/lib.mjs's rawBrowser), since the automation package keeps every page it drives visible. Its tab goes behind another: after the grace the page signs a termination as hidden and cancels its subscription to the other's video, the relay sends no more of it while the audio goes on decoding, the log verifies on the host in wai-rs (wai_live_web termination-verify) and in the Python verifier with the facts the page states, and a segment receipt the page signed, replayed after the termination, is refused after_termination; in front again the video is subscribed under a new log. The screen shrinks below the video: the page declares the panel again and its receipts record the match, the least rendition served. The canvas is moved out of the viewport and the video is terminated as occluded
  e2ee	an end-to-end encrypted room kept by a room service (e2ee = true), the relay keeping a DVR window: the page says so before anyone enters; two publishers key through the service's log (a GroupInfo, a commit, a Welcome), media flows both ways, every object verified against its receipt over the sealed bytes, opened, then decoded; the page shows its epoch and roster; a viewer, outside the group, receives the bytes exactly as sealed (no envelope, the AEAD's length) and opens nothing; every change a relay could make to an object is refused with the extension's reason (bad_tag, key_id_missing, properties_malformed, object_id_too_large, key_id_unknown, sender_not_owner, replayed) and the object opens once; an object changed on the wire is refused bad_tag; a late joiner's FETCH of earlier groups opens nothing sealed before its join (epoch_not_joined); a member that stops publishing and stays is removed by the next commit and opens nothing of the later epoch it still receives (removed_before_epoch), and an object it sealed before is refused sender_removed; the relay's DVR store keeps nothing of the room
  cowatch	a watch room (WAI co-watch) kept by a room service (watch_source, a two-minute tone), behind the same-origin proxy: three viewers, each page fetching the source and checking it by its content hash and entering the co-watch log by a proof of its key; one plays, one pauses, one seeks, each control applied by every page at its instant; a viewer joins late and is brought to the room's position; in the windows after it, the clock's skew statements state every viewer and pair in sync within the 250 ms bound, as each page shows; no record or event is refused; and the room read back from the service verifies by co_watch_verify.py, every skew statement recomputed
  messaging	e2e/messaging.e2e.mjs: a conversation kept by a room service with a history store, behind the same-origin proxy: three pages join and key through the service's log and open what the others seal; one page's network is cut while the others start a thread, reply and react, the service stops and starts again from its store, a root is edited and a message sent and retracted; the page comes back and catches up, opening every message of both epochs, the thread, the reaction, the edit and the retraction as the others show them; each page acknowledges what it read; the owner removes a member, whose page is told it is no longer one and opens nothing written after; the store holds no plaintext
  call	e2e/messaging.e2e.mjs: a call between two pages through a relay: the callee's inbox checks the invitation and its call before it rings, the callee accepts, and both rooms (the room page in a frame, entering with the call's grant) publish camera and microphone and decode the other's video and audio; a call a third page never answers is missed at its deadline, signed by the room's clock, and shown as missed

  With --base <origin> the service scenario runs against a deployment instead: nothing is started, and the page at <origin>/room/ is driven the same way (its production test). Its seek back is skipped there, and reported as skipped rather than passed, since the deployment's relay may keep no window; --seek on runs it, and --seek off skips it on loopback too.

  Every wait is bounded; each browser runs under the host's memguard where it exists.

Not yet implemented

• WebTransport for the public room: its relay is reached over the WebSocket binding through an HTTP front, and offers no UDP endpoint.
• Simulcast and staged layers; publishing a screen; datagrams.
• In an end-to-end encrypted room: watching without being a member, ingest by WHIP or WebRTC, and seeking back (WAI confidentiality §15.3).
• In the fixture's room, a page holding the clock's key authors the lifecycle, driven by the test; with a room service, the service's clock does.

Licence

Apache-2.0.