# joule-moq-web

A browser client for MOQT, `draft-ietf-moq-transport-22`, and the wasm build of
`joule-moq` it runs on. A page publishes its camera and microphone to `joule-moq-relay`
with WebCodecs, over WebTransport or the relay's WebSocket binding, subscribes to the
other participants of a room, verifies every object against its receipt, shows the
room's events and captions, and composes the participants with WAI's integer layout and
mixer (`wai/wai-live-web`, the other wasm build the page loads).

The room page runs two ways: against a test room, every key from a fixture, on loopback;
or against a room service (`joule-moq-room`), as the public room at
`wai.transaction.science/room/` does, where the page makes its own key and the service's
clock authors the room's lifecycle.

## The two wasm builds

| Build | From | What the page uses it for |
|---|---|---|
| `joule_moq_web` | this crate, over `joule-moq` with `default-features = false` | SETUP, every request a client sends, framed control messages and data streams read as their bytes arrive, subgroup streams written; each group signed into its receipt (`TrackSigner`) and each object verified against it (`TrackVerifier`) |
| `wai_live_web` | `wai/wai-live-web`, over `wai-rs` with `live_compose` and `timed_text` | the room descriptor and grants, the lifecycle replay and the composition at an instant, the layout and the mixer, signed events and captions, the MSF event-timeline binding, the `WAI1` envelope |

`joule-moq` without its default feature `runtime` is its codec, its carriage of objects and
receipts, its receipt builder and verifiers and its session's setup rules, with no tokio
and no native dependency. The relay and every other dependent take the default features
and build what they built before.

**Signatures.** Receipts are verified in wasm by `joule-moq`'s `ReceiptVerifier` (its
`ReceiptChains` for the chain), the same code the relay's door runs. WebCrypto's Ed25519 is
not used: a receipt's verification also needs BLAKE3 and the Merkle walk, which WebCrypto
does not have, and one implementation keeps the page's verdicts the relay's.

## The page

`web/room/` is the room client; `web/moq/` its transport, session and media framing;
`web/compose/` runs `live-conformance/compose` in the page.

| Step | What it does |
|---|---|
| Enter (room service) | `?service=<url>` or `data-service` on `#room-app`: the page fetches the room (`GET /room`), turns the camera and microphone on for a publisher, makes an Ed25519 key with the browser's random generator, and asks for a grant for it (`POST /grant`, a publisher's or a viewer's; a full room offers watching). It connects over the WebSocket binding at `/moq` on its own origin (`ws` names another), WebTransport not tried, and presents the grant with a proof by its key. The lifecycle is the service's clock's: a join under a grant the page does not hold has it read the room's grants again (`Room.setGrants`). A viewer publishes nothing. Leave closes the session; the relay ends the tracks and the clock records it. |
| Connect | WebTransport to the relay's URL with `protocols: ["moqt-22"]` (`WT-Available-Protocols`) and, for a development certificate, `serverCertificateHashes`; when the browser has no WebTransport or the session does not open, the WebSocket binding (subprotocol `moqt-22`). `transport=websocket` forces the binding. |
| Session | SETUP with the receipt extension and the participant's room grant: the page waits for the relay's SETUP, which carries the session's challenge, then sends its own with the grant as its authorization token and a proof of the principal's key over that challenge and the URL's `//authority/path`, signed in wasm (WAI live-rooms §10.5.2); requests on bidirectional streams with even Request IDs; every data stream read by a task of its own as it arrives (the relay's README, "A subscriber that does not read"); GOAWAY: no new request, a new session to the URI it names, the tracks published and the subscriptions made again there, the old session closed by the page before the timeout. |
| Publish | `getUserMedia` (the profile's processing as its constraints) into `VideoEncoder` and `AudioEncoder`, the codec the first of the room profile's `codecs` the encoder supports. Each encoded frame is one group of one object (moq-streaming-format §4.1): a `WAI1` envelope whose payload is in SPEC §5's framing for its capability (VP9 in IVF, AV1 in its five-field framing, Opus as an Ogg Opus stream). Its receipt is signed into a group receipt chained to the group before; the receipts-track object, in WAI's binary form (jwp-receipts §2.7.3), goes first, then the media object with its `WAI_RECEIPT` property. A catalog (MSF) names each track's codec, `wai_receipt_alias` and `wai_receipts_track`. |
| Subscribe | Per participant: its events track, its catalog, then each media track's receipts track (no delivery timeout) and the media track, audio at a higher subscriber priority than video. Each receipts-track object is admitted to the track's chain under the participant's key; each object is verified against the receipt of its position before it is decoded; an object whose receipts arrived and which did not is accounted for by inclusion. A late joiner FETCHes every earlier group of each track and its receipts track from group 0. |
| Compose | The room's lifecycle (the clock's `join`, `publish`, `leave` events) replayed by `wai-live-web` gives the participant set; the `WCD1` descriptor and the set give the tiles, drawn on a canvas, re-laid out on each lifecycle event; the voices are mixed by the integer mixer, in blocks of 20 ms, into an AudioWorklet that plays them. |
| Events | Every record verified (signature, chain, the grant's policy, through the room verifier); `caption` events shown with their origin (human, machine, machine-reviewed), the other kinds as a timeline. |

### What a browser needs

| Path | Needs |
|---|---|
| WebTransport | `WebTransport` with `serverCertificateHashes` (for a development certificate: ECDSA P-256, valid at most fourteen days), `protocols` (where it is missing the relay takes the session as -22), and a secure context (`https:` or loopback). |
| WebSocket binding | `WebSocket` with a subprotocol. Nothing else. |
| Publishing | `getUserMedia`, `MediaStreamTrackProcessor`, `VideoEncoder` for one of the profile's codecs (VP9 or AV1 here), `AudioEncoder` for Opus at 48 kHz, `OffscreenCanvas` where the camera's size is not the profile's. |
| Subscribing | `VideoDecoder`, `AudioDecoder`, a 2D canvas, `AudioWorklet`. |
| Both | WebAssembly with BigInt at the boundary (64-bit integers cross as `BigInt`). |

**What the WebSocket binding costs.** Every stream rides one TCP connection, so a lost
packet holds up every stream behind it, which QUIC's independent streams do not; there are
no datagrams in a useful sense (a `DATAGRAM` capsule is reliable and ordered); stream data
is framed again in capsules (a few bytes for each 16 KiB chunk and for each stream's open
and FIN); stream-count credit is granted by capsule; and there is no data flow control
below TCP's, so a receiver that stops reading is bounded by the relay's buffer and then
closed. Delivery timeouts still apply at the relay, before a write.

## Tests

- `cargo test -p joule-moq-web`: the surface's pure-Rust half on the host.
- `node --test web/moq/capsule.test.mjs web/moq/media.test.mjs`: the binding's capsules
  against the bytes the relay's README states, and the payload framings.
- `e2e/run.sh`: everything built (the relay, both wasm builds, the host's half of
  `wai-live-web`), the browser-automation package from `e2e/package-lock.json` with
  install scripts off, then `e2e/room.e2e.mjs` in the host's installed browser (not a
  bundled one) with a fake camera and microphone, the relay on
  loopback under a test configuration (grants required, the door on every media
  namespace):

  | Scenario | Shows |
  |---|---|
  | `compose` | every layout, mix, descriptor and composition case of `live-conformance/compose` through the wasm build gives the corpus's outcome, and `wai-rs`'s corpus check holds in the browser |
  | `webtransport` | two participants publish and receive each other: frames decode, every received object verifies against its receipt and the relay's door refuses none, the canvas's tiles equal the host's layout for the same descriptor and set and show video, the mix reaches the AudioWorklet; a machine caption and a human caption show with their labels; a late joiner gets the earlier groups by FETCH, each verified, its chains admitting them, group 0 among them, as history, and every page re-lays out for three; the receipts objects ride in the binary form, their sizes reported beside the JSON form's; a grant presented without its proof is refused the session |
  | `websocket` | the same over the binding, forced |
  | `goaway` | the relay is stopped with GOAWAY naming a second relay; every page moves, media is verified again there, and the first relay closes no session at its timeout |
  | `service` | a room service keeps the room, as in production: the relay (WebSocket only, its door holding each publisher to its own key, trusting the service's authority) and the service behind a same-origin proxy; pages enter from their buttons with keys of their own. Two publish and see each other, every object verifies, the layout equals the host's, a machine caption and a typed human caption show their labels, the clock (not a page) authors the lifecycle, a late joiner gets earlier groups by FETCH, a viewer watches, a grant with no proof or with a proof by another key is refused the session, a publisher past the room's places is told it is full, and a leave is recorded |

  With `--base <origin>` the `service` scenario runs against a deployment instead: nothing is
  started, and the page at `<origin>/room/` is driven the same way (its production test).

  Every wait is bounded; each browser runs under the host's `memguard` where it exists.

## Not yet implemented

- WebTransport for the public room: its relay is reached over the WebSocket binding
  through an HTTP front, and offers no UDP endpoint.
- Simulcast and staged layers; publishing a screen; datagrams.
- In the fixture's room, a page holding the clock's key authors the lifecycle, driven by
  the test; with a room service, the service's clock does.

## Licence

Apache-2.0.