open-standards/joule-code/joule-code-rs/crates/joule-moq-ingest/README.md

# joule-moq-ingest

A gateway between WebRTC and the MoQ tracks of a WAI room. A standard WebRTC encoder
publishes into a room over **WHIP** (RFC 9725); the gateway becomes its tracks' MoQ
publisher at the relay and signs every group's receipts as the first hop. A standard
WebRTC player watches a room's tracks over **WHEP** (`draft-ietf-wish-whep-05`); the
gateway subscribes at the relay and verifies every object's receipt before it hands the
frame on. The mapping it implements is WAI's `extensions/live-ingest.md`, which also maps
Enhanced RTMP v2 multitrack and SRT (specified, not implemented here).

It is a library (`GatewayServer`, `Gateway`) and a binary, `joule-moq-ingest`. The relay
is `joule-moq-relay`; the MOQT codec is `joule-moq`'s, `draft-ietf-moq-transport-22`.

## Resources

| resource | methods |
|---|---|
| `/whip` | `POST` an `application/sdp` offer: `201`, the answer, `Location`, `ETag`. `OPTIONS` (CORS, `Accept-Post`), `GET`, `HEAD` |
| `/whip/s/<id>` | `PATCH` an `application/trickle-ice-sdpfrag`: trickle ICE (`204`) or an ICE restart (`200`, a fragment, a new `ETag`); `DELETE` (`200`) ends the session and its tracks |
| `/whep/<room>/<owner>[?layer=<n>]` | `POST` an offer for the owner's tracks: `201`, or `409` with `Retry-After` while the owner is not live. `HEAD` answers with `Link: <urn:ietf:params:whep>; rel="profile"` (WHEP §4.2) |
| `/whep/s/<id>` | `PATCH`, `DELETE`, as for WHIP |

Every request but `OPTIONS` carries `Authorization: Bearer <token>`. **The token is a room
grant**: a WAI `WRG1` record (`live-rooms.md` §4), base64url without padding. A grant is
public, so every request that carries one also carries a proof of possession of its key,
`WAI-Room-Proof` (`live-rooms.md` §10.5.3), over a single-use challenge the gateway gave in
`WAI-Room-Challenge`: on a `401` and on each response to a request with a grant. Refusals are
RFC 9457 problem details whose `reason` member is `live-ingest.md` §14's code.

## Admission

The gateway holds each room's signed descriptor (`WRD1`) and checks a token against the
room it names, as the relay checks a token against its grants, here by the room's own
signed records: the grant reads and its signature verifies under the room's authority
(`live-rooms.md` §4.3, its field rules, its fourteen-day bound, its role and the room's
ceiling), it is good now, it is not among the room's revocations, its origins include the
gateway's, a single-use grant has opened no session here, and the request's proof passes
`live-rooms.md` §10.5.4 over a challenge held for its endpoint or session
(`joule_moq::grant_proof::check_proof`, through `gateway::check_http_proof`). What fails is `401` with
`WWW-Authenticate: Bearer error="invalid_token"`. A good grant without the power the
request needs is **`403`** with `error="insufficient_scope"` (RFC 6750 §3.1): the publish
power for each media kind a WHIP offer carries, the subscribe power up to the layer a WHEP
request asks for. A session's resource answers only the token that opened it.

`room.rs` reads descriptors and grants from the extension's text; `tests/room_corpus.rs`
holds it to WAI's corpus `wai/live-conformance/rooms/`: every room id and grant digest,
the registered profiles' records, and every refusal of a descriptor or a grant there, but
an exchanged grant's link and the key log, which a bearer token does not bring. It also
runs the corpus's six HTTP proofs through `check_http_proof`, each with its stated
outcome, and presents each accepted proof again as a replay.

## WHIP into the room

- **Tracks.** Per media kind of the offer, `(room, owner, audio|video)` / `0` and its
  receipts track `0_receipts`; and the owner's catalog, `(room, owner)` / `catalog`, with
  `catalog_receipts`. All are published (PUBLISH) before the answer goes back. Audio and
  the catalog at publisher priority 0, video at 1.
- **Frames.** str0m depacketises RTP into frames: H.264 (RFC 6184), VP8 (RFC 7741), VP9
  (RFC 9628), AV1 (its RTP payload format v1.0.0) and Opus (RFC 7587). Each frame is one
  object, one `WAI1` envelope:

  | codec | capability | payload |
  |---|---|---|
  | H.264 | `wai.video.avc` | one access unit, Annex B |
  | VP8 | `wai.video.vp8` | one frame in IVF (`VP80`), its size from the keyframe header |
  | VP9 | `wai.video.vp9` | one frame in IVF (`VP90`), its size from the uncompressed header |
  | AV1 | `wai.video.av1` | one temporal unit in SPEC §5's AV1 framing, the temporal delimiter restored, the size from the sequence header |
  | Opus | `wai.audio.opus.packet` | one packet |

- **Groups.** Video: one group per keyframe (a PLI goes to the publisher before the first
  one, and when a group passes `max_video_group_ms`). Audio: one group per fixed window of
  `audio_group_ms` on the room clock. One subgroup stream per group, written as the frames
  arrive. A track's first group ID is its first frame's room-clock time in microseconds.
- **Time.** The manifest's `live` member carries the frame's room-clock time `t`, its RTP
  time and clock rate, whether it is a keyframe, and on a group's first object the grant
  reference (`live-rooms.md` §4.5). The mapping is anchored at a track's first frame's
  arrival, `t = A + (R − R0) · 10^6 / rate`, and re-anchored at any frame it would place
  more than `max_skew_ms` from its arrival: so **every frame's `t` lies within
  `max_skew_ms` of its arrival at the gateway**.
- **Receipts.** When a group ends, the gateway signs its group receipt over every object
  and publishes it with the object receipts on the receipts track, chained to the group
  before. Every figure is the unmetered marker: no meter backs the gateway. The chain's
  alias is derived per session, so a publisher that reconnects starts a new chain.
- **The catalog** is MSF JSON with WAI's `wai_*` fields, published once every track has
  begun (or a second after the first did) and again whenever a codec or picture size
  changes, and signed like the media, so a relay that checks every track of the room at its
  door forwards it.
- **The end.** DELETE (or ICE lost, idle, a bound passed) ends each open group, publishes
  its receipts, waits for the relay to acknowledge every stream, and sends PUBLISH_DONE
  `TRACK_ENDED` on every track: the relay's subscribers see the tracks end.

## WHEP out of the room

The gateway reads the owner's latest catalog once its receipt verifies, answers the
player's offer with the codecs the tracks carry, and subscribes to each accepted track and
its receipts track from the start of the latest group (the live part from the Next Object,
the rest of that group by a fill), so the player starts at a keyframe. Before a frame
reaches the player, its group's receipts must name its MOQT group and Object ID and the
track's chain, verify under a trusted first hop's key (`whep.trust`; this gateway's own
key by default), chain to the group before, and prove the object's inclusion; the object
must open as an envelope of the track's codec. What fails is dropped and counted in the
session's record; a group whose receipts never come is dropped after `hold_ms`.

**What a WHEP player cannot see:** the receipts. WebRTC carries media only. The gateway
verifies on the player's behalf and records what it dropped and why; a WAI-aware player
subscribes to the same tracks and their receipts tracks over MoQ, through the relay's
WebTransport or WebSocket paths, and verifies every object itself.

A verified group is released when its receipts arrive, at the group's end, so a WHEP
player is up to a video group behind the publisher.

## Bounds

Each is a field of `[limits]`, `[groups]`, `[timeline]` or `[whep]` in the configuration:

| field | default | past it |
|---|---|---|
| `max_whip_sessions`, `max_whep_sessions` | 64, 256 | `503` with `Retry-After` |
| `max_tracks_per_session` | 2 | `422`; and one track of each kind at most |
| `max_bitrate_kbps` over `bitrate_window_ms` | 8 000 over 2 000 ms | stated per section as `b=AS` in the answer (a grant's publish profile lowers it); a session past it is ended |
| `max_frame_bytes` | 4 MiB | the frame is dropped and counted |
| `max_queue_bytes` | 32 MiB | the session is ended (`relay_backpressure`) |
| `audio_group_ms`, `max_video_group_ms` | 200, 2 000 | — |
| `max_skew_ms` | 250 | the track is re-anchored |
| `whep.hold_ms` | 5 000 | the group is dropped (`receipt_missing`) |

`gateway.example.toml` shows every field. It holds no secret: the relay token and the
signing key are files, and a room is its public, signed descriptor.

```text
joule-moq-ingest --config gateway.toml      # JOULE_MOQ_INGEST_LOG sets the log level
```

Every listener is loopback by default: put a TLS front before the HTTP resources (RFC 9725
§5) and set `media.bind_ip` to an address clients reach.

## The WebRTC stack

[str0m](https://crates.io/crates/str0m) 0.24.1, pinned, MIT OR Apache-2.0, with
`default-features = false` and its `rust-crypto` backend: DTLS by str0m's own `dimpl`,
SRTP and the rest on RustCrypto, so no C library is linked. It is sans-IO: each session
here owns a UDP socket and drives its `Rtc` from a tokio task, feeding it datagrams and the
time and draining its output after every change. It depacketises and packetises H.264,
VP8, VP9, AV1 and Opus at the frame level (AV1 is a hidden codec in its public API; this
crate's tests take AV1 through it end to end), supports ICE lite and full ICE, ICE
restarts, NACK and keyframe requests. Every crate new to the lockfile is MIT, Apache-2.0
or ISC.

## The session record, and live metering

Each session ends with a record of what it carried, signed by the gateway's key
(`record.rs`, format `joule-moq-ingest/session/1`): per track and direction the objects and
bytes, the drops by reason, subscriber sessions and their time by layer class, the grant
with its ceilings and the bytes under it, and the gateway's energy as the unmetered marker.
`Gateway::records` keeps them and `summary.out_path` appends them as JSON lines.

These are the counts of the per-hop session record `WMH1` of WAI's live metering, kept in
its shape, role 3 (ingest gateway). Once that record lands, a `record::HopSink` that writes
`WMH1` turns each record into its track entries, subscriber time and ledger entry and signs
it with the same key, which the room's key log lists under `live/ingest`; nothing else in
the gateway changes.

## Tests

All on loopback, every wait bounded, the relay checking every receipt at its door:

| test | what it shows |
|---|---|
| `whip_publish` | A WHIP publisher (str0m as the client) sends three seconds of H.264 and Opus. A subscriber at the relay receives every group of both tracks, verifies every receipt under the gateway's key and every chain link, and gets the frames' bytes as sent; one group per keyframe, every audio group within its window; DELETE ends the tracks with PUBLISH_DONE; the session's record verifies. |
| `whep_play` | Before the publisher is live a WHEP POST is `409` with `Retry-After`. Then a WHEP player (str0m) gets the publisher's frames, byte for byte, from the keyframe of the group it joined to the last group closed; the WHEP session ends with the tracks, nothing refused. |
| `auth_and_ice` | No token `401`; a token that is not a grant `401 invalid_token`; a viewer's grant `403 insufficient_scope`; a grant past its window, one of another room; `415`; `422`; WHEP without subscribe or above `max_layer` `403`; discovery and CORS. `PATCH` without `If-Match` `428`, stale `412`, trickle `204`, another principal's grant, its own key proved, `403`; a `PATCH` or `DELETE` presenting the grant without a proof `401 pop_missing`; an ICE restart (`If-Match: *`) `200` with new credentials and a new `ETag`, ICE connecting again, and every frame before and after it at the relay. |
| `codecs` | VP8, VP9 and AV1 each through WHIP: every frame whole, one group per keyframe, the size each framing states. |
| `room_corpus` | The room reader against `wai/live-conformance/rooms/`, and its six HTTP proofs of possession through the gateway's proof step, each accepted proof also refused as a replay. |

## Not implemented

E-RTMP v2 and SRT ingest (`live-ingest.md` §12, §13: mapped only); a WHEP counter-offer;
a player's keyframe request carried upstream; publishing into a slot; simulcast within one
WHIP session; STUN and TURN in `Link` headers; reading the key log itself (revocations are
the list a deployment configures).

## Licence

Apache-2.0.