open-standards/joule-code/joule-code-rs/crates/joule-moq-ingest/README.md

# joule-moq-ingest

A gateway between contribution protocols and the MoQ tracks of a WAI room. A standard
WebRTC encoder publishes into a room over **WHIP** (RFC 9725), and an RTMP encoder over
**Enhanced RTMP v2** (`v2-2026-01-31-r2`), single-track or multitrack, over TCP or TLS; the
gateway becomes their tracks' MoQ publisher at the relay and signs every group's receipts as
the first hop. A standard WebRTC player watches a room's tracks over **WHEP**
(`draft-ietf-wish-whep-05`); the gateway subscribes at the relay and verifies every object's
receipt before it hands the frame on. A broadcast automation system drives the gateway's
**SCTE 104** injector, which translates its requests to SCTE 35 and signs them as the room's
cues. It also takes contribution over **SRT** and **RIST** into the same tracks
(`extensions/live-contribution.md`, below), and signs each such link's record of loss,
recovery and latency. The mappings it implements are WAI's `extensions/live-ingest.md`,
`extensions/cue-injection.md` and `extensions/live-contribution.md`.

It is a library (`GatewayServer`, `Gateway`) and a binary, `wai-ingest` (the crate keeps its
name). The relay is `joule-moq-relay` (binary `wai-relay`); the MOQT codec is `joule-moq`'s. The gateway's sessions with the relay speak the draft `[relay] draft` names, offered alone in the ALPN: `22` by default, or `23`, `18` or `16`, bridged to the gateway's -22 model by `joule-moq-relay`'s `transport::bridge`.

`GatewayServer::shutdown` stops listening, ends every session (each signs its record, ended
`deleted`; an RTMP publisher as below), and returns once every task the gateway ran has ended:
every port it listened on is closed by then (HTTP, RTMP, RTMPS, SRT, the injector's, and each
RIST session's, waiting or running), and every connection it accepted. A gateway can start again
on the same ports at once, in the same process. A session that starts once the shutdown has
begun ends at once, `gateway_stopped`; `Gateway::spawn` runs a task the shutdown waits for.
So does one still dialing the relay when the shutdown begins (a dial can last 10 s, past the
5 s the shutdown waits for its tasks): it signs its record, `gateway_stopped`, with nothing
published. A task still running 5 s on is ended; a RIST listener it held is closed all the same
(`Gateway::hold_rist`: whatever binds one for a task of the gateway's hands it there as it
binds it; `contrib::serve_rist` serves a RIST session at its listener so).

`GatewayServer::start` binds every port and makes every check (the RTMPS identity, the
injector's key and grant) before it spawns a task to serve one. A start refused part way
returns its error with every port it bound free and nothing of its running, so the next start
can bind them at once.

## Resources

| resource | methods |
|---|---|
| `/whip` | `POST` an `application/sdp` offer: `201`, the answer, `Location`, `ETag`. `OPTIONS` (CORS, `Accept-Post`), `GET`, `HEAD` |
| `/whip/s/<id>` | `PATCH` an `application/trickle-ice-sdpfrag`: trickle ICE (`204`) or an ICE restart (`200`, a fragment, a new `ETag`); `DELETE` (`200`) ends the session and its tracks |
| `/whep/<room>/<owner>[?layer=<n>]` | `POST` an offer for the owner's tracks: `201`, or `409` with `Retry-After` while the owner is not live. `HEAD` answers with `Link: <urn:ietf:params:whep>; rel="profile"` (WHEP §4.2) |
| `/whep/s/<id>` | `PATCH`, `DELETE`, as for WHIP |
| `/rtmp` | `POST`: `201`, `{"expires_in", "stream_key", "url"}`: a one-time stream key for an Enhanced RTMP publish under the grant |
| `/metrics` | `GET`, `HEAD`: the gateway's counters in the text exposition format ([`wai/wai-metrics`](../../../../wai/wai-metrics)), for the operator at the gateway's own address; it carries no grant |

`/metrics` holds the requests by resource, method and status (`wai_ingest_requests_total`), the
sessions running by protocol (`wai_ingest_sessions`: WHIP, Enhanced RTMP and contribution publish
in, an RTMP connection counting once its publish is admitted; WHEP sessions are the gateway's
fan-out), the WHIP and WHEP sessions opened, the requests refused, the session records signed,
and `wai_ingest_energy_unmetered`, 1: the gateway's own energy in every record it signs is the
unmetered marker, since no meter backs it. No room, principal, grant, session or address is a
label.

Every request but `OPTIONS` and the operator's `/metrics` carries `Authorization: Bearer <token>`.
**The token is a room grant**: a WAI `WRG1` record (`live-rooms.md` §4), base64url without padding.
A grant is public, so every request that carries one also carries a proof of possession of its key,
`WAI-Room-Proof` (`live-rooms.md` §10.5.3), over a single-use challenge the gateway gave in
`WAI-Room-Challenge`: on a `401` and on each response to a request with a grant. Refusals are
RFC 9457 problem details whose `reason` member is `live-ingest.md` §14's code.

## Admission

The gateway holds each room's signed descriptor (`WRD1`) and checks a token against the
room it names, as the relay checks a token against its grants, here by the room's own
signed records: the grant reads and its signature verifies under the room's authority
(`live-rooms.md` §4.3, its field rules, its fourteen-day bound, its role and the room's
ceiling), it is good now, it is not among the room's revocations, its origins include the
gateway's, a single-use grant has opened no session here, and the request's proof passes
`live-rooms.md` §10.5.4 over a challenge held for its endpoint or session
(`joule_moq::grant_proof::check_proof`, through `gateway::check_http_proof`). What fails is `401` with
`WWW-Authenticate: Bearer error="invalid_token"`. A good grant without the power the
request needs is **`403`** with `error="insufficient_scope"` (RFC 6750 §3.1): the publish
power for each media kind a WHIP offer carries, the subscribe power up to the layer a WHEP
request asks for. A session's resource answers only the token that opened it.

`room.rs` reads descriptors and grants from the extension's text; `tests/room_corpus.rs`
holds it to WAI's corpus `wai/live-conformance/rooms/`: every room id and grant digest,
the registered profiles' records, and every refusal of a descriptor or a grant there, but
an exchanged grant's link and the key log, which a bearer token does not bring. It also
runs the corpus's six HTTP proofs through `check_http_proof`, each with its stated
outcome, and presents each accepted proof again as a replay.

## WHIP into the room

- **Tracks.** Per media kind of the offer, `(room, owner, audio|video)` / `0` and its
  receipts track `0_receipts`; and the owner's catalog, `(room, owner)` / `catalog`, with
  `catalog_receipts`. All are published (PUBLISH) before the answer goes back. Audio and
  the catalog at publisher priority 0, video at 1.
- **Frames.** str0m depacketises RTP into frames: H.264 (RFC 6184), VP8 (RFC 7741), VP9
  (RFC 9628), AV1 (its RTP payload format v1.0.0) and Opus (RFC 7587). Each frame is one
  object, one `WAI1` envelope:

  | codec | capability | payload |
  |---|---|---|
  | H.264 | `wai.video.avc` | one access unit, Annex B |
  | VP8 | `wai.video.vp8` | one frame in IVF (`VP80`), its size from the keyframe header |
  | VP9 | `wai.video.vp9` | one frame in IVF (`VP90`), its size from the uncompressed header |
  | AV1 | `wai.video.av1` | one temporal unit in SPEC §5's AV1 framing, the temporal delimiter restored, the size from the sequence header |
  | Opus | `wai.audio.opus.packet` | one packet |

- **Groups.** Video: one group per keyframe (a PLI goes to the publisher before the first
  one, and when a group passes `max_video_group_ms`). Audio: one group per fixed window of
  `audio_group_ms` on the room clock. One subgroup stream per group, written as the frames
  arrive. A track's first group ID is its first frame's room-clock time in microseconds.
- **Time.** The manifest's `live` member carries the frame's room-clock time `t`, its RTP
  time and clock rate, whether it is a keyframe, and on a group's first object the grant
  reference (`live-rooms.md` §4.5). The mapping is anchored at a track's first frame's
  arrival, `t = A + (R − R0) · 10^6 / rate`, and re-anchored at any frame it would place
  more than `max_skew_ms` from its arrival: so **every frame's `t` lies within
  `max_skew_ms` of its arrival at the gateway**.
- **Receipts.** When a group ends, the gateway signs its group receipt over every object
  and publishes it with the object receipts on the receipts track, chained to the group
  before. Every figure is the unmetered marker: no meter backs the gateway. The chain's
  alias is derived per session, so a publisher that reconnects starts a new chain.
- **The catalog** is MSF JSON with WAI's `wai_*` fields, published once every track has
  begun (or a second after the first did) and again whenever a codec or picture size
  changes, and signed like the media, so a relay that checks every track of the room at its
  door forwards it.
- **The end.** DELETE (or ICE lost, idle, a bound passed) ends each open group, publishes
  its receipts, waits for the relay to acknowledge every stream, and sends PUBLISH_DONE
  `TRACK_ENDED` on every track: the relay's subscribers see the tracks end.

## WHEP out of the room

The gateway reads the owner's latest catalog once its receipt verifies, answers the
player's offer with the codecs the tracks carry, and subscribes to each accepted track and
its receipts track from the start of the latest group (the live part from the Next Object,
the rest of that group by a fill), so the player starts at a keyframe. Before a frame
reaches the player, its group's receipts must name its MOQT group and Object ID and the
track's chain, verify under a trusted first hop's key (`whep.trust`; this gateway's own
key by default), chain to the group before, and prove the object's inclusion; the object
must open as an envelope of the track's codec. What fails is dropped and counted in the
session's record; a group whose receipts never come is dropped after `hold_ms`.

**What a WHEP player cannot see:** the receipts. WebRTC carries media only. The gateway
verifies on the player's behalf and records what it dropped and why; a WAI-aware player
subscribes to the same tracks and their receipts tracks over MoQ, through the relay's
WebTransport or WebSocket paths, and verifies every object itself.

A verified group is released when its receipts arrive, at the group's end, so a WHEP
player is up to a video group behind the publisher.

**What the player's sink dispatches first** (live-ingest §9.5). A WAI-aware player writes in
its offer, as SDP capability attributes (RFC 5939, RFC 6871), the WAI capabilities its sink
dispatches, in its order. The gateway reads them with WAI's reference reader (`sdp_caps`,
wai-rs's `src/sdp_caps.rs` compiled here from its source) and forwards, per kind, the owner's
rendition the first of them names at or below the grant's `max_layer`: it answers that
configuration, its payload type alone in the section with `a=acfg`. Where nothing advertised is
published, the floor, the layer the resource names. An offer with none is answered as before,
byte for byte; so is one asking for what this revision does not negotiate
(`caps_unsupported`), with `a=csup`; every other reason of §9.5 is `422`. The session's record
keeps the choice under `detail.caps`.

## Enhanced RTMP into the room

- **Admission.** An encoder has a server URL and a stream key, and cannot answer a
  challenge, so the publisher first POSTs to `/rtmp` with its grant and a proof of
  possession, admitted as a WHIP request is, and gets a stream key: 128 random bits, good for
  one `publish` within `stream_key_ttl_s` (60 s). The `publish` command's stream name is the
  key; the grant is admitted again when it is used. A refusal is `onStatus`
  `NetStream.Publish.BadName` with the reason as its description.
- **The connection** (`rtmp::chunk`, `rtmp::amf0`, `rtmp::session`): the simple handshake,
  the chunk stream with its extended timestamps, Set Chunk Size, Abort, acknowledgements and
  pings, and the commands. The `connect` result declares `capsEx` 0x0F and the FourCCs it
  carries.
- **Tracks** (`rtmp::tracks`, `rtmp::flv`): legacy AVC and AAC, and `avc1`, `hvc1`, `av01`,
  `vp09`, `vp08`, `Opus`, `fLaC` and `mp4a`, through ModEx and multitrack bodies and aggregate
  messages. Each RTMP track is bound to a room track at its first packet: under a grant with a
  layered video profile, the layer its `onMetaData` entry's size and frame rate match; audio
  track 0 only. Each frame is one object: Annex B with the record's parameter sets before a
  key access unit that carries none, the AV1 framing with the sequence header, IVF for VP8 and
  VP9, ADTS for AAC, a FLAC stream. `live.rtp` is the decode time in µs (`rate` 1 000 000) and
  `live.cto` the composition offset; §7's mapping gives `t`.
- **RTMPS** on `rtmp.tls_listen` with a PEM chain and PKCS #8 key (tokio-rustls on the
  workspace's rustls and ring).
- **Shutdown.** A publisher that declared `Reconnect` is sent
  `NetConnection.Connect.ReconnectRequest` (with `rtmp.reconnect_url` as its `tcUrl`, if set),
  its key is good once more, and the session ends when it leaves or after
  `reconnect_grace_ms`.
- **Records.** Each publish ends with a session record (`protocol` `ertmp`) whose `detail`
  counts what was read and not carried, by reason.

## SCTE 104 into the room's cues

`scte` listens on `injector.listen` (SCTE 104's port is 5167) for one automation system at a
time: `init_request`, `alive_request`, and `multiple_operation_message()`s of splice,
time-signal and splice-null requests with avail, DTMF, segmentation, time, tier and private
descriptor requests. Each is answered (`inject_response`, then `inject_complete_response`
once its cues are signed) with SCTE 104's result codes. `scte::scte104::translate` builds each
SCTE 35 section (`scte::scte35`) on the room's 90 kHz clock and the cues it yields;
`scte::Engine` holds each cue until its splice instant, applies cancels and early returns,
drops what live-ads' break rules would refuse, and signs the cue as a `cue` event of the
injector's principal after its injection record (`WIJ1`, a `meta` event carrying the message,
its arrival and the translation's parameters). The events go out on the principal's events
track at the relay, published from the injector's start. The injector's grant must name its
key and permit `cue` and `meta`.

A relay whose door checks receipts names the room's media kinds in its rules
(`[room, "*", "audio"]`, `[room, "*", "video"]`): an events track carries no receipts.

## Bounds

Each is a field of `[limits]`, `[groups]`, `[timeline]` or `[whep]` in the configuration:

| field | default | past it |
|---|---|---|
| `max_whip_sessions`, `max_whep_sessions` | 64, 256 | `503` with `Retry-After` |
| `max_tracks_per_session` | 2 | `422`; and one track of each kind at most |
| `max_bitrate_kbps` over `bitrate_window_ms` | 8 000 over 2 000 ms | stated per section as `b=AS` in the answer (a grant's publish profile lowers it); a session past it is ended |
| `max_frame_bytes` | 4 MiB | the frame is dropped and counted |
| `max_queue_bytes` | 32 MiB | the session is ended (`relay_backpressure`) |
| `audio_group_ms`, `max_video_group_ms` | 200, 2 000 | — |
| `max_skew_ms` | 250 | the track is re-anchored |
| `whep.hold_ms` | 5 000 | the group is dropped (`receipt_missing`) |
| `rtmp.max_sessions` | 64 | one more connection is closed at accept |
| `rtmp.max_message_bytes` | 8 MiB | the connection ends (`message_too_large`) |
| `rtmp.max_bitrate_kbps` over `bitrate_window_ms` | 40 000 over 2 000 ms | the publish ends (`bitrate_exceeded`) |
| `rtmp.connect_timeout_ms`, `rtmp.idle_timeout_ms` | 15 000, 20 000 | the connection ends |
| `rtmp.stream_key_ttl_s` | 60 | the key is refused (`stream_key_expired`) |
| `injector.max_pending` | 256 | a message is answered 124 |

`gateway.example.toml` shows every field. It holds no secret: the relay token and the
signing key are files, and a room is its public, signed descriptor.

```text
wai-ingest --config gateway.toml      # JOULE_MOQ_INGEST_LOG sets the log level
```

Every listener is loopback by default: put a TLS front before the HTTP resources (RFC 9725
§5) and set `media.bind_ip` to an address clients reach. `/metrics` is the operator's: a request
that came through a front, one carrying `X-Forwarded-For` or another of the fields a front sets, is
answered as a path the gateway does not have, and a front that sets none of them is configured not
to pass the path on. Read at the gateway's own address, it carries no CORS field.

## The WebRTC stack

[str0m](https://crates.io/crates/str0m) 0.25.0, pinned, MIT OR Apache-2.0, with
`default-features = false` and its `rust-crypto` backend: DTLS by str0m's own `dimpl`
0.9, SRTP and the rest on RustCrypto, whose elliptic curves (p256 and p384 0.14) run on
crypto-bigint 0.7. The stack builds one C library, aws-lc: each session's self-signed DTLS
certificate is made by `rcgen` on `aws-lc-rs`, which `dimpl`'s certificate feature takes. It
is sans-IO: each session here owns a UDP socket and drives its `Rtc` from a tokio task,
feeding it datagrams and the time and draining its output after every change. It
depacketises and packetises H.264, VP8, VP9, AV1 and Opus at the frame level (AV1 is a
hidden codec in its public API; this crate's tests take AV1 through it end to end), supports
ICE lite and full ICE, ICE restarts, NACK and keyframe requests. Every crate new to the
lockfile is MIT, Apache-2.0 or ISC.

## Uncompressed essence (SMPTE ST 2110)

WAI `extensions/essence-contribution.md`. The readers (`st2110`: the description, RTP and its
sequence, video rows into frames, PCM into blocks, ancillary data into records and the SCTE
104 messages in it) are in every build, with their corpus (`wai/live-conformance/st2110/`,
`joule-moq-ingest-st2110-vectors --out|--check`). The session is built with the feature
`st2110`, which brings the encode stage and the one encoder the crate links (`rav1e`,
BSD-2-Clause, without its assembly: the fallback); the real-time encoders are loaded at run
time and are no build dependency:

| resource | what it does |
|---|---|
| `/st2110` | `POST` with the grant and its proof and `{"sdp": "<the description>", "source": "<a capture's digital source type, optional>"}`: `201`, the session and, for each stream, the UDP `port` to send it to, its clock class and its `TSMODE`. Refusals: a description's own reason (`sdp_*`, 422), `essence_streams_unsupported`, `essence_format_not_carried`, `essence_outside_floor`, `multicast_not_permitted`, `session_conflict` |
| `/st2110/s/<id>` | `DELETE` ends the session |

A whole frame goes onto the video floor as `wai.video.av1`; a frame with a sample missing,
or one that found the encode stage's queue full, is counted and not published. Sound goes
out as `wai.audio.flac`, every sample as it arrived. Each object carries its capture instant;
SCTE 104 messages in the ancillary data go to the injector; every other ancillary data
packet is counted as not carried. The session's record (protocol `st2110`) holds each
stream's statement and every count, the encoder and its stage's counts, and an origin claim
a track where a `source` was stated. The gateway holds no reference clock and claims no
lock. `[st2110]` in `gateway.example.toml` lists the settings; a multicast group is joined
only with `multicast = true`.

### The encode stage

`st2110::encode`: one stage ([`Stage`]) and one trait ([`Backend`]) with three encoders
behind it, chosen by `[st2110] encoder` and probed when the gateway starts.

| `encoder` | what encodes | where it comes from |
|---|---|---|
| `va` | the GPU's AV1 encode engine through VA-API (`libva.so.2`, loaded at run time; MIT), on the render node `va_device` | the host's driver; Linux |
| `svt-av1` | SVT-AV1 in its real-time mode (`libSvtAv1Enc.so.4`, or the path `svt_av1_library`, loaded at run time; BSD-3-Clause-Clear with the AOM Patent License 1.0). The adapter holds the layouts of 4.2 and refuses another version | the operator's build of it |
| `rav1e` | the pure-Rust encoder this crate links, at its fastest preset, with tiles cut for the host's threads | every build: the fallback |
| `auto` | the first of those, in that order, that passes its probe | |

- **Asked for and not here is a refusal.** With `encoder = "va"` or `"svt-av1"` on a host
  without it, the gateway does not start and says why. With `encoder_fallback = true` it
  starts on the next encoder and says so in its log, in each session's answer (`encoder`:
  `backend`, `asked`, `fallback`) and in each session's record. No encoder runs under
  another's name.
- **The probe.** Each candidate encodes a few computed pictures at 8 and at 10 bits; its
  first temporal unit must hold a sequence header within the floor (Main profile, Main
  tier, level 4.1 or below) and state the picture's size. One that does not is refused
  with what was found. `POST /st2110` does the same at the stream's own size and depth
  before it answers (`essence_encoder_refused`), and every keyframe of every session is
  held to the same check: a session whose encoder leaves the floor ends
  `encoder_outside_floor`, that unit unpublished.
- **The picture's size.** A hardware engine may code frames larger than the picture, to
  its own block alignment: the one measured codes 1920 × 1082 for a picture of
  1920 × 1080. The VA-API adapter states the picture's size as the render size in every
  frame header (`encode::obu` writes the sequence header and the start of each frame
  header, which that interface leaves to the application, and reads both back from what
  the driver made); the stage refuses a stream whose keyframes do not state the picture;
  the record's `encoder.stage` has `picture` and `coded_frame`.
- **What the gateway says at start**: the encoder, its version and where it was loaded
  from, every probe's outcome, and how long computed pictures of 1280 × 720 and
  1920 × 1080 took at its fastest setting on this host (`calibrate_ms`; a start-up
  estimate, not a measurement of a stream).
- **The real-time budget.** The stage runs on its own thread behind `encode_queue` frames.
  It converts 4:2:2 to 4:2:0, asks for a keyframe every `key_interval_ms` of the stream's
  timestamps, times every frame against the frame interval, and moves the encoder along
  its rungs: faster when a frame takes more than 85% of the interval or two frames wait,
  straight to the fastest when a frame was dropped, and toward the better picture only
  after a hold with the queue empty and under 45% of the interval used. SVT-AV1's rungs
  are its presets from `svt_av1_preset` to 13, changed mid-stream with no keyframe (with
  `svt_av1_preset = 0` the first is chosen for the stream: 9 up to 1280 × 720 at 30 frames
  a second, 11 up to 1920 × 1080 at 30, 12 above); `rav1e`'s are quantizers, each a new
  sequence; a hardware engine has one. `encode_queue = 0` is the frames of two fifteenths
  of a second, at least 4: a keyframe takes several frames' time, and the queue absorbs it.
- **The drop policy.** A frame that arrives while the queue is full is not encoded.
  Quality is lowered first. Every such frame is counted
  (`not_published.frames_encoder_behind_not_published`, and the stage's
  `frames_not_encoded_queue_full`), and no encoder here drops a frame of its own.
- **The loops.** `st2110::convert`: 4:2:2 pgroups at 8 and 10 bits into planes, a
  word-at-a-time reader for every other pgroup, and chroma at half height by the rounded
  mean of two rows; each held bit for bit to a reference that says the same a sample at a
  time, at every shape.
- **The receive buffer.** Each stream's socket is asked for `receive_buffer_octets`,
  halving the ask while the host refuses it; the host's limit bounds what it gets
  (`net.core.rmem_max` on Linux, `kern.ipc.maxsockbuf` on macOS), and the session's answer
  and record state it, the record in time at the stream's rate too. A 1920 × 1080 stream at
  30 frames a second is about 109 000 datagrams a second, at 60 about 218 000: a default
  buffer holds under a millisecond of them.
- **The way in** (`st2110::rx`). Each stream is read by a thread of its own that does
  nothing else, into batches through the session's one bounded queue. On Linux one
  `recvmmsg` reads up to 64 buffers, `UDP_GRO` lets the kernel hand over a run of datagrams
  of one size as one buffer (cut back here at the size it states), `SO_TIMESTAMPNS` stamps
  each and `SO_RXQ_OVFL` counts the buffers a full one dropped. Elsewhere one `recvmsg` a
  datagram with `SO_TIMESTAMP`; on macOS the thread asks to be favoured
  (`QOS_CLASS_USER_INTERACTIVE`). The readers, and so the sequence, loss and completeness
  rules, are unchanged. The system calls are in `st2110::sock`, each with its SAFETY note.
- **Arrivals against the sender's type** (`st2110::timing`). Where a video stream's
  description names an ST 2110-21 type (`TP`), its arrivals are read against that type's
  Network Compatibility Model and the record counts the frames outside it; it is a reading
  at the gateway's socket, not the sender's compliance (`sender_type_checked` stays `"no"`).
- **The test sender** (`st2110::pace`): ST 2110-21's linear schedule, four packets at an
  instant and never sooner than the model drains them, by `sendmmsg` with `UDP_SEGMENT` on
  Linux and one `send` a datagram elsewhere. `tests/essence_rx.rs` holds the way in to
  1280 × 720 at 30 and 1920 × 1080 at 30 and 60 frames a second without loss, on request.

`examples/st2110_profile.rs` prints where a frame's time goes on the host it is run on
(reader, conversion, encode, packaging). `tests/essence_rt.rs` holds a real session to its
frame rate: 1280 × 720 and 1920 × 1080 at 30 frames a second for 20 seconds with each
real-time encoder the host has, failing on one frame dropped behind the encoder, and
refusing to measure on a loaded host. Every temporal unit is decoded by the host's libdav1d
(`libdav1d.so.7`, loaded at run time; BSD-2-Clause) and held to the picture sent, and the
first frames a second time by `wai-av1`, this repository's own decoder, sample for sample
against the first decode. What each encoder held on which host, dated, is in the
extension's §12.

## The telephony bridge (SIP and RTP)

WAI `extensions/telephony-bridge.md`. The readers and the user agent's answers (`telephony`:
SIP messages, digest authentication with MD5 written from RFC 1321, offers and answers, RTP,
G.711, telephone events, the order and form of every answer) are in every build, with their
corpus (`wai/live-conformance/telephony/`, `joule-moq-ingest-telephony-vectors --out|--check`).
The bridge is built with the feature `telephony` and started by `wai-ingest` where
`[telephony] enabled = true`; it links nothing the crate did not already have.

- **One peer.** A SIP trunk or a PBX at `peer`, over UDP, TCP or TLS (`sip_udp`, `sip_tcp`,
  `sip_tls`; port 0 takes one the host gives, and the bridge logs it). Requests from any other
  address are dropped and counted. The bridge's own requests go to the peer over UDP or TCP. A
  dialog's `Contact` is the listener of the transport it was made over (`;transport=tcp` over
  TCP, a `sips:` URI over TLS), so the peer's requests in it come back the same way.
- **A call into the room.** A number maps to grants (`[[telephony.number]] grants`), or to PIN
  entry (`pin = true`, then `[[telephony.pin]]`, the PIN read from a file). The call publishes
  one audio track of the grant's principal: G.711 as `wai.audio.flac` (every character's
  sample, 20 ms an object, 100 ms groups), Opus as `wai.audio.opus.packet` as it came, each
  object stamped with its capture instant, every group receipted as the first hop.
- **The room back.** Each room's mix subscribes to the tracks its calls hear (the other calls,
  and `hear_owners`), verifies every object against its receipt, decodes FLAC at 8 kHz, and
  every 20 ms sends each call the sum less its own, in the call's law.
- **Placing a call and registering.** `Bridge::place_call` dials the peer; with
  `[telephony.outbound] register = true` the bridge registers and keeps its registration.
- **The record** of each call (protocol `telephony`): its statement, its RTP counts by reason,
  the jitter buffer's, what it heard of each track, its origin claim where `source` is set. No
  record says who called unless `caller_in_record = true`; none holds a PIN or a password.

`tests/telephony_bridge.rs` runs on loopback with the tests' own SIP peer and RTP callers
(`tests/common/telephony.rs`): two calls hearing each other through the room, sample for
sample, their tracks verified at a relay subscriber; credentials, a replayed digest and a PIN;
the planted faults over the wire, among them a call from an unconfigured peer, each with its
reason and the call going on after it; Opus carried as it came; calls over TCP and TLS; a call
the bridge places, challenged, and its registration. `tests/telephony_rt.rs` (ignored unless
asked for) holds one call to real time for 30 seconds both ways and carries 64 calls at once,
the default `max_calls`.

`tests/telephony_interop.rs` (ignored unless asked for) holds the bridge to an outside SIP user
agent: baresip 4.13.0 with libre 4.13.0 (BSD-3-Clause), built from their pinned sources and
named by `TELEPHONY_BARESIP` (WAI decisions D58 has the hashes and the build). It refuses to run
unless loopback is its only network interface: run it in a namespace of its own, as its header
shows. Three calls: the softphone calls in over UDP and over TCP (a PIN entered as its own RFC
4733 events), answering a SHA-256 digest challenge, and is called by the bridge; each time its
sound is its track, verified sample for sample, the room's other caller hears it exactly, and
it hears the room.

## SRT and RIST

| resource | methods |
|---|---|
| `/srt` | `POST` with the grant and its proof: `201`, a one-time session as JSON: its `stream_id`, `passphrase`, `pbkeylen`, the listener's `latency_ms`, `host` and `port`. With `{"call": "ip:port", "passphrase": …}`, the gateway calls the publisher's SRT listener, where `[contrib.srt] call_allow` lists it |
| `/rist` | `POST` with the grant and its proof, `{"profile": "simple"|"main"}`: `201`, the session's `port` (Simple: an even port and the one above), its `cname`, its `buffer_ms`, and for the Main Profile the PSK `passphrase` and its `aes` |
| `/srt/s/<id>`, `/rist/s/<id>` | `DELETE` ends the session, or one not yet connected; `GET`, `HEAD` |

The SRT listener (`[contrib.srt] listen`) checks each caller's stream ID
(`#!::r=<room>/<owner>,m=publish,s=<session>`) against the sessions it gave, and refuses with
the codes of live-ingest §13 (1401, 1402, 1403, 1404, 1405, 1409) and the draft's (1010 for a
wrong passphrase, 1011 for none). A RIST session's port takes the first flow whose RTCP CNAME
is the session's. The link's transport stream is demultiplexed: the first H.264 or HEVC
stream and the first AAC stream go to the video and audio tracks as WHIP's frames do, one
access unit (one ADTS frame) an object, the PTS on the room clock; a PES missing a packet is
dropped whole. The catalog declares the owner's `link` track.

**The link record** (`linkrec`, `WLK1`): every `[contrib] record_interval_ms` and when the
session ends, the gateway signs a part: the sequence numbers passed, each delivered at the
first try, recovered, dropped or outstanding, the loss runs, requests, retransmissions, round
trips and recovery times, chained to the part before and naming each media track's receipt
chain head. Each part goes out on the `link` track with its receipts, is kept
(`Gateway::link_records`) and is appended to `[summary] link_out_path`. The corpus
`wai/live-conformance/links/` is written by `joule-moq-ingest-link-vectors`.

RIST is built only with the feature `rist` (`joule-contrib`'s README, "Patents").

## The session record, and live metering

Each session ends with a record of what it carried, signed by the gateway's key
(`record.rs`, format `joule-moq-ingest/session/1`): per track and direction the objects and
bytes, the drops by reason, subscriber sessions and their time by layer class, the grant
with its ceilings and the bytes under it, and the gateway's energy as the unmetered marker.
`Gateway::records` keeps them and `summary.out_path` appends them as JSON lines.

These are the counts of the per-hop session record `WMH1` of WAI's live metering, kept in
its shape, role 3 (ingest gateway). Once that record lands, a `record::HopSink` that writes
`WMH1` turns each record into its track entries, subscriber time and ledger entry and signs
it with the same key, which the room's key log lists under `live/ingest`; nothing else in
the gateway changes.

## Tests

All on loopback, every wait bounded, the relay checking every receipt at its door:

| test | what it shows |
|---|---|
| `whip_publish` | A WHIP publisher (str0m as the client) sends three seconds of H.264 and Opus. A subscriber at the relay receives every group of both tracks, verifies every receipt under the gateway's key and every chain link, and gets the frames' bytes as sent; one group per keyframe, every audio group within its window; DELETE ends the tracks with PUBLISH_DONE; the session's record verifies. |
| `whep_play` | Before the publisher is live a WHEP POST is `409` with `Retry-After`. Then a WHEP player (str0m) gets the publisher's frames, byte for byte, from the keyframe of the group it joined to the last group closed; the WHEP session ends with the tracks, nothing refused. |
| `whep_caps` | An Enhanced RTMP publisher sends AV1 at layer 0 and H.264 at layer 1 (the `camera-720` profile) with Opus; WHEP players state their sinks' capabilities. H.264 first: the answer is its configuration, `a=acfg` and one payload type, and the player gets layer 1's frames byte for byte from a keyframe. AV1 first: layer 0 in AV1. No attributes, and a required option the gateway does not support: the floor, no `a=acfg` (the second with `a=csup`). Attributes that do not hold together: `422` with the reason. Each record keeps the choice. |
| `auth_and_ice` | No token `401`; a token that is not a grant `401 invalid_token`; a viewer's grant `403 insufficient_scope`; a grant past its window, one of another room; `415`; `422`; WHEP without subscribe or above `max_layer` `403`; discovery and CORS. `PATCH` without `If-Match` `428`, stale `412`, trickle `204`, another principal's grant, its own key proved, `403`; a `PATCH` or `DELETE` presenting the grant without a proof `401 pop_missing`; an ICE restart (`If-Match: *`) `200` with new credentials and a new `ETag`, ICE connecting again, and every frame before and after it at the relay. |
| `codecs` | VP8, VP9 and AV1 each through WHIP: every frame whole, one group per keyframe, the size each framing states. |
| `room_corpus` | The room reader against `wai/live-conformance/rooms/`, and its six HTTP proofs of possession through the gateway's proof step, each accepted proof also refused as a replay. |
| `ertmp` | A publisher of our own over RTMP and RTMPS: legacy AVC and AAC; enhanced HEVC with composition offsets and Opus; AV1 with FLAC and VP9 with AAC, a ModEx nanosecond offset; multitrack layers bound by the grant's profile; every refusal of a stream key; the reconnect request at shutdown. Every frame reaches a relay subscriber in its payload framing, at its mapped time, every receipt verified. |
| `scte104_injection` | An automation system of our own over TCP: init, alive, a second connection refused 110, splices with and without pre-roll, a cancel, time signals, refusals with their result codes; the chain verifies and every cue is its message's translation; a relay subscriber reads the chain from the events track. |
| `conformance_corpus` | `wai/live-conformance/ertmp/` and `wai/live-conformance/injection/` regenerate byte for byte; the texts' reason tables list the codes reported. |
| `ertmp_ffmpeg` (ignored; `--ignored` where a local encoder is present) | A local encoder as the test client: AVC with B-frames and AAC, HEVC and AVC on the platform's media engine, HEVC with Opus, AV1 with FLAC, VP9 with Opus; every frame decoded from what the relay carries. |
| `contrib_srt` | An SRT publisher, through a path that drops datagrams each way, into the gateway: a relay subscriber gets every H.264 access unit and AAC frame, every receipt verified; the link record's parts verify as a series, show the losses recovered and name each track's receipt-chain head as the subscriber found it, and are on the link track. Every listener refusal with its code; the gateway calling an encoder's listener with HEVC; 20 Mbit/s with groups of megabytes through the relay's door. |
| `contrib_rist` (feature `rist`) | A RIST Simple Profile flow and a Main Profile tunnel under PSK into the gateway, received whole; another CNAME opens no session. |
| `shutdown_ports` | From a task on a runtime of one worker, the gateway shut down and started again on the same ports many times: when the shutdown returns, its HTTP, RTMP and SRT ports and a RIST session's ports (waiting or running, feature `rist`) bind at once, the HTTP connection it kept alive and an RTMP connection mid-handshake are closed, and the SRT and RIST sessions that ran have signed their records. A RIST session whose relay does not answer its dial signs its record, `gateway_stopped`, and its ports bind at once; so do a RIST listener's held by a task the shutdown ends past its bound. |
| `start_ports` | From a task on a runtime of one worker, a start on fixed ports refused part way (the RTMP, RTMPS, SRT or SCTE 104 port held, or the RTMPS identity spoiled): when it returns, every other port binds at once, and a start with nothing held comes up on all of them. |
| `link_corpus` | Every vector of `wai/live-conformance/links/` with its stated outcome. |
| `contrib_interop` (ignored) | The reference SRT caller and RIST sender into the gateway, with an encoder's H.264, HEVC and AAC through lossy paths (tools named by `WAI_SRT_LIVE_TRANSMIT`, `WAI_RISTSENDER`, `WAI_FFMPEG`, `WAI_FFPROBE`). |
| `contrib_bench` (ignored) | The gateway's throughput and energy on an SRT link, the sender in a process of its own. |

## Not implemented

`vvc1` over Enhanced RTMP; a stream key another gateway accepts after a reconnect request;
SCTE 104 in a video signal's ancillary data (`cue-injection.md` §10); SRT and RIST egress; a
WHEP counter-offer;
a player's keyframe request carried upstream; publishing into a slot; simulcast within one
WHIP session; STUN and TURN in `Link` headers; reading the key log itself (revocations are
the list a deployment configures).

## Licence

Apache-2.0.