186 lines
11 KiB
Markdown
186 lines
11 KiB
Markdown
# joule-moq
|
|
|
|
MoqT (Media over QUIC Transport) for CommunicationOS broadcast media, with a JWP
|
|
receipt extension: every object carries verifiable provenance and its energy
|
|
figure.
|
|
|
|
## The draft it implements
|
|
|
|
`draft-ietf-moq-transport-22` (1 October 2026), and no other. The draft is chosen
|
|
before the first MOQT byte, by the ALPN `moqt-22` on native QUIC or the
|
|
`WT-Available-Protocols` value `moqt-22` on WebTransport (§6.2); `SETUP` carries no
|
|
version. `MOQT_DRAFT_VERSION` is the conventional code `0xff000016`.
|
|
|
|
Drafts do not interoperate with each other. Deployed relays speak earlier drafts,
|
|
which differ from -22 in their varint, their control streams and their messages, so
|
|
this crate does not talk to them. The codec of -22 is a module of its own
|
|
(`src/draft22/`), selected through `draft::Draft`: a later draft, or an earlier one
|
|
added for interoperation, is a sibling module and a `Draft` variant, and the
|
|
draft-neutral parts (tracks, objects, receipts, the session's state) stay as they are.
|
|
|
|
## What is implemented
|
|
|
|
The codec (`draft22`), every decoder bounded (it reads no byte that is not there and
|
|
allocates nothing from a length before the bytes behind it exist) and every refusal a
|
|
typed `WireError` carrying the draft's consequence: close the session with a code,
|
|
reject the request, or a malformed track.
|
|
|
|
| -22 | module |
|
|
|---|---|
|
|
| §8.1 variable-length integer: 1 to 9 bytes, leading ones, longer forms accepted | `draft22::varint` |
|
|
| §8.2 Location, §8.7 Track Namespace (at most 32 fields, none empty, 4,096 bytes for a Full Track Name), §8.8 rendering | `draft22::names` |
|
|
| §8.3 Key-Value-Pairs (delta types, values up to 2^16-1 bytes), §8.4 and §10 Properties with Immutable Properties, Prior Group and Object ID Gaps and Mandatory Track Properties, §9.1 Setup Options | `draft22::kvp` |
|
|
| §8.9 Authorization Token structure | `draft22::token` |
|
|
| §9 every control message of Table 5, framed by a 16-bit Length | `draft22::control` |
|
|
| §9.20 every Message Parameter, the list each message allows, the rules on repetition; §8.6 Range Filters; §9.20.9 Location Filters; FILL_PARAMETERS | `draft22::params` |
|
|
| §6.3 and §6.4.2 control and request streams, Request IDs by end, §9.1.7 MAX_REQUEST_UPDATES credit, the order of messages on each request stream | `draft22::request` |
|
|
| §11 subgroup streams, object datagrams, fetch streams with End of Range, padding; a whole-stream decoder and an incremental one | `draft22::data` |
|
|
| §12 and §16.11 session, request, PUBLISH_DONE and stream reset codes; §13 grease | `draft22::codes` |
|
|
|
|
On it:
|
|
|
|
- `session`: the draft by ALPN, SETUP options, Request IDs, the receipt extension
|
|
declared by a Setup Option.
|
|
- `transport_quic` (feature `quic`): native QUIC on quinn. A pair of unidirectional
|
|
control streams, each opened by SETUP; a bidirectional stream per request; subgroup,
|
|
fetch and padding streams; object datagrams.
|
|
- `mock`: an in-memory fabric and relay whose objects, receipts and requests cross
|
|
the -22 codec.
|
|
- `carriage`: how this crate's objects and receipts ride -22 objects (below).
|
|
|
|
### Where -22 leaves an encoding open
|
|
|
|
The codec takes one reading of each, and the vector checker takes the same:
|
|
|
|
- FILL_TIMEOUT (0x0A) and RENDEZVOUS_TIMEOUT (0x04) state no encoding; both are
|
|
durations in milliseconds and are read as varints, as the draft encodes its other
|
|
durations.
|
|
- FILL_PARAMETERS is length-prefixed and holds "a sequence of Parameters": read up to
|
|
its length, with no count before them, type deltas starting from 0.
|
|
- End of Range (§11.4.1.2) says "the Group ID and Object ID fields are present"; the
|
|
only such fields of a fetch object are the Group ID Delta and Object ID Delta,
|
|
which bits 0x08 and 0x04 of 0x8C, 0x10C and 0x20C mark. They are read as an
|
|
object's are: absolute on the first entry, and after it a Group ID that moves one
|
|
or more groups in the request's Group Order, with an absolute Object ID.
|
|
- An unknown message, stream or datagram type closes the session; -22 names no code,
|
|
and this codec gives PROTOCOL_VIOLATION.
|
|
|
|
## Carriage of this crate's objects and receipts
|
|
|
|
Object Properties in the range -22 sets aside for an application's own use (§8.4),
|
|
inside the object's Immutable Properties, since a relay must not change them:
|
|
|
|
| type | name | value |
|
|
|---|---|---|
|
|
| `0x3A40` | `MRL_LAYER` | varint: the object's layer, when the header states layers |
|
|
| `0x3A41` | `WAI_RECEIPT` | bytes: WAI `extensions/moq-streaming-format.md` §5's property, laid out below |
|
|
| `0x3A42` | `MRL_TOTAL_LAYERS` | varint, when non-zero |
|
|
| `0x3A43` | `JWP_OBJECT_RECEIPT` | bytes: the object receipt standing alone, in the binary form of WAI `extensions/jwp-receipts.md` §2.7.2 |
|
|
| `0x3A44` | `WAI_ENVELOPE` | varint `1`, when the payload is a WAI envelope |
|
|
|
|
A layer cap on a SUBSCRIBE is an OBJECT_PROPERTY_FILTER on `MRL_LAYER`
|
|
(`messages::mrl_cap_filter`), which needs the peer's MAX_FILTER_RANGES above 0. An
|
|
object's delivery timeout rides as its OBJECT_DELIVERY_TIMEOUT property (§10.2).
|
|
|
|
The `WAI_RECEIPT` value: `content_hash` (32 bytes), `joules_micro` (a -22 varint),
|
|
`acquisition` (a varint length, then UTF-8), `acquisition_evidence` (a varint length,
|
|
then bytes), `group_receipt` (32 bytes), and nothing after.
|
|
|
|
A group's receipts ride its receipts track (moq-streaming-format §4.1): group *g*
|
|
carries one object, the group receipt and the object receipts of group *g* of the media
|
|
track in the binary form of WAI jwp-receipts §2.7.3: `WJR1`, the group receipt's
|
|
canonical bytes (its signing payload in the form signed, then its signature, and for a
|
|
legacy group its parent flag and parent), a u32 count, and each object receipt's body.
|
|
`ReceiptBundle::read` gives the group's link hash computed over the bytes received;
|
|
`read_receipts`, `GroupReceipt::from_binary` and `ObjectReceipt::from_binary` refuse what
|
|
does not read with the §3 step jwp-receipts §2.7.4 names. `tests/wai_jwp_corpus.rs` reads
|
|
and writes every binary vector of WAI's corpus.
|
|
|
|
**Room grants** (WAI `extensions/live-rooms.md` §10.5, `grant_proof`): a grant is public,
|
|
so it is presented with a `WRP1` proof of its principal's key over a challenge the verifier
|
|
chose. Over MOQT the relay's SETUP carries the challenge (`GRANT_CHALLENGE`, `0x3A47`) and
|
|
the client's the grant as an AUTHORIZATION TOKEN beside the proof (`GRANT_PROOF`,
|
|
`0x3A49`), with the context `SETUP //<authority><path>`. `check_proof` is §10.5.4's
|
|
checks, given the grant's digest and principal by whatever read the grant; over HTTP the
|
|
same record rides the `WAI-Room-Proof` header and the challenge `WAI-Room-Challenge`.
|
|
|
|
The receipt extension is declared by the Setup Option `0x3A46`, a varint holding the
|
|
extension's version. -22 has no unregistered range for Setup Options and asks for a
|
|
provisional registration; until one is made the code point is this crate's choice,
|
|
and a peer that does not know it ignores it (§9.1).
|
|
|
|
## Not yet implemented
|
|
|
|
- WebTransport (MOQT over HTTP/3, §6.2.1), a relay with priority fan-out and
|
|
per-object receipts, and the authorization token cache (§8.9) are in
|
|
`joule-moq-relay`, built on this codec; this crate's own session keeps no token
|
|
cache. The browser client is `joule-moq-web`, over this crate built without its
|
|
default feature `runtime` (the codec, the carriage and the receipts, for
|
|
`wasm32-unknown-unknown`).
|
|
- Evaluating filters at a publisher (§3.3), and holding a peer to its
|
|
MAX_FILTER_RANGES: the codec reads and writes every filter and counts their ranges.
|
|
- GOAWAY, delivery-timeout enforcement on real streams and §5.1's scheduling are
|
|
`joule-moq-relay`'s; this crate's QUIC transport has none of them, and its mock
|
|
relay applies delivery timeouts on a caller-driven clock.
|
|
- `draft-ietf-moq-loc` and `draft-ietf-moq-msf` semantics (catalogs, LOC
|
|
properties): their properties ride as any property does.
|
|
|
|
## The vector corpus
|
|
|
|
`tests/vectors/draft-22/vectors.json` holds -22 vectors: varints, every control
|
|
message type, control, subgroup, fetch and padding streams, datagrams, and the
|
|
`WAI_RECEIPT` value; and negative vectors, each with the refusal it must meet.
|
|
`tests/draft22_vectors.rs` generates the file deterministically from typed values
|
|
and holds the crate to it; `JOULE_MOQ_WRITE_VECTORS=1` rewrites it.
|
|
`tests/vectors/draft-22/moqt22_check.py` decodes the corpus independently, from the
|
|
draft's text, standard library only:
|
|
|
|
```sh
|
|
python3 -I tests/vectors/draft-22/moqt22_check.py
|
|
```
|
|
|
|
Each vector has a `name`, a `kind`, its bytes as `hex`, `canonical` (whether this
|
|
crate's encoder makes exactly those bytes) and `expect`; a `uni-stream` vector also
|
|
has `context.group_order` (1 Ascending, 2 Descending), the Group Order of the request
|
|
a fetch stream answers.
|
|
|
|
| kind | bytes | reading |
|
|
|---|---|---|
|
|
| `varint` | one varint | `{"value", "length"}` |
|
|
| `control` | one framed message | the message (below) |
|
|
| `uni-stream` | a whole stream to its FIN | a list of events (below) |
|
|
| `datagram` | one datagram | `{"datagram": {…}}` or `{"padding": n}` |
|
|
| `wai-receipt` | the `WAI_RECEIPT` value | its five fields |
|
|
|
|
The JSON view: byte strings are lowercase hex; a Track Namespace is a list of its
|
|
fields in hex; a Location is `[group, object]`; a Key-Value-Pair is `{"type", "value"}`
|
|
for an even type and `{"type", "bytes"}` for an odd one, with Immutable Properties as
|
|
raw bytes. A message is an object with `"message"` (its name in Table 5) and its
|
|
fields by their names in the draft, lowercased with underscores. A parameter is
|
|
`{"type": NAME}` with `"value"` (a varint or a uint8), `"token"`, `"location"`,
|
|
`"filter"` (a Location Filter's `filter_type` and the fields it has; a Range Filter's
|
|
`set_id`, `property_type` for the property filters, and `ranges` as `[start, end or
|
|
null]`; `null` for Length 0), `"parameters"` (FILL_PARAMETERS) or `"namespace"`.
|
|
Stream events are `{"header": "subgroup" | "fetch" | "control" | "padding", …}`,
|
|
`{"control": message}`, `{"object": …}` (subgroup), `{"fetch_object": …}`,
|
|
`{"end_of_range": {"kind", "location"}}` and `{"padding": n}`.
|
|
|
|
A refusal is `{"error": kind, "consequence": c}`, with `c` one of
|
|
`session:<code>`, `request:<code>` or `malformed-track` (and `property_invalid` for
|
|
the `WAI_RECEIPT` value, which a sink discards). The kinds are `WireError::kind`:
|
|
`truncated`, `trailing-bytes`, `unknown-message-type`, `unknown-stream-type`,
|
|
`invalid-flags`, `unknown-parameter`, `duplicate`, `parameter-not-allowed`,
|
|
`duplicate-filter`, `invalid-filter`, `type-overflow`, `length-exceeds-limit`,
|
|
`too-many-namespace-fields`, `empty-namespace-field`, `invalid-value`, `overflow`,
|
|
`properties-on-non-normal`, `unknown-object-status`, `no-prior-object`,
|
|
`empty-datagram-properties`, `malformed-token`, `unexpected-message`,
|
|
`malformed-track`.
|
|
|
|
## Tests
|
|
|
|
- `cargo test -p joule-moq --features quic --all-targets` runs the codec's unit
|
|
tests, the corpus (`tests/draft22_vectors.rs`), generated round trips and hostile
|
|
input (`tests/draft22_roundtrip.rs`), the QUIC transport over loopback, the mock
|
|
and relay, the staged MoQ trace and WAI's JWP corpus.
|
|
- The independent decoder above runs over the corpus in CI
|
|
(`.github/workflows/joule-code-conformance.yml`).
|