open-standards/wai/moqt-conformance/README.md

# moqt-conformance — a relay held to the MOQT interop cases

The corpus of `wai conform relay --suite moqt`: the cases a MOQT relay is held to, each a test
the interop client `joule-moq-interop` runs against the relay at one draft of the transport (22,
18 or 16). `cases.json` lists them, each named as the client names it, with what the relay must
do for the case to pass and the drafts at which it is a test. A record of a run names the corpus
by the digest of `cases.json` (extensions/conformance-results.md §2.1).

| case | the relay passes when |
|---|---|
| `setup-only` | it completes the session's setup at the draft within 2 s |
| `announce-only` | it accepts a `PUBLISH_NAMESPACE` within 2 s |
| `publish-namespace-done` | after a published namespace is withdrawn, the session still answers a `SUBSCRIBE` within 2 s |
| `subscribe-namespace-lifecycle` | a subscriber to a prefix is told of a namespace published under it and of its withdrawal, each within 2 s |
| `subscribe-error` | a `SUBSCRIBE` to a track no one publishes is refused with `REQUEST_ERROR` within 2 s |
| `rendezvous-timeout` | a `SUBSCRIBE` with a `RENDEZVOUS_TIMEOUT` of 500 ms to a track no one publishes is refused with `REQUEST_ERROR TIMEOUT` within 2 s (drafts 22 and 18; draft 16 has no such parameter) |
| `announce-subscribe` | a `SUBSCRIBE` to a track of a published namespace reaches its publisher, and the subscriber receives its `SUBSCRIBE_OK` within 3 s |
| `subscribe-before-announce` | a `SUBSCRIBE` sent before its namespace is published is answered within 3.5 s |
| `wai-receipts` | a publisher's WAI track, its receipts track and its catalog reach a subscriber, every object verified against the publisher's signed receipts: the group receipt's signature, the object's inclusion, the object receipt carried as a property and on the receipts track, the `WAI_RECEIPT` property, the chain's links and the envelope |

The first eight are the MOQT interop runner's test cases, written against
draft-ietf-moq-transport-18; `wai-receipts` is WAI's.

## Running

    wai conform relay --suite moqt --impl moqt://127.0.0.1:4443/moq --tls-disable-verify
    wai conform relay --suite moqt --impl https://relay.example/moq --draft 22

`--impl` is the relay's URL: `moqt://` reaches it over QUIC with the draft's ALPN, `https://`
over WebTransport. `--draft` is the draft the client speaks (18 unless given), and the record
names it in the client's command line. `--tls-disable-verify` is for a relay whose certificate
is its own (`wai-relay --dev`). Each case is one run of the client,
`joule-moq-interop --relay <url> --draft <n> --test <case>`, and its TAP (version 14) is the
case's outcome: the case passes when its test point is `ok`. A point the client skips passes
only at a draft the case lists no test for (the client skips `rendezvous-timeout` at draft 16);
`not ok`, a `TODO`, no point of the case's name, a client that does not run the test or a run
past `--timeout` (60 s a case) fails it. The record is conformant when every case passed.

The client is found as `wai dev` finds its daemons: `WAI_MOQ_INTEROP_BIN`, else beside `wai`,
else on `PATH`. It builds from `joule-code/joule-code-rs`:

    cargo build --release -p joule-moq-interop

The reference relay, `wai-relay --dev` (crate `joule-moq-relay`), passes every case at each
draft.

**Provenance.** Written here, by hand, from the client's test cases (`joule-moq-interop`,
`src/main.rs`), whose names and order `wai-rs`'s tests hold `cases.json` to. No test vector:
each case is run live against the relay.

## Mappings of MoQ drafts

Eleven corpora of mappings sit beside the relay's cases, each a directory with its own README,
and each with a second implementation here, written from its text and sharing no code with
the reference:

| corpus | what | text | second implementation |
|---|---|---|---|
| [`moq-file/`](moq-file/) | a take's tracks as the files of `draft-jennings-moq-file-05` | live-recording §8.7 | `moq_file_verify.py` |
| [`tempo/`](tempo/) | the playout timing of `draft-nandakumar-moq-tempo-00` held to a room's controls | co-watch §15 | `tempo_verify.py` |
| [`v3c/`](v3c/) | a `wai.pc.vpcc` payload as the tracks of `draft-yuyou-moq-v3c-00` | moq-streaming-format §4.5 | `v3c_verify.py` |
| [`switching/`](switching/) | a subscriber's switch between the tracks of a catalog's rendition set, by `draft-ugur-moq-fast-switching-00` | the corpus's README | `switching_verify.py` |
| [`probe/`](probe/) | the probe tracks of `draft-gurel-moq-probing-00`: names, resolution, limits, errors, objects, catalog and measurement | the corpus's README | `probe_verify.py` |
| [`metrics-track/`](metrics-track/) | metrics over MOQT, `draft-jennings-moq-metrics-03`: names, catalogs, data groups, and captures made from a text exposition and held to it | moq-streaming-format §4.8 | `metrics_track_verify.py` |
| [`log-track/`](log-track/) | logs over MOQT, `draft-jennings-moq-log-04`: names, Log Records, and the groups a publisher sends, one a second | moq-streaming-format §4.8 | `log_track_verify.py` |
| [`qlog/`](qlog/) | the MOQT events of `draft-pardue-moq-qlog-moq-events-08`: a log of every event type, and the order a log is held to | moq-streaming-format §4.8 | `qlog_verify.py` |
| [`moq-test/`](moq-test/) | the test tracks of `draft-afrind-moq-test-01`: namespaces, a publisher's limits, every object of a track with its streams and datagrams, and deliveries a subscriber accepts or refuses | moq-streaming-format §4.10 | `moq_test_verify.py` |
| [`test-tracks/`](test-tracks/) | the namespace of the media test tracks of `draft-marzot-moq-test-tracks-00`: sets of tracks, ladders, bitrates, timing and refusals; no media | moq-streaming-format §4.10 | `test_tracks_verify.py` |
| [`mpeg2ts/`](mpeg2ts/) | the `mpeg2ts` packaging of `draft-gregoire-moq-msfts-02`: transport streams built here from a packaged floor, tracks of the draft's four modes with their catalog and one track's receipts, and deliveries a subscriber accepts or refuses | moq-streaming-format §4.11 | `mpeg2ts_verify.py` |

None of them is a case of `wai conform relay`: `cases.json`, and the digest a record names the
corpus by, are unchanged. The interop client also runs the test tracks of `moq-test/` through
a relay (`joule-moq-interop --test moq-test`, in that corpus's README); that test is run only
when named and is not a case of `cases.json` either.