# http-conformance — HTTP delivery

The conformance corpus of [http-delivery](../extensions/http-delivery.md) (§14): objects
served as `<cid>.wai` with their `WHP1` head proofs (§6) and ETag sidecars, and cases
for §4–§7 — the head, the head proof, the response checks, the plans, selection on the
head and on the whole envelope, the byte-range procedure with its fallback, the
`codecs` grammar, union, floor and support; and for §8.6, §10.4, §11 and Appendix F the
WAI playlists beside their classical ones, their receipts, the live edge, and the
reference adaptation policies. Two implementations check every case and must agree:

- `wai_http_vectors verify .` (also `wai http vectors verify .`), in `wai-rs`, built
  with the features `http` and `video_manifest` (the playlist receipts);
- `python3 http_verify.py .`, written from the extension text and SPEC §7.1, sharing no
  code with `wai-rs`. It needs only the Python standard library, and reads
  `../claim-conformance/_blake3.py` (BLAKE3 from its specification),
  `../claim-conformance/_ed25519.py` (WAI's strict Ed25519 rule, SPEC §7 *Signatures*,
  written from RFC 8032; it checks the playlist receipts) and
  `../registry/capabilities.json` (the published registry export).

**Provenance.** Generated here by `wai_http_vectors`, from a staged-delivery case and from the frozen sources `src/SOURCES.json` lists, each encoded once by an encoder of its format: pictures of the `int_mlicpp` vector's generated 64×64 pattern, AV1 colour-bar test patterns, and a clip of a square moving over a gradient. The test patterns and the clip are generated content whose generators are not in this tree. None is a recording or a photograph.

## Files

| path | what |
|---|---|
| `objects/<cid>.wai` | each object, named by its content hash (§5.2) |
| `objects/<cid>.wai.hp` | its head proof (§6.2) |
| `objects/<cid>.wai.etag` | its `ETag` value, `"<cid>"` (§5.3) |
| `objects.json` | each object's id, cid, path, `n`, `h`, `codecs` element and what it covers |
| `bad/O6.wai` | O5 with one byte after it: not one envelope (§5.1) |
| `cases/*.hp` | head proofs that are wrong in one way each |
| `cases.json` | one case per line: `{name, op, input, expect}` |
| `src/` | the frozen sources, with their SHA-256 in `src/SOURCES.json` |
| `blake3_vectors.json` | a subset of the published BLAKE3 test vectors (CC0), which `http_verify.py` checks itself against first |
| `http_verify.py` | the independent verifier |
| `falsify/*.patch` | defects that `wai_http_vectors verify` must catch (below) |
| `src/live/`, `src/players/`, `src/playlists/` | frozen sources of the playlist side: the integer video payloads of O12 and O13, the classical playlists the WAI ones sit beside, and the player fixture's classical master as authored |
| `playlists/o9/`, `playlists/o10/`, `playlists/o11/` | WAI media and multivariant playlists (and MPDs) over O9 (and over O9 with its second segment a gap, which `wai-gap.mpd` leaves out), O10 with its layers, and O11, each beside its classical playlist in `src/playlists/`; O9's MPDs are on the timeline of `src/playlists/o9-classical.mpd` (90 kHz, written with `@r`, beside an audio set on its own timescale) |
| `live/` | four revisions of a low-latency WAI playlist over O12 beside `src/live/classical-r*.m3u8`, and its multivariant playlist; the parts are listed under aliases `720/<sequence>.<part>.wai` with their cids (the alias files are not in the corpus: the cases serve the objects under them, and `wai http origin --root . --live-edge live/wai-720.m3u8=live/wai-r0.m3u8,live/wai-r1.m3u8,live/wai-r2.m3u8,live/wai-r3.m3u8` replays the edge over HTTP, blocking reloads and hinted parts included) |
| `players/` | the player fixture: `classical/` (frozen) is a playable classical presentation, `master.m3u8` its master as published with the session-data line, and the WAI multivariant and media playlists and MPD over O13 |
| `*/mr/<manifest_hash>.json` | each WAI media playlist's receipt (§8.6, Appendix H), sealed with the corpus's test key (seed 32 bytes of `0x57`, signer `did:example:http-vectors`; it attests nothing) under a grant of no ceilings, O10's under a test grant with both ceilings; the live revisions are chained |
| `cases/live-r2-orphan.json` | the third live revision sealed with no parent: a receipt that verifies and does not chain |
| `*/mr/<receipt_hash>.json` | each receipt again under its receipt hash, where a client walking back from a later revision fetches it by its link (§8.6) |
| `cases/other-key/` | the third live revision's receipt chained to the second by hash and signed by another key (`r2.json`, and in `mr/` under its receipt hash), and the fourth revision's receipt chained to it (`r3.json`) |
| `cases/relist/` | the third live revision listing the alias of part 0 of segment 1201 with part 1's object (`wai-r2.m3u8`), sealed and chained to the second (`r2.json`, and in `mr/`); the fourth revision's receipt chained to it (`r3-after.json`); the fourth revision listing segment 1201's object at sequence 1200 (`wai-r3.m3u8`, `r3.json`); the third revision listing that alias as `./720/1201.0.wai`, with part 1's object and with its own (`wai-r2-dot*.m3u8`, `r2-dot*.json`); and the fourth revision withdrawing segment 1200 as a gap (`wai-r3-gap.m3u8`, `r3-gap.json`) |
| `cases/walk-substituted/`, `cases/walk-invalid/` | parents a walk fetches by their links: the second revision's receipt served under the third's receipt hash; and the third revision changed after it was signed, served under its receipt hash, with the fourth revision's receipt chained to it (`r3.json`) |
| `cases/sig-*.json` | the O9 playlist's receipt re-signed at the edges of Ed25519 (Appendix H): the identity key, canonical and respelled `y` = p + 1, under `(R, S) = (B, 1)`; under the key B (scalar 1), `R` the identity, canonical and respelled, with `S` = k, so the cofactorless equation holds point for point; and under the key B a signature the strict rule accepts (`sig-key-base-point.json`) |

## Objects

| id | object | covers |
|---|---|---|
| O1 | `WAI1`, the pinned `wai.neural.int_mlicpp` vector of `wai-rs/tests/vectors/int_mlicpp/` | a `WAI1` head; a proof that covers the whole object |
| O2 | `WAI2` image: that pinned rendition, AVIF, JPEG XL and PNG of the one picture it decodes to | pins: held, not held |
| O3 | `WAI2` video: AV1 at two rates, HDR10+ metadata bound to the higher by `companion_of_id` | range plans; a companion; tampering |
| O4 | `WAI2` whose first selectable entry states no `sha256` | `component_digest_absent`: the whole object, the same pick |
| O5 | a small `WAI1` | fetched whole (§5.4) |
| O6 | O5 with a trailing byte (`bad/`) | `object_length_mismatch` |
| O7 | `WAI2` whose first entry states a wrong `sha256` | the skip on both paths |
| O8 | `WAI2`: an AVC base, an LCEVC enhancement bound to it, an AV1 alternative | classes; the refinement deadline (§10) |
| O9a–O9c | three segments carrying {AV1, integer learned video}, {AV1}, {AV1} | union and floor (§4.2) |
| OD | Appendix D's envelope | Appendix D's numbers, with `h` and without |
| OH | a `WAI2` whose head spans several chunks | covered subtrees past the first chunk |
| O10, O10L1, O10L2 | the staged-delivery FLAC base of `../staged-conformance/cases/audio_flac_rate_ladder/` and its two layers | layer objects (§10.4): never dispatched, fetched whole at refinement class, described after `+` through `refines.base` |
| OS, ON, OW | an AVC base with an LCEVC enhancement nearly four times its size; the enhancement's digest right, absent, wrong | a refinement never moves its base (§7, §10.3); on a clock (`timed_*`), the base ready before its refinement, the deadline from the response that verified it, and a refinement's whole-object `200` that never reaches the base (§10.1, §10.2) |
| OT | two renditions the first request holds whole | bytes changed in the first response are not reused |
| OF | a `WAI1` naming a record beside a payload fallback | the union lists what SPEC §4 dispatches |
| OE1, OE2s, OE2t | an encrypted `WAI1`; `WAI2`s whose `security` is a string, `true` | SPEC §6.4 on both containers |
| OR | a refinement component whose `refines` is `{"base": "b"}`, the pick's id and nothing else | the envelope does not conform, and the refinement, whose `refines.base` read on its own names the pick, is reported dropped (`components_nonconforming`, §7 step 4) |
| OPf, OPe, OPw | a `WAI1` naming `wai.text.zstd` whose `refines` is `false`, `{}`, and well formed | a payload carrying `refines` is an ordinary object: dispatched, described by its capability, `not_a_layer` when listed as a layer (§10.4) |
| ORN | an AV1 base most of the object and a refinement component stating no `sha256` | the refinement dropped (`component_digest_absent`) on the whole path and the range path alike |
| ONC, OCF | a base most of the object, an HDR10+ companion, and a refinement with its `sha256`; the companion bound to nothing (ONC) or to the base's capability (OCF) | ONC does not conform, so its refinement is dropped (`components_nonconforming`) and never requested; every component of OCF conforms, so its refinement is fetched and delivered; on the whole path and the range path alike (§7 step 4) |
| ONM, ONX | ONC's base and refinement, the refinement's `refines` the base's id alone (ONM) or `x`, no entry's (ONX) | ONM's refinement is reported dropped (`components_nonconforming`); ONX's names no entry, so it is no refinement of the pick and is not reported |
| `bad/OK.wai`, `bad/OKN.wai` | a string key id; a numeric `kind` | the manifest's member types |
| O11a, O11b, O11bL1 | the staged-delivery pair of `../staged-conformance/cases/live_multi_base/`: two `WAI1` int_motion bases of two frames each, and a layer over the second | `#WAI-LAYER` after one segment only; the layer joined to its own base in the union; depth over a base with one layer |
| O12s0, O12s1, O12p0_00 – O12p1_11 | a low-latency set: two segments of 3.84 s, each its own object, and their twenty-four 0.32 s parts, each its own object that decodes alone | Live-P adjacency, parts listed under aliases with cids, the live revision chain, parts verified against the cid the first revision listing them lists |
| O13a – O13c | the player fixture's WAI segments: one second each of the clip the classical presentation in `players/classical/` carries | a WAI playlist beside a playable classical one |

O10 is the layer set of the byte-range side, so the playlist side's objects take the
next numbers: O11 is a layered sequence and O12 the low-latency set. The payloads of
O12 and O13 are `wai.video.int_motion` clips of a synthetic picture, encoded once by
this tree's integer video encoder, frozen in `src/` and checked to decode to the
encoder's reconstruction; every frame of each clip is distinct, so no two objects are
the same bytes.

The HDR10+ block, the AVC bases and the LCEVC enhancements are synthetic: they carry
the right start codes and decode to nothing, which transport does not need. OD's and
OT's payloads are byte patterns, as are OE2s', OE2t's, OR's, ORN's, ONC's, OCF's,
ONM's and ONX's. Every other payload is real: the integer renditions are
the committed vectors; the images and the AV1 streams are in `src/`, encoded once
outside this repository, and the AV1 streams were checked to decode.

## Operations

| op | input | expect |
|---|---|---|
| `parse_head` | an object (or `file`), optionally a `prefix` length, an `envelope_len` and a `patch_object` | `need`, or the head's version, lengths, `covered_len` and component ranges, or `error` |
| `head_proof` | an object, optionally a `proof` file and a `patch_object` | `covered_len`, `n_cv`, `envelope_len`, or `error` |
| `plan` | an object, the `wanted` entries with their classes, the `held` ranges, a `threshold_permille` | `requests` with their classes, or `whole_rest` and why |
| `response` | the request, status, `Content-Range`, `Content-Encoding`, body length and object length | `ok` with the bytes held, or `error` |
| `select` | an object and a sink: `advertises`, `held` (parameter-set files), `order` | `pick` and `skipped`, or `inert` |
| `fetch` | an object, a `fragment`, a sink (with `applies`, `declared` and `layer_base`), a `tamper` | the pick, how it was verified, why the whole object if it was, body bytes, requests, companions, refinements, abandoned refinements, skips and the component's SHA-256, or `error` |
| `fetch_timed` | an object, a sink, a `deadline_ms`, and a `script` of steps each at a time on the caller's clock: `base` (answer every base-class request, and those that follow), `base_once` (those outstanding now), `refinement` (every outstanding refinement request, by its range or by a `200` carrying the whole object with the byte at `flip` changed), `tick` | the step after which the base was first ready and its time, the base then and at the end, what each tick abandoned, and the outcome (null while there is none) |
| `same_pick` | as `fetch` | the pick (or `inert`), which `select` and `fetch` must both give (§7.2) |
| `codecs` | a `codecs` value | each element rendered, `null` for a classical one, or `error` |
| `union_floor` | a sequence of objects | the union and the floor, or `error` |
| `supports` | a floor, the sink's containers and capabilities | `supports` |
| `layer_tag` | a `#WAI-LAYER` line | `n` and `uri`, or `error` |
| `media_playlist` | a `classical` playlist, the `segments` (each an `object`, its `layers`, its `parts` with their `alias`), an `open` segment, a `uri_prefix`, `end_list` and the `file` it is committed as | the rendering's BLAKE3 and length, and whether the file holds those bytes; or `error` |
| `multivariant` | each variant's `uri`, the WAI media `playlist` it names, `resolution`, `frame_rate_milli` | as `media_playlist`; the objects are found by the cids the playlist lists |
| `mpd` | a WAI media `playlist`, the classical timeline (`classical_mpd` and its `classical_set` and `period_id`, Appendix I; else the `classical` HLS playlist at timescale 1000), `set_id`, `rep_id` | as `media_playlist`; a gap segment is left out |
| `dash_adjacency` | a `wai` and a `classical` MPD (files, or `*_text`), optionally edited | `adjacent`, or `error` (§11.2 in DASH, Appendix I) |
| `playlist_parse` | a playlist `file` or `text` | its parameters, segments (duration, discontinuity, date-time, parts, layer numbers, URI), open segment and hint, and whether it is in the rendered form; or `error` |
| `adjacency` | a `wai` and a `classical` playlist (files, or `*_text`), optionally edited (`edit_wai`, `edit_classical`: `[from, to]` pairs, each replaced once) | `adjacent`, or `error` |
| `unchanged` | an `original` and a `published` classical master (files, or `*_text`), `edit_published` | `unchanged`, or `error` |
| `playlist_receipt` | a `playlist`, its `receipt` file and the `name` it is fetched as (the file's own by default), each optionally edited | the receipt hash and its parent, or `error` |
| `live` | `revisions` in the order a client fetched them (each a WAI playlist, its receipt, its classical playlist, `edit_wai`), `fetches` (an `alias` and the `object` served under it, optionally patched), the playlist's `url` (against which its lines resolve; `https://origin.example/live/wai-720.m3u8` by default), where parents are fetched from (`parents`, a directory holding `<receipt_hash>.json`; the first revision's `mr/` by default) and the walk's bound (`walk_bound`, 64 by default) | per revision its adjacency, its listing against what accepted revisions bound, its receipt, what it did to the chain (`true`, `unchanged`, `reanchored`, or the reason) and the parents walked; per fetch the accepted revision that first lists the alias, the outcome and what held (`listed`, or `signed` when a receipt in the chain binds the alias's URL to that cid) |
| `uri_resolve` | a `base` and a `reference` | the `url` (RFC 3986 §5.2 and §6.2.2, without the fragment) |
| `abr` | a `policy` and a `state` (Appendix F) | the `choice` and what the `budget` did, or `error` beyond Appendix F's bounds |
| `abr_order` | an object, what the sink `advertises`, a `policy`, a `state` | the entry `order` |
| `depth` | layer `objects` (or `layers_n`), a `policy`, a `state` | the `depth` and what the `budget` did, or `error` |
| `ilog2` | `a`, `b` | `log2_q16` |
| `render` | `segments`, `render` and `extras` written inline | the rendering's BLAKE3 and length, or `error` |

An input names an object by id, a `file`, or an `envelope_hex` written inline;
`union_floor` also takes `envelopes_hex`. A sink's `declared` is the union it was given
(§4.2), parsed before anything is fetched; `layer_base` (an object id) or
`layer_base_hex` is the base it holds, against which a layer is declared only in the
`alt` of that base's capability. A `declared` that does not parse, or a base that is
not an object, refuses the case with its code.

`fetch`, `select` and `same_pick` also take `abr` (a `policy` and a `state`): the sink's
entry order is then the policy's over the verified head (Appendix F), on both paths.

`fetch` runs the reference client's policy: a first request of 16369 bytes unless `h`
is given, never past `n`; the rest of the object when the planned base-class ranges
reach 900 ‰ of the bytes after `covered_len` (`threshold_permille` changes it); and,
with `listed_as_layer`, one whole request at refinement class. The origin answers
single ranges with `206`. A `tamper` changes what it serves: `patch_object` its bytes;
`kind` `gzip` (every object response labelled gzip), `no_refinement` (no
refinement-class request answered) or `no_proof` (the head proof's request not
answered); `first_range_flip` one byte of the first range response only; `refinement`
every refinement-class range response one way (`gzip`, `bad_range`, `416`,
`complete_length`, `plus_sign`, `short_200`). Body bytes count the head proof; a
delivery states its role (`base` or `layer`), the refinements abandoned and those
dropped with why.

## Regenerating

    cargo run --no-default-features --features http,video_manifest --bin wai_http_vectors -- gen ../http-conformance

from `wai/wai-rs`. Every file is a function of `src/`, the two committed vectors the
objects carry, and the code, so a second run writes the same bytes; `verify`
regenerates in memory and requires them. The generator also holds each negative case
to the reason it exists for, and OD to Appendix D's cid, length and proof size.

## Falsification

Each patch in `falsify/` names the function it breaks and, on its first line, the
cases that must fail. The `falsify` job of `.github/workflows/wai-http.yml` applies
each to a fresh worktree, requires the build to succeed, and requires
`wai_http_vectors verify` to exit non-zero naming those cases. `http_verify.py
--mutate NAME` does the same on the Python side (`left-half`, `no-root`,
`counter-zero` fail the proof cases; `floor-as-codecs` fails `union_O9`; `blake3-iv`
fails the self-test; `base-at-done`, `tick-noop` and `refinement-unclipped` fail the
`timed_*` cases; `pdt-ignored`, `dseq-ignored`, `parts-after-extinf`, `average-floor`,
`render-digest-off`, `name-unchecked`, `chain-unchecked`, `ilog2-no-interp`,
`bola-waits`, `unlabelled-as-zero` and `round-half-down` fail the playlist, receipt and
adaptation cases they reach; `nonconforming-offered`, `conforming-withheld` and
`refines-read-whole` fail the ORN, ONC, OCF, ONM and OR fetches; `falsify/run.sh` runs
every one), and the job checks a differential: `wai_http_vectors diff`
writes random envelopes' covered prefixes and proofs, `http_verify.py diff` verifies
every one, and, with one chaining-value byte changed in ten of them, refuses exactly
those ten.

## Outside the corpus

    wai_http_vectors fuzz . --seed S --count N --out cases.json
    python3 http_verify.py cases cases.json .

`fuzz` writes seeded cases, in the corpus's format and with wai-rs's outcomes, for the
inputs where two readers most easily part, in families:

- `fuzz_response`: `Content-Range` values, half of them the request's own range with
  one number given a sign, a separator, a space or another script's digits;
- `fuzz_codecs`: `codecs` values (the `wai` prefix in any case, duplicates, empty
  elements, non-ASCII); `fuzz_layer_tag`: `#WAI-LAYER` lines;
- `fuzz_wai1_head`, `fuzz_wai1_fetch`, `fuzz_wai2_head`, `fuzz_wai2_select`,
  `fuzz_wai2_fetch`, `fuzz_union`: `WAI1` and `WAI2` manifests (member types,
  `security` in every shape, `refines` well and badly formed, members named twice,
  refinement components with a right, a wrong and no `sha256`);
- `fuzz_wai1_refines_fetch`, `fuzz_wai1_refines_union`: well-formed `WAI1`s whose
  `refines` varies over payload and refinement-only capabilities, fetched as objects
  and as listed layers;
- `fuzz_layer_union`, `fuzz_layer_declared`: a base with a layer that names it or does
  not, and the layer against a declared union and a base — its own base's `alt`, that
  `alt` without it, another `alt` with it, the other container, no base, another base;
- `fuzz_component_layer_union`, `fuzz_component_layer_declared`: a layer whose base is
  a `WAI2` component, named by the hash of the component's exploded envelope;
- `fuzz_large_fetch`: `WAI2` objects past the first request, with a base, a refinement
  or companion and an alternative each stating a right, a wrong or no digest, fetched
  through an origin tampered one way, at thresholds either side of the plan;
  `fuzz_whole_refine`: the whole path, its base's digest absent or the threshold low,
  with a refinement whose digest is right, wrong or absent;
- `fuzz_playlist_parse`: playlists in a packager's order with at most one line changed,
  and lines from every tag the reader reads, valid and not; `fuzz_adjacency`: one
  timeline written as a classical and a WAI playlist, the WAI one perturbed one way or
  not at all; `fuzz_unchanged`: a master and a published form of it; `fuzz_render`:
  renderings from inline inputs, well and badly formed; `fuzz_receipt`: a committed
  receipt file with one member changed;
- `fuzz_abr`: states and parameters of the four policies, sizes out of order, samples
  of 0 ms, empty options and budgets; `fuzz_ilog2`: the integer log at random points
  and the ends of `u64`.

A refusal is an outcome like any other: `http_verify.py cases` must give the same
code. A case wai-rs gives no outcome for stops the run, and `fuzz` prints, per family,
the cases it generated and how many were refused, so a family that only ever refuses
(or never does) shows. `wai-conformance.yml` runs it at two seeds.