open-standards/wai/pack-conformance/README.md

# pack-conformance — the packaging corpus

Vectors for [`extensions/packaging.md`](../extensions/packaging.md): the policy, its
digest and what it may ask a packager to emit (§3), how a source is cut into pieces
(§4), and the integer video payloads a packager writes (§5.5).

**Provenance.** Generated here: the `wai-pack` fixture generator (`wai-pack/src/fixture.rs`, `wai-pack/src/conformance.rs`) writes every frame from integer functions of the frame and pixel index, and a package's floor is an AV1 coding of those frames. No footage.

- `policy/` — policies a packager accepts (every rung one it may emit, §3.6), and
  `expected.json`: each policy's effective canonical JSON and
  digest (`eq-defaults-a.toml` also with `royalty_free_only` forced on, the one way an
  override changes it, §3.2), and the
  pairs `eq-<name>-a.toml` and `eq-<name>-b.toml`, two spellings of one policy that
  must have one digest: every default omitted or written out; standard or inline
  tables, dotted keys, member order, string escapes, integer notation and comments.
- `refusals/` — `policy/*.toml`, one fault each (the byte layer's among them: a
  byte-order mark, bytes that are not UTF-8, an integer at 2^63, nesting to depth 33 and
  80), and `source/*` (Y4M and WAV), and `expected.json`: each one's reason (§12). Every
  file is read as bytes. A policy's reason is a packager's: the
  policy's own (§3.2–§3.4), or its first rung's capability refusal (§3.6), read from
  SPEC §5's registry. The source files are written by the generator.
- `conversions.json` and `conversions/` — milliseconds to a timebase (§4.4), a lead
  start to a component's timebase (§4.5), and Y4M to RGB8 (§4.7) by the SHA-256 of each
  frame, for 4:2:0 with odd dimensions and for 4:4:4. The Y4M files are written by the
  generator.
- `scenecut/` — `policy.toml` and `expected.json`: for each fixture, its SHA-256, the
  score of every frame, every piece (start, frames, flags, kind, and the statistics the
  kind is read from) and the plan digest (§4.2–§4.6).
- `encode/` — `WIV1` payloads of the integer video encoder (§5.5; `wai-rs`
  `int_video_enc`), written by the generator, and `expected.json`: one for every piece
  of every fixture, coding the piece's first 12 frames, under a whole-pel and a
  half-pel parameter set; for each, the encoder's parameters, the payload's BLAKE3,
  the SSE of its reconstruction against the source, and the SHA-256 of every decoded
  frame. Identical content gives identical bytes: `kinds-01`'s pieces 4 and 5 both
  start inside one static shot, so their first frames, and their payloads, are the
  same.

- `qchoice/expected.json` — the step the base encode chooses (§5.6) for every piece of
  every fixture (its first 12 frames), under two bounds: each candidate's SSE and SSE
  per sample in thousandths, the step chosen, and the chosen payload's BLAKE3.

- `packs/video-base-01/` — a golden package (§5–§11). Its inputs: `policy.toml`, the
  supplied floor `floor/v.obu` (an AV1 stream with a key frame at each piece start of
  the plan, each piece coded alone, written by an AV1 encoder from the fixture; the
  policy pins its listing), and `key.hex` (a test signing seed, never a production
  key). `package/` is every file the packer writes from them and the fixture
  `video-base-01`: objects, the stage index, the floor's objects and CMAF files, the
  catalogs, the receipt and `LISTING`. The fixture's bumper plays twice, so its pieces
  are one object each time.
- `mutations/` — stage indexes, objects and version-2 receipts, and `expected.json`:
  each case's reason (§6.2–§6.4, §9.1), a well-formed case of each kind among them,
  and three receipts carrying a signature only a verifier that checks the equation
  alone accepts (a key of small order, that key spelled with y = p + 1, an R of small
  order), each `receipt-signature` (SPEC §7 *Signatures*).
- `receipts/` — version-1 package receipts as JSON (§9.3), with each one's receipt
  hash: the legacy receipt whose hash `wai-rs` pins, an unmetered one, and the
  unmetered one carrying each of those three signatures, marked
  `"signature": "refused"`.
- `catalogs/` — stage indexes no golden package has (refinements on more than one
  deadline), with the MoQ catalog each gives (§8.4).
- `package-mutations/` — packages a verifier must refuse (§11.3), each the golden
  altered and, where the receipt or `LISTING` alone would catch the alteration,
  re-sealed with the golden's key: what a dishonest packager holding the key could
  publish. Each is an overlay on the golden (`<name>/`, the files that differ or are
  new; `expected.json`, the files it removes, the directories it adds that hold
  nothing or cannot be read, and its reason), written out as a directory and read
  strictly by each verifier: a receipt over no component, an index that is not its
  policy's, the policy's own refusals (a `policy.json` that turns `royalty_free_only`
  off among them), a missing or corrupted object, a live window no catalog can be
  written for, floor objects that do not open on a key frame or are not §7.2's, files
  and directories a strict read finds, and a receipt file that is not its canonical
  form; and one the verifier must accept (an extra key frame the policy allows).

The fixtures are not committed. `wai-pack fixture gen <name>` writes each from its
definition (`wai-pack/src/fixture.rs`), every byte an integer function of the frame
index and the pixel position, and the corpus pins its SHA-256:

- `cuts-01` — hard cuts; a flash exactly as long as the lookahead, which is not cut;
  a fade, which is not cut; and two shots cut against each other, each return a cut.
- `kinds-01` — one shot of each kind, then a static shot longer than the longest
  piece, cut where `max` forces it.
- `video-base-01` — 64 × 36: three shots with a one-second bumper between them, the
  bumper the same frames both times (its span reads time from its own start).

Files are compared as bytes; `.gitattributes` turns off line-ending conversion.

**Generation.**

    cd wai/wai-pack
    cargo run --release -- conformance regen ../pack-conformance

**Verifiers.**

- `wai-pack conformance check ../pack-conformance` recomputes every expected file and
  every generated input and requires them to equal the committed ones, requires the
  digests of each equal pair to be equal, decodes every encode payload through
  `decode_int` to the encoder's reconstruction and recomputes its SSE. It packs each
  golden again and compares every file, verifies the committed package
  (`wai-pack verify`), and gives every mutation and every package mutation its
  reason. `cargo test`
  runs the same check, with the scene cuts of each fixture stated as tests
  (`tests/scenecut.rs`).
- `pack_verify.py`, Python standard library and, to sign what it re-derives, `cryptography` (TOML through `tomllib`, so 3.11 or
  later) and sharing no code with wai-pack, written from the extension: its own policy
  schema, constraints, canonical JSON and digest; its own Y4M and WAV readers; the
  fixtures regenerated from their definition; `wai.pack.scenecut/1` in its on-demand
  form, every frame known, against the vectors of the streaming segmenter; the
  capability rules from the registry's published export
  (`../registry/capabilities.json`); and every encode payload decoded to its frames'
  SHA-256, with its SSE recomputed from the regenerated source. BLAKE3 comes from
  `../claim-conformance/_blake3.py`, and the `WIV1` parser and decoder from
  `../int-payload-conformance/int_payload_verify.py`. For the packages it has its own
  stage index reader and rules, object rules (each payload parsed as its pinned format,
  a `WIR1` reader included), version-1 and version-2 receipts (Ed25519 by WAI's strict
  rule, SPEC §7 *Signatures*, from `../claim-conformance/_ed25519.py`), an AV1 stream
  and sequence header reader, the CMAF files, and every catalog: it verifies each golden
  package in §11.3's order, rebuilding the floor's files and re-rendering the catalogs
  byte for byte, gives every mutation its reason, checks the version-1 receipts, and
  verifies every package mutation in the same order, refused with its reason.
  From the regenerated fixture it re-derives what only the source decides: the pieces,
  their rules, moments and class plan, each base's SSE and step, and the encoder's
  choices; and it checks the supplied floor against the policy's listing and the plan.

      python3 pack-conformance/pack_verify.py pack-conformance

- `policy_diff.py` runs the policy reader differentially: byte-level cases (a
  byte-order mark, bytes that are not UTF-8, integers past 2^53 and 2^63, nesting at
  and past the cap, control characters, line endings) and seeded random policies with
  faults in every table, each one's verdict and digest from `wai-pack policy-verdict`
  equal to `pack_verify.py`'s.

      python3 pack-conformance/policy_diff.py <wai-pack binary> 1 3000

- `segment_diff.py` runs `wai.pack.scenecut/1` differentially: seeded random clips
  under random parameters, every piece and score of the streaming segmenter
  (`wai-pack segment-json`) equal to `pack_verify.py`'s on-demand form, and every piece
  within `min` and `max`.

      python3 pack-conformance/segment_diff.py <wai-pack binary> 1 2000

- `source_diff.py` runs the source readers differentially: seeded Y4M and WAV
  mutations (dimensions and channel counts at and past their range, every format
  field, truncations, single-byte changes), each file's verdict from
  `wai-pack source-verdict` equal to `pack_verify.py`'s, and no reader panicking.

      python3 pack-conformance/source_diff.py <wai-pack binary> 1 3000