wai-pack: packages — the stage index, the version-2 package receipt, the floor, and HLS, DASH and Media over QUIC views #172

Merged

dcharlot merged 29 commits from wai/w4-3-packages into main 2026-10-06 10:32:04 -04:00

Conversation 0 Commits 29 Files changed 338 +13207 -84

2026-10-06

Transaction Science

wai.h: regenerated — CHAIN_REQUESTED_LINKS was removed in #171 ...

Stage receipts' last round replaced the requested-links set with the freshness
edge, but the committed header still declared the constant. cbindgen output from
the current tree.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

05783dbb8d

• View at this point in history

2026-10-06 09:23:31 -04:00

Transaction Science

Merge main (stage receipts merged as #171) into wai/w4-3-packages ...

Co-Authored-By: Claude Opus 5.5 <[email protected]>

9586bd950b

• View at this point in history

2026-10-06 09:18:29 -04:00

Transaction Science

wai-pack: an override never relaxes royalty_free_only; empty and unreadable directories refused; pack only into an empty --out; classic in any case; deep alterations never panic ...

Round 3 of the package review.

A. §3.2: an override only forces royalty_free_only on. Overrides.royalty_free_only is a
   bool (force on); Policy::effective ORs it in. Both verifiers read policy.json's
   royalty_free_only as that override, so a policy.json of false over a policy.toml of
   true is policy-json-mismatch. pack_verify.py parses policy.toml, applies the
   override, then checks §3.6 under the effective policy (§11.3 step 4), no longer
   refusing at the parse. Corpus case policy-json-relaxes-royalty-free (the review's
   pk-pol-rfo-false: a patent-pool floor relabelled through the objects and index,
   sealed with policy.json false). golden::reseal_as seals a policy.json given as bytes.
   The policy corpus's override case is now eq-defaults-a.toml forced on (example.toml
   forced off was a relaxation); packs.rs takes the line out of the policy instead.
B. pack_verify.py's strict walk raises on a directory it cannot read (os.walk onerror)
   and refuses it, as read_strict does.
C. Both verifiers refuse a directory below the package that holds nothing
   (unexpected-file); §11.1 says "no other file or directory". Corpus cases
   empty-directory, empty-dot-directory (.git/refs) and unreadable-directory (mode 0,
   or empty where the platform has no modes). Package cases are now written out and
   verified as directories by both verifiers (golden::package_case_verdict).
D. wai-pack pack refuses an --out that exists and is not empty (pack-out-not-empty,
   pack::out_dir_ready), before any work; §11.4 and §12. refusals.rs runs the command
   line: a stale claim left alone, a file in the way, an empty and a new directory
   packed and verified.
E. A class named classic in any case is pack-id (eq_ignore_ascii_case); corpus case
   class-named-classic-in-capitals.
F. review.rs deep_alterations_never_panic: 300 seeded rounds of floor-unit, framing,
   base-payload and index-field alterations through golden::replace_object and a
   re-seal; never a panic; anything accepted is in a stated equivalence class (AV1
   frame data, producer statements, equivalent WIV1 encodings) or the golden itself.

Floors: review 9, refusals 5.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

39375763c7

• View at this point in history

2026-10-06 09:09:09 -04:00

Transaction Science

pack-conformance: a piece with no key frame, a floor presenting another base, a CMAF file altered, a patent-pool floor under the packer's override ...

Falsification found two cases that did not hold their rule alone:

- floor-opens-on-inter rotated the piece, which also put a key frame inside it, so the
  extra-key-frame rule refused it whatever the opening rule did. The case (and its
  packer-side twin in refusals.rs) is now a piece whose first unit is an inter frame
  and which holds no key frame at all.
- patent-pool-floor put royalty_free_only in policy.toml, which the policy reader
  already refuses in pack_verify.py, so the check under the effective policy was never
  reached alone. patent-pool-floor-by-override leaves it out of policy.toml and states
  it in policy.json, the packer's override (§3.2); reseal keeps policy.json's
  royalty_free_only, as the verifier reads it.

New cases floor-motif (floor-motif-mismatch) and floor-cmaf (floor-cmaf-mismatch) give
every floor reason a verifier names a package case, in Rust and Python.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

f3d9e6e56a

• View at this point in history

2026-10-06 08:18:48 -04:00

Transaction Science

pack-conformance: a class record alone against its policy (index-class-rate) ...

A class's rate one below the policy's, the plan, catalogs and receipt following the
index: refused index-policy-mismatch by both verifiers, so the class-record check is held
on its own and not only through the class plan.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

83668b5f70

• View at this point in history

2026-10-06 08:05:45 -04:00

Transaction Science

wai: the encoder-cap test holds the clip cap's own refusal; to_json's doc says an unmetered receipt writes energy_provenance null ...

Co-Authored-By: Claude Opus 5.5 <[email protected]>

27b648044f

• View at this point in history

2026-10-06 08:03:24 -04:00

Transaction Science

pack-conformance: package mutations, refused by both verifiers ...

The review's package cases become a corpus, package-mutations/: each the golden altered
and, where the receipt or LISTING alone would catch it, re-sealed with the golden's key,
held as an overlay (the files that differ or are new, and the files it removes) with its
reason. 26 cases: a receipt over no component (on demand and live), the index against
its policy (class records, a capability, the string order, the class plan), a class
named classic, a patent-pool floor under royalty_free_only, a component of another
medium, a floor object missing and corrupted, a live window starting past 2^53 - 1, floor
objects that do not open on a key frame / hold an extra one / have another sequence
header / framing header / renderer and settings / target, an extra key frame the policy
allows (ok), dot-names and a *.sha256 file read strictly, and receipt files that are not
canonical or name a member twice.

- golden.rs builds them (package_mutations), writes the overlays (regen) and requires
  each, applied to the golden, to get its reason from verify_files (check, so cargo
  test and CI).
- pack_verify.py writes each case out and verifies it in §11.3's order without the
  golden's inputs, taking the reason of the first check that fails. Two fixes the
  corpus found in it: an index no catalog can be written for (a start past 2^53 - 1) is
  catalog-mismatch, not an uncaught ValueError; and "on-demand, one window" is an
  assertion about the golden, not a rule a package is held to (a live window is a
  package too, §6.1, §9.2). An object-rules failure now names its reason in brackets.
- review.rs asserts the same cases by name, keeping what the corpus cannot hold (an
  on-demand index starting past 0 cannot be sealed, a fragment time past 2^64 - 1, the
  random alterations, the verifier's own directory walk).

Co-Authored-By: Claude Opus 5.5 <[email protected]>

b1a7a776dd

• View at this point in history

2026-10-06 08:01:06 -04:00

Transaction Science

wai: window-start tested in wai-rs and in the corpus ...

An on-demand index (no previous window) whose first_piece or first_entry is not 0, or
whose first piece does not start at 0, is refused with window-start: three cases in the
stage index's mutation test, and the same three as corpus index mutations that
pack_verify.py re-derives.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

3d5a146ac4

• View at this point in history

2026-10-06 07:52:20 -04:00

Transaction Science

wai-pack: every packer refusal tested and listed; pack-capability-unsupported names its capability; a meter with no figure refuses the pack ...

- tests/refusals.rs: each supplied-floor reason (floor-missing, floor-listing-mismatch,
  floor-stream-missing, floor-av1-malformed, floor-frame-count, floor-keyframe-mismatch,
  and an extra key frame allowed by the policy); pack-media-unsupported,
  pack-refinement-unsupported, pack-rule-ladder-unsupported,
  pack-floor-encode-unsupported and pack-capability-unsupported:<cap> for a base, a
  second base and a floor; pack-meter for a model past its range and one that rounds to
  nothing; the encoder's caps on a 4K piece of 50 frames.
- pack: the policy's own refusals in the verifier's order (policy_refusal); two base
  rungs name the second's capability; a meter that was asked for and gives no figure
  refuses with pack-meter instead of packing unmetered.
- packaging §11.4 and §12 list pack-encode, pack-piece-length, pack-catalog, pack-meter,
  pack-measurement-claim and pack-listing. CHANGELOG: no package of this revision
  carries a #WAI-LAYER line.
- wai-pack.yml: test-count floors --lib 28, review 8, refusals 4.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

80dbd8d3e4

• View at this point in history

2026-10-06 07:50:50 -04:00

Transaction Science

ci: pack_verify.py's Ed25519 library installed from a hashed requirements file with --require-hashes ...

cryptography, cffi and pycparser pinned to one release each, every distribution file of
each listed by SHA-256; a wrong hash stops the install.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

267eb52e01

• View at this point in history

2026-10-06 07:44:38 -04:00

Transaction Science

wai: a package is bound to the policy its receipt signs; malformed packages are refused, never panicked on; floor key frames and members checked against the package; receipt JSON read strictly; MoQ dependencies never dangle ...

Review round 2, items 1-10:
- 1: §11.3 rewritten in order. The verifier (Rust and Python) requires the receipt's
  components to be the policy's by id, order and media (policy-component-mismatch,
  so a zero-component receipt is refused); runs pack-id and §3.6's capability rules
  under the effective policy; holds each index's timebase, media, string table,
  class records, every entry's role/capability/layer, and every piece's rules,
  moments and class plan to the policy (index-policy-mismatch). Seal and verify
  refuse a receipt with no component (receipt-index-count).
- 2: verify reads objects by get(), skips the floor and catalogs once anything has
  failed; the catalog renderers return Result (a start past 2^53 - 1 is
  catalog-mismatch); the fragment time uses checked_mul; Python maps encoding,
  overflow and value errors to refusals. New index rule window-start: an index with
  no previous window starts at piece 0, entry 0 and time 0 (§4.5), in both.
- 3: each floor piece opens on a shown key frame, and holds no other unless
  extra_keyframes; every sequence header is the first's bytes (both, §11.3).
- 4: both readers refuse a member named twice before reading the version; the
  package's receipt file must be its canonical bytes; Python refuses -0.
- 5: a refinement track depends on the tracks carrying what its entries refine
  (MSF's depends is all-of); never a track the catalog lacks. catalogs/ corpus.
- 6: Rust binds a supplied floor's renderer, settings, framing header and target
  (floor-object-mismatch); SPEC §7.1 says a profile may fix those members.
- 7: verify reads the directory strictly (dot-names, *.sha256 included).
- 8: Python's manifest parse is the typed one; the object differential is 0.
- 9: Python refuses two components of one id (receipt-component).
- 10: v2 writes every member, energy_provenance null when unmetered, evidence
  exactly as its class carries; v1 keeps reading an absent class.
Regression tests: tests/review.rs (the review's probes, re-sealed with the key, and a
seeded run of random alterations that never panics); mutation corpus receipt cases.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

6012a7c3cf

• View at this point in history

2026-10-06 07:42:47 -04:00

Transaction Science

pack-conformance: README names the verifier's one package dependency ...

Co-Authored-By: Claude Opus 5.5 <[email protected]>

30e4fd0c0f

• View at this point in history

2026-10-06 06:47:54 -04:00

Transaction Science

pack-conformance: the README describes pack_verify.py's package checks ...

Co-Authored-By: Claude Opus 5.5 <[email protected]>

4130b1c532

• View at this point in history

2026-10-06 06:47:48 -04:00

Transaction Science

wai: settle what pack_verify.py found ambiguous; the WSIX reader refuses in the stated order ...

The independent verifier, written from the text, found places the text did not
decide; each is now stated and both implementations follow it:
- §6.2: the reader's refusal order (magic, version, sizes from counts, reserved
  bytes, media class, then each entry's role and tier, strings, trailing); wai-rs's
  reader is rewritten to it.
- §6.3: entry order and layer-role for every role; the class-plan values; a
  refinement's object is a WAI1; null is absent; a WAI2's derived_from is its
  entries'; tier-weakest is a layered entry's.
- §6.6 a floor's sse; §7.3 a shown key frame needs a sequence header, and which
  header differs; §7.6 hdlr's name and tkhd's fields; §8.3 the Representation's
  attributes and one component; §8.4 track order; §8.8 media_playlist only when
  the class lists a WAI object (the golden's stages/legacy.json changes); §9.1 the
  reader's order and label rules, counts across indexes; §9.3 the version-1 signing
  payload.
- §5.2: the base manifest's fallback member and derived_from's place.

pack_verify.py follows the same choices (MoQ track order, null labels, WAI2 entries,
shown key frames); 540 of 540 checks hold, and the Rust corpus check passes.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

54ae7206de

• View at this point in history

2026-10-06 06:45:58 -04:00

Transaction Science

pack-conformance: pack_verify.py verifies the golden package, the mutations and the version-1 receipts from the extension text; the mutation objects carry real WIV1 and WIR1 payloads ...

pack_verify.py, written from packaging §5–§11, SPEC and the bindings it cites and
sharing no code with wai-pack: the stage index's layout and rules, every object rule,
the version-1 and version-2 receipts (payload, hash, Ed25519 through the cryptography
package, imported only where a signature is checked), the supplied AV1 floor and its
key frames, the floor objects and the CMAF files rebuilt, every catalog re-rendered,
and the package's verification in §11.3's order. From the regenerated fixture it also
re-derives what only the source decides: the pieces, the rules, moments and class
plan, each base's SSE and step, and the encoder's choices.

The mutations' base and layers now carry real WIV1 and WIR1 payloads, so they meet
§6.2's counted-bytes rule, which a verifier applies after the other object rules; new
cases object-hash (a misnamed file) and object-lengths (a base whose payload is not
its format). wai-pack's objects verdict applies the rule too.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

dac5560877

• View at this point in history

2026-10-06 06:40:07 -04:00

Transaction Science

wai-pack: an object's lengths are its counted bytes (staged measurement's count_object) where WAI checks its payload's format; frame digests through output_tree ...

Co-Authored-By: Claude Opus 5.5 <[email protected]>

25b6fda432

• View at this point in history

2026-10-06 06:29:12 -04:00

Transaction Science

ci: wai-pack's test-count floors for the packer's tests ...

Co-Authored-By: Claude Opus 5.5 <[email protected]>

753051923e

• View at this point in history

2026-10-06 06:24:29 -04:00

Transaction Science

wai-rs: the derived_from test's malformed motif is 64 uppercase digits, so only the lowercase rule refuses it ...

Co-Authored-By: Claude Opus 5.5 <[email protected]>

d54753279c

• View at this point in history

2026-10-06 06:23:22 -04:00

Transaction Science

wai-pack: tests 7 and 14 read a fresh pack ...

Co-Authored-By: Claude Opus 5.5 <[email protected]>

bca471be2c

• View at this point in history

2026-10-06 06:11:47 -04:00

Transaction Science

wai-pack: clippy clean under -D warnings ...

Co-Authored-By: Claude Opus 5.5 <[email protected]>

e8d0469227

• View at this point in history

2026-10-06 06:09:50 -04:00

Transaction Science

ci: wai-pack's media job reads and decodes the golden's floor; the independent job installs the Ed25519 library; the V2 figure-view test needs receipt_claims ...

Co-Authored-By: Claude Opus 5.5 <[email protected]>

915b3340bd

• View at this point in history

2026-10-06 06:09:14 -04:00

Transaction Science

wai: CHANGELOG and the corpus README for packages ...

Co-Authored-By: Claude Opus 5.5 <[email protected]>

238c8bacea

• View at this point in history

2026-10-06 06:08:10 -04:00

Transaction Science

wai: the registry lists packaging's wire identifiers (WSIX, the class and stage-view SESSION-DATA ids and DASH schemes, wai_layer and wai_delivery_timeout_ms) ...

Co-Authored-By: Claude Opus 5.5 <[email protected]>

ed83558841

• View at this point in history

2026-10-06 06:07:05 -04:00

Transaction Science

wai: packaging §5.1–5.4 and §6–§11 as built, the reasons of §12; SPEC §7.1 Derived renditions and its §8 member; wai.video.package version 2 ...

- packaging.md: objects (one envelope, position independence, the name, refines on a
  refinement); the stage index layout, its rules and reasons, tiers, prefix digest
  and SSE; the floor, derived_from on floor objects, a supplied floor and its
  checks, the keyframe plan, the CMAF files; what a class lists, the HLS and DASH
  files in http-delivery's Mode N, the MoQ catalog, priority, deadline, the stage
  view; the version-2 receipt, its JSON, verification and reasons, v1 JSON, the
  measurement claim; royalty-free packaging; the package directory, re-pack
  equivalence and verification; what this revision packs.
- SPEC: §7.1 Derived renditions; §8 lists derived_from among the OPTIONAL members
  that never break a parse; the wai.video.package row names version 2.
- verify: an index whose timebase is not its component's is index-policy-mismatch.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

3b6cef4cb9

• View at this point in history

2026-10-06 06:03:58 -04:00

Transaction Science

wai-pack: test 5-8, 14, 17, 18 over the golden; catalog tests; a refinement's refines names its piece's base and the layers below ...

- tests/packs.rs: the golden packs byte for byte and twice alike (5), every
  mutation's reason (6), the repeated bumper one object (7), royalty-free and
  candidate refusals with nothing written (8), the floor's CMAF structure (14), the
  measurement claim present, missing and forged (17), the receipts' JSON (18).
- catalog: a layered class in every catalog, #WAI-LAYER lines read back by
  http-delivery's parser, adjacency of the WAI and floor playlists.
- stage_index::check_object takes the entry's index and checks staged-delivery §3:
  base is the piece's base, producers are layers below, among them the entry below.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

a0bc29ae83

• View at this point in history

2026-10-06 05:58:50 -04:00

Transaction Science

pack-conformance: the golden video-base-01, the index, object and receipt mutations, and version-1 receipts as JSON ...

wai-pack conformance regen writes, and check recomputes:
- packs/video-base-01: policy, supplied AV1 floor, test key, and the whole package
  the packer writes from the generated fixture, byte for byte, which the verifier
  also verifies;
- mutations: stage indexes, objects and receipts, each with the reason it gets;
- receipts: the legacy version-1 receipt and an unmetered one as JSON.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

d561d372f8

• View at this point in history

2026-10-06 05:55:26 -04:00

Transaction Science

wai-pack: the packer and its verifier, with a supplied AV1 floor, CMAF, and HLS, DASH, MoQ and stage-view catalogs ...

- av1: the low-overhead OBU stream split into temporal units, shown key frames, the
  sequence header's fields, the codecs string and the av1C record.
- floor: a supplied floor checked against its pinned listing and the keyframe plan;
  the reference AV1 framing of a floor object's payload.
- cmaf: init.mp4 (cmfc, av01 + av1C) and one fragment per piece, pure functions of
  the samples.
- objects: base and floor envelopes (derived_from on the floor), named by content
  hash.
- plan: moments, rules and the class plan (max_layer, rules, max_kbps, floor only).
- catalog: on http-delivery's Mode N, the classical master and floor playlist, each
  class's master (one added SESSION-DATA line), its WAI playlist with the codecs
  union and X-WAI-FLOOR, its WAI media playlist with #WAI-LAYER lines and
  #EXT-X-GAP for floor-only pieces; classical and per-class WAI MPDs; the MoQ
  catalog; stage views.
- pack: one video component, an int_motion base whose step choose_q picks per
  piece, a supplied floor; refuses by name what this revision does not pack.
- verify: LISTING, the receipt and policy, the indexes, every object and prefix
  digest, the floor's CMAF rebuilt, the catalogs re-rendered, the measurement claim.
- CLI: plan writes the keyframe plan; pack, verify, listing.
- fixture video-base-01: three shots and a bumper played twice.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

411966b672

• View at this point in history

2026-10-06 05:52:22 -04:00

Transaction Science

wai-rs: keep the stage index's and package receipt's constants out of the C header ...

Co-Authored-By: Claude Opus 5.5 <[email protected]>

a8539781f4

• View at this point in history

2026-10-06 05:36:37 -04:00

Transaction Science

wai-rs: the stage index (WSIX), the version-2 package receipt with JSON, v1 JSON, the V2 figure view, and derived_from ...

- stage_index (video_package): the WSIX layout, parse, digest, stage leaves and
  root, the plan digest, every rule the index alone decides (piece order and
  entries, role and layer, refines, the tier composition of derived-tracks §3, the
  duplicate flag, the class plan), and the object rules (non-envelope-entry,
  lengths, capability, refines, derived_from).
- package (video_package): PackageReceiptV2 under wai:pack-receipt, sealed over the
  indexes with counts every verifier recomputes; verify_detailed with named
  reasons; RFC 8785 JSON both ways; v1 PackageReceipt gains the same JSON form
  (the legacy pin unchanged); AnyPackageReceipt dispatches on version.
- energy_figure: the V2 receipt's figure under wai:video-package-receipt-id, viewed
  only with the indexes it verifies against.
- container: derived_from on a WAI1 manifest and a WAI2 entry, kept as JSON like
  refines, with a typed reader.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

86b9fd7dc9

• View at this point in history

2026-10-06 05:35:41 -04:00