packaging: a title's whole ladder in one package — several floors, a re-pack, CMAF floors, floor-only, a streaming packer, the verifier at scale #426
2026-10-21
Merge origin/main into wai/pack-efficiency (exploration, ad presentation, access services, sandbox)
Some checks failed
WAI energy receipts / energy receipts · linux-arm64 (pull_request) Successful in 6m42s
WAI conformance / quantum core and corpus under wasm32-wasip2 (wasmtime) (pull_request) Successful in 2m11s
WAI conformance / cross-arch byte-equality · linux (pull_request) Successful in 29s
WAI conformance / independent verifiers · claims, JWP, key discovery and integer payloads (Python) (pull_request) Successful in 29m8s
WAI energy receipts / energy receipts · linux (pull_request) Successful in 11m1s
wai-pack / wai-pack · linux (pull_request) Successful in 1m35s
WAI conformance / cargo-class corpora · linux (pull_request) Successful in 19m52s
wai-quantum studio and desktop / studio · browser (pull_request) Successful in 4m12s
wai-web wasm32 / wai-web wasm32 build · corpus, menu and writers (pull_request) Successful in 4m18s
WAI HTTP delivery / HTTP corpus · falsify mutations 0/3 (pull_request) Successful in 31m53s
WAI HTTP delivery / HTTP corpus · falsify patches 0/3 (pull_request) Failing after 1m40s
WAI HTTP delivery / HTTP corpus · falsify patches 1/3 (pull_request) Failing after 1m1s
WAI HTTP delivery / HTTP corpus · falsify patches 2/3 (pull_request) Failing after 1m5s
WAI HTTP delivery / HTTP corpus · every patch and mutation in exactly one shard (pull_request) Successful in 4s
WAI HTTP delivery / HTTP corpus · falsify mutations 1/3 (pull_request) Successful in 31m41s
WAI HTTP delivery / HTTP corpus · verified both ways, and the differential (pull_request) Successful in 2m24s
WAI HTTP delivery / cache rules · controls and labelled caches (pull_request) Successful in 1m34s
wai-pack / wai-pack · linux-arm64 (pull_request) Successful in 1m26s
wai-pack / wai-pack · the floor through a media prober (pull_request) Failing after 17s
wai-pack / wai-pack · dependency licences (pull_request) Successful in 1m16s
wai-pack / wai-pack · independent verifier (Python) (pull_request) Successful in 6m31s
WAI HTTP delivery / HTTP corpus · falsify mutations 2/3 (pull_request) Successful in 32m22s
joule-moq-web browser client / joule-moq-web · browser end-to-end (macos) (pull_request) Successful in 6m19s
standards workspaces build + test / macOS energy backends (pull_request) Successful in 9m13s
WAI conformance / cargo-class corpora · macos (pull_request) Successful in 17m14s
WAI conformance / cross-arch byte-equality · macos (pull_request) Successful in 19s
WAI conformance / codec wrappers · fuzz smoke, text decode limits, C entry points, provenance, the semantic path (pull_request) Successful in 57s
WAI energy receipts / energy receipts · macos (pull_request) Successful in 7m3s
wai-pack / wai-pack · macos (pull_request) Successful in 1m28s
wai-quantum studio and desktop / desktop app · macos (pull_request) Successful in 2m49s
The changelog's packaging section stays at the top of v0.4.0, above main's entries. Co-Authored-By: Claude Opus 5.5 <[email protected]>
pack-conformance: policy_diff.py generates profile, source and the [floor] table
Some checks failed
WAI conformance / quantum core and corpus under wasm32-wasip2 (wasmtime) (pull_request) Has been cancelled
WAI conformance / v2 vocabulary · manifests validate as stated (pull_request) Has been cancelled
WAI conformance / independent verifiers · claims, JWP, key discovery and integer payloads (Python) (pull_request) Has been cancelled
WAI conformance / delivery telemetry · CMCD and CMSD against an independent library (pull_request) Has been cancelled
WAI conformance / cross-arch byte-equality · linux (pull_request) Has been cancelled
WAI conformance / cross-arch byte-equality · linux-arm64 (pull_request) Has been cancelled
WAI conformance / cross-arch byte-equality · macos (pull_request) Has been cancelled
WAI conformance / codec wrappers · fuzz smoke, text decode limits, C entry points, provenance, the semantic path (pull_request) Has been cancelled
WAI conformance / C2PA manifest emission · signed, read back, validated (pull_request) Has been cancelled
WAI energy receipts / energy receipts · linux (pull_request) Has been cancelled
WAI energy receipts / energy receipts · linux-arm64 (pull_request) Has been cancelled
WAI energy receipts / energy receipts · macos (pull_request) Has been cancelled
WAI HTTP delivery / HTTP corpus · falsify mutations 0/3 (pull_request) Has been cancelled
WAI HTTP delivery / HTTP corpus · falsify mutations 1/3 (pull_request) Has been cancelled
WAI HTTP delivery / HTTP corpus · falsify mutations 2/3 (pull_request) Has been cancelled
WAI HTTP delivery / HTTP corpus · falsify patches 0/3 (pull_request) Has been cancelled
WAI HTTP delivery / HTTP corpus · falsify patches 1/3 (pull_request) Has been cancelled
WAI HTTP delivery / HTTP corpus · falsify patches 2/3 (pull_request) Has been cancelled
WAI HTTP delivery / HTTP corpus · every patch and mutation in exactly one shard (pull_request) Has been cancelled
WAI HTTP delivery / HTTP corpus · verified both ways, and the differential (pull_request) Has been cancelled
WAI HTTP delivery / cache rules · controls and labelled caches (pull_request) Has been cancelled
wai-pack / wai-pack · linux (pull_request) Has been cancelled
wai-pack / wai-pack · linux-arm64 (pull_request) Has been cancelled
wai-pack / wai-pack · macos (pull_request) Has been cancelled
wai-pack / wai-pack · independent verifier (Python) (pull_request) Has been cancelled
wai-pack / wai-pack · the floor through a media prober (pull_request) Has been cancelled
wai-pack / wai-pack · dependency licences (pull_request) Has been cancelled
wai-quantum studio and desktop / studio · browser (pull_request) Has been cancelled
wai-quantum studio and desktop / desktop app · macos (pull_request) Has been cancelled
wai-web wasm32 / wai-web wasm32 build · corpus, menu and writers (pull_request) Has been cancelled
The seeded policies now carry component.profile (floor-only, or a value it does not take), a source, and floor.supplied_form and extra_keyframes, so the two policy readers are compared on the new members and their refusals (policy-floor-only among them). Co-Authored-By: Claude Opus 5.5 <[email protected]>
ci(wai-pack): the claim verifier and the OER verifier it imports are among the paths
pack_verify.py now reads a metered package's claim with claim-conformance/claim_verify.py, which imports oer-conformance/oer2_verify.py: a change to either runs this workflow. Co-Authored-By: Claude Opus 5.5 <[email protected]>
pack-conformance: the supplied CMAF reader differentially (cmaf_diff.py)
wai-pack's reader of a supplied CMAF floor (wai-pack cmaf-verdict, a conformance aid) against pack_verify.py's, which shares no code with it, on seeded alterations of a media muxer's segments (packs/video-cmaf-01/floor) and of §7.6's writer's: a flipped byte, an overwritten field, a cut, a changed box size, an inserted box, the init's timescale or edit list, a repeated moof/mdat pair. Every verdict, and every accepted segment's samples, must agree, and neither may crash. CI runs it in the independent job. Co-Authored-By: Claude Opus 5.5 <[email protected]>
packaging §7.3: a key-frame sequence header put back is the stream's own
An informative note: a floor taken from an existing ladder needs a sequence header in every unit that starts at a shown key frame; where a ladder's key-frame samples lack one, the header put back must be the stream's own first sequence header OBU, byte for byte, since §7.3 refuses any other bytes. The sample entry's av1C header can differ from the in-band one in its tool flags and is refused; the packager writes its own av1C from the in-band header. A ladder without in-band headers cannot be taken in the CMAF form. tests/refusals.rs holds the rule: the stream stripped of its key-frame headers is refused, its own header put back is the stream, another header's bytes are refused. Co-Authored-By: Claude Opus 5.5 <[email protected]>
pack-conformance: the staged golden itself among staged-mutations (ok)
Verified without its sources, the staged golden's every layer has its prefix digest checked by a decode the verifier runs however it runs it (serially or in pack_verify.py's workers). Without an accepted layered case, layer digests a verifier got wrong still gave the refusals the corpus expects. The README names the metered package cases. Co-Authored-By: Claude Opus 5.5 <[email protected]>
pack_verify.py at scale, and the measurement claim checked (§9.4)
- Each object is hashed once (http_verify keeps each object's content hash, which pack_verify seeds once an object holds to its entry), and http_verify.resolve_uri finds an object under its own content id before any search: the lookup that hashed every object for every URI is gone. - --jobs N: every object's hash, and every base's decode and its layers, piece by piece, run in N worker processes ahead of the checks, which read their results in the serial run's order with the serial run's reasons. - --native-blake3 (opt-in): BLAKE3 from the blake3 module, taken only after it agrees with the pure implementation on the published vectors and on inputs at every chunk boundary; the docstring says what it gives up (that module is the implementation wai-rs links). - --package <dir>: one package verified as §11.3 does, without its sources or its key. - The measurement claim of a metered package is checked: the claim corpus's verifier (claim_verify.py) reads it, then it is held to the receipt (subject, figure profile, joules, class and evidence, signer). The unconditional measurement-claim-invalid is gone. - package-mutations: a metered package (ok), a claim about another figure (measurement-claim-invalid), none (measurement-claim-missing), each made by the generator with fixed members. - CI: pack_verify.py --jobs 2 over the corpus. Co-Authored-By: Claude Opus 5.5 <[email protected]>
wai-pack tests: the directory pack of a floor-only golden has no source
Co-Authored-By: Claude Opus 5.5 <[email protected]>
packaging: a floor-only profile (§7.8)
A component of profile = "floor-only" is its floor rungs alone: no integer base, no refinement and no source (policy-floor-only otherwise). Its pieces are its first floor rung's media segments (the CMAF form) or shown key frames (the OBU form), each at most max_ms (floor-piece-length); each piece is its floor objects, which present no base and so carry no derived_from, of kind none (stage index kind 6). Every class lists only the floor. It attests delivery: the receipt binds every floor object and the supplied floor's bytes, and the verifier holds them to the AV1 stream's structure and to the CMAF files; it states no decoded output, no fidelity to a source and no primary. - wai-rs stage_index: piece kind 6; a floor carries derived_from exactly when its piece has a base (floor-derived-from). - wai-pack: policy (component.profile; the rule ladder of a floor-only component is its floors), the packer's floor-only path, the verifier's index and floor checks; pack_verify.py mirrors each. - pack-conformance: packs/video-floor-only-01 (CMAF form) and -02 (OBU form), and index cases of a piece of floors alone. - tests/floor_only.rs; wai-rs stage_index test; CI floors and the media probe of the OBU floor-only golden. Co-Authored-By: Claude Opus 5.5 <[email protected]>
wai-pack: a re-pack reads its parent's objects one at a time
The parent's objects were held for the whole pack, so a re-pack's memory grew with the title. The parent is still verified whole (§11.3's verifier reads a package whole); after that only its entries are kept, and each piece's objects are read from the parent when the piece is coded, held to the content hashes the verified index names. Co-Authored-By: Claude Opus 5.5 <[email protected]>
packaging: a streaming on-demand pack (§11.4)
wai-pack pack now reads its source from its file a frame at a time (twice: once to cut, once to code) and the supplied floor from its directory a file and a temporal unit at a time, writes each package file into the output directory as it is made (files::DirSink, which removes what it wrote on a refusal), and reads objects back one at a time for the catalogs. What it holds no longer grows with the title's length, but for a record per piece and per entry and a byte per frame per floor. --jobs <n> codes n pieces at once, each alone, so the package is the same for any n. - packaging §11.4: what the packer holds, and that a refusal leaves nothing written. - tests/pack_alloc.rs: a source eight times as long packs in the same peak allocation (a counting allocator), give or take its records; tests/packs.rs: every golden packs byte for byte into a directory on three threads. CI's test-count floors. Co-Authored-By: Claude Opus 5.5 <[email protected]>
packaging: a floor that is the ladder's own CMAF segments (§7.7)
With floor.supplied_form = "cmaf" (absent, the OBU form; existing policy digests are unchanged), a supplied floor is an existing ladder's init.mp4 and media segments, one per piece. The packer reads them box by box (wai-pack cmaf_read): the floor's temporal units are the samples, each with a temporal delimiter before it, held to §7.3 as a stream is. The package's floor files are the ladder's files byte for byte under their SHA-256, not rewritten, and the package carries the supplied floor's listing (floor.listing), whose SHA-256 the policy pins: the receipt binds every byte of the ladder, and a segment of the same samples that is not the supplied one is refused. New refusals: floor-cmaf-malformed, floor-cmaf-timing, floor-cmaf-pieces. wai-pack pack --link-supplied writes the floor's files as hard links to the supplied ones. - pack-conformance: packs/video-cmaf-01 (video-ladder-01's streams muxed by ffmpeg's HLS muxer, -c copy, by supplied_cmaf.py) and cmaf-mutations; pack_verify.py reads the CMAF form with a box reader of its own, field reads bounded by their boxes as wai-pack's are. - tests/cmaf.rs, cmaf_read's unit tests, CI's test-count floor and the media probe of the CMAF golden's floors. Co-Authored-By: Claude Opus 5.5 <[email protected]>
packaging: a re-pack takes its bases from a parent package (§11.5)
A packer may now take every piece's base and refinements from an earlier package instead of coding them, so a title's integer base is coded once and every later package of the same source takes it: wai-pack pack --base-from <package>. Before it writes anything it verifies the parent as §11.3 does (repack-parent-invalid), matches it piece by piece (start, duration, flags, kind, the rungs and parameters of the base and refinements, the deadline; repack-parent-mismatch), and holds each reused prefix to this source by the SSE its entry states (repack-source-mismatch). The receipt names the parent with parent kind re-pack; every other file is a coding pack's. A verifier holding the parent checks a re-pack against it (wai-pack verify --parent; verify::verify_repack). - pack-conformance: packs/video-repack-01, the ladder golden re-packed from video-base-01, and pack_verify.py's check of a re-pack golden against the parent it names. - tests/repack.rs, and CI's test-count floor. Co-Authored-By: Claude Opus 5.5 <[email protected]>
packaging: several floor rungs per package, over one base
A component may now have several floor rungs: each piece has one floor object per floor rung, in ladder order, over one integer base, so one package carries a title's whole AV1 ladder and its base is coded once (packaging §7.1). Each floor rung has a floor name: the component's id when it has one floor rung, so existing packages are unchanged byte for byte, and the rung's id when it has several. The name names the supplied stream, the CMAF files, the floor playlist, the DASH representation and the MoQ track; a name that cannot be a path segment, or two equal in any case, is refused (pack-id). - wai-pack: the packer reads its source and each floor stream as streams and writes each file to a sink as it is made (files.rs, av1::TuReader, floor::check_stream); a source that does not read the same twice is refused (pack-source-changed). The catalogs list every floor; the verifier holds the k-th floor entry of each piece to the k-th floor rung. - wai-rs: the DASH renderer writes a representation's own codecs and size, and an adaptation set's codecs only when shared. - pack-conformance: packs/video-ladder-01 (two floors over the bases of video-base-01, which are the same objects), ladder-mutations, and pack_verify.py's floor and catalog checks per floor rung, in §11.3's order; supplied_floor.py writes one stream per floor rung. - tests/ladder.rs, and CI's test-count floor and media probe of every floor. Co-Authored-By: Claude Opus 5.5 <[email protected]>