Transaction-Science/open-standards
Watch
1
0
Fork
You've already forked open-standards
 0
Code Issues Pull requests Projects Releases 15 Packages Wiki Activity Actions

wai identify, the carousel schedule and holdings: what a file is, when each object passes on a one-way bearer, and which neighbour holds it #605

Closed
dcharlot wants to merge 6 commits from wai/inspect-carousel-holdings into main
pull from: wai/inspect-carousel-holdings
merge into: Transaction-Science:main
Conversation 0 Commits 6 Files changed 493 +44064 -24
dcharlot commented 2026-11-03 17:15:41 -05:00
Owner
Copy link

This replaces #589, #592 and #595, closed because main moved under them: wai/CHANGELOG.md's Unreleased head conflicted with main's newer entries (from #591's tariff version 6 on), and a stacked branch can neither be force-pushed nor carry a merge commit. The six commits here are theirs, cherry-picked in stack order onto main at 6c8016b61, with messages unchanged and no merge commit: #589's three, #592's carousel commit, the carousel §11 Patents commit (f64d8fb53: the IETF disclosures on RaptorQ and Raptor, and no-code as the default), which was #592's last push, about seven minutes before it closed; its CI there never finished, so this pull request's CI is its first complete run, and #595's holdings commit. Three commits on the stacked branches carried #589's second and third commits again under the same patch IDs (9680914bb, 716dfd41d, f356702ad), so they were skipped.

Conflicts. Only one: wai/CHANGELOG.md, when #589's first commit was picked. This branch's entries go first under Unreleased, newest first (holdings, carousel, identify), above main's entries, which are kept as they are. Main had not changed the shared lists since the stack's base (WAI_FEATURES, the union build's --test and --bin lists, the feature-alone loops, codecs/registry.rs, REGISTRY.md, the registry and vocab exports, catalogue/ops.json), so they took the branch's entries in their stacking positions with no conflict. wai-rs/Cargo.toml merged cleanly: main's receipt_ledger line keeps exact_upscale and exact_tonemap beside the new identify, carousel and holdings features. No generated file needed regenerating: both exports check as current at the head.

Re-run at the head (c303cf55d). These ran on the macOS build host, exactly as the CI steps write them unless a note says otherwise, and all passed:

  • wai registry export --check and wai ops export --check ../catalogue/ops.json.
  • The carriage step, with cli_registry and corpus_provenance.
  • The library unit tests step (902 tests, among them codecs::registry:: and the vocab tests).
  • Each feature alone: cargo check --lib --features identify, its wasm32 build, and cargo check --tests with carousel and with holdings.
  • Media identification: lib identify:: (17), tests/identify (6) and wai_identify_vectors verify, then identify_verify.py: 429 vectors, 0 failures, with every mutated copy failing.
  • cli_mcp, cli_setup_agent, the binary's own tests and lib catalogue::.
  • int_kernels_vectors (8 tests; not the CI step's full test list).
  • Carousel: lib carousel:: claim_receipt::schedule:: (14) and tests/carousel (7). carousel_verify.py gave 99 cases with 0 failures, 10 of 10 mutated copies failed, and 14 of 14 weakened verifiers were caught. claim_verify.py carousel checked 134 claims with 0 failures.
  • Holdings: lib holdings:: (5) and tests/holdings (3). holdings_verify.py gave 39 cases with 0 failures, 9 of 9 mutated copies failed, and 10 of 10 weakened verifiers were caught.
  • Bearer receipts, since claim_verify.py and bearer-receipts.md changed: tests/bearer_receipts, bearer_verify.py, and claim_verify.py bearer (72 claims).
  • claim_verify.py on its own corpus (636 vectors) and in consumption mode (27 claims).
  • http_verify.py, since http_receipts.py changed: 1057 of 1057 cases.
  • scripts/check-lockfiles.sh: 60 lockfiles current.

These ran locally and also passed:

  • A step-by-step comparison of the workflow against main and the three source branches (no step body new to this branch).
  • ruby -ryaml and yaml.safe_load of wai-conformance.yml.
  • check-public-names.sh, check-wai-section-refs.py, check-tests-in-ci.py, check-workflow-concurrency.sh and check-wai-ops-doors.py.

wai identify and media.inspect (was #589)

wai inspect reads envelopes. Most files a toolchain meets are not envelopes, and until now the answer to "what is this file?" came from a file's extension or from a third-party prober. This adds wai identify <file> (catalogue operation media.inspect, agent tool media_inspect): what a file is, from its bytes alone.

What it says

  • The formats the bytes fit. Each candidate has the offsets that say so and how far its structure was followed: signature, structure or parameters. There is no confidence figure, because the strength says what was checked.
  • A container's tracks: each with its codec, size and protection, as the container states them.
  • The registered capabilities those codecs map to (media-identify §5.1).
  • Signatures embedded inside the file (§7).
  • When no format is recognised, measures that still hold: how its byte values are spread (chi-square and uniform windows) and the lags at which it repeats (§8).
  • verdict: identified, ambiguous, signature_only or unknown. A walk that stops says why: overrun, invalid, crc, limit or unknown_size.

Formats it walks: ISO media, Matroska, transport streams (with CRC-32), Ogg, FLAC, RIFF, MPEG audio, ADTS, PNG, JPEG, GIF, IVF, YUV4MPEG2, FLV, AV1 OBU, Annex B streams, and HLS and DASH text. It knows many more by signature, and the §6 and §7 tables are generated from the code into the spec.

Limits:

  • Nothing is decoded and no protected track is opened.
  • Cost is bounded for any file size: it reads 1 MiB at the head and scans at most 16 MiB, in at most 4,096 steps.
  • The agent door marks results untrusted_content, because a file's author chooses the strings in it.

How it is checked

  • Corpus: wai/identify-conformance/, 429 vectors.

    • About 290 cases are generated (including files repeated past the head and the scan window).
    • Files come from real muxers and encoders (tools/make_external.sh).
    • Each file has what an independent prober read of it (prober.json).
    • Negative controls are included.
    • wai_identify_vectors verify and tests/identify.rs require the corpus to regenerate byte for byte, with every result.

  • Second implementation: identify_verify.py.

    • It is written from the extension's text and shares no code with wai-rs.
    • It reads the codec tables, bitrates, ADTS rates and H.264 profiles from the spec.
    • It gives every file its whole result, holds ext/ to the prober's reading, and requires every mutated copy to fail.
    • Result: 429 vectors, 429 files, 0 failures.

  • Hostile bytes: prefix, flip and fuzz tests run through every walk. This found the AVS3 probe panic fixed in #586, which is identical here.

  • Mutation testing: three rounds against the Rust walks. The last round caught 1,999 of 2,185 mutants. The survivors are mostly equivalent mutants: slice bounds that cannot be reached, caps the head or scan limit makes unreachable, and the repeat measure's initial values.

  • Tests:

    • lib identify::: 17;
    • tests/identify: 6, covering the corpus, the §6 and §7 tables, every reason and verdict reached, the CLI equal to the library, and a directory or missing file being an error;
    • cli_ops: 10;
    • format_probe: 7.

    Clippy is clean, check-wai-section-refs and the public-name check pass, and the ops export is current.

  • CI:

    • The conformance workflow checks the identify feature alone and builds it for wasm32.
    • It adds the test and vector binary to the union build.
    • It runs identify_verify.py.
    • A change to media-identify.md triggers it.

The carousel schedule (was #592)

On a bearer with no return path (broadcast, satellite, one-way multicast), a receiver can't ask for what it missed. It holds an object only if the passes it heard carried enough encoding symbols. WAI could declare such a transmission (bearer-receipts) and reconcile it afterwards, but it had no signed statement of the plan in between: when each object passes, and with what forward error correction. Without one, a one-way bearer can't promise a deadline that anyone can check, and a receiver can't plan when to listen.

This adds wai.bearer.carousel (extensions/carousel.md) with the jwp-receipts claim class schedule (§2.4.16).

The claim

  • What it is: a publisher's signed plan, beside the group receipt and the instance's transmission claim, naming the group by its link hash.
  • What each object's entry states:
    • its code: RaptorQ (RFC 6330 with its verified erratum 5548) or Compact No-Code (RFC 5445);
    • that code's object transmission information;
    • the repair symbols each pass adds;
    • whether later passes repeat their ESIs or send new ones;
    • the first pass's instant, the period, the duration and the count of passes.
  • Step 4 holds each code's parameters to its bounds:
    • F up to 942,574,504,275;
    • T a multiple of Al, Z from 1 to 255, and N up to T/Al;
    • at most 56,403 source symbols per block, with RFC 6330's Partition[];
    • RFC 5052 §9.1 blocking within RFC 5445's 16-bit fields;
    • ESIs within their fields, passes that don't overlap, and at most 2^53 − 1 octets sent.
  • Step 6 checks each entry against the group: subject, signer, range, first pass not before group_start, content hash, and length where the bytes are held.

What a verifier derives

  • The pass after which an object is held, for a loss bound in parts per million. The answer holds for every pattern of loss within the bound.
    • A RaptorQ block counts as held at K′ + h distinct symbols, the count RFC 6330 §5.8 states its 10⁻² / 10⁻⁴ / 10⁻⁶ decoding-failure bounds at.
    • A symbol that a repeating pass missed is counted as missed again, so repeat and Compact No-Code never count on a second pass.
    • This is the deadline a one-way bearer can promise, checked with no receiver.
  • Statements that an object or a whole group is held by an instant: holds, availability_after_deadline, availability_not_within_schedule or object_not_scheduled.
  • Schedules set aside when they contradict each other, or when their instance is undeclared or two-way.
  • A receiver's listening plan: the passes that add symbols it is certain to keep, when it holds the object, and how long its radio must be on.

How it is checked

  • Corpus (live-conformance/carousel/):
    • 33 verdict cases;
    • 50 claim cases, covering every reachable bound of both codes, both met and passed;
    • 16 plans.
  • tests/carousel.rs, against an independent RFC 6330 implementation (the Apache-2.0 raptorq crate, added as a dev-dependency only):
    • K′ and Partition[] agree for every block size from 1 to 56,403;
    • every RaptorQ object a case holds available decodes from what a receiver within the case's loss bound holds by the end of that pass. In each pass the first ⌊L·m/10⁶⌋ symbols of each block are dropped, which in the first pass are source symbols, so the decoder must really decode.
  • Second implementation: carousel_verify.py, written from the text.
    • It agrees on all 99 cases.
    • It catches 10 of 10 mutated copies of the corpus.
    • All 14 versions of it with one check removed are caught by the corpus.
  • Claims: claim_verify.py carousel agrees with wai-rs on all 134 claims through steps 1–4. Its own corpus (636 vectors) and the bearer corpus (72 claims) still pass.
  • Independent expectations: the corpus states its expected passes by hand and computes the octets sent by a different formula from the module's, never copying either from the module.
  • HTTP delivery: a receipt index now checks a schedule claim against its group and listed objects, instead of accepting it after step 5. The rule is in http-delivery's text, in wai-rs and in http_receipts.py. The HTTP corpus has no schedule claim yet, so that branch is exercised only by compiling it.

Breaking

New variants in ClaimClass, ClaimBody and Reason; each is named in CHANGELOG.

Holdings (was #595)

Neighbours (a receiver that stores what a broadcast carried, a neighbourhood cache, a peer) can serve objects over a short link, but nothing said who holds what. This adds wai.delivery.holdings (extensions/holdings.md).

The record

WHD1 carries, for each group a node holds objects of:

  • the group's link hash;
  • its leaf count;
  • either whole or a bitmap of the objects held, encoded as a reconciliation's bitmap.

The record also states an instant it holds at and one it expires at, and the holder signs it under SPEC §7 Signatures. Each holding has exactly one encoding.

It uses groups and bitmaps rather than a filter over hashes, for three reasons:

  • WAI names objects by position under group receipts, and nodes hold objects as groups arrive, so holdings are dense within groups.
  • A bitmap is exact: no neighbour is sent after an object its holder lacks.
  • It costs a bit an object, and 41 bytes for a group held whole.

What a verifier derives

  • Instant: a record before as_of or at its expiry is refused.
  • Group receipts: a record that states a leaf count differently from a group receipt the verifier holds is refused whole. An entry for a group with no receipt held is kept.
  • One holder's records: the latest stands. Two at one instant that differ are both set aside.
  • Queries: held, object_not_held, object_out_of_range or group_not_advertised.
  • An asker's plan: each wanted object's holders, in the asker's order, and the objects no neighbour holds, which come from the origin.

An object a neighbour serves is verified under its receipts like any other object; a record is never evidence of bytes. A node advertises only what reached it over a broadcast bearer, and may sign each record with a fresh key.

Checks

  • Corpus: 21 records (a control for each rule and for the signature, including a small-order key), 11 verdicts, 6 queries, and a plan over three neighbours.
  • Second implementation: holdings_verify.py, written from the text, agrees on all 39 cases, catches 9 of 9 mutated copies, and catches 10 of 10 weakened verifiers.
  • wai-rs:
    • the full-feature build compiles;
    • unit tests pass (53 across holdings, carousel and claims), and the holdings and carousel tests pass;
    • clippy is clean on the new files;
    • the registry export is up to date;
    • wai breaking: no new breaking change.

🤖 Generated with Claude Code

Closed: Replaced by #611, which carries #604's two commits and then this PR's six unchanged, linearly on main c5c418e33; the two PRs could not both merge (each adds an entry at the head of wai/CHANGELOG.md). This PR was green apart from the co-watch browser scenario, a room page bug fixed on a separate branch.

This replaces #589, #592 and #595, closed because main moved under them: wai/CHANGELOG.md's Unreleased head conflicted with main's newer entries (from #591's tariff version 6 on), and a stacked branch can neither be force-pushed nor carry a merge commit. The six commits here are theirs, cherry-picked in stack order onto main at 6c8016b61, with messages unchanged and no merge commit: #589's three, #592's carousel commit, the carousel §11 Patents commit (f64d8fb53: the IETF disclosures on RaptorQ and Raptor, and no-code as the default), which was #592's last push, about seven minutes before it closed; its CI there never finished, so this pull request's CI is its first complete run, and #595's holdings commit. Three commits on the stacked branches carried #589's second and third commits again under the same patch IDs (9680914bb, 716dfd41d, f356702ad), so they were skipped. **Conflicts.** Only one: `wai/CHANGELOG.md`, when #589's first commit was picked. This branch's entries go first under Unreleased, newest first (holdings, carousel, identify), above main's entries, which are kept as they are. Main had not changed the shared lists since the stack's base (`WAI_FEATURES`, the union build's `--test` and `--bin` lists, the feature-alone loops, `codecs/registry.rs`, `REGISTRY.md`, the registry and vocab exports, `catalogue/ops.json`), so they took the branch's entries in their stacking positions with no conflict. `wai-rs/Cargo.toml` merged cleanly: main's `receipt_ledger` line keeps `exact_upscale` and `exact_tonemap` beside the new `identify`, `carousel` and `holdings` features. No generated file needed regenerating: both exports check as current at the head. **Re-run at the head (c303cf55d).** These ran on the macOS build host, exactly as the CI steps write them unless a note says otherwise, and all passed: - `wai registry export --check` and `wai ops export --check ../catalogue/ops.json`. - The carriage step, with `cli_registry` and `corpus_provenance`. - The library unit tests step (902 tests, among them `codecs::registry::` and the vocab tests). - Each feature alone: `cargo check --lib --features identify`, its wasm32 build, and `cargo check --tests` with `carousel` and with `holdings`. - Media identification: `lib identify::` (17), `tests/identify` (6) and `wai_identify_vectors verify`, then `identify_verify.py`: 429 vectors, 0 failures, with every mutated copy failing. - `cli_mcp`, `cli_setup_agent`, the binary's own tests and `lib catalogue::`. - `int_kernels_vectors` (8 tests; not the CI step's full test list). - Carousel: `lib carousel:: claim_receipt::schedule::` (14) and `tests/carousel` (7). `carousel_verify.py` gave 99 cases with 0 failures, 10 of 10 mutated copies failed, and 14 of 14 weakened verifiers were caught. `claim_verify.py carousel` checked 134 claims with 0 failures. - Holdings: `lib holdings::` (5) and `tests/holdings` (3). `holdings_verify.py` gave 39 cases with 0 failures, 9 of 9 mutated copies failed, and 10 of 10 weakened verifiers were caught. - Bearer receipts, since `claim_verify.py` and bearer-receipts.md changed: `tests/bearer_receipts`, `bearer_verify.py`, and `claim_verify.py bearer` (72 claims). - `claim_verify.py` on its own corpus (636 vectors) and in `consumption` mode (27 claims). - `http_verify.py`, since `http_receipts.py` changed: 1057 of 1057 cases. - `scripts/check-lockfiles.sh`: 60 lockfiles current. These ran locally and also passed: - A step-by-step comparison of the workflow against main and the three source branches (no step body new to this branch). - `ruby -ryaml` and `yaml.safe_load` of `wai-conformance.yml`. - `check-public-names.sh`, `check-wai-section-refs.py`, `check-tests-in-ci.py`, `check-workflow-concurrency.sh` and `check-wai-ops-doors.py`. ## `wai identify` and `media.inspect` (was #589) `wai inspect` reads envelopes. Most files a toolchain meets are not envelopes, and until now the answer to "what is this file?" came from a file's extension or from a third-party prober. This adds `wai identify <file>` (catalogue operation `media.inspect`, agent tool `media_inspect`): what a file is, from its bytes alone. ### What it says - **The formats the bytes fit.** Each candidate has the offsets that say so and how far its structure was followed: `signature`, `structure` or `parameters`. There is no confidence figure, because the strength says what was checked. - **A container's tracks:** each with its codec, size and protection, as the container states them. - **The registered capabilities** those codecs map to (media-identify §5.1). - **Signatures embedded inside the file** (§7). - **When no format is recognised,** measures that still hold: how its byte values are spread (chi-square and uniform windows) and the lags at which it repeats (§8). - **`verdict`:** `identified`, `ambiguous`, `signature_only` or `unknown`. A walk that stops says why: `overrun`, `invalid`, `crc`, `limit` or `unknown_size`. **Formats it walks:** ISO media, Matroska, transport streams (with CRC-32), Ogg, FLAC, RIFF, MPEG audio, ADTS, PNG, JPEG, GIF, IVF, YUV4MPEG2, FLV, AV1 OBU, Annex B streams, and HLS and DASH text. It knows many more by signature, and the §6 and §7 tables are generated from the code into the spec. **Limits:** - Nothing is decoded and no protected track is opened. - Cost is bounded for any file size: it reads 1 MiB at the head and scans at most 16 MiB, in at most 4,096 steps. - The agent door marks results `untrusted_content`, because a file's author chooses the strings in it. ### How it is checked - **Corpus: `wai/identify-conformance/`, 429 vectors.** - About 290 cases are generated (including files repeated past the head and the scan window). - Files come from real muxers and encoders (`tools/make_external.sh`). - Each file has what an independent prober read of it (`prober.json`). - Negative controls are included. - `wai_identify_vectors verify` and `tests/identify.rs` require the corpus to regenerate byte for byte, with every result. - **Second implementation:** `identify_verify.py`. - It is written from the extension's text and shares no code with wai-rs. - It reads the codec tables, bitrates, ADTS rates and H.264 profiles from the spec. - It gives every file its whole result, holds `ext/` to the prober's reading, and requires every mutated copy to fail. - Result: 429 vectors, 429 files, 0 failures. - **Hostile bytes:** prefix, flip and fuzz tests run through every walk. This found the AVS3 probe panic fixed in #586, which is identical here. - **Mutation testing:** three rounds against the Rust walks. The last round caught 1,999 of 2,185 mutants. The survivors are mostly equivalent mutants: slice bounds that cannot be reached, caps the head or scan limit makes unreachable, and the repeat measure's initial values. - **Tests:** - `lib identify::`: 17; - `tests/identify`: 6, covering the corpus, the §6 and §7 tables, every reason and verdict reached, the CLI equal to the library, and a directory or missing file being an error; - `cli_ops`: 10; - `format_probe`: 7. Clippy is clean, `check-wai-section-refs` and the public-name check pass, and the ops export is current. - **CI:** - The conformance workflow checks the `identify` feature alone and builds it for wasm32. - It adds the test and vector binary to the union build. - It runs `identify_verify.py`. - A change to `media-identify.md` triggers it. ## The carousel schedule (was #592) On a bearer with no return path (broadcast, satellite, one-way multicast), a receiver can't ask for what it missed. It holds an object only if the passes it heard carried enough encoding symbols. WAI could declare such a transmission (bearer-receipts) and reconcile it afterwards, but it had no signed statement of the plan in between: when each object passes, and with what forward error correction. Without one, a one-way bearer can't promise a deadline that anyone can check, and a receiver can't plan when to listen. This adds `wai.bearer.carousel` (extensions/carousel.md) with the jwp-receipts claim class `schedule` (§2.4.16). ### The claim - **What it is:** a publisher's signed plan, beside the group receipt and the instance's transmission claim, naming the group by its link hash. - **What each object's entry states:** - its code: RaptorQ (RFC 6330 with its verified erratum 5548) or Compact No-Code (RFC 5445); - that code's object transmission information; - the repair symbols each pass adds; - whether later passes repeat their ESIs or send new ones; - the first pass's instant, the period, the duration and the count of passes. - **Step 4** holds each code's parameters to its bounds: - F up to 942,574,504,275; - T a multiple of Al, Z from 1 to 255, and N up to T/Al; - at most 56,403 source symbols per block, with RFC 6330's Partition[]; - RFC 5052 §9.1 blocking within RFC 5445's 16-bit fields; - ESIs within their fields, passes that don't overlap, and at most 2^53 − 1 octets sent. - **Step 6** checks each entry against the group: subject, signer, range, first pass not before `group_start`, content hash, and length where the bytes are held. ### What a verifier derives - **The pass after which an object is held,** for a loss bound in parts per million. The answer holds for every pattern of loss within the bound. - A RaptorQ block counts as held at K′ + h distinct symbols, the count RFC 6330 §5.8 states its 10⁻² / 10⁻⁴ / 10⁻⁶ decoding-failure bounds at. - A symbol that a repeating pass missed is counted as missed again, so `repeat` and Compact No-Code never count on a second pass. - This is the deadline a one-way bearer can promise, checked with no receiver. - **Statements** that an object or a whole group is held by an instant: holds, `availability_after_deadline`, `availability_not_within_schedule` or `object_not_scheduled`. - **Schedules set aside** when they contradict each other, or when their instance is undeclared or two-way. - **A receiver's listening plan:** the passes that add symbols it is certain to keep, when it holds the object, and how long its radio must be on. ### How it is checked - **Corpus** (`live-conformance/carousel/`): - 33 verdict cases; - 50 claim cases, covering every reachable bound of both codes, both met and passed; - 16 plans. - **`tests/carousel.rs`**, against an independent RFC 6330 implementation (the Apache-2.0 `raptorq` crate, added as a dev-dependency only): - K′ and Partition[] agree for every block size from 1 to 56,403; - every RaptorQ object a case holds available decodes from what a receiver within the case's loss bound holds by the end of that pass. In each pass the first ⌊L·m/10⁶⌋ symbols of each block are dropped, which in the first pass are source symbols, so the decoder must really decode. - **Second implementation:** `carousel_verify.py`, written from the text. - It agrees on all 99 cases. - It catches 10 of 10 mutated copies of the corpus. - All 14 versions of it with one check removed are caught by the corpus. - **Claims:** `claim_verify.py carousel` agrees with wai-rs on all 134 claims through steps 1–4. Its own corpus (636 vectors) and the bearer corpus (72 claims) still pass. - **Independent expectations:** the corpus states its expected passes by hand and computes the octets sent by a different formula from the module's, never copying either from the module. - **HTTP delivery:** a receipt index now checks a schedule claim against its group and listed objects, instead of accepting it after step 5. The rule is in http-delivery's text, in wai-rs and in `http_receipts.py`. The HTTP corpus has no schedule claim yet, so that branch is exercised only by compiling it. ### Breaking New variants in `ClaimClass`, `ClaimBody` and `Reason`; each is named in CHANGELOG. ## Holdings (was #595) Neighbours (a receiver that stores what a broadcast carried, a neighbourhood cache, a peer) can serve objects over a short link, but nothing said who holds what. This adds `wai.delivery.holdings` (extensions/holdings.md). ### The record `WHD1` carries, for each group a node holds objects of: - the group's link hash; - its leaf count; - either `whole` or a bitmap of the objects held, encoded as a reconciliation's bitmap. The record also states an instant it holds at and one it expires at, and the holder signs it under SPEC §7 *Signatures*. Each holding has exactly one encoding. It uses groups and bitmaps rather than a filter over hashes, for three reasons: - WAI names objects by position under group receipts, and nodes hold objects as groups arrive, so holdings are dense within groups. - A bitmap is exact: no neighbour is sent after an object its holder lacks. - It costs a bit an object, and 41 bytes for a group held whole. ### What a verifier derives - **Instant:** a record before `as_of` or at its expiry is refused. - **Group receipts:** a record that states a leaf count differently from a group receipt the verifier holds is refused whole. An entry for a group with no receipt held is kept. - **One holder's records:** the latest stands. Two at one instant that differ are both set aside. - **Queries:** held, `object_not_held`, `object_out_of_range` or `group_not_advertised`. - **An asker's plan:** each wanted object's holders, in the asker's order, and the objects no neighbour holds, which come from the origin. An object a neighbour serves is verified under its receipts like any other object; a record is never evidence of bytes. A node advertises only what reached it over a broadcast bearer, and may sign each record with a fresh key. ### Checks - **Corpus:** 21 records (a control for each rule and for the signature, including a small-order key), 11 verdicts, 6 queries, and a plan over three neighbours. - **Second implementation:** `holdings_verify.py`, written from the text, agrees on all 39 cases, catches 9 of 9 mutated copies, and catches 10 of 10 weakened verifiers. - **wai-rs:** - the full-feature build compiles; - unit tests pass (53 across holdings, carousel and claims), and the holdings and carousel tests pass; - clippy is clean on the new files; - the registry export is up to date; - `wai breaking`: no new breaking change. 🤖 Generated with [Claude Code](https://claude.com/claude-code) --- **Closed:** Replaced by #611, which carries #604's two commits and then this PR's six unchanged, linearly on main c5c418e33; the two PRs could not both merge (each adds an entry at the head of wai/CHANGELOG.md). This PR was green apart from the co-watch browser scenario, a room page bug fixed on a separate branch. 
`wai inspect` reads envelopes; most files a toolchain meets are not envelopes, and the
first question about one is what it is. `wai identify <file>` (operation media.inspect,
extensions/media-identify.md) answers it for any file:

- the formats the bytes fit, each with the offsets of the bytes that say so, how far its
  structure was followed and where and why that stopped;
- a container's tracks as it states them: codec, size, channels, rate, protection;
- the registered capabilities they map to, and what a format probe reads of them;
- signatures found inside the file;
- where no format is recognised, measures that still hold: how the byte values are
  spread, uniform windows, and the lags at which the bytes repeat.

The verdict is identified, ambiguous, signature_only or unknown, and there is no
confidence figure: a candidate's strength says what was checked. Nothing is decoded, no
protected track is opened, and unknown stays unknown. Walks cover ISO media, Matroska,
transport streams, Ogg, FLAC, RIFF, MPEG audio, ADTS, PNG, JPEG, GIF, IVF, YUV4MPEG2,
FLV, AV1 OBU and Annex B streams, and HLS and DASH text; 35 more formats are known by
their signature. Every walk is bounded, reads a file where its lengths lead (a large file
is never held whole), and has been given hostile bytes; it builds for wasm32.

The agent door has it as media_inspect, its results marked untrusted_content: a file's
author chooses the strings in it.

identify-conformance/ holds files written by real muxers and encoders from test patterns
and tones, what an independent media prober reads of them (prober.json), and generated
cases sitting on each rule's edge. identify_verify.py is a second implementation written
from the text; it reads the extension's tables, gives every file its whole result, and
holds ext/ to the prober's reading. tests/identify.rs holds the reference's tables to
the extension's.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
every_decoded_output_states_a_sha256 requires each conformance directory to state its decoded
outputs' SHA-256 or say why it has none. identify-conformance decodes nothing: each vector states
what a file's bytes are read as, and the file's SHA-256 in vectors.json.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
The core toolset serves media_inspect after envelope_inspect; tests/cli_mcp.rs and the
README's toolset table now name it in that place.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
On a bearer with no return path a receiver cannot ask for what it missed: it holds an object
only if the passes it heard carried enough encoding symbols. bearer-receipts declares the
transmission and reconciles it afterwards; nothing stated the plan in between.

extensions/carousel.md registers `wai.bearer.carousel` and the jwp-receipts claim class
`schedule` (§2.4.16): a publisher's signed plan, beside the group receipt and the instance's
transmission claim, naming for each object its forward error correction (RaptorQ, RFC 6330 with
its verified erratum 5548; Compact No-Code, RFC 5445) and that code's object transmission
information, the repair symbols each pass adds, whether later passes repeat their ESIs or send
new ones, and the passes' times and count. Step 4 holds each code's parameters to its bounds
and division into source blocks; step 6 holds the entries to the group, its objects' content
hashes and, where the bytes are held, their lengths.

From it a verifier derives, for a loss bound in parts per million, the pass after which a
receiver whose losses stay within it holds each object, for every pattern of loss within the
bound: a RaptorQ block at K' + h distinct symbols, the count RFC 6330 §5.8 states its decoding
failure bound at, and a symbol a repeating pass missed counted as missed again. It checks a
statement that an object or a group is held by an instant, sets aside schedules that contradict
each other or whose instance is undeclared or two-way, and a receiver derives the passes it
must listen to and how long its radio must be on.

The reference is the `carousel` module of wai-rs with the class's body in
`claim_receipt::schedule`; the corpus is live-conformance/carousel/ (33 verdicts, 50 claims, 16
plans) with carousel_verify.py as its second implementation and claim_verify.py's new
`carousel` mode for steps 1-4. tests/carousel.rs checks K' and Partition[] against an
independent RFC 6330 implementation (a dev-dependency) for every block size, and decodes each
RaptorQ object a case holds available from what a receiver within the bound holds. A receipt
index served over HTTP now checks a schedule claim against its group and listed objects.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
The IETF holds disclosures on RaptorQ (RFC 6330; #2554 current, updating #1958) and on Raptor
(RFC 5053; #1957 current, with #1176 the holder's general licence statement). Both current
ones give two terms by product: a licence within the holder's standard terms for a device that
implements a wireless wide-area standard, and a non-assertion, conditional on full
implementation and suspended against a party that asserts, for one that implements none. No
disclosure relates to RFC 5445 or RFC 5052.

§11 cites them by number and states each licence class as disclosed. Since neither code is
royalty-free for every implementer, no-code is the default a tool that writes schedules uses,
as SPEC §5 keeps a capability that is not royalty-free from being a default. The reference
implementation encodes and decodes no symbol; its RFC 6330 decoder is a development dependency.
Security and privacy considerations move to §12; nothing cites them by number.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
wai: holdings — which objects of which groups a node holds now, for its neighbours
All checks were successful
WAI energy receipts / energy receipts · linux (pull_request) Successful in 14m13s
Details
wai-quantum studio and desktop / desktop app · linux (pull_request) Successful in 6m16s
Details
WAI mbt2018_mean integer decode (regenerated vectors) / regenerate mbt2018_mean integer vectors → Rust decode, required (pull_request) Successful in 5m25s
Details
wai-web wasm32 / wai-web wasm32 build · corpus, menu and writers (pull_request) Successful in 6m31s
Details
wai-sink-swift native sink package / libwai slices, desktop tests, simulator tests · macos (pull_request) Successful in 17m37s
Details
WAI Studio / Studio in WebKit · linux (pull_request) Successful in 11m12s
Details
wai-quantum docs, course and locales / every command and snippet in the docs · linux (pull_request) Successful in 40m51s
Details
WAI conformance / v2 vocabulary · manifests validate as stated (pull_request) Successful in 21s
Details
WAI conformance / delivery telemetry · CMCD and CMSD against an independent library (pull_request) Successful in 32s
Details
WAI conformance / cross-arch byte-equality · linux-arm64 (pull_request) Successful in 55s
Details
WAI conformance / C2PA manifest emission · signed, read back, validated (pull_request) Successful in 2m16s
Details
WAI conformance / each feature of the independent verifiers' steps compiles alone (pull_request) Successful in 19m1s
Details
WAI conformance / cargo-class corpora · linux-arm64 (pull_request) Successful in 26m32s
Details
WAI examples / WAI examples · android-app · linux-arm64 (pull_request) Successful in 8m8s
Details
WAI energy receipts / energy receipts · linux-arm64 (pull_request) Successful in 12m25s
Details
WAI HTTP delivery / HTTP corpus · falsify patches 0/3 (pull_request) Successful in 7m48s
Details
WAI HTTP delivery / HTTP corpus · falsify patches 1/3 (pull_request) Successful in 7m41s
Details
WAI HTTP delivery / HTTP corpus · falsify mutations 0/3 (pull_request) Successful in 35m19s
Details
WAI HTTP delivery / HTTP corpus · every patch and mutation in exactly one shard (pull_request) Successful in 7s
Details
WAI HTTP delivery / HTTP corpus · verified both ways, and the differential (pull_request) Successful in 2m15s
Details
WAI HTTP delivery / cache rules · controls and labelled caches (pull_request) Successful in 1m31s
Details
WAI HTTP delivery / HTTP corpus · falsify patches 2/3 (pull_request) Successful in 7m13s
Details
wai-pack / wai-pack · linux-arm64 (pull_request) Successful in 1m56s
Details
wai-pack / wai-pack · the floor through a media prober (pull_request) Successful in 26s
Details
wai-pack / wai-pack · dependency licences (pull_request) Successful in 13s
Details
WAI HTTP delivery / HTTP corpus · falsify mutations 2/3 (pull_request) Successful in 34m7s
Details
wai-sink-android native sink library / the JNI layer and the platform-free half on the host JVM · linux-arm64 (pull_request) Successful in 2m33s
Details
WAI HTTP delivery / HTTP corpus · falsify mutations 1/3 (pull_request) Successful in 39m55s
Details
wai-pack / wai-pack · independent verifier (Python) (pull_request) Successful in 7m7s
Details
WAI conformance / independent verifiers · claims, JWP, key discovery and integer payloads (Python) (pull_request) Successful in 1h31m0s
Details
c303cf55dd 
A receiver that stores what a broadcast carried, a neighbourhood cache or a peer can serve those
objects over a short link; a neighbour that knows who holds what fetches from them before it spends
the backhaul. Nothing said who holds what.

extensions/holdings.md registers `wai.delivery.holdings`: a record (WHD1) of, per group a node holds
objects of, the group's link hash, its leaf count and either `whole` or a bitmap (as a
reconciliation's), in one canonical form, with the instant it states them at and the one it expires
at, signed by the holder under SPEC §7 Signatures. Groups and bitmaps rather than a filter over
hashes: WAI delivers everything under group receipts that name objects by position, a node holds
objects as groups arrive, and a bitmap is exact (no neighbour sent after an object its holder
lacks) at a bit an object, 41 bytes for a group held whole.

A verifier holds a record to its instant and to the group receipts it has (a misstated leaf count
refuses it whole; a group with no receipt held is kept), keeps a holder's latest record and sets
aside two of one instant that differ, answers whether a record holds an object, and derives an
asker's plan: each wanted object's holders in the asker's order, and the objects no neighbour holds,
which come from the origin. An object a neighbour serves is verified under its receipts like any
other; a record is never evidence of bytes. A node advertises only what reached it over a broadcast
bearer, and may sign each record with a fresh key.

The reference is the `holdings` module of wai-rs; the corpus is live-conformance/holdings/ (21
records with a control for each rule, 11 verdicts, 6 queries, a plan over three neighbours), with
holdings_verify.py as its second implementation: 39 cases agree, 9 of 9 mutated copies and 10 of 10
weakened verifiers caught.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
dcharlot closed this pull request 2026-11-04 05:22:05 -05:00
Some checks failed
wai-quantum WebGPU member / member · DX12 and Vulkan (cbio-windows) (pull_request) Successful in 12m26s
Details
joule-moq-remote host agent / joule-moq-remote · does this change touch it (pull_request) Successful in 6s
Details
joule-moq-room room service / joule-moq-room · does this change touch it (pull_request) Successful in 7s
Details
joule-moq-web browser client / joule-moq-web · does this change touch it (pull_request) Successful in 5s
Details
wai-web reach builds / wai-web reach · does this change touch it (pull_request) Successful in 7s
Details
wai-app and Create / wai-app · does this change touch it (pull_request) Successful in 15s
Details
standards workspaces build + test / which workspaces this change touches (pull_request) Successful in 16s
Details
wai-app and Create / wai-app · tests, clippy, the line language's verifier (linux) (pull_request) Successful in 39s
Details
wai-web reach builds / wai-web reach builds · the package of record, every output as recorded (pull_request) Successful in 2m26s
Details
wai-web reach builds / wai-web reach builds · this tree's wai-web, every rung on its engine line (pull_request) Successful in 4m37s
Details
joule-moq-remote host agent / joule-moq-remote, every target (macos) (pull_request) Successful in 3m7s
Details
standards workspaces build + test / macOS energy backends (pull_request) Successful in 6m29s
Details
wai-app and Create / wai-app · Create, browser end-to-end (macos) (pull_request) Successful in 6m0s
Details
joule-moq-web browser client / joule-moq-web · browser end-to-end (macos) (pull_request) Failing after 33m50s
Details
wai-audit collector and player integration / does this change touch it (pull_request) Successful in 14s
Details
WAI breaking changes and the conformance runner / WAI breaking · does this change touch WAI (pull_request) Successful in 12s
Details
WAI conformance / does this change touch it (pull_request) Successful in 11s
Details
WAI operation catalogue / does this change touch it (pull_request) Successful in 12s
Details
WAI energy receipts / does this change touch it (pull_request) Successful in 9s
Details
WAI examples / WAI examples · does this change touch them (pull_request) Successful in 10s
Details
wai-fleet service / does this change touch it (pull_request) Successful in 9s
Details
WAI HTTP delivery / WAI HTTP · does this change touch it (pull_request) Successful in 9s
Details
WAI JPEG-AI whole decode (pinned inputs, required) / does this change touch it (pull_request) Successful in 10s
Details
wai-key-proxy / does this change touch it (pull_request) Successful in 11s
Details
wai-ledger service / does this change touch it (pull_request) Successful in 9s
Details
WAI neural sink gates / does this change touch it (pull_request) Successful in 8s
Details
wai-ops-gpu integer operators on GPUs / does this change touch it (pull_request) Successful in 9s
Details
wai-pack / wai-pack · does this change touch it (pull_request) Successful in 9s
Details
wai-quantum docs, course and locales / does this change touch it (pull_request) Successful in 9s
Details
wai-quantum for JavaScript / does this change touch it (pull_request) Successful in 9s
Details
repository checks / lockfiles, advisories, tests in CI, workflow hygiene, spec and section references, public text (pull_request) Successful in 1m43s
Details
wai-quantum studio and desktop / does this change touch it (pull_request) Successful in 10s
Details
wai-quantum WebGPU member / does this change touch it (pull_request) Successful in 9s
Details
wai-sink-android native sink library / does this change touch it (pull_request) Successful in 10s
Details
WAI mbt2018_mean integer decode (regenerated vectors) / does this change touch it (pull_request) Successful in 11s
Details
wai-sink-swift native sink package / does this change touch it (pull_request) Successful in 12s
Details
WAI Studio / does this change touch it (pull_request) Successful in 8s
Details
wai-web wasm32 / does this change touch it (pull_request) Successful in 11s
Details
WAI conformance / fast integer kernels, every x86_64 kernel set (AVX-512 silicon) (pull_request) Successful in 3m3s
Details
joule-moq-remote host agent / joule-moq-remote, every target (linux-arm64) (pull_request) Successful in 2m22s
Details
joule-moq-room room service / joule-moq-room, every target (pull_request) Successful in 5m27s
Details
wai-quantum WebGPU member / member · integrated GPU, Vulkan (framework-128gb) (pull_request) Successful in 7m58s
Details
wai-quantum studio and desktop / desktop app · windows (pull_request) Successful in 10m33s
Details
standards workspaces build + test / joule-code-rs (joule-code/joule-code-rs, joule-code-rs, linux-arm64, true) (pull_request) Successful in 13m28s
Details
wai-quantum WebGPU member / member · GPU, Vulkan (jetson-hub) (pull_request) Successful in 24m24s
Details
wai-quantum WebGPU member / member · GPU, Vulkan (thor) (pull_request) Successful in 24m49s
Details
wai-audit collector and player integration / collector, player integration and the browser end to end · macos (pull_request) Successful in 3m48s
Details
WAI conformance / cross-arch byte-equality · macos (pull_request) Successful in 2m26s
Details
WAI conformance / codec wrappers · fuzz smoke, text decode limits, C entry points, provenance, the semantic path (pull_request) Successful in 2m38s
Details
wai-fleet service / service · macos (pull_request) Successful in 3m6s
Details
wai-ledger service / service · macos (pull_request) Successful in 2m26s
Details
wai-audit collector and player integration / collector · linux (pull_request) Successful in 2m55s
Details
WAI energy receipts / energy receipts · macos (pull_request) Successful in 11m31s
Details
WAI breaking changes and the conformance runner / WAI breaking · wire, record and API against main · linux (pull_request) Successful in 2m6s
Details
WAI conformance / cargo-class corpora · macos (pull_request) Successful in 21m26s
Details
WAI breaking changes and the conformance runner / WAI breaking and conform · their tests, and the independent verifiers through wai conform · linux (pull_request) Successful in 3m33s
Details
WAI conformance / cross-arch byte-equality · linux (pull_request) Successful in 41s
Details
wai-pack / wai-pack · macos (pull_request) Successful in 5m55s
Details
WAI conformance / quantum core and corpus under wasm32-wasip2 (wasmtime) (pull_request) Successful in 3m31s
Details
WAI conformance / faces from keypoints under wasm32-wasip2 (wasmtime) (pull_request) Successful in 4m19s
Details
wai-quantum studio and desktop / desktop app · macos (pull_request) Successful in 4m53s
Details
WAI examples / WAI examples · the archive, the readiness check, the browser and Apple · macos (pull_request) Successful in 17m3s
Details
wai-fleet service / service · linux (pull_request) Successful in 2m29s
Details
wai-ops-gpu integer operators on GPUs / Metal GPU and browser WebGPU (macos) (pull_request) Successful in 12m55s
Details
wai-quantum WebGPU member / member · browser WebGPU (macos) (pull_request) Successful in 2m44s
Details
WAI operation catalogue / the catalogue and every door · linux (pull_request) Successful in 5m12s
Details
wai-quantum WebGPU member / member · Metal (macos) (pull_request) Successful in 4m33s
Details
wai-key-proxy / proxy · linux (pull_request) Successful in 3m29s
Details
WAI neural sink gates / model-free refusals of the neural sink (pull_request) Successful in 2m5s
Details
wai-ledger service / service · linux (pull_request) Successful in 2m23s
Details
wai-quantum docs, course and locales / the Studio's words in every locale · linux (pull_request) Successful in 31s
Details
WAI JPEG-AI whole decode (pinned inputs, required) / fetch pinned JPEG-AI inputs → whole decode + pixel hash, required (pull_request) Successful in 6m21s
Details
wai-pack / wai-pack · linux (pull_request) Successful in 2m49s
Details
wai-ops-gpu integer operators on GPUs / builds · no adapter is not a pass (linux) (pull_request) Successful in 4m11s
Details
wai-quantum docs, course and locales / the course and the command line's languages · linux (pull_request) Successful in 2m14s
Details
wai-sink-android native sink library / every ABI, the AAR, and the instrumented tests in a headless emulator · macos (pull_request) Successful in 7m19s
Details
WAI examples / WAI examples · portable · linux (pull_request) Successful in 10m45s
Details
wai-quantum for JavaScript / javascript · linux (pull_request) Successful in 3m32s
Details
wai-quantum WebGPU member / member · software Vulkan (linux) (pull_request) Successful in 2m12s
Details
wai-quantum studio and desktop / studio · browser (pull_request) Successful in 4m30s
Details
WAI Studio / the catalogue's actions, the unit tests and the core · linux (pull_request) Successful in 1m16s
Details
WAI conformance / cargo-class corpora · linux (pull_request) Successful in 16m19s
Details
wai-quantum WebGPU member / shader checks · browser build · wasip2 inertness · MSRV (pull_request) Successful in 3m50s
Details
WAI Studio / Studio in Chromium and Firefox · macos (pull_request) Successful in 9m58s
Details
WAI energy receipts / energy receipts · linux (pull_request) Successful in 14m13s
Details
wai-quantum studio and desktop / desktop app · linux (pull_request) Successful in 6m16s
Details
WAI mbt2018_mean integer decode (regenerated vectors) / regenerate mbt2018_mean integer vectors → Rust decode, required (pull_request) Successful in 5m25s
Details
wai-web wasm32 / wai-web wasm32 build · corpus, menu and writers (pull_request) Successful in 6m31s
Details
wai-sink-swift native sink package / libwai slices, desktop tests, simulator tests · macos (pull_request) Successful in 17m37s
Details
WAI Studio / Studio in WebKit · linux (pull_request) Successful in 11m12s
Details
wai-quantum docs, course and locales / every command and snippet in the docs · linux (pull_request) Successful in 40m51s
Details
WAI conformance / v2 vocabulary · manifests validate as stated (pull_request) Successful in 21s
Details
WAI conformance / delivery telemetry · CMCD and CMSD against an independent library (pull_request) Successful in 32s
Details
WAI conformance / cross-arch byte-equality · linux-arm64 (pull_request) Successful in 55s
Details
WAI conformance / C2PA manifest emission · signed, read back, validated (pull_request) Successful in 2m16s
Details
WAI conformance / each feature of the independent verifiers' steps compiles alone (pull_request) Successful in 19m1s
Details
WAI conformance / cargo-class corpora · linux-arm64 (pull_request) Successful in 26m32s
Details
WAI examples / WAI examples · android-app · linux-arm64 (pull_request) Successful in 8m8s
Details
WAI energy receipts / energy receipts · linux-arm64 (pull_request) Successful in 12m25s
Details
WAI HTTP delivery / HTTP corpus · falsify patches 0/3 (pull_request) Successful in 7m48s
Details
WAI HTTP delivery / HTTP corpus · falsify patches 1/3 (pull_request) Successful in 7m41s
Details
WAI HTTP delivery / HTTP corpus · falsify mutations 0/3 (pull_request) Successful in 35m19s
Details
WAI HTTP delivery / HTTP corpus · every patch and mutation in exactly one shard (pull_request) Successful in 7s
Details
WAI HTTP delivery / HTTP corpus · verified both ways, and the differential (pull_request) Successful in 2m15s
Details
WAI HTTP delivery / cache rules · controls and labelled caches (pull_request) Successful in 1m31s
Details
WAI HTTP delivery / HTTP corpus · falsify patches 2/3 (pull_request) Successful in 7m13s
Details
wai-pack / wai-pack · linux-arm64 (pull_request) Successful in 1m56s
Details
wai-pack / wai-pack · the floor through a media prober (pull_request) Successful in 26s
Details
wai-pack / wai-pack · dependency licences (pull_request) Successful in 13s
Details
WAI HTTP delivery / HTTP corpus · falsify mutations 2/3 (pull_request) Successful in 34m7s
Details
wai-sink-android native sink library / the JNI layer and the platform-free half on the host JVM · linux-arm64 (pull_request) Successful in 2m33s
Details
WAI HTTP delivery / HTTP corpus · falsify mutations 1/3 (pull_request) Successful in 39m55s
Details
wai-pack / wai-pack · independent verifier (Python) (pull_request) Successful in 7m7s
Details
WAI conformance / independent verifiers · claims, JWP, key discovery and integer payloads (Python) (pull_request) Successful in 1h31m0s
Details

Pull request closed

Please reopen this pull request to perform a merge.
Sign in to join this conversation.
Reviewers
No reviewers
Labels
Clear labels
No items
No labels
Milestone
Clear milestone
No items
No milestone
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
Transaction-Science/open-standards!605
No description provided.