From 22928147f35595866f37ff5f20e8638ed4b9c2a5 Mon Sep 17 00:00:00 2001 From: Transaction Science Date: Thu, 1 Oct 2026 05:51:50 -0400 Subject: [PATCH 01/12] wai: the parameter-set pin, component identity, pin-aware selection MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit SPEC §3.1 defines the pin a manifest carries (conditioning.prior, and a WAI2 rendition's prior): id, sha256, determinism, and the optional form, tier and supersedes. The digest covers the whole parameter set. A set of several files is pinned through its listing, the new registered prior form wai.prior.set. A sink verifies the pin before it decodes, reports a refusal with one of an ordered set of snake_case reasons, and decodes only from the set it verified. Tier is a closed vocabulary; supersedes is informative. A pin that does not resolve makes its capability unsupported for that envelope: SPEC §4 dispatch continues at the fallback, §7.1 selection at the next rendition, and an inert envelope's error names the reasons. A rendition entry may carry id, sha256 and, on a companion, companion_of_id. SPEC §7 gains a Digests subsection, and §8 says how a MINOR revision adds an optional member. In wai-rs: the manifest types are #[non_exhaustive] with builders; a new pin module verifies pins in memory and on disk; Wai::dispatch_verified, WaiMulti::pick and primary check pins and component digests; every neural decoder reads through ParamSource; a codec-free `ffi` feature carries the C envelope surface with wai_envelope_select and wai_neural_decode_ex. wai-web wraps wai::container. New vectors: the int_mlicpp set-form twin and the pin-conformance corpus with an independent Python verifier, plus sha256.json sidecars for the vectors of capabilities that decode against a parameter set. Squashed from wai/stream-a-parameter-set-identity onto link 12. Conflicts are resolved to keep both sides; the integration commits that follow make the two streams agree. Co-Authored-By: Claude Opus 5.5 --- .github/workflows/wai-conformance.yml | 46 +- .github/workflows/wai-jpegai-e2e.yml | 10 +- .github/workflows/wai-neural-gates.yml | 62 +- .github/workflows/wai-web-wasm.yml | 17 +- wai/CHANGELOG.md | 110 + wai/SPEC.md | 463 +++- .../src/int_video_golden.rs | 36 +- .../src/neural_keyframe_golden.rs | 16 + wai/extensions/determinism-tiers.md | 21 +- wai/extensions/prior-carriage.md | 172 +- wai/neural-conformance/README.md | 10 +- wai/neural-conformance/sha256.json | 18 + wai/pin-conformance/.gitattributes | 2 + wai/pin-conformance/README.md | 43 + wai/pin-conformance/cases.json | 481 ++++ .../listings/bad/blank_line.sha256 | 2 + wai/pin-conformance/listings/bad/bom.sha256 | 1 + .../listings/bad/case_fold.sha256 | 2 + wai/pin-conformance/listings/bad/crlf.sha256 | 1 + .../listings/bad/dot_dot.sha256 | 1 + .../listings/bad/duplicate.sha256 | 2 + wai/pin-conformance/listings/bad/empty.sha256 | 0 .../listings/bad/file_and_directory.sha256 | 2 + .../listings/bad/leading_dot.sha256 | 1 + .../listings/bad/no_final_lf.sha256 | 1 + .../listings/bad/one_space.sha256 | 1 + .../listings/bad/trailing_dot.sha256 | 1 + .../listings/bad/unsorted.sha256 | 2 + .../listings/bad/uppercase_hex.sha256 | 1 + wai/pin-conformance/listings/demo.sha256 | 3 + wai/pin-conformance/listings/tiny.sha256 | 1 + wai/pin-conformance/pin_verify.py | 184 ++ wai/pin-conformance/selection.json | 531 ++++ wai/pin-conformance/sets/demo/.hidden | 1 + wai/pin-conformance/sets/demo/a.b | 1 + wai/pin-conformance/sets/demo/a/c.onnx | 1 + .../sets/demo/weights/hs_y.json | 1 + .../sets/tiny-tampered/model.wmb | 1 + wai/pin-conformance/sets/tiny/model.wmb | 1 + wai/pin-conformance/static.json | 465 ++++ wai/research/2030-transport-landscape.md | 22 +- wai/research/syfai-input.md | 24 +- wai/tools/wai_int_mlicpp_export.py | 4 + wai/tools/wai_int_mlicpp_set_form.py | 72 + wai/vocab/README.md | 11 + wai/vocab/v1.jsonld | 9 + wai/vocab/v1.shapes.ttl | 22 + wai/vocab/v1.ttl | 21 +- wai/wai-quantum-serve/Cargo.lock | 1 + wai/wai-relay/Cargo.lock | 1 + wai/wai-rs/Cargo.toml | 20 +- wai/wai-rs/build.rs | 2 + wai/wai-rs/cbindgen.toml | 1 + wai/wai-rs/include/wai.h | 268 +- wai/wai-rs/src/bin/wai.rs | 108 +- wai/wai-rs/src/bin/wai_int_synth_vectors.rs | 22 +- wai/wai-rs/src/bin/wai_pin_vectors.rs | 304 +++ wai/wai-rs/src/bin/wai_prov.rs | 15 +- wai/wai-rs/src/bin/wai_world_vectors.rs | 15 +- wai/wai-rs/src/codecs/mod.rs | 246 +- wai/wai-rs/src/codecs/prior_form.rs | 57 +- wai/wai-rs/src/container.rs | 2257 +++++++++++++---- wai/wai-rs/src/ffi.rs | 517 +--- wai/wai-rs/src/ffi_codecs.rs | 390 +++ wai/wai-rs/src/ffi_neural.rs | 279 +- wai/wai-rs/src/lib.rs | 29 +- wai/wai-rs/src/neural/audio.rs | 7 +- wai/wai-rs/src/neural/context.rs | 17 +- wai/wai-rs/src/neural/glc.rs | 15 +- wai/wai-rs/src/neural/hyperprior.rs | 45 +- wai/wai-rs/src/neural/image.rs | 106 +- wai/wai-rs/src/neural/int_mlicpp.rs | 32 +- wai/wai-rs/src/neural/int_synth.rs | 24 +- wai/wai-rs/src/neural/jpegai.rs | 56 +- wai/wai-rs/src/neural/mlicpp.rs | 21 +- wai/wai-rs/src/neural/mod.rs | 556 ++-- wai/wai-rs/src/neural/params.rs | 121 + wai/wai-rs/src/neural/runtime.rs | 25 - wai/wai-rs/src/neural/snac.rs | 7 +- wai/wai-rs/src/neural/video.rs | 7 +- wai/wai-rs/src/neural/video_dcvc.rs | 11 +- wai/wai-rs/src/ort_session.rs | 17 + wai/wai-rs/src/pin.rs | 589 +++++ wai/wai-rs/src/quantum_envelope.rs | 15 +- wai/wai-rs/tests/carriage.rs | 22 +- wai/wai-rs/tests/ffi_envelope.rs | 120 + wai/wai-rs/tests/ffi_neural_pin.rs | 120 + wai/wai-rs/tests/generate.rs | 4 + wai/wai-rs/tests/int_kernels_vectors.rs | 94 + wai/wai-rs/tests/int_mlicpp.rs | 99 + wai/wai-rs/tests/jpegai_e2e_full.rs | 88 + wai/wai-rs/tests/manifest_builders.rs | 92 + wai/wai-rs/tests/neural_native_decode.rs | 415 ++- wai/wai-rs/tests/semantic_path.rs | 18 +- wai/wai-rs/tests/support/sha256_sidecar.rs | 58 + wai/wai-rs/tests/vectors/generate/sha256.json | 6 + .../tests/vectors/int_audio/sha256.json | 6 + .../tests/vectors/int_hyper/sha256.json | 6 + .../tests/vectors/int_mlicpp/.gitattributes | 1 + .../tests/vectors/int_mlicpp/model.sha256 | 1 + .../tests/vectors/int_mlicpp/stream_set.wai | Bin 0 -> 910 bytes .../tests/vectors/int_splat/sha256.json | 6 + .../tests/vectors/int_video/sha256.json | 38 + .../tests/vectors/int_world/sha256.json | 17 + .../tests/vectors/int_world2/sha256.json | 17 + wai/wai-rs/tests/world_replay.rs | 15 +- wai/wai-spz-gen/Cargo.lock | 1 + wai/wai-spz-gen/src/main.rs | 15 +- wai/wai-web/Cargo.lock | 142 ++ wai/wai-web/Cargo.toml | 5 + wai/wai-web/src/lib.rs | 698 +++-- 111 files changed, 9105 insertions(+), 2102 deletions(-) create mode 100644 wai/neural-conformance/sha256.json create mode 100644 wai/pin-conformance/.gitattributes create mode 100644 wai/pin-conformance/README.md create mode 100644 wai/pin-conformance/cases.json create mode 100644 wai/pin-conformance/listings/bad/blank_line.sha256 create mode 100644 wai/pin-conformance/listings/bad/bom.sha256 create mode 100644 wai/pin-conformance/listings/bad/case_fold.sha256 create mode 100644 wai/pin-conformance/listings/bad/crlf.sha256 create mode 100644 wai/pin-conformance/listings/bad/dot_dot.sha256 create mode 100644 wai/pin-conformance/listings/bad/duplicate.sha256 create mode 100644 wai/pin-conformance/listings/bad/empty.sha256 create mode 100644 wai/pin-conformance/listings/bad/file_and_directory.sha256 create mode 100644 wai/pin-conformance/listings/bad/leading_dot.sha256 create mode 100644 wai/pin-conformance/listings/bad/no_final_lf.sha256 create mode 100644 wai/pin-conformance/listings/bad/one_space.sha256 create mode 100644 wai/pin-conformance/listings/bad/trailing_dot.sha256 create mode 100644 wai/pin-conformance/listings/bad/unsorted.sha256 create mode 100644 wai/pin-conformance/listings/bad/uppercase_hex.sha256 create mode 100644 wai/pin-conformance/listings/demo.sha256 create mode 100644 wai/pin-conformance/listings/tiny.sha256 create mode 100644 wai/pin-conformance/pin_verify.py create mode 100644 wai/pin-conformance/selection.json create mode 100644 wai/pin-conformance/sets/demo/.hidden create mode 100644 wai/pin-conformance/sets/demo/a.b create mode 100644 wai/pin-conformance/sets/demo/a/c.onnx create mode 100644 wai/pin-conformance/sets/demo/weights/hs_y.json create mode 100644 wai/pin-conformance/sets/tiny-tampered/model.wmb create mode 100644 wai/pin-conformance/sets/tiny/model.wmb create mode 100644 wai/pin-conformance/static.json create mode 100644 wai/tools/wai_int_mlicpp_set_form.py create mode 100644 wai/wai-rs/src/bin/wai_pin_vectors.rs create mode 100644 wai/wai-rs/src/ffi_codecs.rs create mode 100644 wai/wai-rs/src/neural/params.rs delete mode 100644 wai/wai-rs/src/neural/runtime.rs create mode 100644 wai/wai-rs/src/pin.rs create mode 100644 wai/wai-rs/tests/ffi_envelope.rs create mode 100644 wai/wai-rs/tests/ffi_neural_pin.rs create mode 100644 wai/wai-rs/tests/manifest_builders.rs create mode 100644 wai/wai-rs/tests/support/sha256_sidecar.rs create mode 100644 wai/wai-rs/tests/vectors/generate/sha256.json create mode 100644 wai/wai-rs/tests/vectors/int_audio/sha256.json create mode 100644 wai/wai-rs/tests/vectors/int_hyper/sha256.json create mode 100644 wai/wai-rs/tests/vectors/int_mlicpp/.gitattributes create mode 100644 wai/wai-rs/tests/vectors/int_mlicpp/model.sha256 create mode 100644 wai/wai-rs/tests/vectors/int_mlicpp/stream_set.wai create mode 100644 wai/wai-rs/tests/vectors/int_splat/sha256.json create mode 100644 wai/wai-rs/tests/vectors/int_video/sha256.json create mode 100644 wai/wai-rs/tests/vectors/int_world/sha256.json create mode 100644 wai/wai-rs/tests/vectors/int_world2/sha256.json diff --git a/.github/workflows/wai-conformance.yml b/.github/workflows/wai-conformance.yml index 9446e6443..e04038a52 100644 --- a/.github/workflows/wai-conformance.yml +++ b/.github/workflows/wai-conformance.yml @@ -24,12 +24,13 @@ on: # The registry export and the two library tests that read files outside # wai-rs (the codec glossary, SPEC.md) must fire this workflow too: a # hand edit of an exported file, of SPEC §5 or of the glossary is checked - # here. + # here. vocab/v1's and v2's pin terms are checked by a library test. - 'wai/registry/**' - 'wai/vocab/**' - 'wai/glossary/**' - 'wai/SPEC.md' - 'wai/REGISTRY.md' + - 'wai/wai-web/**' - '.github/workflows/wai-conformance.yml' pull_request: paths: @@ -42,12 +43,13 @@ on: # The registry export and the two library tests that read files outside # wai-rs (the codec glossary, SPEC.md) must fire this workflow too: a # hand edit of an exported file, of SPEC §5 or of the glossary is checked - # here. + # here. vocab/v1's and v2's pin terms are checked by a library test. - 'wai/registry/**' - 'wai/vocab/**' - 'wai/glossary/**' - 'wai/SPEC.md' - 'wai/REGISTRY.md' + - 'wai/wai-web/**' - '.github/workflows/wai-conformance.yml' # No global `shell:` default. The Windows runner is a host executor whose @@ -151,19 +153,27 @@ jobs: # export, and check each vector's metadata. Cargo.toml declares them, and the other # feature-gated targets named here, with required-features, so if this step ever loses a # feature, cargo stops with an error instead of compiling an empty binary that reports ok. + # Each committed vector's decoded output is also checked against the SHA-256 in + # its sha256.json sidecar (SPEC §7 *Digests*), and + # int_kernels_vectors::every_decoded_output_states_a_sha256 requires a sidecar, + # or a named exclusion, for every corpus directory. tests/generate.rs checks the + # wai.generate.spectral vector, which `generate` enables. - name: Fast integer kernels ≡ reference kernels (overflow checks on) working-directory: wai/wai-rs - run: cargo test --no-default-features --features neural_int,video_int,audio_int,world_int,splat_int --lib --test int_kernels_vectors --test int_kernels_geometry --test int_mlicpp --test int_mlicpp_preflight --test int_audio --test int_splat + run: cargo test --no-default-features --features neural_int,video_int,audio_int,world_int,splat_int,generate --lib --test int_kernels_vectors --test int_kernels_geometry --test int_mlicpp --test int_mlicpp_preflight --test int_audio --test int_splat --test generate - name: Fast integer kernels ≡ reference kernels (release, overflow checks off) working-directory: wai/wai-rs - run: cargo test --release --no-default-features --features neural_int,video_int,audio_int,world_int,splat_int --lib --test int_kernels_vectors --test int_kernels_geometry --test int_mlicpp --test int_mlicpp_preflight --test int_audio --test int_splat + run: cargo test --release --no-default-features --features neural_int,video_int,audio_int,world_int,splat_int,generate --lib --test int_kernels_vectors --test int_kernels_geometry --test int_mlicpp --test int_mlicpp_preflight --test int_audio --test int_splat --test generate # carriage: every registered capability round-trips byte-exact. cli_wrap_intent and # cli_registry run the `wai` binary, which builds without the codecs feature: the # intent rule of `wai wrap`, its refusal of candidate capabilities (SPEC §5), its media # class, and `wai registry export --check` on the tree and on a drifted copy. + # tests/manifest_builders.rs: the manifest types are #[non_exhaustive], and the + # builders write the JSON a struct literal wrote; members the types keep only + # as `extra` (and `security`) survive a round trip. - name: Carriage conformance and the wai CLI (wrap, registry export) working-directory: wai/wai-rs - run: cargo test --no-default-features --test carriage --test cli_wrap_intent --test cli_registry + run: cargo test --no-default-features --test carriage --test manifest_builders --test cli_wrap_intent --test cli_registry # The machine-readable registry (wai/registry/, wai/vocab/v2.*, the table in # wai/REGISTRY.md) is generated from wai-rs/src/codecs/registry.rs. Regenerate it in # memory and fail on any difference; the message says how to regenerate. One leg: the @@ -172,6 +182,22 @@ jobs: if: matrix.os == 'linux' working-directory: wai/wai-rs run: cargo run -q --no-default-features --bin wai -- registry export --check + # The C ABI's envelope surface builds with no codec library (feature `ffi`): + # unpack returns each manifest as the envelope carries it, the WAI2 table carries + # every component's identity and selectability, and wai_envelope_select follows + # SPEC §7.1. + - name: C ABI envelope surface (no codec library) + working-directory: wai/wai-rs + run: cargo test --no-default-features --features ffi --test ffi_envelope + # The browser crate wraps the same envelope types (wai::container), so role, + # companion_of, component identity and the pin reach JS, and its pick is SPEC + # §7.1's. Its tests target the pure-Rust half (`core`), which runs natively; one + # leg suffices, the logic is architecture-independent. The JS surface itself runs + # under wasm32 in wai-web-wasm.yml. + - name: wai-web envelope surface (native tests) + if: matrix.os == 'linux' + working-directory: wai/wai-web + run: cargo test --locked # The corpus-verify loop is genuinely shell-specific (a helper that groups # log output and propagates the first non-zero exit), so it ships once per @@ -197,6 +223,15 @@ jobs: v quantum-ops wai_quantum_ops_vectors ../quantum-ops-conformance v oer wai_oer_vectors ../oer-conformance v claims wai_claim_vectors ../claim-conformance + v pin wai_pin_vectors ../pin-conformance + # The pin corpus's second implementation: Python standard library only, sharing no + # code with wai-rs. One leg is enough — it checks digests and listing rules, which + # do not depend on the architecture — and the macOS host has python3. + - name: The pin corpus checks against an independent verifier (macOS) + if: matrix.os == 'macos' + shell: bash + working-directory: wai + run: python3 pin-conformance/pin_verify.py pin-conformance # OER's second implementation is portable C, not Rust. Rebuild it from source and # require its output to match the pinned corpus byte for byte: the kernel is # deterministic, and wai_oer_vectors above has already verified the pinned copy, @@ -237,6 +272,7 @@ jobs: v quantum-ops wai_quantum_ops_vectors ../quantum-ops-conformance v oer wai_oer_vectors ../oer-conformance v claims wai_claim_vectors ../claim-conformance + v pin wai_pin_vectors ../pin-conformance # The generated v2 vocabulary (wai/vocab/v2.*) is checked as a consumer uses it: # sample manifests, typed and untyped, WAI1 and WAI2, are expanded through the diff --git a/.github/workflows/wai-jpegai-e2e.yml b/.github/workflows/wai-jpegai-e2e.yml index 43b41b85e..f35df3c50 100644 --- a/.github/workflows/wai-jpegai-e2e.yml +++ b/.github/workflows/wai-jpegai-e2e.yml @@ -68,6 +68,10 @@ on: paths: - 'wai/wai-rs/src/jpegai_*.rs' - 'wai/wai-rs/src/reconstruction_binding.rs' + # The set-form pin over the exact bundle (tests/jpegai_e2e_full.rs) verifies + # through the pin module. + - 'wai/wai-rs/src/pin.rs' + - 'wai/wai-rs/src/container.rs' - 'wai/wai-rs/src/int_transformer.rs' - 'wai/wai-rs/src/int_hyper_synth.rs' - 'wai/wai-rs/src/int_kernels.rs' @@ -89,6 +93,10 @@ on: paths: - 'wai/wai-rs/src/jpegai_*.rs' - 'wai/wai-rs/src/reconstruction_binding.rs' + # The set-form pin over the exact bundle (tests/jpegai_e2e_full.rs) verifies + # through the pin module. + - 'wai/wai-rs/src/pin.rs' + - 'wai/wai-rs/src/container.rs' - 'wai/wai-rs/src/int_transformer.rs' - 'wai/wai-rs/src/int_hyper_synth.rs' - 'wai/wai-rs/src/int_kernels.rs' @@ -157,7 +165,7 @@ jobs: # A test that was renamed or compiled out would not fail — it would just # not run. Require each one by name, and the pinned-hash check by its line. for t in codestream_to_rgb_matches_reference_png real_decode_to_reconstruction_binding \ - decode_matches_pinned_hashes; do + decode_matches_pinned_hashes jpegai_e2e_set_pin_verifies; do grep -q "^test $t \.\.\. ok$" "$log" || { echo "::error::$t did not run and pass"; exit 1; } done grep -q "matches the pinned WAI_JPEGAI_EXPECT_SHA256" "$log" \ diff --git a/.github/workflows/wai-neural-gates.yml b/.github/workflows/wai-neural-gates.yml index b3d9573ac..99ecc3a1a 100644 --- a/.github/workflows/wai-neural-gates.yml +++ b/.github/workflows/wai-neural-gates.yml @@ -1,8 +1,8 @@ name: WAI neural sink gates -# The neural sink refuses some envelopes before it touches a model: a pinned -# prior whose hash does not match the registered model, or that declares a form -# the sink cannot read (ModelRegistry::resolve_checked), and a `replicate` that +# The neural sink refuses some envelopes before it touches a model: a pin that +# the parameter sets it holds do not satisfy, with SPEC §3.1's reason +# (ModelRegistry::resolve_verified, decode_envelope, decode_multi), and a `replicate` that # SPEC §7 does not back (decode_envelope_strict): over a capability whose registry # class cannot back it, or with a prior that is float or absent, or that declares a # form not established to reconstruct the same tensors at every reader. Those @@ -12,10 +12,14 @@ name: WAI neural sink gates # them. This job runs exactly those tests and fails if any of them is not run: # a filter that matches nothing reports "0 passed" and exits 0. # -# It also runs the one `neural` dispatch test that needs no model download: -# `wai.neural.int_mlicpp` through the strict gate on its committed conformance vector -# (the pinned image; a one-byte-different model bundle refused by the prior pin; -# `replicate` without a prior refused). +# It also runs the `neural` dispatch tests that need no model download, over the +# committed int_mlicpp vector and the int_synth corpus: the form-less and set-form +# pins through the strict gate, the int_synth pin over the spec it reads, the +# several-file capabilities refusing a form-less pin, a pinned decode reading only +# the bytes it verified, a WAI2 envelope whose pins pick the held model, the strict +# rule read for the capability decoded through (a sound fallback behind an unsound +# primary), SPEC §4 step 6's order of reasons after a set_incomplete decode, and the +# replicate_* reason the strict WAI2 policy reports for an entry it leaves out. on: push: @@ -25,6 +29,14 @@ on: - 'wai/wai-rs/src/container.rs' - 'wai/wai-rs/src/codecs/mod.rs' - 'wai/wai-rs/src/codecs/prior_form.rs' + - 'wai/wai-rs/src/pin.rs' + - 'wai/wai-rs/src/ffi.rs' + - 'wai/wai-rs/src/ffi_neural.rs' + - 'wai/wai-rs/tests/ffi_neural_pin.rs' + - 'wai/wai-rs/src/int_synth.rs' + - 'wai/wai-rs/src/ort_session.rs' + - 'wai/neural-conformance/vectors/**' + - 'wai/pin-conformance/**' - 'wai/wai-rs/tests/neural_native_decode.rs' - 'wai/wai-rs/src/int_mlicpp.rs' - 'wai/wai-rs/tests/vectors/int_mlicpp/**' @@ -37,6 +49,14 @@ on: - 'wai/wai-rs/src/container.rs' - 'wai/wai-rs/src/codecs/mod.rs' - 'wai/wai-rs/src/codecs/prior_form.rs' + - 'wai/wai-rs/src/pin.rs' + - 'wai/wai-rs/src/ffi.rs' + - 'wai/wai-rs/src/ffi_neural.rs' + - 'wai/wai-rs/tests/ffi_neural_pin.rs' + - 'wai/wai-rs/src/int_synth.rs' + - 'wai/wai-rs/src/ort_session.rs' + - 'wai/neural-conformance/vectors/**' + - 'wai/pin-conformance/**' - 'wai/wai-rs/tests/neural_native_decode.rs' - 'wai/wai-rs/src/int_mlicpp.rs' - 'wai/wai-rs/tests/vectors/int_mlicpp/**' @@ -75,6 +95,30 @@ jobs: set -euo pipefail cargo test --no-default-features --features neural --test neural_native_decode -- \ --exact resolve_checked_verifies_prior_hash strict_decode_gates_unsound_replicate \ - int_mlicpp_decodes_through_the_strict_dispatch \ + int_mlicpp_decodes_through_the_strict_dispatch int_mlicpp_set_pin_through_the_strict_dispatch \ + int_synth_pin_covers_the_spec_it_reads several_file_capabilities_need_the_set_form \ + pinned_decode_reads_only_verified_bytes wai2_decode_multi_picks_the_verified_rendition \ + strict_gate_reads_the_capability_it_decodes_through set_incomplete_at_step_3_outranks_the_fallback_refusal \ + decode_multi_strict_reports_replicate_reasons \ 2>&1 | tee "$RUNNER_TEMP/neural-gates.log" - grep -q "test result: ok. 3 passed; 0 failed" "$RUNNER_TEMP/neural-gates.log" + grep -q "test result: ok. 11 passed; 0 failed" "$RUNNER_TEMP/neural-gates.log" + # Library unit tests that exist only under `neural` and need no model: the + # bmshj2018 CDF table cache stays bounded and evicts least recently used. + - name: Model-free neural unit tests + working-directory: wai/wai-rs + shell: bash + run: | + set -euo pipefail + cargo test --no-default-features --features neural --lib -- --exact neural::image::tests::cdf_cache_is_bounded \ + 2>&1 | tee "$RUNNER_TEMP/neural-lib.log" + grep -q "test result: ok. 1 passed; 0 failed" "$RUNNER_TEMP/neural-lib.log" + - name: The neural C ABI reports pin and replicate refusals and decodes WAI2 + working-directory: wai/wai-rs + shell: bash + run: | + set -euo pipefail + cargo test --no-default-features --features neural,ffi --test ffi_neural_pin -- \ + --exact tampered_model_is_wai_err_prior strict_replicate_is_wai_err_unsound_replicate \ + wai2_decode_ex_picks_the_held_pin failed_decode_ex_clears_out \ + 2>&1 | tee "$RUNNER_TEMP/neural-ffi.log" + grep -q "test result: ok. 4 passed; 0 failed" "$RUNNER_TEMP/neural-ffi.log" diff --git a/.github/workflows/wai-web-wasm.yml b/.github/workflows/wai-web-wasm.yml index 793c6fdc4..23a141843 100644 --- a/.github/workflows/wai-web-wasm.yml +++ b/.github/workflows/wai-web-wasm.yml @@ -12,7 +12,8 @@ name: wai-web wasm32 # registry agrees with the export on every candidate; # - demo/verify_pack.mjs: the browser writers `pack` and `packMulti` refuse # every candidate the export lists, in every place an envelope can name one; -# - a byte comparison of the demo's staged int_synth copies with the corpus. +# - a byte comparison of the demo's staged int_synth copies with the corpus; +# - the crate's wasm_tests under wasm-bindgen-test-runner. # # The full-scale int_hyper wasm decode needs vectors regenerated from a # pretrained model; wai-sota-vectors.yml runs it after regenerating them. @@ -70,6 +71,20 @@ jobs: cargo build --locked --release --target wasm32-unknown-unknown wasm-bindgen --target nodejs --out-dir demo/pkg-node --out-name wai_web \ target/wasm32-unknown-unknown/release/wai_web.wasm + # The JS surface itself (pick, companionsOf, verifyPin, rendition() and + # companion() refusing what is never presented or does not match its digest, + # the held-sets conversion, and the WAI2 parse at 32-bit usize) runs only on + # wasm32 (`mod wasm_tests`). wasm-bindgen-test-runner, installed above at the + # version Cargo.lock pins, runs it under Node. The count check turns a test + # that stopped being compiled into a failure instead of a silent "0 passed". + - name: wai-web JS surface (wasm32 tests under Node) + shell: bash + working-directory: wai/wai-web + run: | + set -euo pipefail + CARGO_TARGET_WASM32_UNKNOWN_UNKNOWN_RUNNER=wasm-bindgen-test-runner \ + cargo test --locked --target wasm32-unknown-unknown 2>&1 | tee "$RUNNER_TEMP/wai-web-wasm.log" + grep -q "test result: ok. 4 passed; 0 failed" "$RUNNER_TEMP/wai-web-wasm.log" - name: wasm32 int_synth decode equals the frozen native corpus shell: bash working-directory: wai/wai-web diff --git a/wai/CHANGELOG.md b/wai/CHANGELOG.md index 8e5d007c7..91945827d 100644 --- a/wai/CHANGELOG.md +++ b/wai/CHANGELOG.md @@ -126,6 +126,116 @@ wire format may change and such changes are recorded here. cases §5.1–§5.5 define; verifying against their vectors becomes a MUST once the vectors are published in the tree. +### The parameter-set pin, specified; component identity; pin-aware selection + +- **SPEC §3.1** defines the pin a manifest carries (`conditioning.prior`; a `WAI2` + rendition's `prior`): `id`, `sha256`, `determinism`, and the optional `form`, + `tier` and `supersedes`. The digest covers the whole parameter set: a form-less + pin is admissible only for a capability registered with a one-file set (§5 + registers each capability's shape), and a set of several files is pinned through + its listing, the new registered prior form `wai.prior.set` (`StandardDefined`, + royalty-free, the one form listed as established exact). A sink verifies the pin + before it decodes, reports a refusal with one of an ordered set of snake_case + reasons (`pin_malformed`, `form_not_registered`, `form_unreadable`, + `form_required`, the reserved `prior_revoked`, `prior_not_held` / + `digest_superseded`, `set_malformed`, `part_mismatch`; `set_incomplete` during a + decode), and decodes only from the set it verified. A pin over a capability this + revision does not register is verified as several files. `null` in an optional + member is the same as absent. +- **Tier** is a closed vocabulary (`decode-equivalence`, `entropy-consistency`, + `none`): an encoder writes a registered token, a sink reads any other as `none`, + a verifier reports it (`tier_unregistered`). Pins without `tier` keep the legacy + reading of `determinism`, now stated normatively, so every existing verdict — + `tests/vectors/int_mlicpp/stream.wai`'s included — is unchanged. +- **`supersedes` is informative**: it changes only the reason a sink that holds the + predecessor reports, a malformed one is a static finding the sink ignores, and a + successor never satisfies its predecessor's pin. +- **§4 and §7.1**: a pin that does not resolve makes its capability unsupported for + that envelope; dispatch continues at the fallback or the next rendition, and an + inert envelope's error names the reasons: for `WAI1`, the reason kept for the + capability before the fallback's, `set_incomplete` included; for `WAI2`, every + entry's (`not_selectable`, `missing_capability`, `component_digest_mismatch`, a + pin reason, or, under a policy that serves only sound `replicate` claims, the + entry's `replicate_*` reason). A `WAI1` pin applies to whichever capability + decodes the payload. +- **Component identity (§7.1)**: a rendition entry MAY carry `id` (unique, never a + capability string), `sha256` (its payload digest, checked before the entry is + decoded or applied: `component_digest_mismatch`) and, on a companion, + `companion_of_id`, which binds it to one entry — an enhancement layer can now + bind to one of two renditions of a base capability. `id`, `sha256` and `prior` + are one construct; no extension defines another per-component digest. +- **§7**: `replicate` over a `NeuralIntegerExact` capability MUST carry a pin that + declares `decode-equivalence`, in no form or in `wai.prior.set`. The new + *Digests* subsection names SHA-256 for parameter-set identity, for component + identity, and for the decoded output of capabilities that decode against a + parameter set and of standard codecs in conformance material; FNV-1a and BLAKE3 + stay as diagnostics. The vectors of such capabilities gain `sha256.json` + sidecars; no existing vector file or value changes. +- **§8** settles how an optional member is added under the 1.x freeze, which the + `form` entry below left open: a MINOR revision MAY define an OPTIONAL member, + envelopes keep `wai` `"1.0"`/`"1.1"`, a reader MUST ignore a member it does not + understand whatever the MINOR, and an earlier reader may instead reject such an + envelope (fails closed) or ignore the pin (fails open), which §3.1 states. +- **Envelopes made non-conforming:** the list in SPEC §3.1. None is committed. + One kind stops parsing, as §8 now records: a manifest that carries a member this + revision defines (a pin or pin member, or a rendition entry's `id`, `sha256` or + `companion_of_id`) with another JSON type is malformed, because reading a + mistyped `form` as absent would turn a set-form pin into a form-less one. + New vectors: `tests/vectors/int_mlicpp/{model.sha256, stream_set.wai}` (the + set-form twin, written by `tools/wai_int_mlicpp_set_form.py`) and the + `pin-conformance/` corpus, with an independent verifier in Python. +- **Reference-sink behaviour:** a form-less pin over a several-file capability is + refused (`form_required`), where before it verified one file or failed on a + directory; `wai.neural.int_synth`'s pin covers `int_synth.json`; a digest that is + not 64 lowercase hex is refused, where before it was compared case-insensitively; + a refused pin falls through to a fallback the registry holds; every decoder reads + parameters through `neural::ParamSource`, so a pinned decode reads only verified + bytes (ONNX sessions built from them); `bmshj2018`'s CDF cache is keyed by content + digest and holds a bounded number of tables, least recently used evicted. + `decode_envelope_strict` applies §7's rule to the capability it decodes through, + so a sound fallback behind an unsound primary the sink cannot use still decodes; + it refuses before reading a file only when every capability it could decode + through is refused. A `WAI2` parse sums the rendition lengths in 64 bits, so a + table whose lengths wrap a 32-bit sum is refused as truncated on 32-bit targets + too, where it used to panic. `wai inspect` prints digests through + `pin::digest_preview`, which shows a malformed one whole instead of cutting it + inside a character. +- **Rust API, source-breaking** (the same 0.4.0 release as `form`): the manifest + types (`Manifest`, `ModelRequirement`, `Conditioning`, `Prior`, `Supersedes`, + `ManifestV2`, `RenditionMeta`, `Rendition`, `Security`) are now + `#[non_exhaustive]`, with constructors and `with_*` builders that write the same + JSON a literal wrote, so the next optional member does not break callers again; + every manifest object keeps members it does not model (`extra`), and `Manifest` + gains `security`, so a Rust round trip loses nothing. `Rendition` holds a + `RenditionMeta`. `Wai::dispatch_capability` is replaced by `dispatch_verified`; + `WaiMulti::pick`/`primary` take a pin verifier and return `Picked` (index, entry, + payload, verified set, skipped entries); `companions_of_index` binds by id. + `DecodeError::PriorMismatch` carries a `PinRefusal`, and `DecodeError` gains + `NoRendition`; `ModelRegistry` gains `held`/`hold`/`verify`/`resolve_verified` + (`resolve_checked` is deprecated); new `decode_payload`, `decode_multi`, + `decode_multi_strict`. `sha2` is now a plain dependency, and the new `pin` module + needs no feature. +- **C ABI:** a codec-free `ffi` feature carries the envelope surface (`codecs` + implies it); `WAI_ERR_PRIOR` (5), `WAI_ERR_UNSOUND_REPLICATE` (6) and + `WAI_ERR_NO_RENDITION` (7), with the reason first in `wai_last_error`; the unpack + calls return the original manifest bytes, as `wai.h` already documented; the + `WAI2` table carries identity, pin and `selectable`; new `wai_envelope_select` + and `wai_neural_decode_ex` (`WAI1` or `WAI2`, a registry of held sets, a strict + flag), which writes `*out` on every call, empty on failure, so `out->data` is + always safe to free. +- **wai-web** wraps `wai::container` instead of reduced copies, so `role`, + `companion_of`, identity and pins reach JS, and exposes `pick`, `companionsOf`, + `verifyPin` and `sha256Hex`; a companion-only component is never presented as + media, and `rendition(i)` and `companion(i)` refuse an entry whose payload does not + match its `sha256`. Its JS-facing tests run under wasm32 in CI. +- **Vocabulary:** `vocab/v1` gains the pin terms additively; `sharedPriorUri` and + `sharedPriorHash` are annotated as not being the manifest pin. The SHACL pin shape + targets the objects of `wai:prior` (no class is asserted for them), and + `supersedes` is checked by its own shape: a required digest and an optional form. +- **Open item:** `wai.world.model` names its renderer's model by `model_id`, a + second model-identity construct; it is to fold into the pin once that renderer + is registered as decoding against a parameter set. + ### Typed receipt classes, and a second (soft) binding class - [`extensions/jwp-receipts.md`](extensions/jwp-receipts.md) §2.4.3–§2.4.7: every diff --git a/wai/SPEC.md b/wai/SPEC.md index 6aaea51f4..2437b396e 100644 --- a/wai/SPEC.md +++ b/wai/SPEC.md @@ -8,8 +8,10 @@ file: compact conditioning payload against a *shared ambient prior* (a model). The model is a **requirement of existence**, never transported in the envelope that decodes against it, exactly as `.mp3` - does not ship its decoder. Its identity MAY be pinned by digest (a - `Prior`), and the pinned artifact's wire form MAY be declared + does not ship its decoder. Its identity MAY be pinned: the manifest MAY + carry a *pin*, the SHA-256 of the whole parameter set the payload was coded + against (§3.1), and a sink whose parameter set does not match the pin does + not decode against it (§4, §7.1). The pinned set's wire form MAY be declared ([prior-carriage](extensions/prior-carriage.md)). - the **zeroth condition** — a *menu of registered standard codecs* (AVIF, JPEG XL, PNG, Opus, FLAC, AV1, zstd, XZ and the others §5 @@ -40,7 +42,9 @@ A WAI file declares (in its manifest): - what media it carries, - what *capability* a sink must have to decode it, - a fallback capability the sink can use if it lacks the primary, -- and the conditioning bytes for that capability. +- the conditioning bytes for that capability, +- and, OPTIONALLY, for a capability that decodes against a parameter set, the + pin of the set the payload was coded against (§3.1). Conforming sinks MUST implement the container envelope (§2), the manifest schema (§3), the capability-dispatch algorithm (§4), and at @@ -66,7 +70,9 @@ OPTIONAL; §5 says which are RECOMMENDED. on key order for *semantics*; encoders SHOULD preserve order so files round-trip byte-identically. - The payload is opaque codec bytes whose interpretation is fully - determined by the manifest's `model_requirement.capability`. + determined by the manifest's `model_requirement.capability` and, for a + capability that decodes against a parameter set, by that set, which the + manifest MAY pin (§3.1). All WAI multi-byte integers are little-endian unless stated otherwise. @@ -82,14 +88,15 @@ All WAI multi-byte integers are little-endian unless stated otherwise. "intent": "replicate" | "create" | "improve" | "...", "model_requirement": { "capability": "", "fallback": "" | null }, - "conditioning": { "kind": "" }, + "conditioning": { "kind": "", "prior": { ... } }, "security": { "encryption": "" | "none", "kid": }, "target": { ... } } ``` - `wai` — spec version. This document defines `"1.0"`. Unknown major → - reject. Unknown minor → accept by ignoring unknown fields. + reject. Unknown minor → accept. A reader ignores a member it does not + understand, whatever the minor (§8). - `media` — media class; informational. The capability is authoritative. The values are the registry's media classes (`codecs::capability_media`; [`registry/capabilities.json`](registry/capabilities.json)): `image`, @@ -111,7 +118,7 @@ All WAI multi-byte integers are little-endian unless stated otherwise. what its standard's decoder conformance states, which the registry does not assert is byte identity. For a **neural** capability, `replicate` promises decode-equivalence and is sound - only under the integer-entropy contract (§7): a capability whose decode runs + only under the integer-entropy contract (§7), with a pin (§3.1, §7): a capability whose decode runs float, or whose entropy decode alone is integer, MUST NOT claim it. That is WAI's rule, and this revision applies it to `wai.image.jpegai` as well; it is not a finding that JPEG AI's standard leaves the reconstruction unbounded (§7, @@ -119,10 +126,18 @@ All WAI multi-byte integers are little-endian unless stated otherwise. - `model_requirement.capability` — the capability string a sink MUST resolve. See §5 for the registered set. - `model_requirement.fallback` — a second capability the sink MAY use - if it lacks the primary. `null` means the file is inert without the + if it lacks the primary, or if the envelope's pin does not resolve for the + primary at the sink (§4 step 3). `null` means the file is inert without the primary. - `conditioning.kind` — short codec id matching the capability. Helps multiplexers route bytes without parsing the full capability string. +- `conditioning.prior` — OPTIONAL. The pin (§3.1) of the parameter set the + payload was coded against. It applies to whichever of + `model_requirement.capability` and `model_requirement.fallback` decodes the + payload (§4), where that capability decodes against a parameter set. Absent, or + `null`, the key is not written, and a decode through such a capability is + unpinned: it runs against whatever parameter set the sink holds, and it cannot + back `intent = replicate` over a `NeuralIntegerExact` capability (§7). - `security` — OPTIONAL. Absent, `null`, or `encryption: "none"` ⇒ the payload is plaintext (the default). Otherwise the payload is encrypted and MUST be recovered before dispatch; see §6. The capability, media, @@ -131,6 +146,190 @@ All WAI multi-byte integers are little-endian unless stated otherwise. sample rate, duration, frame count). Informational; the payload is self-describing per the codec's own format. +### 3.1 Parameter-set pin (`prior`) + +A capability **decodes against a parameter set** when its registry class (§7) is +`NeuralIntegerExact`, `NeuralEntropyExact`, `NeuralFloat` or +`GenerativeDeterministic`. Its **parameter set** is every byte, other than the +payload, that its decode reads and that its output depends on: networks, entropy +tables, normative constants. The sink supplies it; the envelope never carries it +(§1). §5 registers each such capability's parameter set as **one file** or +**several files**; a capability registered with neither, and a capability this +revision does not register, is treated as several files. + +A manifest names the parameter set a payload was coded against with a **pin**: + +```json +"prior": { + "id": "", + "sha256": "<64 lowercase hexadecimal digits>", + "determinism": "", + "form": "", + "tier": "decode-equivalence", + "supersedes": { "sha256": "<64 lowercase hexadecimal digits>", "form": "" } +} +``` + +| member | presence | meaning | +|---|---|---| +| `id` | REQUIRED, string | A name for people and logs. A sink MUST NOT select, resolve or accept a parameter set by it. | +| `sha256` | REQUIRED, string | The SHA-256 (FIPS 180-4) of the parameter set in its form (below), as 64 lowercase hexadecimal digits. | +| `determinism` | REQUIRED, string | How the decode is made reproducible, in words. Informative, except where §7 reads it for a pin without `tier`. | +| `form` | OPTIONAL, string | A registered prior form (§5; [prior-carriage](extensions/prior-carriage.md)). | +| `tier` | OPTIONAL, string | The reproducibility tier ([determinism-tiers](extensions/determinism-tiers.md) §2) the sender claims for the decode, from a closed vocabulary: `decode-equivalence`, `entropy-consistency`, or `none` for neither. | +| `supersedes` | OPTIONAL, object | Informative. The parameter set this one replaces, named as a pin names one: its `sha256` (REQUIRED) and, if it is pinned in a declared form, that `form`. | + +A `prior`, or a member of it, whose JSON type is not the one given makes the +manifest malformed, and a reader rejects it as it rejects any manifest that does +not parse. A mistyped member is never read as absent: a `form` dropped that way +would turn a set-form pin into a form-less one, which a single file can satisfy. `null` in an OPTIONAL member is the same as the member being absent. +A member this section does not define is ignored (§8). An optional member that is +absent is not written, so a pin written without it keeps its bytes. + +**What the digest covers.** A pin identifies the whole parameter set. + +- With `form` absent, `sha256` is the SHA-256 of the one file that holds the set. + That is admissible only for a capability registered with a one-file set. +- With `form` present, `sha256` is the SHA-256 of the set serialized in exactly + that form ([prior-carriage](extensions/prior-carriage.md) §2). The form + `wai.prior.set` serializes a set of one or more files as its **listing**, so a + pin in that form covers every file of the set: + - The listing has one line for each file: the file's SHA-256 as 64 lowercase + hexadecimal digits, two spaces (0x20 0x20), the file's path, and a line feed + (0x0A). + - A path names the file below the set's root: one or more segments joined by + `/` (0x2F). A segment begins with one of `A`–`Z`, `a`–`z`, `0`–`9`, `_` and + `-`, continues with any of those and `.`, and does not end with `.`. + - The lines are in strictly ascending order of their paths compared as byte + strings. No two paths are equal once `A`–`Z` are mapped to `a`–`z`, and no + path followed by `/` begins another path. + - The listing is its lines concatenated. It has at least one line and nothing + else: no header, no byte-order mark, no carriage return, no blank line. + + A listing is the text common checksum utilities write and check, so a third + party can check a set against its listing, and a listing against a pin, without + WAI software. + +**Verification.** A sink that is to decode through a capability, for an envelope +whose pin applies to that capability, MUST verify the pin first. A pin applies to +a capability that decodes against a parameter set, and to a capability this +revision does not register when the sink advertises it: such a capability is +verified as several files, so a form-less pin over it is refused +(`form_required`). The pin **resolves** when the sink holds a parameter set for the +capability whose digest, in the pin's form, equals `sha256`: in the form +`wai.prior.set`, a listing whose SHA-256 is `sha256`, together with every file the +listing names below the listing's root, each with the SHA-256 its line gives. +Nothing else satisfies a pin: not a matching `id`, not a set that `supersedes` +names or that names this one, and not a set in another form whose unpacked +parameters compare equal. A sink MAY hold several parameter sets for one +capability; the pin resolves when one of them satisfies it. + +A pin that does not resolve is refused with one reason. The checks run in this +order, and the first that fails is the reason reported: + +1. `pin_malformed`: `sha256` is not 64 lowercase hexadecimal digits. +2. `form_not_registered`: `form` names no registered prior form. +3. `form_unreadable`: `form` is registered and the sink has no reader for it. + This is the sink's gap, and it is never reported as a mismatch. +4. `form_required`: `form` is absent and the capability is not registered with a + one-file set. +5. `prior_revoked`: reserved, for a revocation of the pinned set, keyed on the + pin's `form` and `sha256`. +6. `prior_not_held`: the sink holds no parameter set for the capability whose + digest, in the pin's form, equals `sha256`; a file the sink cannot read is not + held. A sink that holds the set a well-formed `supersedes` names — a file when + `supersedes.form` is absent, a listing when it is `wai.prior.set` — reports + `digest_superseded` instead, so that its operator can tell it holds the set + this one replaces. +7. `set_malformed`: in the form `wai.prior.set`, the listing whose SHA-256 is + `sha256` breaks a rule above. +8. `part_mismatch`: in the form `wai.prior.set`, in every copy of that listing + the sink holds, a file the listing names is missing, cannot be read, or has a + SHA-256 other than its line gives. + +These checks read the envelope, the registry, the forms the sink can read and the +bytes it holds, and nothing that depends on how the sink was built, so two +conforming sinks that read the same forms and hold the same bytes report the same +reason. Refusal reasons are snake_case; tier names and class labels are +kebab-case. + +**Decoding against a verified set.** A pinned decode MUST read its parameters +only from the set it verified. A decode that needs a file the verified set does +not contain is refused with `set_incomplete`; the sink presents no output from it, +and dispatch treats it as a pin that did not resolve. A sink SHOULD decode from +the very bytes it verified: one that reads a file again after verifying it can +decode bytes it did not verify. + +**A refused pin** makes its capability **unsupported for that envelope** at that +sink. It is not a decode error and not a reason to decode anyway, as +[capability-constraints](extensions/capability-constraints.md) §2 treats an unmet +constraint. Dispatch continues as §4 and §7.1 state. A sink MUST NOT decode a +payload through a capability whose pin it refused. + +**Tier.** `tier` is the sender's claim, from the registered vocabulary +`decode-equivalence`, `entropy-consistency` and `none`. An encoder MUST write a +registered token. A sink reads a token this revision does not register as `none`, +and a verifier reports it (`tier_unregistered`). The tier a registry class gives +is `decode-equivalence` for `NeuralIntegerExact`, `entropy-consistency` for +`NeuralEntropyExact`, and `none` for every other class. With the tiers ordered +`none` < `entropy-consistency` < `decode-equivalence`, a `tier` above the class's +does not conform (`tier_exceeds_class`): the `replicate` verdict (§7) does not +rest on it, and a verifier reports it. The decode itself is unaffected. A `tier` +at or below the class's is honoured as stated. + +**Updates.** `supersedes` is informative: it records that the set a pin names +replaces another, changes no decision, and changes only the reason check 6 +reports. A `supersedes` whose `sha256` is not 64 lowercase hexadecimal digits or +equals `sha256`, or whose `form` is not a registered prior form, is malformed: a +verifier reports it (`supersedes_malformed`), and a sink ignores the member and so +never reports `digest_superseded` for that pin. Content coded against a set pins +that set, and a successor never satisfies its predecessor's pin, so content coded +against a predecessor decodes at a sink exactly as long as that sink still holds +the predecessor. A sink can hold both without a second copy of the files they +share, by keeping the predecessor's listing beside the successor's. An update +delivered as a change to an earlier set, such as an NNC incremental update, is not +a parameter set and is never pinned; the self-contained set it yields is +([prior-carriage](extensions/prior-carriage.md) §2). + +**Where a pin sits.** In a `WAI1` manifest, `conditioning.prior` (§3). In a `WAI2` +manifest, a rendition entry's `prior`, which pins that entry's capability and is +part of the entry's identity (§7.1). A pin where no capability it would apply to +decodes against a parameter set, and every such capability is registered, means +nothing: an encoder MUST NOT write one, a verifier reports it +(`prior_not_applicable`), and a sink ignores it. + +**Sinks of earlier revisions.** A sink that implements an earlier text of this +specification may do one of two things with a pin (§8). It may reject the +envelope as carrying a member it does not understand, which fails closed. Or it +may ignore `prior`, and then the pin fails open: such a sink may decode a pinned +payload against another parameter set. The rules of this section bind sinks that +implement this revision; an encoder that must not be decoded against another set +at an earlier sink has no means to require that within `1.x` other than a +capability string earlier sinks do not advertise. + +**Envelopes these rules make non-conforming.** The rules enforce the definition of +`prior` as it was made — a lowercase hexadecimal SHA-256 of the artifact that +determines the decode's output, which a `replicate` over an integer-exact neural +capability requires — and change no payload format and no role (§8). Each of these +envelopes but the last still parses, and does not conform: + +- one whose pin's `sha256` is not 64 lowercase hexadecimal digits (the reference + implementation compared digests without regard to case or length); +- one whose pin has no `form` for a capability registered with several files: its + digest covered one file of the set or, where the registered artifact was a + directory, nothing a sink could hash; +- one that carries a pin where no capability it would apply to decodes against a + parameter set; +- one with `intent = replicate` whose decode capability is `NeuralIntegerExact` + and which pins no parameter set for it, or pins one that does not declare + `decode-equivalence` (§7); the reference implementation already refused these; +- one that carries a member this revision defines — `conditioning.prior` or a + rendition entry's `prior`, a pin member, or a rendition entry's `id`, `sha256` or + `companion_of_id` (§7.1) — with a JSON type other than the one defined. A reader + of an earlier revision may have ignored the member; a reader of this one rejects the + manifest as malformed, so this envelope no longer parses. It is the one + exception §8 allows to a new OPTIONAL member leaving envelopes parseable. + --- ## 4. Capability dispatch @@ -139,13 +338,26 @@ A conforming sink MUST implement this algorithm: 1. Read the manifest. If `wai` major version is unknown → reject. 2. Let `cap = model_requirement.capability`. -3. If `cap` is dispatchable (below) and the sink advertises `cap`, decode - `payload` via the codec registered for `cap` (§5) and return. +3. If `cap` is dispatchable (below), the sink advertises `cap`, and — when a pin applies to `cap` (§3.1) and the + manifest carries `conditioning.prior` — the pin resolves at the sink for + `cap`, decode `payload` via the codec registered for `cap` (§5), against the + verified set, and return. A pin that does not resolve makes `cap` unsupported + for this envelope: keep its reason and continue at step 4. 4. Else, let `fb = model_requirement.fallback`. -5. If `fb != null`, `fb` is dispatchable and the sink advertises `fb`, - decode via `fb`'s codec and return. -6. Else the file is **inert at this sink** — return a clear "missing - capability" error to the calling application. +5. If `fb != null`, `fb` is dispatchable and the sink advertises `fb`, and — when a pin applies to `fb` + and the manifest carries `conditioning.prior` — the pin resolves at the sink + for `fb`, decode via `fb`'s codec and return. A pin that does not resolve: keep + its reason. +6. Else the file is **inert at this sink** — return an error to the calling + application that names why: the reason kept at step 3, else the one kept at + step 5, else `missing_capability`. + +`conditioning.prior` pins the parameter set the payload was coded against, so it +applies to whichever capability decodes the payload. A pin that does not resolve +is never a reason to decode against another parameter set: a learned decode +against the wrong parameters yields plausible output, not an error. An encoder +that needs an alternative for sinks that hold another parameter set carries one +pinned rendition per set in a `WAI2` envelope (§7.1). A capability is **dispatchable** unless this revision registers it as something a sink does not decode on its own: a candidate (§5, *Candidate capabilities*), a @@ -248,6 +460,17 @@ zeroth codecs. The decode route for each is implemented in the reference sink (see the `source` column / `neural/*.rs`); the model weights are sink-supplied. +A capability that decodes against a parameter set (§3.1) is registered with that +set's shape. **One file:** `wai.neural.int_synth` (its model specification, +`int_synth.json`); `wai.neural.int_mlicpp` (its `WMB1` bundle, read from a set as +`model.wmb`); and `wai.neural.encodec32`, `wai.neural.dac`, `wai.neural.mimi`, +`wai.neural.wavtokenizer`, `wai.neural.snac` and `wai.neural.video_bmshj2018` (a +self-contained ONNX model; one that keeps tensors in external files is several +files). **Several files:** every other capability that decodes against a parameter +set. A later revision MAY register a capability of several files as one file when +it defines a one-file format that holds the whole set; that widens the +registration and makes no conforming envelope non-conforming (§8). + *Audio:* | capability | model | operating point | source | @@ -348,7 +571,8 @@ reconstruction, with the acquisition class and coverage C2PA 2.4's | capability | payload | status · extension | |---|---|---| -| `wai.prior.nnc` | a trained network's parameters as an ISO/IEC 15938-17:2024 (NNC, 2nd edition) bitstream | the registered wire form a `Prior` MAY declare (`prior.form`); its digest then covers the coded bitstream. Optional, not recommended: royalty status is not established, and cross-decoder exactness of the reconstruction is not established, so a `replicate` over a neural capability cannot rest on a prior read out of it. As a payload it takes the replicate class of a standard whose decoder conformance the registry has not checked · [prior-carriage](extensions/prior-carriage.md) | +| `wai.prior.nnc` | a trained network's parameters as an ISO/IEC 15938-17:2024 (NNC, 2nd edition) bitstream | a registered wire form a pin MAY declare (`prior.form`); its digest then covers the coded bitstream. Optional, not recommended: royalty status is not established, and cross-decoder exactness of the reconstruction is not established, so a `replicate` over a neural capability cannot rest on a prior read out of it. As a payload it takes the replicate class of a standard whose decoder conformance the registry has not checked · [prior-carriage](extensions/prior-carriage.md) | +| `wai.prior.set` | the listing of a parameter set of one or more files: each file's SHA-256 and path (§3.1) | a registered wire form a pin MAY declare (`prior.form`), and the form in which a set of several files is pinned: the pin covers the listing, and each line covers one file. Reading a set through its listing compares digests and hands the decoder each file's bytes unchanged, so the registry lists it as established exact, and a `replicate` MAY rest on a pin in it. WAI-native, royalty-free; as a payload its output is its own bytes (`StandardDefined`) · [prior-carriage](extensions/prior-carriage.md) | | `wai.tensor.nnc` | general tensorial data (split-inference feature maps, 3D/4D scene parameters) coded with NNC | **candidate**: general-tensor coding is in an unpublished amendment, so the payload format is not fixed and MUST NOT be emitted until pinned (*Candidate capabilities*, below) · [prior-carriage](extensions/prior-carriage.md) | The ten derivation operations of ISO/IEC 23001-16:2021 (derived visual tracks) @@ -599,7 +823,7 @@ The reference implementation checks these rules in `Wai::check_component_roles` (v1.0) and `WaiMulti::check_component_roles` (v1.1: the companion rules of §7.1, including both enhancement-layer rules above), dispatches `WAI1` -envelopes through `Wai::dispatch_capability`, which applies both sentences +envelopes through `Wai::dispatch_verified`, which applies both sentences above, never returns a companion-only component from `pick` / `primary`, and its `wai wrap` tool refuses to write an envelope that fails either check. Parsing stays format-agnostic: a non-conforming envelope still parses, and the check is what @@ -911,6 +1135,7 @@ cited are the ones read; the evidence, and what was not read, is in | class | capabilities | what the named standard fixes | |---|---|---| | `StandardDefined` | `wai.image.png`, `wai.audio.flac`, `wai.text.zstd`, `wai.text.xz` | the decoded output exactly: lossless formats (PNG, W3C Recommendation, Third Edition — its filters are exact integer operations; FLAC, RFC 9639) and lossless byte formats | +| | `wai.prior.set` | its bytes exactly: a listing (§3.1) is its own output | | | `wai.video.avc`, `wai.video.hevc`, `wai.video.vvc` | every sample of every cropped decoded picture a conforming decoder outputs equals the value the specified decoding process produces (ITU-T H.264 (V16, 06/2026) §C.4; H.265 (V11, 01/2026) §C.5.1; H.266 (V4, 01/2026) §C.5.1); processing an SEI message only signals, such as film grain synthesis, is outside that decoding process | | | `wai.video.vp9` | output frames "identical in all respects", in the same order, to those of the specified decoding process (VP9 Bitstream & Decoding Process Specification v0.6 §8.1) | | `StandardTolerance` | `wai.audio.opus` | output within `opus_compare`'s thresholds of the reference decoder, with the same final range-decoder state (RFC 6716 §6) | @@ -996,21 +1221,48 @@ A conforming sink that receives `intent = replicate` for a capability whose clas is `NeuralEntropyExact`, `NeuralFloat`, `GenerativeDeterministic` or `Behavioral` MUST refuse the replicate claim (decode as perceptual or reject). The capability is the one it decodes through: `model_requirement.capability`, or the -fallback when it decodes via §4 step 5. The +fallback when it decodes via §4 step 5, or, in a `WAI2` envelope, the capability +of the rendition it picks (§7.1). + +**The pin under `replicate`.** Over a `NeuralIntegerExact` capability a conforming +sink MUST also refuse the replicate claim unless a pin applies to that capability +(§3.1), declares `decode-equivalence`, and declares no form other than one the +registry lists as established exact ([prior-carriage](extensions/prior-carriage.md) +§5). A pin declares a tier by `tier`. A pin without `tier` declares +`decode-equivalence` exactly when its `determinism`, compared in ASCII lower case, +contains `int`, `fixed` or `integer` and none of `f32`, `f64` and `float`, and +declares no tier otherwise. That legacy reading exists so that a pin written +before `tier` keeps its verdict. It reads a declaration and can misread one (a +contract naming `fp16` contains no float token); the registry class, checked +first, is what guarantees the arithmetic, so a misread declaration cannot admit a +float decode. An encoder SHOULD write `tier`, and SHOULD write a `determinism` +that this reading classes the same way, so that a sink of an earlier revision +reaches the same verdict. The refusals are named `replicate_unregistered` (the +capability is not registered), `replicate_class` (its class cannot back +`replicate`), `replicate_unpinned`, `replicate_tier` (the pin does not declare +`decode-equivalence`) and `replicate_form_not_exact`. They read only the manifest +and the registry, so a sink MAY make them before it verifies the pin; refusing the +claim does not excuse the pin, which every decode of a pinned payload verifies +(§3.1). + +The reference impl checks this per-envelope in `decode_envelope_strict` (`codecs::replicate_refusal`) in two steps. First the capability's registry class: only `NeuralIntegerExact` and the three standard classes pass, whatever the envelope declares (`ReplicateClass::replicate_promise` states what each -promises). Then, for a `NeuralIntegerExact` capability, the pinned prior: it must -declare an integer-exact contract (`DeterminismClass`), and a prior that declares -the wire form of its artifact counts as integer-exact only if that form is -established to reconstruct identical tensors at every reader -([prior-carriage](extensions/prior-carriage.md) §5); no form is yet, so a -`replicate` whose prior declares one is refused. The prior check reads the +promises). Then, for a `NeuralIntegerExact` capability, the pin: it must be present and +declare `decode-equivalence` (`Prior::declared_tier`), and a pin that declares a +form passes only if the registry lists that form as established exact +([prior-carriage](extensions/prior-carriage.md) §5): `wai.prior.set` is listed, +and no coded form is. The prior check reads the sender's declaration; the class check is what ties it to the capability, so an envelope whose prior declares an integer contract over a float or entropy-only -capability is refused. `decode_envelope_strict` decodes only through -`model_requirement.capability`, so that is the one it checks. On the encoder +capability is refused. `decode_envelope_strict` applies the rule to the capability +it decodes through — `model_requirement.capability`, or the fallback when dispatch +reaches step 5 — and, when the rule refuses every capability the sink could +decode through, refuses before it verifies a pin or reads a file; +`decode_multi_strict` considers only the renditions this rule admits, and reports +each one it leaves out with that rendition's `replicate_*` reason (§7.1 step 5). On the encoder side, `wai wrap`, which pins no prior, writes `intent = replicate` only where that check accepts it unpinned — over the three standard classes — for the capability and for any fallback it declares, or, in the `WAI2` companion form (§7.1), for the @@ -1061,7 +1313,52 @@ capability promises, and the precondition for a hard the entropy decode reproduces across devices (integer entropy arithmetic or explicit parameter transmission), the tier at which JPEG AI sets its bit-exact conformance point; it does not back `replicate`. `NeuralFloat` capabilities claim -neither tier. +neither tier. A pin MAY state its decode's tier (§3.1, `tier`); a stated tier above +the class's does not conform. + +### Digests (normative) + +WAI names one digest algorithm for each purpose: + +- **Parameter-set identity** — a pin's `sha256` and `supersedes.sha256`, and each + line of a listing (§3.1): SHA-256 (FIPS 180-4), written as 64 lowercase + hexadecimal digits. +- **Component identity** — a `WAI2` entry's `sha256`, the digest of its payload + bytes (§7.1): SHA-256, written the same way. The content hash a receipt chain + takes over a whole envelope ([jwp-receipts](extensions/jwp-receipts.md) §2.1) + is a different object. +- **Decoded output in conformance material** — a digest of the decoded output of + a capability that decodes against a parameter set (§3.1), or of a standard codec + (§5), in a published conformance vector or corpus, in a golden a conformance test + compares with, or in a reconstruction or energy binding + ([reconstruction-binding](extensions/reconstruction-binding.md) §2, + [energy-binding](extensions/energy-binding.md)): SHA-256 over the capability's + canonical output buffer, written as 64 lowercase hexadecimal digits. +- **Receipt content and WAI-native state** — an envelope's content hash in a + receipt chain, and, for a WAI-native capability that decodes against no + parameter set, its state hash and the digests of its rendered output: the + algorithm the defining extension names ([jwp-receipts](extensions/jwp-receipts.md) + §2.1; the extensions §5 links). + +A conformance vector or corpus that states a digest of decoded output in the third +purpose MUST state its SHA-256 and MUST name the buffer it covers: + +- an image, or one frame of video: `RGB8` as + [reconstruction-binding](extensions/reconstruction-binding.md) §2 defines it; +- audio: the samples as signed 16-bit little-endian integers in output order, + channels interleaved; +- a sequence of frames: each frame's digest, and a clip digest, the SHA-256 of the + frames' 32-byte SHA-256 digests concatenated in output order; +- any other output (an attribute tensor, an intermediate stage): the buffer the + vector's documentation names, with its element type, byte order and layout. + +A vector MAY state other digests of the same buffer, such as a 64-bit FNV-1a or a +domain-separated BLAKE3, to locate a divergence. A verifier MUST NOT accept a +decode on such a digest alone. A hash a capability uses as a step of its own +algorithm (the prompt fold of `wai.generate.spectral`) is part of that capability, +not a digest of its output, and this rule does not reach it. A frozen digest +SHOULD be re-derived on more than one instruction-set architecture before it is +published. ### Machine-checked totality (informative) @@ -1158,7 +1455,38 @@ by which capability happens to be installed. } ``` Each entry has its own `capability` (the dispatch key) and `kind` - (the codec id matching the bytes in the corresponding payload). + (the codec id matching the bytes in the corresponding payload). An entry MAY + carry `prior`, the pin (§3.1) of the parameter set its payload was coded + against. Two entries MAY name one capability with different pins: the same + content coded against two parameter sets, such as a set and its successor: + ```json + "renditions": [ + { "capability": "wai.neural.int_mlicpp", "kind": "int_mlicpp", + "prior": { "id": "int_mlicpp/b", "sha256": "", "determinism": "int-rans/p16+int-mlicpp/i64", + "tier": "decode-equivalence", "supersedes": { "sha256": "" } } }, + { "capability": "wai.neural.int_mlicpp", "kind": "int_mlicpp", + "prior": { "id": "int_mlicpp/a", "sha256": "", "determinism": "int-rans/p16+int-mlicpp/i64", + "tier": "decode-equivalence" } }, + { "capability": "wai.image.avif", "kind": "avif" } + ] + ``` + A sink that holds `A` decodes the second entry; one that holds `B`, the first; + one that holds neither, the third. +- **Component identity.** An entry MAY carry an `id`, a string of 1 to 64 of + `a`–`z`, `0`–`9`, `_` and `-`, unique within the envelope (it contains no `.`, + so it never equals a capability string), and a `sha256`, the SHA-256 of the + entry's payload bytes as 64 lowercase hexadecimal digits. A component's identity + is its `id`, its payload digest `sha256` and its parameter-set pin `prior` + (§3.1). These are one construct: no extension defines another per-component + digest, and staged layers, byte-range and referenced renditions, and + per-rendition tracks address a component by `id` and verify it by `sha256` and + `prior`. A sink MUST check an entry's `sha256` against the entry's payload bytes + before it decodes or applies the entry: a selectable entry that does not match is + skipped (`component_digest_mismatch`), as an entry whose pin is refused is, and a + companion that does not match is not applied. An entry whose `id`, `sha256` or + `companion_of_id` is not a string (`null` is the same as absent) makes the + manifest malformed, as a mistyped pin member does (§3.1), and the reader rejects + the envelope. - Rendition table: `u16 n_renditions | n × (u32 offset, u32 length)`, offsets relative to the start of the payloads block. The table's i-th entry corresponds to `renditions[i]` in the manifest. @@ -1169,17 +1497,38 @@ by which capability happens to be installed. A v1.1-aware sink MUST: -1. Read the manifest, get the ordered list of `model_requirement` - entries. -2. Walk the list in order; pick the first entry whose capability the - sink either advertises OR whose declared backend the sink prefers - under its deployer policy. +1. Read the manifest and take its ordered `renditions` entries. +2. Walk the entries in order, and pick the first that is not a companion (below), + whose capability is not companion-only (§5), that the sink either advertises + OR whose declared backend the sink prefers under its deployer policy, whose + `sha256`, when present, matches its payload bytes, and whose `prior`, when the + entry carries one and a pin applies to its capability (§3.1), resolves at the + sink. An entry whose payload digest does not match, or whose pin is refused, is + skipped as if the sink did not advertise its capability, and the sink keeps the + reason. A deployer policy MAY reorder or restrict the entries the sink + considers; it MUST NOT pick an entry skipped for its digest or its pin. 3. Read the corresponding rendition's bytes from the payload block. -4. Dispatch as in v1.0 §4. +4. Dispatch as in v1.0 §4, with the entry's `capability` and `prior` in place of + `model_requirement.capability` and `conditioning.prior`, and no fallback. A + decode refused with `set_incomplete` (§3.1) is handled as a skipped entry: + continue at step 2 with the next entry. +5. If no entry is picked, the envelope is inert at this sink: return an error + that names, for each entry, its index and why it was skipped: + `not_selectable` (a companion, or a companion-only capability, which step 2 + never picks), `missing_capability`, `component_digest_mismatch`, a pin reason + (§3.1), or, where a deployer policy restricts step 2 to the entries §7's + `replicate` rule admits, the `replicate_*` reason the rule gives for an entry + it leaves out. -A v1.0 sink that encounters a `WAI2` envelope MUST refuse it cleanly -(unknown major). No silent fallback to the first payload — that would -hide a major-version mismatch. +A sink MUST NOT decode one entry's payload against the parameter set another +entry pins. The envelope's `intent` applies to the rendition the sink picks, and +§7's `replicate` rule reads that entry's capability and pin; a deployer policy +that serves only sound `replicate` claims MAY restrict step 2 to the entries that +rule admits. + +A v1.0 sink that encounters a `WAI2` envelope MUST refuse it cleanly (unknown +magic, §2). No silent fallback to the first payload — that would hide the +version mismatch. ### Companion components (descriptors bound to a payload) @@ -1198,8 +1547,10 @@ revision registers no capability for immersive-audio mix metadata; and travels as a rendition. Selection: a sink picks a rendition among the -`role != "companion"` entries, then applies every companion whose -`companion_of` names the chosen component (e.g. the HDR10+ block for the AV1 +`role != "companion"` entries by the algorithm above, then applies every companion +bound to the chosen component — by `companion_of_id` when the companion carries +one, otherwise every companion whose `companion_of` names the chosen component's +capability (e.g. the HDR10+ block for the AV1 video the sink picked). `pick`/`primary` never return a companion, nor a component whose capability is not dispatchable (§4): a companion-only capability, a candidate, a record or a derivation. A component whose capability @@ -1233,6 +1584,14 @@ A component with `role = "companion"` conforms only if: layer (§5). Several descriptor companions MAY bind to one base, and a descriptor MAY bind to several renditions of one capability. +A companion MAY carry `companion_of_id`, the `id` of the one entry it binds to. +That entry MUST exist, MUST have the capability the companion's `companion_of` +names, and MUST be a selectable rendition; the companion then binds to exactly that +entry, and rule 3's "at most one enhancement layer per base" is checked by entry. +`companion_of` keeps its meaning, so a sink that does not read `companion_of_id` +binds by capability: for an enhancement layer over two renditions of one base +capability it finds the base not unique, and does not apply the layer. + An envelope with a companion that breaks any of these does not conform (the reference `WaiMulti::check_component_roles` reports the first break). A sink MUST NOT apply a companion that breaks rule 1 or 2, nor one whose capability it @@ -1303,8 +1662,26 @@ envelope that uses any string a later revision registers. - The `wai` manifest field is `MAJOR.MINOR`. A reader MUST reject an unknown MAJOR. A reader MUST accept an unknown MINOR by ignoring fields it does not understand, provided MAJOR matches. -- The container format (§2) and the manifest schema (§3) are frozen - for all of `1.x`. +- A reader MUST ignore a manifest member it does not understand, whatever the + MINOR. Earlier text required this only for an unknown MINOR, so a reader of an + earlier revision may instead reject an envelope that carries such a member, + which fails closed. Or it ignores the member, and then a pin fails open (§3.1). +- The container format (§2) is frozen for all of `1.x`. The manifest schema (§3, + and a §7.1 rendition entry) is frozen for all of `1.x` in this sense: no member + is removed or renamed, none changes type or meaning, and none becomes REQUIRED. + A MINOR revision MAY define an OPTIONAL member, at the top level or inside an + existing object, whose absence leaves every envelope's bytes and meaning as they + were, and the revision states what it means for a reader of an earlier + revision (above). This revision defines `conditioning.prior`, a rendition + entry's `prior`, `id`, `sha256` and `companion_of_id`, and the pin's members + `form`, `tier` and `supersedes` this way (§3.1, §7.1); envelopes that use them + keep `wai` `"1.0"` or `"1.1"`. A defined member carried with another JSON type + makes the manifest malformed rather than being ignored, because a reader that + ignored a mistyped pin member could decode against a set the pin excludes; an + envelope that carried such a value before the member was defined, and parsed + then, does not parse now, and §3.1 lists it. The reference implementation wrote `prior`, with + `id`, `sha256`, `determinism` and `form`, before this revision defined it; an + envelope written that way keeps its bytes and its meaning, except as §3.1 lists. - New capabilities (§5) MAY be added in any MINOR revision. Existing capabilities MUST NOT be redefined — once a capability string is registered for a codec, its payload format is fixed forever. @@ -1330,6 +1707,9 @@ envelope that uses any string a later revision registers. envelope stops conforming, and a reader of an earlier revision that reports the new binding as not conforming does not apply that companion and can still present its base. +- A registration of a capability that decodes against a parameter set also fixes + that set's shape (§5, §3.1). Changing one file to several narrows it and + requires a new MAJOR; several to one widens it, which a MINOR revision MAY do. - Removing or repurposing an existing capability — changing its payload format or its role — requires a new MAJOR. - A MINOR revision MAY add a conformance rule that enforces a registration, @@ -1339,7 +1719,8 @@ envelope that uses any string a later revision registers. and the revision lists every kind of envelope it makes non-conforming; they still parse. The companion rules of §5 and §7.1 are such rules, and §7.1 lists the envelopes they affect (*Envelopes the companion rules make - non-conforming*). + non-conforming*). §3.1's pin rules are such rules, and §3.1 lists the + envelopes they affect. - Registering a capability gives a string a payload format and a role that no earlier revision gave it, and the rules of §5 and §7.1 then apply to it. An envelope that used the string before then used a capability no revision diff --git a/wai/byte-exact-conformance/src/int_video_golden.rs b/wai/byte-exact-conformance/src/int_video_golden.rs index ec5272f67..777ebead5 100644 --- a/wai/byte-exact-conformance/src/int_video_golden.rs +++ b/wai/byte-exact-conformance/src/int_video_golden.rs @@ -21,8 +21,34 @@ use std::path::PathBuf; use serde_json::Value; +use sha2::{Digest, Sha256}; + use crate::int_video::{decode, fnv1a}; +/// The clips' SHA-256 conformance digests (SPEC §7 *Digests*): the SHA-256 over the +/// frames' 32-byte SHA-256 digests in output order, frames as RGB8. The FNV-1a +/// values beside them locate a divergence; these are what a conforming decode +/// matches. +const SYNTHETIC_CLIP_SHA256: &str = "5afd6477d09525b7db4485027588ba3bc43de3d29c996eb57f2d3274bf8309f9"; +const REAL_CLIP_SHA256: &str = "e6724ca214694e012c3e2b49df4e116a0a7e150435a12d7b144560ce2d5c7bdf"; + +/// Every frame's SHA-256 against the sidecar entry `key`, and the clip digest. +fn assert_sha256(frames: &[Vec], key: &str) -> String { + let side = jload("sha256.json"); + let want = side[key]["frames_sha256"].as_array().expect("frames_sha256"); + assert_eq!(want.len(), frames.len(), "{key}: frame count in sha256.json"); + let mut all = Vec::with_capacity(frames.len() * 32); + for (i, f) in frames.iter().enumerate() { + let d = Sha256::digest(f); + let hex: String = d.iter().map(|b| format!("{b:02x}")).collect(); + assert_eq!(hex, want[i].as_str().expect("frame digest"), "{key}: frame {i} SHA-256"); + all.extend_from_slice(&d); + } + let clip: String = Sha256::digest(&all).iter().map(|b| format!("{b:02x}")).collect(); + assert_eq!(clip, side[key]["clip_sha256"].as_str().expect("clip_sha256"), "{key}: clip SHA-256"); + clip +} + fn vectors() -> PathBuf { PathBuf::from(env!("CARGO_MANIFEST_DIR")).join("../wai-rs/tests/vectors/int_video") } @@ -45,7 +71,7 @@ fn hex16(s: &str) -> [u8; 8] { /// Decode a committed clip and assert every frame hash and the clip hash. /// Returns the clip hash so callers can pin it verbatim. -fn assert_clip(bitstream: &str, conf: &str) -> String { +fn assert_clip(bitstream: &str, conf: &str, key: &str) -> (String, String) { let bs = jload(bitstream); let conf = jload(conf); let clip = decode(&bs).expect("decode must succeed on every architecture"); @@ -71,14 +97,15 @@ fn assert_clip(bitstream: &str, conf: &str) -> String { } let clip_hash = format!("{:016x}", fnv1a(&all)); assert_eq!(clip_hash, conf["clip_hash"].as_str().expect("clip_hash"), "clip hash"); - clip_hash + (clip_hash, assert_sha256(&clip.frames, key)) } /// The synthetic clip: 5 frames of 48×48 RGB, whole-pel motion. #[test] fn int_video_synthetic_clip_is_byte_identical() { - let clip = assert_clip("int_video_bitstream.json", "int_video.json"); + let (clip, sha) = assert_clip("int_video_bitstream.json", "int_video.json", "int_video"); assert_eq!(clip, "27573cdba1ad6628"); + assert_eq!(sha, SYNTHETIC_CLIP_SHA256); println!("int_video (synthetic, 5x48x48): clip {clip}"); } @@ -97,7 +124,8 @@ fn int_video_real_footage_exercises_the_half_pel_predictor() { still pass while silently covering only the whole-pel path" ); - let clip = assert_clip("video_real_bitstream.json", "video_real.json"); + let (clip, sha) = assert_clip("video_real_bitstream.json", "video_real.json", "video_real"); + assert_eq!(sha, REAL_CLIP_SHA256); assert_eq!(clip, "1ec1f0fa7da0d70f"); println!("int_video (real footage, 8x96x96, half-pel): clip {clip}"); } diff --git a/wai/byte-exact-conformance/src/neural_keyframe_golden.rs b/wai/byte-exact-conformance/src/neural_keyframe_golden.rs index 1f4d6934b..762c3def2 100644 --- a/wai/byte-exact-conformance/src/neural_keyframe_golden.rs +++ b/wai/byte-exact-conformance/src/neural_keyframe_golden.rs @@ -22,6 +22,8 @@ use std::path::PathBuf; use serde_json::Value; use crate::int_hyper_synth::{decode_conditional, CondBitstream, CondEntropy, IntModel}; +use sha2::Digest; + use crate::int_video::{decode_pframes, fnv1a}; fn vectors() -> PathBuf { @@ -81,6 +83,20 @@ fn the_neural_keyframe_clip_is_byte_identical() { let clip_hash = format!("{:016x}", fnv1a(&all)); assert_eq!(clip_hash, conf["clip_hash"].as_str().expect("clip_hash")); assert_eq!(clip_hash, "a5f53d477999d159"); + // The SHA-256 conformance digests (SPEC §7 *Digests*), from the vector's sidecar. + let side = jload("int_video/sha256.json"); + let want = side["video_nk"]["frames_sha256"].as_array().expect("frames_sha256"); + assert_eq!(want.len(), clip.frames.len()); + let mut digests = Vec::with_capacity(clip.frames.len() * 32); + for (i, f) in clip.frames.iter().enumerate() { + let d = sha2::Sha256::digest(f); + let hex: String = d.iter().map(|b| format!("{b:02x}")).collect(); + assert_eq!(hex, want[i].as_str().expect("frame digest"), "frame {i} SHA-256"); + digests.extend_from_slice(&d); + } + let clip_sha: String = sha2::Sha256::digest(&digests).iter().map(|b| format!("{b:02x}")).collect(); + assert_eq!(clip_sha, side["video_nk"]["clip_sha256"].as_str().expect("clip_sha256")); + assert_eq!(clip_sha, "4a81f0c94ccd938a7ca80df203303d5735d9dfcdfa150da155d9f2e4ca283e8f"); println!("int_hyper keyframe clip ({} frames {h}x{w}): {clip_hash}", clip.frames.len()); } diff --git a/wai/extensions/determinism-tiers.md b/wai/extensions/determinism-tiers.md index d57d65b97..cf66a3e83 100644 --- a/wai/extensions/determinism-tiers.md +++ b/wai/extensions/determinism-tiers.md @@ -8,7 +8,9 @@ > neural capability promises, what a receipt can hash under each, which WAI > capabilities are registered at each and at what cost, and what the repository > demonstrates today. -> No new wire format; a vocabulary + conformance framing over §7. +> One optional manifest member carries a tier name: a pin's `tier` +> ([SPEC.md §3.1](../SPEC.md)). Otherwise a vocabulary and conformance framing +> over §7. > Keywords **MUST**, **MUST NOT**, **SHOULD**, **MAY** are RFC 2119/8174. ## 1. Why tiers @@ -259,12 +261,17 @@ thing. `NeuralEntropyExact` capabilities at `entropy-consistency`, and `NeuralFloat` capabilities claim neither. §5 lists WAI's capabilities at each tier. A capability MAY declare `entropy-consistency` when it only guarantees the entropy - stage. + stage. A pin MAY also state a tier in the manifest, as `tier` (SPEC §3.1), from + the closed vocabulary `decode-equivalence`, `entropy-consistency`, or `none` for + neither; an encoder writes only those tokens, and a sink reads any other token + as `none`. A stated tier above the capability's registered tier does not conform + (`tier_exceeds_class`): the `replicate` verdict does not rest on it. - **Over a neural capability, `replicate` promises `decode-equivalence` (SPEC §7).** An envelope with `intent = replicate` over a neural capability MUST be over a capability registered at `decode-equivalence`, which SPEC §7 admits only by the integer construction: an integer / fixed-point decode path with no float - operation. + operation. Such an envelope MUST pin the parameter set of that capability with a + pin that declares `decode-equivalence` (SPEC §3.1, §7). `entropy-consistency` MUST NOT back `replicate`. A neural decode that derives its entropy parameters in float meets neither tier; one that reads them from the codestream, or derives them in integer arithmetic, and runs its later stages in @@ -299,10 +306,10 @@ thing. classes, which are not neural, are not refused by this rule). The class does not depend on anything the sender declares, so a `replicate` envelope over `wai.image.jpegai` or a float codec is refused even when its prior declares an integer contract. Then, for an - integer-exact capability, the pinned prior: `Prior::determinism_class` must - classify its sender-declared contract string as integer-exact. That - classification is a substring match (a contract containing `int`, `fixed` or - `integer` and none of `f32`, `f64`, `float`), so on its own it verifies the + integer-exact capability, the pin: it must declare `decode-equivalence` — by + `tier`, or, for a pin without `tier`, by the reading of its `determinism` string + that SPEC §7 fixes, a substring match (a contract containing `int`, `fixed` or + `integer` and none of `f32`, `f64`, `float`). Either way it verifies a declaration, not the arithmetic. Neither step inspects the decode path a sink actually runs; for `wai.neural.int_synth` and `wai.neural.int_mlicpp`, the `NeuralIntegerExact` capabilities the reference neural dispatch decodes, there is diff --git a/wai/extensions/prior-carriage.md b/wai/extensions/prior-carriage.md index a2a27f8b7..100f5c0d7 100644 --- a/wai/extensions/prior-carriage.md +++ b/wai/extensions/prior-carriage.md @@ -1,22 +1,25 @@ # WAI Extension: Prior Carriage -> Status: Draft. Lets a `Prior` declare the **wire form** of the artifact its -> digest covers, registers ISO/IEC 15938-17 (neural-network coding, "NNC") as -> that form, registers a **candidate** general-tensor payload, and fixes how an -> in-band signature inside such an artifact relates to WAI's digest. Reference -> impl: `wai-rs` — `container::Prior::{new, with_form, form, check_form, -> form_preserves_exactness}`, `codecs::prior_form`, `codecs::{CAP_PRIOR_NNC, -> CAP_TENSOR_NNC}`. +> Status: Draft. Registers the wire **forms** a pin's digest may cover — +> `wai.prior.set`, the listing of a parameter set ([SPEC.md §3.1](../SPEC.md)), +> and ISO/IEC 15938-17 (neural-network coding, "NNC") — registers a **candidate** +> general-tensor payload, and fixes how an in-band signature inside such an +> artifact relates to WAI's digest. The pin itself, its verification and its +> refusal reasons are SPEC.md §3.1. Reference impl: `wai-rs` — +> `container::{Prior, PinRefusal}`, `pin::{verify_bytes, verify, parse_listing, +> write_listing, listing_of_dir}`, `codecs::prior_form`, +> `codecs::{CAP_PRIOR_SET, CAP_PRIOR_NNC, CAP_TENSOR_NNC}`. > Keywords **MUST**, **MUST NOT**, **SHOULD**, **MAY** are RFC 2119/8174. ## 1. A prior has a digest, and had no form -A neural capability decodes against a model the envelope does not carry -([SPEC.md §1](../SPEC.md)). WAI pins that model's identity: a `Prior` in the -conditioning names an `id`, a `sha256` over the artifact, and a `determinism` -contract. The reference sink recomputes the digest over the model it registered -and refuses to decode on a mismatch (`neural::ModelRegistry::resolve_checked`), -and `replicate` soundness is gated on the determinism contract (SPEC §7). +A neural capability decodes against a parameter set the envelope does not carry +([SPEC.md §1](../SPEC.md)). WAI pins that set's identity +([SPEC.md §3.1](../SPEC.md)): a pin names an `id`, a `sha256` over the whole +parameter set and a `determinism` description, and MAY state a `tier` and the set +it `supersedes`. A sink verifies the pin before it decodes, refuses a pin its sets +do not satisfy with a named reason, and `replicate` soundness is gated on the +declared tier (SPEC §7). The digest fixes **which bytes**. Until now nothing fixed **what those bytes are**: the sink hashes whatever single file it registered — a serialized graph, @@ -30,25 +33,33 @@ operand needs a form both ends can name. ```json "prior": { "id": "int_hyper/q4", - "sha256": "", + "sha256": "<64 lowercase hexadecimal digits: the SHA-256 of the set, in its declared form>", "determinism": "int-rans/p16+int-synth", - "form": "wai.prior.nnc" + "form": "wai.prior.set", + "tier": "decode-equivalence" } ``` - `form` is OPTIONAL. Absent, the form is undeclared and the digest covers the - artifact as the sink registered it — the existing behaviour, unchanged, and a - manifest without `form` round-trips byte for byte. + one file that holds the parameter set; SPEC §3.1 admits that only for a + capability registered with a one-file set, and a sink refuses such a pin over + any other capability (`form_required`). A manifest without `form` round-trips + byte for byte. - Present, `form` MUST be a registered prior form (§3), and `sha256` MUST be - computed over the artifact serialized in exactly that form: the coded - bitstream, not an unpacking, a transcoding, or a directory of its parts. -- A sink that cannot read the declared form MUST refuse the prior, as it refuses - a digest mismatch. It MUST NOT resolve the prior against an artifact in another - form, even one whose unpacked parameters compare equal: equality after - unpacking is a different fact from the digest the manifest pins. -- A declared form names a **self-contained** artifact. An NNC incremental-update - bitstream decodes only against its base, so it MUST NOT be pinned as a prior on - its own. No form is registered for a base-plus-updates chain. + computed over the parameter set serialized in exactly that form: for a coded + form, the coded bitstream, not an unpacking or a transcoding of it; for + `wai.prior.set`, the listing SPEC §3.1 defines, never the files' bytes + concatenated or an archive of them. +- A sink that does not know the declared form as a prior form refuses the pin + (`form_not_registered`), and one that cannot read it refuses it + (`form_unreadable`, SPEC §3.1). It MUST NOT resolve the pin against an artifact + in another form, even one whose unpacked parameters compare equal: equality + after unpacking is a different fact from the digest the manifest pins. +- A declared form names a **self-contained** parameter set. An NNC + incremental-update bitstream decodes only against its base, so it MUST NOT be + pinned on its own, and no form is registered for a base-plus-updates chain. A + pin names the self-contained set an update yields, and MAY name the set it + replaced in `supersedes` (SPEC §3.1). - A form is a capability string. When a prior artifact travels in a WAI envelope it does so under its form's capability, so the envelope's per-object content hash and the `Prior` digest cover the same bytes. @@ -57,10 +68,11 @@ operand needs a form both ends can name. | capability | payload | status | |---|---|---| -| `wai.prior.nnc` | the parameters of a trained network as a bitstream conforming to **ISO/IEC 15938-17:2024** (NNC, 2nd edition: parameters and their incremental updates, DeepCABAC entropy coding) | registered; the only prior form | +| `wai.prior.set` | the listing of a parameter set of one or more files (SPEC §3.1) | registered; WAI-native; established exact (§5) | +| `wai.prior.nnc` | the parameters of a trained network as a bitstream conforming to **ISO/IEC 15938-17:2024** (NNC, 2nd edition: parameters and their incremental updates, DeepCABAC entropy coding) | registered; a coded form | | `wai.tensor.nnc` | general tensorial data — split-inference feature maps, 3D/4D scene parameters — coded with NNC | **candidate**: payload format not fixed | -Both are sink-supplied: `wai-rs` contains no NNC reader or writer. The +The NNC entries are sink-supplied: `wai-rs` contains no NNC reader or writer. The standard's reference software is itself standardized, in **ISO/IEC 15938-18**, *Conformance and reference software for compression of neural networks* (1st edition 2023, 2nd edition 2025). That part also supplies the conformance @@ -71,6 +83,12 @@ an implementation to use it. Registry classification (`codecs::capability_*`): +- **`wai.prior.set`**: fidelity tier lossless and replicate class + `StandardDefined` — a listing is its own output, and SPEC §3.1 fixes its bytes + exactly; licence royalty-free (WAI-native, Apache-2.0 reference). + +For the NNC entries: + - **Fidelity tier: lossy, by primary role.** The registry's tier names a capability's primary role, as it does for AVIF, which also has a lossless mode; it is not a statement that every NNC artifact approximates its source. NNC's @@ -185,8 +203,10 @@ reasons: published editions and for any later one. - Therefore an envelope with `intent = replicate` MUST NOT rest on a prior whose declared form is not established exact. The registry lists the forms that are - established (`codecs::prior_form::form_reconstruction_is_exact`), and **none is - listed today**. In the reference implementation the rule is part of + established (`codecs::prior_form::form_reconstruction_is_exact`). It lists one, + `wai.prior.set`: reading a set through its listing compares digests and hands + the decoder each file's bytes unchanged, so every sink that holds the set reads + the same parameters. No coded form is listed. In the reference implementation the rule is part of `Prior::is_cross_hardware_exact`, which requires `Prior::form_preserves_exactness` as well as an integer-exact contract. Every `replicate` check reads that predicate, so `container::replicate_is_sound` and `Wai::replicate_is_sound` @@ -228,39 +248,71 @@ WAI's binding sits beside the standard's, never in place of it. ## 7. Reference implementation -- `container::Prior::form` (field, serde-skipped when absent) and - `Prior::form()`; `Prior::new(id, sha256, determinism)`, which leaves the form - absent, and `Prior::with_form(form)`. The new public field breaks Rust code - outside the crate that builds a `Prior` with a struct literal; the CHANGELOG - (v0.4.0) states the break, the fix, and why `Prior` is not - `#[non_exhaustive]`; `Prior::check_form(readable) -> Result<(), PriorFormError>` - (`NotAPriorForm` / `Unreadable`); `Prior::form_preserves_exactness()`, which - `Prior::is_cross_hardware_exact()` requires alongside an integer-exact contract. - `container::replicate_prior_refusal(intent, prior) -> Option` - gives the reason: `PriorRefusal::FormNotExact(form)` when an integer-exact - contract declares a form not listed as exact, with a message that says whether - the form is registered at all. `container::replicate_is_sound` and - `Wai::replicate_is_sound` hold exactly when it returns `None`, and a unit test - checks that the two agree. -- `codecs::prior_form::{prior_forms, is_prior_form, - form_reconstruction_is_exact}`; `codecs::{CAP_PRIOR_NNC, CAP_TENSOR_NNC}` with - their tier and licence entries. They have no replicate-class entry of their own: - a unit test holds their class equal to MPEG-I Haptics' (§3). -- `neural::ModelRegistry::resolve_checked` runs `check_form` against - `neural::READABLE_PRIOR_FORMS`, which is empty: the reference sink refuses any - prior that declares a form. `neural::decode_envelope_strict` reports the - `PriorRefusal` as the reason for `UnsoundReplicate`. -- CI: the library unit tests (`wai-conformance`) cover `check_form`, - `form_preserves_exactness`, every `PriorRefusal`, and the form rule in each - `replicate` check (`Prior::is_cross_hardware_exact`, both `replicate_is_sound` - functions and `replicate_prior_refusal`). The two model-free tests of the - neural-sink wiring (`resolve_checked_verifies_prior_hash`, - `strict_decode_gates_unsound_replicate`) are configured to run in - `wai-neural-gates`, in an Ubuntu 24.04 container image. That job has not yet - run on a runner. +- `container::Prior` has the members of SPEC §3.1; `form`, `tier` and + `supersedes` are serde-skipped when absent, so a pin written without them + round-trips byte for byte, and a member this revision does not define is kept + (`extra`). `Prior` is `#[non_exhaustive]`: it is built with `Prior::new` and + the `with_form`, `with_tier` (a typed `codecs::DeclaredTier`, so the crate + writes only a registered token) and `with_supersedes` builders, so a member a + later revision adds does not break code that builds one. `Prior::check_syntax` + applies SPEC §3.1 check 1, `Prior::supersedes_finding` the `supersedes` rule, + and `Prior::declared_tier` reads `tier` or, without it, the reading SPEC §7 + fixes (`Prior::determinism_class`). `container::PinRefusal::code()` is the + reason SPEC §3.1 names. `Wai::check_pin` and `WaiMulti::check_pins` report what + a verifier can decide without holding a parameter set. +- `pin::verify_bytes` runs SPEC §3.1's checks in order over parameter sets held in + memory (a browser sink calls it); `pin::verify` reads held files and listings + from disk, keeps the bytes it hashed, and calls it. `pin::parse_listing`, + `pin::write_listing` and `pin::listing_of_dir` read and write a listing. + `pin::READABLE_FORMS` holds `wai.prior.set`: the reference sink reads no coded + form. +- `neural::ModelRegistry` holds, for each capability, its registered path and any + further paths given to `hold`. For a form-less pin it hashes each held file — + for `wai.neural.int_synth`, the `int_synth.json` beside it. For a + `wai.prior.set` pin it takes every `*.sha256` file directly inside each held + path's directory (the path itself if it is a directory) as a listing rooted + there. +- `container::Wai::dispatch_verified` applies SPEC §4 steps 3–6 and + `container::WaiMulti::pick` SPEC §7.1 steps 2 and 5, each with a pin verifier + the caller supplies. `neural::decode_envelope` and `neural::decode_multi` + decode through them; their strict variants apply SPEC §7 to the capability they + decode through. +- The decoders read parameters only through `neural::ParamSource`, which hands a + pinned decode the verified bytes; ONNX sessions are built from those bytes, and + a file the verified set lacks is refused with `set_incomplete`. + `wai.neural.bmshj2018` caches its CDF tables by content digest. +- The C ABI (`ffi`) returns each manifest as its original bytes, carries a `WAI2` + rendition's identity and pin in the rendition table, and reports a refused pin + as `WAI_ERR_PRIOR` with its reason first in `wai_last_error`; + `wai_neural_decode_ex` decodes a `WAI1` or `WAI2` envelope against a registry of + held sets. The browser crate (`wai-web`) wraps the same types and exposes `pick`, + `companionsOf` and `verifyPin` to JS. +- CI: the library unit tests cover `container`, `codecs` and `pin`; + `wai_pin_vectors verify ../pin-conformance` checks the corpus, and + `pin-conformance/pin_verify.py`, which shares no code with `wai-rs`, checks it + independently; the model-free neural-sink and neural C ABI tests, and the + browser crate's tests (natively and under wasm32), are configured to run in CI. - Not implemented: an NNC reader or writer, and any cross-decoder measurement of NNC reconstruction. +The files the reference decoders read from a set (informative). A set built for +another decoder of the capability may name other files; a decode that needs a +file the set lacks is refused with `set_incomplete`. + +| capability | shape | files the reference decoder reads from a set | +|---|---|---| +| `wai.neural.int_mlicpp` | one file | `model.wmb` | +| `wai.neural.int_synth` | one file | `int_synth.json` | +| `wai.neural.encodec32`, `.dac`, `.mimi`, `.wavtokenizer`, `.snac`, `.video_bmshj2018` | one file | `decoder.onnx` | +| `wai.neural.bmshj2018` | several | `decoder.onnx`, `cdfs_q.json` | +| `wai.neural.mbt2018_mean` | several | `hyper_cdfs_q.json`, `hyper_synthesis.onnx`, `gauss_cdfs_q.json`, `synthesis.onnx` | +| `wai.neural.elic` | several | the `mbt2018_mean` files, and `context.onnx` | +| `wai.neural.mlicpp`, `.mlicv2`, `.cmic` | several | the `mbt2018_mean` files, and `entropy.onnx` | +| `wai.neural.glc` | several | `hyper_cdfs_q.json`, `hyper_entropy.onnx`, `context_entropy.onnx`, `synthesis.onnx` | +| `wai.neural.dcvc_rt`, `.dcvc_fm` | several | `hyper_cdfs_q.json`, `hyper_synthesis.onnx`, `gauss_cdfs_q.json`, `temporal_decoder.onnx` | +| `wai.image.jpegai` | several | exact path: `norm.json`, `weights/{hs,hyp,synth}_{y,uv}.json`, `weights/mcm{0..3}_y.json`, `weights/icci.json`. Float path: `hyper_scale_decoder_int_weights.json`, `model_constants.json`, `jpegai_normative_tables.json`, `me-tANS/bounds.csv`, `jpegai_factorized_cdfs.json`, `jpegai_scaler_vec.json`, `jpegai_dequant_consts.json`, `model/CCS_SGMM/tools_0/model_{y,uv}/common_modules/hyper_decoder.onnx`, `model/CCS_SGMM/tools_0/model_y/common_modules/MCM/stage{0..3}.onnx`, `model16/CCS_SGMM/tools_0/model_{y,uv}/synthesis16.onnx`, `e2e/pfchain/icci_{y_m6,uv_m3}.onnx`. A set holding only the exact path's files decodes through that path; where that path refuses (a non-default floating-point environment), the float path needs files the set lacks, and the decode is refused with `set_incomplete` instead of running on unverified files. | +| `wai.neural.int_hyper`, `wai.video.int_hyper`, `wai.audio.int_codec`, `wai.splat.int_codec`, `wai.generate.spectral` | several, until an envelope decode path registers its files | — (no envelope decode path in the reference sink) | + ## 8. Sources - ISO/IEC 15938-17:2022 and ISO/IEC 15938-17:2024, *Compression of neural diff --git a/wai/neural-conformance/README.md b/wai/neural-conformance/README.md index c3d2e9811..a25bd85eb 100644 --- a/wai/neural-conformance/README.md +++ b/wai/neural-conformance/README.md @@ -26,9 +26,15 @@ macOS arm64) on every push that touches it. weights/bias as `i32`, stride, requantize shift, integer ReLU flag). - `vectors//latent.bin` — the wire payload (12-byte header + channel-major `int8` latents). +- `sha256.json` — per vector, the SHA-256 of its decoded output as `RGB8` + (SPEC §7, *Digests*): the conformance digest. - `expected_hashes.json` — per vector: `s_log2`, `in_channels`, `layers`, - `width`, `height`, and `rgb_hash` (domain-prefixed BLAKE3 of the decoded - RGB — the conformance quantity). + `width`, `height`, and `rgb_hash`, a domain-prefixed BLAKE3 of the decoded + RGB, kept unchanged as a divergence diagnostic. + +The vectors keep their unpinned `/model` convention. A sink that pins an +`int_synth` parameter set pins `int_synth.json`, the one file that holds it +(SPEC §5). Synthetic vectors (realistic multi-layer decoders, distinct shapes) — model weights/latents authored from a fixed integer SplitMix64 sequence, so `gen` diff --git a/wai/neural-conformance/sha256.json b/wai/neural-conformance/sha256.json new file mode 100644 index 000000000..a5e433dcd --- /dev/null +++ b/wai/neural-conformance/sha256.json @@ -0,0 +1,18 @@ +{ + "gradient": { + "buffer": "RGB8: rows top to bottom, pixels left to right, R, G, B bytes", + "sha256": "ce0d4833e42d670daf90aab70f0ea255d1a0610d3984845ea61dfe2b43db7cd4" + }, + "bloom": { + "buffer": "RGB8: rows top to bottom, pixels left to right, R, G, B bytes", + "sha256": "14d16a169b0fed52b52f0f38f30af1751e22072a87852a654838f48f299d49bd" + }, + "facade": { + "buffer": "RGB8: rows top to bottom, pixels left to right, R, G, B bytes", + "sha256": "fe3578908ce45680c053fdd589c033e50ae0045eaa24330356603fa325becdc0" + }, + "photo": { + "buffer": "RGB8: rows top to bottom, pixels left to right, R, G, B bytes", + "sha256": "2a3db89614cd632829d873cf59d6300b0e49c5647a9ede1115719a6604d42115" + } +} diff --git a/wai/pin-conformance/.gitattributes b/wai/pin-conformance/.gitattributes new file mode 100644 index 000000000..eff1df73a --- /dev/null +++ b/wai/pin-conformance/.gitattributes @@ -0,0 +1,2 @@ +# Hashed files: a line-ending conversion on checkout would change every digest. +* -text diff --git a/wai/pin-conformance/README.md b/wai/pin-conformance/README.md new file mode 100644 index 000000000..068bac99d --- /dev/null +++ b/wai/pin-conformance/README.md @@ -0,0 +1,43 @@ +# pin-conformance — the parameter-set pin + +Vectors for the parameter-set pin (SPEC.md §3.1, §4, §7, §7.1). + +**Reason-equivalence.** A conforming verifier reports exactly the declared outcome +for every case: + +- `cases.json` — given a pin, a capability, the forms the sink reads (default + `wai.prior.set`) and the artifacts it holds, the outcome `ok` or the one reason + SPEC §3.1 names: `pin_malformed`, `form_not_registered`, `form_unreadable`, + `form_required`, `prior_not_held`, `digest_superseded`, `set_malformed`, + `part_mismatch`. A held artifact is a `file`, or a `listing` together with the + `root` its paths are read below. Paths are relative to this directory. +- `selection.json` — given a `WAI2` manifest, its payloads, the capabilities the + sink advertises and the one-file parameter sets it holds, the rendition SPEC §7.1 + picks and the reason for each entry it skips (`not_selectable`, + `missing_capability`, `component_digest_mismatch`, or a pin reason); when nothing + is picked, the reason for every entry. Where stated, also the companions applied + with the pick, and the first component-role finding. +- `static.json` — the first finding a verifier makes about a manifest's pins + without holding a parameter set (`prior_not_applicable`, `pin_malformed`, + `tier_unregistered`, `supersedes_malformed`, `form_not_registered`, + `form_required`, `tier_exceeds_class`, a `replicate_*` reason), or `ok`. For a + `WAI2` manifest the finding is `[entry index, reason]`. + +**Listings.** Every `listings/.sha256` is, byte for byte, the listing of +`sets//`: regular files, names that begin with `.` left out, sorted by path as +bytes. A common checksum utility checks a set against its listing +(`cd sets/demo && sha256sum -c ../../listings/demo.sha256`). Every file under +`listings/bad/` breaks one listing rule and is `set_malformed`. + +Files are compared as bytes; `.gitattributes` turns off line-ending conversion. + +**Verifiers.** + +- `wai_pin_vectors verify pin-conformance` (wai-rs) runs every file through the + reference implementation's in-memory pin core (`wai::pin::verify_bytes`), its + selection (`WaiMulti::pick`) and its static checks (`Wai::check_pin`, + `WaiMulti::check_pins`). It runs in both corpus loops of the conformance workflow. + `wai_pin_vectors generate pin-conformance` rewrites the listings from the sets. +- `pin_verify.py` checks `cases.json` and `listings/` with the Python standard + library alone, re-implementing the listing rules and the ordered checks from the + SPEC text. It shares no code with wai-rs. diff --git a/wai/pin-conformance/cases.json b/wai/pin-conformance/cases.json new file mode 100644 index 000000000..cdd4ec4e7 --- /dev/null +++ b/wai/pin-conformance/cases.json @@ -0,0 +1,481 @@ +{ + "about": "SPEC.md \u00a73.1: the reason a conforming verifier reports for each pin, capability and set of held artifacts. Paths are relative to this directory; a listing's files are read below its root. readable defaults to [\"wai.prior.set\"].", + "cases": [ + { + "name": "one_file_legacy", + "capability": "wai.neural.int_mlicpp", + "pin": { + "id": "conformance", + "sha256": "7ef44f3b1fc4a2df6c3e3b68460f595cae3d31cb7f9018699e29e738340272a4", + "determinism": "int-rans/p16+int/i64" + }, + "held": [ + { + "file": "../wai-rs/tests/vectors/int_mlicpp/model.wmb" + } + ], + "expect": "ok" + }, + { + "name": "uppercase_digest", + "capability": "wai.neural.int_mlicpp", + "pin": { + "id": "conformance", + "sha256": "7EF44F3B1FC4A2DF6C3E3B68460F595CAE3D31CB7F9018699E29E738340272A4", + "determinism": "int-rans/p16+int/i64" + }, + "held": [ + { + "file": "../wai-rs/tests/vectors/int_mlicpp/model.wmb" + } + ], + "expect": "pin_malformed" + }, + { + "name": "digest_63_digits", + "capability": "wai.neural.int_mlicpp", + "pin": { + "id": "conformance", + "sha256": "7ef44f3b1fc4a2df6c3e3b68460f595cae3d31cb7f9018699e29e738340272a", + "determinism": "int-rans/p16+int/i64" + }, + "held": [ + { + "file": "../wai-rs/tests/vectors/int_mlicpp/model.wmb" + } + ], + "expect": "pin_malformed" + }, + { + "name": "malformed_before_unreadable_form", + "capability": "wai.neural.int_mlicpp", + "pin": { + "id": "conformance", + "sha256": "7EF44F3B1FC4A2DF6C3E3B68460F595CAE3D31CB7F9018699E29E738340272A4", + "determinism": "int-rans/p16+int/i64", + "form": "wai.prior.nnc" + }, + "held": [], + "expect": "pin_malformed" + }, + { + "name": "form_unregistered", + "capability": "wai.neural.int_mlicpp", + "pin": { + "id": "conformance", + "sha256": "7ef44f3b1fc4a2df6c3e3b68460f595cae3d31cb7f9018699e29e738340272a4", + "determinism": "int-rans/p16+int/i64", + "form": "wai.prior.nope" + }, + "held": [ + { + "file": "../wai-rs/tests/vectors/int_mlicpp/model.wmb" + } + ], + "expect": "form_not_registered" + }, + { + "name": "form_is_a_payload_codec", + "capability": "wai.neural.int_mlicpp", + "pin": { + "id": "conformance", + "sha256": "7ef44f3b1fc4a2df6c3e3b68460f595cae3d31cb7f9018699e29e738340272a4", + "determinism": "int-rans/p16+int/i64", + "form": "wai.image.png" + }, + "held": [ + { + "file": "../wai-rs/tests/vectors/int_mlicpp/model.wmb" + } + ], + "expect": "form_not_registered" + }, + { + "name": "form_unreadable_nnc", + "capability": "wai.neural.int_mlicpp", + "pin": { + "id": "conformance", + "sha256": "7ef44f3b1fc4a2df6c3e3b68460f595cae3d31cb7f9018699e29e738340272a4", + "determinism": "int-rans/p16+int/i64", + "form": "wai.prior.nnc" + }, + "held": [ + { + "file": "../wai-rs/tests/vectors/int_mlicpp/model.wmb" + } + ], + "expect": "form_unreadable" + }, + { + "name": "form_required_several_files", + "capability": "wai.neural.bmshj2018", + "pin": { + "id": "conformance", + "sha256": "7ef44f3b1fc4a2df6c3e3b68460f595cae3d31cb7f9018699e29e738340272a4", + "determinism": "int-rans/p16+int/i64" + }, + "held": [ + { + "file": "../wai-rs/tests/vectors/int_mlicpp/model.wmb" + } + ], + "expect": "form_required" + }, + { + "name": "form_required_directory_capability", + "capability": "wai.image.jpegai", + "pin": { + "id": "conformance", + "sha256": "7ef44f3b1fc4a2df6c3e3b68460f595cae3d31cb7f9018699e29e738340272a4", + "determinism": "int-rans/p16+int/i64" + }, + "held": [ + { + "file": "../wai-rs/tests/vectors/int_mlicpp/model.wmb" + } + ], + "expect": "form_required" + }, + { + "name": "unregistered_capability_pin_is_verified", + "capability": "wai.neural.future", + "pin": { + "id": "conformance", + "sha256": "7ef44f3b1fc4a2df6c3e3b68460f595cae3d31cb7f9018699e29e738340272a4", + "determinism": "int-rans/p16+int/i64" + }, + "held": [ + { + "file": "../wai-rs/tests/vectors/int_mlicpp/model.wmb" + } + ], + "expect": "form_required" + }, + { + "name": "tampered_file_not_held", + "capability": "wai.neural.int_mlicpp", + "pin": { + "id": "conformance", + "sha256": "29bc4a3984b2ca82ab4656e459e7c34b5af9f507309bcf88ed7b00561692ff0a", + "determinism": "int-rans/p16+int/i64" + }, + "held": [ + { + "file": "sets/tiny-tampered/model.wmb" + } + ], + "expect": "prior_not_held" + }, + { + "name": "id_matches_digest_differs", + "capability": "wai.neural.int_mlicpp", + "pin": { + "id": "conformance", + "sha256": "402709318223af457631c917c5a1081cb8e380e1dde47c6aebd8f264e077f090", + "determinism": "int-rans/p16+int/i64" + }, + "held": [ + { + "file": "../wai-rs/tests/vectors/int_mlicpp/model.wmb" + } + ], + "expect": "prior_not_held" + }, + { + "name": "digest_superseded", + "capability": "wai.neural.int_mlicpp", + "pin": { + "id": "conformance", + "sha256": "402709318223af457631c917c5a1081cb8e380e1dde47c6aebd8f264e077f090", + "determinism": "int-rans/p16+int/i64", + "supersedes": { + "sha256": "7ef44f3b1fc4a2df6c3e3b68460f595cae3d31cb7f9018699e29e738340272a4" + } + }, + "held": [ + { + "file": "../wai-rs/tests/vectors/int_mlicpp/model.wmb" + } + ], + "expect": "digest_superseded" + }, + { + "name": "supersedes_malformed_is_ignored", + "capability": "wai.neural.int_mlicpp", + "pin": { + "id": "conformance", + "sha256": "7ef44f3b1fc4a2df6c3e3b68460f595cae3d31cb7f9018699e29e738340272a4", + "determinism": "int-rans/p16+int/i64", + "supersedes": { + "sha256": "7ef44f3b1fc4a2df6c3e3b68460f595cae3d31cb7f9018699e29e738340272a4" + } + }, + "held": [ + { + "file": "../wai-rs/tests/vectors/int_mlicpp/model.wmb" + } + ], + "expect": "ok" + }, + { + "name": "supersedes_bad_form_never_superseded", + "capability": "wai.neural.int_mlicpp", + "pin": { + "id": "conformance", + "sha256": "402709318223af457631c917c5a1081cb8e380e1dde47c6aebd8f264e077f090", + "determinism": "int-rans/p16+int/i64", + "supersedes": { + "sha256": "7ef44f3b1fc4a2df6c3e3b68460f595cae3d31cb7f9018699e29e738340272a4", + "form": "wai.image.png" + } + }, + "held": [ + { + "file": "../wai-rs/tests/vectors/int_mlicpp/model.wmb" + } + ], + "expect": "prior_not_held" + }, + { + "name": "successor_never_satisfies_predecessor", + "capability": "wai.neural.int_mlicpp", + "pin": { + "id": "conformance", + "sha256": "7ef44f3b1fc4a2df6c3e3b68460f595cae3d31cb7f9018699e29e738340272a4", + "determinism": "int-rans/p16+int/i64" + }, + "held": [ + { + "file": "sets/tiny/model.wmb" + } + ], + "expect": "prior_not_held" + }, + { + "name": "set_ok", + "capability": "wai.neural.int_mlicpp", + "pin": { + "id": "conformance", + "sha256": "85a05b0dac15566c5a657ddeee253eaa9f54ce852ae04924f5b04151901d9d1b", + "determinism": "int-rans/p16+int/i64", + "form": "wai.prior.set", + "tier": "decode-equivalence" + }, + "held": [ + { + "listing": "../wai-rs/tests/vectors/int_mlicpp/model.sha256", + "root": "../wai-rs/tests/vectors/int_mlicpp" + } + ], + "expect": "ok" + }, + { + "name": "set_ok_several_files", + "capability": "wai.neural.elic", + "pin": { + "id": "conformance", + "sha256": "88d72629c0c1d5adef7587afa6926405eefd0b7aaabf608483d8551d2536790f", + "determinism": "int-rans/p16+int/i64", + "form": "wai.prior.set" + }, + "held": [ + { + "listing": "listings/demo.sha256", + "root": "sets/demo" + } + ], + "expect": "ok" + }, + { + "name": "set_pin_is_not_a_file_pin", + "capability": "wai.neural.int_mlicpp", + "pin": { + "id": "conformance", + "sha256": "85a05b0dac15566c5a657ddeee253eaa9f54ce852ae04924f5b04151901d9d1b", + "determinism": "int-rans/p16+int/i64", + "form": "wai.prior.set" + }, + "held": [ + { + "file": "../wai-rs/tests/vectors/int_mlicpp/model.wmb" + } + ], + "expect": "prior_not_held" + }, + { + "name": "file_pin_is_not_a_set_pin", + "capability": "wai.neural.int_mlicpp", + "pin": { + "id": "conformance", + "sha256": "7ef44f3b1fc4a2df6c3e3b68460f595cae3d31cb7f9018699e29e738340272a4", + "determinism": "int-rans/p16+int/i64" + }, + "held": [ + { + "listing": "../wai-rs/tests/vectors/int_mlicpp/model.sha256", + "root": "../wai-rs/tests/vectors/int_mlicpp" + } + ], + "expect": "prior_not_held" + }, + { + "name": "set_malformed_crlf", + "capability": "wai.neural.int_mlicpp", + "pin": { + "id": "conformance", + "sha256": "7703c3e01c81277fa56da7220805d3e96ea5858f1f87b1f372136c3ce112351c", + "determinism": "int-rans/p16+int/i64", + "form": "wai.prior.set" + }, + "held": [ + { + "listing": "listings/bad/crlf.sha256", + "root": "sets/tiny" + } + ], + "expect": "set_malformed" + }, + { + "name": "part_mismatch", + "capability": "wai.neural.int_mlicpp", + "pin": { + "id": "conformance", + "sha256": "ac5a74765058dac5b44f90185af19852e70ce2922fcac2ace28590b633a9c376", + "determinism": "int-rans/p16+int/i64", + "form": "wai.prior.set" + }, + "held": [ + { + "listing": "listings/tiny.sha256", + "root": "sets/tiny-tampered" + } + ], + "expect": "part_mismatch" + }, + { + "name": "part_missing", + "capability": "wai.neural.int_mlicpp", + "pin": { + "id": "conformance", + "sha256": "ac5a74765058dac5b44f90185af19852e70ce2922fcac2ace28590b633a9c376", + "determinism": "int-rans/p16+int/i64", + "form": "wai.prior.set" + }, + "held": [ + { + "listing": "listings/tiny.sha256", + "root": "sets/demo" + } + ], + "expect": "part_mismatch" + }, + { + "name": "two_roots_one_intact", + "capability": "wai.neural.int_mlicpp", + "pin": { + "id": "conformance", + "sha256": "ac5a74765058dac5b44f90185af19852e70ce2922fcac2ace28590b633a9c376", + "determinism": "int-rans/p16+int/i64", + "form": "wai.prior.set" + }, + "held": [ + { + "listing": "listings/tiny.sha256", + "root": "sets/tiny-tampered" + }, + { + "listing": "listings/tiny.sha256", + "root": "sets/tiny" + } + ], + "expect": "ok" + }, + { + "name": "flipped_listing_not_held", + "capability": "wai.neural.int_mlicpp", + "pin": { + "id": "conformance", + "sha256": "ac5a74765058dac5b44f90185af19852e70ce2922fcac2ace28590b633a9c376", + "determinism": "int-rans/p16+int/i64", + "form": "wai.prior.set" + }, + "held": [ + { + "listing": "listings/bad/uppercase_hex.sha256", + "root": "sets/tiny" + } + ], + "expect": "prior_not_held" + }, + { + "name": "set_superseded", + "capability": "wai.neural.int_mlicpp", + "pin": { + "id": "conformance", + "sha256": "402709318223af457631c917c5a1081cb8e380e1dde47c6aebd8f264e077f090", + "determinism": "int-rans/p16+int/i64", + "form": "wai.prior.set", + "supersedes": { + "sha256": "ac5a74765058dac5b44f90185af19852e70ce2922fcac2ace28590b633a9c376", + "form": "wai.prior.set" + } + }, + "held": [ + { + "listing": "listings/tiny.sha256", + "root": "sets/tiny" + } + ], + "expect": "digest_superseded" + }, + { + "name": "int_synth_one_file", + "capability": "wai.neural.int_synth", + "pin": { + "id": "conformance", + "sha256": "1c4705d17c01f865b20d785191c7f63b5c18dcc8fce77e15852cf8240b7368be", + "determinism": "int-rans/p16+int/i64" + }, + "held": [ + { + "file": "../neural-conformance/vectors/gradient/int_synth.json" + } + ], + "expect": "ok" + }, + { + "name": "null_member_equals_absent", + "capability": "wai.neural.int_mlicpp", + "pin": { + "id": "conformance", + "sha256": "7ef44f3b1fc4a2df6c3e3b68460f595cae3d31cb7f9018699e29e738340272a4", + "determinism": "int-rans/p16+int/i64", + "form": null, + "tier": null, + "supersedes": null + }, + "held": [ + { + "file": "../wai-rs/tests/vectors/int_mlicpp/model.wmb" + } + ], + "expect": "ok" + }, + { + "name": "tier_unregistered_still_verifies", + "capability": "wai.neural.int_mlicpp", + "pin": { + "id": "conformance", + "sha256": "7ef44f3b1fc4a2df6c3e3b68460f595cae3d31cb7f9018699e29e738340272a4", + "determinism": "int-rans/p16+int/i64", + "tier": "decode_equivalence" + }, + "held": [ + { + "file": "../wai-rs/tests/vectors/int_mlicpp/model.wmb" + } + ], + "expect": "ok" + } + ] +} diff --git a/wai/pin-conformance/listings/bad/blank_line.sha256 b/wai/pin-conformance/listings/bad/blank_line.sha256 new file mode 100644 index 000000000..3f7259828 --- /dev/null +++ b/wai/pin-conformance/listings/bad/blank_line.sha256 @@ -0,0 +1,2 @@ +29bc4a3984b2ca82ab4656e459e7c34b5af9f507309bcf88ed7b00561692ff0a model.wmb + diff --git a/wai/pin-conformance/listings/bad/bom.sha256 b/wai/pin-conformance/listings/bad/bom.sha256 new file mode 100644 index 000000000..fb9353725 --- /dev/null +++ b/wai/pin-conformance/listings/bad/bom.sha256 @@ -0,0 +1 @@ +29bc4a3984b2ca82ab4656e459e7c34b5af9f507309bcf88ed7b00561692ff0a model.wmb diff --git a/wai/pin-conformance/listings/bad/case_fold.sha256 b/wai/pin-conformance/listings/bad/case_fold.sha256 new file mode 100644 index 000000000..67dd92f82 --- /dev/null +++ b/wai/pin-conformance/listings/bad/case_fold.sha256 @@ -0,0 +1,2 @@ +6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b A.json +d4735e3a265e16eee03f59718b9b5d03019c07d8b6c51f90da3a666eec13ab35 a.json diff --git a/wai/pin-conformance/listings/bad/crlf.sha256 b/wai/pin-conformance/listings/bad/crlf.sha256 new file mode 100644 index 000000000..f726ce199 --- /dev/null +++ b/wai/pin-conformance/listings/bad/crlf.sha256 @@ -0,0 +1 @@ +29bc4a3984b2ca82ab4656e459e7c34b5af9f507309bcf88ed7b00561692ff0a model.wmb diff --git a/wai/pin-conformance/listings/bad/dot_dot.sha256 b/wai/pin-conformance/listings/bad/dot_dot.sha256 new file mode 100644 index 000000000..27fd020c2 --- /dev/null +++ b/wai/pin-conformance/listings/bad/dot_dot.sha256 @@ -0,0 +1 @@ +6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b a/../b diff --git a/wai/pin-conformance/listings/bad/duplicate.sha256 b/wai/pin-conformance/listings/bad/duplicate.sha256 new file mode 100644 index 000000000..4c9abf432 --- /dev/null +++ b/wai/pin-conformance/listings/bad/duplicate.sha256 @@ -0,0 +1,2 @@ +6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b a +6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b a diff --git a/wai/pin-conformance/listings/bad/empty.sha256 b/wai/pin-conformance/listings/bad/empty.sha256 new file mode 100644 index 000000000..e69de29bb diff --git a/wai/pin-conformance/listings/bad/file_and_directory.sha256 b/wai/pin-conformance/listings/bad/file_and_directory.sha256 new file mode 100644 index 000000000..63412a0f0 --- /dev/null +++ b/wai/pin-conformance/listings/bad/file_and_directory.sha256 @@ -0,0 +1,2 @@ +6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b w +d4735e3a265e16eee03f59718b9b5d03019c07d8b6c51f90da3a666eec13ab35 w/x diff --git a/wai/pin-conformance/listings/bad/leading_dot.sha256 b/wai/pin-conformance/listings/bad/leading_dot.sha256 new file mode 100644 index 000000000..9a2fdbf66 --- /dev/null +++ b/wai/pin-conformance/listings/bad/leading_dot.sha256 @@ -0,0 +1 @@ +6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b .hidden diff --git a/wai/pin-conformance/listings/bad/no_final_lf.sha256 b/wai/pin-conformance/listings/bad/no_final_lf.sha256 new file mode 100644 index 000000000..6f820cad5 --- /dev/null +++ b/wai/pin-conformance/listings/bad/no_final_lf.sha256 @@ -0,0 +1 @@ +29bc4a3984b2ca82ab4656e459e7c34b5af9f507309bcf88ed7b00561692ff0a model.wmb \ No newline at end of file diff --git a/wai/pin-conformance/listings/bad/one_space.sha256 b/wai/pin-conformance/listings/bad/one_space.sha256 new file mode 100644 index 000000000..8753e273f --- /dev/null +++ b/wai/pin-conformance/listings/bad/one_space.sha256 @@ -0,0 +1 @@ +29bc4a3984b2ca82ab4656e459e7c34b5af9f507309bcf88ed7b00561692ff0a model.wmb diff --git a/wai/pin-conformance/listings/bad/trailing_dot.sha256 b/wai/pin-conformance/listings/bad/trailing_dot.sha256 new file mode 100644 index 000000000..3fbd9896a --- /dev/null +++ b/wai/pin-conformance/listings/bad/trailing_dot.sha256 @@ -0,0 +1 @@ +6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b a. diff --git a/wai/pin-conformance/listings/bad/unsorted.sha256 b/wai/pin-conformance/listings/bad/unsorted.sha256 new file mode 100644 index 000000000..094a12925 --- /dev/null +++ b/wai/pin-conformance/listings/bad/unsorted.sha256 @@ -0,0 +1,2 @@ +6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b b +d4735e3a265e16eee03f59718b9b5d03019c07d8b6c51f90da3a666eec13ab35 a diff --git a/wai/pin-conformance/listings/bad/uppercase_hex.sha256 b/wai/pin-conformance/listings/bad/uppercase_hex.sha256 new file mode 100644 index 000000000..d30c35f80 --- /dev/null +++ b/wai/pin-conformance/listings/bad/uppercase_hex.sha256 @@ -0,0 +1 @@ +29BC4A3984B2CA82AB4656E459E7C34B5AF9F507309BCF88ED7B00561692FF0A model.wmb diff --git a/wai/pin-conformance/listings/demo.sha256 b/wai/pin-conformance/listings/demo.sha256 new file mode 100644 index 000000000..8ada0a154 --- /dev/null +++ b/wai/pin-conformance/listings/demo.sha256 @@ -0,0 +1,3 @@ +4b9090d289a978de401cf08619aada32980438b296a7c261b760ec1cb3e61a42 a.b +550c3f093fed7ebdf3ac74ecf5c0b5d410a072d7af081a100af4e1c1087a2949 a/c.onnx +a2f36dd7d60b3eb35ba27548a7058a227f2d0d48f4ce8e7a55b5c993b2069e10 weights/hs_y.json diff --git a/wai/pin-conformance/listings/tiny.sha256 b/wai/pin-conformance/listings/tiny.sha256 new file mode 100644 index 000000000..b98f886e3 --- /dev/null +++ b/wai/pin-conformance/listings/tiny.sha256 @@ -0,0 +1 @@ +29bc4a3984b2ca82ab4656e459e7c34b5af9f507309bcf88ed7b00561692ff0a model.wmb diff --git a/wai/pin-conformance/pin_verify.py b/wai/pin-conformance/pin_verify.py new file mode 100644 index 000000000..600945515 --- /dev/null +++ b/wai/pin-conformance/pin_verify.py @@ -0,0 +1,184 @@ +#!/usr/bin/env python3 +"""Independent check of the pin-conformance corpus (SPEC.md §3.1). + +Shares no code with the Rust reference implementation: it re-implements the +listing rules and the ordered pin checks from the SPEC text, with the standard +library only, and checks that + + - every `listings/.sha256` is byte for byte the listing of `sets//` + (regular files, names that begin with `.` skipped), and every file under + `listings/bad/` breaks a listing rule; + - every case in `cases.json` gets its declared reason. + + python3 pin-conformance/pin_verify.py [corpus-dir] + +The registry facts it needs are restated here from SPEC.md §5 and §3.1: the +registered prior forms, the forms the reference sink reads, and the capabilities +registered with a one-file parameter set. +""" +import hashlib +import json +import os +import re +import sys + +PRIOR_FORMS = {'wai.prior.nnc', 'wai.prior.set'} +READABLE = ['wai.prior.set'] +ONE_FILE = { + 'wai.neural.int_synth', 'wai.neural.int_mlicpp', 'wai.neural.encodec32', 'wai.neural.dac', + 'wai.neural.mimi', 'wai.neural.wavtokenizer', 'wai.neural.snac', 'wai.neural.video_bmshj2018', +} +HEX64 = re.compile(r'^[0-9a-f]{64}$') +SEGMENT = re.compile(r'^[A-Za-z0-9_-]([A-Za-z0-9_.-]*[A-Za-z0-9_-])?$') + + +class Malformed(Exception): + pass + + +def sha(b): + return hashlib.sha256(b).hexdigest() + + +def parse_listing(b): + if not b or not b.endswith(b'\n'): + raise Malformed('empty, or no final line feed') + lines = [] + for raw in b[:-1].split(b'\n'): + if len(raw) < 67 or raw[64:66] != b' ': + raise Malformed('line shape') + d = raw[:64].decode('ascii', 'replace') + if not HEX64.match(d): + raise Malformed('digest') + try: + p = raw[66:].decode('ascii') + except UnicodeDecodeError: + raise Malformed('path not ASCII') + if not all(SEGMENT.match(s) for s in p.split('/')): + raise Malformed(f'path {p!r}') + lines.append((p, d)) + paths = [p for p, _ in lines] + if any(a.encode() >= b.encode() for a, b in zip(paths, paths[1:])): + raise Malformed('order') + folded = sorted(p.lower() for p in paths) + if any(a == b for a, b in zip(folded, folded[1:])): + raise Malformed('case fold') + for a in paths: + if any(b.startswith(a + '/') for b in paths): + raise Malformed('file and directory') + return lines + + +def listing_of_dir(root): + files = [] + for dirpath, dirnames, filenames in os.walk(root): + dirnames[:] = [d for d in dirnames if not d.startswith('.')] + rel = os.path.relpath(dirpath, root) + for f in filenames: + if f.startswith('.'): + continue + if rel == '.' and f.endswith('.sha256'): + continue + p = f if rel == '.' else rel.replace(os.sep, '/') + '/' + f + files.append(p) + out = b'' + for p in sorted(files, key=lambda x: x.encode()): + with open(os.path.join(root, p), 'rb') as fh: + out += sha(fh.read()).encode() + b' ' + p.encode() + b'\n' + return out + + +def verify(capability, pin, held, readable): + """SPEC §3.1's checks, in order; returns 'ok' or the reason.""" + if not HEX64.match(pin['sha256']): + return 'pin_malformed' + form = pin.get('form') + if form is not None: + if form not in PRIOR_FORMS: + return 'form_not_registered' + if form not in readable: + return 'form_unreadable' + elif capability not in ONE_FILE: + return 'form_required' + set_form = form == 'wai.prior.set' + digest = lambda h: sha(h['bytes']) if (('files' in h) == set_form) else None + matching = [h for h in held if digest(h) == pin['sha256']] + if not matching: + sup = pin.get('supersedes') + if (sup is not None and HEX64.match(sup['sha256']) and sup['sha256'] != pin['sha256'] + and (sup.get('form') is None or sup.get('form') in PRIOR_FORMS)): + pre_set = sup.get('form') == 'wai.prior.set' + if any(sha(h['bytes']) == sup['sha256'] and (('files' in h) == pre_set) for h in held): + return 'digest_superseded' + return 'prior_not_held' + if not set_form: + return 'ok' + try: + lines = parse_listing(matching[0]['bytes']) + except Malformed: + return 'set_malformed' + for h in matching: + if all(p in h['files'] and sha(h['files'][p]) == d for p, d in lines): + return 'ok' + return 'part_mismatch' + + +def load(corpus, held): + out = [] + for h in held: + if 'file' in h: + with open(os.path.join(corpus, h['file']), 'rb') as f: + out.append({'bytes': f.read()}) + else: + with open(os.path.join(corpus, h['listing']), 'rb') as f: + b = f.read() + files = {} + try: + for p, _ in parse_listing(b): + fp = os.path.join(corpus, h['root'], p) + if os.path.isfile(fp): + with open(fp, 'rb') as f: + files[p] = f.read() + except Malformed: + pass + out.append({'bytes': b, 'files': files}) + return out + + +def main(): + corpus = sys.argv[1] if len(sys.argv) > 1 else os.path.dirname(os.path.abspath(__file__)) + fails, n = [], 0 + ld = os.path.join(corpus, 'listings') + for name in sorted(os.listdir(ld)): + path = os.path.join(ld, name) + if name.endswith('.sha256') and os.path.isfile(path): + n += 1 + with open(path, 'rb') as f: + committed = f.read() + if listing_of_dir(os.path.join(corpus, 'sets', name[:-len('.sha256')])) != committed: + fails.append(f'listings/{name} is not the listing of its set') + for name in sorted(os.listdir(os.path.join(ld, 'bad'))): + n += 1 + with open(os.path.join(ld, 'bad', name), 'rb') as f: + b = f.read() + try: + parse_listing(b) + fails.append(f'listings/bad/{name} parsed') + except Malformed: + pass + with open(os.path.join(corpus, 'cases.json')) as f: + cases = json.load(f)['cases'] + for c in cases: + n += 1 + pin = {k: v for k, v in c['pin'].items() if v is not None} + got = verify(c['capability'], pin, load(corpus, c['held']), c.get('readable', READABLE)) + if got != c['expect']: + fails.append(f"cases.json {c['name']}: expected {c['expect']}, got {got}") + for f in fails: + print('FAIL', f) + print(f'pin_verify.py: {n - len(fails)} of {n} checks as declared') + sys.exit(1 if fails else 0) + + +if __name__ == '__main__': + main() diff --git a/wai/pin-conformance/selection.json b/wai/pin-conformance/selection.json new file mode 100644 index 000000000..402b36ac5 --- /dev/null +++ b/wai/pin-conformance/selection.json @@ -0,0 +1,531 @@ +{ + "about": "SPEC.md \u00a77.1: the rendition a sink picks, the reason it reports for each entry it skips, and the companions it applies with the pick (companions_of_index). held lists the one-file parameter sets the sink holds; advertised is \"all\" or a list of capabilities. roles is the first check_component_roles finding (\"ok\" when the envelope conforms).", + "cases": [ + { + "name": "holds_predecessor", + "manifest": { + "wai": "1.1", + "media": "image", + "intent": "replicate", + "renditions": [ + { + "capability": "wai.neural.int_mlicpp", + "kind": "int_mlicpp", + "prior": { + "id": "conformance", + "sha256": "c4232f2e2d5f76cce4ac782b254a91410a2ab89b22792f8356585549938a0e92", + "determinism": "int-rans/p16+int/i64", + "tier": "decode-equivalence", + "supersedes": { + "sha256": "29bc4a3984b2ca82ab4656e459e7c34b5af9f507309bcf88ed7b00561692ff0a" + } + } + }, + { + "capability": "wai.neural.int_mlicpp", + "kind": "int_mlicpp", + "prior": { + "id": "conformance", + "sha256": "29bc4a3984b2ca82ab4656e459e7c34b5af9f507309bcf88ed7b00561692ff0a", + "determinism": "int-rans/p16+int/i64", + "tier": "decode-equivalence" + } + }, + { + "capability": "wai.image.avif", + "kind": "avif" + } + ], + "target": null + }, + "payloads": [ + "B-CODED", + "A-CODED", + "AVIF" + ], + "advertised": "all", + "held": [ + "sets/tiny/model.wmb" + ], + "expect": { + "picked": 1, + "skipped": [ + [ + 0, + "digest_superseded" + ] + ] + } + }, + { + "name": "holds_successor", + "manifest": { + "wai": "1.1", + "media": "image", + "intent": "replicate", + "renditions": [ + { + "capability": "wai.neural.int_mlicpp", + "kind": "int_mlicpp", + "prior": { + "id": "conformance", + "sha256": "c4232f2e2d5f76cce4ac782b254a91410a2ab89b22792f8356585549938a0e92", + "determinism": "int-rans/p16+int/i64", + "tier": "decode-equivalence", + "supersedes": { + "sha256": "29bc4a3984b2ca82ab4656e459e7c34b5af9f507309bcf88ed7b00561692ff0a" + } + } + }, + { + "capability": "wai.neural.int_mlicpp", + "kind": "int_mlicpp", + "prior": { + "id": "conformance", + "sha256": "29bc4a3984b2ca82ab4656e459e7c34b5af9f507309bcf88ed7b00561692ff0a", + "determinism": "int-rans/p16+int/i64", + "tier": "decode-equivalence" + } + }, + { + "capability": "wai.image.avif", + "kind": "avif" + } + ], + "target": null + }, + "payloads": [ + "B-CODED", + "A-CODED", + "AVIF" + ], + "advertised": "all", + "held": [ + "sets/tiny-tampered/model.wmb" + ], + "expect": { + "picked": 0, + "skipped": [] + } + }, + { + "name": "holds_neither", + "manifest": { + "wai": "1.1", + "media": "image", + "intent": "replicate", + "renditions": [ + { + "capability": "wai.neural.int_mlicpp", + "kind": "int_mlicpp", + "prior": { + "id": "conformance", + "sha256": "c4232f2e2d5f76cce4ac782b254a91410a2ab89b22792f8356585549938a0e92", + "determinism": "int-rans/p16+int/i64", + "tier": "decode-equivalence", + "supersedes": { + "sha256": "29bc4a3984b2ca82ab4656e459e7c34b5af9f507309bcf88ed7b00561692ff0a" + } + } + }, + { + "capability": "wai.neural.int_mlicpp", + "kind": "int_mlicpp", + "prior": { + "id": "conformance", + "sha256": "29bc4a3984b2ca82ab4656e459e7c34b5af9f507309bcf88ed7b00561692ff0a", + "determinism": "int-rans/p16+int/i64", + "tier": "decode-equivalence" + } + }, + { + "capability": "wai.image.avif", + "kind": "avif" + } + ], + "target": null + }, + "payloads": [ + "B-CODED", + "A-CODED", + "AVIF" + ], + "advertised": "all", + "held": [], + "expect": { + "picked": 2, + "skipped": [ + [ + 0, + "prior_not_held" + ], + [ + 1, + "prior_not_held" + ] + ] + } + }, + { + "name": "inert", + "manifest": { + "wai": "1.1", + "media": "image", + "intent": "replicate", + "renditions": [ + { + "capability": "wai.neural.int_mlicpp", + "kind": "int_mlicpp", + "prior": { + "id": "conformance", + "sha256": "c4232f2e2d5f76cce4ac782b254a91410a2ab89b22792f8356585549938a0e92", + "determinism": "int-rans/p16+int/i64", + "tier": "decode-equivalence", + "supersedes": { + "sha256": "29bc4a3984b2ca82ab4656e459e7c34b5af9f507309bcf88ed7b00561692ff0a" + } + } + }, + { + "capability": "wai.neural.int_mlicpp", + "kind": "int_mlicpp", + "prior": { + "id": "conformance", + "sha256": "29bc4a3984b2ca82ab4656e459e7c34b5af9f507309bcf88ed7b00561692ff0a", + "determinism": "int-rans/p16+int/i64", + "tier": "decode-equivalence" + } + }, + { + "capability": "wai.image.avif", + "kind": "avif" + } + ], + "target": null + }, + "payloads": [ + "B-CODED", + "A-CODED", + "AVIF" + ], + "advertised": [ + "wai.neural.int_mlicpp" + ], + "held": [], + "expect": { + "inert": [ + [ + 0, + "prior_not_held" + ], + [ + 1, + "prior_not_held" + ], + [ + 2, + "missing_capability" + ] + ] + } + }, + { + "name": "component_digest_ok", + "manifest": { + "wai": "1.1", + "media": "image", + "intent": "replicate", + "renditions": [ + { + "capability": "wai.image.png", + "kind": "png", + "sha256": "e793c41f42c3766932d69864deeea3b3949a26e1e427bdf261a77755021e58cb" + }, + { + "capability": "wai.image.png", + "kind": "png", + "sha256": "700983acb1832a2b0830405407bcaefaf0909c8631c1603cc5a72ecc4257e4d2" + }, + { + "capability": "wai.meta.hdr10plus", + "kind": "descriptor", + "role": "companion", + "companion_of": "wai.image.png", + "sha256": "2d1edf729cc53756834657ce399b1568debd59b0ceddce5da39b86750d59fbd0" + } + ], + "target": null + }, + "payloads": [ + "PNG-A", + "PNG-B", + "META" + ], + "advertised": "all", + "held": [], + "expect": { + "picked": 0, + "skipped": [], + "companions": [ + 2 + ] + } + }, + { + "name": "component_digest_mismatch", + "manifest": { + "wai": "1.1", + "media": "image", + "intent": "replicate", + "renditions": [ + { + "capability": "wai.image.png", + "kind": "png", + "sha256": "e793c41f42c3766932d69864deeea3b3949a26e1e427bdf261a77755021e58cb" + }, + { + "capability": "wai.image.png", + "kind": "png", + "sha256": "700983acb1832a2b0830405407bcaefaf0909c8631c1603cc5a72ecc4257e4d2" + }, + { + "capability": "wai.meta.hdr10plus", + "kind": "descriptor", + "role": "companion", + "companion_of": "wai.image.png", + "sha256": "2d1edf729cc53756834657ce399b1568debd59b0ceddce5da39b86750d59fbd0" + } + ], + "target": null + }, + "payloads": [ + "PNG-a", + "PNG-B", + "META" + ], + "advertised": "all", + "held": [], + "expect": { + "picked": 1, + "skipped": [ + [ + 0, + "component_digest_mismatch" + ] + ], + "companions": [ + 2 + ] + } + }, + { + "name": "companion_digest_mismatch_not_applied", + "manifest": { + "wai": "1.1", + "media": "image", + "intent": "replicate", + "renditions": [ + { + "capability": "wai.image.png", + "kind": "png", + "sha256": "e793c41f42c3766932d69864deeea3b3949a26e1e427bdf261a77755021e58cb" + }, + { + "capability": "wai.image.png", + "kind": "png", + "sha256": "700983acb1832a2b0830405407bcaefaf0909c8631c1603cc5a72ecc4257e4d2" + }, + { + "capability": "wai.meta.hdr10plus", + "kind": "descriptor", + "role": "companion", + "companion_of": "wai.image.png", + "sha256": "2d1edf729cc53756834657ce399b1568debd59b0ceddce5da39b86750d59fbd0" + } + ], + "target": null + }, + "payloads": [ + "PNG-A", + "PNG-B", + "METa" + ], + "advertised": "all", + "held": [], + "expect": { + "picked": 0, + "skipped": [], + "companions": [] + } + }, + { + "name": "companion_of_id_binds_one_rendition", + "manifest": { + "wai": "1.1", + "media": "video", + "intent": "replicate", + "renditions": [ + { + "capability": "wai.video.hevc", + "kind": "hevc", + "id": "a" + }, + { + "capability": "wai.video.hevc", + "kind": "hevc", + "id": "b" + }, + { + "capability": "wai.video.lcevc", + "kind": "descriptor", + "role": "companion", + "companion_of": "wai.video.hevc", + "companion_of_id": "b" + } + ], + "target": null + }, + "payloads": [ + "HEVC-A", + "HEVC-B", + "ENH" + ], + "advertised": [ + "wai.video.hevc" + ], + "held": [], + "expect": { + "picked": 0, + "skipped": [], + "companions": [], + "roles": "ok" + } + }, + { + "name": "companion_of_id_applies_to_the_picked_rendition", + "manifest": { + "wai": "1.1", + "media": "video", + "intent": "replicate", + "renditions": [ + { + "capability": "wai.video.hevc", + "kind": "hevc", + "id": "a", + "sha256": "4e452595c3c6ccc249140f0ffe2b163b6c57203f0c7cfdcf0e4316e47fb55df7" + }, + { + "capability": "wai.video.hevc", + "kind": "hevc", + "id": "b" + }, + { + "capability": "wai.video.lcevc", + "kind": "descriptor", + "role": "companion", + "companion_of": "wai.video.hevc", + "companion_of_id": "b" + } + ], + "target": null + }, + "payloads": [ + "HEVC-a", + "HEVC-B", + "ENH" + ], + "advertised": "all", + "held": [], + "expect": { + "picked": 1, + "skipped": [ + [ + 0, + "component_digest_mismatch" + ] + ], + "companions": [ + 2 + ], + "roles": "ok" + } + }, + { + "name": "enhancement_base_ambiguous_without_id", + "manifest": { + "wai": "1.1", + "media": "video", + "intent": "replicate", + "renditions": [ + { + "capability": "wai.video.hevc", + "kind": "hevc", + "id": "a" + }, + { + "capability": "wai.video.hevc", + "kind": "hevc", + "id": "b" + }, + { + "capability": "wai.video.lcevc", + "kind": "descriptor", + "role": "companion", + "companion_of": "wai.video.hevc" + } + ], + "target": null + }, + "payloads": [ + "HEVC-A", + "HEVC-B", + "ENH" + ], + "advertised": "all", + "held": [], + "expect": { + "picked": 0, + "skipped": [], + "companions": [ + 2 + ], + "roles": "EnhancementBaseNotUnique" + } + }, + { + "name": "companion_only_never_presented", + "manifest": { + "wai": "1.1", + "media": "video", + "intent": "replicate", + "renditions": [ + { + "capability": "wai.video.lcevc", + "kind": "lcevc" + }, + { + "capability": "wai.video.hevc", + "kind": "hevc" + } + ], + "target": null + }, + "payloads": [ + "ENH", + "HEVC" + ], + "advertised": "all", + "held": [], + "expect": { + "picked": 1, + "skipped": [ + [ + 0, + "not_selectable" + ] + ], + "roles": "CompanionOnlyAsRendition" + } + } + ] +} diff --git a/wai/pin-conformance/sets/demo/.hidden b/wai/pin-conformance/sets/demo/.hidden new file mode 100644 index 000000000..176c0b9d7 --- /dev/null +++ b/wai/pin-conformance/sets/demo/.hidden @@ -0,0 +1 @@ +not listed diff --git a/wai/pin-conformance/sets/demo/a.b b/wai/pin-conformance/sets/demo/a.b new file mode 100644 index 000000000..420db7302 --- /dev/null +++ b/wai/pin-conformance/sets/demo/a.b @@ -0,0 +1 @@ +{"part":"a.b"} diff --git a/wai/pin-conformance/sets/demo/a/c.onnx b/wai/pin-conformance/sets/demo/a/c.onnx new file mode 100644 index 000000000..e713f18e2 --- /dev/null +++ b/wai/pin-conformance/sets/demo/a/c.onnx @@ -0,0 +1 @@ +ONNX-STAND-IN diff --git a/wai/pin-conformance/sets/demo/weights/hs_y.json b/wai/pin-conformance/sets/demo/weights/hs_y.json new file mode 100644 index 000000000..7512d0282 --- /dev/null +++ b/wai/pin-conformance/sets/demo/weights/hs_y.json @@ -0,0 +1 @@ +{"w":[1,2,3]} diff --git a/wai/pin-conformance/sets/tiny-tampered/model.wmb b/wai/pin-conformance/sets/tiny-tampered/model.wmb new file mode 100644 index 000000000..7d0c0c631 --- /dev/null +++ b/wai/pin-conformance/sets/tiny-tampered/model.wmb @@ -0,0 +1 @@ +WAI pin-conformance one-file seT diff --git a/wai/pin-conformance/sets/tiny/model.wmb b/wai/pin-conformance/sets/tiny/model.wmb new file mode 100644 index 000000000..22e3765c9 --- /dev/null +++ b/wai/pin-conformance/sets/tiny/model.wmb @@ -0,0 +1 @@ +WAI pin-conformance one-file set diff --git a/wai/pin-conformance/static.json b/wai/pin-conformance/static.json new file mode 100644 index 000000000..50061ff6d --- /dev/null +++ b/wai/pin-conformance/static.json @@ -0,0 +1,465 @@ +{ + "about": "SPEC.md \u00a73.1, \u00a77: the first finding a verifier makes about a manifest's pins without holding a parameter set (\"ok\" when it makes none). For a WAI2 manifest the finding is [entry index, reason].", + "cases": [ + { + "name": "stream_wai_conforms", + "manifest": { + "wai": "1.0", + "media": "image", + "intent": "replicate", + "model_requirement": { + "capability": "wai.neural.int_mlicpp" + }, + "conditioning": { + "kind": "int_mlicpp", + "prior": { + "id": "int_mlicpp/synthetic-s20260929", + "sha256": "7ef44f3b1fc4a2df6c3e3b68460f595cae3d31cb7f9018699e29e738340272a4", + "determinism": "int-rans/p16+int-mlicpp/i64" + } + }, + "target": { + "w": 64, + "h": 64 + } + }, + "expect": "ok" + }, + { + "name": "prior_not_applicable", + "manifest": { + "wai": "1.0", + "media": "image", + "intent": "replicate", + "model_requirement": { + "capability": "wai.image.png" + }, + "conditioning": { + "kind": "k", + "prior": { + "id": "conformance", + "sha256": "7ef44f3b1fc4a2df6c3e3b68460f595cae3d31cb7f9018699e29e738340272a4", + "determinism": "int-rans/p16+int/i64" + } + }, + "target": null + }, + "expect": "prior_not_applicable" + }, + { + "name": "prior_not_applicable_before_malformed", + "manifest": { + "wai": "1.0", + "media": "image", + "intent": "replicate", + "model_requirement": { + "capability": "wai.image.png" + }, + "conditioning": { + "kind": "k", + "prior": { + "id": "conformance", + "sha256": "AB", + "determinism": "int-rans/p16+int/i64" + } + }, + "target": null + }, + "expect": "prior_not_applicable" + }, + { + "name": "pin_malformed", + "manifest": { + "wai": "1.0", + "media": "image", + "intent": "replicate", + "model_requirement": { + "capability": "wai.neural.int_mlicpp" + }, + "conditioning": { + "kind": "k", + "prior": { + "id": "conformance", + "sha256": "7EF44F3B1FC4A2DF6C3E3B68460F595CAE3D31CB7F9018699E29E738340272A4", + "determinism": "int-rans/p16+int/i64" + } + }, + "target": null + }, + "expect": "pin_malformed" + }, + { + "name": "tier_unregistered", + "manifest": { + "wai": "1.0", + "media": "image", + "intent": "improve", + "model_requirement": { + "capability": "wai.neural.int_mlicpp" + }, + "conditioning": { + "kind": "k", + "prior": { + "id": "conformance", + "sha256": "7ef44f3b1fc4a2df6c3e3b68460f595cae3d31cb7f9018699e29e738340272a4", + "determinism": "int-rans/p16+int/i64", + "tier": "decode_equivalence" + } + }, + "target": null + }, + "expect": "tier_unregistered" + }, + { + "name": "supersedes_malformed", + "manifest": { + "wai": "1.0", + "media": "image", + "intent": "replicate", + "model_requirement": { + "capability": "wai.neural.int_mlicpp" + }, + "conditioning": { + "kind": "k", + "prior": { + "id": "conformance", + "sha256": "7ef44f3b1fc4a2df6c3e3b68460f595cae3d31cb7f9018699e29e738340272a4", + "determinism": "int-rans/p16+int/i64", + "supersedes": { + "sha256": "7ef44f3b1fc4a2df6c3e3b68460f595cae3d31cb7f9018699e29e738340272a4" + } + } + }, + "target": null + }, + "expect": "supersedes_malformed" + }, + { + "name": "form_not_registered", + "manifest": { + "wai": "1.0", + "media": "image", + "intent": "improve", + "model_requirement": { + "capability": "wai.neural.int_mlicpp" + }, + "conditioning": { + "kind": "k", + "prior": { + "id": "conformance", + "sha256": "7ef44f3b1fc4a2df6c3e3b68460f595cae3d31cb7f9018699e29e738340272a4", + "determinism": "int-rans/p16+int/i64", + "form": "wai.image.png" + } + }, + "target": null + }, + "expect": "form_not_registered" + }, + { + "name": "form_required", + "manifest": { + "wai": "1.0", + "media": "image", + "intent": "improve", + "model_requirement": { + "capability": "wai.neural.bmshj2018" + }, + "conditioning": { + "kind": "k", + "prior": { + "id": "conformance", + "sha256": "7ef44f3b1fc4a2df6c3e3b68460f595cae3d31cb7f9018699e29e738340272a4", + "determinism": "int-rans/p16+int/i64" + } + }, + "target": null + }, + "expect": "form_required" + }, + { + "name": "form_required_for_the_fallback", + "manifest": { + "wai": "1.0", + "media": "image", + "intent": "improve", + "model_requirement": { + "capability": "wai.neural.int_mlicpp", + "fallback": "wai.neural.elic" + }, + "conditioning": { + "kind": "k", + "prior": { + "id": "conformance", + "sha256": "7ef44f3b1fc4a2df6c3e3b68460f595cae3d31cb7f9018699e29e738340272a4", + "determinism": "int-rans/p16+int/i64" + } + }, + "target": null + }, + "expect": "form_required" + }, + { + "name": "unregistered_capability_form_required", + "manifest": { + "wai": "1.0", + "media": "image", + "intent": "improve", + "model_requirement": { + "capability": "wai.neural.future" + }, + "conditioning": { + "kind": "k", + "prior": { + "id": "conformance", + "sha256": "7ef44f3b1fc4a2df6c3e3b68460f595cae3d31cb7f9018699e29e738340272a4", + "determinism": "int-rans/p16+int/i64" + } + }, + "target": null + }, + "expect": "form_required" + }, + { + "name": "tier_exceeds_class", + "manifest": { + "wai": "1.0", + "media": "image", + "intent": "improve", + "model_requirement": { + "capability": "wai.image.jpegai" + }, + "conditioning": { + "kind": "k", + "prior": { + "id": "conformance", + "sha256": "7ef44f3b1fc4a2df6c3e3b68460f595cae3d31cb7f9018699e29e738340272a4", + "determinism": "int-rans/p16+int/i64", + "form": "wai.prior.set", + "tier": "decode-equivalence" + } + }, + "target": null + }, + "expect": "tier_exceeds_class" + }, + { + "name": "tier_at_class", + "manifest": { + "wai": "1.0", + "media": "image", + "intent": "improve", + "model_requirement": { + "capability": "wai.image.jpegai" + }, + "conditioning": { + "kind": "k", + "prior": { + "id": "conformance", + "sha256": "7ef44f3b1fc4a2df6c3e3b68460f595cae3d31cb7f9018699e29e738340272a4", + "determinism": "int-rans/p16+int/i64", + "form": "wai.prior.set", + "tier": "entropy-consistency" + } + }, + "target": null + }, + "expect": "ok" + }, + { + "name": "replicate_tier", + "manifest": { + "wai": "1.0", + "media": "image", + "intent": "replicate", + "model_requirement": { + "capability": "wai.neural.int_synth" + }, + "conditioning": { + "kind": "k", + "prior": { + "id": "conformance", + "sha256": "1c4705d17c01f865b20d785191c7f63b5c18dcc8fce77e15852cf8240b7368be", + "determinism": "int-rans/p16+int/i64", + "tier": "none" + } + }, + "target": null + }, + "expect": "replicate_tier" + }, + { + "name": "replicate_tier_legacy_float_words", + "manifest": { + "wai": "1.0", + "media": "image", + "intent": "replicate", + "model_requirement": { + "capability": "wai.neural.int_synth" + }, + "conditioning": { + "kind": "k", + "prior": { + "id": "conformance", + "sha256": "1c4705d17c01f865b20d785191c7f63b5c18dcc8fce77e15852cf8240b7368be", + "determinism": "int-rans/p16+onnx-f32" + } + }, + "target": null + }, + "expect": "replicate_tier" + }, + { + "name": "replicate_unpinned", + "manifest": { + "wai": "1.0", + "media": "image", + "intent": "replicate", + "model_requirement": { + "capability": "wai.neural.int_synth" + }, + "conditioning": { + "kind": "k" + }, + "target": null + }, + "expect": "replicate_unpinned" + }, + { + "name": "replicate_form_not_exact", + "manifest": { + "wai": "1.0", + "media": "image", + "intent": "replicate", + "model_requirement": { + "capability": "wai.neural.int_synth" + }, + "conditioning": { + "kind": "k", + "prior": { + "id": "conformance", + "sha256": "1c4705d17c01f865b20d785191c7f63b5c18dcc8fce77e15852cf8240b7368be", + "determinism": "int-rans/p16+int/i64", + "form": "wai.prior.nnc" + } + }, + "target": null + }, + "expect": "replicate_form_not_exact" + }, + { + "name": "replicate_set_form_sound", + "manifest": { + "wai": "1.0", + "media": "image", + "intent": "replicate", + "model_requirement": { + "capability": "wai.neural.int_mlicpp" + }, + "conditioning": { + "kind": "k", + "prior": { + "id": "conformance", + "sha256": "85a05b0dac15566c5a657ddeee253eaa9f54ce852ae04924f5b04151901d9d1b", + "determinism": "int-rans/p16+int/i64", + "form": "wai.prior.set", + "tier": "decode-equivalence" + } + }, + "target": null + }, + "expect": "ok" + }, + { + "name": "null_member_equals_absent", + "manifest": { + "wai": "1.0", + "media": "image", + "intent": "replicate", + "model_requirement": { + "capability": "wai.neural.int_mlicpp" + }, + "conditioning": { + "kind": "k", + "prior": { + "id": "conformance", + "sha256": "7ef44f3b1fc4a2df6c3e3b68460f595cae3d31cb7f9018699e29e738340272a4", + "determinism": "int-rans/p16+int/i64", + "form": null, + "tier": null, + "supersedes": null + } + }, + "target": null + }, + "expect": "ok" + }, + { + "name": "wai2_entry_index", + "manifest": { + "wai": "1.1", + "media": "image", + "intent": "replicate", + "renditions": [ + { + "capability": "wai.neural.int_mlicpp", + "kind": "int_mlicpp", + "prior": { + "id": "conformance", + "sha256": "c4232f2e2d5f76cce4ac782b254a91410a2ab89b22792f8356585549938a0e92", + "determinism": "int-rans/p16+int/i64", + "tier": "decode-equivalence", + "supersedes": { + "sha256": "29bc4a3984b2ca82ab4656e459e7c34b5af9f507309bcf88ed7b00561692ff0a" + } + } + }, + { + "capability": "wai.neural.int_mlicpp", + "kind": "int_mlicpp", + "prior": { + "id": "conformance", + "sha256": "29bc4a3984b2ca82ab4656e459e7c34b5af9f507309bcf88ed7b00561692ff0a", + "determinism": "int-rans/p16+int/i64", + "tier": "decode-equivalence" + } + }, + { + "capability": "wai.image.avif", + "kind": "avif" + } + ], + "target": null + }, + "expect": "ok" + }, + { + "name": "wai2_entry_finding", + "manifest": { + "wai": "1.1", + "media": "image", + "intent": "improve", + "renditions": [ + { + "capability": "wai.image.avif", + "kind": "avif" + }, + { + "capability": "wai.neural.glc", + "kind": "glc", + "prior": { + "id": "conformance", + "sha256": "7ef44f3b1fc4a2df6c3e3b68460f595cae3d31cb7f9018699e29e738340272a4", + "determinism": "int-rans/p16+int/i64" + } + } + ], + "target": null + }, + "expect": [ + 1, + "form_required" + ] + } + ] +} diff --git a/wai/research/2030-transport-landscape.md b/wai/research/2030-transport-landscape.md index 626737ef0..9ee743cee 100644 --- a/wai/research/2030-transport-landscape.md +++ b/wai/research/2030-transport-landscape.md @@ -865,22 +865,18 @@ analysis* — DeepCABAC entropy coding, pruning/sparsification/low-rank preprocessing, incremental-update support, with reference software in ISO/IEC 15938-18 and a separate open implementation (NNCodec) [name removed 2026-09-30]. The claim that needs no amendment at all is the carriage one: NNC is a ratified -wire format for the artifact a `Prior` names, which WAI currently leaves as an -informal directory. The signing feature is a second surface. It concerns how +wire format for the artifact a `Prior` names; for a set of several files, WAI pins +its listing (`wai.prior.set`, SPEC §3.1). The signing feature is a second surface. It concerns how NNC's in-band signatures relate to WAI's digest, which `extensions/prior-carriage.md` §6 fixes without depending on the amendment's details. -**What WAI already has, stated correctly.** WAI *does* pin model **identity** -normatively: the reference implementation's `Prior { id, sha256, determinism }` -is carried in the conditioning, hash-verified by the registry, with -`replicate`-soundness gated on the determinism contract. (SPEC.md's opening prose -— "never transported and never hash-pinned" — describes transport, not identity; -the two statements should be reconciled in the spec text.) What WAI has **no** -answer for is model/tensor **carriage**: by explicit design the weights are never -carried and decoders are sink-supplied, so the artifact whose `sha256` the -`Prior` names has no normative on-the-wire form — today it is an informal model -bundle directory. +**What WAI already has, stated correctly.** WAI pins model identity +normatively: SPEC §3.1 defines the pin, carried as `conditioning.prior` in a +single-payload manifest and on each rendition of a multi-rendition one, and +requires a sink to verify it before decoding and to refuse with a named reason. +What WAI does not specify is model carriage: the set a pin names travels outside +the envelope, in a form the pin MAY declare. **Three concrete surfaces.** (1) Name NNC as the standards-track normative form for the artifact a `Prior` identifies, so `Prior.sha256` covers a portable @@ -1054,7 +1050,7 @@ Items 1–2 are internal defects that need no external dependency at all. | 6 | **Key discovery**, plus DSC SEI verdict chaining with a precedence rule (§3.6, §6.1) | a trust-anchor mechanism WAI has none of; borrow DSCI's key-location design (certificate URI or ISO/IEC 21617-1 trust record) | ITU-T H.274 V4 (in force) | every WAI signature presumes the verifier already holds the key | | 7 | **JCR-1 alignment, then an RFC 9942 / RFC 9995 emission profile + SCRAPI registration** (§3.6) | move receipts onto the family's canonical envelope; emit the group receipt or checkpoints; decide BLAKE3 registration vs dual-hash | RFCs published; SCRAPI in RFC Editor queue; CCF profile in Last Call | non-equivocation, timestamping and suppression-detection are missing today | | 8 | **Derive acquisition classes from GREEN `data-source-accuracy`**, and declare tolerances (§3.5) | register `HwShunt`/`ModelBased`/`Estimator` as identities derived from the hierarchy; state measurement tolerances | adopted WG draft | the enum is private and coarser than an adopted draft, now | -| 9 | **NNC as the normative artifact form for a `Prior`**; register the general-tensor payload (§6.4) | reference the bitstream; one new capability; reconcile data-unit signing | CDAM stage — track, do not pin | `Prior.sha256` currently covers an informal directory | +| 9 | **NNC as the normative artifact form for a `Prior`**; register the general-tensor payload (§6.4) | reference the bitstream; one new capability; reconcile data-unit signing | CDAM stage — track, do not pin | a pin covers a set of files through its listing; NNC would give its networks a coded form | | 10 | **Sink-evaluated predicate capability + authority class + detached receipt profile** (§4.2, §7.1) | predicate-equivalence contract; priority field + sink-held trust anchor; store-and-forward receipt profile | deployed regional precedent (no dependency) | pre-emption is the highest-authority instruction and WAI names no signer class | | 11 | **NMOS constraint-set grammar over the capability registry** (§4.3) | a sink-side precondition on §4 dispatch; author register entries for the non-broadcast families | BCP-004-01/-02 v1.0.0 (ratified) | makes §7.1 rendition selection evaluable rather than policy-deferred | | 12 | **Register the Part 16 derivation operations with a per-operation determinism tier** (§3.1) | capability strings + tiers; do not adopt compositing | ISO/IEC 23001-16:2021 (published) | prevents restating a 2021 ratification as a WAI invention | diff --git a/wai/research/syfai-input.md b/wai/research/syfai-input.md index d65d4a793..7edbfbe89 100644 --- a/wai/research/syfai-input.md +++ b/wai/research/syfai-input.md @@ -130,10 +130,10 @@ the text catches up (G1). | framework topic | WAI mechanism | state | honest limit | |---|---|---|---| -| (i) AI parameter-data format | **One registered form, optional.** A `Prior` MAY declare that the artifact its digest covers is an ISO/IEC 15938-17:2024 (NNC) bitstream, `wai.prior.nnc`; the digest then covers the coded bitstream ([prior-carriage](../extensions/prior-carriage.md) §2–§3; link 05, PR #106). The model is still never transported in the envelope that decodes against it (SPEC.md's opening section). Three informative formats exist: the integer `int_synth.json` model in [neural-conformance](../neural-conformance/README.md); the JPEG AI weight bundle (`weights/*.json` + `norm.json`) read by `try_jpegai_exact`, whose hyper-decoder, context-model and synthesis networks are integer but whose eICCI network and one `norm.json` table are `f32`; and the one-file `WMB1` bundle of `wai.neural.int_mlicpp` — three integer networks and the entropy tables in one file (link 09, PR #111). | NNC form: specified, not implemented. Informative formats: implemented | `wai-rs` has no NNC reader or writer, and the reference sink refuses any prior that declares a form (`READABLE_PRIOR_FORMS` is empty). NNC is optional and not recommended: its royalty status, and whether conforming decoders reconstruct identical tensors, are not established (prior-carriage §4–§5). On `main` no form exists at all. | -| (ii) manifest | The reference implementation carries `conditioning.prior = { id, sha256, determinism }` in a single-payload (`WAI1`) manifest (`container.rs`, `Prior`), and at link 05 an optional `form`. A multi-rendition (`WAI2`) manifest carries the same `Prior` per rendition (`RenditionMeta.prior`). It names *which* model, exactly, and *how* its decode is made reproducible. prior-carriage §2 (a Draft extension) shows the object with its four members. | **implemented (reference implementation); shown in a Draft extension; not in SPEC.md §3** | SPEC.md §3's manifest schema defines `conditioning` as `{ "kind" }` only, and SPEC.md §7.1 defines no prior on a rendition entry (G1). The pin names identity, not location, and has no update or versioning semantics. Without a declared form, `sha256` covers **one file** (G2). `determinism` is free text (G3). The per-rendition pin is carried and never checked (G12). | +| (i) AI parameter-data format | **Registered forms, each optional.** A pin MAY declare that the set its digest covers is the listing of a set of files, `wai.prior.set` ([SPEC.md §3.1](../SPEC.md)), or an ISO/IEC 15938-17:2024 (NNC) bitstream, `wai.prior.nnc`; the digest then covers the listing or the coded bitstream ([prior-carriage](../extensions/prior-carriage.md) §2–§3). The model is still never transported in the envelope that decodes against it (SPEC.md's opening section). Informative formats exist: the integer `int_synth.json` model in [neural-conformance](../neural-conformance/README.md); the JPEG AI weight bundle (`weights/*.json` + `norm.json`) read by `try_jpegai_exact`, whose hyper-decoder, context-model and synthesis networks are integer but whose eICCI network and one `norm.json` table are `f32`; and the one-file `WMB1` bundle of `wai.neural.int_mlicpp` — its integer networks and the entropy tables in one file. | Listing form: specified, implemented, CI-checked. NNC form: specified, not implemented. Informative formats: implemented | `wai-rs` has no NNC reader or writer; the reference sink reads `wai.prior.set` and no coded form. NNC is optional and not recommended: its royalty status, and whether conforming decoders reconstruct identical tensors, are not established (prior-carriage §4–§5). | +| (ii) manifest | SPEC.md §3.1 defines the pin: `conditioning.prior = { id, sha256, determinism, form?, tier?, supersedes? }` in a single-payload (`WAI1`) manifest, and a rendition entry's `prior` in a multi-rendition (`WAI2`) one, beside the entry's `id` and payload `sha256` (§7.1). It names *which* parameter set, exactly — the SHA-256 of the whole set, a set of several files through its listing — and *how* its decode is made reproducible, with a tier from a closed vocabulary. | **specified (SPEC.md §3, §3.1, §7.1) and implemented** | The pin names identity, not location. Sinks of earlier revisions may ignore it (the pin then fails open) or reject the envelope (SPEC.md §3.1, §8). `id` is informative; `supersedes` is informative and never makes a successor satisfy its predecessor's pin. | | (iii) repository | **None.** SPEC.md §1 scopes WAI as a container and capability-dispatch standard; nothing in the tree specifies a model store or service. Identity-by-digest makes any store content-addressable, but WAI specifies no store. | out of scope | A repository would be SyfAI's. WAI's input is the identity rule a repository must honour, not the repository. | -| (iv) integrity checking | `ModelRegistry::resolve_checked` recomputes SHA-256 over the registered artifact and refuses to decode on mismatch (`DecodeError::PriorMismatch`); at link 05 it also refuses, before hashing, a declared form the sink cannot read. The `wai.neural.int_mlicpp` decoder re-checks a pinned prior against the very bytes it decodes with (link 09). `resolve_checked` is called only from the `WAI1` decode (`decode_envelope`), so no code path checks a `WAI2` rendition's pin (G12). `decode_envelope_strict` checks the capability's registry class and then the pinned prior before a `replicate` decode (link 02, PR #103). Because the manifest sits inside the envelope bytes the JWP receipt hashes ([jwp-receipts](../extensions/jwp-receipts.md) §2.1), a signed receipt authenticates the pin as well as the payload. | **implemented (reference implementation); partly specified** | prior-carriage §2 requires a sink to refuse a prior whose declared form it cannot read, "as it refuses a digest mismatch", but no normative text states the refusal on a digest mismatch itself (G1). SPEC.md §7 requires a sink to refuse a `replicate` claim over a `NeuralEntropyExact`, `NeuralFloat`, `GenerativeDeterministic` or `Behavioral` capability (on `main`: over a `NeuralFloat` one). A digest proves the bytes are the named bytes. It does not say who chose them. That comes from a signed JWP receipt over the envelope, and WAI has no key discovery (landscape §3.6). The lenient `decode_envelope` path decodes unpinned neural capabilities without checking; this is honest for perceptual use and is not a replicate path. | +| (iv) integrity checking | SPEC.md §3.1 requires a sink to verify a pin before it decodes, and to refuse one its parameter sets do not satisfy with one of an ordered set of named reasons; a refused pin makes the capability unsupported for that envelope, and dispatch continues at the fallback (§4) or the next rendition (§7.1). A pinned decode reads only the set it verified. The reference implementation does this for `WAI1` and `WAI2` (`Wai::dispatch_verified`, `WaiMulti::pick`, `pin::verify_bytes`, `neural::decode_envelope`, `neural::decode_multi`), in the C ABI and in the browser crate. `decode_envelope_strict` checks the capability's registry class and then the pin before a `replicate` decode. Because the manifest sits inside the envelope bytes the JWP receipt hashes ([jwp-receipts](../extensions/jwp-receipts.md) §2.1), a signed receipt authenticates the pin as well as the payload. | **specified (SPEC.md §3.1, §4, §7.1) and implemented; the pin vectors are checked on the conformance legs** | A digest proves the bytes are the named bytes. It does not say who chose them. That comes from a signed JWP receipt over the envelope, and WAI has no key discovery (landscape §3.6). The lenient `decode_envelope` path decodes an unpinned neural capability without checking; this is honest for perceptual use and is not a replicate path. | | (v) bit exactness + conformance | Two named tiers, `entropy-consistency` and `decode-equivalence`, and the rule that a tier covers what the sink **emits**, including post-decode processing ([determinism-tiers](../extensions/determinism-tiers.md) §2). Over a neural capability `replicate` promises `decode-equivalence`, which SPEC.md §7 admits only by an integer / fixed-point decode path with no float operation; `entropy-consistency` does not back it (link 02). A per-capability `ReplicateClass` in the registry: the `NeuralIntegerExact` capabilities at `decode-equivalence`, `wai.neural.int_mlicpp` among them (link 09), and `wai.image.jpegai` alone in `NeuralEntropyExact` at `entropy-consistency` (link 02). `codecs::DeterminismTier` carries the tiers in code (link 02). Frozen corpora of decoded-output digests. | tiers: specified, and a type in code at link 02 (on `main`, text only). Emitted-medium rule: **specified only**. Integer-entropy contract, `ReplicateClass` and the strict check: specified + implemented. Corpora and byte-equality goldens: implemented, and CI-checked on `main` for the capabilities listed under *CI scope*, on the legs that ran. | No manifest field names a tier; a capability's tier comes from its registration. `wai-rs` does not handle post-decode processing or operation-reduction hints. `wai.image.jpegai`'s exact decoder pins one reconstruction, reproducible only under its own build and floating-point conditions, and the reference sink can decode the same capability through a float path (G11). The float MLIC++ decoder agrees with its encoder only by convention between the two implementations; `wai.neural.int_mlicpp` makes that agreement a property of the arithmetic (*CI scope*). On `main`, a malformed stream's outcome depends on the build profile; link 01 fixes the known case (G6). Float capabilities have no corpus, deliberately. | **CI scope, precisely.** Two states are described. For `main`, the workflow @@ -383,6 +383,17 @@ party can regenerate. See [energy-exposure-mapping](energy-exposure-mapping.md). ## 4. Candidate requirements +*Status in this tree.* R1, R2, R3, R7 and R9 are now specified: R1 by SPEC.md §3.1 +(the pin covers the whole parameter set; several files through the listing form +`wai.prior.set`); R2 by §3.1's closed `tier` vocabulary; R3 by §3.1's ordered +refusal reasons and §4 / §7.1's dispatch rule; R7 by §7 *Digests* (SHA-256 for +parameter-set identity, component identity and the decoded output of capabilities +that decode against a parameter set, with `sha256.json` sidecars on the committed +vectors); R9 by `supersedes`, which names the predecessor and changes only the +reason a sink that holds it reports — predecessor content decodes exactly while the +sink still holds the predecessor. The entries below record the state they were +written against. + Drafted as an input would phrase them. After each one: what WAI does in the tree at link 10, or the gap. @@ -521,6 +532,13 @@ one of them. It does not need to prefer one route. ## 6. Gaps WAI must close before any of this is contributed +*Status in this tree.* G1 is closed by SPEC.md §3.1 (the pin is normative text, +for `WAI1` and per rendition); G2 by the parameter-set shapes of §5 and the +`wai.prior.set` listing; G3 by the closed `tier` vocabulary, with the tier-less +reading frozen as the legacy rule; G5 by `supersedes`; G7 by §7 *Digests*; G12 by +§4 / §7.1's dispatch rule and its implementation for `WAI2`. The entries below +record the state they were written against. + Each gap was found on `main`. Its status says what the tree at link 10 holds. **G1: The prior pin is in the reference implementation, not in the diff --git a/wai/tools/wai_int_mlicpp_export.py b/wai/tools/wai_int_mlicpp_export.py index 32fd1237d..737de6cfe 100644 --- a/wai/tools/wai_int_mlicpp_export.py +++ b/wai/tools/wai_int_mlicpp_export.py @@ -27,6 +27,8 @@ Writes into : stream.wai a WAI1 envelope over `wai.neural.int_mlicpp`, intent `replicate`, pinning model.wmb by SHA-256 as an integer-exact prior. Its payload is the `WIM1` codestream (see `wai-rs/src/int_mlicpp.rs`). + model.sha256, the set-form twin (tools/wai_int_mlicpp_set_form.py): the listing of + stream_set.wai the one-file set, and stream.wai pinned through it (`wai.prior.set`). expected.json what the decode must reproduce: the RGB hashes (FNV-1a and SHA-256), and FNV-1a over the y stream's bucket indices (u16 LE) and symbols (i32 LE) in coding order. @@ -469,6 +471,8 @@ def main(): with open(os.path.join(a.outdir, 'expected.json'), 'w') as f: json.dump(expected, f, indent=1) f.write('\n') + from wai_int_mlicpp_set_form import write_set_form + write_set_form(a.outdir) print(f'int_mlicpp vector: {W}x{H}, y {CY}x{enc["dims"][2]}x{enc["dims"][3]} in {K} slices, ' f'{len(enc["z_bits"])} z + {len(enc["y_bits"])} y rANS bytes, model {len(wmb)} B, ' f'rgb_fnv1a {expected["rgb_fnv1a"]}') diff --git a/wai/tools/wai_int_mlicpp_set_form.py b/wai/tools/wai_int_mlicpp_set_form.py new file mode 100644 index 000000000..3a675a569 --- /dev/null +++ b/wai/tools/wai_int_mlicpp_set_form.py @@ -0,0 +1,72 @@ +#!/usr/bin/env python3 +"""Write the set-form twin of the `wai.neural.int_mlicpp` conformance vector. + +The committed `stream.wai` pins `model.wmb` form-less: its pin's `sha256` is the +SHA-256 of the one file. SPEC.md §3.1 also lets a pin cover a parameter set through +its listing (the prior form `wai.prior.set`). This tool writes, next to the vector: + + model.sha256 the listing of the one-file set: the line + ` model.wmb` and a line feed -- the text + `sha256sum model.wmb` prints, which `sha256sum -c` checks. + stream_set.wai `stream.wai` with the same payload, its pin replaced by one over + the listing: `sha256` = SHA-256(model.sha256), `form` + `wai.prior.set`, `tier` `decode-equivalence`. + +Standard library only, and independent of the Rust reference implementation, which +checks these files in tests/int_mlicpp.rs and src/pin.rs. + + python tools/wai_int_mlicpp_set_form.py [vector-dir] + +`vector-dir` defaults to wai-rs/tests/vectors/int_mlicpp. `wai_int_mlicpp_export.py` +calls `write_set_form` after it writes the vector. +""" +import hashlib +import json +import os +import struct +import sys + + +def listing(files): + """The listing of `files` (path -> bytes): sorted by path as bytes, one + `<64 hex> \\n` line per file.""" + out = b'' + for path in sorted(files, key=lambda p: p.encode()): + out += hashlib.sha256(files[path]).hexdigest().encode() + b' ' + path.encode() + b'\n' + return out + + +def write_set_form(vector_dir): + with open(os.path.join(vector_dir, 'model.wmb'), 'rb') as f: + wmb = f.read() + with open(os.path.join(vector_dir, 'stream.wai'), 'rb') as f: + env = f.read() + assert env[:4] == b'WAI1' + ml = struct.unpack(' 1 else os.path.join(here, '..', 'wai-rs', 'tests', 'vectors', 'int_mlicpp') + lst, set_env = write_set_form(d) + print(f'model.sha256 {len(lst)} B sha256 {hashlib.sha256(lst).hexdigest()}; ' + f'stream_set.wai {len(set_env)} B sha256 {hashlib.sha256(set_env).hexdigest()}') + + +if __name__ == '__main__': + main() diff --git a/wai/vocab/README.md b/wai/vocab/README.md index d493e5e4d..2ca410f31 100644 --- a/wai/vocab/README.md +++ b/wai/vocab/README.md @@ -36,3 +36,14 @@ JPEG XL, FLAC, Opus, AV1, zstd, XZ) and model the neural and zeroth paths as classes; they are not the registry. v1's SHACL shape rejects a manifest that names any other registered capability, so a validator of current manifests uses v2. + +The parameter-set pin (`wai:ParameterSetPin`, SPEC §3.1) is in v1 as additive terms: +`wai:prior` and the members `wai:pinId`, `wai:pinSha256`, `wai:pinDeterminism`, +`wai:pinForm`, `wai:pinTier` (a closed vocabulary: `decode-equivalence`, +`entropy-consistency`, `none`) and `wai:pinSupersedes`, with a scoped JSON-LD term +`prior` whose members are the manifest's own names, and a `ParameterSetPinShape`. +`wai:sharedPriorUri` and `wai:sharedPriorHash` describe a located prior with a BLAKE3 +hash; they are not the manifest's pin. A library test in `wai-rs` +(`container::tests::vocab_pin_terms_match_prior_members`) holds the scoped term's +members equal to the pin's serialized members and the shape's tier list equal to the +registered tokens. diff --git a/wai/vocab/v1.jsonld b/wai/vocab/v1.jsonld index 559b5b373..5786655f6 100644 --- a/wai/vocab/v1.jsonld +++ b/wai/vocab/v1.jsonld @@ -17,6 +17,15 @@ "codec": { "@id": "wai:codec", "@type": "@id" }, "path": { "@id": "wai:path", "@type": "@vocab" }, "capabilities_required": { "@id": "wai:capabilitiesRequired", "@container": "@list" }, + "ParameterSetPin": "wai:ParameterSetPin", + "prior": { "@id": "wai:prior", "@context": { + "id": { "@id": "wai:pinId", "@type": "xsd:string" }, + "sha256": { "@id": "wai:pinSha256", "@type": "xsd:string" }, + "determinism": { "@id": "wai:pinDeterminism", "@type": "xsd:string" }, + "form": { "@id": "wai:pinForm", "@type": "xsd:string" }, + "tier": { "@id": "wai:pinTier" }, + "supersedes": { "@id": "wai:pinSupersedes" } + } }, "shared_prior_uri": { "@id": "wai:sharedPriorUri", "@type": "@id" }, "shared_prior_hash": { "@id": "wai:sharedPriorHash", "@type": "xsd:string" }, "content_hash": { "@id": "wai:contentHash", "@type": "xsd:string" }, diff --git a/wai/vocab/v1.shapes.ttl b/wai/vocab/v1.shapes.ttl index 20334bd46..8be021f1c 100644 --- a/wai/vocab/v1.shapes.ttl +++ b/wai/vocab/v1.shapes.ttl @@ -26,3 +26,25 @@ wai:NeuralPriorShape a sh:NodeShape ; sh:property [ sh:path wai:sharedPriorUri ; sh:nodeKind sh:IRI ; sh:minCount 1 ; sh:maxCount 1 ] ; sh:property [ sh:path wai:sharedPriorHash ; sh:datatype xsd:string ; sh:minCount 1 ; sh:maxCount 1 ; sh:pattern "^[a-f0-9]{64}$" ] . + +# The parameter-set pin (SPEC §3.1). The JSON-LD context types no `prior` object, +# so the shape targets whatever `wai:prior` points at rather than a class. +# `pinTier` is a closed vocabulary; the tokens are the ones `codecs::TIER_TOKENS` +# holds. +wai:ParameterSetPinShape a sh:NodeShape ; + sh:targetObjectsOf wai:prior ; + sh:property [ sh:path wai:pinId ; sh:datatype xsd:string ; sh:minCount 1 ; sh:maxCount 1 ] ; + sh:property [ sh:path wai:pinSha256 ; sh:datatype xsd:string ; sh:minCount 1 ; sh:maxCount 1 ; + sh:pattern "^[a-f0-9]{64}$" ] ; + sh:property [ sh:path wai:pinDeterminism ; sh:datatype xsd:string ; sh:minCount 1 ; sh:maxCount 1 ] ; + sh:property [ sh:path wai:pinForm ; sh:datatype xsd:string ; sh:maxCount 1 ] ; + sh:property [ sh:path wai:pinTier ; sh:datatype xsd:string ; sh:maxCount 1 ; + sh:in ( "decode-equivalence" "entropy-consistency" "none" ) ] ; + sh:property [ sh:path wai:pinSupersedes ; sh:node wai:SupersedesShape ; sh:maxCount 1 ] . + +# `supersedes` names a set as a pin names one, by `sha256` and, if it is pinned in a +# declared form, `form`; it has no `id`, `determinism` or `tier`. +wai:SupersedesShape a sh:NodeShape ; + sh:property [ sh:path wai:pinSha256 ; sh:datatype xsd:string ; sh:minCount 1 ; sh:maxCount 1 ; + sh:pattern "^[a-f0-9]{64}$" ] ; + sh:property [ sh:path wai:pinForm ; sh:datatype xsd:string ; sh:maxCount 1 ] . diff --git a/wai/vocab/v1.ttl b/wai/vocab/v1.ttl index 15fd1692b..a16ba97f0 100644 --- a/wai/vocab/v1.ttl +++ b/wai/vocab/v1.ttl @@ -58,8 +58,25 @@ wai:modality a owl:ObjectProperty ; rdfs:domain wai:Manifest ; rd wai:codec a owl:ObjectProperty ; rdfs:domain wai:Manifest ; rdfs:range wai:Codec ; rdfs:label "Selected codec" . wai:path a owl:ObjectProperty ; rdfs:domain wai:Manifest ; rdfs:range wai:Path ; rdfs:label "Decode path" . wai:capabilitiesRequired a owl:ObjectProperty ; rdfs:domain wai:Manifest ; rdfs:range wai:Capability ; rdfs:label "Required capabilities" . -wai:sharedPriorUri a owl:ObjectProperty ; rdfs:domain wai:NeuralPrior ; rdfs:label "Shared-prior URI" . -wai:sharedPriorHash a owl:DatatypeProperty ; rdfs:domain wai:NeuralPrior ; rdfs:range xsd:string ; rdfs:label "Shared-prior content hash (BLAKE3 hex)" . +wai:sharedPriorUri a owl:ObjectProperty ; rdfs:domain wai:NeuralPrior ; rdfs:label "Shared-prior URI" ; + rdfs:comment "Describes a located prior with a BLAKE3 hash; not the manifest's pin (SPEC §3.1), which is a SHA-256 over the whole parameter set and carries no location." . +wai:sharedPriorHash a owl:DatatypeProperty ; rdfs:domain wai:NeuralPrior ; rdfs:range xsd:string ; rdfs:label "Shared-prior content hash (BLAKE3 hex)" ; + rdfs:comment "Describes a located prior with a BLAKE3 hash; not the manifest's pin (SPEC §3.1), which is a SHA-256 over the whole parameter set and carries no location." . + +# ─── The parameter-set pin (SPEC §3.1) ─────────────────────────────── + +wai:ParameterSetPin a owl:Class ; rdfs:label "Parameter-set pin" ; + rdfs:comment "The identity of the parameter set a payload was coded against: the SHA-256 of the whole set in its declared form, and how the decode is made reproducible (SPEC §3.1)." . +wai:prior a owl:ObjectProperty ; rdfs:range wai:ParameterSetPin ; rdfs:label "Parameter-set pin" . +wai:pinId a owl:DatatypeProperty ; rdfs:domain wai:ParameterSetPin ; rdfs:range xsd:string ; rdfs:label "Pin name (informative)" . +wai:pinSha256 a owl:DatatypeProperty ; rdfs:range xsd:string ; rdfs:label "SHA-256 of the parameter set in its form (64 lowercase hex)" ; + rdfs:comment "A member of a pin and of its supersedes object." . +wai:pinDeterminism a owl:DatatypeProperty ; rdfs:domain wai:ParameterSetPin ; rdfs:range xsd:string ; rdfs:label "How the decode is made reproducible" . +wai:pinForm a owl:DatatypeProperty ; rdfs:range xsd:string ; rdfs:label "Registered prior form" ; + rdfs:comment "A member of a pin and of its supersedes object." . +wai:pinTier a owl:DatatypeProperty ; rdfs:domain wai:ParameterSetPin ; rdfs:range xsd:string ; rdfs:label "Declared reproducibility tier" . +wai:pinSupersedes a owl:ObjectProperty ; rdfs:domain wai:ParameterSetPin ; rdfs:label "The parameter set this one replaces (informative)" ; + rdfs:comment "Names that set by pinSha256 and, if it is pinned in a declared form, pinForm; it is not itself a pin." . wai:contentHash a owl:DatatypeProperty ; rdfs:range xsd:string ; rdfs:label "Content hash (BLAKE3 hex)" . wai:size a owl:DatatypeProperty ; rdfs:range xsd:nonNegativeInteger ; rdfs:label "Byte size" . wai:mediaType a owl:DatatypeProperty ; rdfs:range xsd:string ; rdfs:label "IANA media type" . diff --git a/wai/wai-quantum-serve/Cargo.lock b/wai/wai-quantum-serve/Cargo.lock index 80d6943e6..00aafc33e 100644 --- a/wai/wai-quantum-serve/Cargo.lock +++ b/wai/wai-quantum-serve/Cargo.lock @@ -583,6 +583,7 @@ dependencies = [ "pkg-config", "serde", "serde_json", + "sha2", "wai-quantum", ] diff --git a/wai/wai-relay/Cargo.lock b/wai/wai-relay/Cargo.lock index fedead0ec..846c6166d 100644 --- a/wai/wai-relay/Cargo.lock +++ b/wai/wai-relay/Cargo.lock @@ -1085,6 +1085,7 @@ dependencies = [ "pkg-config", "serde", "serde_json", + "sha2", ] [[package]] diff --git a/wai/wai-rs/Cargo.toml b/wai/wai-rs/Cargo.toml index a3e8424e7..00bd62858 100644 --- a/wai/wai-rs/Cargo.toml +++ b/wai/wai-rs/Cargo.toml @@ -66,7 +66,10 @@ byteorder = { version = "1", optional = true } ort = { version = "2.0.0-rc.12", default-features = false, features = ["std", "download-binaries", "ndarray", "tls-rustls"], optional = true } ndarray = { version = "0.17", optional = true } zstd-safe = { version = "7", optional = true, features = ["std"] } # for bmshj2018 latent unpack (std → WriteBuf for Vec) -sha2 = { version = "0.10", optional = true } # verify a pinned neural prior against the registered model artifact +# SHA-256 is WAI's digest for parameter-set identity, component identity and decoded-output +# conformance digests (SPEC §3.1, §7 *Digests*, §7.1). Pure Rust, so it is always on: the +# envelope reader verifies a component digest and a pin with no feature enabled. +sha2 = "0.10" # ---- real C2PA manifest emission (optional, behind `c2pa_emit`) ---- # Writes an actual signed manifest carrying the energy + reconstruction @@ -103,7 +106,12 @@ codecs = [ "dep:opus", "dep:ogg", "dep:claxon", "dep:flac-bound", "dep:rav1e", "dep:dav1d", "dep:zstd", "dep:xz2", "dep:byteorder", + "ffi", ] +# The C ABI's envelope surface (buffers, errors, WAI1/WAI2 pack and unpack, §7.1 +# selection), with no codec library: it builds and is tested where libjxl and the +# other C codec libraries are absent. `codecs` adds the codec wrappers to it. +ffi = [] # Pulls in the ONNX runtime + zstd unpack so the native sink can # decode `wai.neural.<...>` envelopes. Off by default — the bare # container + zeroth codecs build with no ML dep chain. @@ -135,7 +143,7 @@ world_int = ["neural_int"] # learned transform over the [14,S,S] attribute image → byte-exact attribute tensor # (splat-equivalence, a portable cloud hash). Reuses int_hyper_synth (neural_int). splat_int = ["neural_int"] -neural = ["neural_int", "dep:ort", "dep:ndarray", "dep:zstd-safe", "dep:sha2"] +neural = ["neural_int", "dep:ort", "dep:ndarray", "dep:zstd-safe"] # The perceptual / generative-semantic path: CLIP-similarity conformance, # regenerate-at-sink. Reuses the ONNX runtime for the reference CLIP embedder; # the generator is always sink-supplied. Off by default like `neural`. @@ -162,12 +170,12 @@ score = ["world"] # Asset provenance (extensions/provenance): a signed, energy-accounted # lineage receipt for opaque captured content WAI does not reconstruct. # Reuses the JWP Merkle + Ed25519; no codec/ML deps. -provenance = ["world", "dep:sha2"] +provenance = ["world"] # OER/2 outcome-and-energy receipts (extensions/oer-receipts.md): a verifier for # receipts that grade what a unit of work cost (projected/simulated/derived/measured), # content-address and chain it, and sign it. Conformance is reason-set equality with # the reference verifier and the portable C kernel over oer-conformance/. -oer = ["dep:sha2", "dep:blake3", "dep:ed25519-dalek", "dep:num-bigint", "serde_json/float_roundtrip"] +oer = ["dep:blake3", "dep:ed25519-dalek", "dep:num-bigint", "serde_json/float_roundtrip"] # Typed receipt classes on the JWP chain (extensions/jwp-receipts.md §2.4.3–§2.4.7, # §2.6, §3.1–§3.2): `claim` receipts of class energy / lawful-basis / log-inclusion, # the soft-binding class, and recovery under the ancestor rule. Needs `provenance` @@ -503,6 +511,10 @@ name = "wai_oer_diff" path = "src/bin/wai_oer_diff.rs" required-features = ["oer"] +[[bin]] +name = "wai_pin_vectors" +path = "src/bin/wai_pin_vectors.rs" + [[bin]] name = "wai_claim_vectors" path = "src/bin/wai_claim_vectors.rs" diff --git a/wai/wai-rs/build.rs b/wai/wai-rs/build.rs index ae84b4b8e..2ea5f8896 100644 --- a/wai/wai-rs/build.rs +++ b/wai/wai-rs/build.rs @@ -27,6 +27,8 @@ fn main() { /// via `rerun-if-changed` declarations). fn generate_c_header() { println!("cargo:rerun-if-changed=src/ffi.rs"); + println!("cargo:rerun-if-changed=src/ffi_codecs.rs"); + println!("cargo:rerun-if-changed=src/ffi_neural.rs"); println!("cargo:rerun-if-changed=src/container.rs"); println!("cargo:rerun-if-changed=cbindgen.toml"); let crate_dir = std::env::var("CARGO_MANIFEST_DIR").unwrap(); diff --git a/wai/wai-rs/cbindgen.toml b/wai/wai-rs/cbindgen.toml index 98d62450e..c44d98c9e 100644 --- a/wai/wai-rs/cbindgen.toml +++ b/wai/wai-rs/cbindgen.toml @@ -40,6 +40,7 @@ prefix = "" # internal helpers out of the public C surface. [export.rename] "WaiBuffer" = "wai_buffer_t" +"WaiDecoded" = "wai_decoded_t" [fn] prefix = "" diff --git a/wai/wai-rs/include/wai.h b/wai/wai-rs/include/wai.h index dc14964de..d18ac8262 100644 --- a/wai/wai-rs/include/wai.h +++ b/wai/wai-rs/include/wai.h @@ -40,6 +40,21 @@ #define WAI_ERR_PANIC 4 +/** + * A pin was refused (SPEC §3.1); `wai_last_error` starts with the reason. + */ +#define WAI_ERR_PRIOR 5 + +/** + * `intent = replicate` is not sound for the capability decoded through (SPEC §7). + */ +#define WAI_ERR_UNSOUND_REPLICATE 6 + +/** + * No rendition of a `WAI2` envelope can be picked (SPEC §7.1). + */ +#define WAI_ERR_NO_RENDITION 7 + /** * Probability output scale of [`softmax`]: results are at `2^PROB_FRAC` * (they sum to ~`2^PROB_FRAC`). @@ -81,6 +96,26 @@ #define MARKER_UDI 65420 +/** + * The media kind of a [`WaiDecoded`]. + */ +#define WAI_DECODED_AUDIO 1 + +/** + * The media kind of a [`WaiDecoded`]. + */ +#define WAI_DECODED_IMAGE 2 + +/** + * The media kind of a [`WaiDecoded`]. + */ +#define WAI_DECODED_VIDEO 3 + +/** + * `wai_neural_decode_ex` flag: apply SPEC §7's replicate rule (strict). + */ +#define WAI_DECODE_STRICT 1 + #define SECTION_CONTRACT 1 #define SECTION_GRAPH 2 @@ -310,6 +345,29 @@ typedef struct wai_buffer_t { uintptr_t cap; } wai_buffer_t; +/** + * What `wai_neural_decode_ex` decoded. + * + * `kind` is `WAI_DECODED_AUDIO` (`data` holds `f32` samples, `sample_rate` set), + * `WAI_DECODED_IMAGE` (`data` holds RGB8, `width` and `height` set) or + * `WAI_DECODED_VIDEO` (`data` holds the frames' RGB8 concatenated, `width`, + * `height`, `n_frames` and `fps_x_1000` set). `index` is the `WAI2` rendition + * picked (0 for a `WAI1` envelope). Free `data` with `wai_buffer_free`. + * `wai_neural_decode_ex` writes the whole struct on every call that passes a + * non-null `out`: on failure `kind` is 0 and `data` is empty, so `data` is always + * safe to free. + */ +typedef struct wai_decoded_t { + uint32_t kind; + uint32_t index; + struct wai_buffer_t data; + uint32_t width; + uint32_t height; + uint32_t sample_rate; + uint32_t n_frames; + uint32_t fps_x_1000; +} wai_decoded_t; + @@ -339,6 +397,105 @@ extern "C" { */ const char *wai_last_error(void); +/** + * Builds a complete WAI file from a codec-produced payload + manifest + * fields. `manifest_json` is a UTF-8 JSON object overriding/supplying + * any fields; `payload` is the bytes from one of the codec encoders. + * Refuses (`WAI_ERR_INVALID_ARG`) a manifest that names a candidate capability + * as its capability or fallback, which an encoder MUST NOT emit (SPEC §5, + * *Candidate capabilities*): it emits through `Wai::to_emit_bytes`. + */ + +int wai_envelope_pack(const char *manifest_json, + const uint8_t *payload, + uintptr_t payload_len, + struct wai_buffer_t *out); + +/** + * Detect WAI version from a byte stream. Returns 1 (WAI1), 2 (WAI2), + * or 0 with an error message in `wai_last_error` for unknown magic. + */ + int wai_envelope_detect_version(const uint8_t *bytes, uintptr_t len); + +/** + * Pack a WAI2 multi-rendition envelope. + * + * - `manifest_json` must declare a `renditions` array of length `n`. + * - `payload_data` is an array of `n` byte-buffer pointers. + * - `payload_lens` is the matching array of `n` lengths. + * - The i-th payload pairs with `manifest.renditions[i]`. + * + * Refuses (`WAI_ERR_INVALID_ARG`) a table with a component, in either role, + * whose capability is a candidate (SPEC §5, *Candidate capabilities*): it emits + * through `WaiMulti::to_emit_bytes`. + * + * # Safety + * All array pointers must point to valid memory of the declared length. + */ + +int wai_envelope_pack_multi(const char *manifest_json, + const uint8_t *const *payload_data, + const uintptr_t *payload_lens, + uintptr_t n_payloads, + struct wai_buffer_t *out); + +/** + * Parse a WAI2 envelope. Writes three outputs: + * + * - `out_manifest` — the manifest JSON exactly as the envelope carries it + * (UTF-8 bytes, no NUL), so members this library does not model survive + * - `out_renditions_json` — a JSON array with one object per component: + * `capability`, `kind`, `offset` (into `out_payload_block`), `length`, and + * `selectable` (whether SPEC §7.1 may pick it: a companion, or a companion-only + * capability, never is), plus `id`, `sha256`, `prior`, `role`, `companion_of` + * and `companion_of_id` when the entry carries them. A C sink selects with + * `wai_envelope_select` rather than by hand. + * - `out_payload_block` — every rendition's payload bytes, packed + * contiguously. The caller indexes via the JSON above. + * + * This avoids exposing a C struct array (and the ownership pain that + * comes with it). Any JSON parser the consumer already has decodes + * the rendition table in two lines. + */ + +int wai_envelope_unpack_multi(const uint8_t *bytes, + uintptr_t len, + struct wai_buffer_t *out_manifest, + struct wai_buffer_t *out_renditions_json, + struct wai_buffer_t *out_payload_block); + +/** + * Parses a WAI envelope. Writes the manifest JSON exactly as the envelope + * carries it to `out_manifest` (so `security`, and members this library does not + * model, survive) and the raw payload to `out_payload`. + */ + +int wai_envelope_unpack(const uint8_t *bytes, + uintptr_t len, + struct wai_buffer_t *out_manifest, + struct wai_buffer_t *out_payload); + +/** + * SPEC §7.1 selection for a `WAI2` envelope at a sink that advertises the + * capabilities in `advertised_json` (a NUL-terminated JSON array of capability + * strings). Writes the index of the picked rendition to `out_index`: the first + * entry that is selectable (never a companion or a companion-only capability), + * advertised, and whose payload matches its `sha256`. This call holds no + * parameter set, so an entry whose pin applies is skipped (`prior_not_held`); a + * sink that holds parameter sets decodes through `wai_neural_decode_ex`, which + * verifies each pin. Returns `WAI_ERR_NO_RENDITION` when nothing is picked, with + * each entry's index and reason in `wai_last_error`. + * + * # Safety + * `envelope` valid for `envelope_len` bytes; `advertised_json` NUL-terminated; + * `out_index` writable. + */ + +int wai_envelope_select(const uint8_t *envelope, + uintptr_t envelope_len, + const char *advertised_json, + uint32_t *out_index); + /** * Encode RGB pixels (row-major, 3 bytes per pixel) as PNG. * @@ -466,87 +623,14 @@ int wai_video_av1_decode(const uint8_t *bytes, uint32_t *out_fps_num, uint32_t *out_fps_den); -/** - * Builds a complete WAI file from a codec-produced payload + manifest - * fields. `manifest_json` is a UTF-8 JSON object overriding/supplying - * any fields; `payload` is the bytes from one of the codec encoders. - * Refuses (`WAI_ERR_INVALID_ARG`) a manifest that names a candidate capability - * as its capability or fallback, which an encoder MUST NOT emit (SPEC §5, - * *Candidate capabilities*): it emits through `Wai::to_emit_bytes`. - */ - -int wai_envelope_pack(const char *manifest_json, - const uint8_t *payload, - uintptr_t payload_len, - struct wai_buffer_t *out); - -/** - * Detect WAI version from a byte stream. Returns 1 (WAI1), 2 (WAI2), - * or 0 with an error message in `wai_last_error` for unknown magic. - */ - int wai_envelope_detect_version(const uint8_t *bytes, uintptr_t len); - -/** - * Pack a WAI2 multi-rendition envelope. - * - * - `manifest_json` must declare a `renditions` array of length `n`. - * - `payload_data` is an array of `n` byte-buffer pointers. - * - `payload_lens` is the matching array of `n` lengths. - * - The i-th payload pairs with `manifest.renditions[i]`. - * - * Refuses (`WAI_ERR_INVALID_ARG`) a table with a component, in either role, - * whose capability is a candidate (SPEC §5, *Candidate capabilities*): it emits - * through `WaiMulti::to_emit_bytes`. - * - * # Safety - * All array pointers must point to valid memory of the declared length. - */ - -int wai_envelope_pack_multi(const char *manifest_json, - const uint8_t *const *payload_data, - const uintptr_t *payload_lens, - uintptr_t n_payloads, - struct wai_buffer_t *out); - -/** - * Parse a WAI2 envelope. Writes three outputs: - * - * - `out_manifest` — the original manifest JSON (UTF-8 bytes, no NUL) - * - `out_renditions_json` — a JSON array of `{capability, kind, offset, - * length}` entries. `offset` is into `out_payload_block`, `length` - * is the rendition's payload byte count. - * - `out_payload_block` — every rendition's payload bytes, packed - * contiguously. The caller indexes via the JSON above. - * - * This avoids exposing a C struct array (and the ownership pain that - * comes with it). Any JSON parser the consumer already has decodes - * the rendition table in two lines. - */ - -int wai_envelope_unpack_multi(const uint8_t *bytes, - uintptr_t len, - struct wai_buffer_t *out_manifest, - struct wai_buffer_t *out_renditions_json, - struct wai_buffer_t *out_payload_block); - -/** - * Parses a WAI envelope. Writes the manifest as JSON to `out_manifest` - * and the raw payload to `out_payload`. - */ - -int wai_envelope_unpack(const uint8_t *bytes, - uintptr_t len, - struct wai_buffer_t *out_manifest, - struct wai_buffer_t *out_payload); - /** * Decode a `wai.neural.