Transaction-Science/open-standards
Watch
1
0
Fork
You've already forked open-standards
 0
Code Issues Pull requests Projects Releases 17 Packages Wiki Activity Actions 60

wai: the parameter-set pin, component identity, pin-aware selection #132

Merged
dcharlot merged 13 commits from wai/land-13-parameter-set-identity into main 2026-10-01 14:50:19 -04:00
Conversation 0 Commits 13 Files changed 121 +9903 -2139
121 changed files with 9903 additions and 2139 deletions
Download patch file Download diff file Expand all files Collapse all files

47
.github/workflows/wai-conformance.yml vendored
View file

@ -24,12 +24,13 @@ on:
# The registry export and the two library tests that read files outside
# wai-rs (the codec glossary, SPEC.md) must fire this workflow too: a
# hand edit of an exported file, of SPEC §5 or of the glossary is checked
# here.
# here. vocab/v1's and v2's pin terms are checked by a library test.
- 'wai/registry/**'
- 'wai/vocab/**'
- 'wai/glossary/**'
- 'wai/SPEC.md'
- 'wai/REGISTRY.md'
- 'wai/wai-web/**'
- '.github/workflows/wai-conformance.yml'
pull_request:
paths:
@ -42,12 +43,13 @@ on:
# The registry export and the two library tests that read files outside
# wai-rs (the codec glossary, SPEC.md) must fire this workflow too: a
# hand edit of an exported file, of SPEC §5 or of the glossary is checked
# here.
# here. vocab/v1's and v2's pin terms are checked by a library test.
- 'wai/registry/**'
- 'wai/vocab/**'
- 'wai/glossary/**'
- 'wai/SPEC.md'
- 'wai/REGISTRY.md'
- 'wai/wai-web/**'
- '.github/workflows/wai-conformance.yml'
# No global `shell:` default. The Windows runner is a host executor whose
@ -151,19 +153,27 @@ jobs:
# export, and check each vector's metadata. Cargo.toml declares them, and the other
# feature-gated targets named here, with required-features, so if this step ever loses a
# feature, cargo stops with an error instead of compiling an empty binary that reports ok.
# Each committed vector's decoded output is also checked against the SHA-256 in
# its sha256.json sidecar (SPEC §7 *Digests*), and
# int_kernels_vectors::every_decoded_output_states_a_sha256 requires a sidecar,
# or a named exclusion, for every corpus directory. tests/generate.rs checks the
# wai.generate.spectral vector, which `generate` enables.
- name: Fast integer kernels ≡ reference kernels (overflow checks on)
working-directory: wai/wai-rs
run: cargo test --no-default-features --features neural_int,video_int,audio_int,world_int,splat_int --lib --test int_kernels_vectors --test int_kernels_geometry --test int_mlicpp --test int_mlicpp_preflight --test int_audio --test int_splat
run: cargo test --no-default-features --features neural_int,video_int,audio_int,world_int,splat_int,generate --lib --test int_kernels_vectors --test int_kernels_geometry --test int_mlicpp --test int_mlicpp_preflight --test int_audio --test int_splat --test generate
- name: Fast integer kernels ≡ reference kernels (release, overflow checks off)
working-directory: wai/wai-rs
run: cargo test --release --no-default-features --features neural_int,video_int,audio_int,world_int,splat_int --lib --test int_kernels_vectors --test int_kernels_geometry --test int_mlicpp --test int_mlicpp_preflight --test int_audio --test int_splat
run: cargo test --release --no-default-features --features neural_int,video_int,audio_int,world_int,splat_int,generate --lib --test int_kernels_vectors --test int_kernels_geometry --test int_mlicpp --test int_mlicpp_preflight --test int_audio --test int_splat --test generate
# carriage: every registered capability round-trips byte-exact. cli_wrap_intent and
# cli_registry run the `wai` binary, which builds without the codecs feature: the
# intent rule of `wai wrap`, its refusal of candidate capabilities (SPEC §5), its media
# class, and `wai registry export --check` on the tree and on a drifted copy.
# tests/manifest_builders.rs: the manifest types are #[non_exhaustive], and the
# builders write the JSON a struct literal wrote; members the types keep only
# as `extra` (and `security`) survive a round trip.
- name: Carriage conformance and the wai CLI (wrap, registry export)
working-directory: wai/wai-rs
run: cargo test --no-default-features --test carriage --test cli_wrap_intent --test cli_registry
run: cargo test --no-default-features --test carriage --test manifest_builders --test cli_wrap_intent --test cli_registry
# The machine-readable registry (wai/registry/, wai/vocab/v2.*, the table in
# wai/REGISTRY.md) is generated from wai-rs/src/codecs/registry.rs. Regenerate it in
# memory and fail on any difference; the message says how to regenerate. One leg: the
@ -172,6 +182,23 @@ jobs:
if: matrix.os == 'linux'
working-directory: wai/wai-rs
run: cargo run -q --no-default-features --bin wai -- registry export --check
# The C ABI's envelope surface builds with no codec library (feature `ffi`):
# unpack returns each manifest as the envelope carries it, the WAI2 table carries
# every component's identity and selectability, wai_envelope_select follows
# SPEC §7.1 and never picks a candidate, and the packers refuse a candidate
# (SPEC §5). Cargo.toml declares the target with required-features.
- name: C ABI envelope surface (no codec library)
working-directory: wai/wai-rs
run: cargo test --no-default-features --features ffi --test ffi_envelope
# The browser crate wraps the same envelope types (wai::container), so role,
# companion_of, component identity and the pin reach JS, and its pick is SPEC
# §7.1's. Its tests target the pure-Rust half (`core`), which runs natively; one
# leg suffices, the logic is architecture-independent. The JS surface itself runs
# under wasm32 in wai-web-wasm.yml.
- name: wai-web envelope surface (native tests)
if: matrix.os == 'linux'
working-directory: wai/wai-web
run: cargo test --locked
# The corpus-verify loop is genuinely shell-specific (a helper that groups
# log output and propagates the first non-zero exit), so it ships once per
@ -197,6 +224,15 @@ jobs:
v quantum-ops wai_quantum_ops_vectors ../quantum-ops-conformance
v oer wai_oer_vectors ../oer-conformance
v claims wai_claim_vectors ../claim-conformance
v pin wai_pin_vectors ../pin-conformance
# The pin corpus's second implementation: Python standard library only, sharing no
# code with wai-rs. One leg is enough — it checks digests and listing rules, which
# do not depend on the architecture — and the macOS host has python3.
- name: The pin corpus checks against an independent verifier (macOS)
if: matrix.os == 'macos'
shell: bash
working-directory: wai
run: python3 pin-conformance/pin_verify.py pin-conformance
# OER's second implementation is portable C, not Rust. Rebuild it from source and
# require its output to match the pinned corpus byte for byte: the kernel is
# deterministic, and wai_oer_vectors above has already verified the pinned copy,
@ -237,6 +273,7 @@ jobs:
v quantum-ops wai_quantum_ops_vectors ../quantum-ops-conformance
v oer wai_oer_vectors ../oer-conformance
v claims wai_claim_vectors ../claim-conformance
v pin wai_pin_vectors ../pin-conformance
# The generated v2 vocabulary (wai/vocab/v2.*) is checked as a consumer uses it:
# sample manifests, typed and untyped, WAI1 and WAI2, are expanded through the

10
.github/workflows/wai-jpegai-e2e.yml vendored
View file

@ -68,6 +68,10 @@ on:
paths:
- 'wai/wai-rs/src/jpegai_*.rs'
- 'wai/wai-rs/src/reconstruction_binding.rs'
# The set-form pin over the exact bundle (tests/jpegai_e2e_full.rs) verifies
# through the pin module.
- 'wai/wai-rs/src/pin.rs'
- 'wai/wai-rs/src/container.rs'
- 'wai/wai-rs/src/int_transformer.rs'
- 'wai/wai-rs/src/int_hyper_synth.rs'
- 'wai/wai-rs/src/int_kernels.rs'
@ -89,6 +93,10 @@ on:
paths:
- 'wai/wai-rs/src/jpegai_*.rs'
- 'wai/wai-rs/src/reconstruction_binding.rs'
# The set-form pin over the exact bundle (tests/jpegai_e2e_full.rs) verifies
# through the pin module.
- 'wai/wai-rs/src/pin.rs'
- 'wai/wai-rs/src/container.rs'
- 'wai/wai-rs/src/int_transformer.rs'
- 'wai/wai-rs/src/int_hyper_synth.rs'
- 'wai/wai-rs/src/int_kernels.rs'
@ -157,7 +165,7 @@ jobs:
# A test that was renamed or compiled out would not fail — it would just
# not run. Require each one by name, and the pinned-hash check by its line.
for t in codestream_to_rgb_matches_reference_png real_decode_to_reconstruction_binding \
decode_matches_pinned_hashes; do
decode_matches_pinned_hashes jpegai_e2e_set_pin_verifies; do
grep -q "^test $t \.\.\. ok$" "$log" || { echo "::error::$t did not run and pass"; exit 1; }
done
grep -q "matches the pinned WAI_JPEGAI_EXPECT_SHA256" "$log" \

72
.github/workflows/wai-neural-gates.yml vendored
View file

@ -1,8 +1,8 @@
name: WAI neural sink gates
# The neural sink refuses some envelopes before it touches a model: a pinned
# prior whose hash does not match the registered model, or that declares a form
# the sink cannot read (ModelRegistry::resolve_checked), and a `replicate` that
# The neural sink refuses some envelopes before it touches a model: a pin that
# the parameter sets it holds do not satisfy, with SPEC §3.1's reason
# (ModelRegistry::resolve_verified, decode_envelope, decode_multi), and a `replicate` that
# SPEC §7 does not back (decode_envelope_strict): over a capability whose registry
# class cannot back it, or with a prior that is float or absent, or that declares a
# form not established to reconstruct the same tensors at every reader. Those
@ -12,10 +12,15 @@ name: WAI neural sink gates
# them. This job runs exactly those tests and fails if any of them is not run:
# a filter that matches nothing reports "0 passed" and exits 0.
#
# It also runs the one `neural` dispatch test that needs no model download:
# `wai.neural.int_mlicpp` through the strict gate on its committed conformance vector
# (the pinned image; a one-byte-different model bundle refused by the prior pin;
# `replicate` without a prior refused).
# It also runs the `neural` dispatch tests that need no model download, over the
# committed int_mlicpp vector and the int_synth corpus: the form-less and set-form
# pins through the strict gate, the int_synth pin over the spec it reads, the
# several-file capabilities refusing a form-less pin, a pinned decode reading only
# the bytes it verified, a WAI2 envelope whose pins pick the held model, the strict
# rule read for the capability decoded through (a sound fallback behind an unsound
# primary), SPEC §4 step 6's order of reasons after a set_incomplete decode, and the
# replicate_* reason the strict WAI2 policy reports for an entry it leaves out,
# and a candidate held at the sink leaving the strict path's reason unchanged.
on:
push:
@ -25,6 +30,14 @@ on:
- 'wai/wai-rs/src/container.rs'
- 'wai/wai-rs/src/codecs/mod.rs'
- 'wai/wai-rs/src/codecs/prior_form.rs'
- 'wai/wai-rs/src/pin.rs'
- 'wai/wai-rs/src/ffi.rs'
- 'wai/wai-rs/src/ffi_neural.rs'
- 'wai/wai-rs/tests/ffi_neural_pin.rs'
- 'wai/wai-rs/src/int_synth.rs'
- 'wai/wai-rs/src/ort_session.rs'
- 'wai/neural-conformance/vectors/**'
- 'wai/pin-conformance/**'
- 'wai/wai-rs/tests/neural_native_decode.rs'
- 'wai/wai-rs/src/int_mlicpp.rs'
- 'wai/wai-rs/tests/vectors/int_mlicpp/**'
@ -37,6 +50,14 @@ on:
- 'wai/wai-rs/src/container.rs'
- 'wai/wai-rs/src/codecs/mod.rs'
- 'wai/wai-rs/src/codecs/prior_form.rs'
- 'wai/wai-rs/src/pin.rs'
- 'wai/wai-rs/src/ffi.rs'
- 'wai/wai-rs/src/ffi_neural.rs'
- 'wai/wai-rs/tests/ffi_neural_pin.rs'
- 'wai/wai-rs/src/int_synth.rs'
- 'wai/wai-rs/src/ort_session.rs'
- 'wai/neural-conformance/vectors/**'
- 'wai/pin-conformance/**'
- 'wai/wai-rs/tests/neural_native_decode.rs'
- 'wai/wai-rs/src/int_mlicpp.rs'
- 'wai/wai-rs/tests/vectors/int_mlicpp/**'
@ -75,6 +96,39 @@ jobs:
set -euo pipefail
cargo test --no-default-features --features neural --test neural_native_decode -- \
--exact resolve_checked_verifies_prior_hash strict_decode_gates_unsound_replicate \
int_mlicpp_decodes_through_the_strict_dispatch \
int_mlicpp_decodes_through_the_strict_dispatch int_mlicpp_set_pin_through_the_strict_dispatch \
int_synth_pin_covers_the_spec_it_reads several_file_capabilities_need_the_set_form \
pinned_decode_reads_only_verified_bytes wai2_decode_multi_picks_the_verified_rendition \
strict_gate_reads_the_capability_it_decodes_through set_incomplete_at_step_3_outranks_the_fallback_refusal \
decode_multi_strict_reports_replicate_reasons strict_gate_ignores_a_held_capability_it_cannot_dispatch \
2>&1 | tee "$RUNNER_TEMP/neural-gates.log"
grep -q "test result: ok. 3 passed; 0 failed" "$RUNNER_TEMP/neural-gates.log"
grep -q "test result: ok. 12 passed; 0 failed" "$RUNNER_TEMP/neural-gates.log"
# Library unit tests that exist only under `neural` and need no model: the
# bmshj2018 CDF table cache stays bounded and evicts least recently used;
# the ONNX reader finds a tensor kept in another file, wherever it sits; and a
# pinned decode refuses such a model with set_incomplete before the runtime
# sees it, while the same graph with its tensor inside loads.
- name: Model-free neural unit tests
working-directory: wai/wai-rs
shell: bash
run: |
set -euo pipefail
cargo test --no-default-features --features neural --lib -- --exact neural::image::tests::cdf_cache_is_bounded \
neural::onnx_external::tests::a_tensor_kept_inside_the_model_is_not_external \
neural::onnx_external::tests::an_external_initializer_is_found_with_its_location \
neural::onnx_external::tests::an_external_tensor_in_a_subgraph_is_found \
neural::onnx_external::tests::external_data_without_a_location_still_marks_the_tensor \
neural::onnx_external::tests::bytes_that_are_not_a_protobuf_message_are_an_error \
neural::params::tests::a_pinned_model_with_external_tensors_is_set_incomplete \
2>&1 | tee "$RUNNER_TEMP/neural-lib.log"
grep -q "test result: ok. 7 passed; 0 failed" "$RUNNER_TEMP/neural-lib.log"
- name: The neural C ABI reports pin and replicate refusals and decodes WAI2
working-directory: wai/wai-rs
shell: bash
run: |
set -euo pipefail
cargo test --no-default-features --features neural,ffi --test ffi_neural_pin -- \
--exact tampered_model_is_wai_err_prior strict_replicate_is_wai_err_unsound_replicate \
wai2_decode_ex_picks_the_held_pin failed_decode_ex_clears_out \
2>&1 | tee "$RUNNER_TEMP/neural-ffi.log"
grep -q "test result: ok. 4 passed; 0 failed" "$RUNNER_TEMP/neural-ffi.log"

17
.github/workflows/wai-web-wasm.yml vendored
View file

@ -12,7 +12,8 @@ name: wai-web wasm32
# registry agrees with the export on every candidate;
# - demo/verify_pack.mjs: the browser writers `pack` and `packMulti` refuse
# every candidate the export lists, in every place an envelope can name one;
# - a byte comparison of the demo's staged int_synth copies with the corpus.
# - a byte comparison of the demo's staged int_synth copies with the corpus;
# - the crate's wasm_tests under wasm-bindgen-test-runner.
#
# The full-scale int_hyper wasm decode needs vectors regenerated from a
# pretrained model; wai-sota-vectors.yml runs it after regenerating them.
@ -70,6 +71,20 @@ jobs:
cargo build --locked --release --target wasm32-unknown-unknown
wasm-bindgen --target nodejs --out-dir demo/pkg-node --out-name wai_web \
target/wasm32-unknown-unknown/release/wai_web.wasm
# The JS surface itself (pick, companionsOf, verifyPin, rendition() and
# companion() refusing what is never presented or does not match its digest,
# the held-sets conversion, and the WAI2 parse at 32-bit usize) runs only on
# wasm32 (`mod wasm_tests`). wasm-bindgen-test-runner, installed above at the
# version Cargo.lock pins, runs it under Node. The count check turns a test
# that stopped being compiled into a failure instead of a silent "0 passed".
- name: wai-web JS surface (wasm32 tests under Node)
shell: bash
working-directory: wai/wai-web
run: |
set -euo pipefail
CARGO_TARGET_WASM32_UNKNOWN_UNKNOWN_RUNNER=wasm-bindgen-test-runner \
cargo test --locked --target wasm32-unknown-unknown 2>&1 | tee "$RUNNER_TEMP/wai-web-wasm.log"
grep -q "test result: ok. 4 passed; 0 failed" "$RUNNER_TEMP/wai-web-wasm.log"
- name: wasm32 int_synth decode equals the frozen native corpus
shell: bash
working-directory: wai/wai-web

139
wai/CHANGELOG.md
View file

@ -39,9 +39,12 @@ wire format may change and such changes are recorded here.
candidate, a companion-only capability, a record or a derivation. A string this
revision does not register stays dispatchable — a later MINOR revision may register
it (§8) — and is decoded only where the sink advertises it. `codecs::is_dispatchable`
implements it and is the one predicate `Wai::dispatch_capability`, `WaiMulti::pick`,
`WaiMulti::primary`, `RenditionMeta::is_selectable` and the CLI's `inspect` use; §7.1's
selection sentence says the same. A later dispatch or selection function uses it too.
implements it and is the one predicate every dispatch and selection function of the
reference implementation uses (`Wai::dispatch_verified`, formerly
`dispatch_capability`; `WaiMulti::pick`; `WaiMulti::primary`;
`RenditionMeta::is_selectable`; the CLI's `inspect`; and, below, the neural sink, the
C ABI and wai-web); §7.1's selection sentence says the same. A later dispatch or
selection function uses it too.
- **`wai registry export`** writes `registry/capabilities.json`, its JSON Schema
(`registry/capabilities.schema.json`), the generated table of
[`REGISTRY.md`](REGISTRY.md) and **vocab v2** (`vocab/v2.{ttl,jsonld,shapes.ttl}`,
@ -131,6 +134,136 @@ wire format may change and such changes are recorded here.
cases §5.1–§5.5 define; verifying against their vectors becomes a MUST once the
vectors are published in the tree.
### The parameter-set pin, specified; component identity; pin-aware selection
- **SPEC §3.1** defines the pin a manifest carries (`conditioning.prior`; a `WAI2`
rendition's `prior`): `id`, `sha256`, `determinism`, and the optional `form`,
`tier` and `supersedes`. The digest covers the whole parameter set: a form-less
pin is admissible only for a capability registered with a one-file set (§5
registers each capability's shape), and a set of several files is pinned through
its listing, the new registered prior form `wai.prior.set` (`StandardDefined`,
royalty-free, the one form listed as established exact). A sink verifies the pin
before it decodes, reports a refusal with one of an ordered set of snake_case
reasons (`pin_malformed`, `form_not_registered`, `form_unreadable`,
`form_required`, the reserved `prior_revoked`, `prior_not_held` /
`digest_superseded`, `set_malformed`, `part_mismatch`; `set_incomplete` during a
decode), and decodes only from the set it verified. A pin over a capability this
revision does not register is verified as several files. `null` in an optional
member is the same as absent.
- **Tier** is a closed vocabulary (`decode-equivalence`, `entropy-consistency`,
`none`): an encoder writes a registered token, a sink reads any other as `none`,
a verifier reports it (`tier_unregistered`). Pins without `tier` keep the legacy
reading of `determinism`, now stated normatively, so every existing verdict —
`tests/vectors/int_mlicpp/stream.wai`'s included — is unchanged.
- **`supersedes` is informative**: it changes only the reason a sink that holds the
predecessor reports, a malformed one is a static finding the sink ignores, and a
successor never satisfies its predecessor's pin.
- **§4 and §7.1**: a pin that does not resolve makes its capability unsupported for
that envelope; dispatch continues at the fallback or the next rendition, and an
inert envelope's error names the reasons: for `WAI1`, the reason kept for the
capability before the fallback's, `set_incomplete` included; for `WAI2`, every
entry's (`not_selectable`, `missing_capability`, `component_digest_mismatch`, a
pin reason, or, under a policy that serves only sound `replicate` claims, the
entry's `replicate_*` reason). A `WAI1` pin applies to whichever capability
decodes the payload.
- **Component identity (§7.1)**: a rendition entry MAY carry `id` (unique, never a
capability string), `sha256` (its payload digest, checked before the entry is
decoded or applied: `component_digest_mismatch`) and, on a companion,
`companion_of_id`, which binds it to one entry — an enhancement layer can now
bind to one of two renditions of a base capability. `id`, `sha256` and `prior`
are one construct; no extension defines another per-component digest.
- **§7**: `replicate` over a `NeuralIntegerExact` capability MUST carry a pin that
declares `decode-equivalence`, in no form or in `wai.prior.set`. The new
*Digests* subsection names SHA-256 for parameter-set identity, for component
identity, and for the decoded output of capabilities that decode against a
parameter set and of standard codecs in conformance material; FNV-1a and BLAKE3
stay as diagnostics. The vectors of such capabilities gain `sha256.json`
sidecars; no existing vector file or value changes.
- **§8** settles how an optional member is added under the 1.x freeze, which the
`form` entry below left open: a MINOR revision MAY define an OPTIONAL member,
envelopes keep `wai` `"1.0"`/`"1.1"`, a reader MUST ignore a member it does not
understand whatever the MINOR, and an earlier reader may instead reject such an
envelope (fails closed) or ignore the pin (fails open), which §3.1 states. §8
states separately the one exception this revision makes to the freeze (next
entry).
- **Envelopes made non-conforming:** the list in SPEC §3.1. None is committed.
One kind stops parsing, the one exception to the `1.x` freeze that §8 states: a
manifest that carries a member this revision defines (a pin or pin member, or a
rendition entry's `id`, `sha256` or `companion_of_id`) with another JSON type is
malformed, because reading a mistyped `form` as absent would turn a set-form pin
into a form-less one. The envelope's `wai` value does not signal it.
New vectors: `tests/vectors/int_mlicpp/{model.sha256, stream_set.wai}` (the
set-form twin, written by `tools/wai_int_mlicpp_set_form.py`) and the
`pin-conformance/` corpus, with an independent verifier in Python.
- **Reference-sink behaviour:** a form-less pin over a several-file capability is
refused (`form_required`), where before it verified one file or failed on a
directory; `wai.neural.int_synth`'s pin covers `int_synth.json`; a digest that is
not 64 lowercase hex is refused, where before it was compared case-insensitively;
a refused pin falls through to a fallback the registry holds; every decoder reads
parameters through `neural::ParamSource`, so a pinned decode reads only verified
bytes (ONNX sessions built from them), and an ONNX model that keeps a tensor's
data in another file, which the runtime would read from outside those bytes, is
refused with `set_incomplete` naming that file, as §3.1 now states; `bmshj2018`'s CDF cache is keyed by content
digest and holds a bounded number of tables, least recently used evicted.
`decode_envelope_strict` applies §7's rule to the capability it decodes through,
so a sound fallback behind an unsound primary the sink cannot use still decodes;
it refuses before reading a file only when every capability it could decode
through (dispatchable and advertised) is refused, so a candidate the registry
happens to hold does not change its reason. A `WAI2` parse sums the rendition lengths in 64 bits, so a
table whose lengths wrap a 32-bit sum is refused as truncated on 32-bit targets
too, where it used to panic. `wai inspect` prints digests through
`pin::digest_preview`, which shows a malformed one whole instead of cutting it
inside a character.
- **Rust API, source-breaking** (the same 0.4.0 release as `form`): the manifest
types (`Manifest`, `ModelRequirement`, `Conditioning`, `Prior`, `Supersedes`,
`ManifestV2`, `RenditionMeta`, `Rendition`, `Security`) are now
`#[non_exhaustive]`, with constructors and `with_*` builders that write the same
JSON a literal wrote, so the next optional member does not break callers again;
every manifest object keeps members it does not model (`extra`), and `Manifest`
gains `security`, so a Rust round trip loses nothing. `Rendition` holds a
`RenditionMeta`. `Wai::dispatch_capability` is replaced by `dispatch_verified`;
`WaiMulti::pick`/`primary` take a pin verifier and return `Picked` (index, entry,
payload, verified set, skipped entries); `companions_of_index` binds by id.
`DecodeError::PriorMismatch` carries a `PinRefusal`, and `DecodeError` gains
`NoRendition`; `ModelRegistry` gains `held`/`hold`/`verify`/`resolve_verified`
(`resolve_checked` is deprecated); new `decode_payload`, `decode_multi`,
`decode_multi_strict`. `sha2` is now a plain dependency, and the new `pin` module
needs no feature.
- **C ABI:** a codec-free `ffi` feature carries the envelope surface (`codecs`
implies it); `WAI_ERR_PRIOR` (5), `WAI_ERR_UNSOUND_REPLICATE` (6) and
`WAI_ERR_NO_RENDITION` (7), with the reason first in `wai_last_error`; the unpack
calls return the original manifest bytes, as `wai.h` already documented; the
`WAI2` table carries identity, pin and `selectable`; new `wai_envelope_select`
and `wai_neural_decode_ex` (`WAI1` or `WAI2`, a registry of held sets, a strict
flag), which writes `*out` on every call, empty on failure, so `out->data` is
always safe to free. The envelope packers build under `ffi` too, and
`tests/ffi_envelope.rs` checks in CI that they refuse a candidate and that
`wai_envelope_select` never picks one.
- **wai-web** wraps `wai::container` instead of reduced copies, so `role`,
`companion_of`, identity and pins reach JS, and exposes `pick`, `companionsOf`,
`verifyPin` and `sha256Hex`; a component whose capability is not dispatchable
(SPEC §4) is never presented as media, and `rendition(i)` and `companion(i)` refuse an entry whose payload does not
match its `sha256`. Its JS-facing tests run under wasm32 in CI.
- **One dispatch predicate:** `dispatch_verified`, `pick`, `primary`,
`RenditionMeta::is_selectable`, `decode_multi`, `decode_envelope_strict`'s early
refusal, `wai_envelope_select`, wai-web and `wai inspect` all take a capability only when `codecs::is_dispatchable`
(above) accepts it, so a candidate, a record or a derivation is never dispatched
or picked, whatever its pin. An unregistered string stays dispatchable where the
sink advertises it, and a pin on it is verified as several files, so it fails
closed. SPEC §7.1 step 2 reads "dispatchable (§4)". `wai.prior.set` has its row in
the registry table, so `registry/capabilities.json`, `REGISTRY.md` and vocab v2
list it.
- **Vocabulary:** `vocab/v1` gains the pin terms additively, and `wai registry
export` writes the same terms and shapes into v2, with the tier list taken from
`codecs::TIER_TOKENS`; `vocab/check_shapes.py` validates conforming and
non-conforming pins against the v2 shapes. `sharedPriorUri` and
`sharedPriorHash` are annotated as not being the manifest pin. The SHACL pin shape
targets the objects of `wai:prior` (no class is asserted for them), and
`supersedes` is checked by its own shape: a required digest and an optional form.
- **Open item:** `wai.world.model` names its renderer's model by `model_id`, a
second model-identity construct; it is to fold into the pin once that renderer
is registered as decoding against a parameter set.
### Typed receipt classes, and a second (soft) binding class
- [`extensions/jwp-receipts.md`](extensions/jwp-receipts.md) §2.4.3–§2.4.7: every

3
wai/REGISTRY.md
View file

@ -10,7 +10,7 @@ together with the class functions of [`wai-rs/src/codecs/mod.rs`](wai-rs/src/cod
|---|---|
| [`registry/capabilities.json`](registry/capabilities.json) | every registered string with its kind, status, media class, role, fidelity tier, licence class, replicate class, set membership, registering section, edition and registration date; the named identifiers that are not capabilities; the unregistered labels |
| [`registry/capabilities.schema.json`](registry/capabilities.schema.json) | JSON Schema (2020-12) for that file; its enums are the label sets below |
| [`vocab/v2.ttl`](vocab/v2.ttl), [`vocab/v2.jsonld`](vocab/v2.jsonld), [`vocab/v2.shapes.ttl`](vocab/v2.shapes.ttl) | the same registry as RDF, a JSON-LD context for manifests, and a SHACL shape that checks a manifest's capability and fallback against it and only the form of its media; a violation for a string the registry does not hold means the string is not registered in this revision, not that the envelope fails to conform or cannot be dispatched (SPEC §4, §8) |
| [`vocab/v2.ttl`](vocab/v2.ttl), [`vocab/v2.jsonld`](vocab/v2.jsonld), [`vocab/v2.shapes.ttl`](vocab/v2.shapes.ttl) | the same registry as RDF, a JSON-LD context for manifests, and a SHACL shape that checks a manifest's capability and fallback against it, only the form of its media, and its parameter-set pins (`vocab/README.md`); a violation for a string the registry does not hold means the string is not registered in this revision, not that the envelope fails to conform or cannot be dispatched (SPEC §4, §8) |
| the table at the end of this file | the same registry, for reading |
A library test (`codecs::registry::tests::registry_export_matches_committed`) and
@ -181,6 +181,7 @@ and a role are fixed once registered.
| `wai.pc.gpcc` | payload | registered | volumetric | standalone | patent-pool | standard-unclassified | — | §5 Immersive audio, cinema, volumetric, haptics, HDR |
| `wai.pc.vpcc` | payload | registered | volumetric | standalone | patent-pool | standard-unclassified | — | §5 Immersive audio, cinema, volumetric, haptics, HDR |
| `wai.prior.nnc` | payload | registered | binary | standalone | patent-pool | standard-unclassified | — | §5 Prior artifacts, general tensors and derivation operations |
| `wai.prior.set` | payload | registered | binary | standalone | royalty-free | standard-defined | — | §5 Prior artifacts, general tensors and derivation operations |
| `wai.quantum.calibration` | record | registered | — | — | — | — | — | §5 Interactive, temporal & computed content |
| `wai.quantum.circuit` | payload | registered | binary | standalone | royalty-free | standard-defined | — | §5 Interactive, temporal & computed content |
| `wai.quantum.job` | record | registered | — | — | — | — | — | §5 Interactive, temporal & computed content |

479
wai/SPEC.md
View file

@ -8,8 +8,10 @@ file:
compact conditioning payload against a *shared ambient prior* (a
model). The model is a **requirement of existence**, never
transported in the envelope that decodes against it, exactly as `.mp3`
does not ship its decoder. Its identity MAY be pinned by digest (a
`Prior`), and the pinned artifact's wire form MAY be declared
does not ship its decoder. Its identity MAY be pinned: the manifest MAY
carry a *pin*, the SHA-256 of the whole parameter set the payload was coded
against (§3.1), and a sink whose parameter set does not match the pin does
not decode against it (§4, §7.1). The pinned set's wire form MAY be declared
([prior-carriage](extensions/prior-carriage.md)).
- the **zeroth condition** — a *menu of registered standard codecs*
(AVIF, JPEG XL, PNG, Opus, FLAC, AV1, zstd, XZ and the others §5
@ -40,7 +42,9 @@ A WAI file declares (in its manifest):
- what media it carries,
- what *capability* a sink must have to decode it,
- a fallback capability the sink can use if it lacks the primary,
- and the conditioning bytes for that capability.
- the conditioning bytes for that capability,
- and, OPTIONALLY, for a capability that decodes against a parameter set, the
pin of the set the payload was coded against (§3.1).
Conforming sinks MUST implement the container envelope (§2), the
manifest schema (§3), the capability-dispatch algorithm (§4), and at
@ -66,7 +70,9 @@ OPTIONAL; §5 says which are RECOMMENDED.
on key order for *semantics*; encoders SHOULD preserve order so files
round-trip byte-identically.
- The payload is opaque codec bytes whose interpretation is fully
determined by the manifest's `model_requirement.capability`.
determined by the manifest's `model_requirement.capability` and, for a
capability that decodes against a parameter set, by that set, which the
manifest MAY pin (§3.1).
All WAI multi-byte integers are little-endian unless stated otherwise.
@ -82,14 +88,15 @@ All WAI multi-byte integers are little-endian unless stated otherwise.
"intent": "replicate" | "create" | "improve" | "...",
"model_requirement": { "capability": "<capability-string>",
"fallback": "<capability-string>" | null },
"conditioning": { "kind": "<codec-id>" },
"conditioning": { "kind": "<codec-id>", "prior": { ... } },
"security": { "encryption": "<scheme>" | "none", "kid": <u64> },
"target": { ... }
}
```
- `wai` — spec version. This document defines `"1.0"`. Unknown major →
reject. Unknown minor → accept by ignoring unknown fields.
reject. Unknown minor → accept. A reader ignores a member it does not
understand, whatever the minor (§8).
- `media` — media class; informational. The capability is authoritative.
The values are the registry's media classes (`codecs::capability_media`;
[`registry/capabilities.json`](registry/capabilities.json)): `image`,
@ -111,7 +118,7 @@ All WAI multi-byte integers are little-endian unless stated otherwise.
what its standard's decoder conformance states, which the registry does not
assert is byte identity.
For a **neural** capability, `replicate` promises decode-equivalence and is sound
only under the integer-entropy contract (§7): a capability whose decode runs
only under the integer-entropy contract (§7), with a pin (§3.1, §7): a capability whose decode runs
float, or whose entropy decode alone is integer, MUST NOT claim it. That is
WAI's rule, and this revision applies it to `wai.image.jpegai` as well; it is
not a finding that JPEG AI's standard leaves the reconstruction unbounded (§7,
@ -119,10 +126,18 @@ All WAI multi-byte integers are little-endian unless stated otherwise.
- `model_requirement.capability` — the capability string a sink MUST
resolve. See §5 for the registered set.
- `model_requirement.fallback` — a second capability the sink MAY use
if it lacks the primary. `null` means the file is inert without the
if it lacks the primary, or if the envelope's pin does not resolve for the
primary at the sink (§4 step 3). `null` means the file is inert without the
primary.
- `conditioning.kind` — short codec id matching the capability. Helps
multiplexers route bytes without parsing the full capability string.
- `conditioning.prior` — OPTIONAL. The pin (§3.1) of the parameter set the
payload was coded against. It applies to whichever of
`model_requirement.capability` and `model_requirement.fallback` decodes the
payload (§4), where that capability decodes against a parameter set. Absent, or
`null`, the key is not written, and a decode through such a capability is
unpinned: it runs against whatever parameter set the sink holds, and it cannot
back `intent = replicate` over a `NeuralIntegerExact` capability (§7).
- `security` — OPTIONAL. Absent, `null`, or `encryption: "none"` ⇒ the
payload is plaintext (the default). Otherwise the payload is encrypted
and MUST be recovered before dispatch; see §6. The capability, media,
@ -131,6 +146,193 @@ All WAI multi-byte integers are little-endian unless stated otherwise.
sample rate, duration, frame count). Informational; the payload is
self-describing per the codec's own format.
### 3.1 Parameter-set pin (`prior`)
A capability **decodes against a parameter set** when its registry class (§7) is
`NeuralIntegerExact`, `NeuralEntropyExact`, `NeuralFloat` or
`GenerativeDeterministic`. Its **parameter set** is every byte, other than the
payload, that its decode reads and that its output depends on: networks, entropy
tables, normative constants. The sink supplies it; the envelope never carries it
(§1). §5 registers each such capability's parameter set as **one file** or
**several files**; a capability registered with neither, and a capability this
revision does not register, is treated as several files.
A manifest names the parameter set a payload was coded against with a **pin**:
```json
"prior": {
"id": "<a name for people>",
"sha256": "<64 lowercase hexadecimal digits>",
"determinism": "<how the decode is made reproducible>",
"form": "<a registered prior form>",
"tier": "decode-equivalence",
"supersedes": { "sha256": "<64 lowercase hexadecimal digits>", "form": "<a registered prior form>" }
}
```
| member | presence | meaning |
|---|---|---|
| `id` | REQUIRED, string | A name for people and logs. A sink MUST NOT select, resolve or accept a parameter set by it. |
| `sha256` | REQUIRED, string | The SHA-256 (FIPS 180-4) of the parameter set in its form (below), as 64 lowercase hexadecimal digits. |
| `determinism` | REQUIRED, string | How the decode is made reproducible, in words. Informative, except where §7 reads it for a pin without `tier`. |
| `form` | OPTIONAL, string | A registered prior form (§5; [prior-carriage](extensions/prior-carriage.md)). |
| `tier` | OPTIONAL, string | The reproducibility tier ([determinism-tiers](extensions/determinism-tiers.md) §2) the sender claims for the decode, from a closed vocabulary: `decode-equivalence`, `entropy-consistency`, or `none` for neither. |
| `supersedes` | OPTIONAL, object | Informative. The parameter set this one replaces, named as a pin names one: its `sha256` (REQUIRED) and, if it is pinned in a declared form, that `form`. |
A `prior`, or a member of it, whose JSON type is not the one given makes the
manifest malformed, and a reader rejects it as it rejects any manifest that does
not parse. A mistyped member is never read as absent: a `form` dropped that way
would turn a set-form pin into a form-less one, which a single file can satisfy. `null` in an OPTIONAL member is the same as the member being absent.
A member this section does not define is ignored (§8). An optional member that is
absent is not written, so a pin written without it keeps its bytes.
**What the digest covers.** A pin identifies the whole parameter set.
- With `form` absent, `sha256` is the SHA-256 of the one file that holds the set.
That is admissible only for a capability registered with a one-file set.
- With `form` present, `sha256` is the SHA-256 of the set serialized in exactly
that form ([prior-carriage](extensions/prior-carriage.md) §2). The form
`wai.prior.set` serializes a set of one or more files as its **listing**, so a
pin in that form covers every file of the set:
- The listing has one line for each file: the file's SHA-256 as 64 lowercase
hexadecimal digits, two spaces (0x20 0x20), the file's path, and a line feed
(0x0A).
- A path names the file below the set's root: one or more segments joined by
`/` (0x2F). A segment begins with one of `A`–`Z`, `a`–`z`, `0`–`9`, `_` and
`-`, continues with any of those and `.`, and does not end with `.`.
- The lines are in strictly ascending order of their paths compared as byte
strings. No two paths are equal once `A`–`Z` are mapped to `a`–`z`, and no
path followed by `/` begins another path.
- The listing is its lines concatenated. It has at least one line and nothing
else: no header, no byte-order mark, no carriage return, no blank line.
A listing is the text common checksum utilities write and check, so a third
party can check a set against its listing, and a listing against a pin, without
WAI software.
**Verification.** A sink that is to decode through a capability, for an envelope
whose pin applies to that capability, MUST verify the pin first. A pin applies to
a capability that decodes against a parameter set, and to a capability this
revision does not register when the sink advertises it (such a string is
dispatchable, §4): such a capability is verified as several files, so a form-less pin over it is refused
(`form_required`). The pin **resolves** when the sink holds a parameter set for the
capability whose digest, in the pin's form, equals `sha256`: in the form
`wai.prior.set`, a listing whose SHA-256 is `sha256`, together with every file the
listing names below the listing's root, each with the SHA-256 its line gives.
Nothing else satisfies a pin: not a matching `id`, not a set that `supersedes`
names or that names this one, and not a set in another form whose unpacked
parameters compare equal. A sink MAY hold several parameter sets for one
capability; the pin resolves when one of them satisfies it.
A pin that does not resolve is refused with one reason. The checks run in this
order, and the first that fails is the reason reported:
1. `pin_malformed`: `sha256` is not 64 lowercase hexadecimal digits.
2. `form_not_registered`: `form` names no registered prior form.
3. `form_unreadable`: `form` is registered and the sink has no reader for it.
This is the sink's gap, and it is never reported as a mismatch.
4. `form_required`: `form` is absent and the capability is not registered with a
one-file set.
5. `prior_revoked`: reserved, for a revocation of the pinned set, keyed on the
pin's `form` and `sha256`.
6. `prior_not_held`: the sink holds no parameter set for the capability whose
digest, in the pin's form, equals `sha256`; a file the sink cannot read is not
held. A sink that holds the set a well-formed `supersedes` names — a file when
`supersedes.form` is absent, a listing when it is `wai.prior.set` — reports
`digest_superseded` instead, so that its operator can tell it holds the set
this one replaces.
7. `set_malformed`: in the form `wai.prior.set`, the listing whose SHA-256 is
`sha256` breaks a rule above.
8. `part_mismatch`: in the form `wai.prior.set`, in every copy of that listing
the sink holds, a file the listing names is missing, cannot be read, or has a
SHA-256 other than its line gives.
These checks read the envelope, the registry, the forms the sink can read and the
bytes it holds, and nothing that depends on how the sink was built, so two
conforming sinks that read the same forms and hold the same bytes report the same
reason. Refusal reasons are snake_case; tier names and class labels are
kebab-case.
**Decoding against a verified set.** A pinned decode MUST read its parameters
only from the set it verified. A decode that needs a file the verified set does
not contain is refused with `set_incomplete`; the sink presents no output from it,
and dispatch treats it as a pin that did not resolve. A file a model names for
part of its own data, such as the file an ONNX tensor stored as external data
names, is a file the decode needs: when the decode cannot take it from the
verified set, it is refused the same way. A sink SHOULD decode from
the very bytes it verified: one that reads a file again after verifying it can
decode bytes it did not verify.
**A refused pin** makes its capability **unsupported for that envelope** at that
sink. It is not a decode error and not a reason to decode anyway, as
[capability-constraints](extensions/capability-constraints.md) §2 treats an unmet
constraint. Dispatch continues as §4 and §7.1 state. A sink MUST NOT decode a
payload through a capability whose pin it refused.
**Tier.** `tier` is the sender's claim, from the registered vocabulary
`decode-equivalence`, `entropy-consistency` and `none`. An encoder MUST write a
registered token. A sink reads a token this revision does not register as `none`,
and a verifier reports it (`tier_unregistered`). The tier a registry class gives
is `decode-equivalence` for `NeuralIntegerExact`, `entropy-consistency` for
`NeuralEntropyExact`, and `none` for every other class. With the tiers ordered
`none` < `entropy-consistency` < `decode-equivalence`, a `tier` above the class's
does not conform (`tier_exceeds_class`): the `replicate` verdict (§7) does not
rest on it, and a verifier reports it. The decode itself is unaffected. A `tier`
at or below the class's is honoured as stated.
**Updates.** `supersedes` is informative: it records that the set a pin names
replaces another, changes no decision, and changes only the reason check 6
reports. A `supersedes` whose `sha256` is not 64 lowercase hexadecimal digits or
equals `sha256`, or whose `form` is not a registered prior form, is malformed: a
verifier reports it (`supersedes_malformed`), and a sink ignores the member and so
never reports `digest_superseded` for that pin. Content coded against a set pins
that set, and a successor never satisfies its predecessor's pin, so content coded
against a predecessor decodes at a sink exactly as long as that sink still holds
the predecessor. A sink can hold both without a second copy of the files they
share, by keeping the predecessor's listing beside the successor's. An update
delivered as a change to an earlier set, such as an NNC incremental update, is not
a parameter set and is never pinned; the self-contained set it yields is
([prior-carriage](extensions/prior-carriage.md) §2).
**Where a pin sits.** In a `WAI1` manifest, `conditioning.prior` (§3). In a `WAI2`
manifest, a rendition entry's `prior`, which pins that entry's capability and is
part of the entry's identity (§7.1). A pin where no capability it would apply to
decodes against a parameter set, and every such capability is registered, means
nothing: an encoder MUST NOT write one, a verifier reports it
(`prior_not_applicable`), and a sink ignores it.
**Sinks of earlier revisions.** A sink that implements an earlier text of this
specification may do one of two things with a pin (§8). It may reject the
envelope as carrying a member it does not understand, which fails closed. Or it
may ignore `prior`, and then the pin fails open: such a sink may decode a pinned
payload against another parameter set. The rules of this section bind sinks that
implement this revision; an encoder that must not be decoded against another set
at an earlier sink has no means to require that within `1.x` other than a
capability string earlier sinks do not advertise.
**Envelopes these rules make non-conforming.** The rules enforce the definition of
`prior` as it was made — a lowercase hexadecimal SHA-256 of the artifact that
determines the decode's output, which a `replicate` over an integer-exact neural
capability requires — and change no payload format and no role (§8). Each of these
envelopes but the last still parses, and does not conform:
- one whose pin's `sha256` is not 64 lowercase hexadecimal digits (the reference
implementation compared digests without regard to case or length);
- one whose pin has no `form` for a capability registered with several files: its
digest covered one file of the set or, where the registered artifact was a
directory, nothing a sink could hash;
- one that carries a pin where no capability it would apply to decodes against a
parameter set;
- one with `intent = replicate` whose decode capability is `NeuralIntegerExact`
and which pins no parameter set for it, or pins one that does not declare
`decode-equivalence` (§7); the reference implementation already refused these;
- one that carries a member this revision defines — `conditioning.prior` or a
rendition entry's `prior`, a pin member, or a rendition entry's `id`, `sha256` or
`companion_of_id` (§7.1) — with a JSON type other than the one defined. A reader
of an earlier revision may have ignored the member; a reader of this one rejects the
manifest as malformed, so this envelope no longer parses. §8 states this as the
one exception this revision makes to the `1.x` freeze.
---
## 4. Capability dispatch
@ -139,13 +341,26 @@ A conforming sink MUST implement this algorithm:
1. Read the manifest. If `wai` major version is unknown → reject.
2. Let `cap = model_requirement.capability`.
3. If `cap` is dispatchable (below) and the sink advertises `cap`, decode
`payload` via the codec registered for `cap` (§5) and return.
3. If `cap` is dispatchable (below), the sink advertises `cap`, and — when a pin applies to `cap` (§3.1) and the
manifest carries `conditioning.prior` — the pin resolves at the sink for
`cap`, decode `payload` via the codec registered for `cap` (§5), against the
verified set, and return. A pin that does not resolve makes `cap` unsupported
for this envelope: keep its reason and continue at step 4.
4. Else, let `fb = model_requirement.fallback`.
5. If `fb != null`, `fb` is dispatchable and the sink advertises `fb`,
decode via `fb`'s codec and return.
6. Else the file is **inert at this sink** — return a clear "missing
capability" error to the calling application.
5. If `fb != null`, `fb` is dispatchable and the sink advertises `fb`, and — when a pin applies to `fb`
and the manifest carries `conditioning.prior` — the pin resolves at the sink
for `fb`, decode via `fb`'s codec and return. A pin that does not resolve: keep
its reason.
6. Else the file is **inert at this sink** — return an error to the calling
application that names why: the reason kept at step 3, else the one kept at
step 5, else `missing_capability`.
`conditioning.prior` pins the parameter set the payload was coded against, so it
applies to whichever capability decodes the payload. A pin that does not resolve
is never a reason to decode against another parameter set: a learned decode
against the wrong parameters yields plausible output, not an error. An encoder
that needs an alternative for sinks that hold another parameter set carries one
pinned rendition per set in a `WAI2` envelope (§7.1).
A capability is **dispatchable** unless this revision registers it as something
a sink does not decode on its own: a candidate (§5, *Candidate capabilities*), a
@ -248,6 +463,17 @@ zeroth codecs. The decode route for each is implemented in the reference
sink (see the `source` column / `neural/*.rs`); the model weights are
sink-supplied.
A capability that decodes against a parameter set (§3.1) is registered with that
set's shape. **One file:** `wai.neural.int_synth` (its model specification,
`int_synth.json`); `wai.neural.int_mlicpp` (its `WMB1` bundle, read from a set as
`model.wmb`); and `wai.neural.encodec32`, `wai.neural.dac`, `wai.neural.mimi`,
`wai.neural.wavtokenizer`, `wai.neural.snac` and `wai.neural.video_bmshj2018` (a
self-contained ONNX model; one that keeps tensors in external files is several
files). **Several files:** every other capability that decodes against a parameter
set. A later revision MAY register a capability of several files as one file when
it defines a one-file format that holds the whole set; that widens the
registration and makes no conforming envelope non-conforming (§8).
*Audio:*
| capability | model | operating point | source |
@ -348,7 +574,8 @@ reconstruction, with the acquisition class and coverage C2PA 2.4's
| capability | payload | status · extension |
|---|---|---|
| `wai.prior.nnc` | a trained network's parameters as an ISO/IEC 15938-17:2024 (NNC, 2nd edition) bitstream | the registered wire form a `Prior` MAY declare (`prior.form`); its digest then covers the coded bitstream. Optional, not recommended: royalty status is not established, and cross-decoder exactness of the reconstruction is not established, so a `replicate` over a neural capability cannot rest on a prior read out of it. As a payload it takes the replicate class of a standard whose decoder conformance the registry has not checked · [prior-carriage](extensions/prior-carriage.md) |
| `wai.prior.nnc` | a trained network's parameters as an ISO/IEC 15938-17:2024 (NNC, 2nd edition) bitstream | a registered wire form a pin MAY declare (`prior.form`); its digest then covers the coded bitstream. Optional, not recommended: royalty status is not established, and cross-decoder exactness of the reconstruction is not established, so a `replicate` over a neural capability cannot rest on a prior read out of it. As a payload it takes the replicate class of a standard whose decoder conformance the registry has not checked · [prior-carriage](extensions/prior-carriage.md) |
| `wai.prior.set` | the listing of a parameter set of one or more files: each file's SHA-256 and path (§3.1) | a registered wire form a pin MAY declare (`prior.form`), and the form in which a set of several files is pinned: the pin covers the listing, and each line covers one file. Reading a set through its listing compares digests and hands the decoder each file's bytes unchanged, so the registry lists it as established exact, and a `replicate` MAY rest on a pin in it. WAI-native, royalty-free; as a payload its output is its own bytes (`StandardDefined`) · [prior-carriage](extensions/prior-carriage.md) |
| `wai.tensor.nnc` | general tensorial data (split-inference feature maps, 3D/4D scene parameters) coded with NNC | **candidate**: general-tensor coding is in an unpublished amendment, so the payload format is not fixed and MUST NOT be emitted until pinned (*Candidate capabilities*, below) · [prior-carriage](extensions/prior-carriage.md) |
The ten derivation operations of ISO/IEC 23001-16:2021 (derived visual tracks)
@ -454,9 +681,9 @@ a `WAI1` envelope that names a candidate as its `capability` or its `fallback`,
a `WAI2` envelope with a component of a candidate; such envelopes still parse. The
reference implementation lists the candidates in `codecs::is_candidate`, reports
such envelopes from `Wai::check_candidates` and `WaiMulti::check_candidates`, and
never dispatches or selects a candidate: `Wai::dispatch_capability`,
`WaiMulti::pick` and `WaiMulti::primary` take only a capability
`codecs::is_dispatchable` accepts (§4's dispatchable). Its `wai wrap` tool, its C
never dispatches or selects a candidate: `Wai::dispatch_verified`,
`WaiMulti::pick`, `WaiMulti::primary` and the C `wai_envelope_select` take only a
capability `codecs::is_dispatchable` accepts (§4's dispatchable). Its `wai wrap` tool, its C
envelope packers (`wai_envelope_pack`, `wai_envelope_pack_multi`) and its browser
writers (`pack`, `packMulti`) refuse to write one.
@ -599,7 +826,7 @@ The reference implementation checks these rules in
`Wai::check_component_roles` (v1.0) and `WaiMulti::check_component_roles`
(v1.1: the companion rules of §7.1, including both enhancement-layer rules
above), dispatches `WAI1`
envelopes through `Wai::dispatch_capability`, which applies both sentences
envelopes through `Wai::dispatch_verified`, which applies both sentences
above, never returns a companion-only component from `pick` / `primary`, and
its `wai wrap` tool refuses to write an envelope that fails either check. Parsing stays
format-agnostic: a non-conforming envelope still parses, and the check is what
@ -911,6 +1138,7 @@ cited are the ones read; the evidence, and what was not read, is in
| class | capabilities | what the named standard fixes |
|---|---|---|
| `StandardDefined` | `wai.image.png`, `wai.audio.flac`, `wai.text.zstd`, `wai.text.xz` | the decoded output exactly: lossless formats (PNG, W3C Recommendation, Third Edition — its filters are exact integer operations; FLAC, RFC 9639) and lossless byte formats |
| | `wai.prior.set` | its bytes exactly: a listing (§3.1) is its own output |
| | `wai.video.avc`, `wai.video.hevc`, `wai.video.vvc` | every sample of every cropped decoded picture a conforming decoder outputs equals the value the specified decoding process produces (ITU-T H.264 (V16, 06/2026) §C.4; H.265 (V11, 01/2026) §C.5.1; H.266 (V4, 01/2026) §C.5.1); processing an SEI message only signals, such as film grain synthesis, is outside that decoding process |
| | `wai.video.vp9` | output frames "identical in all respects", in the same order, to those of the specified decoding process (VP9 Bitstream & Decoding Process Specification v0.6 §8.1) |
| `StandardTolerance` | `wai.audio.opus` | output within `opus_compare`'s thresholds of the reference decoder, with the same final range-decoder state (RFC 6716 §6) |
@ -996,21 +1224,48 @@ A conforming sink that receives `intent = replicate` for a capability whose clas
is `NeuralEntropyExact`, `NeuralFloat`, `GenerativeDeterministic` or
`Behavioral` MUST refuse the replicate claim (decode as perceptual or reject). The
capability is the one it decodes through: `model_requirement.capability`, or the
fallback when it decodes via §4 step 5. The
fallback when it decodes via §4 step 5, or, in a `WAI2` envelope, the capability
of the rendition it picks (§7.1).
**The pin under `replicate`.** Over a `NeuralIntegerExact` capability a conforming
sink MUST also refuse the replicate claim unless a pin applies to that capability
(§3.1), declares `decode-equivalence`, and declares no form other than one the
registry lists as established exact ([prior-carriage](extensions/prior-carriage.md)
§5). A pin declares a tier by `tier`. A pin without `tier` declares
`decode-equivalence` exactly when its `determinism`, compared in ASCII lower case,
contains `int`, `fixed` or `integer` and none of `f32`, `f64` and `float`, and
declares no tier otherwise. That legacy reading exists so that a pin written
before `tier` keeps its verdict. It reads a declaration and can misread one (a
contract naming `fp16` contains no float token); the registry class, checked
first, is what guarantees the arithmetic, so a misread declaration cannot admit a
float decode. An encoder SHOULD write `tier`, and SHOULD write a `determinism`
that this reading classes the same way, so that a sink of an earlier revision
reaches the same verdict. The refusals are named `replicate_unregistered` (the
capability is not registered), `replicate_class` (its class cannot back
`replicate`), `replicate_unpinned`, `replicate_tier` (the pin does not declare
`decode-equivalence`) and `replicate_form_not_exact`. They read only the manifest
and the registry, so a sink MAY make them before it verifies the pin; refusing the
claim does not excuse the pin, which every decode of a pinned payload verifies
(§3.1).
The
reference impl checks this per-envelope in `decode_envelope_strict`
(`codecs::replicate_refusal`) in two steps. First the capability's registry
class: only `NeuralIntegerExact` and the three standard classes pass, whatever the
envelope declares (`ReplicateClass::replicate_promise` states what each
promises). Then, for a `NeuralIntegerExact` capability, the pinned prior: it must
declare an integer-exact contract (`DeterminismClass`), and a prior that declares
the wire form of its artifact counts as integer-exact only if that form is
established to reconstruct identical tensors at every reader
([prior-carriage](extensions/prior-carriage.md) §5); no form is yet, so a
`replicate` whose prior declares one is refused. The prior check reads the
promises). Then, for a `NeuralIntegerExact` capability, the pin: it must be present and
declare `decode-equivalence` (`Prior::declared_tier`), and a pin that declares a
form passes only if the registry lists that form as established exact
([prior-carriage](extensions/prior-carriage.md) §5): `wai.prior.set` is listed,
and no coded form is. The prior check reads the
sender's declaration; the class check is what ties it to the capability, so an
envelope whose prior declares an integer contract over a float or entropy-only
capability is refused. `decode_envelope_strict` decodes only through
`model_requirement.capability`, so that is the one it checks. On the encoder
capability is refused. `decode_envelope_strict` applies the rule to the capability
it decodes through — `model_requirement.capability`, or the fallback when dispatch
reaches step 5 — and, when the rule refuses every capability the sink could
decode through, refuses before it verifies a pin or reads a file;
`decode_multi_strict` considers only the renditions this rule admits, and reports
each one it leaves out with that rendition's `replicate_*` reason (§7.1 step 5). On the encoder
side, `wai wrap`, which pins no prior, writes `intent = replicate` only where that
check accepts it unpinned — over the three standard classes — for the capability
and for any fallback it declares, or, in the `WAI2` companion form (§7.1), for the
@ -1061,7 +1316,52 @@ capability promises, and the precondition for a hard
the entropy decode reproduces across devices (integer entropy arithmetic or
explicit parameter transmission), the tier at which JPEG AI sets its bit-exact
conformance point; it does not back `replicate`. `NeuralFloat` capabilities claim
neither tier.
neither tier. A pin MAY state its decode's tier (§3.1, `tier`); a stated tier above
the class's does not conform.
### Digests (normative)
WAI names one digest algorithm for each purpose:
- **Parameter-set identity** — a pin's `sha256` and `supersedes.sha256`, and each
line of a listing (§3.1): SHA-256 (FIPS 180-4), written as 64 lowercase
hexadecimal digits.
- **Component identity** — a `WAI2` entry's `sha256`, the digest of its payload
bytes (§7.1): SHA-256, written the same way. The content hash a receipt chain
takes over a whole envelope ([jwp-receipts](extensions/jwp-receipts.md) §2.1)
is a different object.
- **Decoded output in conformance material** — a digest of the decoded output of
a capability that decodes against a parameter set (§3.1), or of a standard codec
(§5), in a published conformance vector or corpus, in a golden a conformance test
compares with, or in a reconstruction or energy binding
([reconstruction-binding](extensions/reconstruction-binding.md) §2,
[energy-binding](extensions/energy-binding.md)): SHA-256 over the capability's
canonical output buffer, written as 64 lowercase hexadecimal digits.
- **Receipt content and WAI-native state** — an envelope's content hash in a
receipt chain, and, for a WAI-native capability that decodes against no
parameter set, its state hash and the digests of its rendered output: the
algorithm the defining extension names ([jwp-receipts](extensions/jwp-receipts.md)
§2.1; the extensions §5 links).
A conformance vector or corpus that states a digest of decoded output in the third
purpose MUST state its SHA-256 and MUST name the buffer it covers:
- an image, or one frame of video: `RGB8` as
[reconstruction-binding](extensions/reconstruction-binding.md) §2 defines it;
- audio: the samples as signed 16-bit little-endian integers in output order,
channels interleaved;
- a sequence of frames: each frame's digest, and a clip digest, the SHA-256 of the
frames' 32-byte SHA-256 digests concatenated in output order;
- any other output (an attribute tensor, an intermediate stage): the buffer the
vector's documentation names, with its element type, byte order and layout.
A vector MAY state other digests of the same buffer, such as a 64-bit FNV-1a or a
domain-separated BLAKE3, to locate a divergence. A verifier MUST NOT accept a
decode on such a digest alone. A hash a capability uses as a step of its own
algorithm (the prompt fold of `wai.generate.spectral`) is part of that capability,
not a digest of its output, and this rule does not reach it. A frozen digest
SHOULD be re-derived on more than one instruction-set architecture before it is
published.
### Machine-checked totality (informative)
@ -1158,7 +1458,38 @@ by which capability happens to be installed.
}
```
Each entry has its own `capability` (the dispatch key) and `kind`
(the codec id matching the bytes in the corresponding payload).
(the codec id matching the bytes in the corresponding payload). An entry MAY
carry `prior`, the pin (§3.1) of the parameter set its payload was coded
against. Two entries MAY name one capability with different pins: the same
content coded against two parameter sets, such as a set and its successor:
```json
"renditions": [
{ "capability": "wai.neural.int_mlicpp", "kind": "int_mlicpp",
"prior": { "id": "int_mlicpp/b", "sha256": "<B>", "determinism": "int-rans/p16+int-mlicpp/i64",
"tier": "decode-equivalence", "supersedes": { "sha256": "<A>" } } },
{ "capability": "wai.neural.int_mlicpp", "kind": "int_mlicpp",
"prior": { "id": "int_mlicpp/a", "sha256": "<A>", "determinism": "int-rans/p16+int-mlicpp/i64",
"tier": "decode-equivalence" } },
{ "capability": "wai.image.avif", "kind": "avif" }
]
```
A sink that holds `A` decodes the second entry; one that holds `B`, the first;
one that holds neither, the third.
- **Component identity.** An entry MAY carry an `id`, a string of 1 to 64 of
`a`–`z`, `0`–`9`, `_` and `-`, unique within the envelope (it contains no `.`,
so it never equals a capability string), and a `sha256`, the SHA-256 of the
entry's payload bytes as 64 lowercase hexadecimal digits. A component's identity
is its `id`, its payload digest `sha256` and its parameter-set pin `prior`
(§3.1). These are one construct: no extension defines another per-component
digest, and a later mechanism that addresses a component (a staged layer, a
byte-range or referenced rendition, a per-rendition track) addresses it by `id`
and verifies it by `sha256` and `prior`. A sink MUST check an entry's `sha256` against the entry's payload bytes
before it decodes or applies the entry: a selectable entry that does not match is
skipped (`component_digest_mismatch`), as an entry whose pin is refused is, and a
companion that does not match is not applied. An entry whose `id`, `sha256` or
`companion_of_id` is not a string (`null` is the same as absent) makes the
manifest malformed, as a mistyped pin member does (§3.1), and the reader rejects
the envelope.
- Rendition table: `u16 n_renditions | n × (u32 offset, u32 length)`,
offsets relative to the start of the payloads block. The table's
i-th entry corresponds to `renditions[i]` in the manifest.
@ -1169,17 +1500,38 @@ by which capability happens to be installed.
A v1.1-aware sink MUST:
1. Read the manifest, get the ordered list of `model_requirement`
entries.
2. Walk the list in order; pick the first entry whose capability the
sink either advertises OR whose declared backend the sink prefers
under its deployer policy.
1. Read the manifest and take its ordered `renditions` entries.
2. Walk the entries in order, and pick the first that is not a companion (below),
whose capability is dispatchable (§4), that the sink either advertises
OR whose declared backend the sink prefers under its deployer policy, whose
`sha256`, when present, matches its payload bytes, and whose `prior`, when the
entry carries one and a pin applies to its capability (§3.1), resolves at the
sink. An entry whose payload digest does not match, or whose pin is refused, is
skipped as if the sink did not advertise its capability, and the sink keeps the
reason. A deployer policy MAY reorder or restrict the entries the sink
considers; it MUST NOT pick an entry skipped for its digest or its pin.
3. Read the corresponding rendition's bytes from the payload block.
4. Dispatch as in v1.0 §4.
4. Dispatch as in v1.0 §4, with the entry's `capability` and `prior` in place of
`model_requirement.capability` and `conditioning.prior`, and no fallback. A
decode refused with `set_incomplete` (§3.1) is handled as a skipped entry:
continue at step 2 with the next entry.
5. If no entry is picked, the envelope is inert at this sink: return an error
that names, for each entry, its index and why it was skipped:
`not_selectable` (a companion, or a capability that is not dispatchable,
which step 2 never picks), `missing_capability`, `component_digest_mismatch`,
a pin reason (§3.1), or, where a deployer policy restricts step 2 to the entries §7's
`replicate` rule admits, the `replicate_*` reason the rule gives for an entry
it leaves out.
A v1.0 sink that encounters a `WAI2` envelope MUST refuse it cleanly
(unknown major). No silent fallback to the first payload — that would
hide a major-version mismatch.
A sink MUST NOT decode one entry's payload against the parameter set another
entry pins. The envelope's `intent` applies to the rendition the sink picks, and
§7's `replicate` rule reads that entry's capability and pin; a deployer policy
that serves only sound `replicate` claims MAY restrict step 2 to the entries that
rule admits.
A v1.0 sink that encounters a `WAI2` envelope MUST refuse it cleanly (unknown
magic, §2). No silent fallback to the first payload — that would hide the
version mismatch.
### Companion components (descriptors bound to a payload)
@ -1198,8 +1550,10 @@ revision registers no capability for immersive-audio mix metadata;
and travels as a rendition.
Selection: a sink picks a rendition among the
`role != "companion"` entries, then applies every companion whose
`companion_of` names the chosen component (e.g. the HDR10+ block for the AV1
`role != "companion"` entries by the algorithm above, then applies every companion
bound to the chosen component — by `companion_of_id` when the companion carries
one, otherwise every companion whose `companion_of` names the chosen component's
capability (e.g. the HDR10+ block for the AV1
video the sink picked). `pick`/`primary` never return a companion, nor a
component whose capability is not dispatchable (§4): a companion-only
capability, a candidate, a record or a derivation. A component whose capability
@ -1233,6 +1587,14 @@ A component with `role = "companion"` conforms only if:
layer (§5). Several descriptor companions MAY bind to one base, and a
descriptor MAY bind to several renditions of one capability.
A companion MAY carry `companion_of_id`, the `id` of the one entry it binds to.
That entry MUST exist, MUST have the capability the companion's `companion_of`
names, and MUST be a selectable rendition; the companion then binds to exactly that
entry, and rule 3's "at most one enhancement layer per base" is checked by entry.
`companion_of` keeps its meaning, so a sink that does not read `companion_of_id`
binds by capability: for an enhancement layer over two renditions of one base
capability it finds the base not unique, and does not apply the layer.
An envelope with a companion that breaks any of these does not conform (the
reference `WaiMulti::check_component_roles` reports the first break). A sink
MUST NOT apply a companion that breaks rule 1 or 2, nor one whose capability it
@ -1303,8 +1665,33 @@ envelope that uses any string a later revision registers.
- The `wai` manifest field is `MAJOR.MINOR`. A reader MUST reject an
unknown MAJOR. A reader MUST accept an unknown MINOR by ignoring
fields it does not understand, provided MAJOR matches.
- The container format (§2) and the manifest schema (§3) are frozen
for all of `1.x`.
- A reader MUST ignore a manifest member it does not understand, whatever the
MINOR. Earlier text required this only for an unknown MINOR, so a reader of an
earlier revision may instead reject an envelope that carries such a member,
which fails closed. Or it ignores the member, and then a pin fails open (§3.1).
- The container format (§2) is frozen for all of `1.x`. The manifest schema (§3,
and a §7.1 rendition entry) is frozen for all of `1.x` in this sense: no member
is removed or renamed, none changes type or meaning, and none becomes REQUIRED.
A MINOR revision MAY define an OPTIONAL member, at the top level or inside an
existing object, whose absence leaves every envelope's bytes and meaning as they
were, and the revision states what it means for a reader of an earlier
revision (above). This revision defines `conditioning.prior`, a rendition
entry's `prior`, `id`, `sha256` and `companion_of_id`, and the pin's members
`form`, `tier` and `supersedes` this way (§3.1, §7.1); envelopes that use them
keep `wai` `"1.0"` or `"1.1"`. The reference implementation wrote `prior`, with
`id`, `sha256`, `determinism` and `form`, before this revision defined it; an
envelope written that way keeps its bytes and its meaning, except as §3.1 lists.
- **The one exception to the freeze.** This revision makes one exception to the
sentence above, deliberately. A member it defines, carried with another JSON
type, makes the manifest malformed rather than being ignored, because a reader
that ignored a mistyped pin member could decode against a set the pin excludes
(a mistyped `form`, read as absent, turns a set-form pin into a form-less one).
The rule covers every member listed above, the rendition entry's `id`, `sha256`
and `companion_of_id` included, so one rule decides whether a manifest parses.
An envelope that carried such a value before the member was defined, and parsed
then, does not parse now; §3.1 lists it. Nothing in the envelope signals the
change: its `wai` value is the same, so a reader of this revision rejects the
whole envelope where an earlier reader ignored the member.
- New capabilities (§5) MAY be added in any MINOR revision. Existing
capabilities MUST NOT be redefined — once a capability string is
registered for a codec, its payload format is fixed forever.
@ -1330,6 +1717,9 @@ envelope that uses any string a later revision registers.
envelope stops conforming, and a reader of an earlier revision that reports
the new binding as not conforming does not apply that companion and can
still present its base.
- A registration of a capability that decodes against a parameter set also fixes
that set's shape (§5, §3.1). Changing one file to several narrows it and
requires a new MAJOR; several to one widens it, which a MINOR revision MAY do.
- Removing or repurposing an existing capability — changing its payload format
or its role — requires a new MAJOR.
- A MINOR revision MAY add a conformance rule that enforces a registration,
@ -1339,7 +1729,8 @@ envelope that uses any string a later revision registers.
and the revision lists every kind of envelope it makes non-conforming; they
still parse. The companion rules of §5 and §7.1 are such rules, and §7.1
lists the envelopes they affect (*Envelopes the companion rules make
non-conforming*).
non-conforming*). §3.1's pin rules are such rules, and §3.1 lists the
envelopes they affect.
- Registering a capability gives a string a payload format and a role that no
earlier revision gave it, and the rules of §5 and §7.1 then apply to it. An
envelope that used the string before then used a capability no revision

36
wai/byte-exact-conformance/src/int_video_golden.rs
View file

@ -21,8 +21,34 @@ use std::path::PathBuf;
use serde_json::Value;
use sha2::{Digest, Sha256};
use crate::int_video::{decode, fnv1a};
/// The clips' SHA-256 conformance digests (SPEC §7 *Digests*): the SHA-256 over the
/// frames' 32-byte SHA-256 digests in output order, frames as RGB8. The FNV-1a
/// values beside them locate a divergence; these are what a conforming decode
/// matches.
const SYNTHETIC_CLIP_SHA256: &str = "5afd6477d09525b7db4485027588ba3bc43de3d29c996eb57f2d3274bf8309f9";
const REAL_CLIP_SHA256: &str = "e6724ca214694e012c3e2b49df4e116a0a7e150435a12d7b144560ce2d5c7bdf";
/// Every frame's SHA-256 against the sidecar entry `key`, and the clip digest.
fn assert_sha256(frames: &[Vec<u8>], key: &str) -> String {
let side = jload("sha256.json");
let want = side[key]["frames_sha256"].as_array().expect("frames_sha256");
assert_eq!(want.len(), frames.len(), "{key}: frame count in sha256.json");
let mut all = Vec::with_capacity(frames.len() * 32);
for (i, f) in frames.iter().enumerate() {
let d = Sha256::digest(f);
let hex: String = d.iter().map(|b| format!("{b:02x}")).collect();
assert_eq!(hex, want[i].as_str().expect("frame digest"), "{key}: frame {i} SHA-256");
all.extend_from_slice(&d);
}
let clip: String = Sha256::digest(&all).iter().map(|b| format!("{b:02x}")).collect();
assert_eq!(clip, side[key]["clip_sha256"].as_str().expect("clip_sha256"), "{key}: clip SHA-256");
clip
}
fn vectors() -> PathBuf {
PathBuf::from(env!("CARGO_MANIFEST_DIR")).join("../wai-rs/tests/vectors/int_video")
}
@ -45,7 +71,7 @@ fn hex16(s: &str) -> [u8; 8] {
/// Decode a committed clip and assert every frame hash and the clip hash.
/// Returns the clip hash so callers can pin it verbatim.
fn assert_clip(bitstream: &str, conf: &str) -> String {
fn assert_clip(bitstream: &str, conf: &str, key: &str) -> (String, String) {
let bs = jload(bitstream);
let conf = jload(conf);
let clip = decode(&bs).expect("decode must succeed on every architecture");
@ -71,14 +97,15 @@ fn assert_clip(bitstream: &str, conf: &str) -> String {
}
let clip_hash = format!("{:016x}", fnv1a(&all));
assert_eq!(clip_hash, conf["clip_hash"].as_str().expect("clip_hash"), "clip hash");
clip_hash
(clip_hash, assert_sha256(&clip.frames, key))
}
/// The synthetic clip: 5 frames of 48×48 RGB, whole-pel motion.
#[test]
fn int_video_synthetic_clip_is_byte_identical() {
let clip = assert_clip("int_video_bitstream.json", "int_video.json");
let (clip, sha) = assert_clip("int_video_bitstream.json", "int_video.json", "int_video");
assert_eq!(clip, "27573cdba1ad6628");
assert_eq!(sha, SYNTHETIC_CLIP_SHA256);
println!("int_video (synthetic, 5x48x48): clip {clip}");
}
@ -97,7 +124,8 @@ fn int_video_real_footage_exercises_the_half_pel_predictor() {
still pass while silently covering only the whole-pel path"
);
let clip = assert_clip("video_real_bitstream.json", "video_real.json");
let (clip, sha) = assert_clip("video_real_bitstream.json", "video_real.json", "video_real");
assert_eq!(sha, REAL_CLIP_SHA256);
assert_eq!(clip, "1ec1f0fa7da0d70f");
println!("int_video (real footage, 8x96x96, half-pel): clip {clip}");
}

16
wai/byte-exact-conformance/src/neural_keyframe_golden.rs
View file

@ -22,6 +22,8 @@ use std::path::PathBuf;
use serde_json::Value;
use crate::int_hyper_synth::{decode_conditional, CondBitstream, CondEntropy, IntModel};
use sha2::Digest;
use crate::int_video::{decode_pframes, fnv1a};
fn vectors() -> PathBuf {
@ -81,6 +83,20 @@ fn the_neural_keyframe_clip_is_byte_identical() {
let clip_hash = format!("{:016x}", fnv1a(&all));
assert_eq!(clip_hash, conf["clip_hash"].as_str().expect("clip_hash"));
assert_eq!(clip_hash, "a5f53d477999d159");
// The SHA-256 conformance digests (SPEC §7 *Digests*), from the vector's sidecar.
let side = jload("int_video/sha256.json");
let want = side["video_nk"]["frames_sha256"].as_array().expect("frames_sha256");
assert_eq!(want.len(), clip.frames.len());
let mut digests = Vec::with_capacity(clip.frames.len() * 32);
for (i, f) in clip.frames.iter().enumerate() {
let d = sha2::Sha256::digest(f);
let hex: String = d.iter().map(|b| format!("{b:02x}")).collect();
assert_eq!(hex, want[i].as_str().expect("frame digest"), "frame {i} SHA-256");
digests.extend_from_slice(&d);
}
let clip_sha: String = sha2::Sha256::digest(&digests).iter().map(|b| format!("{b:02x}")).collect();
assert_eq!(clip_sha, side["video_nk"]["clip_sha256"].as_str().expect("clip_sha256"));
assert_eq!(clip_sha, "4a81f0c94ccd938a7ca80df203303d5735d9dfcdfa150da155d9f2e4ca283e8f");
println!("int_hyper keyframe clip ({} frames {h}x{w}): {clip_hash}", clip.frames.len());
}

23
wai/extensions/determinism-tiers.md
View file

@ -8,7 +8,9 @@
> neural capability promises, what a receipt can hash under each, which WAI
> capabilities are registered at each and at what cost, and what the repository
> demonstrates today.
> No new wire format; a vocabulary + conformance framing over §7.
> One optional manifest member carries a tier name: a pin's `tier`
> ([SPEC.md §3.1](../SPEC.md)). Otherwise a vocabulary and conformance framing
> over §7.
> Keywords **MUST**, **MUST NOT**, **SHOULD**, **MAY** are RFC 2119/8174.
## 1. Why tiers
@ -259,12 +261,17 @@ thing.
`NeuralEntropyExact` capabilities at `entropy-consistency`, and `NeuralFloat`
capabilities claim neither. §5 lists WAI's capabilities at each tier. A
capability MAY declare `entropy-consistency` when it only guarantees the entropy
stage.
stage. A pin MAY also state a tier in the manifest, as `tier` (SPEC §3.1), from
the closed vocabulary `decode-equivalence`, `entropy-consistency`, or `none` for
neither; an encoder writes only those tokens, and a sink reads any other token
as `none`. A stated tier above the capability's registered tier does not conform
(`tier_exceeds_class`): the `replicate` verdict does not rest on it.
- **Over a neural capability, `replicate` promises `decode-equivalence` (SPEC
§7).** An envelope with `intent = replicate` over a neural capability MUST be
over a capability registered at `decode-equivalence`, which SPEC §7 admits only
by the integer construction: an integer / fixed-point decode path with no float
operation.
operation. Such an envelope MUST pin the parameter set of that capability with a
pin that declares `decode-equivalence` (SPEC §3.1, §7).
`entropy-consistency` MUST NOT back `replicate`. A neural decode that derives
its entropy parameters in float meets neither tier; one that reads them from the
codestream, or derives them in integer arithmetic, and runs its later stages in
@ -299,10 +306,10 @@ thing.
classes, which are not neural, are not refused by this rule). The class does
not depend on anything the sender declares, so a `replicate` envelope over `wai.image.jpegai` or a float
codec is refused even when its prior declares an integer contract. Then, for an
integer-exact capability, the pinned prior: `Prior::determinism_class` must
classify its sender-declared contract string as integer-exact. That
classification is a substring match (a contract containing `int`, `fixed` or
`integer` and none of `f32`, `f64`, `float`), so on its own it verifies the
integer-exact capability, the pin: it must declare `decode-equivalence` — by
`tier`, or, for a pin without `tier`, by the reading of its `determinism` string
that SPEC §7 fixes, a substring match (a contract containing `int`, `fixed` or
`integer` and none of `f32`, `f64`, `float`). Either way it verifies a
declaration, not the arithmetic. Neither step inspects the decode path a sink
actually runs; for `wai.neural.int_synth` and `wai.neural.int_mlicpp`, the
`NeuralIntegerExact` capabilities the reference neural dispatch decodes, there is
@ -501,7 +508,7 @@ task list has no Windows task after that day. The Windows results below are hist
| `float_repro`, the decode's `exp` and rounding: the same bits over a fixed grid of inputs; `exp` within 2 ULPs of the host math library (1 ULP measured); rounding equal to the standard library's | library unit tests under the library-test step of the conformance workflow (`neural_int`), which has run them on its macOS, Linux x86_64 and Linux arm64 runners since 2026-09-28. The pinned-bits and rounding tests never skip. The host comparison asserts the host library's accuracy as much as this module's. Its gate selects by target, not by host: it runs on macOS arm64, on Linux x86_64 and arm64 with glibc, and on WASI, and is reported as ignored, with the reason, elsewhere, Windows included. On the source of the change that added this row it was run against these hosts' `exp`, with a worst case of 1 ULP on each: the platform math library on macOS arm64; the WASI C library under wasmtime; glibc 2.36 on Linux arm64, and glibc 2.35 in the Ubuntu 22.04 image the workflow's Linux arm64 runner used on 2026-09-28; glibc 2.36 on Linux x86_64 under user-mode emulation, against three of the four `exp` variants glibc chooses among by processor feature (the variant each run used follows from the features glibc's loader reported active): SSE2 (QEMU 7.0, whose emulated processor reports none of AVX, FMA, AVX2 and FMA4), AVX (QEMU 7.2, `-cpu max` with FMA and AVX2 masked by glibc's `hwcaps` tunable, which leaves AVX active) and FMA (QEMU 7.2, `-cpu max`). On a separate probe of 20 million inputs, the SSE2 and AVX variants returned the same bits, and the FMA variant different bits for some of them. Not compared: the FMA4 variant, which glibc chooses on a processor that has FMA4 but not both FMA and AVX2, and which neither emulator offers. Not run on x86_64 hardware or on other glibc versions, where the gate still runs it; a failure there would say that the host's `exp` and this one are more than 2 ULPs apart, not that `float_repro`'s bits changed | `wai-rs/src/float_repro.rs` tests |
| The exact decode path calls no platform math library | two checks. Clippy's `disallowed_methods` (`wai-rs/clippy.toml`), denied in each module on the decode path, rejects any use of a listed float method, including `f64::exp(x)` and `.map(f64::exp)`. `tests/jpegai_decode_symbols.rs` lists the symbols of an executable that links only the decoder's entry points and weight loaders. It rejects any C math-library function the executable imports, and any it defines other than the round-to-integral functions (`floor`, `round`, `rint` and the like), whose results IEEE 754 fixes. It allows those definitions because a definition shows what the linker kept, not what the decode calls: linked with GNU ld 2.40 for x86_64 Linux, the debug executable defined the compiler builtins' `floor`, though nothing in it referred to `floor`. On the source of the change that added this row, both checks pass on macOS arm64, the symbol check in debug and release builds. The symbol check also passes, in debug builds, on Linux arm64 (the container above) and on Linux x86_64 under user-mode emulation, linked with GNU ld (reporting that `floor` as allowed) and with LLD. With a platform `exp` put back as `f64::exp(…)`, both fail on macOS arm64 while the decoder's unit tests still pass, and the symbol check fails on Linux x86_64 with GNU ld. With a float `%` put into the decode, the symbol check fails on macOS arm64 and clippy, which does not see operators, passes. The conformance workflow runs both on its unix legs, on the forge's runners since 2026-09-28 | `wai-rs/clippy.toml`, `wai-rs/tests/jpegai_decode_symbols.rs`, `.github/workflows/wai-conformance.yml` |
| `wai.image.jpegai` scope: the reference codestream is accepted; copies with β, `model_id`, the colour transform or an LSBS flag changed are refused; a flush-to-zero or round-toward-zero environment is detected. The float ONNX path calls the same check | library unit tests over a committed copy of the reference codestream, which never skip and fall under the library-test step of the conformance workflow (`neural_int`); the environment test runs on arm64 only. Run on the four platforms above, on the source of the change that added the cross-platform row (the environment test on macOS arm64 and Linux arm64). The float path's refusal (β and `model_id` edits) is tested only with its ONNX bundle supplied (`jpegai_scope_ort.rs`, skips otherwise) | `wai-rs/src/jpegai_decode.rs` tests, `wai-rs/tests/vectors/jpegai/` |
| SPEC §7's replicate rule: the registry class decides first, so an integer-declared prior over `wai.image.jpegai` or a float codec is refused; `wai.image.jpegai` is the only `NeuralEntropyExact` capability; the standard codecs sit in the classes SPEC §7 lists, and the tolerance and unclassified ones are not refused; an encoder that pins no prior (`wai wrap`) writes `replicate` only over the standard classes, for the capability and for any declared fallback, or, in the `WAI2` companion form, for the base payload's capability and each companion's | library unit tests (`codecs`, `container`) that never skip and fall under the same library-test step. The strict-decode test in `tests/neural_native_decode.rs` needs the `neural` feature; `wai-neural-gates.yml` runs it on the Linux runner, in an Ubuntu 24.04 image because the `neural` test binary does not link in that runner's default 22.04 image. The workflow has run on the forge and passed on every run since 2026-09-28; before that its job passed under `forgejo-runner exec` on the Linux runner's host (Linux x86_64, 2026-09-27). The `wai` binary no longer needs the `codecs` feature, so the tests that run `wai wrap` (`cli_wrap_intent.rs`, and `cli_registry.rs` for the candidate refusal and the registry export) run in the conformance workflow's carriage step on every leg; this row records no forge run of them yet. The C envelope packers emit through `Wai::to_emit_bytes` and `WaiMulti::to_emit_bytes`, whose candidate refusal a library test checks with no feature (`the_emit_path_refuses_a_candidate`, in the library-test step); the C wrappers themselves are compiled only with the `codecs` feature, which no CI job builds. The browser writers' refusal runs in `wai-web-wasm.yml` (`verify_pack.mjs`) | `wai-rs/src/codecs/mod.rs`, `container.rs` tests; `wai-rs/tests/neural_native_decode.rs`, `cli_wrap_intent.rs`, `cli_registry.rs`; `wai-web/demo/verify_pack.mjs` |
| SPEC §7's replicate rule: the registry class decides first, so an integer-declared prior over `wai.image.jpegai` or a float codec is refused; `wai.image.jpegai` is the only `NeuralEntropyExact` capability; the standard codecs sit in the classes SPEC §7 lists, and the tolerance and unclassified ones are not refused; an encoder that pins no prior (`wai wrap`) writes `replicate` only over the standard classes, for the capability and for any declared fallback, or, in the `WAI2` companion form, for the base payload's capability and each companion's | library unit tests (`codecs`, `container`) that never skip and fall under the same library-test step. The strict-decode test in `tests/neural_native_decode.rs` needs the `neural` feature; `wai-neural-gates.yml` runs it on the Linux runner, in an Ubuntu 24.04 image because the `neural` test binary does not link in that runner's default 22.04 image. Its strict-decode job ran on the forge and passed on every run from 2026-09-28; before that the job passed under `forgejo-runner exec` on the Linux runner's host (Linux x86_64, 2026-09-27). The pin-aware tests since added to that workflow (its longer exact list of `neural_native_decode` tests, the step of model-free `neural` library tests, and the neural C ABI step, `tests/ffi_neural_pin.rs`) have no recorded forge run yet. The `wai` binary no longer needs the `codecs` feature, so the tests that run `wai wrap` (`cli_wrap_intent.rs`, and `cli_registry.rs` for the candidate refusal and the registry export) run in the conformance workflow's carriage step on every leg; this row records no forge run of them yet. The C envelope packers emit through `Wai::to_emit_bytes` and `WaiMulti::to_emit_bytes`, whose candidate refusal a library test checks with no feature (`the_emit_path_refuses_a_candidate`, in the library-test step); the C wrappers themselves build under the codec-free `ffi` feature, and `tests/ffi_envelope.rs` (`envelope_pack_refuses_a_candidate`, `envelope_select_never_picks_a_candidate`) runs in the conformance workflow's C ABI envelope step on every leg; this row records no forge run of that step yet. The browser writers' refusal runs in `wai-web-wasm.yml` (`verify_pack.mjs`) | `wai-rs/src/codecs/mod.rs`, `container.rs` tests; `wai-rs/tests/neural_native_decode.rs`, `cli_wrap_intent.rs`, `cli_registry.rs`; `wai-web/demo/verify_pack.mjs` |
| `wai.neural.int_hyper` and `wai.video.int_hyper` at full model scale (the pretrained `mbt2018_mean`, exported to integers): the Rust decode reproduces the export tool's integer reference byte for byte (synthesis, hyper-decoder, the whole conditional bitstream, an eight-frame clip), and costs at most 1 dB against the same model's float decode on one pinned photograph | `wai-sota-vectors.yml` regenerates the vectors on the runner from pinned inputs (the photograph's pixels and the checkpoint's bytes by SHA-256, the Python stack by version) and runs both tests with `WAI_REQUIRE_SOTA_VECTORS=1`, under which a missing vector fails instead of skipping. The job also fails on any skip banner in the test log, and runs `require_vectors_rule.rs`, which checks that without the switch a variable naming absent or incomplete inputs fails as well. Rust is checked against the export of the same run on the same machine: the vectors come from float computation, and some of them differed between machines and between runs, so none of their hashes is pinned and none is compared across machines. The workflow has run on the forge's Linux runner (x86_64 hardware) and passed on every run since 2026-09-28 | `wai-rs/tests/int_hyper_synth.rs`, `int_video_sota_keyframe.rs`, `require_vectors_rule.rs`; `.github/workflows/wai-sota-vectors.yml` |
| `wai.audio.int_codec`, `wai.splat.int_codec` | committed vectors that never skip. Their frozen hashes are the output of an independent Python integer reference, computed when the vectors were exported (`tools/wai_int_audio_export.py`, `tools/wai_int_splat_export.py`); no Python runs in CI. The conformance workflow's fast-kernel steps enable `audio_int` and `splat_int` and, in debug and release builds, decode both vectors against those hashes under the reference kernels and every fast-kernel policy (`tests/int_kernels_vectors.rs`); that passed on the forge's Linux x86_64, Linux arm64 and macOS arm64 runners for the landing chain's head `7cfcecfa` (run 745, 2026-09-30). The same steps also name `tests/int_audio.rs` and `tests/int_splat.rs`, which check each vector against the same hash and against its own metadata (sample count; attribute count, tensor size and grid side); Cargo declares each with its feature as a required feature, so a step that names it without that feature fails instead of running nothing. This row records no forge run of those two | `wai-rs/tests/int_audio.rs`, `int_splat.rs`, `int_kernels_vectors.rs`; `.github/workflows/wai-conformance.yml` |
| `wai.neural.int_mlicpp` on its committed vector (synthetic seeded weights; a 64×64 image, 8 latent channels in 4 slices): the decode reproduces the export tool's independent integer reference byte for byte — the RGB, and every scale bucket and symbol of the latent stream in coding order (512 symbols over all 16 buckets of the scale table, 47 of the symbols coded by the rANS escape) — under the reference kernels and under the fast kernels on 1, 2, 3, 5 and all available threads; one integer unit changed in each of four chosen parameters, one on each path that feeds a value (a hyper-decoder weight, an entropy-network weight, a mean-head bias, a synthesis weight), changes the image; a header whose shapes the model cannot produce, or on which a network would form a buffer over the cap, is refused before anything is allocated (`tests/int_mlicpp_preflight.rs`, which counts allocations); damaged payloads are refused or decode, and never panic in an overflow-checked build | measured 2026-09-29 with rustc 1.98.0 on the source of the change that added this row: macOS arm64 native (the wai-rs tests, with overflow checks on and off; the `byte-exact-conformance` golden, release) and wasm32-wasip2 under wasmtime 49.0.1 (the same tests and golden, release; on wasm the fast kernels run single-threaded as portable scalar code, and `usize` is 32-bit). All pinned hashes matched on both. Not measured: x86_64 — the development machine has no x86_64 translation layer installed, and an x86_64-apple-darwin build of the golden compiles but cannot execute there — nor Linux or Windows. The golden runs on every leg of the byte-exact matrix (`wai-conformance.yml`, `wai-byte-equality.yml`) and `tests/int_mlicpp.rs` on every leg of `wai-conformance.yml` in both profiles. Both passed on the forge's Linux x86_64, Linux arm64 and macOS arm64 runners for the landing chain's head `7cfcecfa` (run 745, 2026-09-30); there is no Windows leg. The dispatch through `decode_envelope_strict` (the prior pinning the one-file model bundle, the strict gate) runs in `wai-neural-gates.yml`. The weights are synthetic, so nothing here measures compression quality | `wai-rs/tests/int_mlicpp.rs`, `wai-rs/tests/int_mlicpp_preflight.rs`, `byte-exact-conformance/src/int_mlicpp_golden.rs`, `wai-rs/tests/neural_native_decode.rs`; `tools/wai_int_mlicpp_export.py` |

172
wai/extensions/prior-carriage.md
View file

@ -1,22 +1,25 @@
# WAI Extension: Prior Carriage
> Status: Draft. Lets a `Prior` declare the **wire form** of the artifact its
> digest covers, registers ISO/IEC 15938-17 (neural-network coding, "NNC") as
> that form, registers a **candidate** general-tensor payload, and fixes how an
> in-band signature inside such an artifact relates to WAI's digest. Reference
> impl: `wai-rs` — `container::Prior::{new, with_form, form, check_form,
> form_preserves_exactness}`, `codecs::prior_form`, `codecs::{CAP_PRIOR_NNC,
> CAP_TENSOR_NNC}`.
> Status: Draft. Registers the wire **forms** a pin's digest may cover —
> `wai.prior.set`, the listing of a parameter set ([SPEC.md §3.1](../SPEC.md)),
> and ISO/IEC 15938-17 (neural-network coding, "NNC") — registers a **candidate**
> general-tensor payload, and fixes how an in-band signature inside such an
> artifact relates to WAI's digest. The pin itself, its verification and its
> refusal reasons are SPEC.md §3.1. Reference impl: `wai-rs` —
> `container::{Prior, PinRefusal}`, `pin::{verify_bytes, verify, parse_listing,
> write_listing, listing_of_dir}`, `codecs::prior_form`,
> `codecs::{CAP_PRIOR_SET, CAP_PRIOR_NNC, CAP_TENSOR_NNC}`.
> Keywords **MUST**, **MUST NOT**, **SHOULD**, **MAY** are RFC 2119/8174.
## 1. A prior has a digest, and had no form
A neural capability decodes against a model the envelope does not carry
([SPEC.md §1](../SPEC.md)). WAI pins that model's identity: a `Prior` in the
conditioning names an `id`, a `sha256` over the artifact, and a `determinism`
contract. The reference sink recomputes the digest over the model it registered
and refuses to decode on a mismatch (`neural::ModelRegistry::resolve_checked`),
and `replicate` soundness is gated on the determinism contract (SPEC §7).
A neural capability decodes against a parameter set the envelope does not carry
([SPEC.md §1](../SPEC.md)). WAI pins that set's identity
([SPEC.md §3.1](../SPEC.md)): a pin names an `id`, a `sha256` over the whole
parameter set and a `determinism` description, and MAY state a `tier` and the set
it `supersedes`. A sink verifies the pin before it decodes, refuses a pin its sets
do not satisfy with a named reason, and `replicate` soundness is gated on the
declared tier (SPEC §7).
The digest fixes **which bytes**. Until now nothing fixed **what those bytes
are**: the sink hashes whatever single file it registered — a serialized graph,
@ -30,25 +33,33 @@ operand needs a form both ends can name.
```json
"prior": {
"id": "int_hyper/q4",
"sha256": "<lowercase hex SHA-256 over the artifact, in its declared form>",
"sha256": "<64 lowercase hexadecimal digits: the SHA-256 of the set, in its declared form>",
"determinism": "int-rans/p16+int-synth",
"form": "wai.prior.nnc"
"form": "wai.prior.set",
"tier": "decode-equivalence"
}
```
- `form` is OPTIONAL. Absent, the form is undeclared and the digest covers the
artifact as the sink registered it — the existing behaviour, unchanged, and a
manifest without `form` round-trips byte for byte.
one file that holds the parameter set; SPEC §3.1 admits that only for a
capability registered with a one-file set, and a sink refuses such a pin over
any other capability (`form_required`). A manifest without `form` round-trips
byte for byte.
- Present, `form` MUST be a registered prior form (§3), and `sha256` MUST be
computed over the artifact serialized in exactly that form: the coded
bitstream, not an unpacking, a transcoding, or a directory of its parts.
- A sink that cannot read the declared form MUST refuse the prior, as it refuses
a digest mismatch. It MUST NOT resolve the prior against an artifact in another
form, even one whose unpacked parameters compare equal: equality after
unpacking is a different fact from the digest the manifest pins.
- A declared form names a **self-contained** artifact. An NNC incremental-update
bitstream decodes only against its base, so it MUST NOT be pinned as a prior on
its own. No form is registered for a base-plus-updates chain.
computed over the parameter set serialized in exactly that form: for a coded
form, the coded bitstream, not an unpacking or a transcoding of it; for
`wai.prior.set`, the listing SPEC §3.1 defines, never the files' bytes
concatenated or an archive of them.
- A sink that does not know the declared form as a prior form refuses the pin
(`form_not_registered`), and one that cannot read it refuses it
(`form_unreadable`, SPEC §3.1). It MUST NOT resolve the pin against an artifact
in another form, even one whose unpacked parameters compare equal: equality
after unpacking is a different fact from the digest the manifest pins.
- A declared form names a **self-contained** parameter set. An NNC
incremental-update bitstream decodes only against its base, so it MUST NOT be
pinned on its own, and no form is registered for a base-plus-updates chain. A
pin names the self-contained set an update yields, and MAY name the set it
replaced in `supersedes` (SPEC §3.1).
- A form is a capability string. When a prior artifact travels in a WAI envelope
it does so under its form's capability, so the envelope's per-object content
hash and the `Prior` digest cover the same bytes.
@ -57,10 +68,11 @@ operand needs a form both ends can name.
| capability | payload | status |
|---|---|---|
| `wai.prior.nnc` | the parameters of a trained network as a bitstream conforming to **ISO/IEC 15938-17:2024** (NNC, 2nd edition: parameters and their incremental updates, DeepCABAC entropy coding) | registered; the only prior form |
| `wai.prior.set` | the listing of a parameter set of one or more files (SPEC §3.1) | registered; WAI-native; established exact (§5) |
| `wai.prior.nnc` | the parameters of a trained network as a bitstream conforming to **ISO/IEC 15938-17:2024** (NNC, 2nd edition: parameters and their incremental updates, DeepCABAC entropy coding) | registered; a coded form |
| `wai.tensor.nnc` | general tensorial data — split-inference feature maps, 3D/4D scene parameters — coded with NNC | **candidate**: payload format not fixed |
Both are sink-supplied: `wai-rs` contains no NNC reader or writer. The
The NNC entries are sink-supplied: `wai-rs` contains no NNC reader or writer. The
standard's reference software is itself standardized, in **ISO/IEC 15938-18**,
*Conformance and reference software for compression of neural networks* (1st
edition 2023, 2nd edition 2025). That part also supplies the conformance
@ -71,6 +83,12 @@ an implementation to use it.
Registry classification (`codecs::capability_*`):
- **`wai.prior.set`**: fidelity tier lossless and replicate class
`StandardDefined` — a listing is its own output, and SPEC §3.1 fixes its bytes
exactly; licence royalty-free (WAI-native, Apache-2.0 reference).
For the NNC entries:
- **Fidelity tier: lossy, by primary role.** The registry's tier names a
capability's primary role, as it does for AVIF, which also has a lossless mode;
it is not a statement that every NNC artifact approximates its source. NNC's
@ -185,8 +203,10 @@ reasons:
published editions and for any later one.
- Therefore an envelope with `intent = replicate` MUST NOT rest on a prior whose
declared form is not established exact. The registry lists the forms that are
established (`codecs::prior_form::form_reconstruction_is_exact`), and **none is
listed today**. In the reference implementation the rule is part of
established (`codecs::prior_form::form_reconstruction_is_exact`). It lists one,
`wai.prior.set`: reading a set through its listing compares digests and hands
the decoder each file's bytes unchanged, so every sink that holds the set reads
the same parameters. No coded form is listed. In the reference implementation the rule is part of
`Prior::is_cross_hardware_exact`, which requires `Prior::form_preserves_exactness`
as well as an integer-exact contract. Every `replicate` check reads that
predicate, so `container::replicate_is_sound` and `Wai::replicate_is_sound`
@ -228,39 +248,71 @@ WAI's binding sits beside the standard's, never in place of it.
## 7. Reference implementation
- `container::Prior::form` (field, serde-skipped when absent) and
`Prior::form()`; `Prior::new(id, sha256, determinism)`, which leaves the form
absent, and `Prior::with_form(form)`. The new public field breaks Rust code
outside the crate that builds a `Prior` with a struct literal; the CHANGELOG
(v0.4.0) states the break, the fix, and why `Prior` is not
`#[non_exhaustive]`; `Prior::check_form(readable) -> Result<(), PriorFormError>`
(`NotAPriorForm` / `Unreadable`); `Prior::form_preserves_exactness()`, which
`Prior::is_cross_hardware_exact()` requires alongside an integer-exact contract.
`container::replicate_prior_refusal(intent, prior) -> Option<PriorRefusal>`
gives the reason: `PriorRefusal::FormNotExact(form)` when an integer-exact
contract declares a form not listed as exact, with a message that says whether
the form is registered at all. `container::replicate_is_sound` and
`Wai::replicate_is_sound` hold exactly when it returns `None`, and a unit test
checks that the two agree.
- `codecs::prior_form::{prior_forms, is_prior_form,
form_reconstruction_is_exact}`; `codecs::{CAP_PRIOR_NNC, CAP_TENSOR_NNC}` with
their tier and licence entries. They have no replicate-class entry of their own:
a unit test holds their class equal to MPEG-I Haptics' (§3).
- `neural::ModelRegistry::resolve_checked` runs `check_form` against
`neural::READABLE_PRIOR_FORMS`, which is empty: the reference sink refuses any
prior that declares a form. `neural::decode_envelope_strict` reports the
`PriorRefusal` as the reason for `UnsoundReplicate`.
- CI: the library unit tests (`wai-conformance`) cover `check_form`,
`form_preserves_exactness`, every `PriorRefusal`, and the form rule in each
`replicate` check (`Prior::is_cross_hardware_exact`, both `replicate_is_sound`
functions and `replicate_prior_refusal`). The two model-free tests of the
neural-sink wiring (`resolve_checked_verifies_prior_hash`,
`strict_decode_gates_unsound_replicate`) are configured to run in
`wai-neural-gates`, in an Ubuntu 24.04 container image. That job has not yet
run on a runner.
- `container::Prior` has the members of SPEC §3.1; `form`, `tier` and
`supersedes` are serde-skipped when absent, so a pin written without them
round-trips byte for byte, and a member this revision does not define is kept
(`extra`). `Prior` is `#[non_exhaustive]`: it is built with `Prior::new` and
the `with_form`, `with_tier` (a typed `codecs::DeclaredTier`, so the crate
writes only a registered token) and `with_supersedes` builders, so a member a
later revision adds does not break code that builds one. `Prior::check_syntax`
applies SPEC §3.1 check 1, `Prior::supersedes_finding` the `supersedes` rule,
and `Prior::declared_tier` reads `tier` or, without it, the reading SPEC §7
fixes (`Prior::determinism_class`). `container::PinRefusal::code()` is the
reason SPEC §3.1 names. `Wai::check_pin` and `WaiMulti::check_pins` report what
a verifier can decide without holding a parameter set.
- `pin::verify_bytes` runs SPEC §3.1's checks in order over parameter sets held in
memory (a browser sink calls it); `pin::verify` reads held files and listings
from disk, keeps the bytes it hashed, and calls it. `pin::parse_listing`,
`pin::write_listing` and `pin::listing_of_dir` read and write a listing.
`pin::READABLE_FORMS` holds `wai.prior.set`: the reference sink reads no coded
form.
- `neural::ModelRegistry` holds, for each capability, its registered path and any
further paths given to `hold`. For a form-less pin it hashes each held file —
for `wai.neural.int_synth`, the `int_synth.json` beside it. For a
`wai.prior.set` pin it takes every `*.sha256` file directly inside each held
path's directory (the path itself if it is a directory) as a listing rooted
there.
- `container::Wai::dispatch_verified` applies SPEC §4 steps 3–6 and
`container::WaiMulti::pick` SPEC §7.1 steps 2 and 5, each with a pin verifier
the caller supplies. `neural::decode_envelope` and `neural::decode_multi`
decode through them; their strict variants apply SPEC §7 to the capability they
decode through.
- The decoders read parameters only through `neural::ParamSource`, which hands a
pinned decode the verified bytes; ONNX sessions are built from those bytes, and
a file the verified set lacks is refused with `set_incomplete`.
`wai.neural.bmshj2018` caches its CDF tables by content digest.
- The C ABI (`ffi`) returns each manifest as its original bytes, carries a `WAI2`
rendition's identity and pin in the rendition table, and reports a refused pin
as `WAI_ERR_PRIOR` with its reason first in `wai_last_error`;
`wai_neural_decode_ex` decodes a `WAI1` or `WAI2` envelope against a registry of
held sets. The browser crate (`wai-web`) wraps the same types and exposes `pick`,
`companionsOf` and `verifyPin` to JS.
- CI: the library unit tests cover `container`, `codecs` and `pin`;
`wai_pin_vectors verify ../pin-conformance` checks the corpus, and
`pin-conformance/pin_verify.py`, which shares no code with `wai-rs`, checks it
independently; the model-free neural-sink and neural C ABI tests, and the
browser crate's tests (natively and under wasm32), are configured to run in CI.
- Not implemented: an NNC reader or writer, and any cross-decoder measurement of
NNC reconstruction.
The files the reference decoders read from a set (informative). A set built for
another decoder of the capability may name other files; a decode that needs a
file the set lacks is refused with `set_incomplete`.
| capability | shape | files the reference decoder reads from a set |
|---|---|---|
| `wai.neural.int_mlicpp` | one file | `model.wmb` |
| `wai.neural.int_synth` | one file | `int_synth.json` |
| `wai.neural.encodec32`, `.dac`, `.mimi`, `.wavtokenizer`, `.snac`, `.video_bmshj2018` | one file | `decoder.onnx` |
| `wai.neural.bmshj2018` | several | `decoder.onnx`, `cdfs_q<q>.json` |
| `wai.neural.mbt2018_mean` | several | `hyper_cdfs_q<q>.json`, `hyper_synthesis.onnx`, `gauss_cdfs_q<q>.json`, `synthesis.onnx` |
| `wai.neural.elic` | several | the `mbt2018_mean` files, and `context.onnx` |
| `wai.neural.mlicpp`, `.mlicv2`, `.cmic` | several | the `mbt2018_mean` files, and `entropy.onnx` |
| `wai.neural.glc` | several | `hyper_cdfs_q<q>.json`, `hyper_entropy.onnx`, `context_entropy.onnx`, `synthesis.onnx` |
| `wai.neural.dcvc_rt`, `.dcvc_fm` | several | `hyper_cdfs_q<q>.json`, `hyper_synthesis.onnx`, `gauss_cdfs_q<q>.json`, `temporal_decoder.onnx` |
| `wai.image.jpegai` | several | exact path: `norm.json`, `weights/{hs,hyp,synth}_{y,uv}.json`, `weights/mcm{0..3}_y.json`, `weights/icci.json`. Float path: `hyper_scale_decoder_int_weights.json`, `model_constants.json`, `jpegai_normative_tables.json`, `me-tANS/bounds.csv`, `jpegai_factorized_cdfs.json`, `jpegai_scaler_vec.json`, `jpegai_dequant_consts.json`, `model/CCS_SGMM/tools_0/model_{y,uv}/common_modules/hyper_decoder.onnx`, `model/CCS_SGMM/tools_0/model_y/common_modules/MCM/stage{0..3}.onnx`, `model16/CCS_SGMM/tools_0/model_{y,uv}/synthesis16.onnx`, `e2e/pfchain/icci_{y_m6,uv_m3}.onnx`. A set holding only the exact path's files decodes through that path; where that path refuses (a non-default floating-point environment), the float path needs files the set lacks, and the decode is refused with `set_incomplete` instead of running on unverified files. |
| `wai.neural.int_hyper`, `wai.video.int_hyper`, `wai.audio.int_codec`, `wai.splat.int_codec`, `wai.generate.spectral` | several, until an envelope decode path registers its files | — (no envelope decode path in the reference sink) |
## 8. Sources
- ISO/IEC 15938-17:2022 and ISO/IEC 15938-17:2024, *Compression of neural

10
wai/neural-conformance/README.md
View file

@ -26,9 +26,15 @@ macOS arm64) on every push that touches it.
weights/bias as `i32`, stride, requantize shift, integer ReLU flag).
- `vectors/<name>/latent.bin` — the wire payload (12-byte header +
channel-major `int8` latents).
- `sha256.json` — per vector, the SHA-256 of its decoded output as `RGB8`
(SPEC §7, *Digests*): the conformance digest.
- `expected_hashes.json` — per vector: `s_log2`, `in_channels`, `layers`,
`width`, `height`, and `rgb_hash` (domain-prefixed BLAKE3 of the decoded
RGB — the conformance quantity).
`width`, `height`, and `rgb_hash`, a domain-prefixed BLAKE3 of the decoded
RGB, kept unchanged as a divergence diagnostic.
The vectors keep their unpinned `<dir>/model` convention. A sink that pins an
`int_synth` parameter set pins `int_synth.json`, the one file that holds it
(SPEC §5).
Synthetic vectors (realistic multi-layer decoders, distinct shapes) — model
weights/latents authored from a fixed integer SplitMix64 sequence, so `gen`

18
wai/neural-conformance/sha256.json Normal file
View file

@ -0,0 +1,18 @@
{
"gradient": {
"buffer": "RGB8: rows top to bottom, pixels left to right, R, G, B bytes",
"sha256": "ce0d4833e42d670daf90aab70f0ea255d1a0610d3984845ea61dfe2b43db7cd4"
},
"bloom": {
"buffer": "RGB8: rows top to bottom, pixels left to right, R, G, B bytes",
"sha256": "14d16a169b0fed52b52f0f38f30af1751e22072a87852a654838f48f299d49bd"
},
"facade": {
"buffer": "RGB8: rows top to bottom, pixels left to right, R, G, B bytes",
"sha256": "fe3578908ce45680c053fdd589c033e50ae0045eaa24330356603fa325becdc0"
},
"photo": {
"buffer": "RGB8: rows top to bottom, pixels left to right, R, G, B bytes",
"sha256": "2a3db89614cd632829d873cf59d6300b0e49c5647a9ede1115719a6604d42115"
}
}

2
wai/pin-conformance/.gitattributes vendored Normal file
View file

@ -0,0 +1,2 @@
# Hashed files: a line-ending conversion on checkout would change every digest.
* -text

43
wai/pin-conformance/README.md Normal file
View file

@ -0,0 +1,43 @@
# pin-conformance — the parameter-set pin
Vectors for the parameter-set pin (SPEC.md §3.1, §4, §7, §7.1).
**Reason-equivalence.** A conforming verifier reports exactly the declared outcome
for every case:
- `cases.json` — given a pin, a capability, the forms the sink reads (default
`wai.prior.set`) and the artifacts it holds, the outcome `ok` or the one reason
SPEC §3.1 names: `pin_malformed`, `form_not_registered`, `form_unreadable`,
`form_required`, `prior_not_held`, `digest_superseded`, `set_malformed`,
`part_mismatch`. A held artifact is a `file`, or a `listing` together with the
`root` its paths are read below. Paths are relative to this directory.
- `selection.json` — given a `WAI2` manifest, its payloads, the capabilities the
sink advertises and the one-file parameter sets it holds, the rendition SPEC §7.1
picks and the reason for each entry it skips (`not_selectable`,
`missing_capability`, `component_digest_mismatch`, or a pin reason); when nothing
is picked, the reason for every entry. Where stated, also the companions applied
with the pick, and the first component-role finding.
- `static.json` — the first finding a verifier makes about a manifest's pins
without holding a parameter set (`prior_not_applicable`, `pin_malformed`,
`tier_unregistered`, `supersedes_malformed`, `form_not_registered`,
`form_required`, `tier_exceeds_class`, a `replicate_*` reason), or `ok`. For a
`WAI2` manifest the finding is `[entry index, reason]`.
**Listings.** Every `listings/<name>.sha256` is, byte for byte, the listing of
`sets/<name>/`: regular files, names that begin with `.` left out, sorted by path as
bytes. A common checksum utility checks a set against its listing
(`cd sets/demo && sha256sum -c ../../listings/demo.sha256`). Every file under
`listings/bad/` breaks one listing rule and is `set_malformed`.
Files are compared as bytes; `.gitattributes` turns off line-ending conversion.
**Verifiers.**
- `wai_pin_vectors verify pin-conformance` (wai-rs) runs every file through the
reference implementation's in-memory pin core (`wai::pin::verify_bytes`), its
selection (`WaiMulti::pick`) and its static checks (`Wai::check_pin`,
`WaiMulti::check_pins`). It runs in both corpus loops of the conformance workflow.
`wai_pin_vectors generate pin-conformance` rewrites the listings from the sets.
- `pin_verify.py` checks `cases.json` and `listings/` with the Python standard
library alone, re-implementing the listing rules and the ordered checks from the
SPEC text. It shares no code with wai-rs.

481
wai/pin-conformance/cases.json Normal file
View file

@ -0,0 +1,481 @@
{
"about": "SPEC.md \u00a73.1: the reason a conforming verifier reports for each pin, capability and set of held artifacts. Paths are relative to this directory; a listing's files are read below its root. readable defaults to [\"wai.prior.set\"].",
"cases": [
{
"name": "one_file_legacy",
"capability": "wai.neural.int_mlicpp",
"pin": {
"id": "conformance",
"sha256": "7ef44f3b1fc4a2df6c3e3b68460f595cae3d31cb7f9018699e29e738340272a4",
"determinism": "int-rans/p16+int/i64"
},
"held": [
{
"file": "../wai-rs/tests/vectors/int_mlicpp/model.wmb"
}
],
"expect": "ok"
},
{
"name": "uppercase_digest",
"capability": "wai.neural.int_mlicpp",
"pin": {
"id": "conformance",
"sha256": "7EF44F3B1FC4A2DF6C3E3B68460F595CAE3D31CB7F9018699E29E738340272A4",
"determinism": "int-rans/p16+int/i64"
},
"held": [
{
"file": "../wai-rs/tests/vectors/int_mlicpp/model.wmb"
}
],
"expect": "pin_malformed"
},
{
"name": "digest_63_digits",
"capability": "wai.neural.int_mlicpp",
"pin": {
"id": "conformance",
"sha256": "7ef44f3b1fc4a2df6c3e3b68460f595cae3d31cb7f9018699e29e738340272a",
"determinism": "int-rans/p16+int/i64"
},
"held": [
{
"file": "../wai-rs/tests/vectors/int_mlicpp/model.wmb"
}
],
"expect": "pin_malformed"
},
{
"name": "malformed_before_unreadable_form",
"capability": "wai.neural.int_mlicpp",
"pin": {
"id": "conformance",
"sha256": "7EF44F3B1FC4A2DF6C3E3B68460F595CAE3D31CB7F9018699E29E738340272A4",
"determinism": "int-rans/p16+int/i64",
"form": "wai.prior.nnc"
},
"held": [],
"expect": "pin_malformed"
},
{
"name": "form_unregistered",
"capability": "wai.neural.int_mlicpp",
"pin": {
"id": "conformance",
"sha256": "7ef44f3b1fc4a2df6c3e3b68460f595cae3d31cb7f9018699e29e738340272a4",
"determinism": "int-rans/p16+int/i64",
"form": "wai.prior.nope"
},
"held": [
{
"file": "../wai-rs/tests/vectors/int_mlicpp/model.wmb"
}
],
"expect": "form_not_registered"
},
{
"name": "form_is_a_payload_codec",
"capability": "wai.neural.int_mlicpp",
"pin": {
"id": "conformance",
"sha256": "7ef44f3b1fc4a2df6c3e3b68460f595cae3d31cb7f9018699e29e738340272a4",
"determinism": "int-rans/p16+int/i64",
"form": "wai.image.png"
},
"held": [
{
"file": "../wai-rs/tests/vectors/int_mlicpp/model.wmb"
}
],
"expect": "form_not_registered"
},
{
"name": "form_unreadable_nnc",
"capability": "wai.neural.int_mlicpp",
"pin": {
"id": "conformance",
"sha256": "7ef44f3b1fc4a2df6c3e3b68460f595cae3d31cb7f9018699e29e738340272a4",
"determinism": "int-rans/p16+int/i64",
"form": "wai.prior.nnc"
},
"held": [
{
"file": "../wai-rs/tests/vectors/int_mlicpp/model.wmb"
}
],
"expect": "form_unreadable"
},
{
"name": "form_required_several_files",
"capability": "wai.neural.bmshj2018",
"pin": {
"id": "conformance",
"sha256": "7ef44f3b1fc4a2df6c3e3b68460f595cae3d31cb7f9018699e29e738340272a4",
"determinism": "int-rans/p16+int/i64"
},
"held": [
{
"file": "../wai-rs/tests/vectors/int_mlicpp/model.wmb"
}
],
"expect": "form_required"
},
{
"name": "form_required_directory_capability",
"capability": "wai.image.jpegai",
"pin": {
"id": "conformance",
"sha256": "7ef44f3b1fc4a2df6c3e3b68460f595cae3d31cb7f9018699e29e738340272a4",
"determinism": "int-rans/p16+int/i64"
},
"held": [
{
"file": "../wai-rs/tests/vectors/int_mlicpp/model.wmb"
}
],
"expect": "form_required"
},
{
"name": "unregistered_capability_pin_is_verified",
"capability": "wai.neural.future",
"pin": {
"id": "conformance",
"sha256": "7ef44f3b1fc4a2df6c3e3b68460f595cae3d31cb7f9018699e29e738340272a4",
"determinism": "int-rans/p16+int/i64"
},
"held": [
{
"file": "../wai-rs/tests/vectors/int_mlicpp/model.wmb"
}
],
"expect": "form_required"
},
{
"name": "tampered_file_not_held",
"capability": "wai.neural.int_mlicpp",
"pin": {
"id": "conformance",
"sha256": "29bc4a3984b2ca82ab4656e459e7c34b5af9f507309bcf88ed7b00561692ff0a",
"determinism": "int-rans/p16+int/i64"
},
"held": [
{
"file": "sets/tiny-tampered/model.wmb"
}
],
"expect": "prior_not_held"
},
{
"name": "id_matches_digest_differs",
"capability": "wai.neural.int_mlicpp",
"pin": {
"id": "conformance",
"sha256": "402709318223af457631c917c5a1081cb8e380e1dde47c6aebd8f264e077f090",
"determinism": "int-rans/p16+int/i64"
},
"held": [
{
"file": "../wai-rs/tests/vectors/int_mlicpp/model.wmb"
}
],
"expect": "prior_not_held"
},
{
"name": "digest_superseded",
"capability": "wai.neural.int_mlicpp",
"pin": {
"id": "conformance",
"sha256": "402709318223af457631c917c5a1081cb8e380e1dde47c6aebd8f264e077f090",
"determinism": "int-rans/p16+int/i64",
"supersedes": {
"sha256": "7ef44f3b1fc4a2df6c3e3b68460f595cae3d31cb7f9018699e29e738340272a4"
}
},
"held": [
{
"file": "../wai-rs/tests/vectors/int_mlicpp/model.wmb"
}
],
"expect": "digest_superseded"
},
{
"name": "supersedes_malformed_is_ignored",
"capability": "wai.neural.int_mlicpp",
"pin": {
"id": "conformance",
"sha256": "7ef44f3b1fc4a2df6c3e3b68460f595cae3d31cb7f9018699e29e738340272a4",
"determinism": "int-rans/p16+int/i64",
"supersedes": {
"sha256": "7ef44f3b1fc4a2df6c3e3b68460f595cae3d31cb7f9018699e29e738340272a4"
}
},
"held": [
{
"file": "../wai-rs/tests/vectors/int_mlicpp/model.wmb"
}
],
"expect": "ok"
},
{
"name": "supersedes_bad_form_never_superseded",
"capability": "wai.neural.int_mlicpp",
"pin": {
"id": "conformance",
"sha256": "402709318223af457631c917c5a1081cb8e380e1dde47c6aebd8f264e077f090",
"determinism": "int-rans/p16+int/i64",
"supersedes": {
"sha256": "7ef44f3b1fc4a2df6c3e3b68460f595cae3d31cb7f9018699e29e738340272a4",
"form": "wai.image.png"
}
},
"held": [
{
"file": "../wai-rs/tests/vectors/int_mlicpp/model.wmb"
}
],
"expect": "prior_not_held"
},
{
"name": "successor_never_satisfies_predecessor",
"capability": "wai.neural.int_mlicpp",
"pin": {
"id": "conformance",
"sha256": "7ef44f3b1fc4a2df6c3e3b68460f595cae3d31cb7f9018699e29e738340272a4",
"determinism": "int-rans/p16+int/i64"
},
"held": [
{
"file": "sets/tiny/model.wmb"
}
],
"expect": "prior_not_held"
},
{
"name": "set_ok",
"capability": "wai.neural.int_mlicpp",
"pin": {
"id": "conformance",
"sha256": "85a05b0dac15566c5a657ddeee253eaa9f54ce852ae04924f5b04151901d9d1b",
"determinism": "int-rans/p16+int/i64",
"form": "wai.prior.set",
"tier": "decode-equivalence"
},
"held": [
{
"listing": "../wai-rs/tests/vectors/int_mlicpp/model.sha256",
"root": "../wai-rs/tests/vectors/int_mlicpp"
}
],
"expect": "ok"
},
{
"name": "set_ok_several_files",
"capability": "wai.neural.elic",
"pin": {
"id": "conformance",
"sha256": "88d72629c0c1d5adef7587afa6926405eefd0b7aaabf608483d8551d2536790f",
"determinism": "int-rans/p16+int/i64",
"form": "wai.prior.set"
},
"held": [
{
"listing": "listings/demo.sha256",
"root": "sets/demo"
}
],
"expect": "ok"
},
{
"name": "set_pin_is_not_a_file_pin",
"capability": "wai.neural.int_mlicpp",
"pin": {
"id": "conformance",
"sha256": "85a05b0dac15566c5a657ddeee253eaa9f54ce852ae04924f5b04151901d9d1b",
"determinism": "int-rans/p16+int/i64",
"form": "wai.prior.set"
},
"held": [
{
"file": "../wai-rs/tests/vectors/int_mlicpp/model.wmb"
}
],
"expect": "prior_not_held"
},
{
"name": "file_pin_is_not_a_set_pin",
"capability": "wai.neural.int_mlicpp",
"pin": {
"id": "conformance",
"sha256": "7ef44f3b1fc4a2df6c3e3b68460f595cae3d31cb7f9018699e29e738340272a4",
"determinism": "int-rans/p16+int/i64"
},
"held": [
{
"listing": "../wai-rs/tests/vectors/int_mlicpp/model.sha256",
"root": "../wai-rs/tests/vectors/int_mlicpp"
}
],
"expect": "prior_not_held"
},
{
"name": "set_malformed_crlf",
"capability": "wai.neural.int_mlicpp",
"pin": {
"id": "conformance",
"sha256": "7703c3e01c81277fa56da7220805d3e96ea5858f1f87b1f372136c3ce112351c",
"determinism": "int-rans/p16+int/i64",
"form": "wai.prior.set"
},
"held": [
{
"listing": "listings/bad/crlf.sha256",
"root": "sets/tiny"
}
],
"expect": "set_malformed"
},
{
"name": "part_mismatch",
"capability": "wai.neural.int_mlicpp",
"pin": {
"id": "conformance",
"sha256": "ac5a74765058dac5b44f90185af19852e70ce2922fcac2ace28590b633a9c376",
"determinism": "int-rans/p16+int/i64",
"form": "wai.prior.set"
},
"held": [
{
"listing": "listings/tiny.sha256",
"root": "sets/tiny-tampered"
}
],
"expect": "part_mismatch"
},
{
"name": "part_missing",
"capability": "wai.neural.int_mlicpp",
"pin": {
"id": "conformance",
"sha256": "ac5a74765058dac5b44f90185af19852e70ce2922fcac2ace28590b633a9c376",
"determinism": "int-rans/p16+int/i64",
"form": "wai.prior.set"
},
"held": [
{
"listing": "listings/tiny.sha256",
"root": "sets/demo"
}
],
"expect": "part_mismatch"
},
{
"name": "two_roots_one_intact",
"capability": "wai.neural.int_mlicpp",
"pin": {
"id": "conformance",
"sha256": "ac5a74765058dac5b44f90185af19852e70ce2922fcac2ace28590b633a9c376",
"determinism": "int-rans/p16+int/i64",
"form": "wai.prior.set"
},
"held": [
{
"listing": "listings/tiny.sha256",
"root": "sets/tiny-tampered"
},
{
"listing": "listings/tiny.sha256",
"root": "sets/tiny"
}
],
"expect": "ok"
},
{
"name": "flipped_listing_not_held",
"capability": "wai.neural.int_mlicpp",
"pin": {
"id": "conformance",
"sha256": "ac5a74765058dac5b44f90185af19852e70ce2922fcac2ace28590b633a9c376",
"determinism": "int-rans/p16+int/i64",
"form": "wai.prior.set"
},
"held": [
{
"listing": "listings/bad/uppercase_hex.sha256",
"root": "sets/tiny"
}
],
"expect": "prior_not_held"
},
{
"name": "set_superseded",
"capability": "wai.neural.int_mlicpp",
"pin": {
"id": "conformance",
"sha256": "402709318223af457631c917c5a1081cb8e380e1dde47c6aebd8f264e077f090",
"determinism": "int-rans/p16+int/i64",
"form": "wai.prior.set",
"supersedes": {
"sha256": "ac5a74765058dac5b44f90185af19852e70ce2922fcac2ace28590b633a9c376",
"form": "wai.prior.set"
}
},
"held": [
{
"listing": "listings/tiny.sha256",
"root": "sets/tiny"
}
],
"expect": "digest_superseded"
},
{
"name": "int_synth_one_file",
"capability": "wai.neural.int_synth",
"pin": {
"id": "conformance",
"sha256": "1c4705d17c01f865b20d785191c7f63b5c18dcc8fce77e15852cf8240b7368be",
"determinism": "int-rans/p16+int/i64"
},
"held": [
{
"file": "../neural-conformance/vectors/gradient/int_synth.json"
}
],
"expect": "ok"
},
{
"name": "null_member_equals_absent",
"capability": "wai.neural.int_mlicpp",
"pin": {
"id": "conformance",
"sha256": "7ef44f3b1fc4a2df6c3e3b68460f595cae3d31cb7f9018699e29e738340272a4",
"determinism": "int-rans/p16+int/i64",
"form": null,
"tier": null,
"supersedes": null
},
"held": [
{
"file": "../wai-rs/tests/vectors/int_mlicpp/model.wmb"
}
],
"expect": "ok"
},
{
"name": "tier_unregistered_still_verifies",
"capability": "wai.neural.int_mlicpp",
"pin": {
"id": "conformance",
"sha256": "7ef44f3b1fc4a2df6c3e3b68460f595cae3d31cb7f9018699e29e738340272a4",
"determinism": "int-rans/p16+int/i64",
"tier": "decode_equivalence"
},
"held": [
{
"file": "../wai-rs/tests/vectors/int_mlicpp/model.wmb"
}
],
"expect": "ok"
}
]
}

2
wai/pin-conformance/listings/bad/blank_line.sha256 Normal file
View file

@ -0,0 +1,2 @@
29bc4a3984b2ca82ab4656e459e7c34b5af9f507309bcf88ed7b00561692ff0a model.wmb

1
wai/pin-conformance/listings/bad/bom.sha256 Normal file
View file

@ -0,0 +1 @@
29bc4a3984b2ca82ab4656e459e7c34b5af9f507309bcf88ed7b00561692ff0a model.wmb

2
wai/pin-conformance/listings/bad/case_fold.sha256 Normal file
View file

@ -0,0 +1,2 @@
6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b A.json
d4735e3a265e16eee03f59718b9b5d03019c07d8b6c51f90da3a666eec13ab35 a.json

1
wai/pin-conformance/listings/bad/crlf.sha256 Normal file
View file

@ -0,0 +1 @@
29bc4a3984b2ca82ab4656e459e7c34b5af9f507309bcf88ed7b00561692ff0a model.wmb

1
wai/pin-conformance/listings/bad/dot_dot.sha256 Normal file
View file

@ -0,0 +1 @@
6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b a/../b

2
wai/pin-conformance/listings/bad/duplicate.sha256 Normal file
View file

@ -0,0 +1,2 @@
6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b a
6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b a

0
wai/pin-conformance/listings/bad/empty.sha256 Normal file
View file

2
wai/pin-conformance/listings/bad/file_and_directory.sha256 Normal file
View file

@ -0,0 +1,2 @@
6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b w
d4735e3a265e16eee03f59718b9b5d03019c07d8b6c51f90da3a666eec13ab35 w/x

1
wai/pin-conformance/listings/bad/leading_dot.sha256 Normal file
View file

@ -0,0 +1 @@
6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b .hidden

1
wai/pin-conformance/listings/bad/no_final_lf.sha256 Normal file
View file

@ -0,0 +1 @@
29bc4a3984b2ca82ab4656e459e7c34b5af9f507309bcf88ed7b00561692ff0a model.wmb

1
wai/pin-conformance/listings/bad/one_space.sha256 Normal file
View file

@ -0,0 +1 @@
29bc4a3984b2ca82ab4656e459e7c34b5af9f507309bcf88ed7b00561692ff0a model.wmb

1
wai/pin-conformance/listings/bad/trailing_dot.sha256 Normal file
View file

@ -0,0 +1 @@
6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b a.

2
wai/pin-conformance/listings/bad/unsorted.sha256 Normal file
View file

@ -0,0 +1,2 @@
6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b b
d4735e3a265e16eee03f59718b9b5d03019c07d8b6c51f90da3a666eec13ab35 a

1
wai/pin-conformance/listings/bad/uppercase_hex.sha256 Normal file
View file

@ -0,0 +1 @@
29BC4A3984B2CA82AB4656E459E7C34B5AF9F507309BCF88ED7B00561692FF0A model.wmb

3
wai/pin-conformance/listings/demo.sha256 Normal file
View file

@ -0,0 +1,3 @@
4b9090d289a978de401cf08619aada32980438b296a7c261b760ec1cb3e61a42 a.b
550c3f093fed7ebdf3ac74ecf5c0b5d410a072d7af081a100af4e1c1087a2949 a/c.onnx
a2f36dd7d60b3eb35ba27548a7058a227f2d0d48f4ce8e7a55b5c993b2069e10 weights/hs_y.json

1
wai/pin-conformance/listings/tiny.sha256 Normal file
View file

@ -0,0 +1 @@
29bc4a3984b2ca82ab4656e459e7c34b5af9f507309bcf88ed7b00561692ff0a model.wmb

184
wai/pin-conformance/pin_verify.py Normal file
View file

@ -0,0 +1,184 @@
#!/usr/bin/env python3
"""Independent check of the pin-conformance corpus (SPEC.md §3.1).
Shares no code with the Rust reference implementation: it re-implements the
listing rules and the ordered pin checks from the SPEC text, with the standard
library only, and checks that
- every `listings/<name>.sha256` is byte for byte the listing of `sets/<name>/`
(regular files, names that begin with `.` skipped), and every file under
`listings/bad/` breaks a listing rule;
- every case in `cases.json` gets its declared reason.
python3 pin-conformance/pin_verify.py [corpus-dir]
The registry facts it needs are restated here from SPEC.md §5 and §3.1: the
registered prior forms, the forms the reference sink reads, and the capabilities
registered with a one-file parameter set.
"""
import hashlib
import json
import os
import re
import sys
PRIOR_FORMS = {'wai.prior.nnc', 'wai.prior.set'}
READABLE = ['wai.prior.set']
ONE_FILE = {
'wai.neural.int_synth', 'wai.neural.int_mlicpp', 'wai.neural.encodec32', 'wai.neural.dac',
'wai.neural.mimi', 'wai.neural.wavtokenizer', 'wai.neural.snac', 'wai.neural.video_bmshj2018',
}
HEX64 = re.compile(r'^[0-9a-f]{64}$')
SEGMENT = re.compile(r'^[A-Za-z0-9_-]([A-Za-z0-9_.-]*[A-Za-z0-9_-])?$')
class Malformed(Exception):
pass
def sha(b):
return hashlib.sha256(b).hexdigest()
def parse_listing(b):
if not b or not b.endswith(b'\n'):
raise Malformed('empty, or no final line feed')
lines = []
for raw in b[:-1].split(b'\n'):
if len(raw) < 67 or raw[64:66] != b' ':
raise Malformed('line shape')
d = raw[:64].decode('ascii', 'replace')
if not HEX64.match(d):
raise Malformed('digest')
try:
p = raw[66:].decode('ascii')
except UnicodeDecodeError:
raise Malformed('path not ASCII')
if not all(SEGMENT.match(s) for s in p.split('/')):
raise Malformed(f'path {p!r}')
lines.append((p, d))
paths = [p for p, _ in lines]
if any(a.encode() >= b.encode() for a, b in zip(paths, paths[1:])):
raise Malformed('order')
folded = sorted(p.lower() for p in paths)
if any(a == b for a, b in zip(folded, folded[1:])):
raise Malformed('case fold')
for a in paths:
if any(b.startswith(a + '/') for b in paths):
raise Malformed('file and directory')
return lines
def listing_of_dir(root):
files = []
for dirpath, dirnames, filenames in os.walk(root):
dirnames[:] = [d for d in dirnames if not d.startswith('.')]
rel = os.path.relpath(dirpath, root)
for f in filenames:
if f.startswith('.'):
continue
if rel == '.' and f.endswith('.sha256'):
continue
p = f if rel == '.' else rel.replace(os.sep, '/') + '/' + f
files.append(p)
out = b''
for p in sorted(files, key=lambda x: x.encode()):
with open(os.path.join(root, p), 'rb') as fh:
out += sha(fh.read()).encode() + b' ' + p.encode() + b'\n'
return out
def verify(capability, pin, held, readable):
"""SPEC §3.1's checks, in order; returns 'ok' or the reason."""
if not HEX64.match(pin['sha256']):
return 'pin_malformed'
form = pin.get('form')
if form is not None:
if form not in PRIOR_FORMS:
return 'form_not_registered'
if form not in readable:
return 'form_unreadable'
elif capability not in ONE_FILE:
return 'form_required'
set_form = form == 'wai.prior.set'
digest = lambda h: sha(h['bytes']) if (('files' in h) == set_form) else None
matching = [h for h in held if digest(h) == pin['sha256']]
if not matching:
sup = pin.get('supersedes')
if (sup is not None and HEX64.match(sup['sha256']) and sup['sha256'] != pin['sha256']
and (sup.get('form') is None or sup.get('form') in PRIOR_FORMS)):
pre_set = sup.get('form') == 'wai.prior.set'
if any(sha(h['bytes']) == sup['sha256'] and (('files' in h) == pre_set) for h in held):
return 'digest_superseded'
return 'prior_not_held'
if not set_form:
return 'ok'
try:
lines = parse_listing(matching[0]['bytes'])
except Malformed:
return 'set_malformed'
for h in matching:
if all(p in h['files'] and sha(h['files'][p]) == d for p, d in lines):
return 'ok'
return 'part_mismatch'
def load(corpus, held):
out = []
for h in held:
if 'file' in h:
with open(os.path.join(corpus, h['file']), 'rb') as f:
out.append({'bytes': f.read()})
else:
with open(os.path.join(corpus, h['listing']), 'rb') as f:
b = f.read()
files = {}
try:
for p, _ in parse_listing(b):
fp = os.path.join(corpus, h['root'], p)
if os.path.isfile(fp):
with open(fp, 'rb') as f:
files[p] = f.read()
except Malformed:
pass
out.append({'bytes': b, 'files': files})
return out
def main():
corpus = sys.argv[1] if len(sys.argv) > 1 else os.path.dirname(os.path.abspath(__file__))
fails, n = [], 0
ld = os.path.join(corpus, 'listings')
for name in sorted(os.listdir(ld)):
path = os.path.join(ld, name)
if name.endswith('.sha256') and os.path.isfile(path):
n += 1
with open(path, 'rb') as f:
committed = f.read()
if listing_of_dir(os.path.join(corpus, 'sets', name[:-len('.sha256')])) != committed:
fails.append(f'listings/{name} is not the listing of its set')
for name in sorted(os.listdir(os.path.join(ld, 'bad'))):
n += 1
with open(os.path.join(ld, 'bad', name), 'rb') as f:
b = f.read()
try:
parse_listing(b)
fails.append(f'listings/bad/{name} parsed')
except Malformed:
pass
with open(os.path.join(corpus, 'cases.json')) as f:
cases = json.load(f)['cases']
for c in cases:
n += 1
pin = {k: v for k, v in c['pin'].items() if v is not None}
got = verify(c['capability'], pin, load(corpus, c['held']), c.get('readable', READABLE))
if got != c['expect']:
fails.append(f"cases.json {c['name']}: expected {c['expect']}, got {got}")
for f in fails:
print('FAIL', f)
print(f'pin_verify.py: {n - len(fails)} of {n} checks as declared')
sys.exit(1 if fails else 0)
if __name__ == '__main__':
main()

531
wai/pin-conformance/selection.json Normal file
View file

@ -0,0 +1,531 @@
{
"about": "SPEC.md \u00a77.1: the rendition a sink picks, the reason it reports for each entry it skips, and the companions it applies with the pick (companions_of_index). held lists the one-file parameter sets the sink holds; advertised is \"all\" or a list of capabilities. roles is the first check_component_roles finding (\"ok\" when the envelope conforms).",
"cases": [
{
"name": "holds_predecessor",
"manifest": {
"wai": "1.1",
"media": "image",
"intent": "replicate",
"renditions": [
{
"capability": "wai.neural.int_mlicpp",
"kind": "int_mlicpp",
"prior": {
"id": "conformance",
"sha256": "c4232f2e2d5f76cce4ac782b254a91410a2ab89b22792f8356585549938a0e92",
"determinism": "int-rans/p16+int/i64",
"tier": "decode-equivalence",
"supersedes": {
"sha256": "29bc4a3984b2ca82ab4656e459e7c34b5af9f507309bcf88ed7b00561692ff0a"
}
}
},
{
"capability": "wai.neural.int_mlicpp",
"kind": "int_mlicpp",
"prior": {
"id": "conformance",
"sha256": "29bc4a3984b2ca82ab4656e459e7c34b5af9f507309bcf88ed7b00561692ff0a",
"determinism": "int-rans/p16+int/i64",
"tier": "decode-equivalence"
}
},
{
"capability": "wai.image.avif",
"kind": "avif"
}
],
"target": null
},
"payloads": [
"B-CODED",
"A-CODED",
"AVIF"
],
"advertised": "all",
"held": [
"sets/tiny/model.wmb"
],
"expect": {
"picked": 1,
"skipped": [
[
0,
"digest_superseded"
]
]
}
},
{
"name": "holds_successor",
"manifest": {
"wai": "1.1",
"media": "image",
"intent": "replicate",
"renditions": [
{
"capability": "wai.neural.int_mlicpp",
"kind": "int_mlicpp",
"prior": {
"id": "conformance",
"sha256": "c4232f2e2d5f76cce4ac782b254a91410a2ab89b22792f8356585549938a0e92",
"determinism": "int-rans/p16+int/i64",
"tier": "decode-equivalence",
"supersedes": {
"sha256": "29bc4a3984b2ca82ab4656e459e7c34b5af9f507309bcf88ed7b00561692ff0a"
}
}
},
{
"capability": "wai.neural.int_mlicpp",
"kind": "int_mlicpp",
"prior": {
"id": "conformance",
"sha256": "29bc4a3984b2ca82ab4656e459e7c34b5af9f507309bcf88ed7b00561692ff0a",
"determinism": "int-rans/p16+int/i64",
"tier": "decode-equivalence"
}
},
{
"capability": "wai.image.avif",
"kind": "avif"
}
],
"target": null
},
"payloads": [
"B-CODED",
"A-CODED",
"AVIF"
],
"advertised": "all",
"held": [
"sets/tiny-tampered/model.wmb"
],
"expect": {
"picked": 0,
"skipped": []
}
},
{
"name": "holds_neither",
"manifest": {
"wai": "1.1",
"media": "image",
"intent": "replicate",
"renditions": [
{
"capability": "wai.neural.int_mlicpp",
"kind": "int_mlicpp",
"prior": {
"id": "conformance",
"sha256": "c4232f2e2d5f76cce4ac782b254a91410a2ab89b22792f8356585549938a0e92",
"determinism": "int-rans/p16+int/i64",
"tier": "decode-equivalence",
"supersedes": {
"sha256": "29bc4a3984b2ca82ab4656e459e7c34b5af9f507309bcf88ed7b00561692ff0a"
}
}
},
{
"capability": "wai.neural.int_mlicpp",
"kind": "int_mlicpp",
"prior": {
"id": "conformance",
"sha256": "29bc4a3984b2ca82ab4656e459e7c34b5af9f507309bcf88ed7b00561692ff0a",
"determinism": "int-rans/p16+int/i64",
"tier": "decode-equivalence"
}
},
{
"capability": "wai.image.avif",
"kind": "avif"
}
],
"target": null
},
"payloads": [
"B-CODED",
"A-CODED",
"AVIF"
],
"advertised": "all",
"held": [],
"expect": {
"picked": 2,
"skipped": [
[
0,
"prior_not_held"
],
[
1,
"prior_not_held"
]
]
}
},
{
"name": "inert",
"manifest": {
"wai": "1.1",
"media": "image",
"intent": "replicate",
"renditions": [
{
"capability": "wai.neural.int_mlicpp",
"kind": "int_mlicpp",
"prior": {
"id": "conformance",
"sha256": "c4232f2e2d5f76cce4ac782b254a91410a2ab89b22792f8356585549938a0e92",
"determinism": "int-rans/p16+int/i64",
"tier": "decode-equivalence",
"supersedes": {
"sha256": "29bc4a3984b2ca82ab4656e459e7c34b5af9f507309bcf88ed7b00561692ff0a"
}
}
},
{
"capability": "wai.neural.int_mlicpp",
"kind": "int_mlicpp",
"prior": {
"id": "conformance",
"sha256": "29bc4a3984b2ca82ab4656e459e7c34b5af9f507309bcf88ed7b00561692ff0a",
"determinism": "int-rans/p16+int/i64",
"tier": "decode-equivalence"
}
},
{
"capability": "wai.image.avif",
"kind": "avif"
}
],
"target": null
},
"payloads": [
"B-CODED",
"A-CODED",
"AVIF"
],
"advertised": [
"wai.neural.int_mlicpp"
],
"held": [],
"expect": {
"inert": [
[
0,
"prior_not_held"
],
[
1,
"prior_not_held"
],
[
2,
"missing_capability"
]
]
}
},
{
"name": "component_digest_ok",
"manifest": {
"wai": "1.1",
"media": "image",
"intent": "replicate",
"renditions": [
{
"capability": "wai.image.png",
"kind": "png",
"sha256": "e793c41f42c3766932d69864deeea3b3949a26e1e427bdf261a77755021e58cb"
},
{
"capability": "wai.image.png",
"kind": "png",
"sha256": "700983acb1832a2b0830405407bcaefaf0909c8631c1603cc5a72ecc4257e4d2"
},
{
"capability": "wai.meta.hdr10plus",
"kind": "descriptor",
"role": "companion",
"companion_of": "wai.image.png",
"sha256": "2d1edf729cc53756834657ce399b1568debd59b0ceddce5da39b86750d59fbd0"
}
],
"target": null
},
"payloads": [
"PNG-A",
"PNG-B",
"META"
],
"advertised": "all",
"held": [],
"expect": {
"picked": 0,
"skipped": [],
"companions": [
2
]
}
},
{
"name": "component_digest_mismatch",
"manifest": {
"wai": "1.1",
"media": "image",
"intent": "replicate",
"renditions": [
{
"capability": "wai.image.png",
"kind": "png",
"sha256": "e793c41f42c3766932d69864deeea3b3949a26e1e427bdf261a77755021e58cb"
},
{
"capability": "wai.image.png",
"kind": "png",
"sha256": "700983acb1832a2b0830405407bcaefaf0909c8631c1603cc5a72ecc4257e4d2"
},
{
"capability": "wai.meta.hdr10plus",
"kind": "descriptor",
"role": "companion",
"companion_of": "wai.image.png",
"sha256": "2d1edf729cc53756834657ce399b1568debd59b0ceddce5da39b86750d59fbd0"
}
],
"target": null
},
"payloads": [
"PNG-a",
"PNG-B",
"META"
],
"advertised": "all",
"held": [],
"expect": {
"picked": 1,
"skipped": [
[
0,
"component_digest_mismatch"
]
],
"companions": [
2
]
}
},
{
"name": "companion_digest_mismatch_not_applied",
"manifest": {
"wai": "1.1",
"media": "image",
"intent": "replicate",
"renditions": [
{
"capability": "wai.image.png",
"kind": "png",
"sha256": "e793c41f42c3766932d69864deeea3b3949a26e1e427bdf261a77755021e58cb"
},
{
"capability": "wai.image.png",
"kind": "png",
"sha256": "700983acb1832a2b0830405407bcaefaf0909c8631c1603cc5a72ecc4257e4d2"
},
{
"capability": "wai.meta.hdr10plus",
"kind": "descriptor",
"role": "companion",
"companion_of": "wai.image.png",
"sha256": "2d1edf729cc53756834657ce399b1568debd59b0ceddce5da39b86750d59fbd0"
}
],
"target": null
},
"payloads": [
"PNG-A",
"PNG-B",
"METa"
],
"advertised": "all",
"held": [],
"expect": {
"picked": 0,
"skipped": [],
"companions": []
}
},
{
"name": "companion_of_id_binds_one_rendition",
"manifest": {
"wai": "1.1",
"media": "video",
"intent": "replicate",
"renditions": [
{
"capability": "wai.video.hevc",
"kind": "hevc",
"id": "a"
},
{
"capability": "wai.video.hevc",
"kind": "hevc",
"id": "b"
},
{
"capability": "wai.video.lcevc",
"kind": "descriptor",
"role": "companion",
"companion_of": "wai.video.hevc",
"companion_of_id": "b"
}
],
"target": null
},
"payloads": [
"HEVC-A",
"HEVC-B",
"ENH"
],
"advertised": [
"wai.video.hevc"
],
"held": [],
"expect": {
"picked": 0,
"skipped": [],
"companions": [],
"roles": "ok"
}
},
{
"name": "companion_of_id_applies_to_the_picked_rendition",
"manifest": {
"wai": "1.1",
"media": "video",
"intent": "replicate",
"renditions": [
{
"capability": "wai.video.hevc",
"kind": "hevc",
"id": "a",
"sha256": "4e452595c3c6ccc249140f0ffe2b163b6c57203f0c7cfdcf0e4316e47fb55df7"
},
{
"capability": "wai.video.hevc",
"kind": "hevc",
"id": "b"
},
{
"capability": "wai.video.lcevc",
"kind": "descriptor",
"role": "companion",
"companion_of": "wai.video.hevc",
"companion_of_id": "b"
}
],
"target": null
},
"payloads": [
"HEVC-a",
"HEVC-B",
"ENH"
],
"advertised": "all",
"held": [],
"expect": {
"picked": 1,
"skipped": [
[
0,
"component_digest_mismatch"
]
],
"companions": [
2
],
"roles": "ok"
}
},
{
"name": "enhancement_base_ambiguous_without_id",
"manifest": {
"wai": "1.1",
"media": "video",
"intent": "replicate",
"renditions": [
{
"capability": "wai.video.hevc",
"kind": "hevc",
"id": "a"
},
{
"capability": "wai.video.hevc",
"kind": "hevc",
"id": "b"
},
{
"capability": "wai.video.lcevc",
"kind": "descriptor",
"role": "companion",
"companion_of": "wai.video.hevc"
}
],
"target": null
},
"payloads": [
"HEVC-A",
"HEVC-B",
"ENH"
],
"advertised": "all",
"held": [],
"expect": {
"picked": 0,
"skipped": [],
"companions": [
2
],
"roles": "EnhancementBaseNotUnique"
}
},
{
"name": "companion_only_never_presented",
"manifest": {
"wai": "1.1",
"media": "video",
"intent": "replicate",
"renditions": [
{
"capability": "wai.video.lcevc",
"kind": "lcevc"
},
{
"capability": "wai.video.hevc",
"kind": "hevc"
}
],
"target": null
},
"payloads": [
"ENH",
"HEVC"
],
"advertised": "all",
"held": [],
"expect": {
"picked": 1,
"skipped": [
[
0,
"not_selectable"
]
],
"roles": "CompanionOnlyAsRendition"
}
}
]
}

1
wai/pin-conformance/sets/demo/.hidden Normal file
View file

@ -0,0 +1 @@
not listed

1
wai/pin-conformance/sets/demo/a.b Normal file
View file

@ -0,0 +1 @@
{"part":"a.b"}

1
wai/pin-conformance/sets/demo/a/c.onnx Normal file
View file

@ -0,0 +1 @@
ONNX-STAND-IN

1
wai/pin-conformance/sets/demo/weights/hs_y.json Normal file
View file

@ -0,0 +1 @@
{"w":[1,2,3]}

1
wai/pin-conformance/sets/tiny-tampered/model.wmb Normal file
View file

@ -0,0 +1 @@
WAI pin-conformance one-file seT

1
wai/pin-conformance/sets/tiny/model.wmb Normal file
View file

@ -0,0 +1 @@
WAI pin-conformance one-file set

465
wai/pin-conformance/static.json Normal file
View file

@ -0,0 +1,465 @@
{
"about": "SPEC.md \u00a73.1, \u00a77: the first finding a verifier makes about a manifest's pins without holding a parameter set (\"ok\" when it makes none). For a WAI2 manifest the finding is [entry index, reason].",
"cases": [
{
"name": "stream_wai_conforms",
"manifest": {
"wai": "1.0",
"media": "image",
"intent": "replicate",
"model_requirement": {
"capability": "wai.neural.int_mlicpp"
},
"conditioning": {
"kind": "int_mlicpp",
"prior": {
"id": "int_mlicpp/synthetic-s20260929",
"sha256": "7ef44f3b1fc4a2df6c3e3b68460f595cae3d31cb7f9018699e29e738340272a4",
"determinism": "int-rans/p16+int-mlicpp/i64"
}
},
"target": {
"w": 64,
"h": 64
}
},
"expect": "ok"
},
{
"name": "prior_not_applicable",
"manifest": {
"wai": "1.0",
"media": "image",
"intent": "replicate",
"model_requirement": {
"capability": "wai.image.png"
},
"conditioning": {
"kind": "k",
"prior": {
"id": "conformance",
"sha256": "7ef44f3b1fc4a2df6c3e3b68460f595cae3d31cb7f9018699e29e738340272a4",
"determinism": "int-rans/p16+int/i64"
}
},
"target": null
},
"expect": "prior_not_applicable"
},
{
"name": "prior_not_applicable_before_malformed",
"manifest": {
"wai": "1.0",
"media": "image",
"intent": "replicate",
"model_requirement": {
"capability": "wai.image.png"
},
"conditioning": {
"kind": "k",
"prior": {
"id": "conformance",
"sha256": "AB",
"determinism": "int-rans/p16+int/i64"
}
},
"target": null
},
"expect": "prior_not_applicable"
},
{
"name": "pin_malformed",
"manifest": {
"wai": "1.0",
"media": "image",
"intent": "replicate",
"model_requirement": {
"capability": "wai.neural.int_mlicpp"
},
"conditioning": {
"kind": "k",
"prior": {
"id": "conformance",
"sha256": "7EF44F3B1FC4A2DF6C3E3B68460F595CAE3D31CB7F9018699E29E738340272A4",
"determinism": "int-rans/p16+int/i64"
}
},
"target": null
},
"expect": "pin_malformed"
},
{
"name": "tier_unregistered",
"manifest": {
"wai": "1.0",
"media": "image",
"intent": "improve",
"model_requirement": {
"capability": "wai.neural.int_mlicpp"
},
"conditioning": {
"kind": "k",
"prior": {
"id": "conformance",
"sha256": "7ef44f3b1fc4a2df6c3e3b68460f595cae3d31cb7f9018699e29e738340272a4",
"determinism": "int-rans/p16+int/i64",
"tier": "decode_equivalence"
}
},
"target": null
},
"expect": "tier_unregistered"
},
{
"name": "supersedes_malformed",
"manifest": {
"wai": "1.0",
"media": "image",
"intent": "replicate",
"model_requirement": {
"capability": "wai.neural.int_mlicpp"
},
"conditioning": {
"kind": "k",
"prior": {
"id": "conformance",
"sha256": "7ef44f3b1fc4a2df6c3e3b68460f595cae3d31cb7f9018699e29e738340272a4",
"determinism": "int-rans/p16+int/i64",
"supersedes": {
"sha256": "7ef44f3b1fc4a2df6c3e3b68460f595cae3d31cb7f9018699e29e738340272a4"
}
}
},
"target": null
},
"expect": "supersedes_malformed"
},
{
"name": "form_not_registered",
"manifest": {
"wai": "1.0",
"media": "image",
"intent": "improve",
"model_requirement": {
"capability": "wai.neural.int_mlicpp"
},
"conditioning": {
"kind": "k",
"prior": {
"id": "conformance",
"sha256": "7ef44f3b1fc4a2df6c3e3b68460f595cae3d31cb7f9018699e29e738340272a4",
"determinism": "int-rans/p16+int/i64",
"form": "wai.image.png"
}
},
"target": null
},
"expect": "form_not_registered"
},
{
"name": "form_required",
"manifest": {
"wai": "1.0",
"media": "image",
"intent": "improve",
"model_requirement": {
"capability": "wai.neural.bmshj2018"
},
"conditioning": {
"kind": "k",
"prior": {
"id": "conformance",
"sha256": "7ef44f3b1fc4a2df6c3e3b68460f595cae3d31cb7f9018699e29e738340272a4",
"determinism": "int-rans/p16+int/i64"
}
},
"target": null
},
"expect": "form_required"
},
{
"name": "form_required_for_the_fallback",
"manifest": {
"wai": "1.0",
"media": "image",
"intent": "improve",
"model_requirement": {
"capability": "wai.neural.int_mlicpp",
"fallback": "wai.neural.elic"
},
"conditioning": {
"kind": "k",
"prior": {
"id": "conformance",
"sha256": "7ef44f3b1fc4a2df6c3e3b68460f595cae3d31cb7f9018699e29e738340272a4",
"determinism": "int-rans/p16+int/i64"
}
},
"target": null
},
"expect": "form_required"
},
{
"name": "unregistered_capability_form_required",
"manifest": {
"wai": "1.0",
"media": "image",
"intent": "improve",
"model_requirement": {
"capability": "wai.neural.future"
},
"conditioning": {
"kind": "k",
"prior": {
"id": "conformance",
"sha256": "7ef44f3b1fc4a2df6c3e3b68460f595cae3d31cb7f9018699e29e738340272a4",
"determinism": "int-rans/p16+int/i64"
}
},
"target": null
},
"expect": "form_required"
},
{
"name": "tier_exceeds_class",
"manifest": {
"wai": "1.0",
"media": "image",
"intent": "improve",
"model_requirement": {
"capability": "wai.image.jpegai"
},
"conditioning": {
"kind": "k",
"prior": {
"id": "conformance",
"sha256": "7ef44f3b1fc4a2df6c3e3b68460f595cae3d31cb7f9018699e29e738340272a4",
"determinism": "int-rans/p16+int/i64",
"form": "wai.prior.set",
"tier": "decode-equivalence"
}
},
"target": null
},
"expect": "tier_exceeds_class"
},
{
"name": "tier_at_class",
"manifest": {
"wai": "1.0",
"media": "image",
"intent": "improve",
"model_requirement": {
"capability": "wai.image.jpegai"
},
"conditioning": {
"kind": "k",
"prior": {
"id": "conformance",
"sha256": "7ef44f3b1fc4a2df6c3e3b68460f595cae3d31cb7f9018699e29e738340272a4",
"determinism": "int-rans/p16+int/i64",
"form": "wai.prior.set",
"tier": "entropy-consistency"
}
},
"target": null
},
"expect": "ok"
},
{
"name": "replicate_tier",
"manifest": {
"wai": "1.0",
"media": "image",
"intent": "replicate",
"model_requirement": {
"capability": "wai.neural.int_synth"
},
"conditioning": {
"kind": "k",
"prior": {
"id": "conformance",
"sha256": "1c4705d17c01f865b20d785191c7f63b5c18dcc8fce77e15852cf8240b7368be",
"determinism": "int-rans/p16+int/i64",
"tier": "none"
}
},
"target": null
},
"expect": "replicate_tier"
},
{
"name": "replicate_tier_legacy_float_words",
"manifest": {
"wai": "1.0",
"media": "image",
"intent": "replicate",
"model_requirement": {
"capability": "wai.neural.int_synth"
},
"conditioning": {
"kind": "k",
"prior": {
"id": "conformance",
"sha256": "1c4705d17c01f865b20d785191c7f63b5c18dcc8fce77e15852cf8240b7368be",
"determinism": "int-rans/p16+onnx-f32"
}
},
"target": null
},
"expect": "replicate_tier"
},
{
"name": "replicate_unpinned",
"manifest": {
"wai": "1.0",
"media": "image",
"intent": "replicate",
"model_requirement": {
"capability": "wai.neural.int_synth"
},
"conditioning": {
"kind": "k"
},
"target": null
},
"expect": "replicate_unpinned"
},
{
"name": "replicate_form_not_exact",
"manifest": {
"wai": "1.0",
"media": "image",
"intent": "replicate",
"model_requirement": {
"capability": "wai.neural.int_synth"
},
"conditioning": {
"kind": "k",
"prior": {
"id": "conformance",
"sha256": "1c4705d17c01f865b20d785191c7f63b5c18dcc8fce77e15852cf8240b7368be",
"determinism": "int-rans/p16+int/i64",
"form": "wai.prior.nnc"
}
},
"target": null
},
"expect": "replicate_form_not_exact"
},
{
"name": "replicate_set_form_sound",
"manifest": {
"wai": "1.0",
"media": "image",
"intent": "replicate",
"model_requirement": {
"capability": "wai.neural.int_mlicpp"
},
"conditioning": {
"kind": "k",
"prior": {
"id": "conformance",
"sha256": "85a05b0dac15566c5a657ddeee253eaa9f54ce852ae04924f5b04151901d9d1b",
"determinism": "int-rans/p16+int/i64",
"form": "wai.prior.set",
"tier": "decode-equivalence"
}
},
"target": null
},
"expect": "ok"
},
{
"name": "null_member_equals_absent",
"manifest": {
"wai": "1.0",
"media": "image",
"intent": "replicate",
"model_requirement": {
"capability": "wai.neural.int_mlicpp"
},
"conditioning": {
"kind": "k",
"prior": {
"id": "conformance",
"sha256": "7ef44f3b1fc4a2df6c3e3b68460f595cae3d31cb7f9018699e29e738340272a4",
"determinism": "int-rans/p16+int/i64",
"form": null,
"tier": null,
"supersedes": null
}
},
"target": null
},
"expect": "ok"
},
{
"name": "wai2_entry_index",
"manifest": {
"wai": "1.1",
"media": "image",
"intent": "replicate",
"renditions": [
{
"capability": "wai.neural.int_mlicpp",
"kind": "int_mlicpp",
"prior": {
"id": "conformance",
"sha256": "c4232f2e2d5f76cce4ac782b254a91410a2ab89b22792f8356585549938a0e92",
"determinism": "int-rans/p16+int/i64",
"tier": "decode-equivalence",
"supersedes": {
"sha256": "29bc4a3984b2ca82ab4656e459e7c34b5af9f507309bcf88ed7b00561692ff0a"
}
}
},
{
"capability": "wai.neural.int_mlicpp",
"kind": "int_mlicpp",
"prior": {
"id": "conformance",
"sha256": "29bc4a3984b2ca82ab4656e459e7c34b5af9f507309bcf88ed7b00561692ff0a",
"determinism": "int-rans/p16+int/i64",
"tier": "decode-equivalence"
}
},
{
"capability": "wai.image.avif",
"kind": "avif"
}
],
"target": null
},
"expect": "ok"
},
{
"name": "wai2_entry_finding",
"manifest": {
"wai": "1.1",
"media": "image",
"intent": "improve",
"renditions": [
{
"capability": "wai.image.avif",
"kind": "avif"
},
{
"capability": "wai.neural.glc",
"kind": "glc",
"prior": {
"id": "conformance",
"sha256": "7ef44f3b1fc4a2df6c3e3b68460f595cae3d31cb7f9018699e29e738340272a4",
"determinism": "int-rans/p16+int/i64"
}
}
],
"target": null
},
"expect": [
1,
"form_required"
]
}
]
}

14
wai/registry/capabilities.json
View file

@ -853,6 +853,20 @@
"edition": "ISO/IEC 15938-17:2024",
"registered": "2026-09-28"
},
{
"capability": "wai.prior.set",
"kind": "payload",
"status": "registered",
"media": "binary",
"role": "standalone",
"tier": "lossless",
"licence": "royalty-free",
"replicate": "standard-defined",
"sets": [],
"spec_section": "§5 Prior artifacts, general tensors and derivation operations",
"edition": null,
"registered": "2026-09-30"
},
{
"capability": "wai.quantum.calibration",
"kind": "record",

22
wai/research/2030-transport-landscape.md
View file

@ -863,22 +863,18 @@ analysis* — DeepCABAC entropy coding, pruning/sparsification/low-rank
preprocessing, incremental-update support, with reference software in
ISO/IEC 15938-18 and a separate open implementation (NNCodec) [name removed 2026-09-30].
The claim that needs no amendment at all is the carriage one: NNC is a ratified
wire format for the artifact a `Prior` names, which WAI currently leaves as an
informal directory. The signing feature is a second surface. It concerns how
wire format for the artifact a `Prior` names; for a set of several files, WAI pins
its listing (`wai.prior.set`, SPEC §3.1). The signing feature is a second surface. It concerns how
NNC's in-band signatures relate to WAI's digest, which
`extensions/prior-carriage.md` §6 fixes without depending on the amendment's
details.
**What WAI already has, stated correctly.** WAI *does* pin model **identity**
normatively: the reference implementation's `Prior { id, sha256, determinism }`
is carried in the conditioning, hash-verified by the registry, with
`replicate`-soundness gated on the determinism contract. (SPEC.md's opening prose
— "never transported and never hash-pinned" — describes transport, not identity;
the two statements should be reconciled in the spec text.) What WAI has **no**
answer for is model/tensor **carriage**: by explicit design the weights are never
carried and decoders are sink-supplied, so the artifact whose `sha256` the
`Prior` names has no normative on-the-wire form — today it is an informal model
bundle directory.
**What WAI already has, stated correctly.** WAI pins model identity
normatively: SPEC §3.1 defines the pin, carried as `conditioning.prior` in a
single-payload manifest and on each rendition of a multi-rendition one, and
requires a sink to verify it before decoding and to refuse with a named reason.
What WAI does not specify is model carriage: the set a pin names travels outside
the envelope, in a form the pin MAY declare.
**Three concrete surfaces.** (1) Name NNC as the standards-track normative form
for the artifact a `Prior` identifies, so `Prior.sha256` covers a portable
@ -1052,7 +1048,7 @@ Items 1–2 are internal defects that need no external dependency at all.
| 6 | **Key discovery**, plus DSC SEI verdict chaining with a precedence rule (§3.6, §6.1) | a trust-anchor mechanism WAI has none of; borrow DSCI's key-location design (certificate URI or ISO/IEC 21617-1 trust record) | ITU-T H.274 V4 (in force) | every WAI signature presumes the verifier already holds the key |
| 7 | **JCR-1 alignment, then an RFC 9942 / RFC 9995 emission profile + SCRAPI registration** (§3.6) | move receipts onto the family's canonical envelope; emit the group receipt or checkpoints; decide BLAKE3 registration vs dual-hash | RFCs published; SCRAPI in RFC Editor queue; CCF profile in Last Call | non-equivocation, timestamping and suppression-detection are missing today |
| 8 | **Derive acquisition classes from GREEN `data-source-accuracy`**, and declare tolerances (§3.5) | register `HwShunt`/`ModelBased`/`Estimator` as identities derived from the hierarchy; state measurement tolerances | adopted WG draft | the enum is private and coarser than an adopted draft, now |
| 9 | **NNC as the normative artifact form for a `Prior`**; register the general-tensor payload (§6.4) | reference the bitstream; one new capability; reconcile data-unit signing | CDAM stage — track, do not pin | `Prior.sha256` currently covers an informal directory |
| 9 | **NNC as the normative artifact form for a `Prior`**; register the general-tensor payload (§6.4) | reference the bitstream; one new capability; reconcile data-unit signing | CDAM stage — track, do not pin | a pin covers a set of files through its listing; NNC would give its networks a coded form |
| 10 | **Sink-evaluated predicate capability + authority class + detached receipt profile** (§4.2, §7.1) | predicate-equivalence contract; priority field + sink-held trust anchor; store-and-forward receipt profile | deployed regional precedent (no dependency) | pre-emption is the highest-authority instruction and WAI names no signer class |
| 11 | **NMOS constraint-set grammar over the capability registry** (§4.3) | a sink-side precondition on §4 dispatch; author register entries for the non-broadcast families | BCP-004-01/-02 v1.0.0 (ratified) | makes §7.1 rendition selection evaluable rather than policy-deferred |
| 12 | **Register the Part 16 derivation operations with a per-operation determinism tier** (§3.1) | capability strings + tiers; do not adopt compositing | ISO/IEC 23001-16:2021 (published) | prevents restating a 2021 ratification as a WAI invention |

24
wai/research/syfai-input.md
View file

@ -130,10 +130,10 @@ the text catches up (G1).
| framework topic | WAI mechanism | state | honest limit |
|---|---|---|---|
| (i) AI parameter-data format | **One registered form, optional.** A `Prior` MAY declare that the artifact its digest covers is an ISO/IEC 15938-17:2024 (NNC) bitstream, `wai.prior.nnc`; the digest then covers the coded bitstream ([prior-carriage](../extensions/prior-carriage.md) §2–§3; link 05, PR #106). The model is still never transported in the envelope that decodes against it (SPEC.md's opening section). Three informative formats exist: the integer `int_synth.json` model in [neural-conformance](../neural-conformance/README.md); the JPEG AI weight bundle (`weights/*.json` + `norm.json`) read by `try_jpegai_exact`, whose hyper-decoder, context-model and synthesis networks are integer but whose eICCI network and one `norm.json` table are `f32`; and the one-file `WMB1` bundle of `wai.neural.int_mlicpp` — three integer networks and the entropy tables in one file (link 09, PR #111). | NNC form: specified, not implemented. Informative formats: implemented | `wai-rs` has no NNC reader or writer, and the reference sink refuses any prior that declares a form (`READABLE_PRIOR_FORMS` is empty). NNC is optional and not recommended: its royalty status, and whether conforming decoders reconstruct identical tensors, are not established (prior-carriage §4–§5). On `main` no form exists at all. |
| (ii) manifest | The reference implementation carries `conditioning.prior = { id, sha256, determinism }` in a single-payload (`WAI1`) manifest (`container.rs`, `Prior`), and at link 05 an optional `form`. A multi-rendition (`WAI2`) manifest carries the same `Prior` per rendition (`RenditionMeta.prior`). It names *which* model, exactly, and *how* its decode is made reproducible. prior-carriage §2 (a Draft extension) shows the object with its four members. | **implemented (reference implementation); shown in a Draft extension; not in SPEC.md §3** | SPEC.md §3's manifest schema defines `conditioning` as `{ "kind" }` only, and SPEC.md §7.1 defines no prior on a rendition entry (G1). The pin names identity, not location, and has no update or versioning semantics. Without a declared form, `sha256` covers **one file** (G2). `determinism` is free text (G3). The per-rendition pin is carried and never checked (G12). |
| (i) AI parameter-data format | **Registered forms, each optional.** A pin MAY declare that the set its digest covers is the listing of a set of files, `wai.prior.set` ([SPEC.md §3.1](../SPEC.md)), or an ISO/IEC 15938-17:2024 (NNC) bitstream, `wai.prior.nnc`; the digest then covers the listing or the coded bitstream ([prior-carriage](../extensions/prior-carriage.md) §2–§3). The model is still never transported in the envelope that decodes against it (SPEC.md's opening section). Informative formats exist: the integer `int_synth.json` model in [neural-conformance](../neural-conformance/README.md); the JPEG AI weight bundle (`weights/*.json` + `norm.json`) read by `try_jpegai_exact`, whose hyper-decoder, context-model and synthesis networks are integer but whose eICCI network and one `norm.json` table are `f32`; and the one-file `WMB1` bundle of `wai.neural.int_mlicpp` — its integer networks and the entropy tables in one file. | Listing form: specified, implemented, CI-checked. NNC form: specified, not implemented. Informative formats: implemented | `wai-rs` has no NNC reader or writer; the reference sink reads `wai.prior.set` and no coded form. NNC is optional and not recommended: its royalty status, and whether conforming decoders reconstruct identical tensors, are not established (prior-carriage §4–§5). |
| (ii) manifest | SPEC.md §3.1 defines the pin: `conditioning.prior = { id, sha256, determinism, form?, tier?, supersedes? }` in a single-payload (`WAI1`) manifest, and a rendition entry's `prior` in a multi-rendition (`WAI2`) one, beside the entry's `id` and payload `sha256` (§7.1). It names *which* parameter set, exactly — the SHA-256 of the whole set, a set of several files through its listing — and *how* its decode is made reproducible, with a tier from a closed vocabulary. | **specified (SPEC.md §3, §3.1, §7.1) and implemented** | The pin names identity, not location. Sinks of earlier revisions may ignore it (the pin then fails open) or reject the envelope (SPEC.md §3.1, §8). `id` is informative; `supersedes` is informative and never makes a successor satisfy its predecessor's pin. |
| (iii) repository | **None.** SPEC.md §1 scopes WAI as a container and capability-dispatch standard; nothing in the tree specifies a model store or service. Identity-by-digest makes any store content-addressable, but WAI specifies no store. | out of scope | A repository would be SyfAI's. WAI's input is the identity rule a repository must honour, not the repository. |
| (iv) integrity checking | `ModelRegistry::resolve_checked` recomputes SHA-256 over the registered artifact and refuses to decode on mismatch (`DecodeError::PriorMismatch`); at link 05 it also refuses, before hashing, a declared form the sink cannot read. The `wai.neural.int_mlicpp` decoder re-checks a pinned prior against the very bytes it decodes with (link 09). `resolve_checked` is called only from the `WAI1` decode (`decode_envelope`), so no code path checks a `WAI2` rendition's pin (G12). `decode_envelope_strict` checks the capability's registry class and then the pinned prior before a `replicate` decode (link 02, PR #103). Because the manifest sits inside the envelope bytes the JWP receipt hashes ([jwp-receipts](../extensions/jwp-receipts.md) §2.1), a signed receipt authenticates the pin as well as the payload. | **implemented (reference implementation); partly specified** | prior-carriage §2 requires a sink to refuse a prior whose declared form it cannot read, "as it refuses a digest mismatch", but no normative text states the refusal on a digest mismatch itself (G1). SPEC.md §7 requires a sink to refuse a `replicate` claim over a `NeuralEntropyExact`, `NeuralFloat`, `GenerativeDeterministic` or `Behavioral` capability (on `main`: over a `NeuralFloat` one). A digest proves the bytes are the named bytes. It does not say who chose them. That comes from a signed JWP receipt over the envelope, and WAI has no key discovery (landscape §3.6). The lenient `decode_envelope` path decodes unpinned neural capabilities without checking; this is honest for perceptual use and is not a replicate path. |
| (iv) integrity checking | SPEC.md §3.1 requires a sink to verify a pin before it decodes, and to refuse one its parameter sets do not satisfy with one of an ordered set of named reasons; a refused pin makes the capability unsupported for that envelope, and dispatch continues at the fallback (§4) or the next rendition (§7.1). A pinned decode reads only the set it verified. The reference implementation does this for `WAI1` and `WAI2` (`Wai::dispatch_verified`, `WaiMulti::pick`, `pin::verify_bytes`, `neural::decode_envelope`, `neural::decode_multi`), in the C ABI and in the browser crate. `decode_envelope_strict` checks the capability's registry class and then the pin before a `replicate` decode. Because the manifest sits inside the envelope bytes the JWP receipt hashes ([jwp-receipts](../extensions/jwp-receipts.md) §2.1), a signed receipt authenticates the pin as well as the payload. | **specified (SPEC.md §3.1, §4, §7.1) and implemented; the pin vectors are checked on the conformance legs** | A digest proves the bytes are the named bytes. It does not say who chose them. That comes from a signed JWP receipt over the envelope, and WAI has no key discovery (landscape §3.6). The lenient `decode_envelope` path decodes an unpinned neural capability without checking; this is honest for perceptual use and is not a replicate path. |
| (v) bit exactness + conformance | Two named tiers, `entropy-consistency` and `decode-equivalence`, and the rule that a tier covers what the sink **emits**, including post-decode processing ([determinism-tiers](../extensions/determinism-tiers.md) §2). Over a neural capability `replicate` promises `decode-equivalence`, which SPEC.md §7 admits only by an integer / fixed-point decode path with no float operation; `entropy-consistency` does not back it (link 02). A per-capability `ReplicateClass` in the registry: the `NeuralIntegerExact` capabilities at `decode-equivalence`, `wai.neural.int_mlicpp` among them (link 09), and `wai.image.jpegai` alone in `NeuralEntropyExact` at `entropy-consistency` (link 02). `codecs::DeterminismTier` carries the tiers in code (link 02). Frozen corpora of decoded-output digests. | tiers: specified, and a type in code at link 02 (on `main`, text only). Emitted-medium rule: **specified only**. Integer-entropy contract, `ReplicateClass` and the strict check: specified + implemented. Corpora and byte-equality goldens: implemented, and CI-checked on `main` for the capabilities listed under *CI scope*, on the legs that ran. | No manifest field names a tier; a capability's tier comes from its registration. `wai-rs` does not handle post-decode processing or operation-reduction hints. `wai.image.jpegai`'s exact decoder pins one reconstruction, reproducible only under its own build and floating-point conditions, and the reference sink can decode the same capability through a float path (G11). The float MLIC++ decoder agrees with its encoder only by convention between the two implementations; `wai.neural.int_mlicpp` makes that agreement a property of the arithmetic (*CI scope*). On `main`, a malformed stream's outcome depends on the build profile; link 01 fixes the known case (G6). Float capabilities have no corpus, deliberately. |
**CI scope, precisely.** Two states are described. For `main`, the workflow
@ -383,6 +383,17 @@ party can regenerate. See [energy-exposure-mapping](energy-exposure-mapping.md).
## 4. Candidate requirements
*Status in this tree.* R1, R2, R3, R7 and R9 are now specified: R1 by SPEC.md §3.1
(the pin covers the whole parameter set; several files through the listing form
`wai.prior.set`); R2 by §3.1's closed `tier` vocabulary; R3 by §3.1's ordered
refusal reasons and §4 / §7.1's dispatch rule; R7 by §7 *Digests* (SHA-256 for
parameter-set identity, component identity and the decoded output of capabilities
that decode against a parameter set, with `sha256.json` sidecars on the committed
vectors); R9 by `supersedes`, which names the predecessor and changes only the
reason a sink that holds it reports — predecessor content decodes exactly while the
sink still holds the predecessor. The entries below record the state they were
written against.
Drafted as an input would phrase them. After each one: what WAI does in the tree
at link 10, or the gap.
@ -521,6 +532,13 @@ one of them. It does not need to prefer one route.
## 6. Gaps WAI must close before any of this is contributed
*Status in this tree.* G1 is closed by SPEC.md §3.1 (the pin is normative text,
for `WAI1` and per rendition); G2 by the parameter-set shapes of §5 and the
`wai.prior.set` listing; G3 by the closed `tier` vocabulary, with the tier-less
reading frozen as the legacy rule; G5 by `supersedes`; G7 by §7 *Digests*; G12 by
§4 / §7.1's dispatch rule and its implementation for `WAI2`. The entries below
record the state they were written against.
Each gap was found on `main`. Its status says what the tree at link 10 holds.
**G1: The prior pin is in the reference implementation, not in the

4
wai/tools/wai_int_mlicpp_export.py
View file

@ -27,6 +27,8 @@ Writes into <out-dir>:
stream.wai a WAI1 envelope over `wai.neural.int_mlicpp`, intent `replicate`,
pinning model.wmb by SHA-256 as an integer-exact prior. Its payload
is the `WIM1` codestream (see `wai-rs/src/int_mlicpp.rs`).
model.sha256, the set-form twin (tools/wai_int_mlicpp_set_form.py): the listing of
stream_set.wai the one-file set, and stream.wai pinned through it (`wai.prior.set`).
expected.json what the decode must reproduce: the RGB hashes (FNV-1a and
SHA-256), and FNV-1a over the y stream's bucket indices (u16 LE)
and symbols (i32 LE) in coding order.
@ -469,6 +471,8 @@ def main():
with open(os.path.join(a.outdir, 'expected.json'), 'w') as f:
json.dump(expected, f, indent=1)
f.write('\n')
from wai_int_mlicpp_set_form import write_set_form
write_set_form(a.outdir)
print(f'int_mlicpp vector: {W}x{H}, y {CY}x{enc["dims"][2]}x{enc["dims"][3]} in {K} slices, '
f'{len(enc["z_bits"])} z + {len(enc["y_bits"])} y rANS bytes, model {len(wmb)} B, '
f'rgb_fnv1a {expected["rgb_fnv1a"]}')

72
wai/tools/wai_int_mlicpp_set_form.py Normal file
View file

@ -0,0 +1,72 @@
#!/usr/bin/env python3
"""Write the set-form twin of the `wai.neural.int_mlicpp` conformance vector.
The committed `stream.wai` pins `model.wmb` form-less: its pin's `sha256` is the
SHA-256 of the one file. SPEC.md §3.1 also lets a pin cover a parameter set through
its listing (the prior form `wai.prior.set`). This tool writes, next to the vector:
model.sha256 the listing of the one-file set: the line
`<sha256 of model.wmb> model.wmb` and a line feed -- the text
`sha256sum model.wmb` prints, which `sha256sum -c` checks.
stream_set.wai `stream.wai` with the same payload, its pin replaced by one over
the listing: `sha256` = SHA-256(model.sha256), `form`
`wai.prior.set`, `tier` `decode-equivalence`.
Standard library only, and independent of the Rust reference implementation, which
checks these files in tests/int_mlicpp.rs and src/pin.rs.
python tools/wai_int_mlicpp_set_form.py [vector-dir]
`vector-dir` defaults to wai-rs/tests/vectors/int_mlicpp. `wai_int_mlicpp_export.py`
calls `write_set_form` after it writes the vector.
"""
import hashlib
import json
import os
import struct
import sys
def listing(files):
"""The listing of `files` (path -> bytes): sorted by path as bytes, one
`<64 hex> <path>\\n` line per file."""
out = b''
for path in sorted(files, key=lambda p: p.encode()):
out += hashlib.sha256(files[path]).hexdigest().encode() + b' ' + path.encode() + b'\n'
return out
def write_set_form(vector_dir):
with open(os.path.join(vector_dir, 'model.wmb'), 'rb') as f:
wmb = f.read()
with open(os.path.join(vector_dir, 'stream.wai'), 'rb') as f:
env = f.read()
assert env[:4] == b'WAI1'
ml = struct.unpack('<I', env[4:8])[0]
manifest = json.loads(env[8:8 + ml])
rest = env[8 + ml:]
lst = listing({'model.wmb': wmb})
prior = manifest['conditioning']['prior']
assert prior['sha256'] == hashlib.sha256(wmb).hexdigest(), 'stream.wai pins model.wmb'
prior['sha256'] = hashlib.sha256(lst).hexdigest()
prior['form'] = 'wai.prior.set'
prior['tier'] = 'decode-equivalence'
mb = json.dumps(manifest, separators=(',', ':')).encode()
set_env = b'WAI1' + struct.pack('<I', len(mb)) + mb + rest
with open(os.path.join(vector_dir, 'model.sha256'), 'wb') as f:
f.write(lst)
with open(os.path.join(vector_dir, 'stream_set.wai'), 'wb') as f:
f.write(set_env)
return lst, set_env
def main():
here = os.path.dirname(os.path.abspath(__file__))
d = sys.argv[1] if len(sys.argv) > 1 else os.path.join(here, '..', 'wai-rs', 'tests', 'vectors', 'int_mlicpp')
lst, set_env = write_set_form(d)
print(f'model.sha256 {len(lst)} B sha256 {hashlib.sha256(lst).hexdigest()}; '
f'stream_set.wai {len(set_env)} B sha256 {hashlib.sha256(set_env).hexdigest()}')
if __name__ == '__main__':
main()

20
wai/vocab/README.md
View file

@ -12,8 +12,8 @@ Namespace: `https://wai.transaction.science/vocab/v2#`.
| File | Format | What |
|------|--------|------|
| [`v2.ttl`](v2.ttl) | Turtle (RDF) | One IRI per string SPEC §5 registers, with its kind, status, media class, role, fidelity tier, licence class, replicate class, registering section, edition and registration date (`registeredOn`); the enumerations as individuals named `<class prefix>-<label>` (`kind-companion`, `role-companion`, `media-image`, …), so no two terms share an IRI; and the manifest terms the context maps. |
| [`v2.jsonld`](v2.jsonld) | JSON-LD `@context` | For a WAI manifest, `WAI1` or `WAI2`, as written on the wire: `capability`, `fallback` and a component's `companion_of` expand to capability IRIs, `media` to a media-class individual; `kind` and a component's `role` are plain strings under terms of their own (`payloadKind`, `componentRole`), and `target` is a JSON literal. |
| [`v2.shapes.ttl`](v2.shapes.ttl) | SHACL Core | Applies to a manifest with or without an `@type` (it targets every node with a `wai` or `intent` field, and every component of `renditions`). Checks that a `WAI1` names a capability and a `WAI2` a component list, never both; the `capability` and every component's capability against the registered payload and companion capabilities that are not candidates; the `fallback` and a companion's base against the registered payload capabilities that are not candidates. `media` is informational and a reader accepts a value it does not know (SPEC §3, §8), so only its form is checked. The shapes check a manifest against this revision's registry: a violation for a capability or fallback this revision does not register means the string is not registered in this revision, not that the envelope fails to conform or cannot be dispatched, since a later MINOR revision may register it (SPEC §4, §8). |
| [`v2.jsonld`](v2.jsonld) | JSON-LD `@context` | For a WAI manifest, `WAI1` or `WAI2`, as written on the wire: `capability`, `fallback` and a component's `companion_of` expand to capability IRIs, `media` to a media-class individual; `kind` and a component's `role` are plain strings under terms of their own (`payloadKind`, `componentRole`), and `target` is a JSON literal. `prior` (`conditioning.prior`, or a component's) maps its members to the pin terms below. |
| [`v2.shapes.ttl`](v2.shapes.ttl) | SHACL Core | Applies to a manifest with or without an `@type` (it targets every node with a `wai` or `intent` field, and every component of `renditions`). Checks that a `WAI1` names a capability and a `WAI2` a component list, never both; the `capability` and every component's capability against the registered payload and companion capabilities that are not candidates; the `fallback` and a companion's base against the registered payload capabilities that are not candidates. `media` is informational and a reader accepts a value it does not know (SPEC §3, §8), so only its form is checked. The shapes check a manifest against this revision's registry: a violation for a capability or fallback this revision does not register means the string is not registered in this revision, not that the envelope fails to conform or cannot be dispatched, since a later MINOR revision may register it (SPEC §4, §8). Also checks every parameter-set pin, wherever the manifest carries one (below). |
The three files are generated by `wai registry export` from the reference
implementation's registry (`wai-rs/src/codecs/registry.rs`), together with
@ -36,3 +36,19 @@ JPEG XL, FLAC, Opus, AV1, zstd, XZ) and model the neural and zeroth paths as
classes; they are not the registry. v1's SHACL shape rejects a manifest that
names any other registered capability, so a validator of current manifests uses
v2.
## The parameter-set pin, in both versions
The parameter-set pin (`wai:ParameterSetPin`, SPEC §3.1) is in v1 as additive terms,
and `wai registry export` writes the same terms into v2: `wai:prior` and the members
`wai:pinId`, `wai:pinSha256`, `wai:pinDeterminism`, `wai:pinForm`, `wai:pinTier` (a
closed vocabulary: `decode-equivalence`, `entropy-consistency`, `none`) and
`wai:pinSupersedes`, with a scoped JSON-LD term `prior` whose members are the
manifest's own names, a `ParameterSetPinShape` and a `SupersedesShape`. In v2 the
tier list is generated from the registered tokens (`codecs::TIER_TOKENS`).
`wai:sharedPriorUri` and `wai:sharedPriorHash` (v1 only) describe a located prior
with a BLAKE3 hash; they are not the manifest's pin. A library test in `wai-rs`
(`container::tests::vocab_pin_terms_match_prior_members`) reads both v1 and v2 and
holds each scoped term's members equal to the pin's serialized members and each
shape's tier list equal to the registered tokens; `check_shapes.py` validates
conforming and non-conforming pins against the v2 shapes.

33
wai/vocab/check_shapes.py
View file

@ -49,6 +49,30 @@ def wai2(*components, intent="improve"):
return {"wai": "1.1", "media": "video", "intent": intent, "renditions": renditions, "target": None}
SHA_A = "ab" * 32
SHA_B = "cd" * 32
def pin(**over):
"""A parameter-set pin (SPEC section 3.1); a member set to None is dropped."""
p = {"id": "m", "sha256": SHA_A, "determinism": "int", "form": "wai.prior.set",
"tier": "decode-equivalence", "supersedes": {"sha256": SHA_B}}
p.update(over)
return {k: v for k, v in p.items() if v is not None}
def pinned(prior):
m = wai1("wai.neural.int_mlicpp")
m["conditioning"] = {"kind": "int_mlicpp", "prior": prior}
return m
def pinned2(prior):
m = wai2("wai.neural.int_mlicpp", intent="replicate")
m["renditions"][0]["prior"] = prior
return m
# (name, manifest, must conform)
CASES = [
("WAI1 png, untyped", wai1("wai.image.png"), True),
@ -66,6 +90,15 @@ CASES = [
("WAI2 candidate companion", wai2("wai.video.av1", ("wai.splat.sog", "wai.video.av1")), False),
("WAI2 companion bound to a candidate base", wai2("wai.video.av1", ("wai.meta.hdr10plus", "wai.tensor.nnc")), False),
("WAI1 and WAI2 at once", dict(wai1("wai.image.png"), renditions=[{"capability": "wai.image.png", "kind": "png"}]), False),
# The parameter-set pin (SPEC section 3.1), under conditioning and on a WAI2 component.
("WAI1 pin with every member", pinned(pin()), True),
("WAI1 pin with only the required members", pinned(pin(form=None, tier=None, supersedes=None)), True),
("WAI1 pin with a tier token no revision registers", pinned(pin(tier="decode_equivalence")), False),
("WAI1 pin whose digest is not lowercase hex", pinned(pin(sha256=SHA_A.upper())), False),
("WAI1 pin with no determinism", pinned(pin(determinism=None)), False),
("WAI1 pin whose supersedes names no digest", pinned(pin(supersedes={"form": "wai.prior.set"})), False),
("WAI2 component pin", pinned2(pin()), True),
("WAI2 component pin with an unregistered tier token", pinned2(pin(tier="exact")), False),
]

9
wai/vocab/v1.jsonld
View file

@ -17,6 +17,15 @@
"codec": { "@id": "wai:codec", "@type": "@id" },
"path": { "@id": "wai:path", "@type": "@vocab" },
"capabilities_required": { "@id": "wai:capabilitiesRequired", "@container": "@list" },
"ParameterSetPin": "wai:ParameterSetPin",
"prior": { "@id": "wai:prior", "@context": {
"id": { "@id": "wai:pinId", "@type": "xsd:string" },
"sha256": { "@id": "wai:pinSha256", "@type": "xsd:string" },
"determinism": { "@id": "wai:pinDeterminism", "@type": "xsd:string" },
"form": { "@id": "wai:pinForm", "@type": "xsd:string" },
"tier": { "@id": "wai:pinTier" },
"supersedes": { "@id": "wai:pinSupersedes" }
} },
"shared_prior_uri": { "@id": "wai:sharedPriorUri", "@type": "@id" },
"shared_prior_hash": { "@id": "wai:sharedPriorHash", "@type": "xsd:string" },
"content_hash": { "@id": "wai:contentHash", "@type": "xsd:string" },

22
wai/vocab/v1.shapes.ttl
View file

@ -26,3 +26,25 @@ wai:NeuralPriorShape a sh:NodeShape ;
sh:property [ sh:path wai:sharedPriorUri ; sh:nodeKind sh:IRI ; sh:minCount 1 ; sh:maxCount 1 ] ;
sh:property [ sh:path wai:sharedPriorHash ; sh:datatype xsd:string ; sh:minCount 1 ; sh:maxCount 1 ;
sh:pattern "^[a-f0-9]{64}$" ] .
# The parameter-set pin (SPEC §3.1). The JSON-LD context types no `prior` object,
# so the shape targets whatever `wai:prior` points at rather than a class.
# `pinTier` is a closed vocabulary; the tokens are the ones `codecs::TIER_TOKENS`
# holds.
wai:ParameterSetPinShape a sh:NodeShape ;
sh:targetObjectsOf wai:prior ;
sh:property [ sh:path wai:pinId ; sh:datatype xsd:string ; sh:minCount 1 ; sh:maxCount 1 ] ;
sh:property [ sh:path wai:pinSha256 ; sh:datatype xsd:string ; sh:minCount 1 ; sh:maxCount 1 ;
sh:pattern "^[a-f0-9]{64}$" ] ;
sh:property [ sh:path wai:pinDeterminism ; sh:datatype xsd:string ; sh:minCount 1 ; sh:maxCount 1 ] ;
sh:property [ sh:path wai:pinForm ; sh:datatype xsd:string ; sh:maxCount 1 ] ;
sh:property [ sh:path wai:pinTier ; sh:datatype xsd:string ; sh:maxCount 1 ;
sh:in ( "decode-equivalence" "entropy-consistency" "none" ) ] ;
sh:property [ sh:path wai:pinSupersedes ; sh:node wai:SupersedesShape ; sh:maxCount 1 ] .
# `supersedes` names a set as a pin names one, by `sha256` and, if it is pinned in a
# declared form, `form`; it has no `id`, `determinism` or `tier`.
wai:SupersedesShape a sh:NodeShape ;
sh:property [ sh:path wai:pinSha256 ; sh:datatype xsd:string ; sh:minCount 1 ; sh:maxCount 1 ;
sh:pattern "^[a-f0-9]{64}$" ] ;
sh:property [ sh:path wai:pinForm ; sh:datatype xsd:string ; sh:maxCount 1 ] .

21
wai/vocab/v1.ttl
View file

@ -58,8 +58,25 @@ wai:modality a owl:ObjectProperty ; rdfs:domain wai:Manifest ; rd
wai:codec a owl:ObjectProperty ; rdfs:domain wai:Manifest ; rdfs:range wai:Codec ; rdfs:label "Selected codec" .
wai:path a owl:ObjectProperty ; rdfs:domain wai:Manifest ; rdfs:range wai:Path ; rdfs:label "Decode path" .
wai:capabilitiesRequired a owl:ObjectProperty ; rdfs:domain wai:Manifest ; rdfs:range wai:Capability ; rdfs:label "Required capabilities" .
wai:sharedPriorUri a owl:ObjectProperty ; rdfs:domain wai:NeuralPrior ; rdfs:label "Shared-prior URI" .
wai:sharedPriorHash a owl:DatatypeProperty ; rdfs:domain wai:NeuralPrior ; rdfs:range xsd:string ; rdfs:label "Shared-prior content hash (BLAKE3 hex)" .
wai:sharedPriorUri a owl:ObjectProperty ; rdfs:domain wai:NeuralPrior ; rdfs:label "Shared-prior URI" ;
rdfs:comment "Describes a located prior with a BLAKE3 hash; not the manifest's pin (SPEC §3.1), which is a SHA-256 over the whole parameter set and carries no location." .
wai:sharedPriorHash a owl:DatatypeProperty ; rdfs:domain wai:NeuralPrior ; rdfs:range xsd:string ; rdfs:label "Shared-prior content hash (BLAKE3 hex)" ;
rdfs:comment "Describes a located prior with a BLAKE3 hash; not the manifest's pin (SPEC §3.1), which is a SHA-256 over the whole parameter set and carries no location." .
# ─── The parameter-set pin (SPEC §3.1) ───────────────────────────────
wai:ParameterSetPin a owl:Class ; rdfs:label "Parameter-set pin" ;
rdfs:comment "The identity of the parameter set a payload was coded against: the SHA-256 of the whole set in its declared form, and how the decode is made reproducible (SPEC §3.1)." .
wai:prior a owl:ObjectProperty ; rdfs:range wai:ParameterSetPin ; rdfs:label "Parameter-set pin" .
wai:pinId a owl:DatatypeProperty ; rdfs:domain wai:ParameterSetPin ; rdfs:range xsd:string ; rdfs:label "Pin name (informative)" .
wai:pinSha256 a owl:DatatypeProperty ; rdfs:range xsd:string ; rdfs:label "SHA-256 of the parameter set in its form (64 lowercase hex)" ;
rdfs:comment "A member of a pin and of its supersedes object." .
wai:pinDeterminism a owl:DatatypeProperty ; rdfs:domain wai:ParameterSetPin ; rdfs:range xsd:string ; rdfs:label "How the decode is made reproducible" .
wai:pinForm a owl:DatatypeProperty ; rdfs:range xsd:string ; rdfs:label "Registered prior form" ;
rdfs:comment "A member of a pin and of its supersedes object." .
wai:pinTier a owl:DatatypeProperty ; rdfs:domain wai:ParameterSetPin ; rdfs:range xsd:string ; rdfs:label "Declared reproducibility tier" .
wai:pinSupersedes a owl:ObjectProperty ; rdfs:domain wai:ParameterSetPin ; rdfs:label "The parameter set this one replaces (informative)" ;
rdfs:comment "Names that set by pinSha256 and, if it is pinned in a declared form, pinForm; it is not itself a pin." .
wai:contentHash a owl:DatatypeProperty ; rdfs:range xsd:string ; rdfs:label "Content hash (BLAKE3 hex)" .
wai:size a owl:DatatypeProperty ; rdfs:range xsd:nonNegativeInteger ; rdfs:label "Byte size" .
wai:mediaType a owl:DatatypeProperty ; rdfs:range xsd:string ; rdfs:label "IANA media type" .

28
wai/vocab/v2.jsonld
View file

@ -43,6 +43,34 @@
"target": {
"@id": "https://wai.transaction.science/vocab/v2#target",
"@type": "@json"
},
"ParameterSetPin": "https://wai.transaction.science/vocab/v2#ParameterSetPin",
"prior": {
"@id": "https://wai.transaction.science/vocab/v2#prior",
"@context": {
"id": {
"@id": "https://wai.transaction.science/vocab/v2#pinId",
"@type": "xsd:string"
},
"sha256": {
"@id": "https://wai.transaction.science/vocab/v2#pinSha256",
"@type": "xsd:string"
},
"determinism": {
"@id": "https://wai.transaction.science/vocab/v2#pinDeterminism",
"@type": "xsd:string"
},
"form": {
"@id": "https://wai.transaction.science/vocab/v2#pinForm",
"@type": "xsd:string"
},
"tier": {
"@id": "https://wai.transaction.science/vocab/v2#pinTier"
},
"supersedes": {
"@id": "https://wai.transaction.science/vocab/v2#pinSupersedes"
}
}
}
}
}

25
wai/vocab/v2.shapes.ttl
View file

@ -65,6 +65,7 @@ wai:ManifestShape a sh:NodeShape ;
wai:wai.pc.gpcc
wai:wai.pc.vpcc
wai:wai.prior.nnc
wai:wai.prior.set
wai:wai.quantum.circuit
wai:wai.semantic.clip
wai:wai.semantic.perco2
@ -142,6 +143,7 @@ wai:ManifestShape a sh:NodeShape ;
wai:wai.pc.gpcc
wai:wai.pc.vpcc
wai:wai.prior.nnc
wai:wai.prior.set
wai:wai.quantum.circuit
wai:wai.semantic.clip
wai:wai.semantic.perco2
@ -233,6 +235,7 @@ wai:ComponentShape a sh:NodeShape ;
wai:wai.pc.gpcc
wai:wai.pc.vpcc
wai:wai.prior.nnc
wai:wai.prior.set
wai:wai.quantum.circuit
wai:wai.semantic.clip
wai:wai.semantic.perco2
@ -310,6 +313,7 @@ wai:ComponentShape a sh:NodeShape ;
wai:wai.pc.gpcc
wai:wai.pc.vpcc
wai:wai.prior.nnc
wai:wai.prior.set
wai:wai.quantum.circuit
wai:wai.semantic.clip
wai:wai.semantic.perco2
@ -337,3 +341,24 @@ wai:ComponentShape a sh:NodeShape ;
wai:wai.world.replay
wai:wai.world.scene
) ] .
# The parameter-set pin (SPEC section 3.1). The JSON-LD context types no prior object,
# so the shape targets whatever wai:prior points at rather than a class. pinTier is a
# closed vocabulary; the tokens are the ones codecs::TIER_TOKENS holds.
wai:ParameterSetPinShape a sh:NodeShape ;
sh:targetObjectsOf wai:prior ;
sh:property [ sh:path wai:pinId ; sh:datatype xsd:string ; sh:minCount 1 ; sh:maxCount 1 ] ;
sh:property [ sh:path wai:pinSha256 ; sh:datatype xsd:string ; sh:minCount 1 ; sh:maxCount 1 ;
sh:pattern "^[a-f0-9]{64}$" ] ;
sh:property [ sh:path wai:pinDeterminism ; sh:datatype xsd:string ; sh:minCount 1 ; sh:maxCount 1 ] ;
sh:property [ sh:path wai:pinForm ; sh:datatype xsd:string ; sh:maxCount 1 ] ;
sh:property [ sh:path wai:pinTier ; sh:datatype xsd:string ; sh:maxCount 1 ;
sh:in ( "decode-equivalence" "entropy-consistency" "none" ) ] ;
sh:property [ sh:path wai:pinSupersedes ; sh:node wai:SupersedesShape ; sh:maxCount 1 ] .
# supersedes names a set as a pin names one, by sha256 and, if it is pinned in a
# declared form, form; it has no id, determinism or tier.
wai:SupersedesShape a sh:NodeShape ;
sh:property [ sh:path wai:pinSha256 ; sh:datatype xsd:string ; sh:minCount 1 ; sh:maxCount 1 ;
sh:pattern "^[a-f0-9]{64}$" ] ;
sh:property [ sh:path wai:pinForm ; sh:datatype xsd:string ; sh:maxCount 1 ] .

27
wai/vocab/v2.ttl
View file

@ -25,6 +25,7 @@ wai:Role a owl:Class ; rdfs:label "Dispatch role" .
wai:Tier a owl:Class ; rdfs:label "Fidelity tier" .
wai:Licence a owl:Class ; rdfs:label "Licence class" .
wai:ReplicateClass a owl:Class ; rdfs:label "Replicate class" .
wai:ParameterSetPin a owl:Class ; rdfs:label "Parameter-set pin (SPEC section 3.1)" .
# ─── Enumerations (individuals named <class prefix>-<label>) ─────────
@ -84,6 +85,20 @@ wai:specSection a owl:DatatypeProperty ; rdfs:domain wai:Capability ; rdfs:range
wai:edition a owl:DatatypeProperty ; rdfs:domain wai:Capability ; rdfs:range xsd:string ; rdfs:label "Edition the registration names" .
wai:registeredOn a owl:DatatypeProperty ; rdfs:domain wai:Capability ; rdfs:range xsd:date ; rdfs:label "Date registered" .
# ─── The parameter-set pin (SPEC section 3.1) ────────────────────────
wai:prior a owl:ObjectProperty ; rdfs:range wai:ParameterSetPin ; rdfs:label "Parameter-set pin (conditioning.prior, or a component's prior)" .
wai:pinId a owl:DatatypeProperty ; rdfs:domain wai:ParameterSetPin ; rdfs:range xsd:string ; rdfs:label "Pin name (informative)" .
wai:pinSha256 a owl:DatatypeProperty ; rdfs:range xsd:string ; rdfs:label "SHA-256 of the parameter set in its form (64 lowercase hex)" ;
rdfs:comment "A member of a pin and of its supersedes object." .
wai:pinDeterminism a owl:DatatypeProperty ; rdfs:domain wai:ParameterSetPin ; rdfs:range xsd:string ; rdfs:label "How the decode is made reproducible" .
wai:pinForm a owl:DatatypeProperty ; rdfs:range xsd:string ; rdfs:label "Registered prior form" ;
rdfs:comment "A member of a pin and of its supersedes object." .
wai:pinSupersedes a owl:ObjectProperty ; rdfs:domain wai:ParameterSetPin ; rdfs:label "The parameter set this one replaces (informative)" ;
rdfs:comment "Names that set by pinSha256 and, if it is pinned in a declared form, pinForm; it is not itself a pin." .
wai:pinTier a owl:DatatypeProperty ; rdfs:domain wai:ParameterSetPin ; rdfs:range xsd:string ; rdfs:label "Declared reproducibility tier" ;
rdfs:comment "A closed vocabulary: decode-equivalence, entropy-consistency, none." .
# ─── Registered strings ──────────────────────────────────────────────
wai:wai.asset.provenance a wai:Capability ;
@ -775,6 +790,18 @@ wai:wai.prior.nnc a wai:Capability ;
wai:specSection "§5 Prior artifacts, general tensors and derivation operations" ;
wai:registeredOn "2026-09-28"^^xsd:date .
wai:wai.prior.set a wai:Capability ;
rdfs:label "wai.prior.set" ;
wai:kind wai:kind-payload ;
wai:status wai:status-registered ;
wai:media wai:media-binary ;
wai:role wai:role-standalone ;
wai:tier wai:tier-lossless ;
wai:licence wai:licence-royalty-free ;
wai:replicateClass wai:replicate-standard-defined ;
wai:specSection "§5 Prior artifacts, general tensors and derivation operations" ;
wai:registeredOn "2026-09-30"^^xsd:date .
wai:wai.quantum.calibration a wai:Capability ;
rdfs:label "wai.quantum.calibration" ;
wai:kind wai:kind-record ;

1
wai/wai-quantum-serve/Cargo.lock generated
View file

@ -583,6 +583,7 @@ dependencies = [
"pkg-config",
"serde",
"serde_json",
"sha2",
"wai-quantum",
]

1
wai/wai-relay/Cargo.lock generated
View file

@ -1085,6 +1085,7 @@ dependencies = [
"pkg-config",
"serde",
"serde_json",
"sha2",
]
[[package]]

45
wai/wai-rs/Cargo.toml
View file

@ -66,7 +66,10 @@ byteorder = { version = "1", optional = true }
ort = { version = "2.0.0-rc.12", default-features = false, features = ["std", "download-binaries", "ndarray", "tls-rustls"], optional = true }
ndarray = { version = "0.17", optional = true }
zstd-safe = { version = "7", optional = true, features = ["std"] } # for bmshj2018 latent unpack (std → WriteBuf for Vec)
sha2 = { version = "0.10", optional = true } # verify a pinned neural prior against the registered model artifact
# SHA-256 is WAI's digest for parameter-set identity, component identity and decoded-output
# conformance digests (SPEC §3.1, §7 *Digests*, §7.1). Pure Rust, so it is always on: the
# envelope reader verifies a component digest and a pin with no feature enabled.
sha2 = "0.10"
# ---- real C2PA manifest emission (optional, behind `c2pa_emit`) ----
# Writes an actual signed manifest carrying the energy + reconstruction
@ -103,7 +106,12 @@ codecs = [
"dep:opus", "dep:ogg", "dep:claxon", "dep:flac-bound",
"dep:rav1e", "dep:dav1d",
"dep:zstd", "dep:xz2", "dep:byteorder",
"ffi",
]
# The C ABI's envelope surface (buffers, errors, WAI1/WAI2 pack and unpack, §7.1
# selection), with no codec library: it builds and is tested where libjxl and the
# other C codec libraries are absent. `codecs` adds the codec wrappers to it.
ffi = []
# Pulls in the ONNX runtime + zstd unpack so the native sink can
# decode `wai.neural.<...>` envelopes. Off by default — the bare
# container + zeroth codecs build with no ML dep chain.
@ -135,7 +143,7 @@ world_int = ["neural_int"]
# learned transform over the [14,S,S] attribute image → byte-exact attribute tensor
# (splat-equivalence, a portable cloud hash). Reuses int_hyper_synth (neural_int).
splat_int = ["neural_int"]
neural = ["neural_int", "dep:ort", "dep:ndarray", "dep:zstd-safe", "dep:sha2"]
neural = ["neural_int", "dep:ort", "dep:ndarray", "dep:zstd-safe"]
# The perceptual / generative-semantic path: CLIP-similarity conformance,
# regenerate-at-sink. Reuses the ONNX runtime for the reference CLIP embedder;
# the generator is always sink-supplied. Off by default like `neural`.
@ -162,12 +170,12 @@ score = ["world"]
# Asset provenance (extensions/provenance): a signed, energy-accounted
# lineage receipt for opaque captured content WAI does not reconstruct.
# Reuses the JWP Merkle + Ed25519; no codec/ML deps.
provenance = ["world", "dep:sha2"]
provenance = ["world"]
# OER/2 outcome-and-energy receipts (extensions/oer-receipts.md): a verifier for
# receipts that grade what a unit of work cost (projected/simulated/derived/measured),
# content-address and chain it, and sign it. Conformance is reason-set equality with
# the reference verifier and the portable C kernel over oer-conformance/.
oer = ["dep:sha2", "dep:blake3", "dep:ed25519-dalek", "dep:num-bigint", "serde_json/float_roundtrip"]
oer = ["dep:blake3", "dep:ed25519-dalek", "dep:num-bigint", "serde_json/float_roundtrip"]
# Typed receipt classes on the JWP chain (extensions/jwp-receipts.md §2.4.3–§2.4.7,
# §2.6, §3.1–§3.2): `claim` receipts of class energy / lawful-basis / log-inclusion,
# the soft-binding class, and recovery under the ancestor rule. Needs `provenance`
@ -503,6 +511,10 @@ name = "wai_oer_diff"
path = "src/bin/wai_oer_diff.rs"
required-features = ["oer"]
[[bin]]
name = "wai_pin_vectors"
path = "src/bin/wai_pin_vectors.rs"
[[bin]]
name = "wai_claim_vectors"
path = "src/bin/wai_claim_vectors.rs"
@ -594,6 +606,31 @@ name = "int_mlicpp_preflight"
path = "tests/int_mlicpp_preflight.rs"
required-features = ["neural_int"]
[[test]]
name = "generate"
path = "tests/generate.rs"
required-features = ["generate"]
[[test]]
name = "ffi_envelope"
path = "tests/ffi_envelope.rs"
required-features = ["ffi"]
[[test]]
name = "ffi_neural_pin"
path = "tests/ffi_neural_pin.rs"
required-features = ["neural", "ffi"]
[[test]]
name = "neural_native_decode"
path = "tests/neural_native_decode.rs"
required-features = ["neural"]
[[test]]
name = "jpegai_e2e_full"
path = "tests/jpegai_e2e_full.rs"
required-features = ["neural_int"]
[dev-dependencies]
serde_json = { version = "1", features = ["preserve_order"] } # construct manifests in tests

2
wai/wai-rs/build.rs
View file

@ -27,6 +27,8 @@ fn main() {
/// via `rerun-if-changed` declarations).
fn generate_c_header() {
println!("cargo:rerun-if-changed=src/ffi.rs");
println!("cargo:rerun-if-changed=src/ffi_codecs.rs");
println!("cargo:rerun-if-changed=src/ffi_neural.rs");
println!("cargo:rerun-if-changed=src/container.rs");
println!("cargo:rerun-if-changed=cbindgen.toml");
let crate_dir = std::env::var("CARGO_MANIFEST_DIR").unwrap();

1
wai/wai-rs/cbindgen.toml
View file

@ -40,6 +40,7 @@ prefix = ""
# internal helpers out of the public C surface.
[export.rename]
"WaiBuffer" = "wai_buffer_t"
"WaiDecoded" = "wai_decoded_t"
[fn]
prefix = ""

269
wai/wai-rs/include/wai.h
View file

@ -40,6 +40,21 @@
#define WAI_ERR_PANIC 4
/**
* A pin was refused (SPEC §3.1); `wai_last_error` starts with the reason.
*/
#define WAI_ERR_PRIOR 5
/**
* `intent = replicate` is not sound for the capability decoded through (SPEC §7).
*/
#define WAI_ERR_UNSOUND_REPLICATE 6
/**
* No rendition of a `WAI2` envelope can be picked (SPEC §7.1).
*/
#define WAI_ERR_NO_RENDITION 7
/**
* Probability output scale of [`softmax`]: results are at `2^PROB_FRAC`
* (they sum to ~`2^PROB_FRAC`).
@ -81,6 +96,26 @@
#define MARKER_UDI 65420
/**
* The media kind of a [`WaiDecoded`].
*/
#define WAI_DECODED_AUDIO 1
/**
* The media kind of a [`WaiDecoded`].
*/
#define WAI_DECODED_IMAGE 2
/**
* The media kind of a [`WaiDecoded`].
*/
#define WAI_DECODED_VIDEO 3
/**
* `wai_neural_decode_ex` flag: apply SPEC §7's replicate rule (strict).
*/
#define WAI_DECODE_STRICT 1
#define SECTION_CONTRACT 1
#define SECTION_GRAPH 2
@ -310,6 +345,29 @@ typedef struct wai_buffer_t {
uintptr_t cap;
} wai_buffer_t;
/**
* What `wai_neural_decode_ex` decoded.
*
* `kind` is `WAI_DECODED_AUDIO` (`data` holds `f32` samples, `sample_rate` set),
* `WAI_DECODED_IMAGE` (`data` holds RGB8, `width` and `height` set) or
* `WAI_DECODED_VIDEO` (`data` holds the frames' RGB8 concatenated, `width`,
* `height`, `n_frames` and `fps_x_1000` set). `index` is the `WAI2` rendition
* picked (0 for a `WAI1` envelope). Free `data` with `wai_buffer_free`.
* `wai_neural_decode_ex` writes the whole struct on every call that passes a
* non-null `out`: on failure `kind` is 0 and `data` is empty, so `data` is always
* safe to free.
*/
typedef struct wai_decoded_t {
uint32_t kind;
uint32_t index;
struct wai_buffer_t data;
uint32_t width;
uint32_t height;
uint32_t sample_rate;
uint32_t n_frames;
uint32_t fps_x_1000;
} wai_decoded_t;
@ -339,6 +397,106 @@ extern "C" {
*/
const char *wai_last_error(void);
/**
* Builds a complete WAI file from a codec-produced payload + manifest
* fields. `manifest_json` is a UTF-8 JSON object overriding/supplying
* any fields; `payload` is the bytes from one of the codec encoders.
* Refuses (`WAI_ERR_INVALID_ARG`) a manifest that names a candidate capability
* as its capability or fallback, which an encoder MUST NOT emit (SPEC §5,
* *Candidate capabilities*): it emits through `Wai::to_emit_bytes`.
*/
int wai_envelope_pack(const char *manifest_json,
const uint8_t *payload,
uintptr_t payload_len,
struct wai_buffer_t *out);
/**
* Detect WAI version from a byte stream. Returns 1 (WAI1), 2 (WAI2),
* or 0 with an error message in `wai_last_error` for unknown magic.
*/
int wai_envelope_detect_version(const uint8_t *bytes, uintptr_t len);
/**
* Pack a WAI2 multi-rendition envelope.
*
* - `manifest_json` must declare a `renditions` array of length `n`.
* - `payload_data` is an array of `n` byte-buffer pointers.
* - `payload_lens` is the matching array of `n` lengths.
* - The i-th payload pairs with `manifest.renditions[i]`.
*
* Refuses (`WAI_ERR_INVALID_ARG`) a table with a component, in either role,
* whose capability is a candidate (SPEC §5, *Candidate capabilities*): it emits
* through `WaiMulti::to_emit_bytes`.
*
* # Safety
* All array pointers must point to valid memory of the declared length.
*/
int wai_envelope_pack_multi(const char *manifest_json,
const uint8_t *const *payload_data,
const uintptr_t *payload_lens,
uintptr_t n_payloads,
struct wai_buffer_t *out);
/**
* Parse a WAI2 envelope. Writes three outputs:
*
* - `out_manifest` — the manifest JSON exactly as the envelope carries it
* (UTF-8 bytes, no NUL), so members this library does not model survive
* - `out_renditions_json` — a JSON array with one object per component:
* `capability`, `kind`, `offset` (into `out_payload_block`), `length`, and
* `selectable` (whether SPEC §7.1 may pick it: a companion, or a companion-only
* capability, never is), plus `id`, `sha256`, `prior`, `role`, `companion_of`
* and `companion_of_id` when the entry carries them. A C sink selects with
* `wai_envelope_select` rather than by hand.
* - `out_payload_block` — every rendition's payload bytes, packed
* contiguously. The caller indexes via the JSON above.
*
* This avoids exposing a C struct array (and the ownership pain that
* comes with it). Any JSON parser the consumer already has decodes
* the rendition table in two lines.
*/
int wai_envelope_unpack_multi(const uint8_t *bytes,
uintptr_t len,
struct wai_buffer_t *out_manifest,
struct wai_buffer_t *out_renditions_json,
struct wai_buffer_t *out_payload_block);
/**
* Parses a WAI envelope. Writes the manifest JSON exactly as the envelope
* carries it to `out_manifest` (so `security`, and members this library does not
* model, survive) and the raw payload to `out_payload`.
*/
int wai_envelope_unpack(const uint8_t *bytes,
uintptr_t len,
struct wai_buffer_t *out_manifest,
struct wai_buffer_t *out_payload);
/**
* SPEC §7.1 selection for a `WAI2` envelope at a sink that advertises the
* capabilities in `advertised_json` (a NUL-terminated JSON array of capability
* strings). Writes the index of the picked rendition to `out_index`: the first
* entry that is selectable (never a companion, nor a capability that is not
* dispatchable under SPEC §4: companion-only, a candidate, a record or a
* derivation), advertised, and whose payload matches its `sha256`. This call holds no
* parameter set, so an entry whose pin applies is skipped (`prior_not_held`); a
* sink that holds parameter sets decodes through `wai_neural_decode_ex`, which
* verifies each pin. Returns `WAI_ERR_NO_RENDITION` when nothing is picked, with
* each entry's index and reason in `wai_last_error`.
*
* # Safety
* `envelope` valid for `envelope_len` bytes; `advertised_json` NUL-terminated;
* `out_index` writable.
*/
int wai_envelope_select(const uint8_t *envelope,
uintptr_t envelope_len,
const char *advertised_json,
uint32_t *out_index);
/**
* Encode RGB pixels (row-major, 3 bytes per pixel) as PNG.
*
@ -466,87 +624,14 @@ int wai_video_av1_decode(const uint8_t *bytes,
uint32_t *out_fps_num,
uint32_t *out_fps_den);
/**
* Builds a complete WAI file from a codec-produced payload + manifest
* fields. `manifest_json` is a UTF-8 JSON object overriding/supplying
* any fields; `payload` is the bytes from one of the codec encoders.
* Refuses (`WAI_ERR_INVALID_ARG`) a manifest that names a candidate capability
* as its capability or fallback, which an encoder MUST NOT emit (SPEC §5,
* *Candidate capabilities*): it emits through `Wai::to_emit_bytes`.
*/
int wai_envelope_pack(const char *manifest_json,
const uint8_t *payload,
uintptr_t payload_len,
struct wai_buffer_t *out);
/**
* Detect WAI version from a byte stream. Returns 1 (WAI1), 2 (WAI2),
* or 0 with an error message in `wai_last_error` for unknown magic.
*/
int wai_envelope_detect_version(const uint8_t *bytes, uintptr_t len);
/**
* Pack a WAI2 multi-rendition envelope.
*
* - `manifest_json` must declare a `renditions` array of length `n`.
* - `payload_data` is an array of `n` byte-buffer pointers.
* - `payload_lens` is the matching array of `n` lengths.
* - The i-th payload pairs with `manifest.renditions[i]`.
*
* Refuses (`WAI_ERR_INVALID_ARG`) a table with a component, in either role,
* whose capability is a candidate (SPEC §5, *Candidate capabilities*): it emits
* through `WaiMulti::to_emit_bytes`.
*
* # Safety
* All array pointers must point to valid memory of the declared length.
*/
int wai_envelope_pack_multi(const char *manifest_json,
const uint8_t *const *payload_data,
const uintptr_t *payload_lens,
uintptr_t n_payloads,
struct wai_buffer_t *out);
/**
* Parse a WAI2 envelope. Writes three outputs:
*
* - `out_manifest` — the original manifest JSON (UTF-8 bytes, no NUL)
* - `out_renditions_json` — a JSON array of `{capability, kind, offset,
* length}` entries. `offset` is into `out_payload_block`, `length`
* is the rendition's payload byte count.
* - `out_payload_block` — every rendition's payload bytes, packed
* contiguously. The caller indexes via the JSON above.
*
* This avoids exposing a C struct array (and the ownership pain that
* comes with it). Any JSON parser the consumer already has decodes
* the rendition table in two lines.
*/
int wai_envelope_unpack_multi(const uint8_t *bytes,
uintptr_t len,
struct wai_buffer_t *out_manifest,
struct wai_buffer_t *out_renditions_json,
struct wai_buffer_t *out_payload_block);
/**
* Parses a WAI envelope. Writes the manifest as JSON to `out_manifest`
* and the raw payload to `out_payload`.
*/
int wai_envelope_unpack(const uint8_t *bytes,
uintptr_t len,
struct wai_buffer_t *out_manifest,
struct wai_buffer_t *out_payload);
/**
* Decode a `wai.neural.<audio>` envelope (encodec32 / dac / mimi /
* wavtokenizer). Writes mono `f32` samples into `out_samples`; the
* wavtokenizer / snac). Writes mono `f32` samples into `out_samples`; the
* caller reads them as `float*` of length `out_samples.len / 4`.
*
* # Safety
* `envelope` valid for `envelope_len` bytes. `model_path` NUL-terminated
* UTF-8 path to the decoder.onnx. `out_samples` writable `WaiBuffer*`.
* UTF-8 path to the decoder. `out_samples` writable `WaiBuffer*`.
* `out_sample_rate` writable `uint32_t*` (or null to skip).
*/
@ -557,9 +642,8 @@ int wai_neural_decode_audio(const uint8_t *envelope,
uint32_t *out_sample_rate);
/**
* Decode a `wai.neural.bmshj2018` envelope. Writes packed RGB
* (row-major, 3 bytes/pixel) into `out_rgb`; writes width/height
* into `*out_width` / `*out_height`.
* Decode a neural image envelope. Writes packed RGB (row-major, 3 bytes/pixel)
* into `out_rgb`; writes width/height into `*out_width` / `*out_height`.
*
* # Safety
* See `wai_neural_decode_audio`.
@ -573,9 +657,9 @@ int wai_neural_decode_image(const uint8_t *envelope,
uint32_t *out_height);
/**
* Decode a `wai.neural.video_bmshj2018` envelope. Writes all frames
* concatenated (frame 0 RGB, then frame 1 RGB, …) into
* `out_frames_concat`; the caller splits at `width*height*3` strides.
* Decode a neural video envelope. Writes all frames concatenated (frame 0
* RGB, then frame 1 RGB, …) into `out_frames_concat`; the caller splits at
* `width*height*3` strides.
*
* # Safety
* See `wai_neural_decode_audio`.
@ -590,6 +674,29 @@ int wai_neural_decode_video(const uint8_t *envelope,
uint32_t *out_height,
uint32_t *out_fps_x_1000);
/**
* Decode a `WAI1` or `WAI2` envelope against the parameter sets in
* `registry_json`: a NUL-terminated JSON object mapping each capability to an
* array of paths. The first path is the registered artifact; the others are
* further sets a pin may resolve against (a set and its successor). Every pin is
* verified first (SPEC §3.1); a `WAI2` rendition is picked by SPEC §7.1. With
* `flags & WAI_DECODE_STRICT`, SPEC §7's replicate rule applies too.
*
* `*out` is written on every call with a non-null `out`, before anything else is
* read: on failure it holds `kind` 0 and an empty `data`, so the caller may free
* `out->data` whatever the status.
*
* # Safety
* `envelope` valid for `envelope_len` bytes; `registry_json` NUL-terminated;
* `out` a writable `WaiDecoded*`, which need not be initialised.
*/
int wai_neural_decode_ex(const uint8_t *envelope,
uintptr_t envelope_len,
const char *registry_json,
uint32_t flags,
struct wai_decoded_t *out);
#ifdef __cplusplus
} // extern "C"
#endif // __cplusplus

108
wai/wai-rs/src/bin/wai.rs
View file

@ -34,6 +34,7 @@ fn usage() {
eprintln!("wai registry export [--root <wai-dir>] [--check]");
eprintln!(" write (or, with --check, compare) registry/capabilities.json, its schema,");
eprintln!(" vocab/v2.* and the generated table of REGISTRY.md");
eprintln!("wai prior-set <root> print the listing of a parameter set (wai.prior.set) and its pin");
eprintln!();
eprintln!("encode and decode need a build with the `codecs` feature.");
eprintln!("codecs: png jpeg avif jxl jxl-lossless opus flac av1 av1-lossless zstd xz");
@ -198,19 +199,7 @@ fn encode(codec: &str, input: &str, out: &str, q: u8) -> Result<(), String> {
c => return Err(format!("unknown codec '{c}'")),
};
let manifest = Manifest {
wai: "1.0".into(),
media: media_for(cap).into(),
intent: "replicate".into(),
model_requirement: ModelRequirement {
capability: cap.into(),
fallback: None,
min_sim: None,
sim_metric: None,
},
conditioning: Conditioning { kind: kind.into(), prior: None },
target: serde_json::Value::Null,
};
let manifest = Manifest::new(media_for(cap), "replicate", ModelRequirement::new(cap), Conditioning::new(kind));
let n = Wai::new(manifest, payload).write(out).map_err(|e| e.to_string())?;
println!("encoded → {out} ({} bytes, capability {})", n, cap);
Ok(())
@ -224,7 +213,10 @@ fn decode(input: &str, out: &str) -> Result<(), String> {
// A companion-only capability is never dispatched standalone (SPEC §5).
let native = wai::sink_capabilities();
let roles = wai.check_component_roles();
let Some(cap) = wai.dispatch_capability(|c| native.contains(&c)) else {
// The capabilities this crate decodes natively decode against no parameter
// set, so no pin applies to them and the verifier is never called.
let dispatched = wai.dispatch_verified(|c| native.contains(&c), wai::container::no_parameter_sets());
let Ok(wai::container::Dispatch { capability: cap, .. }) = dispatched else {
return Err(match roles {
Err(e) => format!("{e} (SPEC \u{a7}5)"),
Ok(()) => format!("inert at this sink: {} is not decoded here{}",
@ -318,12 +310,23 @@ fn inspect(path: &str) -> Result<(), String> {
let usable = |c: &str| known.contains(c) && wai::codecs::is_dispatchable(c);
let cap_ok = usable(cap);
let fb_ok = fb.map(usable);
let chosen = w.dispatch_capability(|c| known.contains(c));
// inspect holds no parameter sets: a capability whose pin applies is
// refused here (prior_not_held), which the verdict reports.
let chosen = w.dispatch_verified(|c| known.contains(c), wai::container::no_parameter_sets())
.ok().map(|d| d.capability);
let roles = w.check_component_roles();
println!(" payload size: {:>10} B", w.payload.len());
println!(" media: {}", w.media());
println!(" intent: {}", w.intent());
println!(" conditioning.kind: {}", w.kind());
match w.prior() {
Some(p) => println!(" pin: {}", wai::pin::describe_pin(p)),
None => println!(" pin: (none)"),
}
match w.check_pin() {
Ok(()) => println!(" pin check: \u{2713} no static finding (SPEC \u{a7}3.1, \u{a7}7)"),
Err(f) => println!(" pin check: \u{2717} {f}"),
}
println!();
println!(" capability: {cap}");
println!(" resolution: {}",
@ -345,6 +348,9 @@ fn inspect(path: &str) -> Result<(), String> {
Some(_) => "DECODABLE via declared fallback only if its decoder accepts this payload (\u{a7}4 step 5)",
None if fb.is_some_and(wai::codecs::is_companion_only) =>
"INERT here \u{2014} capability missing; the fallback is companion-only and never dispatched",
None if w.prior().is_some() && [Some(cap), fb].into_iter().flatten()
.any(|c| known.contains(c) && wai::container::pin_applies_to(c)) =>
"NOT DECODED by inspect \u{2014} the pin applies, and inspect holds no parameter sets (prior_not_held)",
None if fb.is_some() => "INERT here \u{2014} neither capability usable",
None => "INERT here \u{2014} capability missing, no fallback declared",
};
@ -367,28 +373,47 @@ fn inspect(path: &str) -> Result<(), String> {
println!();
println!(" rendition table (deployer-preferred order):");
println!(" {:>3} {:>10} {:<28} {}", "idx", "bytes", "capability", "kind");
let mut picked = None;
for (i, (meta, payload)) in
m.manifest.renditions.iter().zip(m.renditions.iter()).enumerate()
{
let avail = known.contains(meta.capability.as_str());
let mark = if avail { "\u{2713}" } else { "\u{2717}" };
// Companions (and companion-only capabilities) are applied with the
// chosen rendition, never picked instead of it (SPEC §7.1).
if avail && meta.is_selectable() && picked.is_none() { picked = Some(i); }
let role = match (meta.is_companion(), meta.companion_of.as_deref()) {
(true, Some(of)) => format!(" companion of {of}"),
(true, Some(of)) => format!(" companion of {of}{}",
meta.companion_of_id.as_deref().map(|id| format!(" (id {id})")).unwrap_or_default()),
(true, None) => " companion (unbound)".to_string(),
(false, _) => String::new(),
};
println!(" [{i}] {:>10} {:<28} {} {}{role}",
payload.len(), meta.capability, meta.kind, mark);
if let Some(id) = meta.id.as_deref() {
println!(" id {id}");
}
if let Some(d) = meta.sha256.as_deref() {
let ok = if meta.payload_digest_matches(payload) { "\u{2713}" } else { "\u{2717} component_digest_mismatch" };
println!(" sha256 {} {ok}", wai::pin::digest_preview(d));
}
if let Some(p) = meta.prior.as_ref() {
println!(" pin {}", wai::pin::describe_pin(p));
}
}
println!();
match picked {
Some(i) => println!(" default pick: [{i}] {} \u{2014} first rendition this sink can decode",
m.manifest.renditions[i].capability),
None => println!(" default pick: none \u{2014} no rendition's capability installed here"),
// Companions (and companion-only capabilities) are applied with the chosen
// rendition, never picked instead of it (SPEC §7.1). inspect holds no
// parameter sets, so a rendition whose pin applies is skipped.
match m.pick(|c| known.contains(c), wai::container::no_parameter_sets()) {
Ok(p) => println!(" default pick: [{}] {} \u{2014} pins not checked: inspect holds no parameter sets, so a pinned rendition is skipped",
p.index, p.meta.capability),
Err(skips) => {
println!(" default pick: none \u{2014} pins not checked: inspect holds no parameter sets");
for (i, why) in skips {
println!(" [{i}] {why}");
}
}
}
match m.check_pins() {
Ok(()) => println!(" pin check: \u{2713} no static finding (SPEC \u{a7}3.1, \u{a7}7)"),
Err((i, f)) => println!(" pin check: \u{2717} [{i}] {f}"),
}
match m.check_component_roles() {
Ok(()) => println!(" component roles: \u{2713} conform (SPEC \u{a7}5, \u{a7}7.1)"),
@ -411,6 +436,16 @@ fn inspect(path: &str) -> Result<(), String> {
Ok(())
}
/// `wai prior-set <root>` — print the listing of every file below `root` (the
/// `wai.prior.set` form, SPEC §3.1) to stdout, and the pin, its SHA-256, to stderr.
fn prior_set(root: &str) -> Result<(), String> {
let listing = wai::pin::listing_of_dir(std::path::Path::new(root)).map_err(|e| e.to_string())?;
use std::io::Write;
std::io::stdout().write_all(&listing).map_err(|e| e.to_string())?;
eprintln!("pin (form wai.prior.set): sha256 {}", wai::pin::sha256_hex(&listing));
Ok(())
}
/// The manifest `media` value for a capability: the registry's media class
/// (`codecs::capability_media`), or `binary` for a string it does not class.
fn media_for(cap: &str) -> &'static str {
@ -485,19 +520,11 @@ fn wrap(cap: &str, input: &str, out: &str, fallback: Option<&str>,
}
let payload = std::fs::read(input).map_err(|e| format!("read {input}: {e}"))?;
let n_in = payload.len();
let manifest = Manifest {
wai: "1.0".into(),
media: media_for(cap).into(),
intent: intent.into(),
model_requirement: ModelRequirement {
capability: cap.into(),
fallback: fallback.map(String::from),
min_sim: None,
sim_metric: None,
},
conditioning: Conditioning { kind: "bytes".into(), prior: None },
target: serde_json::Value::Null,
let requirement = match fallback {
Some(fb) => ModelRequirement::new(cap).with_fallback(fb),
None => ModelRequirement::new(cap),
};
let manifest = Manifest::new(media_for(cap), intent, requirement, Conditioning::new("bytes"));
let envelope = Wai::new(manifest, payload);
// The reference tool does not write an envelope that names a candidate, or
// one that breaks the role rules.
@ -540,7 +567,7 @@ fn wrap_with_companions(cap: &str, payload: Vec<u8>, out: &str, fallback: Option
reconstruction claim"))?;
let bytes = std::fs::read(p).map_err(|e| format!("read {p}: {e}"))?;
let mut comp = wai::container::Rendition::companion(c.as_str(), bytes, cap);
comp.kind = "bytes".into();
comp.meta.kind = "bytes".into();
components.push(comp);
}
let envelope = wai::container::WaiMulti::new(
@ -615,8 +642,10 @@ fn unwrap_component(input: &str, bytes: &[u8], out: Option<&str>,
let m = wai::container::WaiMulti::from_bytes(bytes).map_err(|e| e.to_string())?;
let idx = match component {
Some(s) => s.parse::<usize>().map_err(|_| format!("--component takes an index, got '{s}'"))?,
None => m.manifest.renditions.iter().position(|r| r.is_selectable())
.ok_or("no selectable rendition; name one with --component <i>")?,
// SPEC §7.1 selection at a sink that advertises everything and holds no
// parameter set (pins not checked: a pinned rendition is skipped).
None => m.primary(wai::container::no_parameter_sets()).map(|p| p.index)
.map_err(|_| "no selectable rendition without a pin; name one with --component <i>")?,
};
let (meta, payload) = m.manifest.renditions.get(idx).zip(m.renditions.get(idx))
.ok_or_else(|| format!("component {idx} out of range ({} components)", m.renditions.len()))?;
@ -745,6 +774,7 @@ fn main() -> ExitCode {
Ok(false) => return ExitCode::from(1),
Err(e) => Err(e),
},
"prior-set" if args.len() >= 2 => prior_set(&args[1]),
_ => { usage(); return ExitCode::from(2); }
};
match r {

22
wai/wai-rs/src/bin/wai_int_synth_vectors.rs
View file

@ -13,9 +13,10 @@
//!
//! Each vector is a realistic multi-layer integer decoder: a committed
//! `<name>.int_synth.json` (the model) + a `<name>.bin` payload (header +
//! int8 latents) + a frozen BLAKE3 of the decoded RGB. `verify` re-decodes
//! and checks the hash — in CI on Linux x86_64, Linux arm64 and macOS arm64,
//! on every push that touches it.
//! int8 latents) + a frozen digest of the decoded RGB: the SHA-256 in
//! `sha256.json` (the conformance digest, SPEC §7 *Digests*) and a domain-separated
//! BLAKE3 in `expected_hashes.json` (a diagnostic). `verify` re-decodes and checks
//! both — on every platform leg of the conformance workflow.
//!
//! wai_int_synth_vectors gen <dir> | verify <dir>
@ -34,6 +35,8 @@ fn main() {
}
}
const RGB8: &str = "RGB8: rows top to bottom, pixels left to right, R, G, B bytes";
fn hex(b: &[u8]) -> String {
b.iter().map(|x| format!("{x:02x}")).collect()
}
@ -146,6 +149,9 @@ fn generate(dir: &Path) {
let v = dir.join("vectors");
std::fs::create_dir_all(&v).expect("mkdir");
let mut manifest = serde_json::Map::new();
// SPEC §7 *Digests*: the SHA-256 of each decoded RGB8 buffer, in a sidecar so
// expected_hashes.json keeps its BLAKE3 values unchanged.
let mut sha = serde_json::Map::new();
for (name, seed, s_log2, c, layers) in vectors() {
let (h, w) = output_hw(1 << s_log2, &layers);
// One stream authors weights then the latent, deterministically.
@ -166,6 +172,7 @@ fn generate(dir: &Path) {
hasher.update(b"wai:int-synth-rgb\x01");
hasher.update(&img.rgb);
let rgb_hash = hex(hasher.finalize().as_bytes());
sha.insert(name.to_string(), serde_json::json!({ "buffer": RGB8, "sha256": wai::pin::sha256_hex(&img.rgb) }));
manifest.insert(
name.to_string(),
@ -210,7 +217,13 @@ fn generate(dir: &Path) {
}
}
manifest.insert("photo".to_string(), entry);
sha.insert("photo".to_string(), serde_json::json!({ "buffer": RGB8, "sha256": wai::pin::sha256_hex(&img.rgb) }));
}
std::fs::write(
dir.join("sha256.json"),
serde_json::to_string_pretty(&serde_json::Value::Object(sha)).unwrap() + "\n",
)
.expect("sha256 sidecar");
std::fs::write(
dir.join("expected_hashes.json"),
serde_json::to_string_pretty(&serde_json::Value::Object(manifest)).unwrap() + "\n",
@ -222,6 +235,8 @@ fn generate(dir: &Path) {
fn verify(dir: &Path) {
let m: serde_json::Value =
serde_json::from_slice(&std::fs::read(dir.join("expected_hashes.json")).expect("manifest")).expect("parse");
let sha: serde_json::Value =
serde_json::from_slice(&std::fs::read(dir.join("sha256.json")).expect("sha256.json")).expect("parse sha256.json");
let v = dir.join("vectors");
let mut fails = 0;
for (name, spec) in m.as_object().expect("obj") {
@ -233,6 +248,7 @@ fn verify(dir: &Path) {
hasher.update(&img.rgb);
let got = hex(hasher.finalize().as_bytes());
let ok = got == spec["rgb_hash"].as_str().unwrap()
&& Some(wai::pin::sha256_hex(&img.rgb).as_str()) == sha[name]["sha256"].as_str()
&& img.width == spec["width"].as_u64().unwrap() as u32
&& img.height == spec["height"].as_u64().unwrap() as u32;
println!("{} {name} ({}×{}, {} layers)", if ok { "PASS" } else { "FAIL" },

304
wai/wai-rs/src/bin/wai_pin_vectors.rs Normal file
View file

@ -0,0 +1,304 @@
//! Conformance corpus for the parameter-set pin (`pin-conformance/`, SPEC.md §3.1,
//! §4, §7, §7.1).
//!
//! wai_pin_vectors verify <corpus-dir> every case gets its declared outcome, and
//! every listing is byte for byte the listing
//! of its set
//! wai_pin_vectors generate <corpus-dir> rewrite `listings/<name>.sha256` from
//! `sets/<name>/`
//!
//! - `cases.json`: for a pin, a capability, the forms the sink reads and the
//! artifacts it holds, the reason SPEC §3.1 names (or `ok`). Run through
//! [`wai::pin::verify_bytes`], the in-memory core every sink in this tree uses.
//! - `selection.json`: for a `WAI2` envelope, the rendition SPEC §7.1 picks, the
//! reason for each entry it skips, the companions applied with the pick, and the
//! first component-role finding.
//! - `static.json`: the first finding a verifier makes without holding a parameter
//! set ([`wai::container::Wai::check_pin`], [`wai::container::WaiMulti::check_pins`]).
//! - `listings/`: each `<name>.sha256` is the listing of `sets/<name>/`;
//! `listings/bad/*` each break a listing rule (`set_malformed`).
//!
//! `pin-conformance/pin_verify.py` checks `cases.json` and `listings/` with code that
//! shares nothing with this crate.
use std::collections::BTreeMap;
use std::path::Path;
use serde_json::Value;
use wai::container::{Manifest, ManifestV2, PinRefusal, Prior, Wai, WaiMulti};
use wai::pin::{listing_of_dir, parse_listing, verify_bytes, HeldBytes, READABLE_FORMS};
fn main() {
let a: Vec<String> = std::env::args().collect();
let code = match (a.get(1).map(String::as_str), a.get(2)) {
(Some("verify"), Some(d)) => verify(Path::new(d)),
(Some("generate"), Some(d)) => generate(Path::new(d)),
_ => {
eprintln!("usage: wai_pin_vectors verify|generate <corpus-dir>");
2
}
};
std::process::exit(code);
}
fn read_json(path: &Path) -> Result<Value, String> {
let s = std::fs::read(path).map_err(|e| format!("{}: {e}", path.display()))?;
serde_json::from_slice(&s).map_err(|e| format!("{}: {e}", path.display()))
}
/// The listings directly in `listings/` and the set directory each names.
fn listing_pairs(dir: &Path) -> Result<Vec<(String, std::path::PathBuf)>, String> {
let mut out = Vec::new();
let ld = dir.join("listings");
for e in std::fs::read_dir(&ld).map_err(|e| format!("{}: {e}", ld.display()))? {
let e = e.map_err(|e| e.to_string())?;
let name = e.file_name().to_string_lossy().into_owned();
if let Some(stem) = name.strip_suffix(".sha256") {
if e.file_type().map_err(|e| e.to_string())?.is_file() {
out.push((name.clone(), dir.join("sets").join(stem)));
}
}
}
out.sort();
Ok(out)
}
fn generate(dir: &Path) -> i32 {
let pairs = match listing_pairs(dir) {
Ok(p) => p,
Err(e) => {
eprintln!("{e}");
return 1;
}
};
for (name, set) in pairs {
match listing_of_dir(&set) {
Ok(l) => {
let p = dir.join("listings").join(&name);
if let Err(e) = std::fs::write(&p, &l) {
eprintln!("write {}: {e}", p.display());
return 1;
}
println!("wrote {} ({} B, sha256 {})", p.display(), l.len(), wai::pin::sha256_hex(&l));
}
Err(e) => {
eprintln!("{}: {e}", set.display());
return 1;
}
}
}
0
}
fn verify(dir: &Path) -> i32 {
let mut failures = Vec::new();
let mut checked = 0usize;
let mut fail = |what: String| failures.push(what);
// listings/: byte equality with the set, and the bad ones are malformed.
match listing_pairs(dir) {
Ok(pairs) => {
for (name, set) in pairs {
checked += 1;
let committed = std::fs::read(dir.join("listings").join(&name)).unwrap_or_default();
match listing_of_dir(&set) {
Ok(l) if l == committed => {}
Ok(_) => fail(format!("listings/{name}: differs from the listing of {}", set.display())),
Err(e) => fail(format!("listings/{name}: {e}")),
}
}
}
Err(e) => fail(e),
}
let bad = dir.join("listings/bad");
if let Ok(rd) = std::fs::read_dir(&bad) {
for e in rd.flatten() {
checked += 1;
let b = std::fs::read(e.path()).unwrap_or_default();
match parse_listing(&b) {
Err(PinRefusal::SetMalformed { .. }) => {}
other => fail(format!("listings/bad/{}: expected set_malformed, got {other:?}",
e.file_name().to_string_lossy())),
}
}
}
// cases.json
match read_json(&dir.join("cases.json")) {
Ok(doc) => {
for c in doc["cases"].as_array().into_iter().flatten() {
checked += 1;
let name = c["name"].as_str().unwrap_or("?");
let got = run_case(dir, c).unwrap_or_else(|e| format!("error: {e}"));
let want = c["expect"].as_str().unwrap_or("?");
if got != want {
fail(format!("cases.json {name}: expected {want}, got {got}"));
}
}
}
Err(e) => fail(e),
}
// selection.json
match read_json(&dir.join("selection.json")) {
Ok(doc) => {
for c in doc["cases"].as_array().into_iter().flatten() {
checked += 1;
let name = c["name"].as_str().unwrap_or("?");
match run_selection(dir, c) {
Ok(got) if got == c["expect"] => {}
Ok(got) => fail(format!("selection.json {name}: expected {}, got {got}", c["expect"])),
Err(e) => fail(format!("selection.json {name}: {e}")),
}
}
}
Err(e) => fail(e),
}
// static.json
match read_json(&dir.join("static.json")) {
Ok(doc) => {
for c in doc["cases"].as_array().into_iter().flatten() {
checked += 1;
let name = c["name"].as_str().unwrap_or("?");
match run_static(&c["manifest"]) {
Ok(got) if got == c["expect"] => {}
Ok(got) => fail(format!("static.json {name}: expected {}, got {got}", c["expect"])),
Err(e) => fail(format!("static.json {name}: {e}")),
}
}
}
Err(e) => fail(e),
}
if failures.is_empty() {
println!("pin-conformance: {checked} checks, all as declared");
0
} else {
for f in &failures {
eprintln!("FAIL {f}");
}
eprintln!("pin-conformance: {} of {checked} checks failed", failures.len());
1
}
}
/// The held artifacts of a case, read into memory: (bytes, files below the root).
type Loaded = Vec<(Vec<u8>, Option<BTreeMap<String, Vec<u8>>>)>;
fn load_held(dir: &Path, held: &Value) -> Result<Loaded, String> {
let mut out = Vec::new();
for h in held.as_array().into_iter().flatten() {
if let Some(f) = h["file"].as_str() {
out.push((std::fs::read(dir.join(f)).map_err(|e| format!("{f}: {e}"))?, None));
} else if let Some(l) = h["listing"].as_str() {
let bytes = std::fs::read(dir.join(l)).map_err(|e| format!("{l}: {e}"))?;
let root = dir.join(h["root"].as_str().ok_or("a held listing names its root")?);
// The files the listing names that exist below the root; a listing that
// does not parse names none.
let mut files = BTreeMap::new();
if let Ok(lines) = parse_listing(&bytes) {
for (p, _) in lines {
if let Ok(b) = std::fs::read(root.join(&p)) {
files.insert(p, b);
}
}
}
out.push((bytes, Some(files)));
} else {
return Err(format!("held entry {h} is neither a file nor a listing"));
}
}
Ok(out)
}
fn with_held<T>(loaded: &Loaded, f: impl FnOnce(&[HeldBytes<'_>]) -> T) -> T {
let views: Vec<Option<BTreeMap<String, &[u8]>>> = loaded.iter()
.map(|(_, files)| files.as_ref().map(|m| m.iter().map(|(k, v)| (k.clone(), v.as_slice())).collect()))
.collect();
let held: Vec<HeldBytes<'_>> = loaded.iter().zip(views.iter())
.map(|((bytes, _), v)| match v {
Some(files) => HeldBytes::Set { listing: bytes, files },
None => HeldBytes::File(bytes),
})
.collect();
f(&held)
}
fn run_case(dir: &Path, c: &Value) -> Result<String, String> {
let cap = c["capability"].as_str().ok_or("no capability")?;
let prior: Prior = serde_json::from_value(c["pin"].clone()).map_err(|e| format!("pin: {e}"))?;
let readable: Vec<String> = match c.get("readable") {
Some(r) => serde_json::from_value(r.clone()).map_err(|e| e.to_string())?,
None => READABLE_FORMS.iter().map(|s| s.to_string()).collect(),
};
let readable: Vec<&str> = readable.iter().map(String::as_str).collect();
let loaded = load_held(dir, &c["held"])?;
Ok(with_held(&loaded, |held| match verify_bytes(cap, &prior, held, &readable) {
Ok(_) => "ok".to_string(),
Err(r) => r.code().to_string(),
}))
}
fn run_selection(dir: &Path, c: &Value) -> Result<Value, String> {
let manifest: ManifestV2 = serde_json::from_value(c["manifest"].clone()).map_err(|e| format!("manifest: {e}"))?;
let payloads: Vec<Vec<u8>> = c["payloads"].as_array().ok_or("no payloads")?
.iter().map(|p| p.as_str().unwrap_or_default().as_bytes().to_vec()).collect();
let m = WaiMulti::from_parts(manifest, payloads).map_err(|e| e.to_string())?;
let advertised: Option<Vec<String>> = c["advertised"].as_array()
.map(|a| a.iter().filter_map(|s| s.as_str().map(String::from)).collect());
let held: Vec<Vec<u8>> = c["held"].as_array().into_iter().flatten()
.map(|f| std::fs::read(dir.join(f.as_str().unwrap_or_default())).map_err(|e| format!("{f}: {e}")))
.collect::<Result<_, _>>()?;
let verify = |cap: &str, p: &Prior| {
let hb: Vec<HeldBytes<'_>> = held.iter().map(|b| HeldBytes::File(b)).collect();
verify_bytes(cap, p, &hb, READABLE_FORMS).map(|_| ())
};
let skips = |s: &[(usize, wai::container::Skip)]| -> Value {
Value::Array(s.iter().map(|(i, k)| serde_json::json!([i, k.code()])).collect())
};
let mut out = serde_json::Map::new();
match m.pick(|cap| advertised.as_ref().is_none_or(|a| a.iter().any(|x| x == cap)), verify) {
Ok(p) => {
out.insert("picked".into(), p.index.into());
out.insert("skipped".into(), skips(&p.skipped));
if c["expect"].get("companions").is_some() {
let comps: Vec<usize> = m.companions_of_index(p.index).into_iter().map(|(i, _, _)| i).collect();
out.insert("companions".into(), serde_json::json!(comps));
}
}
Err(s) => {
out.insert("inert".into(), skips(&s));
}
}
if c["expect"].get("roles").is_some() {
let roles = match m.check_component_roles() {
Ok(()) => "ok".to_string(),
Err(e) => format!("{e:?}").split([' ', '{']).next().unwrap_or_default().to_string(),
};
out.insert("roles".into(), roles.into());
}
Ok(Value::Object(out))
}
fn run_static(manifest: &Value) -> Result<Value, String> {
match manifest["wai"].as_str() {
Some("1.0") => {
let m: Manifest = serde_json::from_value(manifest.clone()).map_err(|e| format!("manifest: {e}"))?;
Ok(match Wai::new(m, Vec::new()).check_pin() {
Ok(()) => "ok".into(),
Err(f) => f.code.into(),
})
}
Some("1.1") => {
let m: ManifestV2 = serde_json::from_value(manifest.clone()).map_err(|e| format!("manifest: {e}"))?;
let n = m.renditions.len();
let w = WaiMulti::from_parts(m, vec![Vec::new(); n]).map_err(|e| e.to_string())?;
Ok(match w.check_pins() {
Ok(()) => "ok".into(),
Err((i, f)) => serde_json::json!([i, f.code]),
})
}
other => Err(format!("unknown wai version {other:?}")),
}
}

15
wai/wai-rs/src/bin/wai_prov.rs
View file

@ -53,19 +53,8 @@ fn main() {
let jpeg = jpeg_encode(&rgb, H, W, 90).expect("jpeg");
// Wrap the JPEG as a content-addressed wai.image.jpeg envelope.
let manifest = Manifest {
wai: "1.0".into(),
media: "image".into(),
intent: "replicate".into(),
model_requirement: ModelRequirement {
capability: "wai.image.jpeg".into(),
fallback: None,
min_sim: None,
sim_metric: None,
},
conditioning: Conditioning { kind: "jpeg".into(), prior: None },
target: serde_json::json!({ "name": "captured", "w": W, "h": H }),
};
let manifest = Manifest::new("image", "replicate", ModelRequirement::new("wai.image.jpeg"), Conditioning::new("jpeg"))
.with_target(serde_json::json!({ "name": "captured", "w": W, "h": H }));
let envelope = Wai::new(manifest, jpeg.clone()).to_bytes().expect("wrap");
let asset_hash = asset_content_hash(&envelope);

15
wai/wai-rs/src/bin/wai_world_vectors.rs
View file

@ -536,19 +536,8 @@ fn gen_gallery(dir: &Path) {
// Wrap the cube glTF as a WAI envelope and content-address it (§4.1).
let glb = cube_glb();
let manifest = Manifest {
wai: "1.0".into(),
media: "mesh".into(),
intent: "replicate".into(),
model_requirement: ModelRequirement {
capability: "wai.mesh.glb".into(),
fallback: None,
min_sim: None,
sim_metric: None,
},
conditioning: Conditioning { kind: "glb".into(), prior: None },
target: serde_json::json!({ "name": "cube" }),
};
let manifest = Manifest::new("mesh", "replicate", ModelRequirement::new("wai.mesh.glb"), Conditioning::new("glb"))
.with_target(serde_json::json!({ "name": "cube" }));
let envelope = Wai::new(manifest, glb).to_bytes().expect("wrap glb");
let hash = asset_content_hash(&envelope);
std::fs::write(assets_dir.join("cube.glb.wai"), &envelope).expect("write asset");

1
wai/wai-rs/src/codecs/glossary_check.rs
View file

@ -114,6 +114,7 @@ const GLOSSARY_NO_ENTRY: &[(&str, &str)] = &[
(CAP_SEMANTIC_CLIP, "a generative-semantic profile, not a codec"),
(CAP_SEMANTIC_PERCO2, "PerCoV2 is a different model from the PerCo entry"),
(CAP_PRIOR_NNC, "neural-network coding codes model parameters, not media"),
(CAP_PRIOR_SET, WAI_NATIVE),
(CAP_TENSOR_NNC, "a candidate with no payload format pinned; neural-network coding codes tensors, not media"),
(CAP_SPLAT_SOG, "a candidate with no payload format pinned"),
];

239
wai/wai-rs/src/codecs/mod.rs
View file

@ -220,6 +220,14 @@ pub const CAP_TERNARY_TT: &str = "wai.ternary.tt";
// crate reads no NNC. The standard's reference software and decoder-conformance
// tests are specified by ISO/IEC 15938-18.
pub const CAP_PRIOR_NNC: &str = "wai.prior.nnc";
// wai.prior.set: the listing of a parameter set of one or more files (SPEC §3.1) —
// each file's SHA-256 and path below the set's root, one line per file, sorted by
// path. The registered prior form in which a set of several files is pinned: the
// pin covers the listing, and each line covers one file. WAI-native; reading a set
// through its listing compares digests and hands the decoder each file's bytes
// unchanged, so it is the one form the registry lists as established exact. Read
// and written by `crate::pin`.
pub const CAP_PRIOR_SET: &str = "wai.prior.set";
// wai.tensor.nnc: general tensorial data (split-inference feature maps, 3D/4D
// scene parameters) coded with NNC. A CANDIDATE (is_candidate; SPEC §5, Candidate
// capabilities): general-tensor coding is in an NNC amendment at committee-draft
@ -488,6 +496,8 @@ pub fn known_capabilities() -> &'static [&'static str] {
CAP_TERNARY_TT,
// neural-network coding: the prior artifact form + the general-tensor candidate
CAP_PRIOR_NNC, CAP_TENSOR_NNC,
// the listing of a parameter set: the form a pin over several files takes
CAP_PRIOR_SET,
// immersive audio (IAMF royalty-free; AC-4 / IVAS license-flagged)
CAP_AUDIO_IAMF, CAP_AUDIO_AC4, CAP_AUDIO_IVAS,
// cinema mastering (JPEG 2000 / DCP)
@ -675,7 +685,9 @@ pub fn capability_tier(cap: &str) -> Option<FidelityTier> {
// IAMF (carries lossless codecs — FLAC/LPCM — so lossless ceiling);
// HDR dynamic metadata reproduces exactly (it IS the descriptor bytes).
| CAP_IMAGE_JP2 | CAP_VIDEO_J2K | CAP_MESH_DRACO | CAP_AUDIO_IAMF
| CAP_META_HDR10PLUS | CAP_META_DOVI => Lossless,
| CAP_META_HDR10PLUS | CAP_META_DOVI
// A parameter-set listing is its own output, byte for byte.
| CAP_PRIOR_SET => Lossless,
// Interactive worlds: replay-equivalence is EXACT state
// reproduction (identical BLAKE3 state hash at every checkpoint,
// no tolerance parameter) — the simulation analog of lossless.
@ -827,7 +839,7 @@ pub fn capability_license(cap: &str) -> Option<LicenseClass> {
| CAP_WORLD_SCENE | CAP_WORLD_REPLAY | CAP_WORLD_LIVE | CAP_WORLD_MODEL
| CAP_AUDIO_SCENE | CAP_AUDIO_SCORE | CAP_HAPTIC_SIGNAL | CAP_FEED_STATE
| CAP_AVATAR_POSE | CAP_SPLAT_4D | CAP_FILM_LINEAR | CAP_FILM_REEL
| CAP_QUANTUM_CIRCUIT | CAP_TERNARY_TT
| CAP_QUANTUM_CIRCUIT | CAP_TERNARY_TT | CAP_PRIOR_SET
// Open asset formats: glTF (Khronos), SPZ (open source).
| CAP_MESH_GLB | CAP_SPLAT_SPZ
// JPEG 2000 Part 1 is fee-free under ITU-T policy 2.1 (extensions /
@ -1166,7 +1178,10 @@ pub fn capability_replicate_class(cap: &str) -> Option<ReplicateClass> {
| CAP_WORLD_SCENE | CAP_WORLD_REPLAY | CAP_WORLD_LIVE | CAP_WORLD_MODEL
| CAP_AUDIO_SCENE | CAP_AUDIO_SCORE | CAP_HAPTIC_SIGNAL | CAP_FEED_STATE
| CAP_AVATAR_POSE | CAP_SPLAT_4D | CAP_FILM_LINEAR | CAP_FILM_REEL
| CAP_QUANTUM_CIRCUIT | CAP_TERNARY_TT => StandardDefined,
| CAP_QUANTUM_CIRCUIT | CAP_TERNARY_TT
// A parameter-set listing: SPEC §3.1 fixes its bytes exactly, and they are
// its own output.
| CAP_PRIOR_SET => StandardDefined,
// A criterion around a reference, not identical output (clauses in the
// variant's doc and in research/media-codec-registry.md): Opus (RFC 6716
// §6), JPEG (T.81 §7, A.3.3), JPEG XL (18181-3 §4.2), JPEG 2000 (15444-4),
@ -1209,6 +1224,143 @@ pub fn is_neural_integer_exact(cap: &str) -> bool {
capability_replicate_class(cap) == Some(ReplicateClass::NeuralIntegerExact)
}
// ---- Parameter sets, the tier vocabulary and dispatchability (SPEC §3.1, §4, §7.1) ----
/// Whether a registered capability **decodes against a parameter set** (SPEC §3.1):
/// its registry class is `NeuralIntegerExact`, `NeuralEntropyExact`, `NeuralFloat`
/// or `GenerativeDeterministic`. Its parameter set is every byte other than the
/// payload that its decode reads and that its output depends on; the sink supplies
/// it, and a manifest MAY pin it. `false` for an unregistered capability: this
/// revision does not know what it decodes against (a sink that advertises one and
/// finds a pin on it verifies the pin as several files, SPEC §3.1).
pub fn decodes_against_parameter_set(cap: &str) -> bool {
use ReplicateClass::*;
matches!(
capability_replicate_class(cap),
Some(NeuralIntegerExact | NeuralEntropyExact | NeuralFloat | GenerativeDeterministic)
)
}
/// The registered shape of a capability's parameter set (SPEC §5): one file, or
/// several. A form-less pin is admissible only over a one-file set (SPEC §3.1,
/// `form_required`); a set of several files is pinned through its listing
/// ([`CAP_PRIOR_SET`]).
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub enum ParameterSetShape {
/// One file holds the whole set.
OneFile,
/// Several files make up the set, or the registration names no one-file format.
SeveralFiles,
}
/// The parameter-set shape SPEC §5 registers for `cap`: `Some` exactly when
/// [`decodes_against_parameter_set`]. A capability is one file when the reference
/// decoder reads one file and that file holds every parameter its output depends
/// on; every other capability that decodes against a parameter set is several
/// files. A later revision MAY register a several-file capability as one file when
/// it defines a one-file format that holds the whole set (a widening, SPEC §8).
pub fn parameter_set_shape(cap: &str) -> Option<ParameterSetShape> {
if !decodes_against_parameter_set(cap) {
return None;
}
Some(match cap {
// int_synth: its model specification, int_synth.json. int_mlicpp: its WMB1
// bundle. The audio codecs and the per-frame video codec: one self-contained
// ONNX decoder (a model that keeps tensors in external files is several files).
CAP_NEURAL_INT_SYNTH | CAP_NEURAL_INT_MLICPP
| CAP_NEURAL_ENCODEC32 | CAP_NEURAL_DAC | CAP_NEURAL_MIMI | CAP_NEURAL_WAVTOKENIZER
| CAP_NEURAL_SNAC | CAP_NEURAL_VIDEO_BMSHJ2018 => ParameterSetShape::OneFile,
_ => ParameterSetShape::SeveralFiles,
})
}
/// The tier a pin's `tier` member declares (SPEC §3.1): the closed, registered
/// vocabulary of `extensions/determinism-tiers.md` §2 plus `none`. Ordered weakest
/// first. An encoder writes only these tokens ([`DeclaredTier::token`]); a sink
/// reads any other token as [`DeclaredTier::None`], and a verifier reports it
/// (`tier_unregistered`).
#[derive(Debug, Clone, Copy, PartialEq, Eq, PartialOrd, Ord, Hash)]
pub enum DeclaredTier {
/// Neither tier is claimed.
None,
/// `entropy-consistency` ([`DeterminismTier::EntropyConsistency`]).
EntropyConsistency,
/// `decode-equivalence` ([`DeterminismTier::DecodeEquivalence`]).
DecodeEquivalence,
}
/// The registered tier tokens, in the one table every reader and writer of the
/// vocabulary uses (SPEC §3.1; the vocabulary's shape; the registry export).
/// Tier tokens and class labels are kebab-case; refusal reasons are snake_case.
pub const TIER_TOKENS: &[(&str, DeclaredTier)] = &[
("decode-equivalence", DeclaredTier::DecodeEquivalence),
("entropy-consistency", DeclaredTier::EntropyConsistency),
("none", DeclaredTier::None),
];
impl DeclaredTier {
/// The registered token this tier is written as.
pub fn token(self) -> &'static str {
TIER_TOKENS.iter().find(|&&(_, t)| t == self).map(|&(tok, _)| tok).unwrap_or("none")
}
/// The tier a registered token names, or `None` for a token this revision
/// does not register (which a sink reads as [`DeclaredTier::None`]).
pub fn from_token(token: &str) -> Option<Self> {
TIER_TOKENS.iter().find(|&&(tok, _)| tok == token).map(|&(_, t)| t)
}
/// The determinism tier this declares, `None` for [`DeclaredTier::None`].
pub fn determinism_tier(self) -> Option<DeterminismTier> {
match self {
Self::None => None,
Self::EntropyConsistency => Some(DeterminismTier::EntropyConsistency),
Self::DecodeEquivalence => Some(DeterminismTier::DecodeEquivalence),
}
}
}
impl From<Option<DeterminismTier>> for DeclaredTier {
fn from(t: Option<DeterminismTier>) -> Self {
match t {
None => Self::None,
Some(DeterminismTier::EntropyConsistency) => Self::EntropyConsistency,
Some(DeterminismTier::DecodeEquivalence) => Self::DecodeEquivalence,
}
}
}
impl DeterminismTier {
/// The registered token (`decode-equivalence`, `entropy-consistency`).
pub fn token(self) -> &'static str {
DeclaredTier::from(Some(self)).token()
}
/// The tier a registered token names; `None` for `none` or an unregistered token.
pub fn from_token(token: &str) -> Option<Self> {
DeclaredTier::from_token(token).and_then(DeclaredTier::determinism_tier)
}
}
/// The tier `cap`'s registry class gives (SPEC §3.1, *Tier*): `decode-equivalence`
/// for `NeuralIntegerExact`, `entropy-consistency` for `NeuralEntropyExact`, `none`
/// for every other class. `None` for an unregistered capability.
pub fn class_tier(cap: &str) -> Option<DeclaredTier> {
capability_replicate_class(cap).map(|c| DeclaredTier::from(c.neural_tier()))
}
/// Whether a pin's stated `tier` claims more than `cap`'s registry class gives
/// (SPEC §3.1, `tier_exceeds_class`). Only a registered `tier` member is read: a
/// pin without one, or with an unregistered token (read as `none`), claims nothing
/// above any class. `false` for an unregistered capability, whose class is unknown.
pub fn tier_exceeds_class(cap: &str, prior: &crate::container::Prior) -> bool {
let Some(class) = class_tier(cap) else { return false };
match prior.tier().and_then(DeclaredTier::from_token) {
Some(claim) => claim > class,
None => false,
}
}
/// Why SPEC §7 refuses an `intent = replicate` envelope. See [`replicate_refusal`].
#[derive(Debug, Clone, PartialEq, Eq)]
pub enum ReplicateRefusal {
@ -1249,6 +1401,18 @@ impl std::fmt::Display for ReplicateRefusal {
}
}
impl ReplicateRefusal {
/// The reason SPEC §7 names: `replicate_unregistered`, `replicate_class`, or the
/// pin's reason ([`crate::container::PriorRefusal::code`]).
pub fn code(&self) -> &'static str {
match self {
Self::Unregistered => "replicate_unregistered",
Self::Class(_) => "replicate_class",
Self::Prior(r) => r.code(),
}
}
}
/// SPEC §7 *replicate-soundness* for one envelope: `None` when `(cap, intent, prior)`
/// may be honoured as `replicate`, otherwise why not.
///
@ -1794,6 +1958,58 @@ mod tests {
/// conformance fixes, and `replicate` over it promises that and no more. A codec
/// whose standard conforms decoders to anything but identical output must not
/// sit in the class that promises the same bytes on every sink.
#[test]
fn tier_claims_and_parameter_set_shapes() {
use crate::container::Prior;
let pin = |tier: &str| {
let mut p = Prior::new("m", "ab".repeat(32), "int");
p.tier = Some(tier.into());
p
};
assert!(tier_exceeds_class(CAP_IMAGE_JPEGAI, &pin("decode-equivalence")));
assert!(tier_exceeds_class(CAP_NEURAL_BMSHJ2018, &pin("entropy-consistency")));
assert!(!tier_exceeds_class(CAP_IMAGE_JPEGAI, &pin("entropy-consistency")));
assert!(!tier_exceeds_class(CAP_NEURAL_INT_SYNTH, &pin("none")));
assert!(!tier_exceeds_class(CAP_IMAGE_JPEGAI, &pin("decode_equivalence")), "unregistered reads as none");
assert!(!tier_exceeds_class(CAP_IMAGE_JPEGAI, &Prior::new("m", "ab", "int")), "no tier, no claim");
assert!(!tier_exceeds_class("wai.neural.future", &pin("decode-equivalence")));
// The closed vocabulary: one table, round-tripping, ordered.
for &(tok, t) in TIER_TOKENS {
assert_eq!(DeclaredTier::from_token(tok), Some(t));
assert_eq!(t.token(), tok);
assert!(tok.chars().all(|c| c.is_ascii_lowercase() || c == '-'), "{tok} is kebab-case");
}
assert!(DeclaredTier::None < DeclaredTier::EntropyConsistency
&& DeclaredTier::EntropyConsistency < DeclaredTier::DecodeEquivalence);
assert_eq!(DeterminismTier::DecodeEquivalence.token(), "decode-equivalence");
assert_eq!(DeterminismTier::from_token("none"), None);
// Shapes: Some exactly for the capabilities that decode against a parameter
// set, OneFile exactly for the capabilities SPEC §5 registers as one file.
let mut one_file = Vec::new();
for &c in known_capabilities() {
assert_eq!(parameter_set_shape(c).is_some(), decodes_against_parameter_set(c), "{c}");
if parameter_set_shape(c) == Some(ParameterSetShape::OneFile) {
one_file.push(c);
}
}
one_file.sort_unstable();
let mut want = vec![CAP_NEURAL_INT_SYNTH, CAP_NEURAL_INT_MLICPP, CAP_NEURAL_ENCODEC32, CAP_NEURAL_DAC,
CAP_NEURAL_MIMI, CAP_NEURAL_WAVTOKENIZER, CAP_NEURAL_SNAC, CAP_NEURAL_VIDEO_BMSHJ2018];
want.sort_unstable();
assert_eq!(one_file, want);
for c in [CAP_IMAGE_PNG, CAP_SEMANTIC_CLIP, CAP_PRIOR_SET, CAP_WORLD_MODEL] {
assert!(!decodes_against_parameter_set(c), "{c}");
}
for c in [CAP_IMAGE_JPEGAI, CAP_NEURAL_GLC, CAP_GENERATE_SPECTRAL, CAP_AUDIO_INT_CODEC] {
assert_eq!(parameter_set_shape(c), Some(ParameterSetShape::SeveralFiles), "{c}");
}
// The listing form: lossless, royalty-free, StandardDefined.
assert_eq!(capability_tier(CAP_PRIOR_SET), Some(FidelityTier::Lossless));
assert!(is_royalty_free(CAP_PRIOR_SET));
assert_eq!(capability_replicate_class(CAP_PRIOR_SET), Some(ReplicateClass::StandardDefined));
assert!(is_dispatchable(CAP_PRIOR_SET) && !is_dispatchable(CAP_VIDEO_LCEVC));
}
#[test]
fn standard_classes_follow_the_named_standards_decoder_conformance() {
use ReplicateClass::*;
@ -1848,7 +2064,7 @@ mod tests {
#[test]
fn replicate_follows_the_registry_class_not_the_declared_prior() {
use crate::container::{DeterminismClass, Prior};
let p = |det: &str| Prior { id: "m".into(), sha256: "ab".repeat(32), determinism: det.into(), form: None };
let p = |det: &str| Prior::new("m", "ab".repeat(32), det);
let integer = p("int-rans/p16+int-synth/i64");
let floaty = p("int-rans/p16+onnx-cpu-f32/1thread");
@ -1930,6 +2146,7 @@ mod tests {
(CAP_FILM_REEL, "extensions/linear-film.md: reel-equivalence"),
(CAP_QUANTUM_CIRCUIT, "extensions/quantum-sim.md: statevector-equivalence"),
(CAP_TERNARY_TT, "extensions/ternary-tt.md: reconstruction-equivalence"),
(CAP_PRIOR_SET, "SPEC.md §3.1: a listing's bytes are fixed exactly and are their own output"),
];
/// Every capability classed NeuralIntegerExact, with the basis its class rests
@ -2080,19 +2297,7 @@ mod tests {
fn an_unpinned_envelope_claims_replicate_only_where_a_sink_accepts_it() {
use crate::container::{Conditioning, Manifest, ModelRequirement, Wai};
let envelope = |cap: &str, intent: &str| Wai::new(
Manifest {
wai: "1.0".into(),
media: "binary".into(),
intent: intent.into(),
model_requirement: ModelRequirement {
capability: cap.into(),
fallback: None,
min_sim: None,
sim_metric: None,
},
conditioning: Conditioning { kind: "bytes".into(), prior: None },
target: serde_json::Value::Null,
},
Manifest::new("binary", intent, ModelRequirement::new(cap), Conditioning::new("bytes")),
vec![],
);

57
wai/wai-rs/src/codecs/prior_form.rs
View file

@ -8,11 +8,11 @@
//! name, and a sink decides whether it can read a form by the same dispatch it
//! uses for payloads.
use super::CAP_PRIOR_NNC;
use super::{CAP_PRIOR_NNC, CAP_PRIOR_SET};
/// Every capability string a `Prior` may declare as its artifact's form.
pub fn prior_forms() -> &'static [&'static str] {
&[CAP_PRIOR_NNC]
&[CAP_PRIOR_NNC, CAP_PRIOR_SET]
}
/// Whether `cap` is a registered prior form.
@ -23,19 +23,23 @@ pub fn is_prior_form(cap: &str) -> bool {
/// Prior forms whose decode to the tensors a network consumes is **established**
/// to give identical tensors at every conforming reader.
///
/// Empty. ISO/IEC 15938-17 already has payload types that carry values without
/// quantisation: its 2022 edition lists decoding methods for an integer payload
/// type (`NNR_PT_INT`) and a raw-float payload type (`NNR_PT_RAW_FLOAT`), and a
/// decoding process for an integer weight tensor. What is not established here is
/// whether every conforming decoder reconstructs bit-identical parameter values,
/// from those payload types or from quantisation indices. Decoder conformance is
/// specified in its clause 7 and tested with the bitstreams and procedure of
/// ISO/IEC 15938-18, and neither text was read. That is what would settle it, for
/// the published editions and for any later one; a lossless mode added by an
/// amendment would not settle it by itself. Until a form is listed, a `replicate`
/// claim cannot rest on a prior carried in it (see
/// One: `wai.prior.set`. Reading a set through its listing compares digests and
/// hands the decoder each file's bytes unchanged, so every sink that holds the set
/// reads the same parameters.
///
/// No coded form is listed. ISO/IEC 15938-17 already has payload types that carry
/// values without quantisation: its 2022 edition lists decoding methods for an
/// integer payload type (`NNR_PT_INT`) and a raw-float payload type
/// (`NNR_PT_RAW_FLOAT`), and a decoding process for an integer weight tensor. What
/// is not established here is whether every conforming decoder reconstructs
/// bit-identical parameter values, from those payload types or from quantisation
/// indices. Decoder conformance is specified in its clause 7 and tested with the
/// bitstreams and procedure of ISO/IEC 15938-18, and neither text was read. That is
/// what would settle it, for the published editions and for any later one; a
/// lossless mode added by an amendment would not settle it by itself. Until a coded
/// form is listed, a `replicate` claim cannot rest on a pin in it (see
/// [`Prior::form_preserves_exactness`](crate::container::Prior::form_preserves_exactness)).
const EXACT_RECONSTRUCTION_FORMS: &[&str] = &[];
const EXACT_RECONSTRUCTION_FORMS: &[&str] = &[CAP_PRIOR_SET];
/// Whether reading an artifact out of `form` is established to reproduce the
/// same tensors at every conforming reader. `false` for an unregistered form.
@ -58,7 +62,7 @@ mod tests {
#[test]
fn prior_forms_are_registered_capabilities() {
assert_eq!(prior_forms(), [CAP_PRIOR_NNC]);
assert_eq!(prior_forms(), [CAP_PRIOR_NNC, CAP_PRIOR_SET]);
for &f in prior_forms() {
assert!(is_known_capability(f), "{f} must be a registered capability");
}
@ -70,7 +74,8 @@ mod tests {
}
#[test]
fn no_form_is_yet_established_exact() {
fn only_the_listing_form_is_established_exact() {
assert!(form_reconstruction_is_exact(CAP_PRIOR_SET));
assert!(!form_reconstruction_is_exact(CAP_PRIOR_NNC));
assert!(!form_reconstruction_is_exact("wai.prior.nope"));
}
@ -112,19 +117,8 @@ mod tests {
let nnc = integer.clone().with_form(CAP_PRIOR_NNC);
let envelope = |intent: &str, prior: &Prior| {
Wai::new(
Manifest {
wai: "1.0".into(),
media: "image".into(),
intent: intent.into(),
model_requirement: ModelRequirement {
capability: CAP_NEURAL_INT_HYPER.into(),
fallback: None,
min_sim: None,
sim_metric: None,
},
conditioning: Conditioning { kind: "int_hyper".into(), prior: Some(prior.clone()) },
target: serde_json::Value::Null,
},
Manifest::new("image", intent, ModelRequirement::new(CAP_NEURAL_INT_HYPER),
Conditioning::new("int_hyper").with_prior(prior.clone())),
vec![],
)
};
@ -143,6 +137,11 @@ mod tests {
Some(PriorRefusal::FormNotExact(CAP_PRIOR_NNC.into())));
// Only `replicate` is gated.
assert!(envelope("improve", &nnc).replicate_is_sound());
// A set-form pin declaring decode-equivalence is sound: the listing form is
// established exact.
let set = integer.clone().with_form(CAP_PRIOR_SET);
assert!(set.is_cross_hardware_exact());
assert!(envelope("replicate", &set).replicate_is_sound());
// The yes/no answer and the reason never disagree.
let floaty = Prior::new("m", "ab".repeat(32), "int-rans/p16+onnx-cpu-f32/1thread");

78
wai/wai-rs/src/codecs/registry.rs
View file

@ -252,6 +252,7 @@ const ROWS: &[(&str, &str, Option<&str>, &str)] = &[
(CAP_VIDEO_PACKAGE, S_NEURAL, None, "2026-09-03"),
(CAP_VIDEO_CHANNEL, S_NEURAL, None, "2026-09-03"),
(CAP_PRIOR_NNC, S_PRIOR, Some("ISO/IEC 15938-17:2024"), "2026-09-28"),
(CAP_PRIOR_SET, S_PRIOR, None, "2026-09-30"),
(CAP_TENSOR_NNC, S_PRIOR, None, "2026-09-28"),
(CAP_WORLD_SCENE, S_INTERACTIVE, None, "2026-06-13"),
(CAP_WORLD_REPLAY, S_INTERACTIVE, None, "2026-06-13"),
@ -469,9 +470,11 @@ pub fn is_candidate(cap: &str) -> bool {
/// can decode it is the sink's to say through what it advertises (SPEC §4, step 3).
///
/// Every dispatch and selection function of the reference implementation uses this
/// one predicate, together with what the sink advertises
/// (`Wai::dispatch_capability`, `WaiMulti::pick`, `WaiMulti::primary`,
/// `RenditionMeta::is_selectable`, the CLI's `inspect`); a new one must use it too.
/// one predicate, together with what the sink advertises and, where a pin applies,
/// the sink's verification of it (`Wai::dispatch_verified`, `WaiMulti::pick`,
/// `WaiMulti::primary`, `RenditionMeta::is_selectable`, the neural sink's
/// `decode_multi`, the C `wai_envelope_select`, wai-web's `core` module, the CLI's
/// `inspect`); a new one must use it too.
pub fn is_dispatchable(cap: &str) -> bool {
match capability_kind(cap) {
None => true,
@ -502,7 +505,7 @@ pub fn capability_media(cap: &str) -> Option<Media> {
| CAP_FILM_LINEAR | CAP_FILM_REEL => Video,
CAP_WORLD_SCENE | CAP_WORLD_REPLAY | CAP_WORLD_LIVE | CAP_WORLD_MODEL
| CAP_FEED_STATE | CAP_AVATAR_POSE => Interactive,
CAP_PRIOR_NNC | CAP_TENSOR_NNC | CAP_QUANTUM_CIRCUIT | CAP_TERNARY_TT => Binary,
CAP_PRIOR_NNC | CAP_PRIOR_SET | CAP_TENSOR_NNC | CAP_QUANTUM_CIRCUIT | CAP_TERNARY_TT => Binary,
c if c.starts_with("wai.image.") => Image,
c if c.starts_with("wai.audio.") => Audio,
c if c.starts_with("wai.video.") => Video,
@ -718,6 +721,50 @@ fn enumerations() -> [(&'static str, &'static str, Vec<&'static str>); 7] {
]
}
/// The registered tier tokens of a pin (SPEC §3.1), from [`TIER_TOKENS`].
fn tier_tokens() -> Vec<&'static str> {
TIER_TOKENS.iter().map(|&(t, _)| t).collect()
}
/// The parameter-set pin's terms (SPEC §3.1), as v1 states them. `pinSha256` and
/// `pinForm` name no domain: they are also the members of a pin's `supersedes`
/// object, which is not itself a pin.
const PIN_TERMS_TTL: &str = "# ─── The parameter-set pin (SPEC section 3.1) ────────────────────────\n\n\
wai:prior a owl:ObjectProperty ; rdfs:range wai:ParameterSetPin ; rdfs:label \"Parameter-set pin (conditioning.prior, or a component's prior)\" .\n\
wai:pinId a owl:DatatypeProperty ; rdfs:domain wai:ParameterSetPin ; rdfs:range xsd:string ; rdfs:label \"Pin name (informative)\" .\n\
wai:pinSha256 a owl:DatatypeProperty ; rdfs:range xsd:string ; rdfs:label \"SHA-256 of the parameter set in its form (64 lowercase hex)\" ;\n rdfs:comment \"A member of a pin and of its supersedes object.\" .\n\
wai:pinDeterminism a owl:DatatypeProperty ; rdfs:domain wai:ParameterSetPin ; rdfs:range xsd:string ; rdfs:label \"How the decode is made reproducible\" .\n\
wai:pinForm a owl:DatatypeProperty ; rdfs:range xsd:string ; rdfs:label \"Registered prior form\" ;\n rdfs:comment \"A member of a pin and of its supersedes object.\" .\n\
wai:pinSupersedes a owl:ObjectProperty ; rdfs:domain wai:ParameterSetPin ; rdfs:label \"The parameter set this one replaces (informative)\" ;\n rdfs:comment \"Names that set by pinSha256 and, if it is pinned in a declared form, pinForm; it is not itself a pin.\" .\n";
/// The SHACL shapes of the parameter-set pin (SPEC §3.1), the same in v1 and v2:
/// the JSON-LD context types no `prior` object, so the pin shape targets whatever
/// `wai:prior` points at; `pinTier` is the closed list [`TIER_TOKENS`] holds; and
/// `supersedes` names a set by its digest and, optionally, its form.
fn pin_shapes() -> String {
let tiers: Vec<String> = tier_tokens().iter().map(|t| format!("\"{t}\"")).collect();
let mut o = String::new();
o.push_str("# The parameter-set pin (SPEC section 3.1). The JSON-LD context types no prior object,\n");
o.push_str("# so the shape targets whatever wai:prior points at rather than a class. pinTier is a\n");
o.push_str("# closed vocabulary; the tokens are the ones codecs::TIER_TOKENS holds.\n");
o.push_str("wai:ParameterSetPinShape a sh:NodeShape ;\n");
o.push_str(" sh:targetObjectsOf wai:prior ;\n");
o.push_str(" sh:property [ sh:path wai:pinId ; sh:datatype xsd:string ; sh:minCount 1 ; sh:maxCount 1 ] ;\n");
o.push_str(" sh:property [ sh:path wai:pinSha256 ; sh:datatype xsd:string ; sh:minCount 1 ; sh:maxCount 1 ;\n");
o.push_str(" sh:pattern \"^[a-f0-9]{64}$\" ] ;\n");
o.push_str(" sh:property [ sh:path wai:pinDeterminism ; sh:datatype xsd:string ; sh:minCount 1 ; sh:maxCount 1 ] ;\n");
o.push_str(" sh:property [ sh:path wai:pinForm ; sh:datatype xsd:string ; sh:maxCount 1 ] ;\n");
o.push_str(&format!(" sh:property [ sh:path wai:pinTier ; sh:datatype xsd:string ; sh:maxCount 1 ;\n sh:in ( {} ) ] ;\n", tiers.join(" ")));
o.push_str(" sh:property [ sh:path wai:pinSupersedes ; sh:node wai:SupersedesShape ; sh:maxCount 1 ] .\n\n");
o.push_str("# supersedes names a set as a pin names one, by sha256 and, if it is pinned in a\n");
o.push_str("# declared form, form; it has no id, determinism or tier.\n");
o.push_str("wai:SupersedesShape a sh:NodeShape ;\n");
o.push_str(" sh:property [ sh:path wai:pinSha256 ; sh:datatype xsd:string ; sh:minCount 1 ; sh:maxCount 1 ;\n");
o.push_str(" sh:pattern \"^[a-f0-9]{64}$\" ] ;\n");
o.push_str(" sh:property [ sh:path wai:pinForm ; sh:datatype xsd:string ; sh:maxCount 1 ] .\n");
o
}
/// `wai/vocab/v2.ttl`: one IRI per registered string, with its registry properties
/// as enumeration individuals, and the manifest terms the JSON-LD context maps.
pub fn vocab_v2_ttl() -> String {
@ -743,6 +790,7 @@ pub fn vocab_v2_ttl() -> String {
("Tier", "Fidelity tier"),
("Licence", "Licence class"),
("ReplicateClass", "Replicate class"),
("ParameterSetPin", "Parameter-set pin (SPEC section 3.1)"),
] {
o.push_str(&format!("wai:{class} a owl:Class ; rdfs:label {} .\n", ttl_str(label)));
}
@ -784,6 +832,10 @@ pub fn vocab_v2_ttl() -> String {
o.push_str(&format!("wai:{p} a owl:DatatypeProperty ;{domain} rdfs:range {range} ; rdfs:label {} .\n", ttl_str(label)));
}
o.push('\n');
o.push_str(PIN_TERMS_TTL);
o.push_str(&format!("wai:pinTier a owl:DatatypeProperty ; rdfs:domain wai:ParameterSetPin ; rdfs:range xsd:string ; rdfs:label \"Declared reproducibility tier\" ;\n rdfs:comment {} .\n",
ttl_str(&format!("A closed vocabulary: {}.", tier_tokens().join(", ")))));
o.push('\n');
o.push_str("# ─── Registered strings ──────────────────────────────────────────────\n");
for r in registrations() {
o.push('\n');
@ -863,7 +915,8 @@ pub fn vocab_v2_shapes() -> String {
o.push_str(" sh:targetClass wai:Component ;\n");
o.push_str(" sh:targetObjectsOf wai:renditions ;\n");
o.push_str(&format!(" sh:property [ sh:path wai:capability ; sh:nodeKind sh:IRI ; sh:minCount 1 ; sh:maxCount 1 ;\n sh:in {} ] ;\n", list(&caps)));
o.push_str(&format!(" sh:property [ sh:path wai:companionOf ; sh:nodeKind sh:IRI ; sh:maxCount 1 ;\n sh:in {} ] .\n", list(&payloads)));
o.push_str(&format!(" sh:property [ sh:path wai:companionOf ; sh:nodeKind sh:IRI ; sh:maxCount 1 ;\n sh:in {} ] .\n\n", list(&payloads)));
o.push_str(&pin_shapes());
o
}
@ -894,7 +947,20 @@ pub fn vocab_v2_jsonld() -> String {
"kind": { "@id": t("payloadKind"), "@type": "xsd:string" },
"role": { "@id": t("componentRole"), "@type": "xsd:string" },
"renditions": { "@id": t("renditions") },
"target": { "@id": t("target"), "@type": "@json" }
"target": { "@id": t("target"), "@type": "@json" },
"ParameterSetPin": t("ParameterSetPin"),
// The pin (SPEC §3.1), as `conditioning.prior` or a component's `prior`:
// its members under the manifest's own names. The scoped context reaches
// the `supersedes` object too. `tier` is an untyped string so that a
// SHACL validator compares it with the shape's tier list.
"prior": { "@id": t("prior"), "@context": {
"id": { "@id": t("pinId"), "@type": "xsd:string" },
"sha256": { "@id": t("pinSha256"), "@type": "xsd:string" },
"determinism": { "@id": t("pinDeterminism"), "@type": "xsd:string" },
"form": { "@id": t("pinForm"), "@type": "xsd:string" },
"tier": { "@id": t("pinTier") },
"supersedes": { "@id": t("pinSupersedes") }
} }
}
}))
}

2349
wai/wai-rs/src/container.rs
View file

File diff suppressed because it is too large Load diff

518
wai/wai-rs/src/ffi.rs
View file

@ -3,6 +3,12 @@
//! talks to this. The Rust crate is built as `cdylib`/`staticlib` so it
//! drops in as `libwai.so`/`libwai.dylib`/`wai.dll`.
//!
//! This module (feature `ffi`, which `codecs` implies) is the envelope surface:
//! buffers, errors, packing and unpacking `WAI1`/`WAI2`, and SPEC §7.1 selection.
//! It needs no codec library. The codec wrappers are [`crate::ffi_codecs`]
//! (feature `codecs`, re-exported here) and the neural sink is
//! [`crate::ffi_neural`] (features `neural` and `ffi`).
//!
//! ## Status codes (returned by every wai_* function)
//! - `0` = ok
//! - `1` = invalid argument (null pointer where one shouldn't be, etc.)
@ -11,6 +17,12 @@
//! - `3` = container/envelope error (bad magic, truncated, bad JSON)
//! - `4` = internal panic (a bug in the wrapper — we catch it instead
//! of unwinding into C, which would be UB)
//! - `5` = a pin was refused (SPEC §3.1); `wai_last_error` starts with the reason
//! (`prior_not_held`, `form_required`, …)
//! - `6` = `intent = replicate` is not sound for the capability decoded through
//! (SPEC §7); `wai_last_error` starts with the reason
//! - `7` = no rendition of a `WAI2` envelope can be picked (SPEC §7.1);
//! `wai_last_error` gives each entry's index and reason
//!
//! ## Memory model
//! All byte buffers crossing the boundary are heap-allocated here and
@ -23,19 +35,27 @@ use std::ffi::{c_char, c_int, CString};
use std::panic;
use std::slice;
use crate::codecs;
pub const WAI_OK: c_int = 0;
pub const WAI_ERR_INVALID_ARG: c_int = 1;
pub const WAI_ERR_CODEC: c_int = 2;
pub const WAI_ERR_CONTAINER: c_int = 3;
pub const WAI_ERR_PANIC: c_int = 4;
/// A pin was refused (SPEC §3.1); `wai_last_error` starts with the reason.
pub const WAI_ERR_PRIOR: c_int = 5;
/// `intent = replicate` is not sound for the capability decoded through (SPEC §7).
pub const WAI_ERR_UNSOUND_REPLICATE: c_int = 6;
/// No rendition of a `WAI2` envelope can be picked (SPEC §7.1).
pub const WAI_ERR_NO_RENDITION: c_int = 7;
#[cfg(feature = "codecs")]
pub use crate::ffi_codecs::*;
thread_local! {
static LAST_ERROR: RefCell<Option<CString>> = const { RefCell::new(None) };
}
fn set_err(e: impl ToString) {
pub(crate) fn set_err(e: impl ToString) {
let cs = CString::new(e.to_string())
.unwrap_or_else(|_| CString::new("invalid utf-8 error").unwrap());
LAST_ERROR.with(|c| *c.borrow_mut() = Some(cs));
@ -56,7 +76,7 @@ impl WaiBuffer {
std::mem::forget(v);
WaiBuffer { data, len, cap }
}
fn empty() -> Self { WaiBuffer { data: std::ptr::null_mut(), len: 0, cap: 0 } }
pub(crate) fn empty() -> Self { WaiBuffer { data: std::ptr::null_mut(), len: 0, cap: 0 } }
}
/// Free a buffer previously returned by any `wai_*_encode` /
@ -84,7 +104,7 @@ pub extern "C" fn wai_last_error() -> *const c_char {
}
// ---- generic helpers ------------------------------------------------
unsafe fn input_slice<'a>(data: *const u8, len: usize) -> Option<&'a [u8]> {
pub(crate) unsafe fn input_slice<'a>(data: *const u8, len: usize) -> Option<&'a [u8]> {
match (data.is_null(), len) {
(true, 0) => Some(&[]),
(true, _) => None, // null but non-zero len
@ -92,13 +112,13 @@ unsafe fn input_slice<'a>(data: *const u8, len: usize) -> Option<&'a [u8]> {
}
}
unsafe fn write_buf(out: *mut WaiBuffer, v: Vec<u8>) -> bool {
pub(crate) unsafe fn write_buf(out: *mut WaiBuffer, v: Vec<u8>) -> bool {
if out.is_null() { return false; }
unsafe { *out = WaiBuffer::from_vec(v); }
true
}
unsafe fn clear_buf(out: *mut WaiBuffer) {
pub(crate) unsafe fn clear_buf(out: *mut WaiBuffer) {
if !out.is_null() {
unsafe { *out = WaiBuffer::empty(); }
}
@ -108,7 +128,7 @@ unsafe fn clear_buf(out: *mut WaiBuffer) {
/// runs the closure inside `catch_unwind` (so a Rust panic returns
/// `WAI_ERR_PANIC` instead of unwinding into C — which would be UB),
/// and clears `out` to an empty buffer on any error path.
fn ffi_buf_call<F>(out: *mut WaiBuffer, f: F) -> c_int
pub(crate) fn ffi_buf_call<F>(out: *mut WaiBuffer, f: F) -> c_int
where F: FnOnce() -> std::result::Result<Vec<u8>, (c_int, String)> + panic::UnwindSafe
{
if out.is_null() {
@ -136,384 +156,6 @@ where F: FnOnce() -> std::result::Result<Vec<u8>, (c_int, String)> + panic::Unwi
}
}
fn ffi_codec<T, F>(f: F) -> std::result::Result<Vec<u8>, (c_int, String)>
where F: FnOnce() -> std::result::Result<T, String>,
T: Into<Vec<u8>>,
{
f().map(Into::into).map_err(|e| (WAI_ERR_CODEC, e))
}
// ---- image: PNG -----------------------------------------------------
/// Encode RGB pixels (row-major, 3 bytes per pixel) as PNG.
///
/// # Safety
/// `rgb` must point to a valid RGB buffer of at least `len` bytes;
/// `out` must be a writable `WaiBuffer*`. Pass `(null, 0)` for empty.
#[unsafe(no_mangle)]
pub unsafe extern "C" fn wai_image_png_encode(
rgb: *const u8, len: usize, h: u32, w: u32, out: *mut WaiBuffer,
) -> c_int {
ffi_buf_call(out, || {
let s = unsafe { input_slice(rgb, len) }
.ok_or((WAI_ERR_INVALID_ARG, "rgb pointer null with non-zero len".into()))?;
let expected = (h as usize)
.checked_mul(w as usize)
.and_then(|n| n.checked_mul(3))
.ok_or((WAI_ERR_INVALID_ARG, "h*w*3 overflows".into()))?;
if s.len() != expected {
return Err((WAI_ERR_INVALID_ARG,
format!("rgb len {} != h*w*3 = {}", s.len(), expected)));
}
ffi_codec(|| codecs::image::png_encode(s, h, w)
.map_err(|e| format!("{e:?}")))
})
}
/// Decode a PNG byte stream to RGB. Returns `(rgb, h, w)`.
#[unsafe(no_mangle)]
pub unsafe extern "C" fn wai_image_png_decode(
bytes: *const u8, len: usize, out: *mut WaiBuffer,
out_h: *mut u32, out_w: *mut u32,
) -> c_int {
if out_h.is_null() || out_w.is_null() {
set_err("out_h/out_w null");
return WAI_ERR_INVALID_ARG;
}
decode_to_rgb_size(out, out_h, out_w, || {
let s = unsafe { input_slice(bytes, len) }
.ok_or((WAI_ERR_INVALID_ARG, "bytes pointer null with non-zero len".into()))?;
codecs::image::png_decode(s)
.map_err(|e| (WAI_ERR_CODEC, format!("{e:?}")))
})
}
fn decode_to_rgb_size<F>(out: *mut WaiBuffer, out_h: *mut u32, out_w: *mut u32, f: F) -> c_int
where F: FnOnce() -> std::result::Result<(Vec<u8>, u32, u32), (c_int, String)> + panic::UnwindSafe
{
if out.is_null() {
set_err("out null"); return WAI_ERR_INVALID_ARG;
}
match panic::catch_unwind(f) {
Ok(Ok((v, h, w))) => unsafe {
*out = WaiBuffer::from_vec(v); *out_h = h; *out_w = w; WAI_OK
}
Ok(Err((code, msg))) => {
set_err(msg);
unsafe { clear_buf(out); *out_h = 0; *out_w = 0; }
code
}
Err(p) => {
let msg = if let Some(s) = p.downcast_ref::<&str>() { (*s).to_string() }
else if let Some(s) = p.downcast_ref::<String>() { s.clone() }
else { "panic with non-string payload".into() };
set_err(format!("internal panic: {msg}"));
unsafe { clear_buf(out); *out_h = 0; *out_w = 0; }
WAI_ERR_PANIC
}
}
}
// ---- image: AVIF ----------------------------------------------------
#[unsafe(no_mangle)]
pub unsafe extern "C" fn wai_image_avif_encode(
rgb: *const u8, len: usize, h: u32, w: u32, quality: u8,
out: *mut WaiBuffer,
) -> c_int {
ffi_buf_call(out, || {
let s = unsafe { input_slice(rgb, len) }
.ok_or((WAI_ERR_INVALID_ARG, "rgb null + len>0".into()))?;
let expected = (h as usize).checked_mul(w as usize)
.and_then(|n| n.checked_mul(3))
.ok_or((WAI_ERR_INVALID_ARG, "h*w*3 overflows".into()))?;
if s.len() != expected {
return Err((WAI_ERR_INVALID_ARG,
format!("rgb len {} != h*w*3 = {}", s.len(), expected)));
}
codecs::image::avif_encode(s, h, w, quality)
.map_err(|e| (WAI_ERR_CODEC, format!("{e:?}")))
})
}
#[unsafe(no_mangle)]
pub unsafe extern "C" fn wai_image_avif_decode(
bytes: *const u8, len: usize, out: *mut WaiBuffer,
out_h: *mut u32, out_w: *mut u32,
) -> c_int {
if out_h.is_null() || out_w.is_null() {
set_err("out_h/out_w null"); return WAI_ERR_INVALID_ARG;
}
decode_to_rgb_size(out, out_h, out_w, || {
let s = unsafe { input_slice(bytes, len) }
.ok_or((WAI_ERR_INVALID_ARG, "bytes null + len>0".into()))?;
codecs::image::avif_decode(s)
.map_err(|e| (WAI_ERR_CODEC, format!("{e:?}")))
})
}
// ---- image: JPEG-XL -------------------------------------------------
/// `quality = 0` ⇒ lossless. Otherwise butteraugli distance proxy
/// (higher quality = smaller distance internally).
#[unsafe(no_mangle)]
pub unsafe extern "C" fn wai_image_jxl_encode(
rgb: *const u8, len: usize, h: u32, w: u32, quality: u8,
out: *mut WaiBuffer,
) -> c_int {
ffi_buf_call(out, || {
let s = unsafe { input_slice(rgb, len) }
.ok_or((WAI_ERR_INVALID_ARG, "rgb null + len>0".into()))?;
let expected = (h as usize).checked_mul(w as usize)
.and_then(|n| n.checked_mul(3))
.ok_or((WAI_ERR_INVALID_ARG, "h*w*3 overflows".into()))?;
if s.len() != expected {
return Err((WAI_ERR_INVALID_ARG,
format!("rgb len {} != h*w*3 = {}", s.len(), expected)));
}
let q = if quality == 0 { None } else { Some(quality as f32) };
codecs::image::jxl_encode(s, h, w, q)
.map_err(|e| (WAI_ERR_CODEC, format!("{e:?}")))
})
}
#[unsafe(no_mangle)]
pub unsafe extern "C" fn wai_image_jxl_decode(
bytes: *const u8, len: usize, out: *mut WaiBuffer,
out_h: *mut u32, out_w: *mut u32,
) -> c_int {
if out_h.is_null() || out_w.is_null() {
set_err("out_h/out_w null"); return WAI_ERR_INVALID_ARG;
}
decode_to_rgb_size(out, out_h, out_w, || {
let s = unsafe { input_slice(bytes, len) }
.ok_or((WAI_ERR_INVALID_ARG, "bytes null + len>0".into()))?;
codecs::image::jxl_decode(s)
.map_err(|e| (WAI_ERR_CODEC, format!("{e:?}")))
})
}
// ---- audio: Opus ----------------------------------------------------
#[unsafe(no_mangle)]
pub unsafe extern "C" fn wai_audio_opus_encode(
samples: *const f32, n: usize, sr: u32, bitrate_bps: i32,
out: *mut WaiBuffer,
) -> c_int {
ffi_buf_call(out, || {
if samples.is_null() && n != 0 {
return Err((WAI_ERR_INVALID_ARG, "samples null + n>0".into()));
}
let s = if n == 0 { &[][..] }
else { unsafe { slice::from_raw_parts(samples, n) } };
codecs::audio::opus_encode(s, sr, bitrate_bps)
.map_err(|e| (WAI_ERR_CODEC, format!("{e:?}")))
})
}
#[unsafe(no_mangle)]
pub unsafe extern "C" fn wai_audio_opus_decode(
bytes: *const u8, len: usize, out: *mut WaiBuffer, out_sr: *mut u32,
) -> c_int {
if out_sr.is_null() {
set_err("out_sr null"); return WAI_ERR_INVALID_ARG;
}
if out.is_null() {
set_err("out null"); return WAI_ERR_INVALID_ARG;
}
match panic::catch_unwind(|| -> std::result::Result<(Vec<f32>, u32), (c_int, String)> {
let s = unsafe { input_slice(bytes, len) }
.ok_or((WAI_ERR_INVALID_ARG, "bytes null + len>0".into()))?;
codecs::audio::opus_decode(s)
.map_err(|e| (WAI_ERR_CODEC, format!("{e:?}")))
}) {
Ok(Ok((samples, sr))) => {
let bytes: Vec<u8> = samples.iter().flat_map(|f| f.to_le_bytes()).collect();
unsafe { *out = WaiBuffer::from_vec(bytes); *out_sr = sr; }
WAI_OK
}
Ok(Err((code, msg))) => {
set_err(msg);
unsafe { clear_buf(out); *out_sr = 0; }
code
}
Err(_) => { set_err("internal panic in opus_decode");
unsafe { clear_buf(out); *out_sr = 0; } WAI_ERR_PANIC }
}
}
// ---- audio: FLAC ----------------------------------------------------
#[unsafe(no_mangle)]
pub unsafe extern "C" fn wai_audio_flac_encode(
samples: *const f32, n: usize, sr: u32, out: *mut WaiBuffer,
) -> c_int {
ffi_buf_call(out, || {
if samples.is_null() && n != 0 {
return Err((WAI_ERR_INVALID_ARG, "samples null + n>0".into()));
}
let s = if n == 0 { &[][..] }
else { unsafe { slice::from_raw_parts(samples, n) } };
codecs::audio::flac_encode(s, sr)
.map_err(|e| (WAI_ERR_CODEC, format!("{e:?}")))
})
}
#[unsafe(no_mangle)]
pub unsafe extern "C" fn wai_audio_flac_decode(
bytes: *const u8, len: usize, out: *mut WaiBuffer, out_sr: *mut u32,
) -> c_int {
if out_sr.is_null() || out.is_null() {
set_err("out/out_sr null"); return WAI_ERR_INVALID_ARG;
}
match panic::catch_unwind(|| -> std::result::Result<(Vec<f32>, u32), (c_int, String)> {
let s = unsafe { input_slice(bytes, len) }
.ok_or((WAI_ERR_INVALID_ARG, "bytes null + len>0".into()))?;
codecs::audio::flac_decode(s)
.map_err(|e| (WAI_ERR_CODEC, format!("{e:?}")))
}) {
Ok(Ok((samples, sr))) => {
let bytes: Vec<u8> = samples.iter().flat_map(|f| f.to_le_bytes()).collect();
unsafe { *out = WaiBuffer::from_vec(bytes); *out_sr = sr; }
WAI_OK
}
Ok(Err((code, msg))) => {
set_err(msg);
unsafe { clear_buf(out); *out_sr = 0; }
code
}
Err(_) => { set_err("internal panic in flac_decode");
unsafe { clear_buf(out); *out_sr = 0; } WAI_ERR_PANIC }
}
}
// ---- text: zstd / xz ------------------------------------------------
#[unsafe(no_mangle)]
pub unsafe extern "C" fn wai_text_zstd_encode(
data: *const u8, len: usize, level: i32, out: *mut WaiBuffer,
) -> c_int {
ffi_buf_call(out, || {
let s = unsafe { input_slice(data, len) }
.ok_or((WAI_ERR_INVALID_ARG, "data null + len>0".into()))?;
codecs::text::zstd_encode(s, level)
.map_err(|e| (WAI_ERR_CODEC, format!("{e:?}")))
})
}
#[unsafe(no_mangle)]
pub unsafe extern "C" fn wai_text_zstd_decode(
bytes: *const u8, len: usize, out: *mut WaiBuffer,
) -> c_int {
ffi_buf_call(out, || {
let s = unsafe { input_slice(bytes, len) }
.ok_or((WAI_ERR_INVALID_ARG, "bytes null + len>0".into()))?;
codecs::text::zstd_decode(s)
.map_err(|e| (WAI_ERR_CODEC, format!("{e:?}")))
})
}
#[unsafe(no_mangle)]
pub unsafe extern "C" fn wai_text_xz_encode(
data: *const u8, len: usize, level: u32, out: *mut WaiBuffer,
) -> c_int {
ffi_buf_call(out, || {
let s = unsafe { input_slice(data, len) }
.ok_or((WAI_ERR_INVALID_ARG, "data null + len>0".into()))?;
codecs::text::xz_encode(s, level)
.map_err(|e| (WAI_ERR_CODEC, format!("{e:?}")))
})
}
#[unsafe(no_mangle)]
pub unsafe extern "C" fn wai_text_xz_decode(
bytes: *const u8, len: usize, out: *mut WaiBuffer,
) -> c_int {
ffi_buf_call(out, || {
let s = unsafe { input_slice(bytes, len) }
.ok_or((WAI_ERR_INVALID_ARG, "bytes null + len>0".into()))?;
codecs::text::xz_decode(s)
.map_err(|e| (WAI_ERR_CODEC, format!("{e:?}")))
})
}
// ---- video: AV1 -----------------------------------------------------
/// Encode a sequence of `n` RGB frames as AV1.
#[unsafe(no_mangle)]
pub unsafe extern "C" fn wai_video_av1_encode(
rgb: *const u8, len: usize,
n: u32, h: u32, w: u32, fps_num: u32, fps_den: u32,
lossless: u8, quality: u8,
out: *mut WaiBuffer,
) -> c_int {
ffi_buf_call(out, || {
let s = unsafe { input_slice(rgb, len) }
.ok_or((WAI_ERR_INVALID_ARG, "rgb null + len>0".into()))?;
let per_frame = (h as usize).checked_mul(w as usize)
.and_then(|x| x.checked_mul(3))
.ok_or((WAI_ERR_INVALID_ARG, "h*w*3 overflows".into()))?;
let total = (n as usize).checked_mul(per_frame)
.ok_or((WAI_ERR_INVALID_ARG, "n*h*w*3 overflows".into()))?;
if s.len() != total {
return Err((WAI_ERR_INVALID_ARG,
format!("video rgb len {} != n*h*w*3 = {}", s.len(), total)));
}
if fps_num == 0 || fps_den == 0 {
return Err((WAI_ERR_INVALID_ARG, "fps_num/fps_den must be non-zero".into()));
}
let frames: Vec<Vec<u8>> = (0..n as usize)
.map(|i| s[i * per_frame..(i + 1) * per_frame].to_vec())
.collect();
crate::codecs::video::av1_encode(&frames, h, w, fps_num, fps_den,
lossless != 0, quality)
.map_err(|e| (WAI_ERR_CODEC, format!("{e:?}")))
})
}
/// Decode an AV1 WAI payload to a packed RGB buffer `n*h*w*3` bytes.
#[unsafe(no_mangle)]
pub unsafe extern "C" fn wai_video_av1_decode(
bytes: *const u8, len: usize, out: *mut WaiBuffer,
out_n: *mut u32, out_h: *mut u32, out_w: *mut u32,
out_fps_num: *mut u32, out_fps_den: *mut u32,
) -> c_int {
if out.is_null() || out_n.is_null() || out_h.is_null() || out_w.is_null()
|| out_fps_num.is_null() || out_fps_den.is_null() {
set_err("video decode: one of the out pointers is null");
return WAI_ERR_INVALID_ARG;
}
match panic::catch_unwind(|| -> std::result::Result<(Vec<Vec<u8>>, u32, u32, u32, u32), (c_int, String)> {
let s = unsafe { input_slice(bytes, len) }
.ok_or((WAI_ERR_INVALID_ARG, "bytes null + len>0".into()))?;
crate::codecs::video::av1_decode(s)
.map_err(|e| (WAI_ERR_CODEC, format!("{e:?}")))
}) {
Ok(Ok((frames, h, w, fps_num, fps_den))) => {
let n = frames.len() as u32;
let mut packed = Vec::with_capacity(frames.len() * (h * w * 3) as usize);
for f in &frames { packed.extend_from_slice(f); }
unsafe {
*out = WaiBuffer::from_vec(packed);
*out_n = n; *out_h = h; *out_w = w;
*out_fps_num = fps_num; *out_fps_den = fps_den;
}
WAI_OK
}
Ok(Err((code, msg))) => {
set_err(msg);
unsafe {
clear_buf(out);
*out_n = 0; *out_h = 0; *out_w = 0;
*out_fps_num = 0; *out_fps_den = 0;
}
code
}
Err(_) => {
set_err("internal panic in av1_decode");
unsafe {
clear_buf(out);
*out_n = 0; *out_h = 0; *out_w = 0;
*out_fps_num = 0; *out_fps_den = 0;
}
WAI_ERR_PANIC
}
}
}
// ---- envelope: wrap an existing payload in a WAI container ----------
/// The C error for an envelope the emit path refused: a candidate is the
/// caller's argument (`WAI_ERR_INVALID_ARG`), a serialisation failure the
@ -614,37 +256,33 @@ pub unsafe extern "C" fn wai_envelope_pack_multi(
return Err((WAI_ERR_INVALID_ARG,
"payload_data/payload_lens null with n_payloads>0".into()));
}
// Collect rendition payloads. The native WaiMulti API takes
// owned Vec<u8>, so copy each input buffer once here.
let mut renditions: Vec<crate::container::Rendition> =
Vec::with_capacity(n_payloads);
// Collect rendition payloads. Every member of the manifest — the component
// identity members, and any a later revision defines — is kept as written.
let mut payloads: Vec<Vec<u8>> = Vec::with_capacity(n_payloads);
for i in 0..n_payloads {
let ptr = unsafe { *payload_data.add(i) };
let len = unsafe { *payload_lens.add(i) };
let s = unsafe { input_slice(ptr, len) }
.ok_or((WAI_ERR_INVALID_ARG,
format!("payload {i} null with len>0")))?;
renditions.push(crate::container::Rendition {
capability: manifest.renditions[i].capability.clone(),
kind: manifest.renditions[i].kind.clone(),
payload: s.to_vec(),
prior: manifest.renditions[i].prior.clone(),
role: manifest.renditions[i].role.clone(),
companion_of: manifest.renditions[i].companion_of.clone(),
});
payloads.push(s.to_vec());
}
crate::container::WaiMulti::new(
manifest.media, manifest.intent, manifest.target, renditions)
crate::container::WaiMulti::from_parts(manifest, payloads)
.map_err(|e| (WAI_ERR_INVALID_ARG, e.to_string()))?
.to_emit_bytes().map_err(emit_err)
})
}
/// Parse a WAI2 envelope. Writes three outputs:
///
/// - `out_manifest` — the original manifest JSON (UTF-8 bytes, no NUL)
/// - `out_renditions_json` — a JSON array of `{capability, kind, offset,
/// length}` entries. `offset` is into `out_payload_block`, `length`
/// is the rendition's payload byte count.
/// - `out_manifest` — the manifest JSON exactly as the envelope carries it
/// (UTF-8 bytes, no NUL), so members this library does not model survive
/// - `out_renditions_json` — a JSON array with one object per component:
/// `capability`, `kind`, `offset` (into `out_payload_block`), `length`, and
/// `selectable` (whether SPEC §7.1 may pick it: a companion, or a companion-only
/// capability, never is), plus `id`, `sha256`, `prior`, `role`, `companion_of`
/// and `companion_of_id` when the entry carries them. A C sink selects with
/// `wai_envelope_select` rather than by hand.
/// - `out_payload_block` — every rendition's payload bytes, packed
/// contiguously. The caller indexes via the JSON above.
///
@ -669,18 +307,29 @@ pub unsafe extern "C" fn wai_envelope_unpack_multi(
.ok_or((WAI_ERR_INVALID_ARG, "bytes null + len>0".into()))?;
let m = crate::container::WaiMulti::from_bytes(s)
.map_err(|e| (WAI_ERR_CONTAINER, format!("envelope parse: {e}")))?;
let mb = serde_json::to_vec(&m.manifest)
.map_err(|e| (WAI_ERR_CONTAINER, format!("manifest serialize: {e}")))?;
let mb = crate::container::manifest_bytes(s)
.map_err(|e| (WAI_ERR_CONTAINER, format!("envelope parse: {e}")))?
.to_vec();
// build rendition table JSON + concatenated payload block
let mut table: Vec<serde_json::Value> = Vec::with_capacity(m.renditions.len());
let mut block: Vec<u8> = Vec::new();
for (meta, payload) in m.manifest.renditions.iter().zip(m.renditions.iter()) {
table.push(serde_json::json!({
let mut e = serde_json::json!({
"capability": meta.capability,
"kind": meta.kind,
"offset": block.len(),
"length": payload.len(),
}));
"selectable": meta.is_selectable(),
});
let o = e.as_object_mut().unwrap();
let mut put = |k: &str, v: Option<serde_json::Value>| if let Some(v) = v { o.insert(k.into(), v); };
put("id", meta.id.clone().map(Into::into));
put("sha256", meta.sha256.clone().map(Into::into));
put("prior", meta.prior.as_ref().and_then(|p| serde_json::to_value(p).ok()));
put("role", meta.role.clone().map(Into::into));
put("companion_of", meta.companion_of.clone().map(Into::into));
put("companion_of_id", meta.companion_of_id.clone().map(Into::into));
table.push(e);
block.extend_from_slice(payload);
}
let tj = serde_json::to_vec(&serde_json::Value::Array(table))
@ -714,8 +363,9 @@ pub unsafe extern "C" fn wai_envelope_unpack_multi(
}
}
/// Parses a WAI envelope. Writes the manifest as JSON to `out_manifest`
/// and the raw payload to `out_payload`.
/// Parses a WAI envelope. Writes the manifest JSON exactly as the envelope
/// carries it to `out_manifest` (so `security`, and members this library does not
/// model, survive) and the raw payload to `out_payload`.
#[unsafe(no_mangle)]
pub unsafe extern "C" fn wai_envelope_unpack(
bytes: *const u8, len: usize,
@ -730,8 +380,9 @@ pub unsafe extern "C" fn wai_envelope_unpack(
.ok_or((WAI_ERR_INVALID_ARG, "bytes null + len>0".into()))?;
let wai = crate::container::Wai::from_bytes(s)
.map_err(|e| (WAI_ERR_CONTAINER, format!("envelope parse: {e}")))?;
let mj = serde_json::to_vec(&wai.manifest)
.map_err(|e| (WAI_ERR_CONTAINER, format!("manifest serialize: {e}")))?;
let mj = crate::container::manifest_bytes(s)
.map_err(|e| (WAI_ERR_CONTAINER, format!("envelope parse: {e}")))?
.to_vec();
Ok((mj, wai.payload))
}) {
Ok(Ok((mj, pl))) => unsafe {
@ -751,3 +402,48 @@ pub unsafe extern "C" fn wai_envelope_unpack(
}
}
}
/// SPEC §7.1 selection for a `WAI2` envelope at a sink that advertises the
/// capabilities in `advertised_json` (a NUL-terminated JSON array of capability
/// strings). Writes the index of the picked rendition to `out_index`: the first
/// entry that is selectable (never a companion, nor a capability that is not
/// dispatchable under SPEC §4: companion-only, a candidate, a record or a
/// derivation), advertised, and whose payload matches its `sha256`. This call holds no
/// parameter set, so an entry whose pin applies is skipped (`prior_not_held`); a
/// sink that holds parameter sets decodes through `wai_neural_decode_ex`, which
/// verifies each pin. Returns `WAI_ERR_NO_RENDITION` when nothing is picked, with
/// each entry's index and reason in `wai_last_error`.
///
/// # Safety
/// `envelope` valid for `envelope_len` bytes; `advertised_json` NUL-terminated;
/// `out_index` writable.
#[unsafe(no_mangle)]
pub unsafe extern "C" fn wai_envelope_select(
envelope: *const u8, envelope_len: usize,
advertised_json: *const c_char,
out_index: *mut u32,
) -> c_int {
if out_index.is_null() || advertised_json.is_null() {
set_err("envelope_select: out_index/advertised_json null");
return WAI_ERR_INVALID_ARG;
}
let r = panic::catch_unwind(|| -> std::result::Result<u32, (c_int, String)> {
let s = unsafe { input_slice(envelope, envelope_len) }
.ok_or((WAI_ERR_INVALID_ARG, "envelope null + len>0".into()))?;
let adv = unsafe { std::ffi::CStr::from_ptr(advertised_json) }.to_string_lossy().to_string();
let advertised: Vec<String> = serde_json::from_str(&adv)
.map_err(|e| (WAI_ERR_INVALID_ARG, format!("advertised_json: {e}")))?;
let m = crate::container::WaiMulti::from_bytes(s)
.map_err(|e| (WAI_ERR_CONTAINER, format!("envelope parse: {e}")))?;
match m.pick(|c| advertised.iter().any(|a| a == c), crate::container::no_parameter_sets()) {
Ok(p) => Ok(p.index as u32),
Err(skips) => Err((WAI_ERR_NO_RENDITION, skips.iter()
.map(|(i, k)| format!("{i}: {k}")).collect::<Vec<_>>().join("; "))),
}
});
match r {
Ok(Ok(i)) => { unsafe { *out_index = i; } WAI_OK }
Ok(Err((code, msg))) => { set_err(msg); code }
Err(_) => { set_err("internal panic in envelope_select"); WAI_ERR_PANIC }
}
}

390
wai/wai-rs/src/ffi_codecs.rs Normal file
View file

@ -0,0 +1,390 @@
//! C ABI for the zeroth-condition codec wrappers (feature `codecs`): encode and
//! decode through PNG, AVIF, JPEG XL, Opus, FLAC, zstd, XZ and AV1. The envelope,
//! buffer and error surface they share is [`crate::ffi`] (feature `ffi`), which
//! re-exports these functions.
use std::ffi::c_int;
use std::panic;
use std::slice;
use crate::codecs;
use crate::ffi::*;
fn ffi_codec<T, F>(f: F) -> std::result::Result<Vec<u8>, (c_int, String)>
where F: FnOnce() -> std::result::Result<T, String>,
T: Into<Vec<u8>>,
{
f().map(Into::into).map_err(|e| (WAI_ERR_CODEC, e))
}
// ---- image: PNG -----------------------------------------------------
/// Encode RGB pixels (row-major, 3 bytes per pixel) as PNG.
///
/// # Safety
/// `rgb` must point to a valid RGB buffer of at least `len` bytes;
/// `out` must be a writable `WaiBuffer*`. Pass `(null, 0)` for empty.
#[unsafe(no_mangle)]
pub unsafe extern "C" fn wai_image_png_encode(
rgb: *const u8, len: usize, h: u32, w: u32, out: *mut WaiBuffer,
) -> c_int {
ffi_buf_call(out, || {
let s = unsafe { input_slice(rgb, len) }
.ok_or((WAI_ERR_INVALID_ARG, "rgb pointer null with non-zero len".into()))?;
let expected = (h as usize)
.checked_mul(w as usize)
.and_then(|n| n.checked_mul(3))
.ok_or((WAI_ERR_INVALID_ARG, "h*w*3 overflows".into()))?;
if s.len() != expected {
return Err((WAI_ERR_INVALID_ARG,
format!("rgb len {} != h*w*3 = {}", s.len(), expected)));
}
ffi_codec(|| codecs::image::png_encode(s, h, w)
.map_err(|e| format!("{e:?}")))
})
}
/// Decode a PNG byte stream to RGB. Returns `(rgb, h, w)`.
#[unsafe(no_mangle)]
pub unsafe extern "C" fn wai_image_png_decode(
bytes: *const u8, len: usize, out: *mut WaiBuffer,
out_h: *mut u32, out_w: *mut u32,
) -> c_int {
if out_h.is_null() || out_w.is_null() {
set_err("out_h/out_w null");
return WAI_ERR_INVALID_ARG;
}
decode_to_rgb_size(out, out_h, out_w, || {
let s = unsafe { input_slice(bytes, len) }
.ok_or((WAI_ERR_INVALID_ARG, "bytes pointer null with non-zero len".into()))?;
codecs::image::png_decode(s)
.map_err(|e| (WAI_ERR_CODEC, format!("{e:?}")))
})
}
fn decode_to_rgb_size<F>(out: *mut WaiBuffer, out_h: *mut u32, out_w: *mut u32, f: F) -> c_int
where F: FnOnce() -> std::result::Result<(Vec<u8>, u32, u32), (c_int, String)> + panic::UnwindSafe
{
if out.is_null() {
set_err("out null"); return WAI_ERR_INVALID_ARG;
}
match panic::catch_unwind(f) {
Ok(Ok((v, h, w))) => unsafe {
*out = WaiBuffer::from_vec(v); *out_h = h; *out_w = w; WAI_OK
}
Ok(Err((code, msg))) => {
set_err(msg);
unsafe { clear_buf(out); *out_h = 0; *out_w = 0; }
code
}
Err(p) => {
let msg = if let Some(s) = p.downcast_ref::<&str>() { (*s).to_string() }
else if let Some(s) = p.downcast_ref::<String>() { s.clone() }
else { "panic with non-string payload".into() };
set_err(format!("internal panic: {msg}"));
unsafe { clear_buf(out); *out_h = 0; *out_w = 0; }
WAI_ERR_PANIC
}
}
}
// ---- image: AVIF ----------------------------------------------------
#[unsafe(no_mangle)]
pub unsafe extern "C" fn wai_image_avif_encode(
rgb: *const u8, len: usize, h: u32, w: u32, quality: u8,
out: *mut WaiBuffer,
) -> c_int {
ffi_buf_call(out, || {
let s = unsafe { input_slice(rgb, len) }
.ok_or((WAI_ERR_INVALID_ARG, "rgb null + len>0".into()))?;
let expected = (h as usize).checked_mul(w as usize)
.and_then(|n| n.checked_mul(3))
.ok_or((WAI_ERR_INVALID_ARG, "h*w*3 overflows".into()))?;
if s.len() != expected {
return Err((WAI_ERR_INVALID_ARG,
format!("rgb len {} != h*w*3 = {}", s.len(), expected)));
}
codecs::image::avif_encode(s, h, w, quality)
.map_err(|e| (WAI_ERR_CODEC, format!("{e:?}")))
})
}
#[unsafe(no_mangle)]
pub unsafe extern "C" fn wai_image_avif_decode(
bytes: *const u8, len: usize, out: *mut WaiBuffer,
out_h: *mut u32, out_w: *mut u32,
) -> c_int {
if out_h.is_null() || out_w.is_null() {
set_err("out_h/out_w null"); return WAI_ERR_INVALID_ARG;
}
decode_to_rgb_size(out, out_h, out_w, || {
let s = unsafe { input_slice(bytes, len) }
.ok_or((WAI_ERR_INVALID_ARG, "bytes null + len>0".into()))?;
codecs::image::avif_decode(s)
.map_err(|e| (WAI_ERR_CODEC, format!("{e:?}")))
})
}
// ---- image: JPEG-XL -------------------------------------------------
/// `quality = 0` ⇒ lossless. Otherwise butteraugli distance proxy
/// (higher quality = smaller distance internally).
#[unsafe(no_mangle)]
pub unsafe extern "C" fn wai_image_jxl_encode(
rgb: *const u8, len: usize, h: u32, w: u32, quality: u8,
out: *mut WaiBuffer,
) -> c_int {
ffi_buf_call(out, || {
let s = unsafe { input_slice(rgb, len) }
.ok_or((WAI_ERR_INVALID_ARG, "rgb null + len>0".into()))?;
let expected = (h as usize).checked_mul(w as usize)
.and_then(|n| n.checked_mul(3))
.ok_or((WAI_ERR_INVALID_ARG, "h*w*3 overflows".into()))?;
if s.len() != expected {
return Err((WAI_ERR_INVALID_ARG,
format!("rgb len {} != h*w*3 = {}", s.len(), expected)));
}
let q = if quality == 0 { None } else { Some(quality as f32) };
codecs::image::jxl_encode(s, h, w, q)
.map_err(|e| (WAI_ERR_CODEC, format!("{e:?}")))
})
}
#[unsafe(no_mangle)]
pub unsafe extern "C" fn wai_image_jxl_decode(
bytes: *const u8, len: usize, out: *mut WaiBuffer,
out_h: *mut u32, out_w: *mut u32,
) -> c_int {
if out_h.is_null() || out_w.is_null() {
set_err("out_h/out_w null"); return WAI_ERR_INVALID_ARG;
}
decode_to_rgb_size(out, out_h, out_w, || {
let s = unsafe { input_slice(bytes, len) }
.ok_or((WAI_ERR_INVALID_ARG, "bytes null + len>0".into()))?;
codecs::image::jxl_decode(s)
.map_err(|e| (WAI_ERR_CODEC, format!("{e:?}")))
})
}
// ---- audio: Opus ----------------------------------------------------
#[unsafe(no_mangle)]
pub unsafe extern "C" fn wai_audio_opus_encode(
samples: *const f32, n: usize, sr: u32, bitrate_bps: i32,
out: *mut WaiBuffer,
) -> c_int {
ffi_buf_call(out, || {
if samples.is_null() && n != 0 {
return Err((WAI_ERR_INVALID_ARG, "samples null + n>0".into()));
}
let s = if n == 0 { &[][..] }
else { unsafe { slice::from_raw_parts(samples, n) } };
codecs::audio::opus_encode(s, sr, bitrate_bps)
.map_err(|e| (WAI_ERR_CODEC, format!("{e:?}")))
})
}
#[unsafe(no_mangle)]
pub unsafe extern "C" fn wai_audio_opus_decode(
bytes: *const u8, len: usize, out: *mut WaiBuffer, out_sr: *mut u32,
) -> c_int {
if out_sr.is_null() {
set_err("out_sr null"); return WAI_ERR_INVALID_ARG;
}
if out.is_null() {
set_err("out null"); return WAI_ERR_INVALID_ARG;
}
match panic::catch_unwind(|| -> std::result::Result<(Vec<f32>, u32), (c_int, String)> {
let s = unsafe { input_slice(bytes, len) }
.ok_or((WAI_ERR_INVALID_ARG, "bytes null + len>0".into()))?;
codecs::audio::opus_decode(s)
.map_err(|e| (WAI_ERR_CODEC, format!("{e:?}")))
}) {
Ok(Ok((samples, sr))) => {
let bytes: Vec<u8> = samples.iter().flat_map(|f| f.to_le_bytes()).collect();
unsafe { *out = WaiBuffer::from_vec(bytes); *out_sr = sr; }
WAI_OK
}
Ok(Err((code, msg))) => {
set_err(msg);
unsafe { clear_buf(out); *out_sr = 0; }
code
}
Err(_) => { set_err("internal panic in opus_decode");
unsafe { clear_buf(out); *out_sr = 0; } WAI_ERR_PANIC }
}
}
// ---- audio: FLAC ----------------------------------------------------
#[unsafe(no_mangle)]
pub unsafe extern "C" fn wai_audio_flac_encode(
samples: *const f32, n: usize, sr: u32, out: *mut WaiBuffer,
) -> c_int {
ffi_buf_call(out, || {
if samples.is_null() && n != 0 {
return Err((WAI_ERR_INVALID_ARG, "samples null + n>0".into()));
}
let s = if n == 0 { &[][..] }
else { unsafe { slice::from_raw_parts(samples, n) } };
codecs::audio::flac_encode(s, sr)
.map_err(|e| (WAI_ERR_CODEC, format!("{e:?}")))
})
}
#[unsafe(no_mangle)]
pub unsafe extern "C" fn wai_audio_flac_decode(
bytes: *const u8, len: usize, out: *mut WaiBuffer, out_sr: *mut u32,
) -> c_int {
if out_sr.is_null() || out.is_null() {
set_err("out/out_sr null"); return WAI_ERR_INVALID_ARG;
}
match panic::catch_unwind(|| -> std::result::Result<(Vec<f32>, u32), (c_int, String)> {
let s = unsafe { input_slice(bytes, len) }
.ok_or((WAI_ERR_INVALID_ARG, "bytes null + len>0".into()))?;
codecs::audio::flac_decode(s)
.map_err(|e| (WAI_ERR_CODEC, format!("{e:?}")))
}) {
Ok(Ok((samples, sr))) => {
let bytes: Vec<u8> = samples.iter().flat_map(|f| f.to_le_bytes()).collect();
unsafe { *out = WaiBuffer::from_vec(bytes); *out_sr = sr; }
WAI_OK
}
Ok(Err((code, msg))) => {
set_err(msg);
unsafe { clear_buf(out); *out_sr = 0; }
code
}
Err(_) => { set_err("internal panic in flac_decode");
unsafe { clear_buf(out); *out_sr = 0; } WAI_ERR_PANIC }
}
}
// ---- text: zstd / xz ------------------------------------------------
#[unsafe(no_mangle)]
pub unsafe extern "C" fn wai_text_zstd_encode(
data: *const u8, len: usize, level: i32, out: *mut WaiBuffer,
) -> c_int {
ffi_buf_call(out, || {
let s = unsafe { input_slice(data, len) }
.ok_or((WAI_ERR_INVALID_ARG, "data null + len>0".into()))?;
codecs::text::zstd_encode(s, level)
.map_err(|e| (WAI_ERR_CODEC, format!("{e:?}")))
})
}
#[unsafe(no_mangle)]
pub unsafe extern "C" fn wai_text_zstd_decode(
bytes: *const u8, len: usize, out: *mut WaiBuffer,
) -> c_int {
ffi_buf_call(out, || {
let s = unsafe { input_slice(bytes, len) }
.ok_or((WAI_ERR_INVALID_ARG, "bytes null + len>0".into()))?;
codecs::text::zstd_decode(s)
.map_err(|e| (WAI_ERR_CODEC, format!("{e:?}")))
})
}
#[unsafe(no_mangle)]
pub unsafe extern "C" fn wai_text_xz_encode(
data: *const u8, len: usize, level: u32, out: *mut WaiBuffer,
) -> c_int {
ffi_buf_call(out, || {
let s = unsafe { input_slice(data, len) }
.ok_or((WAI_ERR_INVALID_ARG, "data null + len>0".into()))?;
codecs::text::xz_encode(s, level)
.map_err(|e| (WAI_ERR_CODEC, format!("{e:?}")))
})
}
#[unsafe(no_mangle)]
pub unsafe extern "C" fn wai_text_xz_decode(
bytes: *const u8, len: usize, out: *mut WaiBuffer,
) -> c_int {
ffi_buf_call(out, || {
let s = unsafe { input_slice(bytes, len) }
.ok_or((WAI_ERR_INVALID_ARG, "bytes null + len>0".into()))?;
codecs::text::xz_decode(s)
.map_err(|e| (WAI_ERR_CODEC, format!("{e:?}")))
})
}
// ---- video: AV1 -----------------------------------------------------
/// Encode a sequence of `n` RGB frames as AV1.
#[unsafe(no_mangle)]
pub unsafe extern "C" fn wai_video_av1_encode(
rgb: *const u8, len: usize,
n: u32, h: u32, w: u32, fps_num: u32, fps_den: u32,
lossless: u8, quality: u8,
out: *mut WaiBuffer,
) -> c_int {
ffi_buf_call(out, || {
let s = unsafe { input_slice(rgb, len) }
.ok_or((WAI_ERR_INVALID_ARG, "rgb null + len>0".into()))?;
let per_frame = (h as usize).checked_mul(w as usize)
.and_then(|x| x.checked_mul(3))
.ok_or((WAI_ERR_INVALID_ARG, "h*w*3 overflows".into()))?;
let total = (n as usize).checked_mul(per_frame)
.ok_or((WAI_ERR_INVALID_ARG, "n*h*w*3 overflows".into()))?;
if s.len() != total {
return Err((WAI_ERR_INVALID_ARG,
format!("video rgb len {} != n*h*w*3 = {}", s.len(), total)));
}
if fps_num == 0 || fps_den == 0 {
return Err((WAI_ERR_INVALID_ARG, "fps_num/fps_den must be non-zero".into()));
}
let frames: Vec<Vec<u8>> = (0..n as usize)
.map(|i| s[i * per_frame..(i + 1) * per_frame].to_vec())
.collect();
crate::codecs::video::av1_encode(&frames, h, w, fps_num, fps_den,
lossless != 0, quality)
.map_err(|e| (WAI_ERR_CODEC, format!("{e:?}")))
})
}
/// Decode an AV1 WAI payload to a packed RGB buffer `n*h*w*3` bytes.
#[unsafe(no_mangle)]
pub unsafe extern "C" fn wai_video_av1_decode(
bytes: *const u8, len: usize, out: *mut WaiBuffer,
out_n: *mut u32, out_h: *mut u32, out_w: *mut u32,
out_fps_num: *mut u32, out_fps_den: *mut u32,
) -> c_int {
if out.is_null() || out_n.is_null() || out_h.is_null() || out_w.is_null()
|| out_fps_num.is_null() || out_fps_den.is_null() {
set_err("video decode: one of the out pointers is null");
return WAI_ERR_INVALID_ARG;
}
match panic::catch_unwind(|| -> std::result::Result<(Vec<Vec<u8>>, u32, u32, u32, u32), (c_int, String)> {
let s = unsafe { input_slice(bytes, len) }
.ok_or((WAI_ERR_INVALID_ARG, "bytes null + len>0".into()))?;
crate::codecs::video::av1_decode(s)
.map_err(|e| (WAI_ERR_CODEC, format!("{e:?}")))
}) {
Ok(Ok((frames, h, w, fps_num, fps_den))) => {
let n = frames.len() as u32;
let mut packed = Vec::with_capacity(frames.len() * (h * w * 3) as usize);
for f in &frames { packed.extend_from_slice(f); }
unsafe {
*out = WaiBuffer::from_vec(packed);
*out_n = n; *out_h = h; *out_w = w;
*out_fps_num = fps_num; *out_fps_den = fps_den;
}
WAI_OK
}
Ok(Err((code, msg))) => {
set_err(msg);
unsafe {
clear_buf(out);
*out_n = 0; *out_h = 0; *out_w = 0;
*out_fps_num = 0; *out_fps_den = 0;
}
code
}
Err(_) => {
set_err("internal panic in av1_decode");
unsafe {
clear_buf(out);
*out_n = 0; *out_h = 0; *out_w = 0;
*out_fps_num = 0; *out_fps_den = 0;
}
WAI_ERR_PANIC
}
}
}

279
wai/wai-rs/src/ffi_neural.rs
View file

@ -1,36 +1,35 @@
//! C ABI for the native neural sink (behind `feature = "neural"`).
//! C ABI for the native neural sink (features `neural` and `ffi`).
//!
//! Three entry points, one per media class:
//! One-model entry points, one per media class, for a `WAI1` envelope:
//!
//! wai_neural_decode_audio(envelope, len, model_path, *samples, *sr)
//! wai_neural_decode_image(envelope, len, model_path, *rgb, *w, *h)
//! wai_neural_decode_video(envelope, len, model_path,
//! *frames_concat, *n_frames, *w, *h, *fps_x_1000)
//!
//! Each caller passes the model path as a NUL-terminated C string;
//! that lets a C consumer point at any deployer-installed ONNX file
//! without re-linking. The Rust side picks the right decoder by parsing
//! the envelope's capability and validating media class matches.
//! and one entry point for a `WAI1` or `WAI2` envelope against a registry of the
//! parameter sets the sink holds:
//!
//! Output buffers are heap-allocated here; free with `wai_buffer_free`.
//! wai_neural_decode_ex(envelope, len, registry_json, flags, *WaiDecoded)
//!
//! Every pin is verified before anything is decoded (SPEC §3.1), and a pinned
//! decode reads only the bytes it verified. Errors carry a status code and a
//! message in `wai_last_error`: a refused pin is `WAI_ERR_PRIOR` with the reason
//! first (`prior_not_held: …`), an unsound `replicate` under the strict flag is
//! `WAI_ERR_UNSOUND_REPLICATE`, and a `WAI2` envelope with nothing to pick is
//! `WAI_ERR_NO_RENDITION`. Output buffers are heap-allocated here; free them with
//! `wai_buffer_free`.
use std::ffi::{c_char, c_int, CStr};
use std::panic;
use std::path::Path;
use std::slice;
use crate::container::Wai;
use crate::ffi::{WaiBuffer, WAI_ERR_CODEC, WAI_ERR_CONTAINER, WAI_ERR_INVALID_ARG,
WAI_ERR_PANIC, WAI_OK};
use crate::neural::{decode_envelope, Decoded, ModelRegistry};
// Neural FFI currently returns only status codes (no chained
// `wai_last_error` payload). Rationale: `set_err` in the main FFI module
// is private to that module, and forwarding the rich Rust-side error
// would require either making it `pub(crate)` or duplicating the
// thread-local. Status codes carry the essential branch info
// (invalid arg / container / codec / panic); for richer diagnostics the
// caller can drop into the Rust-side `wai::neural` API directly.
use crate::container::{Wai, WaiMulti};
use crate::ffi::{set_err, WaiBuffer, WAI_ERR_CODEC, WAI_ERR_CONTAINER, WAI_ERR_INVALID_ARG,
WAI_ERR_NO_RENDITION, WAI_ERR_PANIC, WAI_ERR_PRIOR, WAI_ERR_UNSOUND_REPLICATE, WAI_OK};
use crate::neural::{decode_envelope, decode_envelope_strict, decode_multi, decode_multi_strict, DecodeError,
Decoded, ModelRegistry};
unsafe fn input_envelope<'a>(bytes: *const u8, len: usize) -> Option<&'a [u8]> {
if bytes.is_null() && len > 0 { return None; }
@ -38,27 +37,62 @@ unsafe fn input_envelope<'a>(bytes: *const u8, len: usize) -> Option<&'a [u8]> {
Some(unsafe { slice::from_raw_parts(bytes, len) })
}
unsafe fn input_cstr<'a>(s: *const c_char) -> Option<&'a Path> {
unsafe fn input_cstr<'a>(s: *const c_char) -> Option<&'a str> {
if s.is_null() { return None; }
let cstr = unsafe { CStr::from_ptr(s) };
cstr.to_str().ok().map(Path::new)
unsafe { CStr::from_ptr(s) }.to_str().ok()
}
fn run<F: FnOnce() -> Result<(), c_int> + panic::UnwindSafe>(f: F) -> c_int {
match panic::catch_unwind(f) {
Ok(Ok(())) => WAI_OK,
Ok(Err(code)) => code,
Err(_) => WAI_ERR_PANIC,
/// The status code and message for a decode error.
fn decode_status(e: DecodeError) -> (c_int, String) {
match e {
DecodeError::PriorMismatch(r) => (WAI_ERR_PRIOR, format!("{}: {r}", r.code())),
DecodeError::UnsoundReplicate(why) => (WAI_ERR_UNSOUND_REPLICATE, why),
DecodeError::NoRendition(_) => (WAI_ERR_NO_RENDITION, e.to_string()),
other => (WAI_ERR_CODEC, other.to_string()),
}
}
fn run<F: FnOnce() -> Result<(), (c_int, String)> + panic::UnwindSafe>(f: F) -> c_int {
match panic::catch_unwind(f) {
Ok(Ok(())) => WAI_OK,
Ok(Err((code, msg))) => {
set_err(msg);
code
}
Err(_) => {
set_err("internal panic in the neural sink");
WAI_ERR_PANIC
}
}
}
/// Parse a `WAI1` envelope and decode it with `model_path` registered for its
/// capability.
fn decode_one(envelope: *const u8, envelope_len: usize, model_path: *const c_char) -> Result<Decoded, (c_int, String)> {
let bytes = unsafe { input_envelope(envelope, envelope_len) }
.ok_or((WAI_ERR_INVALID_ARG, "envelope null with len>0".to_string()))?;
let path = unsafe { input_cstr(model_path) }
.ok_or((WAI_ERR_INVALID_ARG, "model_path null or not UTF-8".to_string()))?;
let env = Wai::from_bytes(bytes).map_err(|e| (WAI_ERR_CONTAINER, format!("envelope parse: {e}")))?;
let reg = ModelRegistry::new().register(env.capability().to_string(), Path::new(path));
decode_envelope(&env, &reg).map_err(decode_status)
}
fn wrong_media(want: &str) -> (c_int, String) {
(WAI_ERR_CONTAINER, format!("the envelope's capability does not decode to {want}"))
}
fn f32_bytes(samples: &[f32]) -> Vec<u8> {
samples.iter().flat_map(|s| s.to_ne_bytes()).collect()
}
/// Decode a `wai.neural.<audio>` envelope (encodec32 / dac / mimi /
/// wavtokenizer). Writes mono `f32` samples into `out_samples`; the
/// wavtokenizer / snac). Writes mono `f32` samples into `out_samples`; the
/// caller reads them as `float*` of length `out_samples.len / 4`.
///
/// # Safety
/// `envelope` valid for `envelope_len` bytes. `model_path` NUL-terminated
/// UTF-8 path to the decoder.onnx. `out_samples` writable `WaiBuffer*`.
/// UTF-8 path to the decoder. `out_samples` writable `WaiBuffer*`.
/// `out_sample_rate` writable `uint32_t*` (or null to skip).
#[unsafe(no_mangle)]
pub unsafe extern "C" fn wai_neural_decode_audio(
@ -67,31 +101,17 @@ pub unsafe extern "C" fn wai_neural_decode_audio(
) -> c_int {
if out_samples.is_null() { return WAI_ERR_INVALID_ARG; }
run(|| {
let bytes = unsafe { input_envelope(envelope, envelope_len) }
.ok_or(WAI_ERR_INVALID_ARG)?;
let path = unsafe { input_cstr(model_path) }.ok_or(WAI_ERR_INVALID_ARG)?;
let env = Wai::from_bytes(bytes).map_err(|_| WAI_ERR_CONTAINER)?;
let reg = ModelRegistry::new()
.register(env.manifest.model_requirement.capability.clone(), path);
let dec = decode_envelope(&env, &reg).map_err(|_| WAI_ERR_CODEC)?;
let Decoded::Audio(audio) = dec else { return Err(WAI_ERR_CONTAINER); };
// f32 samples → bytes (LE on supported targets; native byte order).
let byte_len = audio.samples.len() * std::mem::size_of::<f32>();
let mut buf = Vec::<u8>::with_capacity(byte_len);
unsafe {
std::ptr::copy_nonoverlapping(
audio.samples.as_ptr() as *const u8, buf.as_mut_ptr(), byte_len);
buf.set_len(byte_len);
}
unsafe { *out_samples = WaiBuffer::from_vec(buf); }
let Decoded::Audio(audio) = decode_one(envelope, envelope_len, model_path)? else {
return Err(wrong_media("audio"));
};
unsafe { *out_samples = WaiBuffer::from_vec(f32_bytes(&audio.samples)); }
if !out_sample_rate.is_null() { unsafe { *out_sample_rate = audio.sample_rate; } }
Ok(())
})
}
/// Decode a `wai.neural.bmshj2018` envelope. Writes packed RGB
/// (row-major, 3 bytes/pixel) into `out_rgb`; writes width/height
/// into `*out_width` / `*out_height`.
/// Decode a neural image envelope. Writes packed RGB (row-major, 3 bytes/pixel)
/// into `out_rgb`; writes width/height into `*out_width` / `*out_height`.
///
/// # Safety
/// See `wai_neural_decode_audio`.
@ -102,26 +122,21 @@ pub unsafe extern "C" fn wai_neural_decode_image(
) -> c_int {
if out_rgb.is_null() { return WAI_ERR_INVALID_ARG; }
run(|| {
let bytes = unsafe { input_envelope(envelope, envelope_len) }
.ok_or(WAI_ERR_INVALID_ARG)?;
let path = unsafe { input_cstr(model_path) }.ok_or(WAI_ERR_INVALID_ARG)?;
let env = Wai::from_bytes(bytes).map_err(|_| WAI_ERR_CONTAINER)?;
let reg = ModelRegistry::new()
.register(env.manifest.model_requirement.capability.clone(), path);
let dec = decode_envelope(&env, &reg).map_err(|_| WAI_ERR_CODEC)?;
let Decoded::Image(img) = dec else { return Err(WAI_ERR_CONTAINER); };
let Decoded::Image(img) = decode_one(envelope, envelope_len, model_path)? else {
return Err(wrong_media("an image"));
};
unsafe {
*out_rgb = WaiBuffer::from_vec(img.rgb);
if !out_width.is_null() { *out_width = img.width; }
if !out_height.is_null() { *out_height = img.height; }
*out_rgb = WaiBuffer::from_vec(img.rgb);
}
Ok(())
})
}
/// Decode a `wai.neural.video_bmshj2018` envelope. Writes all frames
/// concatenated (frame 0 RGB, then frame 1 RGB, …) into
/// `out_frames_concat`; the caller splits at `width*height*3` strides.
/// Decode a neural video envelope. Writes all frames concatenated (frame 0
/// RGB, then frame 1 RGB, …) into `out_frames_concat`; the caller splits at
/// `width*height*3` strides.
///
/// # Safety
/// See `wai_neural_decode_audio`.
@ -134,24 +149,142 @@ pub unsafe extern "C" fn wai_neural_decode_video(
) -> c_int {
if out_frames_concat.is_null() { return WAI_ERR_INVALID_ARG; }
run(|| {
let bytes = unsafe { input_envelope(envelope, envelope_len) }
.ok_or(WAI_ERR_INVALID_ARG)?;
let path = unsafe { input_cstr(model_path) }.ok_or(WAI_ERR_INVALID_ARG)?;
let env = Wai::from_bytes(bytes).map_err(|_| WAI_ERR_CONTAINER)?;
let reg = ModelRegistry::new()
.register(env.manifest.model_requirement.capability.clone(), path);
let dec = decode_envelope(&env, &reg).map_err(|_| WAI_ERR_CODEC)?;
let Decoded::Video(v) = dec else { return Err(WAI_ERR_CONTAINER); };
let frame_bytes = (v.width as usize) * (v.height as usize) * 3;
let mut concat = Vec::with_capacity(frame_bytes * v.frames_rgb.len());
for f in &v.frames_rgb { concat.extend_from_slice(f); }
let Decoded::Video(v) = decode_one(envelope, envelope_len, model_path)? else {
return Err(wrong_media("video"));
};
let concat: Vec<u8> = v.frames_rgb.concat();
unsafe {
*out_frames_concat = WaiBuffer::from_vec(concat);
if !out_n_frames.is_null() { *out_n_frames = v.frames_rgb.len() as u32; }
if !out_width.is_null() { *out_width = v.width; }
if !out_height.is_null() { *out_height = v.height; }
if !out_fps_x_1000.is_null() { *out_fps_x_1000 = (v.fps * 1000.0).round() as u32; }
*out_frames_concat = WaiBuffer::from_vec(concat);
}
Ok(())
})
}
/// The media kind of a [`WaiDecoded`].
pub const WAI_DECODED_AUDIO: u32 = 1;
/// The media kind of a [`WaiDecoded`].
pub const WAI_DECODED_IMAGE: u32 = 2;
/// The media kind of a [`WaiDecoded`].
pub const WAI_DECODED_VIDEO: u32 = 3;
/// `wai_neural_decode_ex` flag: apply SPEC §7's replicate rule (strict).
pub const WAI_DECODE_STRICT: u32 = 1;
/// What `wai_neural_decode_ex` decoded.
///
/// `kind` is `WAI_DECODED_AUDIO` (`data` holds `f32` samples, `sample_rate` set),
/// `WAI_DECODED_IMAGE` (`data` holds RGB8, `width` and `height` set) or
/// `WAI_DECODED_VIDEO` (`data` holds the frames' RGB8 concatenated, `width`,
/// `height`, `n_frames` and `fps_x_1000` set). `index` is the `WAI2` rendition
/// picked (0 for a `WAI1` envelope). Free `data` with `wai_buffer_free`.
/// `wai_neural_decode_ex` writes the whole struct on every call that passes a
/// non-null `out`: on failure `kind` is 0 and `data` is empty, so `data` is always
/// safe to free.
#[repr(C)]
pub struct WaiDecoded {
pub kind: u32,
pub index: u32,
pub data: WaiBuffer,
pub width: u32,
pub height: u32,
pub sample_rate: u32,
pub n_frames: u32,
pub fps_x_1000: u32,
}
impl WaiDecoded {
/// Nothing decoded: `kind` 0, `data` empty.
fn empty() -> Self {
WaiDecoded {
kind: 0, index: 0, data: WaiBuffer::empty(),
width: 0, height: 0, sample_rate: 0, n_frames: 0, fps_x_1000: 0,
}
}
}
/// Decode a `WAI1` or `WAI2` envelope against the parameter sets in
/// `registry_json`: a NUL-terminated JSON object mapping each capability to an
/// array of paths. The first path is the registered artifact; the others are
/// further sets a pin may resolve against (a set and its successor). Every pin is
/// verified first (SPEC §3.1); a `WAI2` rendition is picked by SPEC §7.1. With
/// `flags & WAI_DECODE_STRICT`, SPEC §7's replicate rule applies too.
///
/// `*out` is written on every call with a non-null `out`, before anything else is
/// read: on failure it holds `kind` 0 and an empty `data`, so the caller may free
/// `out->data` whatever the status.
///
/// # Safety
/// `envelope` valid for `envelope_len` bytes; `registry_json` NUL-terminated;
/// `out` a writable `WaiDecoded*`, which need not be initialised.
#[unsafe(no_mangle)]
pub unsafe extern "C" fn wai_neural_decode_ex(
envelope: *const u8, envelope_len: usize,
registry_json: *const c_char, flags: u32,
out: *mut WaiDecoded,
) -> c_int {
if out.is_null() {
set_err("decode_ex: out null");
return WAI_ERR_INVALID_ARG;
}
// `write`, not assignment: the caller's struct may be uninitialised, and its
// old contents are not ours to drop.
unsafe { out.write(WaiDecoded::empty()) };
run(|| {
let bytes = unsafe { input_envelope(envelope, envelope_len) }
.ok_or((WAI_ERR_INVALID_ARG, "envelope null with len>0".to_string()))?;
let rj = unsafe { input_cstr(registry_json) }
.ok_or((WAI_ERR_INVALID_ARG, "registry_json null or not UTF-8".to_string()))?;
let map: std::collections::BTreeMap<String, Vec<String>> = serde_json::from_str(rj)
.map_err(|e| (WAI_ERR_INVALID_ARG, format!("registry_json: {e}")))?;
let mut reg = ModelRegistry::new();
for (cap, paths) in map {
let mut it = paths.into_iter();
if let Some(first) = it.next() {
reg = reg.register(cap.clone(), first);
}
for p in it {
reg = reg.hold(cap.clone(), p);
}
}
let strict = flags & WAI_DECODE_STRICT != 0;
let version = crate::container::detect_version(bytes)
.map_err(|e| (WAI_ERR_CONTAINER, e.to_string()))?;
let (index, decoded) = if version == 1 {
let env = Wai::from_bytes(bytes).map_err(|e| (WAI_ERR_CONTAINER, format!("envelope parse: {e}")))?;
let d = if strict { decode_envelope_strict(&env, &reg) } else { decode_envelope(&env, &reg) };
(0, d.map_err(decode_status)?)
} else {
let m = WaiMulti::from_bytes(bytes).map_err(|e| (WAI_ERR_CONTAINER, format!("envelope parse: {e}")))?;
let r = if strict { decode_multi_strict(&m, &reg) } else { decode_multi(&m, &reg) };
r.map_err(decode_status)?
};
let mut o = WaiDecoded { index: index as u32, ..WaiDecoded::empty() };
match decoded {
Decoded::Audio(a) => {
o.kind = WAI_DECODED_AUDIO;
o.sample_rate = a.sample_rate;
o.data = WaiBuffer::from_vec(f32_bytes(&a.samples));
}
Decoded::Image(i) => {
o.kind = WAI_DECODED_IMAGE;
o.width = i.width;
o.height = i.height;
o.data = WaiBuffer::from_vec(i.rgb);
}
Decoded::Video(v) => {
o.kind = WAI_DECODED_VIDEO;
o.width = v.width;
o.height = v.height;
o.n_frames = v.frames_rgb.len() as u32;
o.fps_x_1000 = (v.fps * 1000.0).round() as u32;
o.data = WaiBuffer::from_vec(v.frames_rgb.concat());
}
}
unsafe { out.write(o) };
Ok(())
})
}

35
wai/wai-rs/src/lib.rs
View file

@ -22,8 +22,13 @@
pub mod codecs;
pub mod container;
#[cfg(feature = "codecs")]
// Parameter-set pins (SPEC §3.1): the listing form, and verifying a pin against
// the sets a sink holds. Pure Rust over bytes, so it is always on.
pub mod pin;
#[cfg(feature = "ffi")]
pub mod ffi;
#[cfg(feature = "codecs")]
pub mod ffi_codecs;
// The integer-exact synthesis decoder lives at the crate root behind the
// light `neural_int` feature: pure i64 arithmetic, no ONNX runtime, so it
@ -161,10 +166,10 @@ pub mod int_splat;
#[cfg(feature = "neural")]
pub mod neural;
// The neural C FFI wraps the codec FFI's shared buffer/error types, so it is
// available only when the codec chain is too. The Rust neural decoders (above)
// need neither — they build and are tested without the C codec chain.
#[cfg(all(feature = "neural", feature = "codecs"))]
// The neural C FFI uses the `ffi` feature's shared buffer and error types, so it
// builds with `neural` and `ffi`; it needs no codec library. The Rust neural
// decoders (above) need neither the C ABI nor the codec chain.
#[cfg(all(feature = "neural", feature = "ffi"))]
pub mod ffi_neural;
// Shared deterministic ONNX session loader, used by both ort-backed paths.
@ -622,19 +627,13 @@ mod integration_tests {
}
}
let payload = codecs::image::jxl_encode(&rgb, h, w, None).unwrap();
let manifest = Manifest {
wai: "1.0".into(),
media: "image".into(),
intent: "replicate".into(),
model_requirement: ModelRequirement {
capability: CAP_IMAGE_JXL.into(),
fallback: Some(CAP_IMAGE_PNG.into()),
min_sim: None,
sim_metric: None,
},
conditioning: Conditioning { kind: "jxl".into(), prior: None },
target: serde_json::json!({"h": h, "w": w}),
};
let manifest = Manifest::new(
"image",
"replicate",
ModelRequirement::new(CAP_IMAGE_JXL).with_fallback(CAP_IMAGE_PNG),
Conditioning::new("jxl"),
)
.with_target(serde_json::json!({"h": h, "w": w}));
let wai = Wai::new(manifest, payload);
let bytes = wai.to_bytes().unwrap();
let recovered = Wai::from_bytes(&bytes).unwrap();

7
wai/wai-rs/src/neural/audio.rs
View file

@ -10,12 +10,11 @@
//! audio_scales float [1]
//! output: audio_values float [1, 1, samples]
use std::path::Path;
use ndarray::{Array1, Array4, IxDyn};
use ort::value::Value;
use super::{runtime::load_session, DecodeError};
use super::{params::ParamSource, DecodeError};
/// Decoded audio: float samples in [-1, 1], mono.
#[derive(Debug, Clone)]
@ -24,7 +23,7 @@ pub struct DecodedAudio {
pub sample_rate: u32,
}
pub(crate) fn decode(payload: &[u8], model_path: &Path, sample_rate: u32)
pub(crate) fn decode(payload: &[u8], params: &ParamSource, sample_rate: u32)
-> Result<DecodedAudio, DecodeError>
{
if payload.len() < 12 {
@ -52,7 +51,7 @@ pub(crate) fn decode(payload: &[u8], model_path: &Path, sample_rate: u32)
}
let scales = Array1::<f32>::from_elem(1, 1.0_f32);
let mut sess = load_session(model_path)?;
let mut sess = params.session_model("decoder.onnx")?;
let codes_v = Value::from_array(codes)
.map_err(|e| DecodeError::Ort(format!("codes tensor: {e}")))?;
let scales_v = Value::from_array(scales)

17
wai/wai-rs/src/neural/context.rs
View file

@ -26,13 +26,12 @@
//! Sink files: hyper_synthesis.onnx, context.onnx, synthesis.onnx,
//! hyper_cdfs_q<q>.json, gauss_cdfs_q<q>.json.
use std::path::Path;
use ndarray::Array4;
use super::hyperprior::{build_index, read_json, run_onnx, sibling, FactorizedCdf, GaussCdf};
use super::hyperprior::{build_index, read_json, run_onnx, FactorizedCdf, GaussCdf};
use super::image::DecodedImage;
use super::{rans::RansDecoder, DecodeError};
use super::{params::ParamSource, rans::RansDecoder, DecodeError};
const HEADER: usize = 24;
@ -70,7 +69,7 @@ fn decode_pass(
}
}
pub(crate) fn decode(payload: &[u8], model_path: &Path) -> Result<DecodedImage, DecodeError> {
pub(crate) fn decode(payload: &[u8], params: &ParamSource) -> Result<DecodedImage, DecodeError> {
if payload.len() < HEADER {
return Err(DecodeError::InvalidPayload(format!(
"context payload {} B < {HEADER} B header", payload.len())));
@ -90,7 +89,7 @@ pub(crate) fn decode(payload: &[u8], model_path: &Path) -> Result<DecodedImage,
let y_bits = &payload[HEADER + lz..HEADER + lz + ly];
// 1. z (factorized).
let hcdf: FactorizedCdf = read_json(model_path, &format!("hyper_cdfs_q{q}"))?;
let hcdf: FactorizedCdf = read_json(params, &format!("hyper_cdfs_q{q}"))?;
if hcdf.channels != cz {
return Err(DecodeError::InvalidPayload(format!("z Cz={cz} != cdf {}", hcdf.channels)));
}
@ -109,7 +108,7 @@ pub(crate) fn decode(payload: &[u8], model_path: &Path) -> Result<DecodedImage,
}
// 2. base params from the hyperprior.
let (base, bd) = run_onnx(&sibling(model_path, "hyper_synthesis.onnx"), z, 2 * cy)?;
let (base, bd) = run_onnx(params, "hyper_synthesis.onnx", z, 2 * cy)?;
if bd[1] != 2 * cy || bd[2] != sy || bd[3] != sy {
return Err(DecodeError::Ort(format!(
"hyper_synthesis [1,{},{},{}]; want [1,{},{sy},{sy}]", bd[1], bd[2], bd[3], 2 * cy)));
@ -117,7 +116,7 @@ pub(crate) fn decode(payload: &[u8], model_path: &Path) -> Result<DecodedImage,
let plane = sy * sy;
let (base_scales, base_means) = (&base[0..cy * plane], &base[cy * plane..2 * cy * plane]);
let g: GaussCdf = read_json(model_path, &format!("gauss_cdfs_q{q}"))?;
let g: GaussCdf = read_json(params, &format!("gauss_cdfs_q{q}"))?;
let mut y = vec![0f32; cy * plane];
// 3. PASS A — anchors, base params, single y stream.
@ -127,7 +126,7 @@ pub(crate) fn decode(payload: &[u8], model_path: &Path) -> Result<DecodedImage,
// 4. context network sees the anchor-only latent, predicts the rest.
let mut y_anchor = Array4::<f32>::zeros((1, cy, sy, sy));
y_anchor.as_slice_mut().unwrap().copy_from_slice(&y);
let (ctx, cd) = run_onnx(&sibling(model_path, "context.onnx"), y_anchor, 2 * cy)?;
let (ctx, cd) = run_onnx(params, "context.onnx", y_anchor, 2 * cy)?;
if cd[1] != 2 * cy || cd[2] != sy || cd[3] != sy {
return Err(DecodeError::Ort(format!(
"context [1,{},{},{}]; want [1,{},{sy},{sy}]", cd[1], cd[2], cd[3], 2 * cy)));
@ -140,7 +139,7 @@ pub(crate) fn decode(payload: &[u8], model_path: &Path) -> Result<DecodedImage,
// 6. synthesis.
let mut y4 = Array4::<f32>::zeros((1, cy, sy, sy));
y4.as_slice_mut().unwrap().copy_from_slice(&y);
let (img, id) = run_onnx(&sibling(model_path, "synthesis.onnx"), y4, 3)?;
let (img, id) = run_onnx(params, "synthesis.onnx", y4, 3)?;
if id[2] != h || id[3] != w {
return Err(DecodeError::Ort(format!(
"synthesis [1,3,{},{}]; want [1,3,{h},{w}]", id[2], id[3])));

15
wai/wai-rs/src/neural/glc.rs
View file

@ -27,13 +27,12 @@
//! context_entropy.onnx (anchor-index grid [1,1,Sy,Sy]→logits),
//! synthesis.onnx (index grid [1,1,Sy,Sy]→image), + hyper_cdfs json.
use std::path::Path;
use ndarray::Array4;
use super::hyperprior::{read_json, run_onnx, sibling, FactorizedCdf};
use super::hyperprior::{read_json, run_onnx, FactorizedCdf};
use super::image::DecodedImage;
use super::{rans::RansDecoder, DecodeError};
use super::{params::ParamSource, rans::RansDecoder, DecodeError};
const HEADER: usize = 24;
const PRECISION: u32 = 16;
@ -109,7 +108,7 @@ fn decode_index(dec: &mut RansDecoder, logits: &[f32]) -> u32 {
dec.decode_categorical(&cdf)
}
pub(crate) fn decode(payload: &[u8], model_path: &Path) -> Result<DecodedImage, DecodeError> {
pub(crate) fn decode(payload: &[u8], params: &ParamSource) -> Result<DecodedImage, DecodeError> {
if payload.len() < HEADER {
return Err(DecodeError::InvalidPayload(format!(
"glc payload {} B < {HEADER} B header", payload.len())));
@ -131,7 +130,7 @@ pub(crate) fn decode(payload: &[u8], model_path: &Path) -> Result<DecodedImage,
let plane = sy * sy;
// z (factorized).
let hcdf: FactorizedCdf = read_json(model_path, &format!("hyper_cdfs_q{q}"))?;
let hcdf: FactorizedCdf = read_json(params, &format!("hyper_cdfs_q{q}"))?;
if hcdf.channels != cz {
return Err(DecodeError::InvalidPayload(format!("z Cz={cz} != cdf {}", hcdf.channels)));
}
@ -150,7 +149,7 @@ pub(crate) fn decode(payload: &[u8], model_path: &Path) -> Result<DecodedImage,
}
// base logits[K] per position from the hyperprior.
let (base, bd) = run_onnx(&sibling(model_path, "hyper_entropy.onnx"), z, k)?;
let (base, bd) = run_onnx(params, "hyper_entropy.onnx", z, k)?;
if bd[1] != k || bd[2] != sy || bd[3] != sy {
return Err(DecodeError::Ort(format!(
"hyper_entropy [1,{},{},{}]; want [1,{k},{sy},{sy}]", bd[1], bd[2], bd[3])));
@ -184,7 +183,7 @@ pub(crate) fn decode(payload: &[u8], model_path: &Path) -> Result<DecodedImage,
}
}
}
let (ctx, cd) = run_onnx(&sibling(model_path, "context_entropy.onnx"), anchor_grid, k)?;
let (ctx, cd) = run_onnx(params, "context_entropy.onnx", anchor_grid, k)?;
if cd[1] != k || cd[2] != sy || cd[3] != sy {
return Err(DecodeError::Ort(format!(
"context_entropy [1,{},{},{}]; want [1,{k},{sy},{sy}]", cd[1], cd[2], cd[3])));
@ -205,7 +204,7 @@ pub(crate) fn decode(payload: &[u8], model_path: &Path) -> Result<DecodedImage,
// synthesis: index grid → image (codebook + generative decoder in the sink).
let mut grid = Array4::<f32>::zeros((1, 1, sy, sy));
grid.as_slice_mut().unwrap().iter_mut().zip(&idx).for_each(|(g, &v)| *g = v as f32);
let (img, id) = run_onnx(&sibling(model_path, "synthesis.onnx"), grid, 3)?;
let (img, id) = run_onnx(params, "synthesis.onnx", grid, 3)?;
if id[2] != h || id[3] != w {
return Err(DecodeError::Ort(format!(
"glc synthesis [1,3,{},{}]; want [1,3,{h},{w}]", id[2], id[3])));

45
wai/wai-rs/src/neural/hyperprior.rs
View file

@ -30,14 +30,13 @@
//! hyper_synthesis.onnx z -> [1, 2*Cy, Sy, Sy] (scales ‖ means, CompressAI order)
//! synthesis.onnx y_hat -> [1, 3, H, W]
use std::path::Path;
use ndarray::Array4;
use ort::value::Value;
use serde::Deserialize;
use super::image::DecodedImage;
use super::{rans::RansDecoder, runtime::load_session, DecodeError};
use super::{params::ParamSource, rans::RansDecoder, DecodeError};
const HEADER: usize = 24; // 4+4+4+4 + 2+2 + 1+1 + 2
@ -63,18 +62,8 @@ pub(crate) struct GaussCdf {
pub(crate) cdf: Vec<Vec<i32>>,
}
pub(crate) fn read_json<T: for<'de> Deserialize<'de>>(model_path: &Path, stem: &str) -> Result<T, DecodeError> {
let dir = model_path.parent()
.ok_or_else(|| DecodeError::InvalidPayload("model path has no parent".into()))?;
let p = dir.join(format!("{stem}.json"));
let raw = std::fs::read(&p)
.map_err(|e| DecodeError::InvalidPayload(format!("{}: {e}", p.display())))?;
serde_json::from_slice(&raw)
.map_err(|e| DecodeError::InvalidPayload(format!("{} parse: {e}", p.display())))
}
pub(crate) fn sibling(model_path: &Path, name: &str) -> std::path::PathBuf {
model_path.parent().map(|d| d.join(name)).unwrap_or_else(|| name.into())
pub(crate) fn read_json<T: for<'de> Deserialize<'de>>(params: &ParamSource, stem: &str) -> Result<T, DecodeError> {
params.json(stem)
}
/// CompressAI `GaussianConditional.build_indexes`: bucket a σ to a scale-table
@ -93,21 +82,21 @@ pub(crate) fn build_index(sigma: f32, scale_table: &[f32]) -> usize {
idx
}
pub(crate) fn run_onnx(model: &Path, input: Array4<f32>, want_c: usize) -> Result<(Vec<f32>, [usize; 4]), DecodeError> {
let mut sess = load_session(model)?;
pub(crate) fn run_onnx(params: &ParamSource, model: &str, input: Array4<f32>, want_c: usize) -> Result<(Vec<f32>, [usize; 4]), DecodeError> {
let mut sess = params.session_sibling(model)?;
let in_name = sess.inputs().first().map(|i| i.name().to_string()).unwrap_or_else(|| "input".into());
let out_name = sess.outputs().first().map(|o| o.name().to_string()).unwrap_or_else(|| "output".into());
let v = Value::from_array(input).map_err(|e| DecodeError::Ort(format!("tensor: {e}")))?;
let outputs = sess.run(ort::inputs![in_name => v])
.map_err(|e| DecodeError::Ort(format!("{}: {e}", model.display())))?;
.map_err(|e| DecodeError::Ort(format!("{}: {e}", model)))?;
let out = outputs.get(out_name.as_str())
.ok_or_else(|| DecodeError::Ort(format!("{} missing output", model.display())))?;
.ok_or_else(|| DecodeError::Ort(format!("{} missing output", model)))?;
let view = out.try_extract_array::<f32>().map_err(|e| DecodeError::Ort(format!("extract: {e}")))?;
let view = view.view();
let d = view.shape();
if d.len() != 4 || (want_c != 0 && d[1] != want_c) {
return Err(DecodeError::Ort(format!(
"{} output shape {d:?}; want [1, {}, _, _]", model.display(),
"{} output shape {d:?}; want [1, {}, _, _]", model,
if want_c == 0 { "C".into() } else { want_c.to_string() })));
}
Ok((view.as_slice().expect("contiguous").to_vec(), [d[0], d[1], d[2], d[3]]))
@ -120,10 +109,10 @@ pub(crate) fn run_onnx(model: &Path, input: Array4<f32>, want_c: usize) -> Resul
/// CompressAI-validated entropy path.
#[allow(clippy::too_many_arguments)]
pub(crate) fn decode_yhat(
z_bits: &[u8], y_bits: &[u8], model_path: &Path,
z_bits: &[u8], y_bits: &[u8], params: &ParamSource,
q: u32, cz: usize, cy: usize, sz: usize, sy: usize,
) -> Result<Array4<f32>, DecodeError> {
let hcdf: FactorizedCdf = read_json(model_path, &format!("hyper_cdfs_q{q}"))?;
let hcdf: FactorizedCdf = read_json(params, &format!("hyper_cdfs_q{q}"))?;
if hcdf.channels != cz {
return Err(DecodeError::InvalidPayload(format!("z Cz={cz} != cdf {}", hcdf.channels)));
}
@ -140,16 +129,16 @@ pub(crate) fn decode_yhat(
}
}
}
let (params, pd) = run_onnx(&sibling(model_path, "hyper_synthesis.onnx"), z, 2 * cy)?;
let (hyper_out, pd) = run_onnx(params, "hyper_synthesis.onnx", z, 2 * cy)?;
if pd[1] / 2 != cy || pd[2] != sy || pd[3] != sy {
return Err(DecodeError::Ort(format!(
"hyper_synthesis gave [1,{},{},{}]; want means‖scales for [1,{cy},{sy},{sy}]",
pd[1], pd[2], pd[3])));
}
let plane = sy * sy;
let scales = &params[0..cy * plane];
let means = &params[cy * plane..2 * cy * plane];
let gcdf: GaussCdf = read_json(model_path, &format!("gauss_cdfs_q{q}"))?;
let scales = &hyper_out[0..cy * plane];
let means = &hyper_out[cy * plane..2 * cy * plane];
let gcdf: GaussCdf = read_json(params, &format!("gauss_cdfs_q{q}"))?;
let mut y = Array4::<f32>::zeros((1, cy, sy, sy));
{
let ys = y.as_slice_mut().unwrap();
@ -163,7 +152,7 @@ pub(crate) fn decode_yhat(
Ok(y)
}
pub(crate) fn decode(payload: &[u8], model_path: &Path) -> Result<DecodedImage, DecodeError> {
pub(crate) fn decode(payload: &[u8], params: &ParamSource) -> Result<DecodedImage, DecodeError> {
if payload.len() < HEADER {
return Err(DecodeError::InvalidPayload(format!(
"payload too short ({} B); need ≥ {HEADER} B header", payload.len())));
@ -183,10 +172,10 @@ pub(crate) fn decode(payload: &[u8], model_path: &Path) -> Result<DecodedImage,
let y_bits = &payload[HEADER + lz..HEADER + lz + ly];
// 1-3. Factorized z → h_s → Gaussian-conditional y (shared with video).
let y = decode_yhat(z_bits, y_bits, model_path, q, cz, cy, sz, sy)?;
let y = decode_yhat(z_bits, y_bits, params, q, cz, cy, sz, sy)?;
// 4. g_s(y_hat) -> image.
let (img, id) = run_onnx(&sibling(model_path, "synthesis.onnx"), y, 3)?;
let (img, id) = run_onnx(params, "synthesis.onnx", y, 3)?;
if id[2] != h || id[3] != w {
return Err(DecodeError::Ort(format!(
"synthesis gave [1,3,{},{}]; want [1,3,{h},{w}]", id[2], id[3])));

106
wai/wai-rs/src/neural/image.rs
View file

@ -11,15 +11,14 @@
//! 3. Add per-channel medians → float32 [1, C, S, S] latents.
//! 4. Run g_s through ONNX → float32 [1, 3, H, W] in [0, 1].
use std::collections::HashMap;
use std::path::Path;
use std::sync::{Mutex, OnceLock};
use std::collections::VecDeque;
use std::sync::{Arc, Mutex, OnceLock};
use ndarray::Array4;
use ort::value::Value;
use serde::Deserialize;
use super::{rans::RansDecoder, runtime::load_session, DecodeError};
use super::{rans::RansDecoder, params::ParamSource, DecodeError};
/// Decoded image: RGB row-major u8, H × W × 3.
#[derive(Debug, Clone)]
@ -38,30 +37,59 @@ struct CdfBlob {
cdf: Vec<Vec<i32>>,
}
fn cdf_cache() -> &'static Mutex<HashMap<String, &'static CdfBlob>> {
static CACHE: OnceLock<Mutex<HashMap<String, &'static CdfBlob>>> = OnceLock::new();
CACHE.get_or_init(|| Mutex::new(HashMap::new()))
/// How many parsed CDF tables the process keeps. A sink decodes against a few
/// parameter sets at a time; the bound keeps a long-running sink that meets many
/// distinct tables (updated sets, unpinned directories that change) from growing
/// without limit.
const CDF_CACHE_CAP: usize = 8;
/// Parsed CDF tables by the SHA-256 of the bytes they were parsed from, oldest
/// first; at most [`CDF_CACHE_CAP`] entries.
type CdfCache = VecDeque<(String, Arc<CdfBlob>)>;
fn cdf_cache() -> &'static Mutex<CdfCache> {
static CACHE: OnceLock<Mutex<CdfCache>> = OnceLock::new();
CACHE.get_or_init(|| Mutex::new(VecDeque::new()))
}
fn load_cdfs_for(model_path: &Path, quality: u32) -> Result<&'static CdfBlob, DecodeError> {
let dir = model_path.parent()
.ok_or_else(|| DecodeError::InvalidPayload("model path has no parent".into()))?;
let cdf_path = dir.join(format!("cdfs_q{quality}.json"));
let key = cdf_path.to_string_lossy().into_owned();
{
let cache = cdf_cache().lock().unwrap();
if let Some(b) = cache.get(&key) { return Ok(*b); }
/// The entry for `key`, moved to the newest end, or `None`.
fn cache_get(cache: &mut CdfCache, key: &str) -> Option<Arc<CdfBlob>> {
let at = cache.iter().position(|(k, _)| k == key)?;
let hit = cache.remove(at)?;
let blob = Arc::clone(&hit.1);
cache.push_back(hit);
Some(blob)
}
/// Insert `blob` under `key`, evicting the least recently used entries beyond
/// [`CDF_CACHE_CAP`]. A decode still holding an evicted table keeps its `Arc`.
fn cache_put(cache: &mut CdfCache, key: String, blob: Arc<CdfBlob>) {
if cache_get(cache, &key).is_some() {
return;
}
cache.push_back((key, blob));
while cache.len() > CDF_CACHE_CAP {
cache.pop_front();
}
let raw = std::fs::read(&cdf_path)
.map_err(|e| DecodeError::InvalidPayload(format!("cdf file {key}: {e}")))?;
let blob: CdfBlob = serde_json::from_slice(&raw)
.map_err(|e| DecodeError::InvalidPayload(format!("cdf parse {key}: {e}")))?;
let leaked: &'static CdfBlob = Box::leak(Box::new(blob));
cdf_cache().lock().unwrap().insert(key, leaked);
Ok(leaked)
}
pub(crate) fn decode(payload: &[u8], model_path: &Path) -> Result<DecodedImage, DecodeError> {
/// The CDF tables for `quality`, parsed once per distinct content: the cache is
/// keyed by the SHA-256 of the bytes read, so a changed file, or another set's
/// tables, is never answered from a stale entry.
fn load_cdfs_for(params: &ParamSource, quality: u32) -> Result<Arc<CdfBlob>, DecodeError> {
let name = format!("cdfs_q{quality}.json");
let raw = params.sibling(&name)?;
let key = crate::pin::sha256_hex(&raw);
if let Some(b) = cache_get(&mut cdf_cache().lock().unwrap(), &key) {
return Ok(b);
}
let blob: Arc<CdfBlob> = Arc::new(serde_json::from_slice(&raw)
.map_err(|e| DecodeError::InvalidPayload(format!("cdf parse {name}: {e}")))?);
cache_put(&mut cdf_cache().lock().unwrap(), key, Arc::clone(&blob));
Ok(blob)
}
pub(crate) fn decode(payload: &[u8], params: &ParamSource) -> Result<DecodedImage, DecodeError> {
if payload.len() < 18 {
return Err(DecodeError::InvalidPayload(
format!("payload too short ({} B); need ≥ 18 B header", payload.len())));
@ -78,7 +106,7 @@ pub(crate) fn decode(payload: &[u8], model_path: &Path) -> Result<DecodedImage,
}
let rans_bytes = &payload[18..18 + l];
let cdfs = load_cdfs_for(model_path, q)?;
let cdfs = load_cdfs_for(params, q)?;
if cdfs.channels != c {
return Err(DecodeError::InvalidPayload(format!(
"envelope C={c} != cdf channels={}", cdfs.channels)));
@ -103,7 +131,7 @@ pub(crate) fn decode(payload: &[u8], model_path: &Path) -> Result<DecodedImage,
}
}
let mut sess = load_session(model_path)?;
let mut sess = params.session_model("decoder.onnx")?;
let input_name = sess.inputs().first().map(|i| i.name().to_string())
.unwrap_or_else(|| "latents".into());
let output_name = sess.outputs().first().map(|o| o.name().to_string())
@ -135,3 +163,31 @@ pub(crate) fn decode(payload: &[u8], model_path: &Path) -> Result<DecodedImage,
}
Ok(DecodedImage { rgb, width: w as u32, height: h as u32 })
}
#[cfg(test)]
mod tests {
use super::*;
fn blob(channels: usize) -> Arc<CdfBlob> {
Arc::new(CdfBlob { channels, cdf_length: vec![], offset: vec![], medians: vec![], cdf: vec![] })
}
/// The CDF cache holds at most CDF_CACHE_CAP tables, evicting the least
/// recently used; a table a decode still holds outlives its eviction.
#[test]
fn cdf_cache_is_bounded() {
let mut c = CdfCache::new();
let first = blob(0);
cache_put(&mut c, "k0".into(), Arc::clone(&first));
for i in 1..3 * CDF_CACHE_CAP {
cache_put(&mut c, format!("k{i}"), blob(i));
// k1 is read on every round, so it stays.
assert!(i < 2 || cache_get(&mut c, "k1").is_some());
}
assert_eq!(c.len(), CDF_CACHE_CAP);
assert!(cache_get(&mut c, "k0").is_none(), "the oldest unused table is evicted");
assert_eq!(cache_get(&mut c, "k1").unwrap().channels, 1);
assert_eq!(first.channels, 0, "an evicted table a decode holds is still valid");
assert_eq!(Arc::strong_count(&first), 1);
}
}

32
wai/wai-rs/src/neural/int_mlicpp.rs
View file

@ -1,29 +1,19 @@
//! Neural-dispatch shim for `wai.neural.int_mlicpp`.
//!
//! The decoder lives at the crate root in [`crate::int_mlicpp`] behind the light
//! `neural_int` feature (no ONNX runtime). This shim loads the registered model
//! bundle and maps the result onto the neural module's types. The registered path is
//! the one-file `WMB1` bundle, so a pinned prior's SHA-256 covers every network and
//! table the decode uses; the shim checks the digest of the very bytes it then
//! parses, rather than of a separate read of the same path.
//! `neural_int` feature (no ONNX runtime). This shim reads the model bundle and maps
//! the result onto the neural module's types. The parameter set is one file, the
//! `WMB1` bundle (`model.wmb` within a set): a pinned decode parses the very bytes
//! its pin verified, so nothing is hashed or read twice.
use std::path::Path;
use super::{image::DecodedImage, params::ParamSource, DecodeError};
use crate::container::Prior;
use super::{image::DecodedImage, DecodeError};
pub(crate) fn decode(payload: &[u8], model_path: &Path, prior: Option<&Prior>) -> Result<DecodedImage, DecodeError> {
use sha2::{Digest, Sha256};
let bytes = std::fs::read(model_path)
.map_err(|e| DecodeError::InvalidPayload(format!("int_mlicpp model {}: {e}", model_path.display())))?;
if let Some(p) = prior {
let got: String = Sha256::digest(&bytes).iter().map(|b| format!("{b:02x}")).collect();
if !got.eq_ignore_ascii_case(&p.sha256) {
return Err(DecodeError::PriorMismatch(format!(
"wai.neural.int_mlicpp: model {} hashes to {got}, the manifest pins {}", model_path.display(), p.sha256)));
}
}
pub(crate) fn decode(payload: &[u8], params: &ParamSource) -> Result<DecodedImage, DecodeError> {
let bytes = params.model("model.wmb").map_err(|e| match e {
// An unreadable unpinned model stays an invalid payload, as before.
DecodeError::InvalidPayload(why) => DecodeError::InvalidPayload(format!("int_mlicpp model {why}")),
other => other,
})?;
let model = crate::int_mlicpp::IntMlicModel::from_bundle(&bytes)
.map_err(|e| DecodeError::InvalidPayload(e.to_string()))?;
let img = crate::int_mlicpp::decode(&model, payload).map_err(|e| DecodeError::InvalidPayload(e.to_string()))?;

24
wai/wai-rs/src/neural/int_synth.rs
View file

@ -3,19 +3,21 @@
//! The actual integer-exact decode lives at the crate root in
//! [`crate::int_synth`] behind the light `neural_int` feature — it needs no
//! ONNX runtime, so it stays buildable (and cross-arch-checkable) without
//! `ort`. The `neural` feature pulls `neural_int` in; this shim adapts the
//! standalone decoder's self-contained image/error types onto the neural
//! module's `DecodedImage` / `DecodeError`.
//! `ort`. This shim adapts its image and error types onto the neural module's.
//!
//! The parameter set is one file, the model specification `int_synth.json`
//! (SPEC §5). A pinned decode reads it from the bytes the pin verified; an unpinned
//! one reads `int_synth.json` beside the registered path, as before.
use std::path::Path;
use super::{image::DecodedImage, params::ParamSource, DecodeError};
use super::{image::DecodedImage, DecodeError};
pub(crate) fn decode(payload: &[u8], model_path: &Path) -> Result<DecodedImage, DecodeError> {
match crate::int_synth::decode(payload, model_path) {
pub(crate) fn decode(payload: &[u8], params: &ParamSource) -> Result<DecodedImage, DecodeError> {
let r = match params {
ParamSource::Unpinned(path) => crate::int_synth::decode(payload, path),
ParamSource::Pinned(_) => crate::int_synth::decode_from_spec(payload, &params.model("int_synth.json")?),
};
match r {
Ok(img) => Ok(DecodedImage { rgb: img.rgb, width: img.width, height: img.height }),
Err(crate::int_synth::IntSynthError::InvalidPayload(e)) => {
Err(DecodeError::InvalidPayload(e))
}
Err(crate::int_synth::IntSynthError::InvalidPayload(e)) => Err(DecodeError::InvalidPayload(e)),
}
}

56
wai/wai-rs/src/neural/jpegai.rs
View file

@ -24,12 +24,13 @@
//! header conditions), so both `wai.image.jpegai` paths accept the same codestreams.
//! This path runs in float through ONNX Runtime and is not decode-equivalent.
use std::path::{Path, PathBuf};
use std::path::Path;
use ort::session::{builder::GraphOptimizationLevel, Session};
use ort::session::Session;
use ort::value::Value;
use crate::neural::image::DecodedImage;
use crate::neural::{params::ParamSource, DecodeError};
use crate::neural::jpegai_bitstream::{parse_ae_substream, parse_codestream};
use crate::neural::jpegai_filters::{efe_linear_plane, efe_nonlinear_plane, lef_apply, EfeLinearPlane};
use crate::neural::jpegai_header::parse_headers;
@ -38,8 +39,12 @@ use crate::neural::jpegai_mans::{build_decode_transitions, MansDecoder};
/// JPEG AI decode error.
#[derive(Debug)]
#[non_exhaustive]
pub enum JpegAiError {
Model(String),
/// A pinned decode needed a file its verified parameter set does not contain
/// (`set_incomplete`, SPEC §3.1).
Pin(crate::container::PinRefusal),
Bitstream(String),
Unsupported(String),
}
@ -47,6 +52,7 @@ impl std::fmt::Display for JpegAiError {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
match self {
Self::Model(e) => write!(f, "jpeg-ai model: {e}"),
Self::Pin(r) => write!(f, "jpeg-ai model: {}: {r}", r.code()),
Self::Bitstream(e) => write!(f, "jpeg-ai bitstream: {e}"),
Self::Unsupported(e) => write!(f, "jpeg-ai unsupported: {e}"),
}
@ -66,7 +72,7 @@ const SKIP_THR: i64 = 382; // jpeg-ai-ref skip_mode.thr_skip: code iff sigma_ski
/// The model constants a JPEG AI decode needs, loaded from a model directory.
/// All are fixed per checkpoint (image-invariant); none is per-image.
pub struct JpegAiModel {
dir: PathBuf,
source: ParamSource,
sigma_weights: serde_json::Value, // hyper_scale_decoder_int_weights.json
consts: serde_json::Value, // model_constants.json (gain_mlog, rvs, lef)
norm_pmf: Vec<Vec<i64>>, // jpegai_normative_tables.json["pmf"]
@ -80,9 +86,14 @@ pub struct JpegAiModel {
lsbs_t1: Vec<i64>,
}
fn jload(p: &Path) -> R<serde_json::Value> {
serde_json::from_slice(&std::fs::read(p).map_err(|e| m(format!("{}: {e}", p.display())))?)
.map_err(|e| m(format!("{}: {e}", p.display())))
fn param_err(e: DecodeError) -> JpegAiError {
match e {
DecodeError::PriorMismatch(r) => JpegAiError::Pin(r),
other => m(other),
}
}
fn jload(src: &ParamSource, name: &str) -> R<serde_json::Value> {
serde_json::from_slice(&src.sibling(name).map_err(param_err)?).map_err(|e| m(format!("{name}: {e}")))
}
fn i64s(v: &serde_json::Value) -> Vec<i64> {
v.as_array().map(|a| a.iter().filter_map(|n| n.as_i64()).collect()).unwrap_or_default()
@ -95,18 +106,27 @@ impl JpegAiModel {
/// Load every model constant from a model directory (the sink's installed
/// JPEG AI model bundle).
pub fn load(dir: impl AsRef<Path>) -> R<Self> {
let dir = dir.as_ref().to_path_buf();
let sigma_weights = jload(&dir.join("hyper_scale_decoder_int_weights.json"))?;
let consts = jload(&dir.join("model_constants.json"))?;
let norm = jload(&dir.join("jpegai_normative_tables.json"))?;
Self::load_from(ParamSource::Unpinned(dir.as_ref().to_path_buf()))
}
/// Load every model constant from a parameter source: a model directory
/// (unpinned), or the parameter set a pin verified, from which every file —
/// the ONNX networks included — is read as the bytes that were hashed. A file
/// the verified set does not contain is [`JpegAiError::Pin`] (`set_incomplete`).
pub fn load_from(source: ParamSource) -> R<Self> {
let src = &source;
let sigma_weights = jload(src, "hyper_scale_decoder_int_weights.json")?;
let consts = jload(src, "model_constants.json")?;
let norm = jload(src, "jpegai_normative_tables.json")?;
let norm_pmf: Vec<Vec<i64>> =
norm["pmf"].as_array().ok_or_else(|| m("normative pmf"))?.iter().map(i64s).collect();
let bounds: Vec<u8> = std::fs::read_to_string(dir.join("me-tANS/bounds.csv"))
let bounds_raw = src.sibling("me-tANS/bounds.csv").map_err(param_err)?;
let bounds: Vec<u8> = std::str::from_utf8(&bounds_raw)
.map_err(|e| m(format!("bounds.csv: {e}")))?
.lines().filter(|l| !l.trim().is_empty())
.map(|l| l.trim().parse().map_err(|e| m(format!("bounds: {e}")))).collect::<R<_>>()?;
let fac = jload(&dir.join("jpegai_factorized_cdfs.json"))?;
let fac = jload(src, "jpegai_factorized_cdfs.json")?;
let plane_cdfs = |p: &str| -> Vec<Vec<u8>> {
fac[p].as_array().map(|a| a.iter()
.map(|row| i64s(row).iter().map(|&v| v as u8).collect()).collect()).unwrap_or_default()
@ -114,25 +134,21 @@ impl JpegAiModel {
let fac_cdfs = [plane_cdfs("y"), plane_cdfs("uv")];
let fac_per_channel = fac["per_channel"].as_u64().ok_or_else(|| m("per_channel"))? as usize;
let sv = jload(&dir.join("jpegai_scaler_vec.json"))?;
let sv = jload(src, "jpegai_scaler_vec.json")?;
let scaler_vec = [f32s(&sv["y"]), f32s(&sv["uv"])];
let rvs_dq = [i64s(&consts["rvs_dq_t_flag0"]), i64s(&consts["rvs_dq_t_flag1"])];
let dq = jload(&dir.join("jpegai_dequant_consts.json"))?;
let dq = jload(src, "jpegai_dequant_consts.json")?;
let z_offsets = [i64s(&dq["z_offsets"]["y"]), i64s(&dq["z_offsets"]["uv"])];
let lsbs_t0 = i64s(&dq["lsbs_t0"]);
let lsbs_t1 = i64s(&dq["lsbs_t1"]);
Ok(Self { dir, sigma_weights, consts, norm_pmf, bounds, fac_cdfs, fac_per_channel,
Ok(Self { source, sigma_weights, consts, norm_pmf, bounds, fac_cdfs, fac_per_channel,
scaler_vec, rvs_dq, z_offsets, lsbs_t0, lsbs_t1 })
}
fn sess(&self, rel: &str) -> R<Session> {
let p = self.dir.join(rel);
Session::builder().map_err(m)?
.with_intra_threads(1).map_err(m)?.with_inter_threads(1).map_err(m)?
.with_optimization_level(GraphOptimizationLevel::Level3).map_err(m)?
.commit_from_file(&p).map_err(|e| m(format!("load {}: {e}", p.display())))
self.source.session_sibling(rel).map_err(param_err)
}
}

21
wai/wai-rs/src/neural/mlicpp.rs
View file

@ -26,13 +26,12 @@
//! u16 q, u16 K
//! Lz bytes z | Ly bytes y (slice0 anchors, slice0 non-anchors, slice1 …)
use std::path::Path;
use ndarray::Array4;
use super::hyperprior::{build_index, read_json, run_onnx, sibling, FactorizedCdf, GaussCdf};
use super::hyperprior::{build_index, read_json, run_onnx, FactorizedCdf, GaussCdf};
use super::image::DecodedImage;
use super::{rans::RansDecoder, DecodeError};
use super::{params::ParamSource, rans::RansDecoder, DecodeError};
const HEADER: usize = 26;
@ -81,7 +80,7 @@ fn entropy_input(y: &[f32], hyper: &[f32], cy: usize, sy: usize) -> Array4<f32>
t
}
pub(crate) fn decode(payload: &[u8], model_path: &Path) -> Result<DecodedImage, DecodeError> {
pub(crate) fn decode(payload: &[u8], params: &ParamSource) -> Result<DecodedImage, DecodeError> {
if payload.len() < HEADER {
return Err(DecodeError::InvalidPayload(format!(
"mlicpp payload {} B < {HEADER} B header", payload.len())));
@ -106,7 +105,7 @@ pub(crate) fn decode(payload: &[u8], model_path: &Path) -> Result<DecodedImage,
let y_bits = &payload[HEADER + lz..HEADER + lz + ly];
// 1. z (factorized).
let hcdf: FactorizedCdf = read_json(model_path, &format!("hyper_cdfs_q{q}"))?;
let hcdf: FactorizedCdf = read_json(params, &format!("hyper_cdfs_q{q}"))?;
if hcdf.channels != cz {
return Err(DecodeError::InvalidPayload(format!("z Cz={cz} != cdf {}", hcdf.channels)));
}
@ -125,14 +124,14 @@ pub(crate) fn decode(payload: &[u8], model_path: &Path) -> Result<DecodedImage,
}
// 2. hyperprior params (the channel-independent base reference).
let (hyper, hd) = run_onnx(&sibling(model_path, "hyper_synthesis.onnx"), z, 2 * cy)?;
let (hyper, hd) = run_onnx(params, "hyper_synthesis.onnx", z, 2 * cy)?;
if hd[1] != 2 * cy || hd[2] != sy || hd[3] != sy {
return Err(DecodeError::Ort(format!(
"hyper_synthesis [1,{},{},{}]; want [1,{},{sy},{sy}]", hd[1], hd[2], hd[3], 2 * cy)));
}
let plane = sy * sy;
let g: GaussCdf = read_json(model_path, &format!("gauss_cdfs_q{q}"))?;
let entropy = sibling(model_path, "entropy.onnx");
let g: GaussCdf = read_json(params, &format!("gauss_cdfs_q{q}"))?;
let entropy = "entropy.onnx";
// 3. channel-autoregressive slices × checkerboard, one y rANS stream.
let mut y = vec![0f32; cy * plane];
@ -140,12 +139,12 @@ pub(crate) fn decode(payload: &[u8], model_path: &Path) -> Result<DecodedImage,
for k_i in 0..k {
let (c0, c1) = (k_i * slice_w, (k_i + 1) * slice_w);
for anchor in [true, false] {
let (params, pd) = run_onnx(&entropy, entropy_input(&y, &hyper, cy, sy), 2 * cy)?;
let (ent_out, pd) = run_onnx(params, entropy, entropy_input(&y, &hyper, cy, sy), 2 * cy)?;
if pd[1] != 2 * cy {
return Err(DecodeError::Ort(format!(
"entropy out ch {} != 2*Cy={}", pd[1], 2 * cy)));
}
let (scales, means) = (&params[0..cy * plane], &params[cy * plane..2 * cy * plane]);
let (scales, means) = (&ent_out[0..cy * plane], &ent_out[cy * plane..2 * cy * plane]);
decode_slice_pass(&mut dy, &mut y, c0, c1, sy, scales, means, &g, anchor);
}
}
@ -153,7 +152,7 @@ pub(crate) fn decode(payload: &[u8], model_path: &Path) -> Result<DecodedImage,
// 4. synthesis.
let mut y4 = Array4::<f32>::zeros((1, cy, sy, sy));
y4.as_slice_mut().unwrap().copy_from_slice(&y);
let (img, id) = run_onnx(&sibling(model_path, "synthesis.onnx"), y4, 3)?;
let (img, id) = run_onnx(params, "synthesis.onnx", y4, 3)?;
if id[2] != h || id[3] != w {
return Err(DecodeError::Ort(format!(
"synthesis [1,3,{},{}]; want [1,3,{h},{w}]", id[2], id[3])));

561
wai/wai-rs/src/neural/mod.rs
View file

@ -18,9 +18,10 @@
use std::collections::HashMap;
use std::path::{Path, PathBuf};
use crate::container::{Prior, Wai};
use crate::container::{Inert, PinRefusal, Prior, Skip, Wai, WaiMulti};
mod runtime;
mod params;
mod onnx_external;
mod audio;
mod snac;
mod image;
@ -42,94 +43,122 @@ pub(crate) use crate::rans;
pub use audio::DecodedAudio;
pub use image::DecodedImage;
pub use params::ParamSource;
pub use video::DecodedVideo;
/// Maps `wai.neural.<name>` capability strings to ONNX decoder paths.
/// Maps capability strings to the parameter sets this sink holds.
///
/// Build it once at startup; the runtime caches sessions internally.
/// The browser equivalent is the `NEURAL_MODELS` map in
/// `wai-web/demo/index.html`.
/// `paths` is the registered artifact of each capability: an ONNX decoder, a model
/// bundle, or a model directory. `held` adds further artifacts a capability's pins
/// may resolve against, so a sink can hold several parameter sets for one
/// capability (a set and its successor, SPEC §3.1). For a pin the registry offers,
/// for each registered and held path:
///
/// - form-less: the file itself — for `wai.neural.int_synth`, the `int_synth.json`
/// beside it, the one file its decode reads;
/// - `wai.prior.set`: every `*.sha256` file directly inside the path's directory
/// (the path itself if it is a directory), each a listing rooted there.
///
/// The browser equivalent is the `NEURAL_MODELS` map in `wai-web/demo/index.html`.
#[derive(Default, Debug, Clone)]
pub struct ModelRegistry {
pub paths: HashMap<String, PathBuf>,
pub held: HashMap<String, Vec<PathBuf>>,
}
impl ModelRegistry {
pub fn new() -> Self { Self::default() }
/// Register `capability`'s artifact: the sink advertises the capability, and an
/// unpinned decode reads this path.
pub fn register(mut self, capability: impl Into<String>, path: impl Into<PathBuf>) -> Self {
self.paths.insert(capability.into(), path.into());
self
}
/// Hold a further artifact a pin over `capability` may resolve against.
pub fn hold(mut self, capability: impl Into<String>, path: impl Into<PathBuf>) -> Self {
self.held.entry(capability.into()).or_default().push(path.into());
self
}
pub fn get(&self, capability: &str) -> Option<&PathBuf> {
self.paths.get(capability)
}
/// Resolve the model path for `capability`, verifying it against a pinned
/// `prior` when the envelope carries one.
///
/// This is the difference between a floor codec and a neural one. A floor
/// capability is standardized by its reference library, so the name is the
/// contract. A neural capability is only well-defined against a specific
/// prior — so when the manifest pins one, the decode proceeds ONLY against
/// the exact model whose bytes hash to `prior.sha256`. Any other checkpoint
/// is refused with [`DecodeError::PriorMismatch`] rather than silently
/// reconstructing a different image. With no pinned prior this is identical
/// to [`get`](Self::get) (unverified) — correct for floor capabilities,
/// and the honest meaning of an unpinned neural capability: perceptual,
/// not normative.
pub fn resolve_checked(
&self,
capability: &str,
prior: Option<&Prior>,
) -> Result<&PathBuf, DecodeError> {
let path = self.paths.get(capability)
.ok_or_else(|| DecodeError::ModelNotRegistered(capability.to_string()))?;
if let Some(p) = prior {
// A declared form this sink cannot read is refused before anything is
// hashed: resolving it against the registered artifact would mean
// reading bytes in some other form than the one the pin names.
p.check_form(READABLE_PRIOR_FORMS)
.map_err(|e| DecodeError::PriorMismatch(format!("{capability}: {e}")))?;
let got = file_sha256_hex(path)?;
if !got.eq_ignore_ascii_case(&p.sha256) {
return Err(DecodeError::PriorMismatch(format!(
"{capability}: registered model {} hashes to {got}, but the \
manifest pins prior {} (sha256 {})",
path.display(), p.id, p.sha256)));
/// Whether this sink advertises `capability` (it is registered).
pub fn advertises(&self, capability: &str) -> bool {
self.paths.contains_key(capability)
}
/// The artifacts a pin over `capability` is checked against (see the type's doc).
fn held_artifacts(&self, capability: &str, set_form: bool) -> Vec<crate::pin::Held> {
use crate::pin::Held;
let paths = self.paths.get(capability).into_iter().chain(self.held.get(capability).into_iter().flatten());
let mut out = Vec::new();
for p in paths {
if set_form {
let dir = if p.is_dir() { p.clone() } else { p.parent().map(Path::to_path_buf).unwrap_or_default() };
let mut listings: Vec<PathBuf> = std::fs::read_dir(&dir).into_iter().flatten().flatten()
.map(|e| e.path())
.filter(|f| f.is_file() && f.extension().is_some_and(|x| x == "sha256"))
.collect();
listings.sort();
out.extend(listings.into_iter().map(Held::Listing));
} else if capability == crate::codecs::CAP_NEURAL_INT_SYNTH
&& p.file_name().is_some_and(|n| n != "int_synth.json")
{
out.push(Held::File(p.parent().map(|d| d.join("int_synth.json")).unwrap_or_default()));
} else if p.is_file() {
out.push(Held::File(p.clone()));
}
}
Ok(path)
out
}
/// Verify `prior` for `capability` against the parameter sets this sink holds
/// (SPEC §3.1), returning the verified bytes.
pub fn verify(&self, capability: &str, prior: &Prior) -> Result<crate::pin::Verified, PinRefusal> {
let set_form = prior.form() == Some(crate::codecs::CAP_PRIOR_SET);
crate::pin::verify(capability, prior, &self.held_artifacts(capability, set_form), READABLE_PRIOR_FORMS)
}
/// The parameters a decode through `capability` reads: verified against `prior`
/// when a pin applies to the capability ([`crate::container::pin_applies_to`]),
/// otherwise the registered artifact, unverified — the honest meaning of an
/// unpinned decode.
pub fn resolve_verified(&self, capability: &str, prior: Option<&Prior>) -> Result<ParamSource, DecodeError> {
let path = self.paths.get(capability)
.ok_or_else(|| DecodeError::ModelNotRegistered(capability.to_string()))?;
match prior {
Some(p) if crate::container::pin_applies_to(capability) => {
self.verify(capability, p).map(ParamSource::Pinned).map_err(DecodeError::PriorMismatch)
}
_ => Ok(ParamSource::Unpinned(path.clone())),
}
}
/// Verify the pin and return the registered path.
#[deprecated(note = "returns a path whose bytes can change after verification; use `resolve_verified`, \
which hands the decoder the bytes it verified")]
pub fn resolve_checked(&self, capability: &str, prior: Option<&Prior>) -> Result<&PathBuf, DecodeError> {
self.resolve_verified(capability, prior)?;
Ok(&self.paths[capability])
}
}
/// The declared prior forms (`Prior::form`) this sink can read. None: the
/// registry loads each model artifact as registered and contains no reader for a
/// coded form such as `wai.prior.nnc`, so a prior that declares one is refused.
pub const READABLE_PRIOR_FORMS: &[&str] = &[];
/// The prior forms this sink reads: `wai.prior.set`. The registry contains no
/// reader for a coded form such as `wai.prior.nnc`, so a pin that declares one is
/// refused (`form_unreadable`).
pub const READABLE_PRIOR_FORMS: &[&str] = crate::pin::READABLE_FORMS;
/// Lowercase-hex SHA-256 of a model artifact — compute the value to pin in a
/// neural capability's [`Prior::sha256`](crate::container::Prior). Minting a
/// neural envelope and verifying one at decode time use the same hash, so the
/// pin a publisher writes is exactly what the sink recomputes.
/// Lowercase-hex SHA-256 of a one-file parameter set — the value to pin, form-less,
/// in a [`Prior::sha256`](crate::container::Prior). For a set of several files, pin
/// the SHA-256 of its listing ([`crate::pin::listing_of_dir`]) in the form
/// `wai.prior.set`.
pub fn model_sha256(path: &Path) -> Result<String, DecodeError> {
file_sha256_hex(path)
}
/// Lowercase-hex SHA-256 of a file's bytes — the prior identity check.
fn file_sha256_hex(path: &Path) -> Result<String, DecodeError> {
use sha2::{Digest, Sha256};
let bytes = std::fs::read(path).map_err(|e| {
DecodeError::PriorMismatch(format!("cannot read model {}: {e}", path.display()))
})?;
let digest = Sha256::digest(&bytes);
let mut hex = String::with_capacity(64);
for byte in digest {
use std::fmt::Write;
let _ = write!(hex, "{byte:02x}");
}
Ok(hex)
std::fs::read(path).map(|b| crate::pin::sha256_hex(&b))
.map_err(|e| DecodeError::InvalidPayload(format!("cannot read {}: {e}", path.display())))
}
/// Output of a neural-capability decode. The variant is determined by
@ -143,18 +172,22 @@ pub enum Decoded {
}
#[derive(Debug)]
#[non_exhaustive]
pub enum DecodeError {
UnknownCapability(String),
ModelNotRegistered(String),
/// The registered model does not hash to the prior the manifest pins —
/// decoding would reconstruct against the wrong model. Refused loudly.
PriorMismatch(String),
/// The envelope claims `intent = replicate`, but its capability's registry
/// class does not guarantee decode-equivalence (a float or entropy-only neural
/// decode), or its prior is absent or not integer-exact — so the reconstruction
/// is not byte-identical on every conforming sink and the `replicate` claim is
/// unfounded. Raised by [`decode_envelope_strict`].
/// The pin was refused (SPEC §3.1): the sink holds no parameter set that
/// satisfies it, or a pinned decode needed a file the verified set does not
/// contain (`set_incomplete`). Decoding would reconstruct against the wrong
/// parameters, so nothing is decoded.
PriorMismatch(PinRefusal),
/// The envelope claims `intent = replicate`, but SPEC §7 does not back it for
/// the capability it decodes through. Raised by [`decode_envelope_strict`] and
/// [`decode_multi_strict`].
UnsoundReplicate(String),
/// A `WAI2` envelope is inert at this sink: for each entry, its index and why it
/// was skipped (SPEC §7.1 step 5).
NoRendition(Vec<(usize, String)>),
Ort(String),
InvalidPayload(String),
Zstd(String),
@ -164,9 +197,16 @@ impl std::fmt::Display for DecodeError {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
match self {
Self::UnknownCapability(c) => write!(f, "unknown capability: {c}"),
Self::ModelNotRegistered(c) => write!(f, "no ONNX model registered for {c}"),
Self::PriorMismatch(e) => write!(f, "pinned-prior mismatch: {e}"),
Self::ModelNotRegistered(c) => write!(f, "no model registered for {c}"),
Self::PriorMismatch(r) => write!(f, "pinned-prior refused: {}: {r}", r.code()),
Self::UnsoundReplicate(e) => write!(f, "unsound replicate claim: {e}"),
Self::NoRendition(skips) => {
write!(f, "no rendition decodes at this sink:")?;
for (i, why) in skips {
write!(f, " [{i}] {why};")?;
}
Ok(())
}
Self::Ort(e) => write!(f, "onnxruntime error: {e}"),
Self::InvalidPayload(e) => write!(f, "invalid payload: {e}"),
Self::Zstd(e) => write!(f, "zstd error: {e}"),
@ -175,112 +215,306 @@ impl std::fmt::Display for DecodeError {
}
impl std::error::Error for DecodeError {}
/// Decode a parsed WAI1 envelope using the neural runtime.
pub fn decode_envelope(env: &Wai, registry: &ModelRegistry) -> Result<Decoded, DecodeError> {
let cap = &env.manifest.model_requirement.capability;
// Verify the registered model against the manifest's pinned prior (if any)
// before decoding — a pinned neural capability decodes only against the
// exact model it names.
let path = registry.resolve_checked(cap, env.prior())?;
match cap.as_str() {
"wai.neural.encodec32" => audio::decode(&env.payload, path, 32_000).map(Decoded::Audio),
"wai.neural.dac" => audio::decode(&env.payload, path, 44_100).map(Decoded::Audio),
"wai.neural.mimi" => audio::decode(&env.payload, path, 24_000).map(Decoded::Audio),
"wai.neural.wavtokenizer" => audio::decode(&env.payload, path, 24_000).map(Decoded::Audio),
fn is_set_incomplete(e: &DecodeError) -> bool {
matches!(e, DecodeError::PriorMismatch(PinRefusal::SetIncomplete { .. }))
}
/// Decode `payload` through `capability` against `params` — the per-capability
/// dispatch, shared by the `WAI1` and `WAI2` paths.
pub fn decode_payload(capability: &str, payload: &[u8], params: &ParamSource) -> Result<Decoded, DecodeError> {
match capability {
"wai.neural.encodec32" => audio::decode(payload, params, 32_000).map(Decoded::Audio),
"wai.neural.dac" => audio::decode(payload, params, 44_100).map(Decoded::Audio),
"wai.neural.mimi" => audio::decode(payload, params, 24_000).map(Decoded::Audio),
"wai.neural.wavtokenizer" => audio::decode(payload, params, 24_000).map(Decoded::Audio),
// SNAC carries its own sample rate (24/32/44 kHz) + N hierarchical levels.
"wai.neural.snac" => snac::decode(&env.payload, path).map(Decoded::Audio),
"wai.neural.bmshj2018" => image::decode(&env.payload, path).map(Decoded::Image),
"wai.neural.mbt2018_mean" => hyperprior::decode(&env.payload, path).map(Decoded::Image),
"wai.neural.int_synth" => int_synth::decode(&env.payload, path).map(Decoded::Image),
// The MLIC++ entropy topology, integer end to end (crate::int_mlicpp). `path` is
// the one-file WMB1 model bundle; the shim re-checks a pinned prior against the
// exact bytes it decodes with.
"wai.neural.int_mlicpp" => int_mlicpp::decode(&env.payload, path, env.prior()).map(Decoded::Image),
"wai.neural.elic" => context::decode(&env.payload, path).map(Decoded::Image),
"wai.neural.mlicpp" => mlicpp::decode(&env.payload, path).map(Decoded::Image),
// MLICv2 is the SAME multi-reference
// channel-autoregressive decode; its gains live in the sink nets.
"wai.neural.mlicv2" => mlicpp::decode(&env.payload, path).map(Decoded::Image),
"wai.neural.snac" => snac::decode(payload, params).map(Decoded::Audio),
"wai.neural.bmshj2018" => image::decode(payload, params).map(Decoded::Image),
"wai.neural.mbt2018_mean" => hyperprior::decode(payload, params).map(Decoded::Image),
"wai.neural.int_synth" => int_synth::decode(payload, params).map(Decoded::Image),
// The MLIC++ entropy topology, integer end to end (crate::int_mlicpp), over
// its one-file WMB1 bundle.
"wai.neural.int_mlicpp" => int_mlicpp::decode(payload, params).map(Decoded::Image),
"wai.neural.elic" => context::decode(payload, params).map(Decoded::Image),
"wai.neural.mlicpp" => mlicpp::decode(payload, params).map(Decoded::Image),
// MLICv2 is the SAME multi-reference channel-autoregressive decode; its gains
// live in the sink nets.
"wai.neural.mlicv2" => mlicpp::decode(payload, params).map(Decoded::Image),
// CMIC's entropy model is SCCTX (space-channel context = channel slices
// + checkerboard) — the mlicpp orchestration. Its Mamba is in the
// sink-supplied transform, not the entropy decode.
"wai.neural.cmic" => mlicpp::decode(&env.payload, path).map(Decoded::Image),
"wai.neural.cmic" => mlicpp::decode(payload, params).map(Decoded::Image),
// GLC: categorical/VQ entropy (not Gaussian) — codebook indices via a
// per-position softmax → quantized CDF → categorical rANS.
"wai.neural.glc" => glc::decode(&env.payload, path).map(Decoded::Image),
"wai.neural.video_bmshj2018" => video::decode(&env.payload, path).map(Decoded::Video),
"wai.neural.glc" => glc::decode(payload, params).map(Decoded::Image),
"wai.neural.video_bmshj2018" => video::decode(payload, params).map(Decoded::Video),
// DCVC inter-frame video: frame loop + reference-feature propagation,
// CPU-capable via ort (a GPU is not required). dcvc_rt + dcvc_fm share the
// orchestration; their temporal nets are sink-supplied.
"wai.neural.dcvc_rt" | "wai.neural.dcvc_fm"
=> video_dcvc::decode(&env.payload, path).map(Decoded::Video),
// ISO/IEC 6048 JPEG AI: a self-sufficient native decode (codestream parse,
// factorized z, integer σ-net, the residual coding mask DERIVED from the
// bitstream, the dequant derived) over a sink-supplied model BUNDLE — here
// `path` is the model directory, not a single ONNX. The learned transforms
// (hyper-decoder, MCM value nets, synthesis, eICCI) are sink ONNX via ort.
=> video_dcvc::decode(payload, params).map(Decoded::Video),
// ISO/IEC 6048 JPEG AI over a sink-supplied model bundle (several files).
"wai.image.jpegai" => {
// Both decode paths implement one configuration (tools_0, β = 0, the
// high-operation-point synthesis, one eICCI model pair, 256×256 4:4:4);
// a codestream that asks for anything else is refused, not mis-decoded.
crate::jpegai_decode::check_codestream(&env.payload).map_err(|why| {
crate::jpegai_decode::check_codestream(payload).map_err(|why| {
DecodeError::InvalidPayload(format!("wai.image.jpegai: unsupported codestream: {why}"))
})?;
// Prefer the exact decoder when the sink ships the integer weight bundle
// (`<dir>/weights/*.json` + `<dir>/norm.json`) — that path pins one
// Prefer the exact decoder when the parameters include the integer weight
// bundle (`weights/*.json` + `norm.json`) — that path pins one
// reconstruction, reproducible across machines under the conditions in
// `extensions/determinism-tiers.md` §5. Otherwise (no bundle, or a
// non-default floating-point environment) fall back to the ORT float
// decode. Both paths share the integer entropy decode, which is all the
// capability guarantees (entropy-consistency, `NeuralEntropyExact`). The
// result does not say which path ran.
if let Some(img) = try_jpegai_exact(&env.payload, path) {
// decode, which needs files a set holding only the exact bundle does not
// contain: a pinned decode is then refused (`set_incomplete`) rather than
// run on unverified files. Both paths share the integer entropy decode,
// which is all the capability guarantees (entropy-consistency,
// `NeuralEntropyExact`). The result does not say which path ran.
if let Some(img) = try_jpegai_exact(payload, params) {
Ok(Decoded::Image(img))
} else {
let model = jpegai::JpegAiModel::load(path)
.map_err(|e| DecodeError::InvalidPayload(e.to_string()))?;
jpegai::decode(&env.payload, &model)
.map(Decoded::Image)
.map_err(|e| DecodeError::InvalidPayload(e.to_string()))
let to_err = |e: jpegai::JpegAiError| match e {
jpegai::JpegAiError::Pin(r) => DecodeError::PriorMismatch(r),
other => DecodeError::InvalidPayload(other.to_string()),
};
let model = jpegai::JpegAiModel::load_from(params.clone()).map_err(to_err)?;
jpegai::decode(payload, &model).map(Decoded::Image).map_err(to_err)
}
}
other => Err(DecodeError::UnknownCapability(other.to_string())),
}
}
/// Load the byte-exact JPEG-AI integer weight bundle from a model directory and run the
/// deterministic [`crate::jpegai_decode::decode`]. Returns `None` — so the caller falls
/// back to the ORT decode — if the bundle's directory or `norm.json` is absent, a file
/// does not parse as JSON, or `decode` refuses (codestream out of scope, a non-default
/// SPEC §4 for a parsed `WAI1` envelope at a sink whose parameter sets `registry`
/// holds, through the capabilities `gate` admits. A refused pin makes its capability
/// unsupported for this envelope and dispatch moves on to the fallback; a pinned
/// decode refused with `set_incomplete` does too. When nothing decodes, the error is
/// step 6's: the reason kept for the capability (step 3) before the one kept for
/// the fallback (step 5), whether that reason is a refused pin or `set_incomplete`.
fn decode_envelope_with(
env: &Wai,
registry: &ModelRegistry,
gate: impl Fn(&str) -> Result<(), DecodeError>,
) -> Result<Decoded, DecodeError> {
let mut excluded: Vec<String> = Vec::new();
// The `set_incomplete` refusal, with the capability whose decode raised it.
let mut incomplete: Option<(String, DecodeError)> = None;
loop {
let d = env.dispatch_verified(
|c| registry.advertises(c) && !excluded.iter().any(|x| x == c),
|c, p| registry.verify(c, p),
);
let d = match d {
Ok(d) => d,
Err(Inert::PinRefused { refusal, .. }) => {
// A set_incomplete at step 3 outranks the fallback's refusal; one at
// step 5 is outranked by the capability's.
return Err(match incomplete {
Some((cap, e)) if cap == env.capability() => e,
_ => DecodeError::PriorMismatch(refusal),
});
}
Err(_) => {
return Err(incomplete.map(|(_, e)| e)
.unwrap_or_else(|| DecodeError::ModelNotRegistered(env.capability().to_string())));
}
};
gate(d.capability)?;
let params = match d.verified {
Some(v) => ParamSource::Pinned(v),
None => ParamSource::Unpinned(registry.paths[d.capability].clone()),
};
match decode_payload(d.capability, &env.payload, &params) {
Err(e) if is_set_incomplete(&e) => {
excluded.push(d.capability.to_string());
if incomplete.is_none() {
incomplete = Some((d.capability.to_string(), e));
}
}
r => return r,
}
}
}
/// Decode a parsed `WAI1` envelope (SPEC §4): the capability if the registry holds
/// it and, where a pin applies, a parameter set that satisfies the pin (SPEC §3.1);
/// else the fallback on the same terms. A pinned decode reads only the bytes it
/// verified. Returns [`DecodeError::PriorMismatch`] with the first refusal when no
/// capability's pin resolves, and [`DecodeError::ModelNotRegistered`] when the
/// registry holds neither capability.
pub fn decode_envelope(env: &Wai, registry: &ModelRegistry) -> Result<Decoded, DecodeError> {
decode_envelope_with(env, registry, |_| Ok(()))
}
/// The SPEC §7 replicate gate for one capability, as a [`DecodeError`].
fn replicate_gate(cap: &str, refusal: Option<crate::codecs::ReplicateRefusal>) -> Result<(), DecodeError> {
use crate::codecs::ReplicateRefusal;
match refusal {
None => Ok(()),
Some(ReplicateRefusal::Unregistered) => Err(DecodeError::UnknownCapability(cap.to_string())),
// The capability is integer-exact; the pin is what fails. The reason (none
// pinned, a tier other than decode-equivalence, or a form not established
// exact) comes from the library, whose unit tests run without this feature.
Some(ReplicateRefusal::Prior(why)) => Err(DecodeError::UnsoundReplicate(format!(
"intent=replicate over {cap} but {why} ({})", why.code()
))),
Some(why) => Err(DecodeError::UnsoundReplicate(format!(
"intent=replicate over {cap}: {why}. Over this capability `replicate` would \
promise the same reconstruction bytes on every conforming sink, which its \
registry class does not provide; tag this improve/perceptual, or use an \
integer-exact capability with an integer-only prior ({})", why.code()
))),
}
}
/// Decode like [`decode_envelope`], but first apply WAI's replicate-soundness rule
/// (SPEC §7, [`crate::codecs::replicate_refusal`]). Over a neural capability
/// `replicate` promises decode-equivalence — the same reconstruction bytes on every
/// conforming sink — so an envelope whose `intent` is `replicate` is decoded only when
///
/// 1. its capability's registry class is `NeuralIntegerExact` (integer end to end,
/// entropy and synthesis), and
/// 2. it pins a parameter set with a pin that declares `decode-equivalence` (by
/// `tier`, or, without one, by the legacy reading of `determinism`), in no form
/// or in one the registry lists as established exact (`wai.prior.set`).
///
/// The rule is applied to the capability the envelope decodes through (SPEC §7):
/// `model_requirement.capability`, or the fallback when dispatch reaches it, so an
/// envelope whose unsound primary this sink does not advertise still decodes
/// through a sound fallback. The capabilities this sink could decode through are
/// those [`crate::codecs::is_dispatchable`] accepts and the sink advertises; a
/// candidate, companion-only capability, record or derivation the registry holds is
/// never one of them, so it counts as not held. When the rule refuses every
/// capability this sink could decode through (or, when there is none, refuses the
/// capability), the refusal is returned before anything is resolved, so it reads
/// no file. Float
/// inference through a runtime is not byte-identical across hardware, and even a
/// fixed-order float stage depends on the build and on the process's floating-point
/// environment, which neither the envelope nor the registry can state.
pub fn decode_envelope_strict(env: &Wai, registry: &ModelRegistry) -> Result<Decoded, DecodeError> {
let cap = env.capability();
let mut candidates: Vec<&str> = [Some(cap), env.fallback()].into_iter().flatten()
.filter(|c| crate::codecs::is_dispatchable(c) && registry.advertises(c))
.collect();
if candidates.is_empty() {
candidates.push(cap);
}
if candidates.iter().all(|c| env.replicate_refusal_for(c).is_some()) {
let first = candidates[0];
return Err(replicate_gate(first, env.replicate_refusal_for(first)).unwrap_err());
}
decode_envelope_with(env, registry, |c| replicate_gate(c, env.replicate_refusal_for(c)))
}
/// SPEC §7.1 for a parsed `WAI2` envelope: pick the first selectable entry whose
/// capability the registry holds, whose payload matches its `sha256`, and whose pin
/// resolves against a held parameter set, then decode it from the verified bytes. A
/// pinned decode refused with `set_incomplete` excludes that entry and picks again.
/// Returns the picked index with the output, or [`DecodeError::NoRendition`] with a
/// reason per entry.
pub fn decode_multi(m: &WaiMulti, registry: &ModelRegistry) -> Result<(usize, Decoded), DecodeError> {
decode_multi_where(m, registry, |_| Ok(()))
}
fn decode_multi_where(
m: &WaiMulti,
registry: &ModelRegistry,
admit: impl Fn(usize) -> Result<(), Skip>,
) -> Result<(usize, Decoded), DecodeError> {
let mut incomplete: Vec<(usize, String)> = Vec::new();
loop {
let picked = m.pick_where(
|i, meta| {
admit(i)?;
// An entry excluded after `set_incomplete` is reported with that
// reason below.
if !registry.advertises(&meta.capability) || incomplete.iter().any(|(x, _)| *x == i) {
return Err(Skip::NotEligible);
}
Ok(())
},
|c, p| registry.verify(c, p),
);
match picked {
Ok(p) => {
let params = match p.verified {
Some(v) => ParamSource::Pinned(v),
None => ParamSource::Unpinned(registry.paths[&p.meta.capability].clone()),
};
match decode_payload(&p.meta.capability, p.payload, &params) {
Err(e) if is_set_incomplete(&e) => {
let DecodeError::PriorMismatch(r) = &e else { unreachable!() };
incomplete.push((p.index, format!("{}: {r}", r.code())));
}
Err(e) => return Err(e),
Ok(d) => return Ok((p.index, d)),
}
}
Err(skips) => {
let reasons = skips.into_iter()
.map(|(i, s)| match incomplete.iter().find(|(x, _)| *x == i) {
Some((_, why)) => (i, why.clone()),
None => (i, s.to_string()),
})
.collect();
return Err(DecodeError::NoRendition(reasons));
}
}
}
}
/// [`decode_multi`] under SPEC §7's replicate rule: with `intent = replicate`, only
/// the entries that rule admits are considered ([`WaiMulti::replicate_refusal`]);
/// one it leaves out is reported with its `replicate_*` reason.
/// When none is admitted the result is [`DecodeError::UnsoundReplicate`] with the
/// first selectable entry's reason.
pub fn decode_multi_strict(m: &WaiMulti, registry: &ModelRegistry) -> Result<(usize, Decoded), DecodeError> {
if m.intent() == "replicate" {
let selectable: Vec<usize> = m.manifest.renditions.iter().enumerate()
.filter(|(_, r)| r.is_selectable()).map(|(i, _)| i).collect();
if let Some(&first) = selectable.first() {
if selectable.iter().all(|&i| m.replicate_refusal(i).is_some()) {
let cap = &m.manifest.renditions[first].capability;
return Err(replicate_gate(cap, m.replicate_refusal(first)).unwrap_err());
}
}
return decode_multi_where(m, registry, |i| match m.replicate_refusal(i) {
Some(why) => Err(Skip::Replicate(why)),
None => Ok(()),
});
}
decode_multi(m, registry)
}
/// Load the byte-exact JPEG-AI integer weight bundle from the decode's parameters
/// (`norm.json` and `weights/*.json`) and run the deterministic
/// [`crate::jpegai_decode::decode`]. Returns `None` — so the caller falls back to
/// the ORT decode — if `norm.json` or a weights file is absent, a file does not parse
/// as JSON, or `decode` refuses (codestream out of scope, a non-default
/// floating-point environment, table shapes). A JSON file with missing or mistyped
/// fields panics in the loaders rather than returning `None`. The bundle is what
/// `tools/wai_jpegai_build_e2e_inputs.py` produces:
/// a `weights/` dir of per-plane / MCM / synthesis / eICCI JSON plus a `norm.json` of
/// the normative tables. Keeps `jpegai_decode` itself I/O-free; the file reads live here.
fn try_jpegai_exact(payload: &[u8], dir: &Path) -> Option<image::DecodedImage> {
/// `tools/wai_jpegai_build_e2e_inputs.py` produces. Keeps `jpegai_decode` itself
/// I/O-free; the reads live here.
fn try_jpegai_exact(payload: &[u8], params: &ParamSource) -> Option<image::DecodedImage> {
use crate::jpegai_decode::{decode, DecoderWeights, NormTables, PlaneWeights};
use crate::jpegai_icci::IcciNet;
use crate::jpegai_synthesis::{IntNet, Weights};
let wdir = dir.join("weights");
let norm_path = dir.join("norm.json");
if !wdir.is_dir() || !norm_path.exists() {
return None;
}
let jload = |p: PathBuf| -> Option<serde_json::Value> {
serde_json::from_slice(&std::fs::read(p).ok()?).ok()
let jload = |name: &str| -> Option<serde_json::Value> {
serde_json::from_slice(&params.try_sibling(name)?).ok()
};
let w = |name: &str| -> Option<Weights> { Some(Weights::from_json(&jload(wdir.join(name))?)) };
let net = |name: &str| -> Option<IntNet> { Some(IntNet::from_json(&jload(wdir.join(name))?)) };
let nv = jload("norm.json")?;
let w = |name: &str| -> Option<Weights> { Some(Weights::from_json(&jload(&format!("weights/{name}"))?)) };
let net = |name: &str| -> Option<IntNet> { Some(IntNet::from_json(&jload(&format!("weights/{name}"))?)) };
let weights = DecoderWeights {
luma: PlaneWeights { hyper_scale: net("hs_y.json")?, hyper_decoder: w("hyp_y.json")?, synthesis: w("synth_y.json")? },
chroma: PlaneWeights { hyper_scale: net("hs_uv.json")?, hyper_decoder: w("hyp_uv.json")?, synthesis: w("synth_uv.json")? },
mcm_y: [w("mcm0_y.json")?, w("mcm1_y.json")?, w("mcm2_y.json")?, w("mcm3_y.json")?],
icci: IcciNet::from_json(&jload(wdir.join("icci.json"))?),
icci: IcciNet::from_json(&jload("weights/icci.json")?),
};
let nv = jload(norm_path)?;
let i64v = |k: &str| nv[k].as_array().unwrap().iter().map(|x| x.as_i64().unwrap()).collect::<Vec<i64>>();
let u8v = |k: &str| nv[k].as_array().unwrap().iter().map(|x| x.as_u64().unwrap() as u8).collect::<Vec<u8>>();
let i32v = |k: &str| nv[k].as_array().unwrap().iter().map(|x| x.as_i64().unwrap() as i32).collect::<Vec<i32>>();
@ -300,53 +534,6 @@ fn try_jpegai_exact(payload: &[u8], dir: &Path) -> Option<image::DecodedImage> {
Some(image::DecodedImage { rgb: rgb.data, width: rgb.w as u32, height: rgb.h as u32 })
}
/// Decode like [`decode_envelope`], but first apply WAI's replicate-soundness rule
/// (SPEC §7, [`crate::codecs::replicate_refusal`]). Over a neural capability
/// `replicate` promises decode-equivalence — the same reconstruction bytes on every
/// conforming sink — so an envelope whose `intent` is `replicate` is decoded only when
///
/// 1. its capability's registry class is `NeuralIntegerExact` (integer end to end,
/// entropy and synthesis), and
/// 2. it pins a prior that declares an integer-exact contract, and whose declared
/// form, if any, is established to reconstruct identical tensors at every reader
/// ([`crate::container::Prior::form_preserves_exactness`]; none is yet).
///
/// The class is checked first and does not depend on anything the sender declares:
/// a `replicate` envelope over `wai.image.jpegai` (`NeuralEntropyExact`: integer
/// entropy decode, float later stages) or over a float codec (`NeuralFloat`) is
/// refused even when its prior declares an integer contract. Float inference through
/// a runtime is not byte-identical across hardware (the runtime picks kernels,
/// accumulation order and fused multiply-adds per device, and one diverging bit
/// corrupts a learned entropy decode); even a fixed-order float stage depends on the
/// build and on the process's floating-point environment, which neither the
/// envelope nor the registry can state.
///
/// Returns [`DecodeError::UnsoundReplicate`], with the reason, *before* touching the
/// model otherwise, and [`DecodeError::UnknownCapability`] for an unregistered
/// capability. `decode_envelope` stays lenient (it decodes regardless, honest about
/// being "exact on this build").
pub fn decode_envelope_strict(env: &Wai, registry: &ModelRegistry) -> Result<Decoded, DecodeError> {
use crate::codecs::ReplicateRefusal;
let cap = &env.manifest.model_requirement.capability;
match env.replicate_refusal() {
None => decode_envelope(env, registry),
Some(ReplicateRefusal::Unregistered) => Err(DecodeError::UnknownCapability(cap.clone())),
// The capability is integer-exact; the pinned prior is what fails. The reason
// (none pinned, a contract that is not integer-exact, or a declared form not
// established exact) comes from the library, whose unit tests run without
// this feature.
Some(ReplicateRefusal::Prior(why)) => Err(DecodeError::UnsoundReplicate(format!(
"intent=replicate over {cap} but {why}"
))),
Some(why) => Err(DecodeError::UnsoundReplicate(format!(
"intent=replicate over {cap}: {why}. Over this capability `replicate` would \
promise the same reconstruction bytes on every conforming sink, which its \
registry class does not provide; tag this improve/perceptual, or use an \
integer-exact capability with an integer-only prior"
))),
}
}
#[cfg(test)]
mod jpegai_exact_dispatch_tests {
use std::path::Path;
@ -364,7 +551,7 @@ mod jpegai_exact_dispatch_tests {
return;
};
let Ok(payload) = std::fs::read(&cs) else { eprintln!("skip: no codestream"); return };
let img = super::try_jpegai_exact(&payload, Path::new(&dir))
let img = super::try_jpegai_exact(&payload, &super::ParamSource::Unpinned(Path::new(&dir).into()))
.expect("integer bundle present → deterministic decode");
assert_eq!((img.width, img.height), (256, 256));
assert_eq!(img.rgb.len(), 256 * 256 * 3);
@ -373,6 +560,6 @@ mod jpegai_exact_dispatch_tests {
/// Absent bundle → `None`, so dispatch falls back to the ORT path unchanged.
#[test]
fn missing_bundle_falls_back() {
assert!(super::try_jpegai_exact(&[], Path::new("/nonexistent/wai/model/dir")).is_none());
assert!(super::try_jpegai_exact(&[], &super::ParamSource::Unpinned("/nonexistent/wai/model/dir".into())).is_none());
}
}

297
wai/wai-rs/src/neural/onnx_external.rs Normal file
View file

@ -0,0 +1,297 @@
//! Whether an ONNX model keeps tensor data outside its own bytes (SPEC §3.1,
//! *Decoding against a verified set*).
//!
//! A pinned decode builds its session from the bytes the pin verified. An ONNX
//! tensor can instead name another file for its data (`data_location = EXTERNAL`,
//! with the file in `external_data`'s `location` entry), and a runtime given only
//! the model's bytes resolves that file itself, outside the verified set. So before
//! a session is built from verified bytes, the model is read for such a tensor and
//! refused when it has one.
//!
//! This reads the protobuf wire format of `onnx.proto` directly and visits every
//! place a tensor can sit: the graph's initializers and sparse initializers, the
//! tensors and sparse tensors of every node attribute, the subgraphs of attributes
//! (control flow), the graphs of the training information, and the nodes of the
//! model's local functions. A field read with an unexpected wire type is skipped,
//! as a protobuf parser treats it as an unknown field. Bytes that are not a
//! well-formed protobuf message are an error, so a model this reader cannot walk is
//! never taken as free of external data.
/// Deepest nesting of messages read before the model is refused as malformed.
const MAX_DEPTH: usize = 64;
/// The `location` of the first tensor in `model`, an ONNX `ModelProto`, whose data
/// is stored outside the model (`""` when it names none); `None` when every tensor
/// keeps its data inside. `Err` when `model` is not a well-formed protobuf message.
pub(crate) fn external_tensor(model: &[u8]) -> Result<Option<String>, String> {
walk(model, Msg::Model, 0)
}
#[derive(Clone, Copy)]
enum Msg {
Model,
TrainingInfo,
Function,
Graph,
Node,
Attribute,
SparseTensor,
Tensor,
}
impl Msg {
/// The message type of field `field` of `self`, when it can hold a tensor.
fn child(self, field: u64) -> Option<Msg> {
use Msg::*;
Some(match (self, field) {
// ModelProto: graph = 7, training_info = 20, functions = 25.
(Model, 7) => Graph,
(Model, 20) => TrainingInfo,
(Model, 25) => Function,
// TrainingInfoProto: initialization = 1, algorithm = 2.
(TrainingInfo, 1 | 2) => Graph,
// FunctionProto: node = 7.
(Function, 7) => Node,
// GraphProto: node = 1, initializer = 5, sparse_initializer = 15.
(Graph, 1) => Node,
(Graph, 5) => Tensor,
(Graph, 15) => SparseTensor,
// NodeProto: attribute = 5.
(Node, 5) => Attribute,
// AttributeProto: t = 5, g = 6, tensors = 10, graphs = 11,
// sparse_tensor = 22, sparse_tensors = 23.
(Attribute, 5 | 10) => Tensor,
(Attribute, 6 | 11) => Graph,
(Attribute, 22 | 23) => SparseTensor,
// SparseTensorProto: values = 1, indices = 2.
(SparseTensor, 1 | 2) => Tensor,
_ => return None,
})
}
}
fn varint(b: &[u8], at: &mut usize) -> Result<u64, String> {
let mut v = 0u64;
for shift in (0..64).step_by(7) {
let byte = *b.get(*at).ok_or("truncated varint")?;
*at += 1;
v |= u64::from(byte & 0x7f) << shift;
if byte & 0x80 == 0 {
return Ok(v);
}
}
Err("varint longer than 10 bytes".into())
}
/// The bytes of a length-delimited field at `at`.
fn delimited<'a>(b: &'a [u8], at: &mut usize) -> Result<&'a [u8], String> {
let len = usize::try_from(varint(b, at)?).map_err(|_| "length out of range")?;
let end = at.checked_add(len).filter(|&e| e <= b.len()).ok_or("truncated field")?;
let s = &b[*at..end];
*at = end;
Ok(s)
}
fn walk(b: &[u8], msg: Msg, depth: usize) -> Result<Option<String>, String> {
if depth > MAX_DEPTH {
return Err(format!("messages nested deeper than {MAX_DEPTH}"));
}
let mut at = 0;
// For a TensorProto: whether it is external, and its `location`.
let mut external = false;
let mut location: Option<String> = None;
while at < b.len() {
let key = varint(b, &mut at)?;
let (field, wire) = (key >> 3, key & 7);
if field == 0 {
return Err("field number 0".into());
}
match wire {
0 => {
let v = varint(b, &mut at)?;
// TensorProto.data_location = 14; EXTERNAL = 1.
if matches!(msg, Msg::Tensor) && field == 14 && v == 1 {
external = true;
}
}
1 => at = at.checked_add(8).filter(|&e| e <= b.len()).ok_or("truncated fixed64")?,
5 => at = at.checked_add(4).filter(|&e| e <= b.len()).ok_or("truncated fixed32")?,
2 => {
let body = delimited(b, &mut at)?;
if matches!(msg, Msg::Tensor) && field == 13 {
// TensorProto.external_data = 13: StringStringEntryProto
// { key = 1, value = 2 }. Any entry marks the tensor external.
external = true;
if let Some(l) = location_entry(body)? {
location.get_or_insert(l);
}
} else if let Some(child) = msg.child(field) {
if let Some(found) = walk(body, child, depth + 1)? {
return Ok(Some(found));
}
}
}
_ => return Err(format!("wire type {wire} (groups are not used by onnx.proto)")),
}
}
Ok(external.then(|| location.unwrap_or_default()))
}
/// The value of a `StringStringEntryProto` whose key is `location`.
fn location_entry(b: &[u8]) -> Result<Option<String>, String> {
let mut at = 0;
let (mut key, mut value) = (None, None);
while at < b.len() {
let k = varint(b, &mut at)?;
match k & 7 {
0 => {
varint(b, &mut at)?;
}
1 => at = at.checked_add(8).filter(|&e| e <= b.len()).ok_or("truncated fixed64")?,
5 => at = at.checked_add(4).filter(|&e| e <= b.len()).ok_or("truncated fixed32")?,
2 => {
let s = String::from_utf8_lossy(delimited(b, &mut at)?).into_owned();
match k >> 3 {
1 => key = Some(s),
2 => value = Some(s),
_ => {}
}
}
w => return Err(format!("wire type {w} in an external_data entry")),
}
}
Ok((key.as_deref() == Some("location")).then(|| value.unwrap_or_default()))
}
#[cfg(test)]
pub(crate) mod tests {
use super::external_tensor;
fn varint(mut v: u64, out: &mut Vec<u8>) {
loop {
let b = (v & 0x7f) as u8;
v >>= 7;
if v == 0 {
out.push(b);
return;
}
out.push(b | 0x80);
}
}
/// A length-delimited field.
pub(crate) fn field(n: u64, body: &[u8]) -> Vec<u8> {
let mut out = Vec::new();
varint(n << 3 | 2, &mut out);
varint(body.len() as u64, &mut out);
out.extend_from_slice(body);
out
}
/// A varint field.
pub(crate) fn int(n: u64, v: u64) -> Vec<u8> {
let mut out = Vec::new();
varint(n << 3, &mut out);
varint(v, &mut out);
out
}
/// A float tensor of `dims` named `name`, its data inside (`raw_data`).
pub(crate) fn inline_tensor(name: &str, dims: &[u64]) -> Vec<u8> {
let n: u64 = dims.iter().product();
let mut t: Vec<u8> = dims.iter().flat_map(|&d| int(1, d)).collect();
t.extend(int(2, 1)); // FLOAT
t.extend(field(8, name.as_bytes()));
t.extend(field(9, &vec![0u8; 4 * n as usize]));
t
}
/// The same tensor with its data in the file `location`.
pub(crate) fn external_tensor_proto(name: &str, dims: &[u64], location: &str) -> Vec<u8> {
let mut t: Vec<u8> = dims.iter().flat_map(|&d| int(1, d)).collect();
t.extend(int(2, 1));
t.extend(field(8, name.as_bytes()));
let mut entry = field(1, b"location");
entry.extend(field(2, location.as_bytes()));
t.extend(field(13, &entry));
t.extend(int(14, 1)); // EXTERNAL
t
}
/// A ModelProto whose graph is one `Add` of input `x` and initializer `w`.
pub(crate) fn model(initializer: &[u8]) -> Vec<u8> {
let float_1 = {
// TypeProto { tensor_type = 1 { elem_type = 1, shape = 2 { dim = 1 { dim_value = 1: 1 } } } }
let shape = field(1, &int(1, 1));
let mut tt = int(1, 1);
tt.extend(field(2, &shape));
field(1, &tt)
};
let value_info = |name: &str| {
let mut v = field(1, name.as_bytes());
v.extend(field(2, &float_1));
v
};
let mut node = field(1, b"x");
node.extend(field(1, b"w"));
node.extend(field(2, b"y"));
node.extend(field(4, b"Add"));
let mut graph = field(1, &node);
graph.extend(field(2, b"g"));
graph.extend(field(5, initializer));
graph.extend(field(11, &value_info("x")));
graph.extend(field(12, &value_info("y")));
let mut m = int(1, 8); // ir_version
m.extend(field(8, &int(2, 17))); // opset_import { version = 17 }
m.extend(field(7, &graph));
m
}
#[test]
fn a_tensor_kept_inside_the_model_is_not_external() {
assert_eq!(external_tensor(&model(&inline_tensor("w", &[1]))), Ok(None));
}
#[test]
fn an_external_initializer_is_found_with_its_location() {
let m = model(&external_tensor_proto("w", &[1], "weights.bin"));
assert_eq!(external_tensor(&m), Ok(Some("weights.bin".into())));
}
#[test]
fn an_external_tensor_in_a_subgraph_is_found() {
// A node whose attribute `then_branch` (g = 6) holds a graph with an
// external initializer, as an `If` does.
let mut sub = field(2, b"then");
sub.extend(field(5, &external_tensor_proto("w", &[1], "../outside.bin")));
let mut attr = field(1, b"then_branch");
attr.extend(field(6, &sub));
let mut node = field(4, b"If");
node.extend(field(5, &attr));
let graph = field(1, &node);
let m = field(7, &graph);
assert_eq!(external_tensor(&m), Ok(Some("../outside.bin".into())));
}
#[test]
fn external_data_without_a_location_still_marks_the_tensor() {
let mut t = int(2, 1);
t.extend(int(14, 1));
assert_eq!(external_tensor(&model(&t)), Ok(Some(String::new())));
}
#[test]
fn bytes_that_are_not_a_protobuf_message_are_an_error() {
let m = model(&inline_tensor("w", &[1]));
assert!(external_tensor(&m[..m.len() - 3]).is_err());
assert!(external_tensor(&[0x0b]).is_err()); // field 1, wire type 3 (group)
// Nesting past the bound: attributes' subgraphs inside each other.
let mut g = Vec::new();
for _ in 0..40 {
let attr = field(6, &g);
let node = field(5, &attr);
g = field(1, &node);
}
assert!(external_tensor(&field(7, &g)).is_err());
}
}

153
wai/wai-rs/src/neural/params.rs Normal file
View file

@ -0,0 +1,153 @@
//! Where a neural decode reads its parameters from (SPEC §3.1, *Decoding against
//! a verified set*).
//!
//! An unpinned decode reads the registered artifact and the files beside it, as a
//! sink always has. A pinned decode reads only the bytes the pin verified
//! ([`crate::pin::Verified`]) — the very bytes that were hashed, never a second read
//! of the same path — and a file the verified set does not contain is refused with
//! `set_incomplete` instead of being read from disk unverified.
//!
//! Sessions are built for determinism (`crate::ort_session`): one intra-op and one
//! inter-op thread and a fixed graph-optimization level, because multi-threaded
//! reductions sum partial results in a nondeterministic order. That makes a float
//! decode reproducible on one build; it does not make it reproducible across
//! hardware, which only an integer decode is (SPEC §7).
use std::borrow::Cow;
use std::path::{Path, PathBuf};
use ort::session::Session;
use crate::container::PinRefusal;
use crate::pin::Verified;
use super::DecodeError;
/// The parameters one decode reads.
#[derive(Debug, Clone)]
pub enum ParamSource {
/// No pin applied: the registered path, and the files beside it (the path
/// itself when it is a directory).
Unpinned(PathBuf),
/// A pin verified this set; the decode reads nothing else.
Pinned(Verified),
}
fn incomplete(name: &str) -> DecodeError {
DecodeError::PriorMismatch(PinRefusal::SetIncomplete { path: name.to_string() })
}
impl ParamSource {
/// The directory an unpinned decode reads siblings from.
fn dir(path: &Path) -> PathBuf {
if path.is_dir() {
path.to_path_buf()
} else {
path.parent().map(Path::to_path_buf).unwrap_or_default()
}
}
/// Whether a pin was verified for this decode.
pub fn is_pinned(&self) -> bool {
matches!(self, Self::Pinned(_))
}
/// The artifact the capability registers, named `set_name` within a set: for an
/// unpinned decode the registered file itself; for a form-less pin the one
/// verified file; for a set, the file it lists as `set_name`.
pub(crate) fn model(&self, set_name: &str) -> Result<Cow<'_, [u8]>, DecodeError> {
match self {
Self::Unpinned(p) => std::fs::read(p).map(Cow::Owned)
.map_err(|e| DecodeError::InvalidPayload(format!("{}: {e}", p.display()))),
Self::Pinned(Verified::File(b)) => Ok(Cow::Borrowed(b)),
Self::Pinned(v) => v.file(set_name).map(Cow::Borrowed).map_err(DecodeError::PriorMismatch),
}
}
/// The file `name` beside the registered artifact (unpinned), or in the verified
/// set. A form-less pin verified one file, so it holds no sibling.
pub(crate) fn sibling(&self, name: &str) -> Result<Cow<'_, [u8]>, DecodeError> {
match self {
Self::Unpinned(p) => {
let f = Self::dir(p).join(name);
std::fs::read(&f).map(Cow::Owned)
.map_err(|e| DecodeError::InvalidPayload(format!("{}: {e}", f.display())))
}
Self::Pinned(Verified::File(_)) => Err(incomplete(name)),
Self::Pinned(v) => v.file(name).map(Cow::Borrowed).map_err(DecodeError::PriorMismatch),
}
}
/// [`Self::sibling`], `None` when the file is absent (an unreadable unpinned
/// file counts as absent).
pub(crate) fn try_sibling(&self, name: &str) -> Option<Cow<'_, [u8]>> {
match self {
Self::Unpinned(p) => std::fs::read(Self::dir(p).join(name)).ok().map(Cow::Owned),
Self::Pinned(Verified::File(_)) => None,
Self::Pinned(v) => v.file(name).ok().map(Cow::Borrowed),
}
}
/// The JSON file `<stem>.json`, parsed.
pub(crate) fn json<T: for<'de> serde::Deserialize<'de>>(&self, stem: &str) -> Result<T, DecodeError> {
let name = format!("{stem}.json");
let raw = self.sibling(&name)?;
serde_json::from_slice(&raw).map_err(|e| DecodeError::InvalidPayload(format!("{name} parse: {e}")))
}
/// A deterministic session over the registered artifact ([`Self::model`]). An
/// unpinned decode loads it from its path, as before (so a model that keeps
/// tensors in external files still loads); a pinned one from the verified bytes
/// alone ([`session_from_verified`]).
pub(crate) fn session_model(&self, set_name: &str) -> Result<Session, DecodeError> {
match self {
Self::Unpinned(p) => crate::ort_session::load_deterministic(p).map_err(DecodeError::Ort),
Self::Pinned(_) => session_from_verified(&self.model(set_name)?),
}
}
/// A deterministic session over the sibling `name` ([`Self::sibling`]).
pub(crate) fn session_sibling(&self, name: &str) -> Result<Session, DecodeError> {
match self {
Self::Unpinned(p) => crate::ort_session::load_deterministic(&Self::dir(p).join(name)).map_err(DecodeError::Ort),
Self::Pinned(_) => session_from_verified(&self.sibling(name)?),
}
}
}
/// A deterministic session over verified model bytes. A tensor whose data the model
/// keeps in another file would be read by the runtime from outside the verified
/// set, so such a model is refused with `set_incomplete`, naming that file
/// ([`super::onnx_external`]); bytes that do not parse as a protobuf message are
/// refused as an invalid model.
fn session_from_verified(bytes: &[u8]) -> Result<Session, DecodeError> {
match super::onnx_external::external_tensor(bytes) {
Ok(None) => crate::ort_session::load_deterministic_from_memory(bytes).map_err(DecodeError::Ort),
Ok(Some(location)) => Err(incomplete(&location)),
Err(e) => Err(DecodeError::Ort(format!("verified model is not an ONNX protobuf: {e}"))),
}
}
#[cfg(test)]
mod tests {
use super::super::onnx_external::tests::{external_tensor_proto, inline_tensor, model};
use super::*;
/// A pinned decode never builds a session over a model whose tensors live in
/// another file: it is refused with `set_incomplete` naming that file, before
/// the runtime sees it. The same graph with its tensor inside loads.
#[test]
fn a_pinned_model_with_external_tensors_is_set_incomplete() {
let ext = ParamSource::Pinned(Verified::File(model(&external_tensor_proto("w", &[1], "weights.bin"))));
match ext.session_model("model.onnx") {
Err(DecodeError::PriorMismatch(r)) => {
assert_eq!(r.code(), "set_incomplete");
assert_eq!(r, PinRefusal::SetIncomplete { path: "weights.bin".into() });
}
Err(e) => panic!("expected set_incomplete, got {e:?}"),
Ok(_) => panic!("expected set_incomplete, got a session"),
}
let inline = ParamSource::Pinned(Verified::File(model(&inline_tensor("w", &[1]))));
assert!(inline.session_model("model.onnx").is_ok(), "a self-contained model loads");
}
}

25
wai/wai-rs/src/neural/runtime.rs
View file

@ -1,25 +0,0 @@
//! Thin `ort::Session` wrapper. The browser side uses onnxruntime-web;
//! this is the matching native runtime that loads the same .onnx files
//! the browser does. CPU-only by default; users wanting a GPU or NPU execution provider
//! enable extra ort features in their own Cargo.toml.
//!
//! The session is built for **determinism**, which is half the
//! `Prior.determinism` contract a neural capability declares. Float ONNX
//! synthesis is only reproducible if the reductions are: multi-threaded
//! intra-op execution sums partial results in nondeterministic order, so a
//! "replicate" claim evaporates. We therefore pin a single intra-op and
//! single inter-op thread and a fixed graph-optimization level. Together
//! with the integer rANS entropy stage (exact by construction) and a
//! pinned-prior model hash, two sinks honoring the same contract reproduce
//! the same bytes. Throughput is traded for reproducibility on purpose;
//! callers who only want perceptual output and not bit-exactness can build
//! their own session.
use std::path::Path;
use ort::session::Session;
use super::DecodeError;
pub(crate) fn load_session(path: &Path) -> Result<Session, DecodeError> {
crate::ort_session::load_deterministic(path).map_err(DecodeError::Ort)
}

7
wai/wai-rs/src/neural/snac.rs
View file

@ -17,18 +17,17 @@
//! output: audio_values float [1, 1, samples]
use std::borrow::Cow;
use std::path::Path;
use ndarray::{Array2, IxDyn};
use ort::session::SessionInputValue;
use ort::value::Value;
use super::audio::DecodedAudio;
use super::{runtime::load_session, DecodeError};
use super::{params::ParamSource, DecodeError};
const HEADER: usize = 12;
pub(crate) fn decode(payload: &[u8], model_path: &Path) -> Result<DecodedAudio, DecodeError> {
pub(crate) fn decode(payload: &[u8], params: &ParamSource) -> Result<DecodedAudio, DecodeError> {
if payload.len() < HEADER {
return Err(DecodeError::InvalidPayload(format!(
"snac payload {} B < {HEADER} B header", payload.len())));
@ -58,7 +57,7 @@ pub(crate) fn decode(payload: &[u8], model_path: &Path) -> Result<DecodedAudio,
}
// Parse each level's codes → [1, T_i] i64, build the named ONNX inputs.
let mut sess = load_session(model_path)?;
let mut sess = params.session_model("decoder.onnx")?;
let mut inputs: Vec<(Cow<'static, str>, SessionInputValue<'static>)> =
Vec::with_capacity(n_levels);
for (i, &t) in lengths.iter().enumerate() {

7
wai/wai-rs/src/neural/video.rs
View file

@ -8,13 +8,12 @@
//! Reuses the same `bmshj2018-factorized` synthesis-transform ONNX as
//! the still-image capability — one model on disk, two capabilities.
use std::path::Path;
use ndarray::Array4;
use ort::session::Session;
use ort::value::Value;
use super::{runtime::load_session, DecodeError};
use super::{params::ParamSource, DecodeError};
#[derive(Debug, Clone)]
pub struct DecodedVideo {
@ -24,7 +23,7 @@ pub struct DecodedVideo {
pub fps: f32,
}
pub(crate) fn decode(payload: &[u8], model_path: &Path) -> Result<DecodedVideo, DecodeError> {
pub(crate) fn decode(payload: &[u8], params: &ParamSource) -> Result<DecodedVideo, DecodeError> {
if payload.len() < 18 {
return Err(DecodeError::InvalidPayload(
format!("payload too short ({} B); need ≥ 18 B header", payload.len())));
@ -47,7 +46,7 @@ pub(crate) fn decode(payload: &[u8], model_path: &Path) -> Result<DecodedVideo,
lens.push(u32::from_le_bytes(payload[off..off + 4].try_into().unwrap()) as usize);
}
let mut sess = load_session(model_path)?;
let mut sess = params.session_model("decoder.onnx")?;
let in_name = sess.inputs().first().map(|i| i.name().to_string()).unwrap_or_else(|| "latents".into());
let out_name = sess.outputs().first().map(|o| o.name().to_string()).unwrap_or_else(|| "image".into());

11
wai/wai-rs/src/neural/video_dcvc.rs
View file

@ -29,18 +29,17 @@
//! outputs: image float [1, 3, H, W]
//! feature float [1, Cf, Sy, Sy] (propagated to the next frame)
use std::path::Path;
use ndarray::Array4;
use ort::value::Value;
use super::hyperprior::{decode_yhat, sibling};
use super::hyperprior::decode_yhat;
use super::video::DecodedVideo;
use super::{runtime::load_session, DecodeError};
use super::{params::ParamSource, DecodeError};
const HEADER: usize = 26;
pub(crate) fn decode(payload: &[u8], model_path: &Path) -> Result<DecodedVideo, DecodeError> {
pub(crate) fn decode(payload: &[u8], params: &ParamSource) -> Result<DecodedVideo, DecodeError> {
if payload.len() < HEADER {
return Err(DecodeError::InvalidPayload(format!(
"dcvc payload {} B < {HEADER} B header", payload.len())));
@ -67,7 +66,7 @@ pub(crate) fn decode(payload: &[u8], model_path: &Path) -> Result<DecodedVideo,
}
// Load the temporal decoder once; reuse across frames.
let mut tdec = load_session(&sibling(model_path, "temporal_decoder.onnx"))?;
let mut tdec = params.session_sibling("temporal_decoder.onnx")?;
// Reference feature, propagated frame→frame. Zero before the first frame.
let mut feat = Array4::<f32>::zeros((1, cf, sy, sy));
@ -82,7 +81,7 @@ pub(crate) fn decode(payload: &[u8], model_path: &Path) -> Result<DecodedVideo,
let y_bits = &payload[off + lz..off + lz + ly];
off += lz + ly;
let y_hat = decode_yhat(z_bits, y_bits, model_path, q, cz, cy, sz, sy)?;
let y_hat = decode_yhat(z_bits, y_bits, params, q, cz, cy, sz, sy)?;
let y_v = Value::from_array(y_hat)
.map_err(|e| DecodeError::Ort(format!("dcvc frame {i} latents: {e}")))?;

19
wai/wai-rs/src/ort_session.rs
View file

@ -23,3 +23,22 @@ pub(crate) fn load_deterministic(path: &Path) -> Result<Session, String> {
.commit_from_file(path)
.map_err(|e| format!("load {} failed: {e}", path.display()))
}
/// [`load_deterministic`] over model bytes already in memory — the bytes a pin
/// verified, so the session runs exactly what was hashed. A model that keeps
/// tensors in external files would have the runtime read them from outside those
/// bytes; the pinned decode refuses such a model before calling this
/// (`neural::params`).
#[cfg(feature = "neural")]
pub(crate) fn load_deterministic_from_memory(bytes: &[u8]) -> Result<Session, String> {
Session::builder()
.map_err(|e| format!("session builder: {e}"))?
.with_intra_threads(1)
.map_err(|e| format!("intra-threads: {e}"))?
.with_inter_threads(1)
.map_err(|e| format!("inter-threads: {e}"))?
.with_optimization_level(GraphOptimizationLevel::Level3)
.map_err(|e| format!("optimization level: {e}"))?
.commit_from_memory(bytes)
.map_err(|e| format!("load from verified bytes failed: {e}"))
}

589
wai/wai-rs/src/pin.rs Normal file
View file

@ -0,0 +1,589 @@
//! Verifying a parameter-set pin (SPEC §3.1) against the parameter sets a sink
//! holds, and reading and writing the listing a set of several files is pinned
//! through (`wai.prior.set`).
//!
//! The core, [`verify_bytes`], works on bytes in memory, so a sink with no
//! filesystem (a browser) runs the same checks as one that reads files; [`verify`]
//! reads the held artifacts from disk and calls it. Both run SPEC §3.1's checks in
//! order and report the first that fails as a [`PinRefusal`].
//!
//! A **listing** has one line per file: the file's SHA-256 as 64 lowercase
//! hexadecimal digits, two spaces, the file's path below the set's root, and a line
//! feed. Paths are `/`-joined segments of `A`–`Z`, `a`–`z`, `0`–`9`, `_`, `-` and
//! `.` (a segment starts with a letter, digit, `_` or `-` and does not end with
//! `.`); lines are sorted by path as byte strings; no two paths are equal once
//! `A`–`Z` are folded to `a`–`z`; no path followed by `/` begins another path. The
//! listing has at least one line and nothing else. It is the text common checksum
//! utilities write and check.
use std::collections::BTreeMap;
use std::path::{Path, PathBuf};
use sha2::{Digest, Sha256};
use crate::codecs::{ParameterSetShape, CAP_PRIOR_SET};
use crate::container::{is_sha256_hex, pin_shape, PinRefusal, Prior};
/// The prior forms this crate reads: `wai.prior.set`. It reads no coded form.
pub const READABLE_FORMS: &[&str] = &[CAP_PRIOR_SET];
/// SHA-256 of `bytes`.
pub fn sha256(bytes: &[u8]) -> [u8; 32] {
Sha256::digest(bytes).into()
}
/// SHA-256 of `bytes` as 64 lowercase hexadecimal digits.
pub fn sha256_hex(bytes: &[u8]) -> String {
hex(&sha256(bytes))
}
fn hex(d: &[u8]) -> String {
const H: &[u8; 16] = b"0123456789abcdef";
let mut s = String::with_capacity(d.len() * 2);
for &b in d {
s.push(H[(b >> 4) as usize] as char);
s.push(H[(b & 15) as usize] as char);
}
s
}
/// A digest shortened for display: the first 16 digits and an ellipsis when it is
/// a SHA-256 in WAI's grammar ([`is_sha256_hex`]); otherwise the whole value,
/// quoted and escaped, so a malformed digest is shown as it is rather than cut
/// at a byte that may fall inside a character.
pub fn digest_preview(d: &str) -> String {
if is_sha256_hex(d) {
format!("{}\u{2026}", &d[..16])
} else {
format!("{d:?} (not a SHA-256)")
}
}
/// One line describing a pin for people: its `id`, its digest
/// ([`digest_preview`]), and the optional members it states.
pub fn describe_pin(p: &Prior) -> String {
let mut s = format!("id {} sha256 {}", p.id, digest_preview(&p.sha256));
if let Some(f) = p.form() {
s += &format!(" form {f}");
}
if let Some(t) = p.tier() {
s += &format!(" tier {t}");
}
if let Some(sup) = p.supersedes() {
s += &format!(" supersedes {}", digest_preview(&sup.sha256));
}
s
}
fn malformed(why: impl Into<String>) -> PinRefusal {
PinRefusal::SetMalformed { why: why.into() }
}
/// Whether `path` is a listing path: one or more `/`-joined segments, each made of
/// `A`–`Z`, `a`–`z`, `0`–`9`, `_`, `-` and `.`, starting with a character other
/// than `.` and not ending with `.`.
pub fn is_listing_path(path: &str) -> bool {
!path.is_empty()
&& path.split('/').all(|seg| {
let b = seg.as_bytes();
!b.is_empty()
&& b[0] != b'.'
&& b[b.len() - 1] != b'.'
&& b.iter().all(|&c| c.is_ascii_alphanumeric() || c == b'_' || c == b'-' || c == b'.')
})
}
/// Check that a sorted list of paths obeys the listing rules beyond the per-path
/// grammar: strictly ascending as byte strings, no case-folded duplicates, and no
/// path that is a directory prefix of another.
fn check_paths(paths: &[&str]) -> Result<(), PinRefusal> {
if paths.is_empty() {
return Err(malformed("a listing has at least one line"));
}
for p in paths {
if !is_listing_path(p) {
return Err(malformed(format!("'{p}' is not a listing path")));
}
}
for w in paths.windows(2) {
if w[0].as_bytes() >= w[1].as_bytes() {
return Err(malformed(format!("'{}' does not sort strictly before '{}'", w[0], w[1])));
}
}
let mut folded: Vec<String> = paths.iter().map(|p| p.to_ascii_lowercase()).collect();
folded.sort_unstable();
if let Some(w) = folded.windows(2).find(|w| w[0] == w[1]) {
return Err(malformed(format!("two paths fold to '{}'", w[0])));
}
for a in paths {
let dir = format!("{a}/");
if let Some(b) = paths.iter().find(|b| b.starts_with(&dir)) {
return Err(malformed(format!("'{a}' is a file and the directory of '{b}'")));
}
}
Ok(())
}
/// Parse a listing into `(path, digest)` lines, enforcing every listing rule.
/// Any rule broken gives `set_malformed`.
pub fn parse_listing(bytes: &[u8]) -> Result<Vec<(String, [u8; 32])>, PinRefusal> {
if bytes.is_empty() {
return Err(malformed("a listing has at least one line"));
}
if *bytes.last().unwrap() != b'\n' {
return Err(malformed("a listing ends with a line feed"));
}
let mut out = Vec::new();
for (n, line) in bytes[..bytes.len() - 1].split(|&b| b == b'\n').enumerate() {
// 64 hex digits, two spaces, a path of at least one byte.
if line.len() < 67 || &line[64..66] != b" " {
return Err(malformed(format!("line {} is not '<64 hex> <path>'", n + 1)));
}
let digest = std::str::from_utf8(&line[..64]).ok().filter(|d| is_sha256_hex(d))
.ok_or_else(|| malformed(format!("line {} does not start with 64 lowercase hexadecimal digits", n + 1)))?;
let path = std::str::from_utf8(&line[66..])
.map_err(|_| malformed(format!("line {} has a path that is not ASCII", n + 1)))?;
let mut d = [0u8; 32];
for (i, out) in d.iter_mut().enumerate() {
*out = u8::from_str_radix(&digest[2 * i..2 * i + 2], 16).unwrap();
}
out.push((path.to_string(), d));
}
let paths: Vec<&str> = out.iter().map(|(p, _)| p.as_str()).collect();
check_paths(&paths)?;
Ok(out)
}
/// Write the listing of the files `files` (path → bytes). The paths must obey the
/// listing rules; the lines come out sorted.
pub fn listing_from(files: &BTreeMap<String, &[u8]>) -> Result<Vec<u8>, PinRefusal> {
let mut paths: Vec<&str> = files.keys().map(String::as_str).collect();
paths.sort_unstable_by(|a, b| a.as_bytes().cmp(b.as_bytes()));
check_paths(&paths)?;
let mut out = Vec::new();
for p in paths {
out.extend_from_slice(sha256_hex(files[p]).as_bytes());
out.extend_from_slice(b" ");
out.extend_from_slice(p.as_bytes());
out.push(b'\n');
}
Ok(out)
}
/// Write the listing of the files at `paths` below `root`, hashing each one.
pub fn write_listing(root: &Path, paths: &[String]) -> Result<Vec<u8>, PinRefusal> {
let mut bytes = BTreeMap::new();
for p in paths {
if !is_listing_path(p) {
return Err(malformed(format!("'{p}' is not a listing path")));
}
let b = std::fs::read(root.join(p)).map_err(|e| malformed(format!("{p}: {e}")))?;
bytes.insert(p.clone(), b);
}
let view: BTreeMap<String, &[u8]> = bytes.iter().map(|(k, v)| (k.clone(), v.as_slice())).collect();
listing_from(&view)
}
/// The listing of every regular file below `root`, skipping names that begin with
/// `.` and the `*.sha256` files directly in `root` (the listings themselves). A
/// symbolic link, or a name a listing path cannot carry, is refused.
pub fn listing_of_dir(root: &Path) -> Result<Vec<u8>, PinRefusal> {
fn walk(root: &Path, rel: &str, out: &mut Vec<String>) -> Result<(), PinRefusal> {
let dir = if rel.is_empty() { root.to_path_buf() } else { root.join(rel) };
let rd = std::fs::read_dir(&dir).map_err(|e| malformed(format!("{}: {e}", dir.display())))?;
for entry in rd {
let entry = entry.map_err(|e| malformed(e.to_string()))?;
let name = entry.file_name().into_string()
.map_err(|n| malformed(format!("{n:?} is not a listing path")))?;
if name.starts_with('.') {
continue;
}
let ty = entry.file_type().map_err(|e| malformed(e.to_string()))?;
let path = if rel.is_empty() { name.clone() } else { format!("{rel}/{name}") };
if ty.is_symlink() {
return Err(malformed(format!("'{path}' is a symbolic link")));
} else if ty.is_dir() {
walk(root, &path, out)?;
} else if rel.is_empty() && name.ends_with(".sha256") {
continue;
} else {
out.push(path);
}
}
Ok(())
}
let mut paths = Vec::new();
walk(root, "", &mut paths)?;
write_listing(root, &paths)
}
/// A parameter set a sink holds for a capability, in memory: one file, or a
/// listing together with the files below its root (path → bytes; a file the sink
/// could not read is simply absent).
#[derive(Debug, Clone, Copy)]
pub enum HeldBytes<'a> {
File(&'a [u8]),
Set { listing: &'a [u8], files: &'a BTreeMap<String, &'a [u8]> },
}
/// Which held artifact satisfied a pin: its index in the `held` slice given to
/// [`verify_bytes`], and, for a set, the verified files' paths.
#[derive(Debug, Clone, PartialEq, Eq)]
pub enum VerifiedRef {
/// Form-less: the one file at `held[index]`.
File { index: usize },
/// `wai.prior.set`: the set at `held[index]`, every file its listing names
/// verified.
Set { index: usize, paths: Vec<String> },
}
/// Run SPEC §3.1's checks for `prior` over `capability` against the parameter sets
/// in `held`, reading the forms in `readable`. The first check that fails is the
/// reason: `pin_malformed`, `form_not_registered`, `form_unreadable`,
/// `form_required`, `prior_not_held` / `digest_superseded`, `set_malformed`,
/// `part_mismatch`. (Check 5, `prior_revoked`, is reserved.) The checks read only
/// the pin, the registry, `readable` and the bytes, so two sinks that hold the same
/// bytes and read the same forms report the same reason.
pub fn verify_bytes(
capability: &str,
prior: &Prior,
held: &[HeldBytes<'_>],
readable: &[&str],
) -> Result<VerifiedRef, PinRefusal> {
// 1. pin_malformed (only `sha256`: a bad `supersedes` is ignored).
prior.check_syntax()?;
// 2. form_not_registered, 3. form_unreadable.
prior.check_form(readable)?;
// 4. form_required.
if prior.form().is_none() && pin_shape(capability) != ParameterSetShape::OneFile {
return Err(PinRefusal::FormRequired { capability: capability.to_string() });
}
// 5. reserved: prior_revoked, keyed on (form, sha256).
let set_form = prior.form() == Some(CAP_PRIOR_SET);
let digest_of = |h: &HeldBytes<'_>| -> Option<String> {
match (h, set_form) {
(HeldBytes::File(b), false) => Some(sha256_hex(b)),
(HeldBytes::Set { listing, .. }, true) => Some(sha256_hex(listing)),
_ => None,
}
};
let digests: Vec<Option<String>> = held.iter().map(digest_of).collect();
let matching: Vec<usize> = (0..held.len())
.filter(|&i| digests[i].as_deref() == Some(prior.sha256.as_str()))
.collect();
// 6. prior_not_held / digest_superseded.
if matching.is_empty() {
if let Some(pre) = prior.usable_supersedes() {
let pre_set = pre.form.as_deref() == Some(CAP_PRIOR_SET);
let holds_pre = held.iter().any(|h| match (h, pre_set) {
(HeldBytes::File(b), false) => sha256_hex(b) == pre.sha256,
(HeldBytes::Set { listing, .. }, true) => sha256_hex(listing) == pre.sha256,
_ => false,
});
if holds_pre {
return Err(PinRefusal::Superseded {
capability: capability.to_string(),
predecessor: pre.sha256.clone(),
});
}
}
return Err(PinRefusal::NotHeld { capability: capability.to_string() });
}
if !set_form {
return Ok(VerifiedRef::File { index: matching[0] });
}
// 7. set_malformed: every matching listing has the same bytes, so parse one.
let HeldBytes::Set { listing, .. } = held[matching[0]] else { unreachable!() };
let lines = parse_listing(listing)?;
// 8. part_mismatch: the first copy whose files are all intact wins.
let mut first_failure: Option<String> = None;
for &i in &matching {
let HeldBytes::Set { files, .. } = held[i] else { unreachable!() };
let bad = lines.iter().find(|(path, d)| files.get(path).is_none_or(|b| sha256(b) != *d));
match bad {
None => {
return Ok(VerifiedRef::Set { index: i, paths: lines.into_iter().map(|(p, _)| p).collect() });
}
Some((path, _)) => {
first_failure.get_or_insert_with(|| path.clone());
}
}
}
Err(PinRefusal::PartMismatch { path: first_failure.unwrap_or_default() })
}
/// An artifact a sink holds, on disk: a file (for a form-less pin), or a listing
/// whose root is the directory it sits in (for a `wai.prior.set` pin).
#[derive(Debug, Clone, PartialEq, Eq)]
pub enum Held {
File(PathBuf),
Listing(PathBuf),
}
/// The parameter set a pin verified, as the very bytes that were hashed. A pinned
/// decode reads its parameters only from here.
#[derive(Debug, Clone, PartialEq, Eq)]
pub enum Verified {
/// Form-less: the one file.
File(Vec<u8>),
/// `wai.prior.set`: every file the listing names, by path.
Set { root: PathBuf, files: BTreeMap<String, Vec<u8>> },
}
impl Verified {
/// The one file of a form-less pin.
pub fn one(&self) -> Option<&[u8]> {
match self {
Self::File(b) => Some(b),
Self::Set { .. } => None,
}
}
/// The file at `path` of a verified set; `set_incomplete` when the set does
/// not contain it (or the pin was form-less).
pub fn file(&self, path: &str) -> Result<&[u8], PinRefusal> {
match self {
Self::Set { files, .. } => files.get(path).map(Vec::as_slice),
Self::File(_) => None,
}
.ok_or_else(|| PinRefusal::SetIncomplete { path: path.to_string() })
}
}
/// [`verify_bytes`] over artifacts on disk. Each held file is read once and the
/// bytes that were hashed are the bytes returned; a file that cannot be read is not
/// held. For a listing, only the files of a listing whose digest matches the pin
/// are read.
pub fn verify(capability: &str, prior: &Prior, held: &[Held], readable: &[&str]) -> Result<Verified, PinRefusal> {
// The cheap checks first, so a malformed or form-less pin reads no file.
prior.check_syntax()?;
prior.check_form(readable)?;
if prior.form().is_none() && pin_shape(capability) != ParameterSetShape::OneFile {
return Err(PinRefusal::FormRequired { capability: capability.to_string() });
}
let set_form = prior.form() == Some(CAP_PRIOR_SET);
struct Loaded {
bytes: Vec<u8>,
root: Option<PathBuf>,
files: BTreeMap<String, Vec<u8>>,
}
let mut loaded: Vec<Loaded> = Vec::new();
for h in held {
match (h, set_form) {
(Held::File(p), false) => {
if let Ok(bytes) = std::fs::read(p) {
loaded.push(Loaded { bytes, root: None, files: BTreeMap::new() });
}
}
(Held::Listing(p), true) => {
let Ok(bytes) = std::fs::read(p) else { continue };
let root = p.parent().map(Path::to_path_buf).unwrap_or_default();
let mut files = BTreeMap::new();
if sha256_hex(&bytes) == prior.sha256 {
if let Ok(lines) = parse_listing(&bytes) {
for (path, _) in lines {
if let Ok(b) = std::fs::read(root.join(&path)) {
files.insert(path, b);
}
}
}
}
loaded.push(Loaded { bytes, root: Some(root), files });
}
_ => {}
}
}
let views: Vec<BTreeMap<String, &[u8]>> = loaded.iter()
.map(|l| l.files.iter().map(|(k, v)| (k.clone(), v.as_slice())).collect())
.collect();
let held_bytes: Vec<HeldBytes<'_>> = loaded.iter().zip(views.iter())
.map(|(l, v)| if l.root.is_some() {
HeldBytes::Set { listing: &l.bytes, files: v }
} else {
HeldBytes::File(&l.bytes)
})
.collect();
match verify_bytes(capability, prior, &held_bytes, readable)? {
VerifiedRef::File { index } => Ok(Verified::File(loaded.swap_remove(index).bytes)),
VerifiedRef::Set { index, paths } => {
let l = loaded.swap_remove(index);
let files = paths.into_iter().filter_map(|p| l.files.get(&p).cloned().map(|b| (p, b))).collect();
Ok(Verified::Set { root: l.root.unwrap_or_default(), files })
}
}
}
#[cfg(test)]
mod tests {
use super::*;
use crate::codecs::{CAP_NEURAL_BMSHJ2018, CAP_NEURAL_INT_MLICPP, CAP_PRIOR_NNC};
use crate::container::Supersedes;
fn line(d: &[u8], path: &str) -> Vec<u8> {
format!("{} {path}\n", sha256_hex(d)).into_bytes()
}
/// A digest that is not 64 lowercase hex digits is shown whole and escaped, so
/// a multi-byte character across byte 16 does not panic the preview.
#[test]
fn digest_preview_never_slices_inside_a_character() {
let good = "ab".repeat(32);
assert_eq!(digest_preview(&good), format!("{}\u{2026}", &good[..16]));
let bad = format!("a{}", "\u{e9}".repeat(8)); // byte 16 falls inside a character
assert!(!bad.is_char_boundary(16));
assert_eq!(digest_preview(&bad), format!("{bad:?} (not a SHA-256)"));
let p = Prior::new("m", bad.clone(), "int")
.with_supersedes(crate::container::Supersedes::new(bad.clone()));
let line = describe_pin(&p);
assert!(line.starts_with("id m sha256 \"a"), "{line}");
assert!(line.contains("supersedes \"a"), "{line}");
}
#[test]
fn listing_rules() {
let ok: Vec<u8> = [line(b"a", "a.b"), line(b"c", "a/c.onnx"), line(b"z", "z")].concat();
assert_eq!(parse_listing(&ok).unwrap().len(), 3, "'a.b' sorts before 'a/c.onnx' as bytes");
let h = sha256_hex(b"x");
let bad: &[(&str, Vec<u8>)] = &[
("CRLF", format!("{h} a\r\n").into_bytes()),
("BOM", [b"\xef\xbb\xbf".to_vec(), line(b"x", "a")].concat()),
("no final LF", format!("{h} a").into_bytes()),
("blank line", [line(b"x", "a"), b"\n".to_vec(), line(b"x", "b")].concat()),
("one-space separator", format!("{h} a\n").into_bytes()),
("uppercase hex", format!("{} a\n", h.to_ascii_uppercase()).into_bytes()),
("unsorted", [line(b"x", "b"), line(b"x", "a")].concat()),
("duplicate", [line(b"x", "a"), line(b"x", "a")].concat()),
("case fold", [line(b"x", "A.json"), line(b"x", "a.json")].concat()),
("file and directory", [line(b"x", "w"), line(b"x", "w/x")].concat()),
("leading dot", line(b"x", ".hidden")),
("trailing dot", line(b"x", "a.")),
("dot-dot", line(b"x", "a/../b")),
("empty segment", line(b"x", "a//b")),
("absolute", line(b"x", "/a")),
("space in path", line(b"x", "a b")),
("empty", Vec::new()),
];
for (what, b) in bad {
assert_eq!(parse_listing(b).map(|_| ()).unwrap_err().code(), "set_malformed", "{what}");
}
}
#[test]
fn write_listing_matches_the_checksum_format() {
let d = Path::new(env!("CARGO_MANIFEST_DIR")).join("tests/vectors/int_mlicpp");
let listing = write_listing(&d, &["model.wmb".to_string()]).unwrap();
assert_eq!(listing.len(), 76);
assert_eq!(listing, std::fs::read(d.join("model.sha256")).unwrap(), "the committed listing");
assert_eq!(sha256_hex(&listing), "85a05b0dac15566c5a657ddeee253eaa9f54ce852ae04924f5b04151901d9d1b");
assert_eq!(parse_listing(&listing).unwrap()[0].0, "model.wmb");
}
fn files<'a>(pairs: &[(&str, &'a [u8])]) -> BTreeMap<String, &'a [u8]> {
pairs.iter().map(|&(k, v)| (k.to_string(), v)).collect()
}
/// Every precedence edge of SPEC §3.1's order, table-driven.
#[test]
fn refusals_come_in_order() {
let model: &[u8] = b"one-file model";
let m_hex = sha256_hex(model);
let f = files(&[("model.wmb", model)]);
let listing = listing_from(&f).unwrap();
let l_hex = sha256_hex(&listing);
let set = |p: Prior| p.with_form(CAP_PRIOR_SET);
let one = Prior::new("m", m_hex.clone(), "int");
let held_file = [HeldBytes::File(model)];
let held_set = [HeldBytes::Set { listing: &listing, files: &f }];
let r = |cap: &str, p: &Prior, h: &[HeldBytes<'_>], readable: &[&str]| {
verify_bytes(cap, p, h, readable).map(|_| "ok").unwrap_or_else(|e| e.code())
};
let mlic = CAP_NEURAL_INT_MLICPP;
// ok, both shapes.
assert_eq!(r(mlic, &one, &held_file, READABLE_FORMS), "ok");
assert_eq!(r(mlic, &set(Prior::new("m", l_hex.clone(), "int")), &held_set, READABLE_FORMS), "ok");
// 1 before everything: an uppercase digest over an unreadable form.
let upper = Prior::new("m", m_hex.to_ascii_uppercase(), "int").with_form(CAP_PRIOR_NNC);
assert_eq!(r(mlic, &upper, &held_file, &[]), "pin_malformed");
assert_eq!(r(mlic, &Prior::new("m", &m_hex[..63], "int"), &held_file, READABLE_FORMS), "pin_malformed");
// 2 before 3.
assert_eq!(r(mlic, &one.clone().with_form("wai.image.png"), &held_file, &[]), "form_not_registered");
assert_eq!(r(mlic, &one.clone().with_form("wai.prior.nope"), &held_file, &[]), "form_not_registered");
// 3 before 4 and 6.
assert_eq!(r(CAP_NEURAL_BMSHJ2018, &one.clone().with_form(CAP_PRIOR_NNC), &[], READABLE_FORMS), "form_unreadable");
// 4 before 6: a several-file capability with a form-less pin, even when a
// held file matches.
assert_eq!(r(CAP_NEURAL_BMSHJ2018, &one, &held_file, READABLE_FORMS), "form_required");
// An unregistered capability is several files: fail closed.
assert_eq!(r("wai.neural.future", &one, &held_file, READABLE_FORMS), "form_required");
// 6: not held; superseded when the predecessor is held.
let other = Prior::new("m", sha256_hex(b"other"), "int");
assert_eq!(r(mlic, &other, &held_file, READABLE_FORMS), "prior_not_held");
let succ = other.clone().with_supersedes(Supersedes::new(m_hex.clone()));
assert_eq!(r(mlic, &succ, &held_file, READABLE_FORMS), "digest_superseded");
// A malformed supersedes is ignored: not held, never superseded.
let bad_sup = other.clone().with_supersedes(Supersedes::new(m_hex.clone()).with_form("wai.image.png"));
assert_eq!(r(mlic, &bad_sup, &held_file, READABLE_FORMS), "prior_not_held");
// A successor in set form whose predecessor is a set.
let succ_set = set(other.clone()).with_supersedes(Supersedes::new(l_hex.clone()).with_form(CAP_PRIOR_SET));
assert_eq!(r(mlic, &succ_set, &held_set, READABLE_FORMS), "digest_superseded");
// A file never satisfies a set pin, a listing never a file pin.
assert_eq!(r(mlic, &set(one.clone()), &held_file, READABLE_FORMS), "prior_not_held");
assert_eq!(r(mlic, &Prior::new("m", l_hex.clone(), "int"), &held_set, READABLE_FORMS), "prior_not_held");
// 7: the pinned listing is malformed (CRLF).
let crlf = String::from_utf8(listing.clone()).unwrap().replace('\n', "\r\n").into_bytes();
let held_crlf = [HeldBytes::Set { listing: &crlf, files: &f }];
assert_eq!(r(mlic, &set(Prior::new("m", sha256_hex(&crlf), "int")), &held_crlf, READABLE_FORMS), "set_malformed");
// 8: a file the listing names is flipped, or missing.
let flipped = files(&[("model.wmb", b"one-file modeL")]);
let held_flip = [HeldBytes::Set { listing: &listing, files: &flipped }];
let pin = set(Prior::new("m", l_hex.clone(), "int"));
assert_eq!(r(mlic, &pin, &held_flip, READABLE_FORMS), "part_mismatch");
let empty = BTreeMap::new();
assert_eq!(r(mlic, &pin, &[HeldBytes::Set { listing: &listing, files: &empty }], READABLE_FORMS), "part_mismatch");
// A flipped listing is a different digest: not held.
let mut lflip = listing.clone();
lflip[0] ^= 1;
assert_eq!(r(mlic, &pin, &[HeldBytes::Set { listing: &lflip, files: &f }], READABLE_FORMS), "prior_not_held");
}
#[test]
fn two_copies_one_intact_resolves() {
let model: &[u8] = b"weights";
let good = files(&[("model.wmb", model)]);
let bad = files(&[("model.wmb", b"weightz")]);
let listing = listing_from(&good).unwrap();
let pin = Prior::new("m", sha256_hex(&listing), "int").with_form(CAP_PRIOR_SET);
let held = [
HeldBytes::Set { listing: &listing, files: &bad },
HeldBytes::Set { listing: &listing, files: &good },
];
assert_eq!(
verify_bytes(CAP_NEURAL_INT_MLICPP, &pin, &held, READABLE_FORMS),
Ok(VerifiedRef::Set { index: 1, paths: vec!["model.wmb".into()] })
);
}
#[test]
fn verified_bytes_are_the_bytes_hashed() {
let dir = std::env::temp_dir().join(format!("wai-pin-verified-{}", std::process::id()));
std::fs::create_dir_all(dir.join("weights")).unwrap();
std::fs::write(dir.join("a.json"), b"{\"a\":1}").unwrap();
std::fs::write(dir.join("weights/b.json"), b"{\"b\":2}").unwrap();
std::fs::write(dir.join(".hidden"), b"skipped").unwrap();
let listing = listing_of_dir(&dir).unwrap();
assert_eq!(parse_listing(&listing).unwrap().iter().map(|(p, _)| p.as_str()).collect::<Vec<_>>(),
["a.json", "weights/b.json"], "hidden files are not listed");
std::fs::write(dir.join("set.sha256"), &listing).unwrap();
let pin = Prior::new("s", sha256_hex(&listing), "int").with_form(CAP_PRIOR_SET);
let v = verify("wai.neural.elic", &pin, &[Held::Listing(dir.join("set.sha256"))], READABLE_FORMS).unwrap();
// Overwrite a file after verification: the verified bytes do not change.
std::fs::write(dir.join("a.json"), b"{\"a\":2}").unwrap();
assert_eq!(v.file("a.json").unwrap(), b"{\"a\":1}");
assert_eq!(v.file("weights/b.json").unwrap(), b"{\"b\":2}");
assert_eq!(v.file("c.json").unwrap_err().code(), "set_incomplete");
// Now the held copy is damaged.
let again = verify("wai.neural.elic", &pin, &[Held::Listing(dir.join("set.sha256"))], READABLE_FORMS);
assert_eq!(again.unwrap_err(), PinRefusal::PartMismatch { path: "a.json".into() });
std::fs::remove_dir_all(&dir).ok();
}
}

15
wai/wai-rs/src/quantum_envelope.rs
View file

@ -11,19 +11,8 @@ use wai_quantum::quantum::{to_wqc, Circuit, Measure, CAPABILITY};
pub fn wrap_envelope(c: &Circuit, measure: Option<Measure>) -> crate::container::Wai {
use crate::container::{Conditioning, Manifest, ModelRequirement};
let payload = to_wqc(c, measure);
let manifest = Manifest {
wai: "1.0".into(),
media: "quantum".into(),
intent: "replicate".into(),
model_requirement: ModelRequirement {
capability: CAPABILITY.into(),
fallback: None,
min_sim: None,
sim_metric: None,
},
conditioning: Conditioning { kind: "wqc".into(), prior: None },
target: serde_json::json!({ "n_qubits": c.n_qubits, "dim": 1u64 << c.n_qubits }),
};
let manifest = Manifest::new("quantum", "replicate", ModelRequirement::new(CAPABILITY), Conditioning::new("wqc"))
.with_target(serde_json::json!({ "n_qubits": c.n_qubits, "dim": 1u64 << c.n_qubits }));
crate::container::Wai::new(manifest, payload)
}

22
wai/wai-rs/tests/carriage.rs
View file

@ -52,19 +52,12 @@ fn payload_for(cap: &str, n: usize) -> Vec<u8> {
fn wrap(cap: &str, payload: Vec<u8>) -> Wai {
Wai::new(
Manifest {
wai: "1.0".into(),
media: media_for(cap).into(),
intent: "replicate".into(),
model_requirement: ModelRequirement {
capability: cap.into(),
fallback: Some("wai.image.png".into()),
min_sim: None,
sim_metric: None,
},
conditioning: Conditioning { kind: "bytes".into(), prior: None },
target: serde_json::Value::Null,
},
Manifest::new(
media_for(cap),
"replicate",
ModelRequirement::new(cap).with_fallback("wai.image.png"),
Conditioning::new("bytes"),
),
payload,
)
}
@ -94,7 +87,8 @@ fn every_registered_capability_carries_bytes_byte_exact() {
assert_eq!(bound.len(), 1, "{cap}: companion not bound to its base");
assert_eq!(bound[0].0.capability, cap, "{cap}: capability not preserved");
assert_eq!(bound[0].1, payload.as_slice(), "{cap}: payload not carried byte-identically at {len} B");
assert_eq!(back.primary().unwrap().0.capability, "wai.video.hevc", "{cap}: companion was selectable");
assert_eq!(back.primary(wai::container::no_parameter_sets()).unwrap().meta.capability,
"wai.video.hevc", "{cap}: companion was selectable");
assert!(capability_tier(cap).is_some(), "{cap}: no fidelity tier");
assert!(capability_license(cap).is_some(), "{cap}: no license class");
continue;

12
wai/wai-rs/tests/cli_registry.rs
View file

@ -87,8 +87,10 @@ fn registry_without_a_subcommand_is_refused() {
assert!(String::from_utf8_lossy(&run.stderr).contains("registry export"));
}
fn payload() -> PathBuf {
let dir = scratch_dir("payload");
/// A payload file in a directory of the calling test's own: tests run in
/// parallel, and `scratch_dir` empties the directory it returns.
fn payload(test: &str) -> PathBuf {
let dir = scratch_dir(&format!("payload-{test}"));
let p = dir.join("bytes");
std::fs::write(&p, b"opaque payload bytes").unwrap();
p
@ -96,7 +98,7 @@ fn payload() -> PathBuf {
#[test]
fn wrap_refuses_a_candidate_and_writes_nothing() {
let p = payload();
let p = payload("wrap_refuses_a_candidate_and_writes_nothing");
let p = p.to_str().unwrap();
let companion = scratch_dir("companion").join("hdr");
std::fs::write(&companion, b"metadata").unwrap();
@ -130,7 +132,7 @@ fn wrap_refuses_a_candidate_and_writes_nothing() {
#[test]
fn wrap_writes_the_registry_media_class() {
let p = payload();
let p = payload("wrap_writes_the_registry_media_class");
for (cap, media, extra) in [
("wai.mesh.draco", "volumetric", &[][..]),
("wai.neural.bmshj2018", "image", &["--intent", "improve"][..]),
@ -151,7 +153,7 @@ fn wrap_writes_the_registry_media_class() {
/// may say a PNG envelope decodes here or list PNG as native.
#[test]
fn inspect_and_capabilities_claim_native_decode_only_with_codecs() {
let p = payload();
let p = payload("inspect_and_capabilities_claim_native_decode_only_with_codecs");
let out = scratch_dir("inspect").join("png.wai");
let run = wai(&["wrap", "wai.image.png", p.to_str().unwrap(), "-o", out.to_str().unwrap(), "--intent", "improve"]);
assert!(run.status.success(), "{}", String::from_utf8_lossy(&run.stderr));

182
wai/wai-rs/tests/ffi_envelope.rs Normal file
View file

@ -0,0 +1,182 @@
//! The C ABI's envelope surface (feature `ffi`, no codec library): unpack returns
//! each manifest exactly as the envelope carries it, the `WAI2` rendition table
//! carries every component's identity and whether SPEC §7.1 may select it, packing
//! keeps members this library does not model, `wai_envelope_select` picks by §7.1
//! and reports each skipped entry's reason, and the packers refuse a candidate
//! capability (SPEC §5).
#![cfg(feature = "ffi")]
use std::ffi::{CStr, CString};
use std::ptr;
use wai::container::{Prior, Rendition, WaiMulti};
use wai::ffi::*;
fn empty() -> WaiBuffer {
WaiBuffer { data: ptr::null_mut(), len: 0, cap: 0 }
}
fn take(b: WaiBuffer) -> Vec<u8> {
let v = if b.data.is_null() { Vec::new() } else { unsafe { std::slice::from_raw_parts(b.data, b.len) }.to_vec() };
unsafe { wai_buffer_free(b) };
v
}
fn last_error() -> String {
let p = wai_last_error();
if p.is_null() { String::new() } else { unsafe { CStr::from_ptr(p) }.to_string_lossy().into_owned() }
}
fn stream_wai() -> Vec<u8> {
std::fs::read(std::path::Path::new(env!("CARGO_MANIFEST_DIR")).join("tests/vectors/int_mlicpp/stream.wai")).unwrap()
}
#[test]
fn unpack_returns_the_original_manifest_bytes() {
let env = stream_wai();
let (mut m, mut p) = (empty(), empty());
assert_eq!(unsafe { wai_envelope_unpack(env.as_ptr(), env.len(), &mut m, &mut p) }, WAI_OK);
let m = take(m);
assert_eq!(m.len(), 339);
assert_eq!(m, env[8..8 + 339]);
assert_eq!(take(p).len(), env.len() - 8 - 339 - 4);
// A manifest with `security`, whitespace and a member this library does not
// model comes back unchanged.
let man = br#"{"wai":"1.0", "media":"text","intent":"replicate","model_requirement":{"capability":"wai.text.zstd"},"conditioning":{"kind":"zstd"},"security":{"encryption":"mls-sframe","kid":2},"target":null,"later":1}"#;
let mut e = b"WAI1".to_vec();
e.extend_from_slice(&(man.len() as u32).to_le_bytes());
e.extend_from_slice(man);
e.extend_from_slice(&1u32.to_le_bytes());
e.push(7);
let (mut m, mut p) = (empty(), empty());
assert_eq!(unsafe { wai_envelope_unpack(e.as_ptr(), e.len(), &mut m, &mut p) }, WAI_OK);
assert_eq!(take(m), man);
take(p);
}
fn multi() -> Vec<u8> {
WaiMulti::new("video", "replicate", serde_json::Value::Null, vec![
Rendition::media("wai.video.lcevc", "lcevc", b"MISLABELLED".to_vec()),
Rendition::media("wai.neural.int_mlicpp", "int_mlicpp", b"PINNED".to_vec())
.with_id("pinned")
.with_prior(Prior::new("m", "ab".repeat(32), "int")),
Rendition::media("wai.video.hevc", "hevc", b"HEVC".to_vec()).with_id("base").with_payload_digest(),
Rendition::companion("wai.meta.hdr10plus", b"META".to_vec(), "wai.video.hevc").with_companion_of_id("base"),
]).to_bytes().unwrap()
}
#[test]
fn unpack_multi_table_carries_identity_and_selectability() {
let env = multi();
let (mut m, mut t, mut b) = (empty(), empty(), empty());
assert_eq!(unsafe { wai_envelope_unpack_multi(env.as_ptr(), env.len(), &mut m, &mut t, &mut b) }, WAI_OK);
let ml = u32::from_le_bytes(env[4..8].try_into().unwrap()) as usize;
assert_eq!(take(m), env[8..8 + ml]);
let table: serde_json::Value = serde_json::from_slice(&take(t)).unwrap();
take(b);
let sel: Vec<bool> = table.as_array().unwrap().iter().map(|e| e["selectable"].as_bool().unwrap()).collect();
assert_eq!(sel, [false, true, true, false], "a companion-only entry and a companion are never selectable");
assert_eq!(table[1]["prior"]["sha256"], "ab".repeat(32));
assert_eq!(table[1]["id"], "pinned");
assert_eq!(table[2]["sha256"], wai::pin::sha256_hex(b"HEVC"));
assert_eq!(table[3]["role"], "companion");
assert_eq!(table[3]["companion_of"], "wai.video.hevc");
assert_eq!(table[3]["companion_of_id"], "base");
assert!(table[0].get("prior").is_none() && table[0].get("id").is_none());
}
#[test]
fn envelope_select_follows_spec_7_1() {
let env = multi();
let all = CString::new(r#"["wai.video.lcevc","wai.neural.int_mlicpp","wai.video.hevc","wai.meta.hdr10plus"]"#).unwrap();
let mut idx = u32::MAX;
assert_eq!(unsafe { wai_envelope_select(env.as_ptr(), env.len(), all.as_ptr(), &mut idx) }, WAI_OK);
// Entry 0 is companion-only, entry 1's pin applies and this call holds no set.
assert_eq!(idx, 2);
let none = CString::new(r#"["wai.neural.int_mlicpp"]"#).unwrap();
assert_eq!(unsafe { wai_envelope_select(env.as_ptr(), env.len(), none.as_ptr(), &mut idx) }, WAI_ERR_NO_RENDITION);
let why = last_error();
assert!(why.contains("0: not_selectable") && why.contains("1: prior_not_held: ") && why.contains("2: missing_capability"), "{why}");
// A bad argument is reported, not a panic.
let bad = CString::new("not json").unwrap();
assert_eq!(unsafe { wai_envelope_select(env.as_ptr(), env.len(), bad.as_ptr(), &mut idx) }, WAI_ERR_INVALID_ARG);
assert_eq!(unsafe { wai_envelope_select(env.as_ptr(), env.len(), all.as_ptr(), ptr::null_mut()) }, WAI_ERR_INVALID_ARG);
}
#[test]
fn pack_multi_keeps_every_member() {
let manifest = CString::new(r#"{"wai":"1.1","media":"image","intent":"replicate","renditions":[{"capability":"wai.image.png","kind":"png","id":"p","sha256":"0000000000000000000000000000000000000000000000000000000000000000","later":{"x":1}}],"target":null}"#).unwrap();
let payload = b"PNG";
let ptrs = [payload.as_ptr()];
let lens = [payload.len()];
let mut out = empty();
assert_eq!(unsafe { wai_envelope_pack_multi(manifest.as_ptr(), ptrs.as_ptr(), lens.as_ptr(), 1, &mut out) }, WAI_OK);
let env = take(out);
let ml = u32::from_le_bytes(env[4..8].try_into().unwrap()) as usize;
assert_eq!(&env[8..8 + ml], manifest.as_bytes());
// A mismatched payload count is an argument error.
let mut out = empty();
assert_eq!(unsafe { wai_envelope_pack_multi(manifest.as_ptr(), ptrs.as_ptr(), lens.as_ptr(), 0, &mut out) }, WAI_ERR_INVALID_ARG);
}
/// SPEC §4, §5 and §7.1: a candidate is never selected, whatever the sink
/// advertises; the table marks it not selectable, and selection reports why.
#[test]
fn envelope_select_never_picks_a_candidate() {
let env = WaiMulti::new("binary", "improve", serde_json::Value::Null, vec![
Rendition::media("wai.tensor.nnc", "bytes", b"TENSOR".to_vec()),
Rendition::media("wai.image.png", "png", b"PNG".to_vec()),
]).to_bytes().unwrap();
let (mut m, mut t, mut b) = (empty(), empty(), empty());
assert_eq!(unsafe { wai_envelope_unpack_multi(env.as_ptr(), env.len(), &mut m, &mut t, &mut b) }, WAI_OK);
take(m);
take(b);
let table: serde_json::Value = serde_json::from_slice(&take(t)).unwrap();
let sel: Vec<bool> = table.as_array().unwrap().iter().map(|e| e["selectable"].as_bool().unwrap()).collect();
assert_eq!(sel, [false, true], "a candidate is never selectable");
let all = CString::new(r#"["wai.tensor.nnc","wai.image.png"]"#).unwrap();
let mut idx = u32::MAX;
assert_eq!(unsafe { wai_envelope_select(env.as_ptr(), env.len(), all.as_ptr(), &mut idx) }, WAI_OK);
assert_eq!(idx, 1);
let only = CString::new(r#"["wai.tensor.nnc"]"#).unwrap();
assert_eq!(unsafe { wai_envelope_select(env.as_ptr(), env.len(), only.as_ptr(), &mut idx) }, WAI_ERR_NO_RENDITION);
let why = last_error();
assert!(why.contains("0: not_selectable") && why.contains("1: missing_capability"), "{why}");
}
/// SPEC §5, *Candidate capabilities*: an encoder MUST NOT emit one, as the
/// capability, the fallback, or a `WAI2` component, and the C packers refuse it.
#[test]
fn envelope_pack_refuses_a_candidate() {
let man = |cap: &str, fb: &str| CString::new(format!(
r#"{{"wai":"1.0","media":"binary","intent":"improve","model_requirement":{{"capability":"{cap}","fallback":{fb}}},"conditioning":{{"kind":"bytes"}},"target":null}}"#)).unwrap();
let payload = b"bytes";
for (cap, fb) in [("wai.splat.sog", "null"), ("wai.image.png", "\"wai.tensor.nnc\"")] {
let m = man(cap, fb);
let mut out = empty();
let rc = unsafe { wai_envelope_pack(m.as_ptr(), payload.as_ptr(), payload.len(), &mut out) };
assert_eq!(rc, WAI_ERR_INVALID_ARG, "{cap} / {fb}");
assert!(out.data.is_null());
assert!(last_error().contains("MUST NOT emit"), "{}", last_error());
}
let ok = man("wai.image.png", "null");
let mut out = empty();
assert_eq!(unsafe { wai_envelope_pack(ok.as_ptr(), payload.as_ptr(), payload.len(), &mut out) }, WAI_OK);
take(out);
let ptrs = [payload.as_ptr(), payload.as_ptr()];
let lens = [payload.len(), payload.len()];
for comps in [
r#"[{"capability":"wai.splat.sog","kind":"bytes"}]"#,
r#"[{"capability":"wai.video.av1","kind":"av1"},{"capability":"wai.tensor.nnc","kind":"bytes","role":"companion","companion_of":"wai.video.av1"}]"#,
] {
let multi = CString::new(format!(
r#"{{"wai":"1.1","media":"binary","intent":"improve","renditions":{comps},"target":null}}"#)).unwrap();
let n = serde_json::from_str::<serde_json::Value>(comps).unwrap().as_array().unwrap().len();
let mut out = empty();
let rc = unsafe { wai_envelope_pack_multi(multi.as_ptr(), ptrs.as_ptr(), lens.as_ptr(), n, &mut out) };
assert_eq!(rc, WAI_ERR_INVALID_ARG, "{comps}");
assert!(out.data.is_null());
}
}

120
wai/wai-rs/tests/ffi_neural_pin.rs Normal file
View file

@ -0,0 +1,120 @@
//! The neural C ABI reports a refused pin as `WAI_ERR_PRIOR` with its reason first
//! in `wai_last_error`, an unsound `replicate` under the strict flag as
//! `WAI_ERR_UNSOUND_REPLICATE`, and decodes a `WAI2` envelope through the
//! rendition whose pin the registry holds — all on the committed int_mlicpp
//! vector, with no model to fetch.
#![cfg(all(feature = "neural", feature = "ffi"))]
use std::ffi::{CStr, CString};
use std::path::{Path, PathBuf};
use std::ptr;
use wai::codecs::DeclaredTier;
use wai::container::{Prior, Rendition, Supersedes, Wai, WaiMulti};
use wai::ffi::*;
use wai::ffi_neural::*;
fn dir() -> PathBuf {
Path::new(env!("CARGO_MANIFEST_DIR")).join("tests/vectors/int_mlicpp")
}
fn last_error() -> String {
let p = wai_last_error();
if p.is_null() { String::new() } else { unsafe { CStr::from_ptr(p) }.to_string_lossy().into_owned() }
}
fn decoded() -> WaiDecoded {
WaiDecoded {
kind: 0, index: 0, data: WaiBuffer { data: ptr::null_mut(), len: 0, cap: 0 },
width: 0, height: 0, sample_rate: 0, n_frames: 0, fps_x_1000: 0,
}
}
fn fnv_of(d: &WaiDecoded) -> String {
let rgb = unsafe { std::slice::from_raw_parts(d.data.data, d.data.len) };
format!("{:016x}", wai::int_hyper_synth::fnv1a(rgb))
}
#[test]
fn tampered_model_is_wai_err_prior() {
let env = std::fs::read(dir().join("stream.wai")).unwrap();
let tampered = std::env::temp_dir().join(format!("wai-ffi-tampered-{}.wmb", std::process::id()));
let mut b = std::fs::read(dir().join("model.wmb")).unwrap();
b[20] ^= 1;
std::fs::write(&tampered, &b).unwrap();
let path = CString::new(tampered.to_str().unwrap()).unwrap();
let mut rgb = WaiBuffer { data: ptr::null_mut(), len: 0, cap: 0 };
let (mut w, mut h) = (0u32, 0u32);
let code = unsafe { wai_neural_decode_image(env.as_ptr(), env.len(), path.as_ptr(), &mut rgb, &mut w, &mut h) };
std::fs::remove_file(&tampered).ok();
assert_eq!(code, WAI_ERR_PRIOR);
assert!(last_error().starts_with("prior_not_held: "), "{}", last_error());
// The true model decodes.
let path = CString::new(dir().join("model.wmb").to_str().unwrap()).unwrap();
assert_eq!(unsafe { wai_neural_decode_image(env.as_ptr(), env.len(), path.as_ptr(), &mut rgb, &mut w, &mut h) }, WAI_OK);
assert_eq!((w, h), (64, 64));
unsafe { wai_buffer_free(rgb) };
}
/// A failed decode leaves `*out` empty, whatever it held before: a caller that
/// frees `out->data` after any status frees nothing it did not get.
#[test]
fn failed_decode_ex_clears_out() {
let env = std::fs::read(dir().join("stream.wai")).unwrap();
let empty = CString::new("{}").unwrap();
let mut out = WaiDecoded {
kind: 7, index: 9, data: WaiBuffer { data: 0x5a5a as *mut u8, len: 5, cap: 5 },
width: 1, height: 1, sample_rate: 1, n_frames: 1, fps_x_1000: 1,
};
let code = unsafe { wai_neural_decode_ex(env.as_ptr(), env.len(), empty.as_ptr(), 0, &mut out) };
assert_ne!(code, WAI_OK);
assert_eq!((out.kind, out.index, out.width, out.height), (0, 0, 0, 0));
assert!(out.data.data.is_null() && out.data.len == 0 && out.data.cap == 0);
unsafe { wai_buffer_free(out.data) };
// A bad argument after the null check clears it too.
let mut out2 = WaiDecoded { kind: 3, ..decoded() };
let code = unsafe { wai_neural_decode_ex(env.as_ptr(), env.len(), ptr::null(), 0, &mut out2) };
assert_eq!(code, WAI_ERR_INVALID_ARG);
assert_eq!(out2.kind, 0);
}
#[test]
fn strict_replicate_is_wai_err_unsound_replicate() {
let mut env = Wai::from_bytes(&std::fs::read(dir().join("stream.wai")).unwrap()).unwrap();
env.manifest.conditioning.prior = None;
let bytes = env.to_bytes().unwrap();
let reg = CString::new(format!(r#"{{"wai.neural.int_mlicpp":["{}"]}}"#, dir().join("model.wmb").display())).unwrap();
let mut out = decoded();
let code = unsafe { wai_neural_decode_ex(bytes.as_ptr(), bytes.len(), reg.as_ptr(), WAI_DECODE_STRICT, &mut out) };
assert_eq!(code, WAI_ERR_UNSOUND_REPLICATE);
assert!(last_error().contains("replicate_unpinned"), "{}", last_error());
// Without the strict flag the unpinned envelope decodes.
assert_eq!(unsafe { wai_neural_decode_ex(bytes.as_ptr(), bytes.len(), reg.as_ptr(), 0, &mut out) }, WAI_OK);
assert_eq!((out.kind, out.width, out.height), (WAI_DECODED_IMAGE, 64, 64));
unsafe { wai_buffer_free(out.data) };
}
#[test]
fn wai2_decode_ex_picks_the_held_pin() {
let pl = Wai::from_bytes(&std::fs::read(dir().join("stream.wai")).unwrap()).unwrap().payload;
let committed = wai::pin::sha256_hex(&std::fs::read(dir().join("model.wmb")).unwrap());
let det = "int-rans/p16+int-mlicpp/i64";
let env = WaiMulti::new("image", "replicate", serde_json::Value::Null, vec![
Rendition::media("wai.neural.int_mlicpp", "int_mlicpp", pl.clone())
.with_prior(Prior::new("next", "5c".repeat(32), det).with_tier(DeclaredTier::DecodeEquivalence)
.with_supersedes(Supersedes::new(committed.clone()))),
Rendition::media("wai.neural.int_mlicpp", "int_mlicpp", pl)
.with_prior(Prior::new("committed", committed, det).with_tier(DeclaredTier::DecodeEquivalence)),
]).to_bytes().unwrap();
let reg = CString::new(format!(r#"{{"wai.neural.int_mlicpp":["{}"]}}"#, dir().join("model.wmb").display())).unwrap();
let mut out = decoded();
let code = unsafe { wai_neural_decode_ex(env.as_ptr(), env.len(), reg.as_ptr(), WAI_DECODE_STRICT, &mut out) };
assert_eq!(code, WAI_OK, "{}", last_error());
assert_eq!((out.index, out.kind), (1, WAI_DECODED_IMAGE));
assert_eq!(fnv_of(&out), "7c0b9a4b3423d775");
unsafe { wai_buffer_free(out.data) };
// With no set held, nothing is picked.
let empty = CString::new("{}").unwrap();
let mut out = decoded();
assert_eq!(unsafe { wai_neural_decode_ex(env.as_ptr(), env.len(), empty.as_ptr(), 0, &mut out) }, WAI_ERR_NO_RENDITION);
}

4
wai/wai-rs/tests/generate.rs
View file

@ -9,6 +9,9 @@ use std::path::PathBuf;
use wai::generate::{fnv1a, generate, SpectralModel};
#[path = "support/sha256_sidecar.rs"]
mod sha256_sidecar;
fn dir() -> PathBuf {
PathBuf::from(env!("CARGO_MANIFEST_DIR")).join("tests/vectors/generate")
}
@ -38,6 +41,7 @@ fn generation_is_deterministic_and_pinned() {
let got = hash(&rgb);
println!("generate.spectral: {h}x{w} '{prompt}' seed={seed} steps={steps} -> fnv1a {got}");
assert_eq!(got, pin["rgb_hash"].as_str().unwrap(), "pinned generation must reproduce the committed hash");
sha256_sidecar::one(&dir(), "generate", "RGB8: rows top to bottom, pixels left to right, R, G, B bytes", &rgb);
// 2. DETERMINISM: regenerating the same inputs is byte-identical.
assert_eq!(generate(&m, prompt, seed, steps, h, w), rgb, "regeneration must be byte-identical");

94
wai/wai-rs/tests/int_kernels_vectors.rs
View file

@ -15,8 +15,15 @@
use std::path::{Path, PathBuf};
use serde_json::Value;
use wai::int_kernels::{with_policy, Policy};
#[path = "support/sha256_sidecar.rs"]
mod sha256_sidecar;
const RGB8: &str = "RGB8: rows top to bottom, pixels left to right, R, G, B bytes";
const POLICIES: [Policy; 5] = [Policy::reference(), Policy::fast(1), Policy::fast(2), Policy::fast(5), Policy::fast(0)];
fn vroot() -> PathBuf {
@ -47,6 +54,7 @@ fn int_hyper_tiny_conditional_codec() {
let h_s = IntModel::from_bytes(&std::fs::read(d.join("tiny_h_s.bin")).unwrap()).unwrap();
let (rgb, _, _) = same_under_every_policy("int_hyper tiny", || decode_conditional(&g_s, &h_s, &ent, &bs).unwrap());
assert_eq!(format!("{:016x}", fnv1a(&rgb)), want);
sha256_sidecar::one(&d, "tiny", RGB8, &rgb);
}
#[test]
@ -68,6 +76,8 @@ fn int_audio_clip() {
let bs = jload(&d.join("audio_bitstream.json"));
let pcm = same_under_every_policy("int_audio", || decode(&model, &bs).unwrap());
assert_eq!(format!("{:016x}", pcm_fnv1a(&pcm)), jload(&d.join("audio.json"))["pcm_fnv1a"].as_str().unwrap());
let s16le: Vec<u8> = pcm.iter().flat_map(|s| s.to_le_bytes()).collect();
sha256_sidecar::one(&d, "audio", "PCM: signed 16-bit little-endian samples in output order, channels interleaved", &s16le);
}
#[test]
@ -80,6 +90,7 @@ fn int_splat_cloud() {
let attr = same_under_every_policy("int_splat", || decode(&gs, &bs).unwrap().attr);
let hash = wai::int_hyper_synth::fnv1a(&attr);
assert_eq!(format!("{hash:016x}"), bs["attr_hash"].as_str().unwrap());
sha256_sidecar::one(&d, "splat", "DecodedSplat::attr: the decoded attribute tensor's bytes in decoder output order", &attr);
}
#[test]
@ -97,6 +108,7 @@ fn int_world_clips() {
let frames = same_under_every_policy("int_world", || decode(&t, &gs, &z0, &actions, c, lat).frames);
let clip = wai::int_world::WorldClip { frames, frame_px: 0 };
assert_eq!(format!("{:016x}", clip.clip_hash()), j["clip_hash"].as_str().unwrap());
sha256_sidecar::frames(&d, "world", RGB8, &clip.frames);
// two agents
let d = vroot().join("int_world2");
@ -121,6 +133,7 @@ fn int_world_clips() {
for (f, w) in frames.iter().zip(want) {
assert_eq!(format!("{:016x}", wai::int_hyper_synth::fnv1a(f)), w.as_str().unwrap());
}
sha256_sidecar::frames(&d, "world2", RGB8, &frames);
}
#[test]
@ -143,6 +156,7 @@ fn int_video_neural_keyframe_and_real_footage() {
for (f, w) in frames.iter().zip(want) {
assert_eq!(format!("{:016x}", fnv1a(f)), w.as_str().unwrap());
}
sha256_sidecar::frames(&iv, "video_nk", RGB8, &frames);
// integer block-MC only (no learned kernels) — unaffected, checked for completeness
let rbs = jload(&iv.join("video_real_bitstream.json"));
let frames = same_under_every_policy("real-footage video", || decode(&rbs).unwrap().frames);
@ -152,6 +166,15 @@ fn int_video_neural_keyframe_and_real_footage() {
for (f, w) in frames.iter().zip(want) {
assert_eq!(format!("{:016x}", fnv1a(f)), w.as_str().unwrap());
}
sha256_sidecar::frames(&iv, "video_real", RGB8, &frames);
// The plain block-motion clip (wai.video.int_motion), for completeness.
let pbs = jload(&iv.join("int_video_bitstream.json"));
let frames = decode(&pbs).unwrap().frames;
let want = jload(&iv.join("int_video.json"));
for (f, w) in frames.iter().zip(want["frame_hashes"].as_array().unwrap()) {
assert_eq!(format!("{:016x}", fnv1a(f)), w.as_str().unwrap());
}
sha256_sidecar::frames(&iv, "int_video", RGB8, &frames);
}
#[test]
@ -170,6 +193,7 @@ fn neural_conformance_int_synth_corpus() {
h.update(&rgb);
let got: String = h.finalize().as_bytes().iter().map(|b| format!("{b:02x}")).collect();
assert_eq!(got, spec["rgb_hash"].as_str().unwrap(), "{name}");
sha256_sidecar::one(&dir, name, RGB8, &rgb);
if let Ok(r) = std::fs::read(vd.join("ref_rgb.bin")) {
assert_eq!(rgb, r, "{name}: must equal the python integer-forward reference");
}
@ -177,3 +201,73 @@ fn neural_conformance_int_synth_corpus() {
}
assert!(n >= 4, "expected the four committed int_synth vectors, found {n}");
}
/// SPEC §7 *Digests*: every committed vector or corpus that states a digest of the
/// decoded output of a capability that decodes against a parameter set states its
/// SHA-256, in a `sha256.json` sidecar that names the buffer it covers. The test
/// enumerates every `wai/*-conformance/` directory and every `tests/vectors/`
/// directory: each has a sidecar, or is listed below with the reason it has none.
#[test]
fn every_decoded_output_states_a_sha256() {
const NATIVE: &str = "a WAI-native capability that decodes against no parameter set: its state and \
rendered-output digests are the algorithm its extension names (SPEC §7 *Digests*)";
let excluded: &[(&str, &str)] = &[
("audio-conformance", NATIVE), ("avatar-conformance", NATIVE), ("feed-conformance", NATIVE),
("film-conformance", NATIVE), ("haptic-conformance", NATIVE), ("score-conformance", NATIVE),
("volumetric-conformance", NATIVE), ("world-conformance", NATIVE),
("quantum-conformance", NATIVE), ("quantum-ops-conformance", NATIVE),
("claim-conformance", "receipt claims: receipt content, no decoded output"),
("oer-conformance", "OER/2 receipts: receipt content, no decoded output"),
("pin-conformance", "parameter-set pins: every digest there is a SHA-256 of a parameter set"),
("byte-exact-conformance", "goldens over tests/vectors, whose sidecars state the SHA-256"),
("vectors/int_mlicpp", "expected.json states rgb_sha256 (RGB8)"),
("vectors/jpegai", "inputs only; the decoded pixels' SHA-256 is pinned by the JPEG AI e2e job"),
];
let wai = PathBuf::from(env!("CARGO_MANIFEST_DIR")).join("..");
let mut dirs: Vec<(String, PathBuf)> = Vec::new();
for e in std::fs::read_dir(&wai).unwrap().flatten() {
let n = e.file_name().to_string_lossy().into_owned();
if n.ends_with("-conformance") && e.path().is_dir() {
dirs.push((n, e.path()));
}
}
for e in std::fs::read_dir(vroot()).unwrap().flatten() {
if e.path().is_dir() {
dirs.push((format!("vectors/{}", e.file_name().to_string_lossy()), e.path()));
}
}
dirs.sort();
let mut with_sidecar = 0;
for (name, path) in &dirs {
if let Some((_, why)) = excluded.iter().find(|(n, _)| n == name) {
assert!(!why.is_empty(), "{name}: an exclusion names its reason");
continue;
}
let side = path.join("sha256.json");
let doc = jload(&side);
let entries = doc.as_object().unwrap_or_else(|| panic!("{}: an object", side.display()));
assert!(!entries.is_empty(), "{}: no entries", side.display());
for (k, e) in entries {
assert!(e["buffer"].as_str().is_some_and(|b| !b.is_empty()), "{name}/{k}: names its buffer");
let mut digests = 0;
for (field, v) in e.as_object().unwrap() {
if !field.ends_with("sha256") {
continue;
}
let all: Vec<&Value> = match v { Value::Array(a) => a.iter().collect(), other => vec![other] };
for d in all {
let d = d.as_str().unwrap_or_default();
assert!(d.len() == 64 && d.bytes().all(|b| b.is_ascii_digit() || (b'a'..=b'f').contains(&b)),
"{name}/{k}/{field}: {d} is not 64 lowercase hexadecimal digits");
digests += 1;
}
}
assert!(digests > 0, "{name}/{k}: states a SHA-256");
}
with_sidecar += 1;
}
for (n, _) in excluded {
assert!(dirs.iter().any(|(d, _)| d == n), "exclusion {n} names no directory");
}
assert!(with_sidecar > 0);
}

99
wai/wai-rs/tests/int_mlicpp.rs
View file

@ -234,3 +234,102 @@ fn damaging_the_committed_payload_never_panics() {
}
assert!(decoded > 0 && refused > 0, "{decoded} decoded, {refused} refused");
}
// ---- the parameter-set pin over the committed vector (SPEC §3.1, §7.1) ----
const LISTING_SHA256: &str = "85a05b0dac15566c5a657ddeee253eaa9f54ce852ae04924f5b04151901d9d1b";
const STREAM_SET_SHA256: &str = "8f23e85cd3d2690817c8935c2473dc2ba3af3cb302e3dca3cf3feb60f1c263d0";
const RGB_SHA256: &str = "45040d17b4178b91a0547b435e57a50fa1021101e290a94d1c57e6fbb4583a10";
fn scratch(tag: &str) -> PathBuf {
let d = std::env::temp_dir().join(format!("wai-int-mlicpp-{tag}-{}", std::process::id()));
std::fs::create_dir_all(&d).unwrap();
d
}
/// The set-form twin of `stream.wai` (`tools/wai_int_mlicpp_set_form.py`): the same
/// payload, pinned through the listing of the one-file set. The pin verifies against
/// the committed listing and bundle, and the decode from the very bytes it verified
/// is the pinned image. A flipped listing is not held; a flipped bundle under an
/// intact listing is a part mismatch; a listing that names another file verifies, and
/// the decode then needs a file the set does not contain.
#[test]
fn set_form_pin_resolves_and_decodes_the_same_image() {
use wai::pin::{sha256_hex, verify, Held, READABLE_FORMS};
let set_bytes = read("stream_set.wai");
assert_eq!(sha256_hex(&set_bytes), STREAM_SET_SHA256);
assert_eq!(sha256_hex(&read("model.sha256")), LISTING_SHA256);
let set = Wai::from_bytes(&set_bytes).unwrap();
let legacy = envelope();
assert_eq!(set.payload, legacy.payload, "only the pin differs");
let pin = set.prior().expect("a pin");
assert_eq!((pin.sha256.as_str(), pin.form()), (LISTING_SHA256, Some("wai.prior.set")));
assert_eq!(set.replicate_refusal(), None, "a set-form pin declaring decode-equivalence backs replicate");
let v = verify("wai.neural.int_mlicpp", pin, &[Held::Listing(dir().join("model.sha256"))], READABLE_FORMS)
.expect("the committed listing and bundle verify");
let m = IntMlicModel::from_bundle(v.file("model.wmb").unwrap()).unwrap();
let img = decode(&m, &set.payload).unwrap();
assert_eq!(hex(fnv1a(&img.rgb)), RGB_FNV1A);
assert_eq!(sha256_hex(&img.rgb), RGB_SHA256);
// Twins on temporary copies.
let code = |d: &PathBuf| verify("wai.neural.int_mlicpp", pin, &[Held::Listing(d.join("model.sha256"))], READABLE_FORMS)
.map(|_| "ok").unwrap_or_else(|e| e.code());
let flipped_listing = scratch("flipped-listing");
let mut l = read("model.sha256");
l[0] ^= 1;
std::fs::write(flipped_listing.join("model.sha256"), &l).unwrap();
std::fs::write(flipped_listing.join("model.wmb"), read("model.wmb")).unwrap();
assert_eq!(code(&flipped_listing), "prior_not_held");
let flipped_bundle = scratch("flipped-bundle");
let mut b = read("model.wmb");
let at = b.len() - 2;
b[at] ^= 1;
std::fs::write(flipped_bundle.join("model.sha256"), read("model.sha256")).unwrap();
std::fs::write(flipped_bundle.join("model.wmb"), &b).unwrap();
assert_eq!(code(&flipped_bundle), "part_mismatch");
let other = scratch("other-name");
std::fs::write(other.join("other.wmb"), read("model.wmb")).unwrap();
let other_listing = wai::pin::write_listing(&other, &["other.wmb".to_string()]).unwrap();
std::fs::write(other.join("model.sha256"), &other_listing).unwrap();
let mut other_pin = pin.clone();
other_pin.sha256 = sha256_hex(&other_listing);
let v = verify("wai.neural.int_mlicpp", &other_pin, &[Held::Listing(other.join("model.sha256"))], READABLE_FORMS)
.expect("a listing of other.wmb verifies");
assert_eq!(v.file("model.wmb").unwrap_err().code(), "set_incomplete");
for d in [flipped_listing, flipped_bundle, other] {
std::fs::remove_dir_all(d).ok();
}
}
/// SPEC §7.1: one envelope, the same payload twice, pinned to two parameter sets — a
/// successor (which names the committed set in `supersedes`) and the committed set.
/// A sink holding only the committed bundle skips the first entry
/// (`digest_superseded`), picks the second, and decodes the pinned image.
#[test]
fn wai2_pins_select_the_held_model() {
use wai::codecs::DeclaredTier;
use wai::container::{Prior, Rendition, Supersedes, WaiMulti};
use wai::pin::{verify, Held, READABLE_FORMS};
let pl = envelope().payload;
let successor = "5ccc".repeat(16);
let m = WaiMulti::new("image", "replicate", serde_json::Value::Null, vec![
Rendition::media("wai.neural.int_mlicpp", "int_mlicpp", pl.clone())
.with_prior(Prior::new("next", successor, "int-rans/p16+int-mlicpp/i64")
.with_tier(DeclaredTier::DecodeEquivalence)
.with_supersedes(Supersedes::new(MODEL_SHA256))),
Rendition::media("wai.neural.int_mlicpp", "int_mlicpp", pl)
.with_prior(Prior::new("committed", MODEL_SHA256, "int-rans/p16+int-mlicpp/i64")
.with_tier(DeclaredTier::DecodeEquivalence)),
]);
let m = WaiMulti::from_bytes(&m.to_bytes().unwrap()).unwrap();
let held = [Held::File(dir().join("model.wmb"))];
let p = m.pick(|_| true, |cap, prior| verify(cap, prior, &held, READABLE_FORMS)).expect("a rendition");
assert_eq!(p.index, 1);
assert_eq!(p.skipped.iter().map(|(i, s)| (*i, s.code())).collect::<Vec<_>>(), [(0, "digest_superseded")]);
let bundle = p.verified.as_ref().unwrap().one().unwrap();
let img = decode(&IntMlicModel::from_bundle(bundle).unwrap(), p.payload).unwrap();
assert_eq!(hex(fnv1a(&img.rgb)), RGB_FNV1A);
assert_eq!(m.replicate_refusal(1), None);
}

Some files were not shown because too many files have changed in this diff Show more