Transaction-Science/open-standards
Watch
1
0
Fork
You've already forked open-standards
 0
Code Issues Pull requests Projects Releases 17 Packages Wiki Activity Actions 137

wai: one registry table, candidate capabilities, public-text checks #131

Merged
dcharlot merged 5 commits from wai/land-12-registry-and-text into main 2026-10-01 14:50:01 -04:00
Conversation 0 Commits 5 Files changed 139 +9063 -1878
139 changed files with 9063 additions and 1878 deletions
Download patch file Download diff file Expand all files Collapse all files

87
.github/workflows/no-vanity-metrics.yml vendored
View file

@ -6,20 +6,67 @@ name: no-vanity-metrics
# permanent sync debt with the websites; the open-standards posture is
# "describe the shape, not the size." See CONTRIBUTING.md for the
# rationale and the opt-out marker.
#
# Two more jobs hold WAI's public text (wai/) to its writing rules
# (wai/CONTRIBUTING.md, *Public text*): no comparative or promotional phrasing
# (a public phrase list), and no company, vendor, customer or product names
# (patterns kept private, in the WAI_NAME_DENYLIST secret; prints locations
# only). Both checks fail closed: a grep error or a pattern that does not compile
# fails the job, and scripts/test-public-text-checks.sh proves it on every run.
# The secret is given to one step, on a push to main only: a pull request runs
# code from its own branch, and a step of it that held the list could print it to
# a public log. Without the secret, a push to main fails; a pull request shows a
# failed (continue-on-error) step saying the name check did not run.
on:
pull_request:
paths:
- '**/*.md'
# The public-text checks below scan WAI's code comments and data too.
- 'wai/**/*.rs'
- 'wai/**/*.toml'
- 'wai/**/*.py'
- 'wai/**/*.sh'
- 'wai/**/*.js'
- 'wai/**/*.mjs'
- 'wai/**/*.json'
- 'wai/**/*.html'
- 'wai/**/*.css'
- 'wai/**/*.ttl'
- 'wai/**/*.h'
- 'wai/**/*.c'
- 'wai/**/*.txt'
- 'wai/**/.gitignore'
- '.github/workflows/no-vanity-metrics.yml'
- 'scripts/check-no-vanity-metrics.sh'
- 'scripts/check-comparative-prose.sh'
- 'scripts/check-public-names.sh'
- 'scripts/test-public-text-checks.sh'
push:
branches:
- main
paths:
- '**/*.md'
# The public-text checks below scan WAI's code comments and data too.
- 'wai/**/*.rs'
- 'wai/**/*.toml'
- 'wai/**/*.py'
- 'wai/**/*.sh'
- 'wai/**/*.js'
- 'wai/**/*.mjs'
- 'wai/**/*.json'
- 'wai/**/*.html'
- 'wai/**/*.css'
- 'wai/**/*.ttl'
- 'wai/**/*.h'
- 'wai/**/*.c'
- 'wai/**/*.txt'
- 'wai/**/.gitignore'
- '.github/workflows/no-vanity-metrics.yml'
- 'scripts/check-no-vanity-metrics.sh'
- 'scripts/check-comparative-prose.sh'
- 'scripts/check-public-names.sh'
- 'scripts/test-public-text-checks.sh'
jobs:
scan:
@ -31,3 +78,43 @@ jobs:
- name: Run scanner
run: bash scripts/check-no-vanity-metrics.sh
wai-comparative-prose:
name: WAI public text names no comparison
runs-on: linux
steps:
- name: Checkout
uses: actions/checkout@v4
- name: The two public-text checks fail closed (self-test)
run: bash scripts/test-public-text-checks.sh
- name: Run the comparative-prose check
run: bash scripts/check-comparative-prose.sh
wai-public-names:
name: WAI public text names no company or product
runs-on: linux
steps:
- name: Checkout
uses: actions/checkout@v4
# The list reaches this step only, and only on a push to main, whose code
# has been merged. Without the list the check cannot run, and that fails.
- name: Run the public-name check
if: github.event_name != 'pull_request'
env:
# Newline-separated patterns, never committed: a public list would
# publish the names it keeps out.
WAI_NAME_DENYLIST: ${{ secrets.WAI_NAME_DENYLIST }}
run: |
if [ -z "$WAI_NAME_DENYLIST" ]; then
echo "::error::WAI_NAME_DENYLIST is not set: the public-name check cannot run"
exit 1
fi
bash scripts/check-public-names.sh
# A pull request never receives the list. The step fails visibly without
# failing the job; the push to main runs the check.
- name: The public-name check did not run on this pull request
if: github.event_name == 'pull_request'
continue-on-error: true
run: |
echo "::warning::the public-name check does not run on a pull request (its pattern list is never given to pull-request code); it runs on the push to main"
exit 1

14
.github/workflows/wai-byte-equality.yml vendored
View file

@ -2,8 +2,8 @@ name: wai cross-architecture byte-equality
# The `replicate` determinism contract, checked on real silicon. WAI claims its
# integer-exact decode paths (rANS entropy coder, integer fixed-point synthesis)
# produce BYTE-IDENTICAL output on every machine. This matrix proves it: five
# OS/arch combinations decode the same fixtures and assert the same golden
# produce BYTE-IDENTICAL output on every machine. This matrix proves it: every
# OS/arch leg of the matrix below decodes the same fixtures and asserts the same golden
# bytes — rANS streams produced by CompressAI's upstream encoder, and an
# int_synth output hash produced on the reference machine (macOS arm64). One
# diverging byte anywhere fails the suite.
@ -77,16 +77,18 @@ jobs:
strategy:
fail-fast: false
matrix:
# Self-hosted fleet labels, not GitHub-hosted images. macos-x86_64 is
# Self-hosted fleet labels, not hosted images. macos-x86_64 is
# dropped: no Intel-Mac runner exists in the fleet, and by the same
# argument the cargo-classes matrix already makes, it adds no arch
# coverage — x86_64 is held by linux-x86_64 and windows-x86_64, arm64
# by linux-aarch64 and the macos-arm64 reference.
# coverage — x86_64 is held by linux-x86_64, arm64 by linux-aarch64
# and the macos-arm64 reference.
# Windows x86_64 leg removed 2026-09-30: runner offline. Restore by
# re-adding `- { os: windows, label: windows-x86_64 }` here once the
# runner is back online.
include:
- { os: linux, label: linux-x86_64 }
- { os: linux-arm64, label: linux-aarch64 }
- { os: macos, label: macos-arm64 (reference) }
- { os: windows, label: windows-x86_64 }
steps:
- name: Checkout
uses: actions/checkout@v4

102
.github/workflows/wai-conformance.yml vendored
View file

@ -2,10 +2,10 @@ name: WAI conformance
# WAI's load-bearing claim is bit-exact reconstruction on every machine,
# no tolerance parameter. This workflow enforces it mechanically: every
# cargo-class corpus — and the one neural decoder that can make the same
# promise, the integer-synthesis path (i64-only, no float ONNX) — is
# re-derived on a four-platform matrix (x86-64 + arm64 across Linux, macOS,
# Windows) and must reproduce its frozen BLAKE3 hashes; and the integer
# cargo-class corpus — and the integer-exact neural decoders, which make the
# same promise (the registry's NeuralIntegerExact class: i64-only, no float
# ONNX) — is re-derived on every OS/arch leg of the matrix below (x86-64 and
# arm64) and must reproduce its frozen BLAKE3 hashes; and the integer
# byte-equality goldens (rANS / int_synth / JPEG-AI mANS, generated once on
# a reference machine) must decode identically everywhere. All of it is pure
# Rust (no C codec chain, no ONNX Runtime), so the matrix needs no system
@ -21,6 +21,15 @@ on:
- 'wai/wai-quantum/**'
- 'wai/*-conformance/**'
- 'wai/byte-exact-conformance/**'
# The registry export and the two library tests that read files outside
# wai-rs (the codec glossary, SPEC.md) must fire this workflow too: a
# hand edit of an exported file, of SPEC §5 or of the glossary is checked
# here.
- 'wai/registry/**'
- 'wai/vocab/**'
- 'wai/glossary/**'
- 'wai/SPEC.md'
- 'wai/REGISTRY.md'
- '.github/workflows/wai-conformance.yml'
pull_request:
paths:
@ -30,12 +39,23 @@ on:
- 'wai/wai-quantum/**'
- 'wai/*-conformance/**'
- 'wai/byte-exact-conformance/**'
# The registry export and the two library tests that read files outside
# wai-rs (the codec glossary, SPEC.md) must fire this workflow too: a
# hand edit of an exported file, of SPEC §5 or of the glossary is checked
# here.
- 'wai/registry/**'
- 'wai/vocab/**'
- 'wai/glossary/**'
- 'wai/SPEC.md'
- 'wai/REGISTRY.md'
- '.github/workflows/wai-conformance.yml'
# No global `shell:` default. The Windows runner is a host executor whose
# `bash` is the WSL stub (a different OS), so shell-specific steps below pick
# their shell per-OS: bash on unix, pwsh on Windows. Bare `cargo …` steps take
# each runner's native default, which runs cargo identically either way.
# each runner's native default, which runs cargo identically either way. The
# Windows legs are out of the matrices for now (see the comment on each matrix);
# their pwsh steps stay, inactive, so restoring a leg is a one-line change.
jobs:
cargo-classes:
@ -44,12 +64,13 @@ jobs:
strategy:
fail-fast: false
matrix:
# Self-hosted fleet labels. Both architectures across all three OS
# families: x86_64 by linux + windows, arm64 by linux-arm64 + macos —
# what matters for the Fx integer floor is the arch, not the OS. No
# Intel-Mac runner exists in the fleet, and it would add no arch
# coverage anyway.
os: [linux, linux-arm64, macos, windows]
# Self-hosted fleet labels. Both architectures: x86_64 by linux, arm64
# by linux-arm64 + macos — what matters for the Fx integer floor is the
# arch, not the OS. No Intel-Mac runner exists in the fleet, and it
# would add no arch coverage anyway.
# Windows x86_64 leg removed 2026-09-30: runner offline. Restore by
# re-adding `windows` here once the runner is back online.
os: [linux, linux-arm64, macos]
steps:
- uses: actions/checkout@v5
@ -125,15 +146,32 @@ jobs:
# tests/int_mlicpp_preflight.rs (its own binary: it installs a counting global
# allocator) checks that a header the model cannot fit is refused before the
# decode allocates anything sized from it.
# tests/int_audio.rs and tests/int_splat.rs decode the committed wai.audio.int_codec and
# wai.splat.int_codec vectors against the hashes the Python integer reference computed at
# export, and check each vector's metadata. Cargo.toml declares them, and the other
# feature-gated targets named here, with required-features, so if this step ever loses a
# feature, cargo stops with an error instead of compiling an empty binary that reports ok.
- name: Fast integer kernels ≡ reference kernels (overflow checks on)
working-directory: wai/wai-rs
run: cargo test --no-default-features --features neural_int,video_int,audio_int,world_int,splat_int --lib --test int_kernels_vectors --test int_kernels_geometry --test int_mlicpp --test int_mlicpp_preflight
run: cargo test --no-default-features --features neural_int,video_int,audio_int,world_int,splat_int --lib --test int_kernels_vectors --test int_kernels_geometry --test int_mlicpp --test int_mlicpp_preflight --test int_audio --test int_splat
- name: Fast integer kernels ≡ reference kernels (release, overflow checks off)
working-directory: wai/wai-rs
run: cargo test --release --no-default-features --features neural_int,video_int,audio_int,world_int,splat_int --lib --test int_kernels_vectors --test int_kernels_geometry --test int_mlicpp --test int_mlicpp_preflight
- name: Carriage conformance — every registered capability round-trips byte-exact
run: cargo test --release --no-default-features --features neural_int,video_int,audio_int,world_int,splat_int --lib --test int_kernels_vectors --test int_kernels_geometry --test int_mlicpp --test int_mlicpp_preflight --test int_audio --test int_splat
# carriage: every registered capability round-trips byte-exact. cli_wrap_intent and
# cli_registry run the `wai` binary, which builds without the codecs feature: the
# intent rule of `wai wrap`, its refusal of candidate capabilities (SPEC §5), its media
# class, and `wai registry export --check` on the tree and on a drifted copy.
- name: Carriage conformance and the wai CLI (wrap, registry export)
working-directory: wai/wai-rs
run: cargo test --no-default-features --test carriage
run: cargo test --no-default-features --test carriage --test cli_wrap_intent --test cli_registry
# The machine-readable registry (wai/registry/, wai/vocab/v2.*, the table in
# wai/REGISTRY.md) is generated from wai-rs/src/codecs/registry.rs. Regenerate it in
# memory and fail on any difference; the message says how to regenerate. One leg: the
# export is the same text on every platform.
- name: The registry export is up to date (linux)
if: matrix.os == 'linux'
working-directory: wai/wai-rs
run: cargo run -q --no-default-features --bin wai -- registry export --check
# The corpus-verify loop is genuinely shell-specific (a helper that groups
# log output and propagates the first non-zero exit), so it ships once per
@ -200,18 +238,40 @@ jobs:
v oer wai_oer_vectors ../oer-conformance
v claims wai_claim_vectors ../claim-conformance
# The generated v2 vocabulary (wai/vocab/v2.*) is checked as a consumer uses it:
# sample manifests, typed and untyped, WAI1 and WAI2, are expanded through the
# JSON-LD context exactly as written on the wire and validated against the SHACL
# shapes, each with the result it must give (a candidate in any place is
# rejected; a media value no revision lists is accepted). One leg: RDF tooling,
# not arithmetic. The tool versions are pinned.
vocab-shapes:
name: v2 vocabulary · manifests validate as stated
runs-on: linux
steps:
- uses: actions/checkout@v5
- uses: actions/setup-python@v5
with:
python-version: '3.11'
- name: Install the RDF tools
shell: bash
run: python -m pip install rdflib==7.6.0 PyLD==3.3.0 pyshacl==0.40.1
- name: Sample manifests against vocab/v2.shapes.ttl
shell: bash
run: python wai/vocab/check_shapes.py
byte-exact:
name: cross-arch byte-equality · ${{ matrix.os }}
runs-on: ${{ matrix.os }}
strategy:
fail-fast: false
matrix:
# Self-hosted fleet labels. Both architectures across all three OS
# families: x86_64 by linux + windows, arm64 by linux-arm64 + macos —
# what matters for the Fx integer floor is the arch, not the OS. No
# Intel-Mac runner exists in the fleet, and it would add no arch
# coverage anyway.
os: [linux, linux-arm64, macos, windows]
# Self-hosted fleet labels. Both architectures: x86_64 by linux, arm64
# by linux-arm64 + macos — what matters for the Fx integer floor is the
# arch, not the OS. No Intel-Mac runner exists in the fleet, and it
# would add no arch coverage anyway.
# Windows x86_64 leg removed 2026-09-30: runner offline. Restore by
# re-adding `windows` here once the runner is back online.
os: [linux, linux-arm64, macos]
steps:
- uses: actions/checkout@v5

4
.github/workflows/wai-energy-receipts.yml vendored
View file

@ -44,8 +44,8 @@ name: WAI energy receipts
# - build wai-web, or the wasm32-wasip2 component of wai-quantum-serve.
# - verify the conformance corpora; wai-conformance.yml does that.
# - run on Windows. The pinned bytes are integers and hashes and make no
# OS-specific claim, and the conformance matrix covers Windows for the
# byte-exact decode.
# OS-specific claim; the conformance matrix has no Windows leg either (its
# runner is offline).
#
# It triggers only on changes under the paths listed below; a change to
# wai-web alone does not trigger it.

26
.github/workflows/wai-sota-vectors.yml vendored
View file

@ -107,6 +107,8 @@ on:
- 'wai/tools/wai_sota_*'
- 'wai/tools/wai_int_video_sota_keyframe_encode.py'
- 'wai/tools/requirements-sota-int.txt'
# The wasm32 decode of the same vectors (the last steps below).
- 'wai/wai-web/**'
- '.github/workflows/wai-sota-vectors.yml'
pull_request:
paths:
@ -127,6 +129,8 @@ on:
- 'wai/tools/wai_sota_*'
- 'wai/tools/wai_int_video_sota_keyframe_encode.py'
- 'wai/tools/requirements-sota-int.txt'
# The wasm32 decode of the same vectors (the last steps below).
- 'wai/wai-web/**'
- '.github/workflows/wai-sota-vectors.yml'
# Weekly, independent of changes: the job depends on artifacts fetched from
# outside the repository (the checkpoint, the pinned wheels), and their
@ -252,3 +256,25 @@ jobs:
echo "::error::a vector test skipped instead of checking its input"
exit 1
fi
# The same vectors decoded by the wasm32 build of wai-web (the browser
# sink), in the same job so they share $WAI_SOTA_INT_DIR. The verify script
# fails on a missing artifact here (WAI_REQUIRE_SOTA_VECTORS=1, and the
# directory is named), and compares the wasm decode's RGB hash, for the
# JSON and the compact binary model, with the native export's.
- name: wasm32 toolchain (target, wasm-bindgen CLI pinned to wai-web's lock)
shell: bash
run: |
rustup target add wasm32-unknown-unknown
cargo install --locked wasm-bindgen-cli --version 0.2.127
- uses: actions/setup-node@v4
with:
node-version: '22'
- name: wasm32 decode of the int_hyper vectors equals the native reference
shell: bash
working-directory: wai/wai-web
run: |
set -euo pipefail
cargo build --locked --release --target wasm32-unknown-unknown
wasm-bindgen --target nodejs --out-dir demo/pkg-node --out-name wai_web \
target/wasm32-unknown-unknown/release/wai_web.wasm
node demo/verify_int_hyper_wasm.mjs ./pkg-node/wai_web.js "$WAI_SOTA_INT_DIR"

95
.github/workflows/wai-web-wasm.yml vendored Normal file
View file

@ -0,0 +1,95 @@
name: wai-web wasm32
# The browser sink (wai/wai-web) is the same Rust as the native reference,
# compiled to wasm32-unknown-unknown. This job builds it and runs, under node:
#
# - demo/verify_int_synth_wasm.mjs against the committed conformance corpus
# wai/neural-conformance/ (not the demo's staged copies): the wasm32
# int_synth decode must hash to the corpus's frozen native hashes;
# - demo/verify_menu.mjs: every capability the browser sink advertises is
# registered in wai/registry/capabilities.json as a payload or companion
# capability, none is a candidate (SPEC §5), and the wasm build's own
# registry agrees with the export on every candidate;
# - demo/verify_pack.mjs: the browser writers `pack` and `packMulti` refuse
# every candidate the export lists, in every place an envelope can name one;
# - a byte comparison of the demo's staged int_synth copies with the corpus.
#
# The full-scale int_hyper wasm decode needs vectors regenerated from a
# pretrained model; wai-sota-vectors.yml runs it after regenerating them.
on:
push:
branches: [main]
paths:
- 'wai/wai-web/**'
- 'wai/wai-rs/**'
- 'wai/wai-quantum/**'
- 'wai/neural-conformance/**'
- 'wai/registry/**'
- '.github/workflows/wai-web-wasm.yml'
pull_request:
paths:
- 'wai/wai-web/**'
- 'wai/wai-rs/**'
- 'wai/wai-quantum/**'
- 'wai/neural-conformance/**'
- 'wai/registry/**'
- '.github/workflows/wai-web-wasm.yml'
jobs:
wasm32:
name: wai-web wasm32 build · corpus, menu and writers
runs-on: linux
timeout-minutes: 45
steps:
- uses: actions/checkout@v5
# The docker runner image ships no Rust.
- name: Install Rust 1.98.0 with the wasm32 target
shell: bash
run: |
export PATH="$HOME/.cargo/bin:$PATH"
command -v rustup >/dev/null 2>&1 || \
curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh -s -- -y --profile minimal --default-toolchain 1.98.0
rustup toolchain install 1.98.0 --profile minimal
rustup default 1.98.0
rustup target add wasm32-unknown-unknown
echo "$HOME/.cargo/bin" >> "$GITHUB_PATH"
# Pinned to the wasm-bindgen version in wai-web/Cargo.lock: the CLI and the
# crate must match exactly.
- name: wasm-bindgen CLI 0.2.127
shell: bash
run: cargo install --locked wasm-bindgen-cli --version 0.2.127
- uses: actions/setup-node@v4
with:
node-version: '22'
- name: Build wai-web for wasm32 (node bindings)
shell: bash
working-directory: wai/wai-web
run: |
set -euo pipefail
cargo build --locked --release --target wasm32-unknown-unknown
wasm-bindgen --target nodejs --out-dir demo/pkg-node --out-name wai_web \
target/wasm32-unknown-unknown/release/wai_web.wasm
- name: wasm32 int_synth decode equals the frozen native corpus
shell: bash
working-directory: wai/wai-web
run: node demo/verify_int_synth_wasm.mjs ./pkg-node/wai_web.js ../neural-conformance
- name: The browser menu is registered and names no candidate
shell: bash
working-directory: wai/wai-web
run: node demo/verify_menu.mjs ./pkg-node/wai_web.js ../registry/capabilities.json
- name: The browser writers refuse every candidate
shell: bash
working-directory: wai/wai-web
run: node demo/verify_pack.mjs ./pkg-node/wai_web.js ../registry/capabilities.json
- name: The demo's staged int_synth vectors are byte copies of the corpus
shell: bash
working-directory: wai
run: |
set -euo pipefail
cmp wai-web/demo/int_synth/expected_hashes.json neural-conformance/expected_hashes.json
for n in gradient bloom facade photo; do
for f in latent.bin int_synth.json; do
cmp "wai-web/demo/int_synth/$n/$f" "neural-conformance/vectors/$n/$f"
done
done

125
scripts/check-comparative-prose.sh Executable file
View file

@ -0,0 +1,125 @@
#!/usr/bin/env bash
#
# check-comparative-prose.sh
#
# Holds WAI's public text to its writing rule: it states what WAI does,
# plainly, and does not rank WAI against anything else. The phrases below are
# the comparative and promotional forms the tree has had to remove; each is a
# POSIX extended regular expression, matched case-sensitively.
#
# Scope: the same files as scripts/check-public-names.sh — the tracked files
# under wai/ with the extensions .md .rs .toml .py .sh .mjs .js .json .html
# .css .ttl .h .c .txt and wai/'s .gitignore files, except the settled OER/2
# documents and corpus, and except wai/CHANGELOG.md: its history is a record,
# and only names are redacted from it. Files given as arguments replace that
# list (the self-test, scripts/test-public-text-checks.sh, uses this).
#
# Opt-out, for a functional identifier or a quotation only (an environment
# variable or file name that contains a listed word, a phrase quoted from a
# cited document): put `allow-phrase: <reason>` on the same line, in a comment
# of the file's own syntax. The reason is required: it must contain a letter or
# a digit.
#
# Exit status: 0 clean; 1 a hit; 2 a pattern grep cannot compile, or any other
# grep error. The check never reads an error as "clean".
set -euo pipefail
REPO_ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"
cd "$REPO_ROOT"
PHRASES=(
'\bSOTA\b'
'\bwin vs\b'
'\b[Mm]oat\b'
'[Nn]obody signs'
'[Nn]ot one signs'
'[Nn]one signs'
'[Bb]est[- ]in[- ]class'
'\b[Bb]eats\b [A-Z]'
'\b[Ss]uperior(ity)?\b'
'% better'
'\b[Tt]he opposite\b'
'[Ss]trictly worse'
'[Tt]oll[- ]road'
'\b[Pp]arity \+'
'[Ff]eature[- ]parity'
'\bparity with\b'
'\b[Uu]nderperforms\b'
'\b[Oo]utperforms\b'
'royalty trap'
'well below [A-Z]'
'well above [A-Z]'
'among the (lowest|best|highest|fastest|smallest)'
'nothing else .* offers?'
'\b[Nn]o other .* (offers?|signs?|provides?)'
'[Cc]ommercial wedge'
'\b[Uu]nclaimed\b'
'\b[Ff]ast win\b'
'\b[Dd]ifferentiator\b'
'[Rr]ate-competitive'
)
args=()
for p in "${PHRASES[@]}"; do args+=(-e "$p"); done
# Compile every phrase once before scanning: grep exits 1 when nothing matches
# and 2 on an error, and only 0 and 1 mean the list is usable.
set +e
grep -E "${args[@]}" /dev/null >/dev/null
rc=$?
set -e
if [ $rc -gt 1 ]; then
echo "::error::a phrase in $0 does not compile (grep exit $rc); nothing was checked" >&2
exit 2
fi
files=()
if [ $# -gt 0 ]; then
files=("$@")
else
while IFS= read -r f; do
case "$f" in
wai/extensions/oer-receipts.md|wai/wai-rs/src/oer.rs|wai/oer-conformance/*) continue ;;
wai/CHANGELOG.md) continue ;;
esac
files+=("$f")
done < <(git ls-files 'wai/*.md' 'wai/*.rs' 'wai/*.toml' 'wai/*.py' 'wai/*.sh' 'wai/*.mjs' 'wai/*.js' \
'wai/*.json' 'wai/*.html' 'wai/*.css' 'wai/*.ttl' 'wai/*.h' 'wai/*.c' 'wai/*.txt' \
'wai/*.gitignore')
fi
if [ ${#files[@]} -eq 0 ]; then
echo "::error::no file to check" >&2
exit 2
fi
out="$(mktemp)"
trap 'rm -f "$out"' EXIT
set +e
grep -HnE "${args[@]}" -- "${files[@]}" > "$out"
rc=$?
set -e
if [ $rc -gt 1 ]; then
echo "::error::grep failed (exit $rc) while scanning; nothing was checked" >&2
exit 2
fi
hits=0
while IFS= read -r line; do
content="${line#*:}"
content="${content#*:}"
if printf '%s' "$content" | grep -qE 'allow-phrase:[[:space:]]*[[:alnum:]]'; then
continue
fi
if [ $hits -eq 0 ]; then
echo "::error::comparative or promotional phrasing in public text (scripts/check-comparative-prose.sh)"
fi
echo "$line"
hits=$((hits + 1))
done < "$out"
if [ $hits -gt 0 ]; then
echo "Fix: state what WAI does, measured where a figure is given, without ranking it against"
echo "anything else; for an identifier or a quotation, mark the line with allow-phrase: <reason>."
exit 1
fi
echo "comparative-prose check: clean (${#files[@]} files)"

115
scripts/check-public-names.sh Executable file
View file

@ -0,0 +1,115 @@
#!/usr/bin/env bash
#
# check-public-names.sh
#
# Holds WAI's public text to its naming rule: it names no company, vendor,
# customer or commercial product. Standards, standards bodies, protocols,
# papers and open-source projects may be named, factually.
#
# The patterns are NOT in this repository. They are read from the
# environment variable WAI_NAME_DENYLIST: newline-separated POSIX extended
# regular expressions, matched case-sensitively (write `[Ff]oo` where both
# cases matter). In CI the variable comes from a forge Actions secret of the
# same name; locally, export it from a private file before committing:
#
# WAI_NAME_DENYLIST="$(cat /path/to/private/patterns)" scripts/check-public-names.sh
#
# A public list would itself publish the names it exists to keep out.
#
# Scope: the tracked files under wai/ with the extensions .md .rs .toml .py
# .sh .mjs .js .json .html .css .ttl .h .c .txt, and wai/'s .gitignore files;
# or, when files are given as arguments, those files (the self-test,
# scripts/test-public-text-checks.sh, uses this). Excluded:
# - the settled OER/2 documents and corpus (wai/extensions/oer-receipts.md,
# wai/wai-rs/src/oer.rs, wai/oer-conformance/), which no change may edit.
#
# Opt-out, for a functional identifier only (a model-hub id a tool defaults
# to, a repository URL, an OS API name, a security advisory id, a capability
# string): put `allow-name: <reason>` on the same line, in a comment of the
# file's own syntax (`<!-- allow-name: reason -->`, `// allow-name: reason`,
# `# allow-name: reason`). The reason is required: it must contain a letter or
# a digit. Prose around such an identifier describes the standard or the
# function, not the owner.
#
# Exit status: 0 clean; 1 a hit; 2 WAI_NAME_DENYLIST unset or empty, a pattern
# grep cannot compile, or any other grep error. The check never reads an error
# as "clean".
set -euo pipefail
REPO_ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"
cd "$REPO_ROOT"
if [ -z "${WAI_NAME_DENYLIST:-}" ]; then
echo "WAI_NAME_DENYLIST is not set: nothing to check against (see the header of $0)" >&2
exit 2
fi
patterns="$(mktemp)"
out="$(mktemp)"
trap 'rm -f "$patterns" "$out"' EXIT
printf '%s\n' "$WAI_NAME_DENYLIST" | grep -v '^[[:space:]]*$' > "$patterns" || true
if [ ! -s "$patterns" ]; then
echo "WAI_NAME_DENYLIST holds no pattern" >&2
exit 2
fi
# Compile every pattern once before scanning. grep exits 1 when nothing matches
# and 2 on an error; only 0 and 1 mean the patterns are usable. grep's message
# is suppressed: it could quote a private pattern.
set +e
grep -E -f "$patterns" /dev/null >/dev/null 2>&1
rc=$?
set -e
if [ $rc -gt 1 ]; then
echo "::error::WAI_NAME_DENYLIST holds a pattern grep cannot compile (grep exit $rc); nothing was checked" >&2
exit 2
fi
files=()
if [ $# -gt 0 ]; then
files=("$@")
else
while IFS= read -r f; do
case "$f" in
wai/extensions/oer-receipts.md|wai/wai-rs/src/oer.rs|wai/oer-conformance/*) continue ;;
esac
files+=("$f")
done < <(git ls-files 'wai/*.md' 'wai/*.rs' 'wai/*.toml' 'wai/*.py' 'wai/*.sh' 'wai/*.mjs' 'wai/*.js' \
'wai/*.json' 'wai/*.html' 'wai/*.css' 'wai/*.ttl' 'wai/*.h' 'wai/*.c' 'wai/*.txt' \
'wai/*.gitignore')
fi
if [ ${#files[@]} -eq 0 ]; then
echo "::error::no file to check" >&2
exit 2
fi
set +e
grep -HnE -f "$patterns" -- "${files[@]}" > "$out" 2>/dev/null
rc=$?
set -e
if [ $rc -gt 1 ]; then
echo "::error::grep failed (exit $rc) while scanning; nothing was checked" >&2
exit 2
fi
hits=0
while IFS= read -r line; do
content="${line#*:}"
content="${content#*:}"
if printf '%s' "$content" | grep -qE 'allow-name:[[:space:]]*[[:alnum:]]'; then
continue
fi
# Print the location only: the matched text may be a name the list keeps private.
if [ $hits -eq 0 ]; then
echo "::error::public text names a company, vendor, customer or product (patterns: WAI_NAME_DENYLIST)"
fi
echo "${line%%:*}:$(printf '%s' "${line#*:}" | cut -d: -f1)"
hits=$((hits + 1))
done < "$out"
if [ $hits -gt 0 ]; then
echo "Fix: name the standard, the standards body, the open-source project or the function instead;"
echo "for a functional identifier, mark the line with allow-name: <reason>."
exit 1
fi
echo "public-name check: clean (${#files[@]} files)"

87
scripts/test-public-text-checks.sh Executable file
View file

@ -0,0 +1,87 @@
#!/usr/bin/env bash
#
# test-public-text-checks.sh
#
# Self-test of scripts/check-public-names.sh and scripts/check-comparative-prose.sh.
# Each case feeds a check a fixture file (or a pattern list) and states the exit
# status it must give:
# - a hit exits 1, a clean file exits 0;
# - an opt-out with a reason is honoured, and one with an empty reason (the
# comment form `<!-- allow-name: -->`, whose `-->` must not count as the
# reason) is not;
# - a pattern grep cannot compile exits 2, never "clean".
# The fixture names a made-up company, never a real one.
#
# Exit status: 0 when every case gives its stated status, 1 otherwise.
set -uo pipefail
REPO_ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"
names="$REPO_ROOT/scripts/check-public-names.sh"
prose="$REPO_ROOT/scripts/check-comparative-prose.sh"
tmp="$(mktemp -d)"
trap 'rm -rf "$tmp"' EXIT
failures=0
expect() { # expect <status> <label> <command...>
local want="$1" label="$2"
shift 2
"$@" >"$tmp/out" 2>&1
local got=$?
if [ "$got" -eq "$want" ]; then
echo "ok $label (exit $got)"
else
echo "FAIL $label: exit $got, expected $want"
sed 's/^/ /' "$tmp/out"
failures=$((failures + 1))
fi
}
printf 'This line names Zyxwvut Holdings.\n' > "$tmp/bare.md"
printf 'Zyxwvut Holdings <!-- allow-name: a model-hub id the tool defaults to -->\n' > "$tmp/reasoned.md"
printf 'Zyxwvut Holdings <!-- allow-name: -->\n' > "$tmp/empty-comment.md"
printf 'Zyxwvut Holdings // allow-name:\n' > "$tmp/empty-line.rs"
printf 'This line names nothing.\n' > "$tmp/clean.md"
list='Zyxwvut'
expect 1 "names: a bare name is a hit" env WAI_NAME_DENYLIST="$list" bash "$names" "$tmp/bare.md"
expect 0 "names: an opt-out with a reason is honoured" env WAI_NAME_DENYLIST="$list" bash "$names" "$tmp/reasoned.md"
expect 1 "names: an opt-out with an empty reason in a comment is not" env WAI_NAME_DENYLIST="$list" bash "$names" "$tmp/empty-comment.md"
expect 1 "names: an opt-out with an empty reason at end of line is not" env WAI_NAME_DENYLIST="$list" bash "$names" "$tmp/empty-line.rs"
expect 0 "names: a clean file is clean" env WAI_NAME_DENYLIST="$list" bash "$names" "$tmp/clean.md"
expect 2 "names: a malformed pattern fails closed" env WAI_NAME_DENYLIST="$(printf 'Zyxwvut\n([unclosed')" bash "$names" "$tmp/clean.md"
expect 2 "names: an unset list fails closed" env -u WAI_NAME_DENYLIST bash "$names" "$tmp/clean.md"
expect 2 "names: a missing file fails closed" env WAI_NAME_DENYLIST="$list" bash "$names" "$tmp/no-such-file.md"
printf 'Its reproducibility is the superiority claim.\n' > "$tmp/superiority.md"
printf 'About 8 kbps, well below Xcodec at the same quality.\n' > "$tmp/well-below.md"
printf 'Its token rate is among the lowest of any codec.\n' > "$tmp/among.md"
printf 'One property nothing else in the field offers.\n' > "$tmp/nothing-else.md"
printf 'State-feeds are the biggest commercial wedge.\n' > "$tmp/wedge.md"
printf 'Joules-per-decode, the unclaimed metric.\n' > "$tmp/unclaimed.md"
printf 'Haptics is a fast win.\n' > "$tmp/fast-win.md"
printf 'Carriage is not a differentiator any more.\n' > "$tmp/differentiator.md"
printf 'It is not a rate-competitive codec.\n' > "$tmp/rate-competitive.md"
printf 'This is the moat <!-- allow-phrase: -->\n' > "$tmp/prose-empty.md"
printf 'WAI_MOAT_DIR <!-- allow-phrase: an environment variable name, not prose -->\n' > "$tmp/prose-reasoned.md"
printf 'The decoder is integer all the way down.\n' > "$tmp/prose-clean.md"
expect 1 "prose: superiority" bash "$prose" "$tmp/superiority.md"
expect 1 "prose: well below <Name>" bash "$prose" "$tmp/well-below.md"
expect 1 "prose: among the lowest" bash "$prose" "$tmp/among.md"
expect 1 "prose: nothing else ... offers" bash "$prose" "$tmp/nothing-else.md"
expect 1 "prose: commercial wedge" bash "$prose" "$tmp/wedge.md"
expect 1 "prose: unclaimed" bash "$prose" "$tmp/unclaimed.md"
expect 1 "prose: fast win" bash "$prose" "$tmp/fast-win.md"
expect 1 "prose: differentiator" bash "$prose" "$tmp/differentiator.md"
expect 1 "prose: rate-competitive" bash "$prose" "$tmp/rate-competitive.md"
expect 1 "prose: an opt-out with an empty reason is not honoured" bash "$prose" "$tmp/prose-empty.md"
expect 0 "prose: an opt-out with a reason is honoured" bash "$prose" "$tmp/prose-reasoned.md"
expect 0 "prose: a clean file is clean" bash "$prose" "$tmp/prose-clean.md"
expect 2 "prose: a missing file fails closed" bash "$prose" "$tmp/no-such-file.md"
if [ $failures -gt 0 ]; then
echo "$failures case(s) gave the wrong status"
exit 1
fi
echo "public-text check self-test: every case gave its stated status"

3
wai/.gitattributes vendored Normal file
View file

@ -0,0 +1,3 @@
# REGISTRY.md carries a table `wai registry export` generates, which a library test
# compares byte for byte (as registry/ and vocab/v2.* are): no line-ending conversion.
REGISTRY.md -text

149
wai/.github/workflows/ci.yml vendored
View file

@ -1,149 +0,0 @@
name: ci
on:
push:
branches: [ main ]
pull_request:
env:
CARGO_TERM_COLOR: always
RUST_BACKTRACE: 1
jobs:
# Build + run all tests (lib + ffi robustness + fuzz smoke) on both
# macOS and Linux. Production deployments will hit both; CI gates
# both.
test:
name: test ${{ matrix.os }}
runs-on: ${{ matrix.os }}
strategy:
fail-fast: false
matrix:
os: [ubuntu-latest, macos-latest]
steps:
- uses: actions/checkout@v4
- name: Install system codec libraries (Linux)
if: runner.os == 'Linux'
run: |
sudo apt-get update
sudo apt-get install -y --no-install-recommends \
pkg-config \
libjxl-dev \
libavif-dev \
libdav1d-dev \
libopus-dev \
libflac-dev \
libzstd-dev \
liblzma-dev \
ffmpeg \
zstd \
xz-utils
- name: Install system codec libraries (macOS)
if: runner.os == 'macOS'
run: |
brew update
brew install \
pkg-config \
jpeg-xl \
libavif \
dav1d \
opus \
flac \
zstd \
xz \
ffmpeg
- name: Install Rust (stable)
uses: dtolnay/rust-toolchain@stable
- name: Cache cargo registry + target
uses: actions/cache@v4
with:
path: |
~/.cargo/registry
~/.cargo/git
wai-rs/target
key: ${{ matrix.os }}-cargo-${{ hashFiles('wai-rs/Cargo.lock') }}
- name: cargo build --release (lib + bins)
working-directory: wai-rs
run: cargo build --release --lib --bins
- name: Verify SDK artifacts (cdylib + staticlib)
working-directory: wai-rs
shell: bash
run: |
if [[ "$RUNNER_OS" == "Linux" ]]; then
test -f target/release/libwai.so
test -f target/release/libwai.a
else
test -f target/release/libwai.dylib
test -f target/release/libwai.a
fi
- name: cargo test (lib + ffi robustness + fuzz smoke)
working-directory: wai-rs
run: cargo test --release
# Quick-mode bench on the always-shippable subset of the corpus.
# NOTE: the full corpus (~22 GB) is not committed and CI uses synthetic
# inputs only. For full-corpus regression gating, run on a self-hosted
# runner that mounts the corpus volume.
bench-quick:
name: bench-quick + regression diff
runs-on: ubuntu-latest
needs: test
steps:
- uses: actions/checkout@v4
- name: Install codec libraries
run: |
sudo apt-get update
sudo apt-get install -y --no-install-recommends \
pkg-config libjxl-dev libavif-dev libdav1d-dev \
libopus-dev libflac-dev libzstd-dev liblzma-dev \
ffmpeg zstd xz-utils
- name: Install Rust
uses: dtolnay/rust-toolchain@stable
- name: Build wai_bench + wai_bench_diff
working-directory: wai-rs
run: cargo build --release --bin wai_bench --bin wai_bench_diff
# Synthetic-only quick bench (CI doesn't have the 22 GB corpus).
# Generates synthetic test inputs into a temporary corpus tree so
# the bench has something to walk. Real-corpus regression checks
# happen on the self-hosted runner job below.
- name: Run quick bench (synthetic inputs)
working-directory: wai-rs
run: |
mkdir -p ../corpus/image/{kodak,synthetic} ../corpus/audio/sqam \
../corpus/text ../corpus/video/derf
# tiny PNG fixture so image_interop has at least one input
python3 -c "
from urllib.request import urlretrieve
urlretrieve('https://r0k.us/graphics/kodak/kodak/kodim01.png',
'../corpus/image/kodak/kodim01.png')
"
target/release/wai_bench --report bench-head.json || true
ls -la bench-head.json
- name: Diff against committed baseline (if present)
working-directory: wai-rs
run: |
if [[ -f ../bench-baseline.json ]]; then
target/release/wai_bench_diff ../bench-baseline.json bench-head.json
else
echo "(no bench-baseline.json checked in — skipping regression diff)"
fi
- name: Upload bench report
uses: actions/upload-artifact@v4
with:
name: bench-report-${{ github.sha }}
path: wai-rs/bench-head.json
retention-days: 30

2
wai/.gitignore vendored
View file

@ -1,4 +1,4 @@
# Test corpora — 22 GB, far over GitHub's 100 MB/file limit. Never commit.
# Test corpora: too large to commit. Never commit.
# Fetch scripts belong in-repo; the data does not.
/corpus/

157
wai/CHANGELOG.md
View file

@ -6,6 +6,131 @@ wire format may change and such changes are recorded here.
## v0.4.0 — current
### The registry as one table, candidate capabilities, the codec glossary under test, and the public text retrofit
- **Candidate capabilities** (SPEC §5): a registered string whose payload format no
revision has pinned — `wai.tensor.nnc` and `wai.splat.sog` — MUST NOT be emitted, and a
sink treats it as unsupported. This is a MINOR rule: it enforces the candidate
registrations as they were made (§8). Envelopes it makes non-conforming: a `WAI1` naming
a candidate as its `capability` or `fallback`, and a `WAI2` with a component of a
candidate; they still parse. `codecs::is_candidate`, `Wai::check_candidates` and
`WaiMulti::check_candidates` report them. `Wai::to_emit_bytes` and
`WaiMulti::to_emit_bytes` check, then serialise: the C packers `wai_envelope_pack` and
`wai_envelope_pack_multi` emit through them, the browser writers `pack` and `packMulti`
refuse the same envelopes (`container::check_candidates_v1`,
`check_component_candidates`), and `wai wrap` refuses a candidate before any other rule,
in every place including a `WAI2` companion. A candidate's licence class is provisional:
`wai.splat.sog` is classified patent-pool, conservatively (it was royalty-free, never
established). `wai.splat.sog` no longer claims an MPEG attribution: no MPEG page or
output document read names the format it was registered for ([research
registry](research/media-codec-registry.md), *MPEG's Gaussian-splat and point-cloud
work*).
- **One registry table** (`wai-rs/src/codecs/registry.rs`, no feature): every registered
string with its **kind** (`payload`, `companion`, `derivation`, `record`), its
**status** (`registered` or `candidate`; `RegistrationStatus`), its **media** class,
the SPEC §5 heading that registers it, the edition its registration names and the date
it was registered; the named identifiers that are not capabilities (`wai.det.fixed64`,
`wai.world.sim.*`); and the receipt labels no revision registers
(`wai.quantum.{mitigate,decode,compile,rearrange,qbom}`). SPEC §5 states the kinds:
records and derivations name no payload format. Every payload and companion
capability carries a licence class (records do not).
- **Dispatchable** (SPEC §4, normative): steps 3 and 5 take only a dispatchable
capability, and a capability is dispatchable unless this revision registers it as a
candidate, a companion-only capability, a record or a derivation. A string this
revision does not register stays dispatchable — a later MINOR revision may register
it (§8) — and is decoded only where the sink advertises it. `codecs::is_dispatchable`
implements it and is the one predicate `Wai::dispatch_capability`, `WaiMulti::pick`,
`WaiMulti::primary`, `RenditionMeta::is_selectable` and the CLI's `inspect` use; §7.1's
selection sentence says the same. A later dispatch or selection function uses it too.
- **`wai registry export`** writes `registry/capabilities.json`, its JSON Schema
(`registry/capabilities.schema.json`), the generated table of
[`REGISTRY.md`](REGISTRY.md) and **vocab v2** (`vocab/v2.{ttl,jsonld,shapes.ttl}`,
namespace `https://wai.transaction.science/vocab/v2#`, one IRI per registered string,
enumeration individuals named `<class prefix>-<label>` so no two terms share an IRI,
the registration date as `registeredOn`; its JSON-LD context maps a `WAI1` or `WAI2`
manifest as written, with `kind` and a component's `role` under terms of their own;
its SHACL shapes apply to a manifest with no `@type` and to every `WAI2` component,
check the capabilities against the registry, and leave `media` open, as SPEC §3 and §8
require). `--check` fails on drift; a library test and a CI step run it, and
`vocab/check_shapes.py` validates typed and untyped sample manifests in CI. vocab v1
is stable: no IRI is removed and no term changes meaning. REGISTRY.md states the
registration procedure; CONTRIBUTING.md is rewritten around it.
- **The `wai` binary builds without the `codecs` feature** (encode and decode are
compiled out); `tests/cli_wrap_intent.rs` and the new `tests/cli_registry.rs` run in
CI. `wai wrap` now writes the registry's media class: `volumetric` for `wai.mesh.*`,
`wai.pc.*` and `wai.splat.*` (was `model`), `image` for the learned image codecs such
as `wai.neural.bmshj2018` (was `binary`). SPEC §3 lists the media values, informational.
Without `codecs`, `inspect`, `capabilities` and `unwrap` claim no native decode.
- **Editions** (SPEC §8): a registration that names no edition carries the edition of
its standard current on the date it was registered, and no later edition or
amendment, which takes a new string. `wai.pc.gpcc` and `wai.pc.vpcc` therefore do not
carry the Gaussian-splat amendments of ISO/IEC 23090-9 and 23090-5; Enhanced G-PCC
(ISO/IEC 23090-38) is not a `wai.pc.gpcc` payload. Which edition of each part was
current on their registration date is unresolved, and what they admit beyond the
part's first edition is unspecified until a revision records it.
- SPEC: `wai.world.replay`, `wai.world.live`, `wai.semantic.clip` and
`wai.semantic.perco2` are spelled out or given rows, and a test holds SPEC and the
registry to the same strings in both directions; the opening, §1 and the floor
paragraph agree with §5; citations: PNG (W3C Recommendation, Third Edition), FLAC
(RFC 9639), Zstandard (RFC 8878), AVIF (the AOM AVIF specification, HEIF-based,
MIAF-conforming), JPEG AI (ISO/IEC 6048-1:2025); the neural tables state each model's
published operating point; Appendices B and C are rewritten; confidentiality's
implementation is a companion implementation under its own licence.
- **The codec glossary** links each registered capability that has an entry, with the
registry's licence and replicate classes, through a reviewed table
(`GLOSSARY_LINKS`); every other payload and companion capability is listed with a
reason (`GLOSSARY_NO_ENTRY`). `org` became `body`, from a closed allowlist of
standards bodies, designations and open-source projects; new modality values `text`,
`volumetric`, `haptic`, `metadata`; new entries for the registered formats that had
none; descriptions scrubbed of company names and comparisons; entries titled by the
format's own name, FourCC, file extension or designation, never by a company, a
product or a person (the research codecs by their method); HEVC regraded `success`;
JPEG AI dated 2025; AVS3 video's body is the designation the registry pins. The file
now carries `"schema_version": 2`: a reader of the earlier form, which had `org` and no
version, finds no `org` field in version 2.
`glossary_links_agree_with_the_registry` holds it to its schema and the registry.
- **quantum-ops vectors regenerated with neutral ids** (`example:lab3-device`,
`example:qpu-a`) through `wai_quantum_ops_vectors gen`. Changed receipt hashes:
`cal_q3` `7c53298929a1261959315c91f9540781b4bff0669b5f4b783becee93dbf5f8fa`, `cal_cz`
`4c6374ee05c956f928455f86326357f10d08f3466e88d569a6ef49975001f526`, `job_bell`
`08b142d29dadc466b294b89d2127f39ee9fa1aa0b6b5a4c6c5d226baf343b9e3`. The wai-quantum
calibration pin (`receipt_pins::calibration_legacy_bytes`) was recomputed by the code
before the acquisition class, with the neutral id. wai-quantum's version is unchanged.
- **CI**: the feature-gated test targets the conformance workflow names are declared
with `required-features`, so dropping a feature stops cargo instead of running nothing;
`tests/int_audio.rs` and `tests/int_splat.rs` now run; the registry export is checked;
the glossary, SPEC, registry and vocab paths fire the conformance workflow. wai-web is
built for wasm32 (`wai-web-wasm.yml`, and the int_hyper vectors in
`wai-sota-vectors.yml`), and its verify scripts fail on a missing artifact when
`WAI_REQUIRE_SOTA_VECTORS` is set or a directory is named. The Windows x86_64 legs are
out of the conformance and byte-equality matrices while their runner is offline (each
matrix says how to restore it). The nested `wai/.github/workflows/ci.yml`, which no
forge reads, is removed.
- **Public text**: two checks, `scripts/check-comparative-prose.sh` (a public phrase
list) and `scripts/check-public-names.sh` (patterns kept private), now run in CI over
`wai/`'s text, code, data, markup and ignore files. Both fail closed (a grep error or
a pattern that does not compile exits 2), an opt-out needs a reason, and
`scripts/test-public-text-checks.sh` proves both on every run. Without the private
list a push to main fails. The list is given only to the run on a push to main, never to
a pull request's code; a pull request shows a failed step saying the name check did not
run. The specification, extensions, research notes, READMEs, CONTRIBUTING, the
glossary and source comments name no company, vendor or product and make no
comparisons; functional identifiers (capability strings, model-hub ids, OS API names,
advisory ids) stay, marked. Dated research records keep their findings: corrections are
dated annotations (the MoQ Transport revision, the MPEG splat stages, WebTransport's
status, the October 2026 call for content); names are redacted, and comparative
framing is restated, in place. In this changelog's history, names are redacted and
nothing else changes.
- Notes for changes in flight: a change that adds a dispatch or selection function
uses `codecs::is_dispatchable`; a change that registers a string (for example a prior
set) adds it to the registry table, runs `wai registry export`, regenerates vocab v2
(not v1) and lists the string in the glossary's `GLOSSARY_NO_ENTRY` or links it; the
quantum-ops corpus and the wai-quantum calibration pin have the values above; the
pwsh steps of the conformance workflow remain, inactive.
- [`extensions/jwp-receipts.md`](extensions/jwp-receipts.md) §5: the MUST is over the
cases §5.1–§5.5 define; verifying against their vectors becomes a MUST once the
vectors are published in the tree.
### Typed receipt classes, and a second (soft) binding class
- [`extensions/jwp-receipts.md`](extensions/jwp-receipts.md) §2.4.3–§2.4.7: every
@ -421,8 +546,8 @@ wire format may change and such changes are recorded here.
against the Python integer reference on one runner; cross-architecture
byte-equality remains the committed tiny vector's job in
`wai-byte-equality.yml`. The regenerated vectors are not portable, and part
of them is not repeatable run to run on every machine. Between an Apple
Silicon Mac and the x86_64 runner, the synthesis-only test's input latent,
of them is not repeatable run to run on every machine. Between an arm64
macOS machine [name removed 2026-09-30] and the x86_64 runner, the synthesis-only test's input latent,
which the float model computes, differed, and with it its expected
`rgb_hash`; each side wrote the same latent in all of its runs. Two runs on
the one runner wrote different `cond.json` and `int_synth_sota.json`, most
@ -438,7 +563,7 @@ wire format may change and such changes are recorded here.
- New test `integer_decode_keeps_the_float_codec_quality`: the Rust decoder
measures its own reconstruction against the source pixels and must stay within
1 dB of the same model's float decode on the same image. On the pinned
photograph, measured on an Apple Silicon Mac: integer decode 35.985 dB (8-bit
photograph, measured on an arm64 macOS machine [name removed 2026-09-30]: integer decode 35.985 dB (8-bit
PSNR of the Rust decode), float decode 36.175 dB (the export tool's PSNR of the
float reconstruction), a cost of 0.19 dB, from 6328 bytes of rANS payload for
256×256 pixels. This is a relative bar on one image: nothing here compares
@ -453,7 +578,7 @@ wire format may change and such changes are recorded here.
WAI's versioned models root (`jpegai/v1/`) by
`tools/wai_jpegai_fetch_e2e_inputs.sh` and checked against the SHA-256 manifest
`tools/jpegai-e2e-v1.sha256`. The job also pins the SHA-256 of the decoded
pixels (`WAI_JPEGAI_EXPECT_SHA256`): measured natively on an Apple Silicon Mac
pixels (`WAI_JPEGAI_EXPECT_SHA256`): measured natively on an arm64 macOS machine [name removed 2026-09-30]
and reproduced by an x86_64 Linux build under QEMU user-mode emulation, and
required again on the x86_64 runner. That extends a measurement, not a
registry guarantee: the registry classes `wai.image.jpegai` as
@ -620,7 +745,7 @@ wire format may change and such changes are recorded here.
enhancement layer. The check previously let an HDR descriptor bind to an audio
rendition, LCEVC bind to any `wai.video.*` rendition (JPEG 2000 included), and
any standalone codec ride as a companion; the reference test that carried an
IAMF stream as a companion now carries Dolby Vision metadata instead.
IAMF stream as a companion now carries ST 2094-10 dynamic metadata [name removed 2026-09-30] instead.
- SPEC §8 now says what these rules are, so they do not read as a redefinition
inside `1.x`: a registration fixes a capability's payload format **and its
role** (standalone media, or companion-only); changing either, or narrowing
@ -1172,7 +1297,7 @@ wire format may change and such changes are recorded here.
Four follow-on builds across the new cargo classes:
- **Real-SPZ-derived 4D clip** (`wai-spz-gen 4d`): the `bloom` clip animates
1200 genuine Niantic-SPZ Gaussians (the same cloud the static
1200 genuine SPZ [name removed 2026-09-30] Gaussians (the same cloud the static
`wai.splat.spz` asset ships, round-tripped through the SPZ library) —
real cloud, authored motion. LIVE on `/volumetric`. Honest framing: the
win is a verifiable per-frame state, not capture fidelity.
@ -1181,7 +1306,7 @@ Four follow-on builds across the new cargo classes:
same one the worlds demo uses) — per-Gaussian covariance projection,
conic, 3σ bounding quad, premultiplied blend, back-to-front sort — with
a 2D alpha-disc fallback where WebGPU is unavailable. The hash-verify
badge is render-independent. Confirmed on WebGPU (Chrome) + 2D fallback.
badge is render-independent. Confirmed on WebGPU (a browser [name removed 2026-09-30]) + 2D fallback.
- **Per-class receipts with measured joules**: `HapticReceipt`
(`haptic/receipt.rs`) + `Splat4dReceipt` (`splat4d/receipt.rs`) — the
worlds/audio JWP receipt profile transposed to touch (per-channel
@ -1345,10 +1470,10 @@ Four follow-on builds across the new cargo classes:
§4 step-6 semantics so callers can fall back per the envelope.
- **Native energy meter — measured joules, no sudo (2026-06-13)**:
corrected a wrong assumption that real energy needed RAPL/sudo
(unavailable on the arm64 prod box). On Apple Silicon the SoC energy
(unavailable on the arm64 prod box). On arm64 macOS [name removed 2026-09-30] the SoC energy
counters are readable through IOReport with **no privileges** (the
source `macmon`/`asitop` use; verified euid≠0 reads CPU/GPU/ANE power
on the M3 Ultra), and `proc_pid_rusage` gives exact per-process
on the arm64 machine [name removed 2026-09-30]), and `proc_pid_rusage` gives exact per-process
retired cycles/instructions. New `wai_world_meter` bin (`world`
feature): brackets a tight replay burst of a sealed `.wwld` against an
idle baseline, samples CPU power via macmon, and reports the marginal
@ -1361,7 +1486,7 @@ Four follow-on builds across the new cargo classes:
arg the meter emits a full §8 `SessionReceipt` carrying the MEASURED
joules + exact `work_steps`, signed and self-verified — real measured
energy end to end. Spec §8 + the `/worlds` page corrected: the
sink-metering path needs no sudo (IOReport on Apple Silicon, RAPL on
sink-metering path needs no sudo (IOReport on arm64 macOS [name removed 2026-09-30], RAPL on
x86); the browser estimate is the fallback, not the ceiling. clippy
clean, 94 tests green.
- **Phase 2 begun — haptics (`wai.haptic.signal`) (2026-06-13)**: touch
@ -1400,7 +1525,7 @@ Four follow-on builds across the new cargo classes:
in-browser against the corpus** (mixdown-equivalence, the same proof
worlds does for state), plays the bit-exact stereo mix via an
AudioBuffer, and shows a top-down radar of the objects panning. Proven
on prod: all three scenes verify, zero errors — wasm-in-Chrome mixes
on prod: all three scenes verify, zero errors — wasm in a browser [name removed 2026-09-30] mixes
byte-identically to native. Honest split kept: plays the exact stereo
floor; binaural/HRTF noted as the richer optional presentation.
- **Spatial-audio conformance corpus (2026-06-13)**: `wai_audio_vectors
@ -1444,7 +1569,7 @@ Four follow-on builds across the new cargo classes:
green. Remaining phases: haptics, live state-feeds, digital humans,
volumetric/4D splats.
- **Real SPZ decode — `wai.splat.spz` end to end (2026-06-13)**: the
splat gallery now renders a genuine Niantic-SPZ file, not a
splat gallery now renders a genuine SPZ [name removed 2026-09-30] file, not a
mesh-derived stand-in. New `wai-spz-gen` crate depends on the field's
`spz` reference library (the sink-supplied codec WAI dispatches to),
encodes a 2,400-Gaussian cube cloud to a real `.spz` (validated by
@ -1566,10 +1691,10 @@ Four follow-on builds across the new cargo classes:
and wai-web binds it (`WorldSink`: advance/particles/members/
state-hash/verify-seal; sim state stays exact Q32.32, f32 only for
display). WebGPU renderer (instanced SDF circles + member lines).
Proven in real Chrome via Playwright: the 1,282-byte truss_fracture
Proven in a real browser [name removed 2026-09-30] via Playwright: the 1,282-byte truss_fracture
payload replays 120 ticks, the member snaps, and the browser's
BLAKE3 state hash equals the native corpus hash byte-for-byte —
**replay-equivalence across arm64-native and wasm32-in-Chrome,
**replay-equivalence across arm64-native and wasm32 in a browser [name removed 2026-09-30],
demonstrated**, with the Ed25519 seal verified in-page.
- **Solver-backed registrant `wai.world.sim.truss0`** (2026-06-12,
`world/truss.rs`): structural simulation — particles + distance
@ -1591,7 +1716,7 @@ Four follow-on builds across the new cargo classes:
### Inter-frame video, categorical/VQ images, and real-model validation
- **`wai.neural.dcvc_rt` / `wai.neural.dcvc_fm` — inter-frame video decoder
implemented** (`neural/video_dcvc.rs`). "Requires CUDA" was a *speed* claim:
implemented** (`neural/video_dcvc.rs`). "Requires a GPU" [name removed 2026-09-30] was a *speed* claim:
inter-frame decode is a frame loop that propagates a reference feature, and
ONNX Runtime runs it on CPU. The per-frame entropy decode reuses the
CompressAI-validated hyperprior path; the temporal net is sink-supplied.
@ -1777,7 +1902,7 @@ thread, populated on any non-OK return.
`wai_bench_diff baseline.json head.json` flags regressions on bytes
(±5% default tolerance), encode time (±25%), or new failures. Non-zero
exit on regression — ready for CI gating.
- **GitHub Actions CI** (`.github/workflows/ci.yml`): ubuntu + macOS
- **Hosted CI** [name removed 2026-09-30] (`.github/workflows/ci.yml`): ubuntu + macOS
matrix, apt/brew codec deps, `cargo test --release`, cdylib/staticlib
artifact verification, quick bench, JSON report uploaded as artifact.

164
wai/CONTRIBUTING.md
View file

@ -1,94 +1,118 @@
# Contributing to WAI
WAI has two layers — a spec and a reference implementation. The
contribution paths are different.
WAI is a specification ([`SPEC.md`](SPEC.md) and its [`extensions/`](extensions/))
with a reference implementation in Rust ([`wai-rs/`](wai-rs/), with
[`wai-quantum/`](wai-quantum/) and the browser crate [`wai-web/`](wai-web/)). Both
are pre-1.0 drafts: substantive changes are welcome.
## To the spec (`SPEC.md`)
The canonical repository is the forge at
`https://git.transaction.science/Transaction-Science/open-standards`. Send changes
there; mirrors of it are read-only.
Open for revision. WAI is a v1.0 **draft**; until it freezes at v1.0
final, substantive changes are welcome.
## To the specification
A spec change is in scope if it:
A change is in scope if it:
- Names a real-world ambiguity in the existing text (point at the
section and the case that breaks).
- Adds a missing structural requirement (byte layout, conformance rule,
dispatch behavior) needed to make a third implementation falsifiable.
- Tightens the capability-dispatch or container semantics.
- Adds a new codec or capability — with a byte-precise wire definition
and cross-implementation fixtures, not a placeholder.
- names a real ambiguity in the text (point at the section and the case that
breaks);
- adds a missing structural requirement (byte layout, conformance rule, dispatch
behaviour) needed to make another implementation falsifiable;
- registers a capability with a byte-precise payload format and conformance
vectors, not a placeholder (see *Registering a capability*, below).
A spec change is out of scope if it:
A change is out of scope if it adds optional fields with no testable conformance
effect. §8 says what a revision may change: a registered capability's payload
format and role are fixed once registered, and a later edition of a standard that
changes its bitstream takes a new capability string.
- Adds optional fields with no testable conformance impact. WAI leans
toward fewer well-defined fields.
- Re-litigates the capability-as-requirement model (§1 is intentional:
the model is named, never transported, never hash-pinned).
- Substitutes an existing primitive for a different one (e.g. another
entropy coder for rANS) without a concrete failure mode in the
existing one.
## Registering a capability
How to propose a change: open a draft in a parallel `SPEC-vX-draft.md`,
quote the section it modifies, and regenerate + run the
cross-implementation conformance suite (below) to show the change does
not regress the two reference implementations.
[`REGISTRY.md`](REGISTRY.md) states the procedure: add the string to the registry
table in `wai-rs/src/codecs/`, register it in SPEC §5 under the same heading, run
`wai registry export`, and link it from the codec glossary or list it there as
having no entry. The tests hold SPEC, the registry and the exported files to the
same set of strings.
## To the reference implementations
## To the reference implementation
Code is Apache-2.0. There are two interoperable references:
### Building
- **`wai/`** — Python, readability-first. Container, static rANS,
`entropy_zeroth`, and the image/audio/video zeroth codecs;
`python -m wai encode|play`. Neural paths (EnCodec, TAESD) live here
and are out of scope for byte-level conformance (the model is a
requirement, not part of the standard).
- **`wai-rs/`** — Rust, performance + embeddability. Same wire format;
`wai` binary with `encode`/`play`.
The two MUST stay byte-interoperable on every lossless path. Before
submitting:
The default `codecs` feature links system codec libraries (libjxl, libavif,
libopus, libdav1d, liblzma, libflac); without them the build stops in a codec
crate's build script. Everything else builds without them, and CI builds that way:
```bash
# Rust: 10 library + 9 cross-Python conformance tests
cd wai-rs
cargo test
cargo build --release --bin wai
# Regenerate the canonical Python reference fixtures and re-check the
# Rust port still agrees byte-for-byte (run from the wai-rs root; the
# sibling `wai/` Python package must be importable)
python tests/gen_cross_fixtures.py
cargo test --test cross_python
cargo test --lib --no-default-features --features world,audio_scene,haptic,splat4d,score,provenance,neural_int,film
cargo run --no-default-features --bin wai -- capabilities
```
Both must pass. The cross-Python suite is what catches wire drift
between the two implementations — it is the executable definition of
conformance for v1.0 (`SPEC.md` §9).
Name the features a change touches explicitly; `cargo test` with the defaults is
not what CI runs.
## A third implementation
### Tests that CI must see
Conformance is **cross-implementation byte-equivalence**, not "passes
its own tests." A new-language implementation is conformant when, given
the fixtures emitted by `wai-rs/tests/gen_cross_fixtures.py`, it
produces byte-identical output for the bit-exact paths (rANS,
`entropy_zeroth`, container) and reconstruction-equivalent output
(identical quantized arrays) for the lossy codecs (image 4:4:4 + 4:2:0,
audio MDCT, video I+P). Validate against the same fixtures the Rust
reference is held to.
- A test file gated on a feature (`#![cfg(feature = "...")]`) compiles to an empty
binary, which reports success, in any build without that feature. Declare it in
`wai-rs/Cargo.toml` with a `[[test]]` stanza and `required-features`, and add
`--test <name>` with the right `--features` to a step in
`.github/workflows/`. A test no workflow names runs nowhere.
- A test that reads inputs from outside the tree skips without them. CI sets
`WAI_REQUIRE_SOTA_VECTORS=1` where it provides them, and a missing input then
fails.
- Falsify every new test once: change the code it guards, show it fails, restore
the code through git, and show it passes again.
A third implementation is in scope when it brings up a language
ecosystem that doesn't have one yet and is validated against those
fixtures. It is *not* in scope when it duplicates an existing reference
for ergonomic reasons; consolidate instead.
### Vectors
## License
Committed conformance vectors and pinned hashes change only through their
generator (the `wai_*_vectors gen` binaries and the export tools in
[`tools/`](tools/)), never by hand. A test whose comment says its hex must never
be edited to make it pass means exactly that.
By contributing, you agree your contribution is licensed under the same
terms as the corresponding layer:
### Independent implementations
- Contributions to the spec: published under the same open terms
(royalty-free; contributors grant patent rights for compliant
implementations).
- Contributions to either reference implementation: Apache-2.0.
Conformance is defined by the committed corpora (`*-conformance/`) and the
criteria of SPEC §7, not by agreement with this implementation's own tests. A
second implementation in another language is validated against the same corpora;
the OER receipt kernel in C (`oer-conformance/settle/`) is one.
No CLA. The license is the agreement.
## Public text
The specification, the extensions, READMEs, research notes, code comments and
the CHANGELOG state what WAI does, plainly:
- name no company, vendor, customer or commercial product. Standards, standards
bodies, protocols, papers and open-source projects may be named, factually;
- make no comparisons with other products or approaches;
- give no counts that drift (crate counts, test counts, line counts).
Three checks enforce this. `scripts/check-no-vanity-metrics.sh` (opt-out marker
`allow-vanity: <reason>`) and `scripts/check-comparative-prose.sh` (opt-out
`allow-phrase: <reason>`, for an identifier or a quotation) run on every change.
`scripts/check-public-names.sh` reads its patterns from a private list (the
`WAI_NAME_DENYLIST` secret in CI, given only to the run on a push to main, so a
pull request is checked when it lands); a functional identifier — a capability string,
a model-hub id a tool defaults to, a repository URL, an OS API name, a security
advisory id — is kept and its line marked `allow-name: <reason>`. Put the marker
in a comment of the file's own syntax (`<!-- ... -->`, `// ...`, `# ...`); the
reason must contain a letter or a digit. JSON has no comments, so a JSON file
carries no such identifier: the codec glossary titles a format by its own name,
FourCC, file extension or designation, and never by a company, a product or a
person ([`glossary/SCHEMA.md`](glossary/SCHEMA.md)). The checks fail closed: a
pattern that does not compile, or any grep error, fails them, and
`scripts/test-public-text-checks.sh` proves that on every run.
Dated research records are not rewritten: a correction is a dated annotation
after the original text, and a name is redacted in place with a dated marker.
## Licence
The specification and the reference implementation are Apache-2.0. By
contributing, you agree your contribution is licensed under the same terms. Each
payload and companion capability carries a licence class (royalty-free,
patent-pool, proprietary, non-commercial) for implementing it, recorded in the
registry; derivations carry the class `codecs::derivation` gives them; records
carry none. Companion implementations outside this tree keep their own licences.
No CLA. The licence is the agreement.

43
wai/README.md
View file

@ -1,10 +1,11 @@
# WAI — Web AI Media Transport & Execution
A container + capability-dispatch standard for media. **WAI does not
re-implement codecs**; it dispatches to SOTA standard libraries (AVIF,
JPEG-XL, PNG, JPEG, Opus, FLAC, AV1, zstd, XZ) and adds an envelope
that lets the *neural-shared-prior* path coexist with the model-free
floor.
A container + capability-dispatch standard for media. A standard codec
(AVIF, JPEG XL, PNG, JPEG, Opus, FLAC, AV1, zstd, XZ and the others the
registry names) is dispatched to the library a sink already has; WAI also
defines its own capability classes — integer codecs, worlds, scores, films —
each with a conformance criterion, and an envelope that lets the
*neural-shared-prior* path coexist with the model-free floor.
Two paths open every conforming file:
@ -12,8 +13,8 @@ Two paths open every conforming file:
shared ambient prior (a model). The model is a *requirement of
existence*, never shipped (an `.mp3` doesn't ship its decoder
either).
- **Zeroth condition** — a *menu of registered SOTA codecs*. Always
satisfiable on any device that ships ffmpeg or equivalent.
- **Zeroth condition** — a *menu of registered standard codecs*, decoded with
the libraries a device already ships.
The capability a file requires is *named*, not supplied.
@ -45,16 +46,20 @@ cargo check -p wai --no-default-features
|------|----------|
| [`SPEC.md`](SPEC.md) | WAI v1.0 spec — envelope, manifest, capability menu |
| [`wai-rs/`](wai-rs/) | Rust reference implementation (lib + cdylib + staticlib + C FFI) |
| [`glossary/`](glossary/) | Reference catalog of media codecs surveyed for capability registration. [`codecs.json`](glossary/codecs.json) is the canonical survey dataset; [`SCHEMA.md`](glossary/SCHEMA.md) defines the shape and contribution rules. Rendered at [wai.transaction.science/glossary](https://wai.transaction.science/glossary). |
| [`corpus/`](corpus/) | Standard test material (Kodak/CLIC/Tecnick/SIPI images, Xiph Derf + UVG video, SQAM-class audio, enwik8 + Silesia text). Not committed; `CORPUS.md` documents reacquisition. |
| [`REGISTRY.md`](REGISTRY.md), [`registry/capabilities.json`](registry/capabilities.json) | The registry: every registered string with its kind, status, classes and edition, generated from the reference implementation (`wai registry export`), with its JSON Schema and the policy for registering a string |
| [`vocab/`](vocab/) | RDF vocabulary: v2 generated from the registry, v1 frozen |
| [`extensions/`](extensions/) | Sub-specifications (confidentiality, receipts, worlds, film, quantum operations, …) |
| [`research/`](research/) | Research notes and dated records: graded, not normative |
| [`glossary/`](glossary/) | Reference catalog of media codecs surveyed for capability registration. [`codecs.json`](glossary/codecs.json) is the dataset, linked to the registry; [`SCHEMA.md`](glossary/SCHEMA.md) defines the shape and contribution rules. Rendered at [wai.transaction.science/glossary](https://wai.transaction.science/glossary). |
| [`CONTRIBUTING.md`](CONTRIBUTING.md) | How to change the specification, register a capability and add tests CI runs |
## Quickstart
```bash
cd wai-rs
cargo test --lib # every registered capability + envelope round-trip
cargo build --release --bin wai
cargo build --release --lib # produces libwai.dylib/.so/.dll + libwai.a
cargo test --no-default-features --lib --test carriage # registry, envelope, every capability round-trips
cargo run --no-default-features --bin wai -- capabilities # the registry, one row per capability
cargo build --release --lib # with the codecs feature: libwai.dylib/.so/.dll + libwai.a
```
On macOS, libflac lives under the Homebrew "flac" formula prefix that
@ -135,12 +140,17 @@ See [`SPEC.md`](SPEC.md) §5 and [`extensions/`](extensions/) for each.
WAI carries a full quantum stack in pure Rust: circuit simulation and the whole
operational layer around it, every stage byte-exact and sealable into a signed,
energy-accounted receipt. No QPU, no cloud service, no vendor SDK. Because it is
energy-accounted receipt. No QPU and no cloud service. Because it is
pure Rust with no host bindings, **one source builds for native and
`wasm32-wasip2` and produces byte-identical results** — the same circuit
reconstructs to the same statevector hash on a laptop or at the edge.
| Capability | What it does | Conformance |
Only `wai.quantum.circuit` is a registered payload capability, and
`wai.quantum.calibration` and `wai.quantum.job` are registered records; the other
names below are module labels ([`REGISTRY.md`](REGISTRY.md) lists the receipt
labels no revision registers).
| Module | What it does | Conformance |
|---|---|---|
| `wai.quantum.circuit` | byte-exact statevector simulation (dyadic Clifford+T+P(k)) | statevector-equivalence |
| `wai.quantum.toolchain` | four backends (statevector / stabilizer / tensor-network MPS / sparse-Pauli), algorithm library, OpenQASM 3.0 in-and-out, **QIR export + ingest**, Bloch + entanglement analysis | statevector-equivalence |
@ -238,4 +248,7 @@ the [toolchain IDE](https://wai.transaction.science/quantum-ide), the
## License
Apache-2.0 — see [`LICENSE`](LICENSE).
Apache-2.0 — see [`LICENSE`](LICENSE). Implementing a registered capability
carries that capability's own licence class (royalty-free, patent-pool,
proprietary or non-commercial), recorded in the registry. Companion
implementations outside this tree keep their own licences.

234
wai/REGISTRY.md Normal file
View file

@ -0,0 +1,234 @@
# WAI registry
The registry is the set of strings [SPEC §5](SPEC.md) registers, with the classes
the specification gives each one. Its source of truth is one table in the
reference implementation, [`wai-rs/src/codecs/registry.rs`](wai-rs/src/codecs/registry.rs)
together with the class functions of [`wai-rs/src/codecs/mod.rs`](wai-rs/src/codecs/mod.rs).
`wai registry export` writes it out in machine-readable form:
| file | what |
|---|---|
| [`registry/capabilities.json`](registry/capabilities.json) | every registered string with its kind, status, media class, role, fidelity tier, licence class, replicate class, set membership, registering section, edition and registration date; the named identifiers that are not capabilities; the unregistered labels |
| [`registry/capabilities.schema.json`](registry/capabilities.schema.json) | JSON Schema (2020-12) for that file; its enums are the label sets below |
| [`vocab/v2.ttl`](vocab/v2.ttl), [`vocab/v2.jsonld`](vocab/v2.jsonld), [`vocab/v2.shapes.ttl`](vocab/v2.shapes.ttl) | the same registry as RDF, a JSON-LD context for manifests, and a SHACL shape that checks a manifest's capability and fallback against it and only the form of its media; a violation for a string the registry does not hold means the string is not registered in this revision, not that the envelope fails to conform or cannot be dispatched (SPEC §4, §8) |
| the table at the end of this file | the same registry, for reading |
A library test (`codecs::registry::tests::registry_export_matches_committed`) and
a CI step (`wai registry export --check` in `.github/workflows/wai-conformance.yml`)
fail when a committed file differs from what the table renders. Edit the table,
never the exported files.
## Kinds
Every registered string has one kind.
- **payload** — a capability whose payload a sink decodes: it may be a `WAI1`
`capability` or `fallback`, or a `WAI2` rendition.
- **companion** — a companion-only capability (SPEC §5, §7.1): its payload is
applied with a base rendition and never on its own, so it is only ever a `WAI2`
companion.
- **derivation** — an operation of ISO/IEC 23001-16 (`wai.derive.*`,
[derived-tracks](extensions/derived-tracks.md)). It acts on samples another
capability decoded and names no payload format.
- **record** — a signed record a WAI tool emits and verifies (a receipt kind:
`wai.asset.provenance`, the quantum-operations receipts, the delivery records
`wai.video.manifest` and its siblings). It names no payload format.
A derivation or record string is not a payload capability, so it is not
dispatchable (SPEC §4): dispatch never takes an envelope's capability that names
one, and step 5 may still take a usable fallback.
Two further tables hold strings that look like capabilities and are not:
- **identifiers** name a part of a capability's definition (`wai.det.fixed64`, the
fixed-point floor; `wai.world.sim.*`, the simulation steps of the feed and avatar
capabilities);
- **unregistered labels** are strings the reference implementation uses as labels
that no revision registers, each with the reason.
## Status
- **registered** — the payload format (for a record or derivation, the
definition) is pinned.
- **candidate** — the string is registered but no revision has pinned its payload
format. An encoder MUST NOT emit a candidate, and a sink MUST treat it as
unsupported (SPEC §5, *Candidate capabilities*). The reference implementation
lists candidates in `codecs::is_candidate`, reports an envelope that names one
from `Wai::check_candidates` and `WaiMulti::check_candidates`, and never
dispatches or selects one.
Every dispatch and selection function uses one predicate,
`codecs::is_dispatchable(cap)`, SPEC §4's *dispatchable*: false for a
companion-only capability, a candidate, a record and a derivation; true for a
registered payload capability that is not a candidate, and for a string this
revision does not register, which a later MINOR revision may register (SPEC §8)
and which a sink decodes only if it advertises it. A new dispatch or selection
function MUST use it too.
## Editions
A registration that names an edition of its standard carries that edition. A
registration that names none carries the edition of its named standard current on
the date it was registered, and no later edition or amendment: a later edition or
amendment that changes the bitstream gets a capability string of its own
(SPEC §8, *Editions*). The export records both: `edition` (a designation, or
`null` for the edition current on the registration date) and `registered` (the
date of the first commit of this repository that registers the string; the
specification entered the repository on 2026-06-12, so a string registered before
then carries that date).
## Licence classes
Every payload and companion capability carries a licence class:
`royalty-free`, `patent-pool`, `proprietary` or `non-commercial`
(`codecs::capability_license`). It is a property of the capability, not a
per-file claim. The classification has no default: a capability registered
without a class is a registry defect the tests reject. The sources and residual
caveats for each class are in
[`research/media-codec-registry.md`](research/media-codec-registry.md). A
capability whose royalty status is not established is classified conservatively
(`patent-pool`), never `royalty-free`; a candidate's class is provisional until a
revision pins its payload format, since the licensing of a format not yet chosen
cannot be established. Every
derivation operation carries the class `codecs::derivation::derivation_license`
gives it; a record carries none.
## How to register a string
1. Add the capability constant to `wai-rs/src/codecs/mod.rs` and, for a payload or
companion capability, list it in `known_capabilities()` and give it an arm in
`capability_tier`, `capability_license` and `capability_replicate_class` (each
function has no default, and the tests reject a registered capability without
one). A record goes in `RECORDS`, a derivation in `codecs::derivation`.
2. Give it a row in `ROWS` of `wai-rs/src/codecs/registry.rs`: the SPEC §5 heading
it is registered under, the edition it carries, and the registration date. A
string whose payload format is not yet pinned also goes in `CANDIDATES`.
3. Register it in SPEC §5, under that heading, with its payload format and role.
The test `spec_and_registry_name_the_same_strings` holds SPEC and the table to
the same set of strings, in both directions.
4. Run `cargo run --no-default-features --bin wai -- registry export` from
`wai/wai-rs`, and commit the files it writes.
5. Link it from the codec glossary entry that describes its bitstream, or list it
in the glossary's `GLOSSARY_NO_ENTRY` table with the reason
([`glossary/SCHEMA.md`](glossary/SCHEMA.md)).
6. Add a CHANGELOG entry.
Changing what a registered string carries is governed by SPEC §8: a payload format
and a role are fixed once registered.
## The registry
<!-- generated: wai registry export; do not edit by hand -->
| string | kind | status | media | role | licence | replicate | sets | registered in |
|---|---|---|---|---|---|---|---|---|
| `wai.asset.provenance` | record | registered | — | — | — | — | — | §5 Interactive, temporal & computed content |
| `wai.audio.aac` | payload | registered | audio | standalone | patent-pool | standard-unclassified | — | §5 Broadcast and streaming codecs |
| `wai.audio.ac4` | payload | registered | audio | standalone | proprietary | standard-unclassified | — | §5 Immersive audio, cinema, volumetric, haptics, HDR |
| `wai.audio.avs3` | payload | registered | audio | standalone | patent-pool | standard-unclassified | — | §5 Broadcast and streaming codecs |
| `wai.audio.flac` | payload | registered | audio | standalone | royalty-free | standard-defined | floor | §5 Mandatory floor |
| `wai.audio.iamf` | payload | registered | audio | standalone | royalty-free | standard-tolerance | — | §5 Immersive audio, cinema, volumetric, haptics, HDR |
| `wai.audio.int_codec` | payload | registered | audio | standalone | royalty-free | neural-int-exact | — | §5 Interactive, temporal & computed content |
| `wai.audio.ivas` | payload | registered | audio | standalone | patent-pool | standard-tolerance | — | §5 Immersive audio, cinema, volumetric, haptics, HDR |
| `wai.audio.mpegh` | payload | registered | audio | standalone | patent-pool | standard-unclassified | — | §5 Broadcast and streaming codecs |
| `wai.audio.opus` | payload | registered | audio | standalone | royalty-free | standard-tolerance | recommended | §5 Recommended modern set |
| `wai.audio.scene` | payload | registered | audio | standalone | royalty-free | standard-defined | — | §5 Interactive, temporal & computed content |
| `wai.audio.score` | payload | registered | audio | standalone | royalty-free | standard-defined | — | §5 Interactive, temporal & computed content |
| `wai.avatar.pose` | payload | registered | interactive | standalone | royalty-free | standard-defined | — | §5 Interactive, temporal & computed content |
| `wai.derive.crop` | derivation | registered | — | — | patent-pool | — | — | §5 Prior artifacts, general tensors and derivation operations |
| `wai.derive.dissolve` | derivation | registered | — | — | patent-pool | — | — | §5 Prior artifacts, general tensors and derivation operations |
| `wai.derive.grid_composition` | derivation | registered | — | — | patent-pool | — | — | §5 Prior artifacts, general tensors and derivation operations |
| `wai.derive.identity` | derivation | registered | — | — | patent-pool | — | — | §5 Prior artifacts, general tensors and derivation operations |
| `wai.derive.mirror` | derivation | registered | — | — | patent-pool | — | — | §5 Prior artifacts, general tensors and derivation operations |
| `wai.derive.overlay_composition` | derivation | registered | — | — | patent-pool | — | — | §5 Prior artifacts, general tensors and derivation operations |
| `wai.derive.roi_selection` | derivation | registered | — | — | patent-pool | — | — | §5 Prior artifacts, general tensors and derivation operations |
| `wai.derive.rotation` | derivation | registered | — | — | patent-pool | — | — | §5 Prior artifacts, general tensors and derivation operations |
| `wai.derive.scaling` | derivation | registered | — | — | patent-pool | — | — | §5 Prior artifacts, general tensors and derivation operations |
| `wai.derive.srgb_fill` | derivation | registered | — | — | patent-pool | — | — | §5 Prior artifacts, general tensors and derivation operations |
| `wai.feed.state` | payload | registered | interactive | standalone | royalty-free | standard-defined | — | §5 Interactive, temporal & computed content |
| `wai.film.linear` | payload | registered | video | standalone | royalty-free | standard-defined | — | §5 Interactive, temporal & computed content |
| `wai.film.reel` | payload | registered | video | standalone | royalty-free | standard-defined | — | §5 Interactive, temporal & computed content |
| `wai.generate.spectral` | payload | registered | image | standalone | royalty-free | generative-exact | — | §5 Neural capabilities |
| `wai.haptic.mihs` | payload | registered | haptic | standalone | patent-pool | standard-unclassified | — | §5 Immersive audio, cinema, volumetric, haptics, HDR |
| `wai.haptic.signal` | payload | registered | haptic | standalone | royalty-free | standard-defined | — | §5 Interactive, temporal & computed content |
| `wai.image.avif` | payload | registered | image | standalone | royalty-free | standard-tolerance | recommended | §5 Recommended modern set |
| `wai.image.jp2` | payload | registered | image | standalone | royalty-free | standard-tolerance | — | §5 Immersive audio, cinema, volumetric, haptics, HDR |
| `wai.image.jpeg` | payload | registered | image | standalone | royalty-free | standard-tolerance | recommended | §5 Recommended modern set |
| `wai.image.jpegai` | payload | registered | image | standalone | royalty-free | neural-entropy-exact | — | §5 Neural capabilities |
| `wai.image.jxl` | payload | registered | image | standalone | royalty-free | standard-tolerance | recommended | §5 Recommended modern set |
| `wai.image.png` | payload | registered | image | standalone | royalty-free | standard-defined | floor | §5 Mandatory floor |
| `wai.mesh.draco` | payload | registered | volumetric | standalone | royalty-free | standard-unclassified | — | §5 Immersive audio, cinema, volumetric, haptics, HDR |
| `wai.mesh.glb` | payload | registered | volumetric | standalone | royalty-free | standard-unclassified | — | §5 Interactive, temporal & computed content |
| `wai.meta.dovi` | companion | registered | metadata | companion | proprietary | standard-unclassified | — | §5 Immersive audio, cinema, volumetric, haptics, HDR |
| `wai.meta.hdr10plus` | companion | registered | metadata | companion | royalty-free | standard-unclassified | — | §5 Immersive audio, cinema, volumetric, haptics, HDR |
| `wai.neural.bmshj2018` | payload | registered | image | standalone | royalty-free | neural-float | — | §5 Neural capabilities |
| `wai.neural.cmic` | payload | registered | image | standalone | royalty-free | neural-float | — | §5 Neural capabilities |
| `wai.neural.dac` | payload | registered | audio | standalone | royalty-free | neural-float | — | §5 Neural capabilities |
| `wai.neural.dcvc_fm` | payload | registered | video | standalone | royalty-free | neural-float | — | §5 Neural capabilities |
| `wai.neural.dcvc_rt` | payload | registered | video | standalone | royalty-free | neural-float | — | §5 Neural capabilities |
| `wai.neural.elic` | payload | registered | image | standalone | royalty-free | neural-float | — | §5 Neural capabilities |
| `wai.neural.encodec32` | payload | registered | audio | standalone | royalty-free | neural-float | — | §5 Neural capabilities |
| `wai.neural.glc` | payload | registered | image | standalone | royalty-free | neural-float | — | §5 Neural capabilities |
| `wai.neural.int_hyper` | payload | registered | image | standalone | royalty-free | neural-int-exact | — | §5 Neural capabilities |
| `wai.neural.int_mlicpp` | payload | registered | image | standalone | royalty-free | neural-int-exact | — | §5 Neural capabilities |
| `wai.neural.int_synth` | payload | registered | image | standalone | royalty-free | neural-int-exact | — | §5 Neural capabilities |
| `wai.neural.mbt2018_mean` | payload | registered | image | standalone | royalty-free | neural-float | — | §5 Neural capabilities |
| `wai.neural.mimi` | payload | registered | audio | standalone | royalty-free | neural-float | — | §5 Neural capabilities |
| `wai.neural.mlicpp` | payload | registered | image | standalone | royalty-free | neural-float | — | §5 Neural capabilities |
| `wai.neural.mlicv2` | payload | registered | image | standalone | royalty-free | neural-float | — | §5 Neural capabilities |
| `wai.neural.snac` | payload | registered | audio | standalone | royalty-free | neural-float | — | §5 Neural capabilities |
| `wai.neural.video_bmshj2018` | payload | registered | video | standalone | royalty-free | neural-float | — | §5 Neural capabilities |
| `wai.neural.wavtokenizer` | payload | registered | audio | standalone | royalty-free | neural-float | — | §5 Neural capabilities |
| `wai.pc.gpcc` | payload | registered | volumetric | standalone | patent-pool | standard-unclassified | — | §5 Immersive audio, cinema, volumetric, haptics, HDR |
| `wai.pc.vpcc` | payload | registered | volumetric | standalone | patent-pool | standard-unclassified | — | §5 Immersive audio, cinema, volumetric, haptics, HDR |
| `wai.prior.nnc` | payload | registered | binary | standalone | patent-pool | standard-unclassified | — | §5 Prior artifacts, general tensors and derivation operations |
| `wai.quantum.calibration` | record | registered | — | — | — | — | — | §5 Interactive, temporal & computed content |
| `wai.quantum.circuit` | payload | registered | binary | standalone | royalty-free | standard-defined | — | §5 Interactive, temporal & computed content |
| `wai.quantum.job` | record | registered | — | — | — | — | — | §5 Interactive, temporal & computed content |
| `wai.semantic.clip` | payload | registered | image | standalone | royalty-free | behavioral | — | §5 Neural capabilities |
| `wai.semantic.perco2` | payload | registered | image | standalone | non-commercial | behavioral | — | §5 Neural capabilities |
| `wai.splat.4d` | payload | registered | volumetric | standalone | royalty-free | standard-defined | — | §5 Interactive, temporal & computed content |
| `wai.splat.int_codec` | payload | registered | volumetric | standalone | royalty-free | neural-int-exact | — | §5 Interactive, temporal & computed content |
| `wai.splat.sog` | payload | candidate | volumetric | standalone | patent-pool | standard-unclassified | — | §5 Immersive audio, cinema, volumetric, haptics, HDR |
| `wai.splat.spz` | payload | registered | volumetric | standalone | royalty-free | standard-unclassified | — | §5 Interactive, temporal & computed content |
| `wai.tensor.nnc` | payload | candidate | binary | standalone | patent-pool | standard-unclassified | — | §5 Prior artifacts, general tensors and derivation operations |
| `wai.ternary.tt` | payload | registered | binary | standalone | royalty-free | standard-defined | — | §5 Interactive, temporal & computed content |
| `wai.text.xz` | payload | registered | text | standalone | royalty-free | standard-defined | recommended | §5 Recommended modern set |
| `wai.text.zstd` | payload | registered | text | standalone | royalty-free | standard-defined | floor | §5 Mandatory floor |
| `wai.video.av1` | payload | registered | video | standalone | royalty-free | standard-tolerance | recommended | §5 Recommended modern set |
| `wai.video.av1.lossless` | payload | registered | video | standalone | royalty-free | standard-tolerance | recommended | §5 Recommended modern set |
| `wai.video.av2` | payload | registered | video | standalone | royalty-free | standard-tolerance | — | §5 Broadcast and streaming codecs |
| `wai.video.avc` | payload | registered | video | standalone | patent-pool | standard-defined | — | §5 Broadcast and streaming codecs |
| `wai.video.avs3` | payload | registered | video | standalone | patent-pool | standard-unclassified | — | §5 Broadcast and streaming codecs |
| `wai.video.channel` | record | registered | — | — | — | — | — | §5 Neural capabilities |
| `wai.video.evc` | payload | registered | video | standalone | patent-pool | standard-unclassified | — | §5 Broadcast and streaming codecs |
| `wai.video.evc.baseline` | payload | registered | video | standalone | royalty-free | standard-unclassified | — | §5 Broadcast and streaming codecs |
| `wai.video.hevc` | payload | registered | video | standalone | patent-pool | standard-defined | — | §5 Broadcast and streaming codecs |
| `wai.video.int_hyper` | payload | registered | video | standalone | royalty-free | neural-int-exact | — | §5 Neural capabilities |
| `wai.video.int_motion` | payload | registered | video | standalone | royalty-free | standard-defined | — | §5 Neural capabilities |
| `wai.video.j2k` | payload | registered | video | standalone | royalty-free | standard-tolerance | — | §5 Immersive audio, cinema, volumetric, haptics, HDR |
| `wai.video.keys` | record | registered | — | — | — | — | — | §5 Neural capabilities |
| `wai.video.lcevc` | companion | registered | video | enhancement | patent-pool | standard-unclassified | — | §5 Broadcast and streaming codecs |
| `wai.video.manifest` | record | registered | — | — | — | — | — | §5 Neural capabilities |
| `wai.video.package` | record | registered | — | — | — | — | — | §5 Neural capabilities |
| `wai.video.ssai` | record | registered | — | — | — | — | — | §5 Neural capabilities |
| `wai.video.vp9` | payload | registered | video | standalone | royalty-free | standard-defined | — | §5 Broadcast and streaming codecs |
| `wai.video.vvc` | payload | registered | video | standalone | patent-pool | standard-defined | — | §5 Broadcast and streaming codecs |
| `wai.world.live` | payload | registered | interactive | standalone | royalty-free | standard-defined | — | §5 Interactive, temporal & computed content |
| `wai.world.model` | payload | registered | interactive | standalone | royalty-free | standard-defined | — | §5 Interactive, temporal & computed content |
| `wai.world.replay` | payload | registered | interactive | standalone | royalty-free | standard-defined | — | §5 Interactive, temporal & computed content |
| `wai.world.scene` | payload | registered | interactive | standalone | royalty-free | standard-defined | — | §5 Interactive, temporal & computed content |
| identifier (not a capability) | meaning | in |
|---|---|---|
| `wai.det.fixed64` | the fixed-point arithmetic floor the deterministic *-equivalence criteria are computed in | §5 Interactive, temporal & computed content |
| `wai.world.sim.feed0` | the identity simulation step of wai.feed.state | §5 Interactive, temporal & computed content |
| `wai.world.sim.skeletal0` | the skeletal-pose simulation step of wai.avatar.pose | §5 Interactive, temporal & computed content |
| unregistered label | reason |
|---|---|
| `wai.quantum.mitigate` | a receipt-kind label the wai-quantum crate seals; no revision of SPEC or of an extension registers it, so it is not a registered record |
| `wai.quantum.decode` | a receipt-kind label the wai-quantum crate seals; no revision of SPEC or of an extension registers it, so it is not a registered record |
| `wai.quantum.compile` | a receipt-kind label the wai-quantum crate seals; no revision of SPEC or of an extension registers it, so it is not a registered record |
| `wai.quantum.rearrange` | a receipt-kind label the wai-quantum crate seals; no revision of SPEC or of an extension registers it, so it is not a registered record |
| `wai.quantum.qbom` | a receipt-kind label the wai-quantum crate seals; no revision of SPEC or of an extension registers it, so it is not a registered record |
<!-- end generated -->

343
wai/SPEC.md
View file

@ -11,11 +11,14 @@ file:
does not ship its decoder. Its identity MAY be pinned by digest (a
`Prior`), and the pinned artifact's wire form MAY be declared
([prior-carriage](extensions/prior-carriage.md)).
- the **zeroth condition** — a *menu of registered SOTA standard
codecs*. Always satisfiable on any device that ships ffmpeg or
equivalent. WAI does **not** define new codecs; it dispatches to the
field's best-in-class lossy+lossless libraries (AVIF, JPEG-XL, PNG,
Opus, FLAC, AV1, zstd, XZ).
- the **zeroth condition** — a *menu of registered standard codecs*
(AVIF, JPEG XL, PNG, Opus, FLAC, AV1, zstd, XZ and the others §5
registers), each decoded by the sink with the library it already has.
WAI dispatches registered standard codecs and also defines WAI-native
capability classes (§5): integer codecs, worlds, scores, films and the
other extensions, each with its own payload format and conformance
criterion.
The capability a file requires is *named*, not supplied. Sinks that
have the named capability use it; sinks that don't fall back to a
@ -27,10 +30,11 @@ Keywords **MUST**, **MUST NOT**, **SHOULD**, **MAY** are RFC 2119.
## 1. Scope and model
WAI is a **container + capability dispatch standard**. It is media-
general (image, audio, video, text), royalty-free, and built around the
principle that the standard's value is the envelope and the dispatch
rules — not in re-implementing existing SOTA codecs.
WAI is a **container + capability dispatch standard**. It is
media-general; its floor and recommended set are royalty-free. Its
substance is the envelope, the dispatch rules and the registry's classes:
a standard codec is dispatched to a sink's existing decoder, not
re-implemented.
A WAI file declares (in its manifest):
- what media it carries,
@ -40,8 +44,8 @@ A WAI file declares (in its manifest):
Conforming sinks MUST implement the container envelope (§2), the
manifest schema (§3), the capability-dispatch algorithm (§4), and at
least the **mandatory floor** capabilities (§5). Other capabilities
(neural and additional zeroth) are OPTIONAL but RECOMMENDED.
least the **mandatory floor** capabilities (§5). Other capabilities are
OPTIONAL; §5 says which are RECOMMENDED.
---
@ -73,7 +77,8 @@ All WAI multi-byte integers are little-endian unless stated otherwise.
```json
{
"wai": "1.0",
"media": "image" | "audio" | "video" | "text",
"media": "image" | "audio" | "video" | "text" | "volumetric" | "haptic"
| "metadata" | "interactive" | "binary",
"intent": "replicate" | "create" | "improve" | "...",
"model_requirement": { "capability": "<capability-string>",
"fallback": "<capability-string>" | null },
@ -86,6 +91,13 @@ All WAI multi-byte integers are little-endian unless stated otherwise.
- `wai` — spec version. This document defines `"1.0"`. Unknown major →
reject. Unknown minor → accept by ignoring unknown fields.
- `media` — media class; informational. The capability is authoritative.
The values are the registry's media classes (`codecs::capability_media`;
[`registry/capabilities.json`](registry/capabilities.json)): `image`,
`audio`, `video`, `text`, `volumetric` (meshes, point clouds, Gaussian
splats), `haptic`, `metadata` (data applied with another payload),
`interactive` (worlds, feeds, avatars) and `binary` (anything else). A
reader MUST accept a value it does not know: the field is informational,
and a MINOR revision may add a value (§8).
- `intent` — generative intent. `replicate` (reconstruct the source) is
the only intent with normative reconstruction semantics in v1.0. What it
promises is set by the capability's registry class (§7, *Replicate-soundness*):
@ -127,14 +139,24 @@ A conforming sink MUST implement this algorithm:
1. Read the manifest. If `wai` major version is unknown → reject.
2. Let `cap = model_requirement.capability`.
3. If the sink advertises `cap`, decode `payload` via the codec
registered for `cap` (§5) and return.
3. If `cap` is dispatchable (below) and the sink advertises `cap`, decode
`payload` via the codec registered for `cap` (§5) and return.
4. Else, let `fb = model_requirement.fallback`.
5. If `fb != null` and the sink advertises `fb`, decode via `fb`'s
codec and return.
5. If `fb != null`, `fb` is dispatchable and the sink advertises `fb`,
decode via `fb`'s codec and return.
6. Else the file is **inert at this sink** — return a clear "missing
capability" error to the calling application.
A capability is **dispatchable** unless this revision registers it as something
a sink does not decode on its own: a candidate (§5, *Candidate capabilities*), a
companion-only capability (§5, *Companion-only capabilities*), a record or a
derivation (§5, *Kinds of registered string*). A sink MUST NOT take step 3 or
step 5 with a capability that is not dispatchable, whatever it advertises. A
string this revision does not register is dispatchable: a later MINOR revision
may register it (§8), and a sink that has a codec for it says so by advertising
it. A `WAI1` whose `cap` is companion-only does not conform and is inert: its
payload is companion data, so step 5 is not taken either (§5).
A WAI **payload always corresponds to exactly one capability**. v1.0
does not support multi-codec multiplexed payloads.
@ -148,25 +170,55 @@ this algorithm on the plaintext; see §6.4.
WAI v1.0 registers the following capabilities. Each names a standard
codec or a WAI-native class. The mandatory floor and the recommended
set are royalty-free; every registered capability carries a licence
class (see the end of this section), and one that is not royalty-free
is never a default. The "payload format" column is the
set are royalty-free; every payload and companion capability carries a
licence class (see the end of this section), and one that is not
royalty-free is never a default. The "payload format" column is the
on-the-wire payload bytes; in every case it is the canonical bytes
that codec normally emits (a JXL file, an Opus stream, etc.) — WAI
does not wrap or re-frame them.
The registry is also published in machine-readable form, generated from
the reference implementation: [`registry/capabilities.json`](registry/capabilities.json)
and its schema, with the policy that governs it in [`REGISTRY.md`](REGISTRY.md).
Where the two differ, this section is normative and the generated files are
wrong.
**Kinds of registered string.** Every string this section registers has one
kind:
- a **payload** capability: an envelope's `capability`, `fallback` or `WAI2`
rendition names the codec its payload is decoded with;
- a **companion** capability: companion-only (*Companion-only capabilities*,
below), carried only as a `WAI2` companion;
- a **derivation** operation (`wai.derive.*`): an operation on samples another
capability decoded, which names no payload format;
- a **record**: a signed record a WAI tool emits and verifies (a receipt kind —
`wai.asset.provenance`, `wai.quantum.calibration`, `wai.quantum.job`, and the
delivery records `wai.video.manifest`, `wai.video.ssai`, `wai.video.keys`,
`wai.video.package` and `wai.video.channel`), which names no payload format.
Only payload and companion capabilities have a media class, a fidelity tier, a
replicate class and a role. A derivation or record string is not a payload
capability and is not dispatchable (§4): step 3 does not take an envelope's
`capability` that names one, and step 5 may still take a usable fallback. A row of the tables below that
registers a record says what the record attests, not a payload format. Named
sub-identifiers this section uses — `wai.det.fixed64`, `wai.world.sim.feed0`,
`wai.world.sim.skeletal0` — name parts of a capability's definition and are
not capabilities.
### Mandatory floor
Every conforming sink MUST implement:
| capability | payload | role |
|---|---|---|
| `wai.image.png` | PNG (RFC 2083) | universal lossless image |
| `wai.audio.flac` | FLAC (xiph.org/flac) | universal lossless audio |
| `wai.text.zstd` | zstd (RFC 8478) | universal compressed bytes |
| `wai.image.png` | PNG (W3C Recommendation, *Portable Network Graphics (PNG) Specification*, Third Edition) | universal lossless image |
| `wai.audio.flac` | FLAC (RFC 9639) | universal lossless audio |
| `wai.text.zstd` | Zstandard (RFC 8878) | universal compressed bytes |
These three guarantee that any sink with the standard system codec
libraries can open *some* file in every WAI-supported media class.
libraries can open a file in the image, audio and text classes. The floor
has no video capability.
### Recommended modern set
@ -174,11 +226,11 @@ Every conforming sink SHOULD implement:
| capability | payload | role |
|---|---|---|
| `wai.image.jxl` | JPEG-XL (ISO/IEC 18181) | lossless + lossy, modern SOTA |
| `wai.image.avif` | AVIF (ISO/IEC 23000-22) | lossy SOTA (AV1-based) |
| `wai.image.jpeg` | JPEG (ITU-T T.81) | legacy compatibility |
| `wai.audio.opus` | Opus (RFC 6716) | lossy SOTA |
| `wai.video.av1` | AV1 (AOMedia) | lossy SOTA |
| `wai.image.jxl` | JPEG XL (ISO/IEC 18181) | lossless + lossy |
| `wai.image.avif` | AVIF (the AOM AVIF specification: an AV1 image format based on HEIF, conforming to MIAF) | lossy (AV1-based) |
| `wai.image.jpeg` | JPEG (ITU-T T.81) | compatibility with existing JPEG decoders |
| `wai.audio.opus` | Opus (RFC 6716) | lossy speech and music |
| `wai.video.av1` | AV1 (AOMedia) | lossy |
| `wai.video.av1.lossless` | AV1 (AOMedia), `quantizer=0` | mathematically lossless YUV (lossy in RGB due to color conversion) |
| `wai.text.xz` | XZ/LZMA2 | maximum classical text ratio |
@ -189,74 +241,69 @@ conditioning representation specific to the capability (latent tokens,
prompt embeddings, etc.). The standard does NOT specify the byte
layout of these payloads — the capability owner does.
Neural capabilities are picked on the **best-quality-per-bit at each
bitrate band** principle — not on whatever model is famous or small.
2026 SOTA per medium, with the wire-byte win each one delivers. The
decode route for each is implemented in the reference sink (see the
`source` column / `neural/*.rs`); the model weights are sink-supplied.
Each row names the model and the operating point its own publication
reports (bitrate, sample rate) or the property WAI's integer construction
gives it; this section states no comparison between models or with the
zeroth codecs. The decode route for each is implemented in the reference
sink (see the `source` column / `neural/*.rs`); the model weights are
sink-supplied.
*Audio:*
| capability | model | win vs zeroth | source |
| capability | model | operating point | source |
|---|---|---|---|
| `wai.neural.encodec32` | EnCodec at 32 kHz, 4 codebooks | ~3 kbps acceptable music | `facebook/encodec_32khz` |
| `wai.neural.dac` | DAC at 44.1 kHz | 6 kbps near-transparent music | `descript/dac_44khz` |
| `wai.neural.mimi` | Mimi, 12.5 Hz frame rate | ~1.1 kbps real-time speech | `kyutai/mimi` |
| `wai.neural.wavtokenizer` | WavTokenizer single-codebook VQ | 0.9 kbps beats DAC @ 9 kbps on UTMOS | `novateur/WavTokenizer-large-*` |
| `wai.neural.snac` | SNAC multi-resolution RVQ (hierarchical code levels, own sample rate) | low-bitrate TTS standard | `neural/snac.rs` |
| `wai.neural.encodec32` | EnCodec at 32 kHz, 4 codebooks | 32 kHz; 4 codebooks of 2048 at 50 Hz (2.2 kbps) | EnCodec (arXiv 2210.13438) |
| `wai.neural.dac` | DAC at 44.1 kHz | 44.1 kHz; 9 codebooks at 86 Hz (about 8 kbps) | DAC (arXiv 2306.06546) |
| `wai.neural.mimi` | Mimi, 12.5 Hz frame rate | 24 kHz; 8 codebooks of 2048 at 12.5 Hz (1.1 kbps), real-time speech | Mimi (arXiv 2410.00037) |
| `wai.neural.wavtokenizer` | WavTokenizer single-codebook VQ | 24 kHz; one codebook at 40 or 75 tokens per second (under 1 kbps) | WavTokenizer (arXiv 2408.16532) |
| `wai.neural.snac` | SNAC multi-resolution RVQ (hierarchical code levels, own sample rate) | hierarchical code levels at the model's own sample rate | SNAC (arXiv 2410.14411), `neural/snac.rs` |
*Image:*
| capability | model | win vs zeroth | source |
| capability | model | operating point or property | source |
|---|---|---|---|
| `wai.neural.bmshj2018` | bmshj2018-factorized + rANS (byte-exact CompressAI port) | ~80× vs raw RGB at q=3 (~0.26 bpp, 29 dB PSNR) | CompressAI |
| `wai.neural.mbt2018_mean` | mean-scale hyperprior (Gaussian-conditional entropy) | conditional-entropy family, stronger than factorized | `neural/hyperprior.rs` |
| `wai.neural.bmshj2018` | bmshj2018-factorized + rANS (byte-exact CompressAI port) | about 0.26 bpp at 29 dB PSNR (quality 3) | CompressAI |
| `wai.neural.mbt2018_mean` | mean-scale hyperprior (Gaussian-conditional entropy) | the conditional-entropy family (arXiv 1809.02736) | `neural/hyperprior.rs` |
| `wai.neural.int_synth` | integer fixed-point synthesis (all-i64 transposed-conv, no float/ONNX) | bit-identical on every machine — soundly backs `intent = replicate` | `neural/int_synth.rs` (QAT weights sink-supplied) |
| `wai.neural.int_hyper` | integer fixed-point **conditional** codec — whole bitstream→image path integer (integer hyper-decoder σ → integer-σ entropy → integer synthesis) | the integer-entropy contract fully assembled; high quality (PTQ of mbt2018_mean) bit-identical on every machine | `int_hyper_synth.rs` (tables sink-supplied) |
| `wai.neural.int_hyper` | integer fixed-point **conditional** codec — whole bitstream→image path integer (integer hyper-decoder σ → integer-σ entropy → integer synthesis) | the integer-entropy contract fully assembled: a PTQ of mbt2018_mean, bit-identical on every machine, whose integer decode CI holds within 1 dB of the same model's float decode on a pinned photograph | `int_hyper_synth.rs` (tables sink-supplied) |
| `wai.neural.int_mlicpp` | integer fixed-point **MLIC++ entropy topology** — `K` channel slices of the latent, each decoded as a checkerboard anchor pass then a non-anchor pass, with an entropy network over concat[latent decoded so far, hyper] re-run before every pass to give σ‖μ; hyper-decoder, entropy network and synthesis all integer | the channel-autoregressive × checkerboard entropy model at **decode-equivalence**: the encoder and every decoder compute the same σ, so the same distribution for every element, at any thread count; bit-identical on every machine. The committed conformance vector has synthetic weights, so it demonstrates decode-equivalence and states no quality figure | `int_mlicpp.rs` (one-file model bundle sink-supplied; `tools/wai_int_mlicpp_export.py`) |
| `wai.image.jpegai` | ISO/IEC 6048 JPEG-AI codestream → RGB — integer entropy front end (mANS, with an integer scale-index derivation) + 16-bit-PTQ integer hyper-decoder, context-model and synthesis networks + dequantisation, latent refinement, post-filters (the eICCI CNN among them) and colour conversion in IEEE-754 float with a fixed evaluation order. Scope: one configuration — 256×256 4:4:4, `model_id` 0 (tools_0), `beta_displacement_log` 0, the high-operation-point synthesis, the eICCI model pair the weight bundle carries, and the other header conditions in [determinism-tiers](extensions/determinism-tiers.md) §5; both decode paths refuse any other codestream. Within it, a sink without the integer weight bundle, or in a non-default floating-point environment, takes a float ONNX decode instead. Only checkpoint constants are sink-supplied | **entropy-consistency** (registry class `NeuralEntropyExact`): the entropy decode, including the scale-index derivation, is integer on both decode paths, so the decoded symbols reproduce on every machine. The exact decoder also pins one reconstruction to RGB, reproducible across machines under the build and floating-point conditions in [determinism-tiers](extensions/determinism-tiers.md) §5–§6 (it checks the floating-point environment before decoding) — a property of that decoder, not of the capability, so `wai.image.jpegai` does not back `replicate` (§7). On one reference codestream the exact decoder stays within max\|Δ\| ≤ 2 of the reference decoder (recorded max\|Δ\| = 1 on ~4.4% of samples); evidence in [determinism-tiers](extensions/determinism-tiers.md) §6. **Downgraded in v0.4.0:** this row previously claimed decode-equivalence, and the registry classed the capability `StandardDefined` | `jpegai_decode.rs` (+ `jpegai_{bitstream,mans,pipeline,synthesis,icci}.rs`) |
| `wai.image.jpegai` | JPEG AI (ISO/IEC 6048-1:2025) codestream → RGB — integer entropy front end (mANS, with an integer scale-index derivation) + 16-bit-PTQ integer hyper-decoder, context-model and synthesis networks + dequantisation, latent refinement, post-filters (the eICCI CNN among them) and colour conversion in IEEE-754 float with a fixed evaluation order. Scope: one configuration — 256×256 4:4:4, `model_id` 0 (tools_0), `beta_displacement_log` 0, the high-operation-point synthesis, the eICCI model pair the weight bundle carries, and the other header conditions in [determinism-tiers](extensions/determinism-tiers.md) §5; both decode paths refuse any other codestream. Within it, a sink without the integer weight bundle, or in a non-default floating-point environment, takes a float ONNX decode instead. Only checkpoint constants are sink-supplied | **entropy-consistency** (registry class `NeuralEntropyExact`): the entropy decode, including the scale-index derivation, is integer on both decode paths, so the decoded symbols reproduce on every machine. The exact decoder also pins one reconstruction to RGB, reproducible across machines under the build and floating-point conditions in [determinism-tiers](extensions/determinism-tiers.md) §5–§6 (it checks the floating-point environment before decoding) — a property of that decoder, not of the capability, so `wai.image.jpegai` does not back `replicate` (§7). On one reference codestream the exact decoder stays within max\|Δ\| ≤ 2 of the reference decoder (recorded max\|Δ\| = 1 on ~4.4% of samples); evidence in [determinism-tiers](extensions/determinism-tiers.md) §6. **Downgraded in v0.4.0:** this row previously claimed decode-equivalence, and the registry classed the capability `StandardDefined` | `jpegai_decode.rs` (+ `jpegai_{bitstream,mans,pipeline,synthesis,icci}.rs`) |
| `wai.generate.spectral` | deterministic integer GENERATIVE synthesis (`intent=create`) — prompt+seed → integer-LUT spectral composition, no float/ONNX | byte-identical on every machine, pinned to (model, prompt, seed, steps); the create-side of "conformance is a hash" (generation-equivalence) | `generate.rs` (small pinned model bundle) |
| `wai.neural.elic` | checkerboard context model (hyperprior + 2-pass spatial context) | the JPEG-AI / MLIC++ entropy class | `neural/context.rs` |
| `wai.neural.mlicpp` | MLIC++ multi-reference entropy (channel-AR slices + checkerboard + global attention) | learned image, low-bitrate high-fidelity | research code, `neural/mlicpp.rs` |
| `wai.neural.mlicv2` | MLICv2 / MLICv2+ (2025), MetaFormer transform | learned-image fidelity frontier; shares the mlicpp route | research code, `neural/mlicpp.rs` |
| `wai.neural.cmic` | Content-Adaptive Mamba (ICLR 2026), SSM transform + SCCTX entropy | learned-image frontier; shares the mlicpp route | arXiv 2508.02192, `neural/mlicpp.rs` |
| `wai.neural.glc` | Generative Latent Coding — categorical / VQ entropy | <0.05 bpp where JPEG-XL collapses | research code, `neural/glc.rs` |
| `wai.neural.elic` | checkerboard context model (hyperprior + 2-pass spatial context) | the checkerboard-context entropy class (arXiv 2203.10886) | `neural/context.rs` |
| `wai.neural.mlicpp` | MLIC++ multi-reference entropy (channel-AR slices + checkerboard + global attention) | the multi-reference entropy model (arXiv 2307.15421) | research code, `neural/mlicpp.rs` |
| `wai.neural.mlicv2` | MLICv2 / MLICv2+ (2025), MetaFormer transform | shares the mlicpp route | research code, `neural/mlicpp.rs` |
| `wai.neural.cmic` | Content-Adaptive Mamba (ICLR 2026), SSM transform + SCCTX entropy | shares the mlicpp route | arXiv 2508.02192, `neural/mlicpp.rs` |
| `wai.neural.glc` | Generative Latent Coding — categorical / VQ entropy | ultra-low-bitrate image coding | research code, `neural/glc.rs` |
| `wai.semantic.clip` | a CLIP-embedding package regenerated by any conforming sink generator (the generative-semantic path, `semantic.rs`) | conforms by similarity above a declared threshold, never by bytes (`Behavioral`, tier `Perceptual`); generators are sink-supplied | `semantic.rs` |
| `wai.semantic.perco2` | PerCoV2 ultra-low-bitrate generative image coding | as `wai.semantic.clip`; its model licence is non-commercial (CC BY-NC-SA), so it is licence-flagged and never a default | `semantic.rs` |
*Video:*
| capability | model | win vs zeroth | source |
| capability | model | operating point or property | source |
|---|---|---|---|
| `wai.neural.video_bmshj2018` | per-frame bmshj2018 | 1000×+ vs raw RGB, browser-decodable on WebGPU | CompressAI |
| `wai.neural.dcvc_rt` | DCVC-RT (2025), inter-frame with reference-feature propagation | real-time neural video at 1080p/4K | research code, `neural/video_dcvc.rs` |
| `wai.neural.dcvc_fm` | DCVC-FM (feature modulation), wide single-model quality range (RGB + YUV) | SOTA-RD neural video; shares the inter-frame route | research code, `neural/video_dcvc.rs` |
| `wai.neural.video_bmshj2018` | per-frame bmshj2018 | per-frame image coding; decodable in a browser on WebGPU | CompressAI |
| `wai.neural.dcvc_rt` | DCVC-RT (2025), inter-frame with reference-feature propagation | inter-frame neural video; its paper reports real-time 1080p and 4K decode on a GPU | research code, `neural/video_dcvc.rs` |
| `wai.neural.dcvc_fm` | DCVC-FM (feature modulation), wide single-model quality range (RGB + YUV) | a wide single-model quality range; shares the inter-frame route | research code, `neural/video_dcvc.rs` |
| `wai.video.int_motion` | DETERMINISTIC INTEGER inter-frame — I-frame + integer block motion-compensated P-frames + integer-quantized residual over rANS, no float/ONNX; motion is whole-pel or **half-pel** (`mv_precision`), the latter predicting at the half-grid by integer bilinear interpolation (`(a+b+1)>>1`, `(a+b+c+d+2)>>2`) so real footage — which pans a non-integer number of pixels per frame — costs fewer residual bits while staying byte-exact | byte-identical frame-by-frame on every machine (*frame-state-equivalence*, a per-frame portable hash); the temporal extension of the integer contract, where a float inter-frame decode desyncs down the GOP | `int_video.rs` (whole-pel `tools/wai_int_video_encode.py`; half-pel from real footage `tools/wai_int_video_real_encode.py`) |
| `wai.video.int_hyper` | NEURAL-KEYFRAME inter-frame — the I-frame is the LEARNED integer codec `int_hyper` decoded byte-exact, the P-frames are integer block-MC residuals on top | a learned keyframe you can put in a receipt + integer inter-frames; whole decode integer, byte-identical everywhere | `int_video::decode_pframes` ∘ `int_hyper_synth::decode_conditional` |
| `wai.video.manifest` | ATTESTED PER-SESSION MANIFEST — the OTT delivery layer, not a codec: a personalised HLS manifest (ads stitched at `EXT-X-DISCONTINUITY`, blackouts + geo enforced) rendered as a **pure function of the session *class*** `(content, entitlement, geo, blackout set, ad pod, window)` rather than the viewer, and sealed in an Ed25519 receipt that binds the JCP grant (capability, joule ceiling, `funds_ceiling` = ad budget), the composition, the stitched ads and the policy applied | **manifest-equivalence**: `manifest_hash == hash(render(segments))` and `session_class_hash` are recomputable by anyone holding the bytes, so the ads, blackouts and compose-work stop being dashboard claims; class-determinism means one generation serves every viewer of the class, and `uj_per_viewer` is the amortised energy that a 1:1-unique generator cannot report below its per-viewer cost. Only `joules_micro` is measured-attested | `wai_video_manifest.rs` |
| `wai.video.ssai` | ATTESTED SERVER-SIDE AD INSERTION — a **deterministic** ad decision (greedy by price within the avail duration and the funds remaining) plus a per-impression Ed25519 receipt binding the candidate pod, the manifest it belongs to, the served `creative_hash`es and the VAST quartile each reached | **spend-equivalence**: `budget_spent_micros` is recomputed from the *billable* impressions and rejected if it exceeds the grant's `funds_ceiling` — overspend cannot be signed; every creative and quartile is checkable, so a buyer and seller reconcile impressions from the same bytes instead of two logs that only a dashboard joins. Only `joules_micro` is measured-attested | `wai_video_ssai.rs` |
| `wai.video.ssai` | ATTESTED SERVER-SIDE AD INSERTION — a **deterministic** ad decision (greedy by price within the avail duration and the funds remaining) plus a per-impression Ed25519 receipt binding the candidate pod, the manifest it belongs to, the served `creative_hash`es and the VAST quartile each reached | **spend-equivalence**: `budget_spent_micros` is recomputed from the *billable* impressions and rejected if it exceeds the grant's `funds_ceiling` — an overspent receipt fails verification; every creative and quartile is checkable, so a buyer and seller reconcile impressions from the same bytes instead of two logs that only a dashboard joins. Only `joules_micro` is measured-attested | `wai_video_ssai.rs` |
| `wai.video.keys` | ATTESTED CONTENT-KEY RELEASE — the DRM licensing decision as a record: every key release, and every refusal, is an Ed25519 receipt binding the KID, the DRM system, the entitlement demanded and the window enforced, with a BLAKE3 `key_commit` proving *which* key was released without carrying it | **decision-equivalence**: `verify()` re-runs the release policy over the receipt's own `(grant capability, entitlement, issued, expiry)` and rejects any misreported outcome, and enforces granted-iff-committed — so "keys went only to entitled, unexpired sessions" is checkable against the signed log, not asserted. Only `joules_micro` is measured-attested | `wai_video_keys.rs` |
| `wai.video.package` | DETERMINISTIC PACKAGING + ATTESTED SEGMENTMAP — a source sliced into content-hashed segments on GOP boundaries; the ordered `(index → content_hash)` map is a signed Merkle commitment, and identical content (ads, bumpers, replays, simulcast overlap) dedups to one stored/encoded copy | **map-equivalence + verifiable dedup**: `segment_map_root` lets a `wai.video.manifest` reference be checked against the package that produced it, and `verify()` recomputes `unique_segments`/`stored_bytes` from the segments so the storage/encode saving cannot be overstated; `uj_per_unique_segment` is the encode-once energy. Only `joules_micro` is measured-attested | `wai_video_package.rs` |
| `wai.video.channel` | VIRTUAL LINEAR / FAST ASSEMBLY — a continuous schedule of assets over wall-clock time, sliced into attested playback windows; windows chain via `parent_receipt_hash`, so a channel's history is a hash chain of program-as-run records | **schedule-continuity + recomputable program-as-run**: `verify()` rejects a schedule with a gap or overlap (no dead air, no double-booking) and recomputes `aired` from the schedule and the window, so what actually aired is checkable by a licensor, advertiser or regulator against the signed chain rather than an operator-held play-out log. Only `joules_micro` is measured-attested | `wai_video_channel.rs` |
**Sink-architecture note:** the inter-frame neural video codecs (DCVC-RT,
DCVC-FM) decode through `ort` and are **CPU-capable** for native sinks —
their real-time figures come from GPU execution, but the decode is not
CUDA-only. They are heavier than per-frame image codecs (a temporal net
their real-time figures come from GPU execution, but a GPU is not required
to decode. They are heavier than per-frame image codecs (a temporal net
plus reference-feature state), so a browser sink that cannot host the
temporal net should declare only `wai.neural.video_bmshj2018` (or another
browser-decodable neural video codec) for the `video` media class. The
choice is the deployer's, advertised per sink — the envelope is identical.
**TAESD is intentionally absent.** TAESD is a distilled SD VAE decoder
— it inverts a generative pipeline's encoder, not a general image
codec. Outside the SD pipeline it underperforms JPEG-XL at most
bitrates while costing ~5 MB of weights and ONNX compute. Don't ship
codecs for the brand; ship them for the bit budget.
**Image/video scope is honest:** zeroth codecs (JPEG-XL, AV1) are
already SOTA-adjacent in the bitrate bands where most media lives.
Neural image/video capabilities are registered for the bands where
zeroth codecs collapse (ultra-low bpp images, bandwidth-constrained
streaming) — not as defaults.
**Scope of the neural rows.** A neural image or video capability is an
option a deployer may declare and a sink may advertise; none is in the
floor or the recommended set, and none is a default.
Each capability owns its payload format. Decoder implementations
(Python source-side, ONNX-export, browser onnxruntime-web) live in
@ -302,7 +349,7 @@ reconstruction, with the acquisition class and coverage C2PA 2.4's
| capability | payload | status · extension |
|---|---|---|
| `wai.prior.nnc` | a trained network's parameters as an ISO/IEC 15938-17:2024 (NNC, 2nd edition) bitstream | the registered wire form a `Prior` MAY declare (`prior.form`); its digest then covers the coded bitstream. Optional, not recommended: royalty status is not established, and cross-decoder exactness of the reconstruction is not established, so a `replicate` over a neural capability cannot rest on a prior read out of it. As a payload it takes the replicate class of a standard whose decoder conformance the registry has not checked · [prior-carriage](extensions/prior-carriage.md) |
| `wai.tensor.nnc` | general tensorial data (split-inference feature maps, 3D/4D scene parameters) coded with NNC | **candidate**: general-tensor coding is in an unpublished amendment, so the payload format is not fixed and MUST NOT be emitted until pinned · [prior-carriage](extensions/prior-carriage.md) |
| `wai.tensor.nnc` | general tensorial data (split-inference feature maps, 3D/4D scene parameters) coded with NNC | **candidate**: general-tensor coding is in an unpublished amendment, so the payload format is not fixed and MUST NOT be emitted until pinned (*Candidate capabilities*, below) · [prior-carriage](extensions/prior-carriage.md) |
The ten derivation operations of ISO/IEC 23001-16:2021 (derived visual tracks)
are registered as `wai.derive.*` strings, each with a per-operation determinism
@ -322,10 +369,10 @@ standard. See the linked extension for each.
| capability | payload | conformance · extension |
|---|---|---|
| `wai.world.scene` / `.replay` / `.live` | WWLD container (op log) | replay-equivalence · [interactive-worlds](extensions/interactive-worlds.md) |
| `wai.world.scene` / `wai.world.replay` / `wai.world.live` | WWLD container (op log) | replay-equivalence · [interactive-worlds](extensions/interactive-worlds.md) |
| `wai.world.model` | generative-world transport: a signed **action trace** (the world CausalOp log) + a declared `model_id` (`world_model.rs`) | the wire carries the *action trace*, not the pixels — it replays under canonical total order to one **session hash** on every machine (replay-equivalence), the generated frames vouched for by a signed, joules-accounted provenance receipt. The neural sibling of `.replay`; with an `int_*` renderer the frames hash-converge too (a generative interactive world you can put in a receipt) |
| `wai.mesh.glb` | glTF 2.0 Binary (Khronos) | asset, content-addressed (decoder sink-supplied) |
| `wai.splat.spz` | SPZ (Niantic) | asset, content-addressed; presentation only |
| `wai.splat.spz` | SPZ (an open-source Gaussian-splat format) | asset, content-addressed; presentation only |
| `wai.audio.scene` | WSCN scene (objects + trajectories) | mixdown-equivalence · [spatial-audio](extensions/spatial-audio.md) |
| `wai.audio.score` | WSCR container (note events) | music as instructions; sink synthesizes, mixdown-equivalence · [score](extensions/score.md) |
| `wai.audio.int_codec` | LEARNED 1-D transform codec — integer transposed-conv synthesis + integer IGDN over a factorized-rANS latent (model + bitstream from `tools/wai_int_audio_export.py`) | **sample-equivalence**: int16 PCM byte-identical on every machine (a portable PCM hash); the audio analog of `wai.neural.int_hyper`, where a float learned-audio decode desyncs on ~1e-5 round-off. Pure i64, no float/ONNX (`int_audio.rs`) |
@ -336,7 +383,7 @@ standard. See the linked extension for each.
| `wai.splat.int_codec` | LEARNED Gaussian-splat ATTRIBUTE codec — a learned transform over the `[14,S,S]` attribute image (pos/scale/rot/color/opacity), integer synthesis (int_hyper machinery, 14 channels) over a factorized-rANS latent (`int_splat.rs`; model + bitstream from `tools/wai_int_splat_export.py`) | **splat-equivalence**: the decoded attribute tensor is byte-identical on every machine (a portable cloud hash) — the int_hyper of 3D, where a float splat-attribute decode desyncs. The float rasterizer stays sink-supplied presentation; the *attributes* are the contract. Pure i64, no float/ONNX |
| `wai.film.linear` | WFLM container (sealed world + camera track + frame schedule) | film-equivalence · [linear-film](extensions/linear-film.md) |
| `wai.film.reel` | WREL container (sealed world + a sequence of shots = camera track + schedule each) | reel-equivalence (film-equivalence with cuts) · [linear-film](extensions/linear-film.md) |
| `wai.ternary.tt` | WQTT container (balanced-ternary MPS cores — packed trits + integer per-column scales) | a **quantum-inspired** (tensor-network) transform ported from the ternary/MPS fabric with ONE change: the per-column scale is an **integer**, so the whole decode is `i64`/`i128` and the reconstruction is byte-identical on every machine — **reconstruction-equivalence** (a portable recon hash), receiptable. Honest scope: a **provenance** transform, NOT a rate-competitive codec (measured RD is mediocre — `Lossy` tier; see the extension's §5 verdict). Encoder-side float SVD only · [ternary-tt](extensions/ternary-tt.md) |
| `wai.ternary.tt` | WQTT container (balanced-ternary MPS cores — packed trits + integer per-column scales) | a **quantum-inspired** (tensor-network) transform ported from the ternary/MPS fabric with ONE change: the per-column scale is an **integer**, so the whole decode is `i64`/`i128` and the reconstruction is byte-identical on every machine — **reconstruction-equivalence** (a portable recon hash), receiptable. Scope: a **provenance** transform, measured at 13.3 dB SNR for 3.8× with six residual layers (`Lossy` tier; the extension's §5). Encoder-side float SVD only · [ternary-tt](extensions/ternary-tt.md) |
| `wai.quantum.circuit` | WQC container (gate op-log — Clifford+T + dyadic phase) | the wire carries a **circuit** (kilobytes, linear in gate count), the sink CLASSICALLY simulates it in a fixed-point **complex** floor (`Amp` = i64 re/im), and the reconstructed `2^n`-amplitude statevector is byte-identical on every machine — **statevector-equivalence** (a portable statevector hash); measurement is sampled by the pinned `splitmix64` so a shot histogram at a pinned `(seed, shots)` converges too (**shot-histogram-equivalence**). The dyadic phase ladder gives the **QFT** ("quantum FFT"), transported as a circuit and reconstructed byte-exact. No quantum hardware — NISQ noise breaks the determinism the standard rests on; this makes a quantum *computation* transportable + receiptable, with **no** speedup claimed · [quantum-sim](extensions/quantum-sim.md) |
And the verify half — a signed, energy-accounted lineage for opaque
@ -362,11 +409,56 @@ named standard's canonical bytes.
| class | capabilities |
|---|---|
| immersive audio | `wai.audio.iamf` (IAMF/Eclipsa, royalty-free) · `wai.audio.ac4` (Dolby AC-4, proprietary) · `wai.audio.ivas` (3GPP IVAS, SEP-pool) · `wai.audio.mpegh` (MPEG-H 3D Audio, SEP-pool) · `wai.audio.avs3` (AVS3-P3 / Audio Vivid, SEP-pool) — the last two with the broadcast codecs below |
| immersive audio | `wai.audio.iamf` (IAMF, royalty-free) · `wai.audio.ac4` (AC-4, ETSI TS 103 190, proprietary) · `wai.audio.ivas` (3GPP IVAS, SEP-pool) · `wai.audio.mpegh` (MPEG-H 3D Audio, SEP-pool) · `wai.audio.avs3` (AVS3-P3 / Audio Vivid, SEP-pool) — the last two with the broadcast codecs below |
| cinema mastering | `wai.image.jp2` · `wai.video.j2k` (JPEG 2000 / DCP, OpenJPEG) |
| volumetric / 3D | `wai.mesh.draco` (royalty-free) · `wai.pc.gpcc` · `wai.pc.vpcc` (MPEG point clouds) · `wai.splat.sog` (Gaussian-splat compression, candidate) |
| volumetric / 3D | `wai.mesh.draco` (royalty-free) · `wai.pc.gpcc` (G-PCC, ISO/IEC 23090-9) · `wai.pc.vpcc` (V-PCC, ISO/IEC 23090-5) — MPEG point clouds · `wai.splat.sog` (Gaussian-splat compression; a **candidate**, no payload format pinned — *Candidate capabilities*, below) |
| haptics | `wai.haptic.mihs` (MPEG-I Haptics, ISO/IEC 23090-31; its 23090-33 ships bit-exact conformance vectors) |
| HDR dynamic metadata | `wai.meta.hdr10plus` (royalty-free) · `wai.meta.dovi` (Dolby Vision, proprietary) — **companion descriptors** to a video payload, not standalone media |
| HDR dynamic metadata | `wai.meta.hdr10plus` (SMPTE ST 2094-40 dynamic metadata, royalty-free) · `wai.meta.dovi` (SMPTE ST 2094-10 dynamic metadata, proprietary) — **companion descriptors** to a video payload, not standalone media |
**Point clouds: what the two registrations name.** `wai.pc.gpcc` names G-PCC,
ISO/IEC 23090-9, and `wai.pc.vpcc` names V-PCC, ISO/IEC 23090-5; each payload is
a bitstream of the part it names. A bitstream of another part of MPEG-I is not a
payload of either unless it is also a bitstream of that part. Enhanced G-PCC,
ISO/IEC 23090-38, is such another part, with its own conformance part, so it
would be registered, if at all, under a capability string of its own (§8). Neither
registration names an edition, so each carries the edition of its part current
when it was registered, on 2026-06-17 (§8, *Editions*). Which edition that was is
unresolved: the research registry records it as open, as it does whether
`wai.pc.vpcc` carries the V3C bitstream of MIV. Until a revision records the
edition, what either registration admits beyond the tools of its part's first
edition is unspecified: a sink that advertises either capability is not known to
decode a bitstream that uses a later tool, and an encoder that emits one makes no
claim a sink can rely on. The amendments in preparation that add Gaussian splats
to both parts were not published on the registration date, so each needs a
capability string of its own (§8). Whether V-DMC, which the fourth edition of
ISO/IEC 23090-5 makes a V3C application, is admitted turns on the unresolved
edition.
**Candidate capabilities.** A *candidate* is a registered capability string whose
payload format no revision has pinned. The candidates are `wai.tensor.nnc`
(general-tensor coding is in an unpublished NNC amendment; *Prior artifacts*
above) and `wai.splat.sog` (registered for a family of Gaussian-splat formats; no
specification of the format is cited). A conforming encoder's payload is the
canonical byte stream of its capability (§7), and a candidate has none. So, until a
revision pins a candidate's payload format:
- an encoder MUST NOT emit it: not as a `WAI1` `capability` or `fallback`, and
not as a `WAI2` component in either role;
- a sink MUST treat it as unsupported: it takes neither step 3 nor step 5 of §4
with a candidate, and §7.1's selection never picks a component of one.
The revision that pins a candidate states its payload format, and §8 fixes that
format from that revision on. This rule enforces the candidate registrations as
they were made (§8); it changes no payload format and no role. It makes non-conforming
a `WAI1` envelope that names a candidate as its `capability` or its `fallback`, and
a `WAI2` envelope with a component of a candidate; such envelopes still parse. The
reference implementation lists the candidates in `codecs::is_candidate`, reports
such envelopes from `Wai::check_candidates` and `WaiMulti::check_candidates`, and
never dispatches or selects a candidate: `Wai::dispatch_capability`,
`WaiMulti::pick` and `WaiMulti::primary` take only a capability
`codecs::is_dispatchable` accepts (§4's dispatchable). Its `wai wrap` tool, its C
envelope packers (`wai_envelope_pack`, `wai_envelope_pack_multi`) and its browser
writers (`pack`, `packMulti`) refuse to write one.
### Broadcast and streaming codecs (registered, sink-supplied)
@ -397,7 +489,8 @@ re-frame it.
*SEP-pool* means the standard's essential patents are licensed for royalties —
through patent pools, a single standard-essential-patent licensor, or FRAND
holders. Per-entry sources and caveats are in the research registry linked
holders; it is the `patent-pool` licence class (*Licence classes*, below).
Per-entry sources and caveats are in the research registry linked
below. Each is tiered `Lossy` (its distribution role; the lossless modes some
of them have are signalled in the bitstream, as for AVIF). No model WAI runs
is involved in any of them. For replication (§7) they fall in three classes:
@ -435,8 +528,9 @@ codec's decoder accepts the primary's bitstream (`wai.video.evc.baseline` →
bitstream). A deployer that must not ship pool-licensed decode filters on the
licence class below, not on a hand-kept list.
**AV2 placement.** AV2 is royalty-free, with a published specification and a
reference decoder (AVM v1.0.0). It is registered here and **not** in the
**AV2 placement.** AV2 is licensed royalty-free by its standards body (footnote ⁶
of the research registry records third-party assertions), with a published
specification and a reference decoder (AVM v1.0.0). It is registered here and **not** in the
recommended set, because that set states what every conforming sink SHOULD
implement, and in September 2026 that is not yet a reasonable ask: AVM is a
reference model, a production software decoder is still in development, and
@ -517,18 +611,19 @@ align picture by picture with its base outside a timed container; this
revision does not pin that alignment, so the embedded form is the unambiguous
one until it does.
**Licence classes.** Every registered capability also carries an IP class —
`codecs::capability_license(cap)` ∈ {royalty-free, patent-pool, proprietary,
non-commercial}, with `is_royalty_free(cap)` as the predicate. It is a registry
property (not a per-file manifest claim — a codec's licence can't be honestly
asserted in an envelope's header), so a deployer can audit a title or a whole
catalogue in code: encode only to royalty-free capabilities, or flag every
pool/proprietary one. The per-title choice of which toll road to pay is a
property of the format, enforceable, not a policy on a slide. The
classification is exhaustive with no default class: a capability registered
**Licence classes.** Every payload and companion capability carries a licence
class — `codecs::capability_license(cap)` ∈ {royalty-free, patent-pool,
proprietary, non-commercial}, with `is_royalty_free(cap)` as the predicate. It is
a registry property (not a per-file manifest claim — a codec's licence can't be
honestly asserted in an envelope's header), so a deployer can audit a title or a
whole catalogue in code: encode only to royalty-free capabilities, or flag every
pool/proprietary one. The classification is exhaustive with no default class: a capability registered
without an explicit class is a registry defect the reference tests reject, so no
codec becomes royalty-free by omission. The same tests hold that every floor and
recommended-set capability is royalty-free.
codec becomes royalty-free by omission. Where a capability's royalty status is
not established, it is classified conservatively (patent-pool), never
royalty-free; a candidate's class is provisional until a revision pins its
payload format. The same tests hold that every floor and recommended-set
capability is royalty-free.
See [`research/media-codec-registry.md`](research/media-codec-registry.md) for
the per-capability standard / spec number / verified royalty status / reference
@ -562,8 +657,9 @@ A manifest MAY carry a `security` object:
- `security.encryption` — encryption scheme id. Absent, `null`, or
`"none"` ⇒ the payload is plaintext (the default; §1–§5 unchanged).
`"mls-sframe"` ⇒ the payload is an SFrame (RFC 9605) ciphertext object
whose group key is agreed by MLS (RFC 9420) — the
`draft-jennings-moq-e2ee-mls` binding.
whose group key is agreed by MLS (RFC 9420), following the construction
of `draft-jennings-moq-e2ee-mls`, an individual Internet-Draft that has
since expired ([confidentiality](extensions/confidentiality.md) §1).
- `security.kid` — the SFrame key id selecting which sender/epoch key
opens the object. It is also carried in the SFrame header; the
manifest copy lets a multiplexer route without parsing the payload.
@ -616,11 +712,12 @@ cleanly, rather than mis-decode ciphertext.
Byte-level SFrame framing, the SFrame key schedule, and the MLS binding
(including per-sender keying for multi-sender channels) are specified in
[`extensions/confidentiality.md`](extensions/confidentiality.md). The
reference implementation is the `moq-sframe` (SFrame) and
`joule-comm-mls` (MLS exporter → base key) crates, with browser bindings
in `joule-comm-mls-wasm`; `Track::e2e_encrypted` is the switch that
selects this path. Live: https://wai.transaction.science/private-channel
[`extensions/confidentiality.md`](extensions/confidentiality.md). Its
implementation is a companion implementation, outside this tree and under
its own licence: the `moq-sframe` (SFrame) and `joule-comm-mls` (MLS
exporter → base key) crates, with browser bindings in
`joule-comm-mls-wasm`; `Track::e2e_encrypted` is the switch that selects
this path. Live: https://wai.transaction.science/private-channel
---
@ -668,7 +765,7 @@ the standard codecs whose named standard fixes less than the bytes
For a **neural** capability `replicate` promises decode-equivalence, and this
revision does not substitute a standard's decoder conformance for it. The one
registered neural capability with a named external standard is
`wai.image.jpegai` (ISO/IEC 6048). That standard's conformance part may set a
`wai.image.jpegai` (ISO/IEC 6048-1:2025). That standard's conformance part may set a
criterion on the reconstruction — an overview of the standard says the part
"considers the possibility that a decoder is conformant to the standard without
having bit-exact reconstruction" — but this revision has not read it, so holding
@ -813,7 +910,7 @@ cited are the ones read; the evidence, and what was not read, is in
| class | capabilities | what the named standard fixes |
|---|---|---|
| `StandardDefined` | `wai.image.png`, `wai.audio.flac`, `wai.text.zstd`, `wai.text.xz` | the decoded output exactly: lossless formats (PNG, RFC 2083 — its filters are exact integer operations; FLAC, RFC 9639) and lossless byte formats |
| `StandardDefined` | `wai.image.png`, `wai.audio.flac`, `wai.text.zstd`, `wai.text.xz` | the decoded output exactly: lossless formats (PNG, W3C Recommendation, Third Edition — its filters are exact integer operations; FLAC, RFC 9639) and lossless byte formats |
| | `wai.video.avc`, `wai.video.hevc`, `wai.video.vvc` | every sample of every cropped decoded picture a conforming decoder outputs equals the value the specified decoding process produces (ITU-T H.264 (V16, 06/2026) §C.4; H.265 (V11, 01/2026) §C.5.1; H.266 (V4, 01/2026) §C.5.1); processing an SEI message only signals, such as film grain synthesis, is outside that decoding process |
| | `wai.video.vp9` | output frames "identical in all respects", in the same order, to those of the specified decoding process (VP9 Bitstream & Decoding Process Specification v0.6 §8.1) |
| `StandardTolerance` | `wai.audio.opus` | output within `opus_compare`'s thresholds of the reference decoder, with the same final range-decoder state (RFC 6716 §6) |
@ -824,14 +921,14 @@ cited are the ones read; the evidence, and what was not read, is in
| | `wai.video.av2` | as AV1: identical output unless film grain is signalled, then one of the same two options (AV2 v1.0.0 §7.1) |
| | `wai.audio.ivas` | output bit-exact to the test sequences, or, for an implementation based on the floating-point reference code whose output is not, a non-bit-exact procedure: TS 26.444's for mono operation, a per-frame loudness-difference corridor for stereo and immersive operation (3GPP TS 26.252 V19.2.0 §7.1, §7.2.1) |
| | `wai.audio.iamf` | its substreams include Opus (above), and renderers are out of its scope |
| `StandardUnclassified` | `wai.video.evc`, `wai.video.evc.baseline`, `wai.video.avs3`, `wai.video.lcevc`, `wai.audio.mpegh`, `wai.audio.avs3`, `wai.audio.ac4`, `wai.mesh.draco`, `wai.pc.gpcc`, `wai.pc.vpcc`, `wai.splat.sog`, `wai.haptic.mihs`, `wai.meta.hdr10plus`, `wai.meta.dovi`, `wai.prior.nnc`, `wai.tensor.nnc` | not checked in this revision (all sink-supplied; `wai.splat.sog` names no frozen standard yet) |
| `StandardUnclassified` | `wai.video.evc`, `wai.video.evc.baseline`, `wai.video.avs3`, `wai.video.lcevc`, `wai.audio.mpegh`, `wai.audio.avs3`, `wai.audio.ac4`, `wai.mesh.draco`, `wai.pc.gpcc`, `wai.pc.vpcc`, `wai.splat.sog`, `wai.haptic.mihs`, `wai.meta.hdr10plus`, `wai.meta.dovi`, `wai.prior.nnc`, `wai.tensor.nnc` | not checked in this revision (all sink-supplied; `wai.splat.sog` and `wai.tensor.nnc` are candidates, with no payload format pinned — §5) |
| | `wai.audio.aac` | not checked in this revision; three decoders on one host gave three different PCM streams from one ADTS stream, so it is not placed where byte identity would be promised |
| | `wai.mesh.glb`, `wai.splat.spz` | the payload bytes pass through intact; what a sink decodes from them is not classified here (a glTF binary may carry JPEG images in its buffer and compressed geometry through extensions, and glTF 2.0 specifies an asset format, not a decoder conformance over what is rendered; SPZ stores quantised attributes) |
A conversion a sink applies after the standard's decode is not part of the
promise. A standard fixes its decoded samples at their own depth, layout and
colour representation, and none of those above fixes a conversion to another: PNG
sample-depth reduction (RFC 2083 §10.4 gives two methods), YUV→RGB for AV1 and
sample-depth reduction (the PNG specification gives more than one method), YUV→RGB for AV1 and
AVIF, JPEG's colour space (outside T.81's scope, §4.1), rendering an IAMF mix to a
loudspeaker layout. The reference sink returns `RGB8` images and f32 PCM, so its
output buffers include such conversions; they are the reference implementation's,
@ -969,7 +1066,8 @@ neither tier.
### Machine-checked totality (informative)
The cross-architecture matrix establishes byte-exactness by **sampling**: it
decodes fixed vectors on five real OS/arch combinations. `wai-verify/` supplies
decodes fixed vectors on every OS/arch leg of the byte-equality matrix
(`.github/workflows/wai-byte-equality.yml`). `wai-verify/` supplies
the complementary half by **reasoning** — a model checker (Kani/CBMC) over the
same `#[path]`-included sources, for every input in a bounded window.
@ -1103,7 +1201,9 @@ Selection: a sink picks a rendition among the
`role != "companion"` entries, then applies every companion whose
`companion_of` names the chosen component (e.g. the HDR10+ block for the AV1
video the sink picked). `pick`/`primary` never return a companion, nor a
capability §5 registers as companion-only. Static HDR
component whose capability is not dispatchable (§4): a companion-only
capability, a candidate, a record or a derivation. A component whose capability
this revision does not register may be picked when the sink advertises it. Static HDR
characteristics (ST 2086 mastering display, HLG transfer) stay in-band in the
video bitstream / a manifest color field and need no companion.
@ -1208,6 +1308,21 @@ envelope that uses any string a later revision registers.
- New capabilities (§5) MAY be added in any MINOR revision. Existing
capabilities MUST NOT be redefined — once a capability string is
registered for a codec, its payload format is fixed forever.
- **Editions.** A registration that names an edition (or version) of its
standard carries that edition. A registration that names none carries the
edition of its named standard current on the date it was registered, and no
later edition or amendment. A later edition or amendment that changes the
bitstream a decoder must accept is registered, if at all, under a capability
string of its own; it never widens an existing string. The machine-readable
registry records both (`edition`, `registered`;
[`registry/capabilities.json`](registry/capabilities.json)). Where which
edition was current on the registration date has not been established, the
registration says so (§5), and what it admits beyond the tools of the
standard's first edition is unspecified until a revision records the edition. Citing a later
edition of the same specification as the reference for a format, where the
bitstream a registration admits does not change, is not a redefinition.
- A MINOR revision MAY add a value of an informational manifest field (the
`media` classes of §3); a reader accepts a value it does not know.
- A registration also fixes the capability's **role** (§5, §7.1): standalone
media, or companion-only — applied with a base rendition, never on its own.
Changing a capability's role, or narrowing the renditions a companion-only
@ -1271,19 +1386,21 @@ harnesses of §7 do not reach these kernels) — see
[`research/decode-throughput.md`](research/decode-throughput.md) for the
argument, its tests, its limits and measured throughput.
## Appendix B — Why a *capability menu*, not a custom codec
## Appendix B — Why a *capability menu*
Earlier drafts of WAI defined custom transform + entropy stages for
each medium. That direction was dropped: re-implementing JPEG / MPEG /
Opus poorly is strictly worse than calling the field's mature
libraries. WAI's value is the envelope + capability dispatch + the
neural-shared-prior model, all of which are absent from existing
standards. The zeroth condition's purpose is **availability**, not
codec novelty.
For a format a standard already specifies, WAI names the standard and the sink
decodes it with the library it already has: the zeroth condition's purpose is
**availability**, so a file opens wherever that library is present. Where WAI
needs a property no registered standard specifies — a reconstruction that is
byte-identical on every machine, a world or a film that replays to the same
state — it defines a WAI-native capability class with its own payload format
and its own conformance criterion (§5, §7). The envelope, the dispatch rules,
the registry's classes and the shared-prior model are what WAI itself
specifies.
## Appendix C — License
Apache-2.0. The standard and reference implementation are open;
royalty-free implementation is a precondition for adoption (avoiding
the H.264/H.265 royalty trap; following the VP9/AV1/AVIF/JPEG-XL open-
codec precedent).
Apache-2.0. The standard and the reference implementation are open;
royalty-free implementation is a precondition for adoption, so every default
capability (§5) is royalty-free. Companion implementations outside this tree
keep their own licences.

2
wai/claim-conformance/README.md
View file

@ -158,7 +158,7 @@ cargo build --no-default-features --features receipt_claims --bin wai_claim_vect
python3 differential.py <target-dir>/debug/wai_claim_vectors [--random N] [--seed S] [--keep FILE]
```
CI runs the first on the four-platform conformance matrix. The second and third are
CI runs the first on the conformance matrix (Linux x86_64, Linux arm64, macOS arm64). The second and third are
run by hand; they are not CI steps.
`differential.py` writes its inputs in the corpus's format with no declared outcome

4
wai/extensions/capability-constraints.md
View file

@ -80,5 +80,5 @@ author itself**, and until it does, those capabilities remain unconstrained unde
The grammar is thin — close to a JSON-Schema subset — and that is fine. The value
is not expressiveness; it is that a ratified, register-backed way of saying
"supported, to here" already exists, and WAI's dispatch is strictly worse for
having only "supported".
"supported, to here" already exists, and WAI's dispatch, which has only
"supported", gains a range by adopting it.

12
wai/extensions/confidentiality.md
View file

@ -1,10 +1,10 @@
# WAI Extension: Confidentiality (MLS + SFrame)
> Status: Draft. Sub-spec of WAI v1.0. Reference implementation: the
> Status: Draft. Sub-spec of WAI v1.0. Its implementation is a companion
> implementation, outside this tree and under its own licence: the
> `moq-sframe` (SFrame object encryption) and `joule-comm-mls` (MLS group
> key agreement) crates of the WAI reference tree, with browser bindings
> in `joule-comm-mls-wasm`. Live demonstration:
> https://wai.transaction.science/private-channel
> key agreement) crates, with browser bindings in `joule-comm-mls-wasm`.
> Live demonstration: https://wai.transaction.science/private-channel
Keywords **MUST**, **MUST NOT**, **SHOULD**, **MAY**, **REQUIRED**,
**OPTIONAL** are RFC 2119 / RFC 8174.
@ -20,7 +20,9 @@ them**. It is the `e2e_encrypted` path — the confidential counterpart to
the public floor, where a shared world or a broadcast film is plaintext
by design.
The construction is the `draft-jennings-moq-e2ee-mls` binding:
The construction is the `draft-jennings-moq-e2ee-mls` binding *(2026-09-30:
draft-jennings-moq-e2ee-mls is an individual Internet-Draft that has expired; the
MoQ working group's draft-ietf-moq-secure-objects is related work)*:
- **MLS** (RFC 9420) — the group agrees a shared secret out of band and
every member derives the same **SFrame base key** from it. Membership,

34
wai/extensions/determinism-tiers.md
View file

@ -473,8 +473,10 @@ bits. A capability's tier states which trade it made.
## 6. Evidence status
What the repository demonstrates, claim by claim, as of 2026-09-27 (main at
`afc7ece`). A test that skips reports as passing, and a matrix leg whose runner did
not run reports nothing; neither is evidence.
`afc7ece`), except where a row names a later run: the rows that cite run 745 read
the forge's runs of the landing chain's head `7cfcecfa` on 2026-09-30. A test that
skips reports as passing, and a matrix leg whose runner did not run reports
nothing; neither is evidence.
The CI rows below come from the forge's commit statuses
(`/api/v1/repos/Transaction-Science/open-standards/commits/<sha>/statuses`), read
@ -482,11 +484,12 @@ for every commit on main since 2026-08-10 and for the pre-merge commits named, n
from the workflow definitions. Two workflows run the same golden-byte suite
(`wai/byte-exact-conformance`, `cargo test --release`): the *cross-arch
byte-equality* job of `wai-conformance.yml`, and `wai-byte-equality.yml`. Both
declare Linux x86_64, Linux arm64, macOS arm64 and Windows x86_64 legs; there is no
macOS x86_64 or Windows arm64 leg. No Windows x86_64 job has run for this
repository since 2026-08-27: every later Windows status on main is "Waiting to run"
or "Has been cancelled", and the forge's task list has no Windows task after that
day.
declared Linux x86_64, Linux arm64, macOS arm64 and Windows x86_64 legs until
2026-09-30, when the Windows x86_64 legs were taken out of both workflows because its
runner is offline; there is no Windows leg now, and no macOS x86_64 or Windows arm64
leg. No Windows x86_64 job ran for this repository after 2026-08-27: every later
Windows status on main is "Waiting to run" or "Has been cancelled", and the forge's
task list has no Windows task after that day. The Windows results below are history.
| claim | evidence | where |
|---|---|---|
@ -495,13 +498,13 @@ day.
| `wai.image.jpegai` exact decoder: the integer derivation of the distribution indices matches the reference decoder exactly | a test on one machine that skips unless its fixtures (sink-supplied weights, normative tables, codestream) are provided; CI does not provide them | `wai-rs/tests/jpegai_hyperscale_calibration.rs` |
| `wai.image.jpegai` exact decoder, whole decode through the one public `decode()` call: it stays within a stated bound of the reference decoder's output (max\|Δ\| ≤ 2, ≤ 5% of samples); two decodes in one process are byte-identical, and a reconstruction binding taken over one verifies the other | `wai-jpegai-e2e.yml` fetches the pinned `jpegai/v1` inputs (an extraction of the reference-software assets, published under WAI's models root), checks each file against the SHA-256 manifest `tools/jpegai-e2e-v1.sha256`, and runs both tests with `WAI_REQUIRE_SOTA_VECTORS=1`, under which a missing input fails instead of skipping. The job also fails on any skip banner in the test log, and requires the decoded pixels to hash to a pinned SHA-256. One 256×256 codestream. The workflow has run on the forge's Linux runner (x86_64 hardware) and passed on every run since 2026-09-28. Before that, its job passed under `forgejo-runner exec` on the same host (2026-09-27) with the exact decoder as it stood on main at `afc7ece`, which calls the platform `exp`; there too the decoded pixels matched the pinned hash, the value measured on macOS arm64 | `wai-rs/tests/jpegai_e2e_full.rs`, `jpegai_reconstruction_binding.rs`; `.github/workflows/wai-jpegai-e2e.yml` |
| `wai.image.jpegai` exact decoder across platforms: the `jpegai/v1` codestream decodes to the same RGB8 bytes, and to the same bits in the float values the decoder reports after each stage (values inside a stage are not hashed), on each platform measured | measured 2026-09-28 with rustc 1.98.0, release builds, inputs checked against their SHA-256, all four on the source of the change that added this row: macOS arm64 (native, on the development machine); wasm32-wasip2 (wasmtime 49, on the same machine); Linux arm64 (glibc 2.36, a container in an arm64 Linux VM on that machine); Linux x86_64 (glibc 2.36, the x86_64 build run in that container under user-mode emulation, QEMU 7.0, not on x86_64 hardware). All nine pinned hashes matched on each platform. On all four they also matched on an earlier state of that change, before the decode's `exp` and rounding moved to `float_repro`. Before the move the decode called the platform `exp` on every target, and its roundings compiled to calls to the symbols `round`, `roundf` and `rint` on x86_64 and `round` and `roundf` on wasm32, which the link resolves: an x86_64 Linux test executable inspected after the move, whose other code still rounds this way, took them from Rust's compiler builtins, not the C library (for wasm32 this was not checked). Not measured: macOS x86_64 (no x86_64 translation layer is installed on the development machine), Windows, x86_64 hardware, 32-bit targets. The test skips unless the inputs are supplied. `wai-jpegai-e2e.yml` supplies them and requires this test to run and pass on its Linux runner; that workflow has run on the forge's Linux x86_64 runner and passed on every run since 2026-09-28 | `wai-rs/tests/jpegai_cross_platform.rs` |
| `float_repro`, the decode's `exp` and rounding: the same bits over a fixed grid of inputs; `exp` within 2 ULPs of the host math library (1 ULP measured); rounding equal to the standard library's | library unit tests under the library-test step of the conformance workflow (`neural_int`), which has run them on its macOS, Linux x86_64 and Linux arm64 runners since 2026-09-28. The pinned-bits and rounding tests never skip. The host comparison asserts the host library's accuracy as much as this module's. Its gate selects by target, not by host: it runs on macOS arm64, on Linux x86_64 and arm64 with glibc, and on WASI, and is reported as ignored, with the reason, elsewhere, Windows included. On the source of the change that added this row it was run against these hosts' `exp`, with a worst case of 1 ULP on each: Apple's math library on macOS arm64; the WASI C library under wasmtime; glibc 2.36 on Linux arm64, and glibc 2.35 in the Ubuntu 22.04 image the workflow's Linux arm64 runner used on 2026-09-28; glibc 2.36 on Linux x86_64 under user-mode emulation, against three of the four `exp` variants glibc chooses among by processor feature (the variant each run used follows from the features glibc's loader reported active): SSE2 (QEMU 7.0, whose emulated processor reports none of AVX, FMA, AVX2 and FMA4), AVX (QEMU 7.2, `-cpu max` with FMA and AVX2 masked by glibc's `hwcaps` tunable, which leaves AVX active) and FMA (QEMU 7.2, `-cpu max`). On a separate probe of 20 million inputs, the SSE2 and AVX variants returned the same bits, and the FMA variant different bits for some of them. Not compared: the FMA4 variant, which glibc chooses on a processor that has FMA4 but not both FMA and AVX2, and which neither emulator offers. Not run on x86_64 hardware or on other glibc versions, where the gate still runs it; a failure there would say that the host's `exp` and this one are more than 2 ULPs apart, not that `float_repro`'s bits changed | `wai-rs/src/float_repro.rs` tests |
| `float_repro`, the decode's `exp` and rounding: the same bits over a fixed grid of inputs; `exp` within 2 ULPs of the host math library (1 ULP measured); rounding equal to the standard library's | library unit tests under the library-test step of the conformance workflow (`neural_int`), which has run them on its macOS, Linux x86_64 and Linux arm64 runners since 2026-09-28. The pinned-bits and rounding tests never skip. The host comparison asserts the host library's accuracy as much as this module's. Its gate selects by target, not by host: it runs on macOS arm64, on Linux x86_64 and arm64 with glibc, and on WASI, and is reported as ignored, with the reason, elsewhere, Windows included. On the source of the change that added this row it was run against these hosts' `exp`, with a worst case of 1 ULP on each: the platform math library on macOS arm64; the WASI C library under wasmtime; glibc 2.36 on Linux arm64, and glibc 2.35 in the Ubuntu 22.04 image the workflow's Linux arm64 runner used on 2026-09-28; glibc 2.36 on Linux x86_64 under user-mode emulation, against three of the four `exp` variants glibc chooses among by processor feature (the variant each run used follows from the features glibc's loader reported active): SSE2 (QEMU 7.0, whose emulated processor reports none of AVX, FMA, AVX2 and FMA4), AVX (QEMU 7.2, `-cpu max` with FMA and AVX2 masked by glibc's `hwcaps` tunable, which leaves AVX active) and FMA (QEMU 7.2, `-cpu max`). On a separate probe of 20 million inputs, the SSE2 and AVX variants returned the same bits, and the FMA variant different bits for some of them. Not compared: the FMA4 variant, which glibc chooses on a processor that has FMA4 but not both FMA and AVX2, and which neither emulator offers. Not run on x86_64 hardware or on other glibc versions, where the gate still runs it; a failure there would say that the host's `exp` and this one are more than 2 ULPs apart, not that `float_repro`'s bits changed | `wai-rs/src/float_repro.rs` tests |
| The exact decode path calls no platform math library | two checks. Clippy's `disallowed_methods` (`wai-rs/clippy.toml`), denied in each module on the decode path, rejects any use of a listed float method, including `f64::exp(x)` and `.map(f64::exp)`. `tests/jpegai_decode_symbols.rs` lists the symbols of an executable that links only the decoder's entry points and weight loaders. It rejects any C math-library function the executable imports, and any it defines other than the round-to-integral functions (`floor`, `round`, `rint` and the like), whose results IEEE 754 fixes. It allows those definitions because a definition shows what the linker kept, not what the decode calls: linked with GNU ld 2.40 for x86_64 Linux, the debug executable defined the compiler builtins' `floor`, though nothing in it referred to `floor`. On the source of the change that added this row, both checks pass on macOS arm64, the symbol check in debug and release builds. The symbol check also passes, in debug builds, on Linux arm64 (the container above) and on Linux x86_64 under user-mode emulation, linked with GNU ld (reporting that `floor` as allowed) and with LLD. With a platform `exp` put back as `f64::exp(…)`, both fail on macOS arm64 while the decoder's unit tests still pass, and the symbol check fails on Linux x86_64 with GNU ld. With a float `%` put into the decode, the symbol check fails on macOS arm64 and clippy, which does not see operators, passes. The conformance workflow runs both on its unix legs, on the forge's runners since 2026-09-28 | `wai-rs/clippy.toml`, `wai-rs/tests/jpegai_decode_symbols.rs`, `.github/workflows/wai-conformance.yml` |
| `wai.image.jpegai` scope: the reference codestream is accepted; copies with β, `model_id`, the colour transform or an LSBS flag changed are refused; a flush-to-zero or round-toward-zero environment is detected. The float ONNX path calls the same check | library unit tests over a committed copy of the reference codestream, which never skip and fall under the library-test step of the conformance workflow (`neural_int`); the environment test runs on arm64 only. Run on the four platforms above, on the source of the change that added the cross-platform row (the environment test on macOS arm64 and Linux arm64). The float path's refusal (β and `model_id` edits) is tested only with its ONNX bundle supplied (`jpegai_scope_ort.rs`, skips otherwise) | `wai-rs/src/jpegai_decode.rs` tests, `wai-rs/tests/vectors/jpegai/` |
| SPEC §7's replicate rule: the registry class decides first, so an integer-declared prior over `wai.image.jpegai` or a float codec is refused; `wai.image.jpegai` is the only `NeuralEntropyExact` capability; the standard codecs sit in the classes SPEC §7 lists, and the tolerance and unclassified ones are not refused; an encoder that pins no prior (`wai wrap`) writes `replicate` only over the standard classes, for the capability and for any declared fallback, or, in the `WAI2` companion form, for the base payload's capability and each companion's | library unit tests (`codecs`, `container`) that never skip and fall under the same library-test step. The strict-decode test in `tests/neural_native_decode.rs` needs the `neural` feature; `wai-neural-gates.yml` runs it on the Linux runner, in an Ubuntu 24.04 image because the `neural` test binary does not link in that runner's default 22.04 image. The workflow has run on the forge and passed on every run since 2026-09-28; before that its job passed under `forgejo-runner exec` on the Linux runner's host (Linux x86_64, 2026-09-27). The test that runs the `wai wrap` binary needs the `codecs` feature, which no CI job builds. Otherwise run so far on macOS arm64 only | `wai-rs/src/codecs/mod.rs`, `container.rs` tests; `wai-rs/tests/neural_native_decode.rs`, `cli_wrap_intent.rs` |
| SPEC §7's replicate rule: the registry class decides first, so an integer-declared prior over `wai.image.jpegai` or a float codec is refused; `wai.image.jpegai` is the only `NeuralEntropyExact` capability; the standard codecs sit in the classes SPEC §7 lists, and the tolerance and unclassified ones are not refused; an encoder that pins no prior (`wai wrap`) writes `replicate` only over the standard classes, for the capability and for any declared fallback, or, in the `WAI2` companion form, for the base payload's capability and each companion's | library unit tests (`codecs`, `container`) that never skip and fall under the same library-test step. The strict-decode test in `tests/neural_native_decode.rs` needs the `neural` feature; `wai-neural-gates.yml` runs it on the Linux runner, in an Ubuntu 24.04 image because the `neural` test binary does not link in that runner's default 22.04 image. The workflow has run on the forge and passed on every run since 2026-09-28; before that its job passed under `forgejo-runner exec` on the Linux runner's host (Linux x86_64, 2026-09-27). The `wai` binary no longer needs the `codecs` feature, so the tests that run `wai wrap` (`cli_wrap_intent.rs`, and `cli_registry.rs` for the candidate refusal and the registry export) run in the conformance workflow's carriage step on every leg; this row records no forge run of them yet. The C envelope packers emit through `Wai::to_emit_bytes` and `WaiMulti::to_emit_bytes`, whose candidate refusal a library test checks with no feature (`the_emit_path_refuses_a_candidate`, in the library-test step); the C wrappers themselves are compiled only with the `codecs` feature, which no CI job builds. The browser writers' refusal runs in `wai-web-wasm.yml` (`verify_pack.mjs`) | `wai-rs/src/codecs/mod.rs`, `container.rs` tests; `wai-rs/tests/neural_native_decode.rs`, `cli_wrap_intent.rs`, `cli_registry.rs`; `wai-web/demo/verify_pack.mjs` |
| `wai.neural.int_hyper` and `wai.video.int_hyper` at full model scale (the pretrained `mbt2018_mean`, exported to integers): the Rust decode reproduces the export tool's integer reference byte for byte (synthesis, hyper-decoder, the whole conditional bitstream, an eight-frame clip), and costs at most 1 dB against the same model's float decode on one pinned photograph | `wai-sota-vectors.yml` regenerates the vectors on the runner from pinned inputs (the photograph's pixels and the checkpoint's bytes by SHA-256, the Python stack by version) and runs both tests with `WAI_REQUIRE_SOTA_VECTORS=1`, under which a missing vector fails instead of skipping. The job also fails on any skip banner in the test log, and runs `require_vectors_rule.rs`, which checks that without the switch a variable naming absent or incomplete inputs fails as well. Rust is checked against the export of the same run on the same machine: the vectors come from float computation, and some of them differed between machines and between runs, so none of their hashes is pinned and none is compared across machines. The workflow has run on the forge's Linux runner (x86_64 hardware) and passed on every run since 2026-09-28 | `wai-rs/tests/int_hyper_synth.rs`, `int_video_sota_keyframe.rs`, `require_vectors_rule.rs`; `.github/workflows/wai-sota-vectors.yml` |
| `wai.audio.int_codec`, `wai.splat.int_codec` | committed vectors that never skip, matched against an independent Python integer reference; the conformance workflow's fast-kernel steps enable their cargo features (`audio_int`, `splat_int`) and decode both vectors against their frozen hashes under every kernel policy (`tests/int_kernels_vectors.rs`); the comparisons with the Python reference in `int_audio.rs` and `int_splat.rs` run in no CI job | `wai-rs/tests/int_audio.rs`, `int_splat.rs` |
| `wai.neural.int_mlicpp` on its committed vector (synthetic seeded weights; a 64×64 image, 8 latent channels in 4 slices): the decode reproduces the export tool's independent integer reference byte for byte — the RGB, and every scale bucket and symbol of the latent stream in coding order (512 symbols over all 16 buckets of the scale table, 47 of the symbols coded by the rANS escape) — under the reference kernels and under the fast kernels on 1, 2, 3, 5 and all available threads; one integer unit changed in each of four chosen parameters, one on each path that feeds a value (a hyper-decoder weight, an entropy-network weight, a mean-head bias, a synthesis weight), changes the image; a header whose shapes the model cannot produce, or on which a network would form a buffer over the cap, is refused before anything is allocated (`tests/int_mlicpp_preflight.rs`, which counts allocations); damaged payloads are refused or decode, and never panic in an overflow-checked build | measured 2026-09-29 with rustc 1.98.0 on the source of the change that added this row: macOS arm64 native (the wai-rs tests, with overflow checks on and off; the `byte-exact-conformance` golden, release) and wasm32-wasip2 under wasmtime 49.0.1 (the same tests and golden, release; on wasm the fast kernels run single-threaded as portable scalar code, and `usize` is 32-bit). All pinned hashes matched on both. Not measured: x86_64 — the development machine has no x86_64 translation layer installed, and an x86_64-apple-darwin build of the golden compiles but cannot execute there — nor Linux or Windows. The golden runs on every leg of the byte-exact matrix (`wai-conformance.yml`, `wai-byte-equality.yml`) and `tests/int_mlicpp.rs` on every leg of `wai-conformance.yml` in both profiles; neither has run through the forge yet. The dispatch through `decode_envelope_strict` (the prior pinning the one-file model bundle, the strict gate) runs in `wai-neural-gates.yml`. The weights are synthetic, so nothing here measures compression quality | `wai-rs/tests/int_mlicpp.rs`, `wai-rs/tests/int_mlicpp_preflight.rs`, `byte-exact-conformance/src/int_mlicpp_golden.rs`, `wai-rs/tests/neural_native_decode.rs`; `tools/wai_int_mlicpp_export.py` |
| `wai.audio.int_codec`, `wai.splat.int_codec` | committed vectors that never skip. Their frozen hashes are the output of an independent Python integer reference, computed when the vectors were exported (`tools/wai_int_audio_export.py`, `tools/wai_int_splat_export.py`); no Python runs in CI. The conformance workflow's fast-kernel steps enable `audio_int` and `splat_int` and, in debug and release builds, decode both vectors against those hashes under the reference kernels and every fast-kernel policy (`tests/int_kernels_vectors.rs`); that passed on the forge's Linux x86_64, Linux arm64 and macOS arm64 runners for the landing chain's head `7cfcecfa` (run 745, 2026-09-30). The same steps also name `tests/int_audio.rs` and `tests/int_splat.rs`, which check each vector against the same hash and against its own metadata (sample count; attribute count, tensor size and grid side); Cargo declares each with its feature as a required feature, so a step that names it without that feature fails instead of running nothing. This row records no forge run of those two | `wai-rs/tests/int_audio.rs`, `int_splat.rs`, `int_kernels_vectors.rs`; `.github/workflows/wai-conformance.yml` |
| `wai.neural.int_mlicpp` on its committed vector (synthetic seeded weights; a 64×64 image, 8 latent channels in 4 slices): the decode reproduces the export tool's independent integer reference byte for byte — the RGB, and every scale bucket and symbol of the latent stream in coding order (512 symbols over all 16 buckets of the scale table, 47 of the symbols coded by the rANS escape) — under the reference kernels and under the fast kernels on 1, 2, 3, 5 and all available threads; one integer unit changed in each of four chosen parameters, one on each path that feeds a value (a hyper-decoder weight, an entropy-network weight, a mean-head bias, a synthesis weight), changes the image; a header whose shapes the model cannot produce, or on which a network would form a buffer over the cap, is refused before anything is allocated (`tests/int_mlicpp_preflight.rs`, which counts allocations); damaged payloads are refused or decode, and never panic in an overflow-checked build | measured 2026-09-29 with rustc 1.98.0 on the source of the change that added this row: macOS arm64 native (the wai-rs tests, with overflow checks on and off; the `byte-exact-conformance` golden, release) and wasm32-wasip2 under wasmtime 49.0.1 (the same tests and golden, release; on wasm the fast kernels run single-threaded as portable scalar code, and `usize` is 32-bit). All pinned hashes matched on both. Not measured: x86_64 — the development machine has no x86_64 translation layer installed, and an x86_64-apple-darwin build of the golden compiles but cannot execute there — nor Linux or Windows. The golden runs on every leg of the byte-exact matrix (`wai-conformance.yml`, `wai-byte-equality.yml`) and `tests/int_mlicpp.rs` on every leg of `wai-conformance.yml` in both profiles. Both passed on the forge's Linux x86_64, Linux arm64 and macOS arm64 runners for the landing chain's head `7cfcecfa` (run 745, 2026-09-30); there is no Windows leg. The dispatch through `decode_envelope_strict` (the prior pinning the one-file model bundle, the strict gate) runs in `wai-neural-gates.yml`. The weights are synthetic, so nothing here measures compression quality | `wai-rs/tests/int_mlicpp.rs`, `wai-rs/tests/int_mlicpp_preflight.rs`, `byte-exact-conformance/src/int_mlicpp_golden.rs`, `wai-rs/tests/neural_native_decode.rs`; `tools/wai_int_mlicpp_export.py` |
So at `entropy-consistency`, WAI's JPEG AI symbol decode is demonstrated across
architectures (x86_64 against arm64) once the distribution indices are given; the
@ -510,13 +513,16 @@ machine, by a test CI does not run. At `decode-equivalence`, cross-architecture
identity (x86_64 against arm64) is demonstrated for `int_synth`, integer video and
`int_hyper` on committed small-scale vectors; of these, only `int_synth` has also
been decoded on Windows on main. `wai.audio.int_codec` and `wai.splat.int_codec`
reproduce a separate implementation's output, but no CI job runs them and no
cross-architecture comparison exists. For full-scale `int_hyper`, identity rests on
decode their committed vectors to the hashes a separate implementation computed, on
Linux x86_64, Linux arm64 and macOS arm64 (run 745): cross-architecture identity
(x86_64 against arm64) against one pinned hash per vector, on small vectors; neither
has been decoded on Windows. For full-scale `int_hyper`, identity rests on
construction, not on a cross-machine comparison of decoded output: CI checks its
decode against the export's integer reference on one machine per run.
`wai.neural.int_mlicpp`'s committed vector decodes to the same pinned bytes on macOS
arm64 and on wasm32-wasip2 (a 32-bit target), measured on the development machine;
its x86_64 and Linux legs await the byte-exact matrix.
it also passed on the byte-exact matrix's Linux x86_64, Linux arm64 and macOS arm64
legs in run 745.
For the JPEG AI exact decoder's pinned reconstruction (§5), identity across
machines rests on construction, and on one comparison: a single codestream on the

4
wai/extensions/energy-binding.md
View file

@ -128,8 +128,8 @@ measurement.
## 5. Relationship to the standard assertion
Interoperability is the reason both are emitted. A validator that understands
only C2PA 2.4 reads `energy_kwh` and a populated `measurementMethod`, which is
already better than the common case. A validator that understands this
only C2PA 2.4 reads `energy_kwh` and a populated `measurementMethod`. A
validator that understands this
extension additionally learns how the figure was acquired, how much of the work
it covers, and which reconstruction it describes — and can check the last of
those itself.

13
wai/extensions/interactive-worlds.md
View file

@ -191,8 +191,8 @@ keeps conformance state-level while assets stream in.
For geometry, materials, textures, and skeletal armatures the
registered asset capability is **`wai.mesh.glb`**: the payload is a
canonical glTF 2.0 Binary (`.glb`, Khronos, royalty-free) — the same
zeroth-condition philosophy as the rest of the core registry: dispatch
to the field's SOTA standard, don't invent. Decoders are sink- or
zeroth-condition approach as the rest of the core registry: dispatch to
a published standard. Decoders are sink- or
library-supplied (every engine and DCC tool ships one). Further asset
media register in the core capability table as usual.
@ -500,7 +500,7 @@ as `SessionReceipt::parent_receipt_hash` + `receipt_hash()` +
function of the world and its ops alone.
- `joules_micro` — a **carried, measured** figure, never derived inside
the receipt. Native sinks supply real energy via the sink-metering
path, which needs **no elevated privileges**: on Apple Silicon the SoC
path, which needs **no elevated privileges**: on arm64 macOS the SoC
energy counters are readable through IOReport (the source `macmon` /
`asitop` use — no sudo), on x86 via RAPL, and per-process retired
cycles/instructions via `proc_pid_rusage` everywhere. The reference
@ -564,10 +564,9 @@ floor.
**Why the standard has no physics.** The moment the extension
specifies a solver it stops being a container standard and starts
being an engine, and every engine vendor becomes a competitor instead
of an implementer. The contract — deterministic, pure, convergent,
receipted — is the part no engine ships today and the only part that
must be common.
being an engine, which an engine would have to replace rather than
implement. The contract — deterministic, pure, convergent, receipted — is
the only part that must be common.
**Why state-level, not pixel-level, conformance.** Pixels are a
function of state plus presentation assets plus renderer. State is the

26
wai/extensions/jwp-receipts.md
View file

@ -1,7 +1,8 @@
# WAI Extension: JWP Receipt Chain
> Status: Draft. Sub-spec of WAI v1.0. Mirrors the Rust reference impl at
> https://github.com/Transaction-Science/open-standards/tree/main/wai/extensions/jwp-receipts.
> Status: Draft. Sub-spec of WAI v1.0. The verifier of the reference
> implementation is in
> https://git.transaction.science/Transaction-Science/open-standards/src/branch/main/wai/wai-rs.
Keywords **MUST**, **MUST NOT**, **SHOULD**, **MAY**, **REQUIRED**,
**OPTIONAL** are RFC 2119 / RFC 8174.
@ -1021,10 +1022,13 @@ chain across hops, a conforming relay:
## 5. Test vectors
Conforming implementations MUST verify against the canonical test
vector set generated by
`joulesperbit/crates/joule-moq-publisher/examples/jwp_receipts_vectors.rs`.
The generator emits five vectors covering the cases below. Each
A conforming verifier MUST give, for each case §5.1–§5.5 defines, the outcome
that case states: accept, or reject at the stated step with the stated reason.
The cases are normative as written here. Vectors that instantiate them are
generated outside this tree, by a companion implementation, and are not yet
published in this repository; once a revision publishes them in the tree, a
conforming verifier MUST also verify against them. (The claim-receipt corpus of
§5.6 is published in this tree.) Each
vector consists of: the WAI envelope bytes for every object in the
group, the canonical group-receipt JSON, the canonical object-receipt
JSON for every object, and a `result.json` declaring the expected
@ -1067,13 +1071,11 @@ Two group receipts G0, G1. G1 omits the `parent_receipt_hash` field
entirely (rather than setting it to `null` or a hash). Expected
outcome: reject at §3 step 1 with reason `missing_required_field`.
The generator writes vectors to
`joulesperbit/crates/joule-moq-publisher/examples/jwp_receipts_vectors/`
as one directory per case. A conforming verifier MUST accept the
The generator writes the vectors as one directory per case. A conforming
verifier MUST accept the
`accept` cases bit-exactly and MUST reject the named cases at the
declared step with the declared reason. The Rust reference verifier
in `inv-moq::receipt::ReceiptVerifier` exercises all five vectors
under `cargo test --lib -p inv-moq`.
declared step with the declared reason. The companion implementation's verifier
exercises all five vectors in its own test suite.
### 5.6. Claim receipts and recovery (`claim-conformance/`)

9
wai/extensions/linear-film.md
View file

@ -22,9 +22,12 @@ of three parts:
The sink renders the frames. The wire carries the *instructions* that
generate the film (the world + the camera), not the rendered result —
lever 3 (instructions-at-the-sink) applied to the largest
captured-as-pixels category there is. A two-hour animated feature is the
world's op log plus a camera spline; the pixels are reconstituted at the
sink's native resolution and frame rate.
captured-as-pixels category there is. A two-hour feature generated from a
world is the world's op log plus a camera spline, together with the assets the
world references (meshes, splats, textures, audio), which travel as their own
content-addressed capabilities and can be large; the pixels are reconstituted at
the sink's native resolution and frame rate. A film captured from a camera has no
world to replay and is not this capability.
In scope: the container, the camera-track model, the frame schedule, the
deterministic **film-equivalence** conformance criterion, and the receipt

4
wai/extensions/prior-carriage.md
View file

@ -108,7 +108,9 @@ Registry classification (`codecs::capability_*`):
**`wai.tensor.nnc` is a candidate.** Coding of general tensorial data is part of
an NNC amendment that has not been published (§4), so this payload's format is
not fixed. Until the capability is pinned to a published edition, an encoder
MUST NOT emit it and a sink MUST treat it as unsupported. SPEC §8's rule that a
MUST NOT emit it and a sink MUST treat it as unsupported. SPEC §5
(*Candidate capabilities*) states the rule for every candidate, and
`codecs::is_candidate` lists them. SPEC §8's rule that a
registered payload format never changes applies from the edition it is pinned to.
It is registered now because split-inference feature maps and scene tensors are a
payload class WAI otherwise has no name for. For Gaussian-splat attributes

58
wai/extensions/quantum-ops.md
View file

@ -4,23 +4,20 @@ Status: **Draft.** Engine: `wai-rs/src/quantum_ops.rs` (feature `quantum_ops`).
The **verify-half** of WAI (`wai.asset.provenance`'s lineage, specialized) applied
to the quantum processing industry.
## 0. The gap this closes
## 0. What this extension adds
Every quantum-control stack in the field — Q-CTRL, QuantrolOx, Qruise, Quantum
Machines, Qblox, Zurich Instruments, OrangeQS, QuEL, Origin, Conductor, Classiq —
emits the same three **unsigned** things: an HDF5 dataset, a JSON state snapshot,
and a dashboard/log. Not one **signs**, **content-addresses**, **ports**, or
**meters the joules** of a single operation. The industry already reaches for
reproducibility (QM's `__package_versions__`, QBOM bills-of-materials, QCIVET
hash-chained audit traces, "Vibe Calibration" internal audit records) but stops
exactly short of two axes:
A calibration run or a QPU job leaves a dataset (commonly HDF5), a state snapshot
and a log. This extension adds a receipt over each operation with two properties:
1. **cryptographic signature + portability** — a receipt a *third party* verifies,
not a mutable vendor-cloud dashboard;
2. **measured joules** — every stack sits on a kW-scale dilution fridge + control
racks, yet none emits joules-per-calibration or joules-per-job.
1. **cryptographic signature + portability** — a record a *third party* verifies
from the bytes alone;
2. **measured joules** — the energy of the operation, carried with its
acquisition class, on hardware whose dilution refrigerator and control racks
draw kilowatts.
This extension is those two axes. It is not a control system and **runs no
Related reproducibility work exists (version capture of the software that ran, QBOM
bills of materials, hash-chained audit traces such as QCIVET's); the receipt here
is signed, content-addressed and joule-metered. It is not a control system and **runs no
hardware**: the calibration/job is performed by the sink's control stack, exactly
as WAI dispatches a decode to a sink's codec. WAI owns the **receipt**.
@ -43,8 +40,8 @@ A signed record of one bring-up / tune-up run:
([energy-measurement](energy-measurement.md) §2.8). An unlabelled receipt
keeps its legacy bytes.
- `grant` — the authorization (below)
- `parent_receipt_hash` — the prior device state this supersedes: the signed
"digital twin" lineage vendors keep only as mutable, unsigned data
- `parent_receipt_hash` — the prior device state this supersedes: a signed
lineage of device states
`merkle_root` covers `[config_hash] ++ evidence leaves`; one Ed25519 signature
covers the whole payload.
@ -53,7 +50,7 @@ covers the whole payload.
A signed record of one dispatched circuit:
- `backend_id` — which QPU/sim actually ran it (`"ionq:aria-1"`, `"wai.quantum.circuit"`)
- `backend_id` — which QPU/sim actually ran it (`"example:qpu-a"`, `"wai.quantum.circuit"`)
- `circuit_hash`, `result_hash` (counts histogram / statevector), `shots`
- **`calibration_ref`** — the `CalibrationReceipt::receipt_hash` of the device
state the job ran under. `verify_against_calibration(cal)` confirms the job
@ -103,23 +100,16 @@ The natural attach point is the **control-system → cloud boundary**: the momen
calibration node or a job completes, the control system already holds everything a
receipt needs (config, evidence datasets, result, timing, backend identity), and
the power rails are already instrumented — so adding a signature + a joule figure
is *incremental*. Ranked open-source insertion points:
is *incremental*. Open-source insertion points:
1. **Quantify** (`quantify-core`) — one HDF5-write chokepoint, co-owned by Qblox +
OrangeQS: one adapter covers two vendors (Phase 1 target).
2. **OrangeQS Juice** — open lab-OS whose Task-Management + DB layer is a natural
control plane; production chip-testing, where a signed pass/fail has direct
commercial value.
3. **QUAlibrate/QUAM** — largest install base; the per-node boundary + the existing
`__package_versions__` hook upgrade cleanly to a signed, metered receipt.
4. **Qubex/qube-calib** (QuEL) — fully open, clean `CalibrationNote` artifact.
1. **Quantify** (`quantify-core`) — one HDF5-write point covers every node.
2. A lab-OS task-management and database layer, as a control plane; in
production chip testing a signed pass/fail is the record a test produces.
3. **QUAlibrate/QUAM** — the per-node boundary, and its existing record of
package versions, map onto a signed, metered receipt.
4. **Qubex/qube-calib** — a `CalibrationNote` artifact per node.
Adoptability is openness-first: Japan (QuEL/OQTOPUS), India (NQM), Korea (SDT/
NVQLink, Norma/PQC), and the Western merchant vendors (receipt as an anti-lock-in
feature). Sovereign closed champions are parallel, not addressable.
## 6. The metric
## 6. The unclaimed metric
"Joules per verified calibration" and "joules per QPU job" are unmeasured and
unclaimed across the entire field. This extension is the object that carries them —
signed, portable, content-addressed, and grant-bounded.
This extension carries joules per verified calibration and joules per QPU job as
signed, portable, content-addressed, grant-bounded records.

2
wai/extensions/quantum-sim.md
View file

@ -219,7 +219,7 @@ Two verification depths:
`to_json()` / `from_json()` give the portable, sorted-key JSON an out-of-band
verifier reads. `wai_quantum_meter <circuit.wqc> [receipt.json]` fills the
energy honestly: on Apple Silicon it samples CPU power via `macmon` (IOReport, no
energy honestly: on arm64 macOS it samples CPU power via `macmon` (IOReport, no
sudo), brackets a reconstruction burst against an idle baseline, and writes the
**measured** marginal microjoules — never a fabricated number; where no meter is
present it emits an UNMETERED receipt (`joules_micro = 0`) with the exact work

7
wai/extensions/spatial-audio.md
View file

@ -160,11 +160,10 @@ impl):
- `parent_receipt_hash` chains successive renders into an append-only
signed timeline, exactly as worlds' receipts do.
## Appendix — why this is the cleanest first cargo class
## Appendix — why this is the first cargo class
Mixdown-equivalence is replay-equivalence with the tick replaced by the
audio sample; the `Fx` floor, the content-addressed assets, the seal,
the receipt profile, and the fallback floor are all reused verbatim. A
spatial-audio scene is, structurally, a world whose "sim" is a mixer —
which is exactly why it proves the interactive-worlds thesis
generalizes to a second sense.
spatial-audio scene is, structurally, a world whose "sim" is a mixer,
so the interactive-worlds rules carry over to a second sense.

14
wai/extensions/ternary-tt.md
View file

@ -7,10 +7,10 @@ deterministic transform.
## 0. What this is — and what it is not (read first)
`wai.ternary.tt` is a **provenance** capability, not a rate-competitive codec.
It exists to demonstrate — and make usable — one property nothing else in the
learned-compression field offers: a quantum-inspired ternary/tensor-network
transform whose reconstruction is **byte-exact, portable, and receiptable**.
`wai.ternary.tt` is a **provenance** capability; rate reduction is not its purpose.
It exists to demonstrate, and make usable, one property: a quantum-inspired
ternary/tensor-network transform whose reconstruction is **byte-exact, portable,
and receiptable**.
It is **not** a good general media codec, and this document says so up front with
numbers (§5). If you need compression ratio, use the zeroth menu (JPEG-XL, AVIF,
@ -78,7 +78,7 @@ here over the integer decode. The registry classes it accordingly:
StandardDefined` (the reconstruction is fixed by WAI's deterministic floor, not a
float neural decode), `license = RoyaltyFree`.
## 5. Rate–distortion verdict (measured, honest)
## 5. Rate–distortion verdict (measured)
Measured on a realistic 256-sample multi-tone + noise stream (raw 2048 B, bond
≤ 4), reconstruction SNR vs on-wire bytes as residual layers accumulate:
@ -91,9 +91,9 @@ Measured on a realistic 256-sample multi-tone + noise stream (raw 2048 B, bond
| K=4 | 10.1 dB | 370 B (5.5×) |
| K=6 | 13.3 dB | 546 B (3.8×) |
Two honest negatives, both recorded so the scope is not oversold:
Two limits, both recorded so the scope is stated plainly:
- **Not rate-competitive.** ~13 dB at 3.6× is far below the zeroth menu; the
- **Low rate–distortion.** 13.3 dB at 3.8× with six layers (the table above); the
noise floor is incompressible and multi-tone content out-ranks bond-4. On a toy
sinusoid it reaches ~20 dB @ 2×, but that is a flattering input.
- **Entropy coding of the trits is break-even.** The trit stream is ~54% zeros

123
wai/glossary/SCHEMA.md
View file

@ -2,42 +2,61 @@
Canonical, machine-readable catalog of media codecs surveyed for WAI capability registration. Lives in [`codecs.json`](codecs.json) alongside this schema.
The glossary is reference material — it is **not** the registry of WAI capabilities (that lives in [`../SPEC.md §5`](../SPEC.md)). Listing a codec here means it exists and is documented; listing a codec in the spec means a conforming sink MAY or MUST implement it.
The glossary is reference material — it is **not** the registry of WAI capabilities (that is [`../SPEC.md §5`](../SPEC.md), published in machine-readable form as [`../registry/capabilities.json`](../registry/capabilities.json)). Listing a codec here means it exists and is documented; listing a codec in the spec means a conforming sink MAY or MUST implement it.
## Provenance
Originally extracted from the inline JavaScript array in `wai-info-page/wai-glossary.html` (the user's purpose-built reference page) into typed form, then ported to JSON for language-neutral consumption.
Originally extracted from the inline JavaScript array of an earlier reference page (`wai-info-page/wai-glossary.html`, not part of this repository) into typed form, then ported to JSON for language-neutral consumption.
Mirrored as a typed Astro module at [`wai-transaction-science-web/src/data/codecs.ts`](https://wai.transaction.science/glossary) — the site renders the JSON statically at build time so search and filters work without JavaScript.
The site's glossary page (https://wai.transaction.science/glossary) renders a hand-ported copy of this file, so a change here reaches the site only when that copy is updated to match.
## Top-level shape
```json
{
"schema": "https://github.com/Transaction-Science/open-standards/blob/main/wai/glossary/SCHEMA.md",
"entries": [ Codec, ... ]
"schema": "https://git.transaction.science/Transaction-Science/open-standards/src/branch/main/wai/glossary/SCHEMA.md",
"schema_version": 2,
"entries": [ Codec, ... ]
}
```
`schema_version` is the version of this schema the file follows. Version 2 is the form described here. The earlier form carried no `schema_version`, and its entries had an `org` field where version 2 has `body`; a reader that finds no `schema_version` has that form. A change that renames or removes a field, or changes what a field means, raises the version.
## `Codec` entry
```ts
{
name: string; // canonical short name, e.g. "AV1", "HEVC", "JPEG-XL"
year: number; // year of first published spec or first release
org: string; // standards body, vendor, or research group that owns lineage
modality: "image" | "video" | "audio" | "speech";
name: string; // the format's name or designation, e.g. "AV1", "HEVC / H.265"
year: number; // year of first published spec or first release
body: string | null; // the standards body, standard designation or open-source
// project the format belongs to, from the allowlist below;
// null when it has none (a proprietary format, a paper)
modality: "image" | "video" | "audio" | "speech" | "text" | "volumetric" | "haptic" | "metadata";
era: "classical" | "neural";
status: "success" | "niche" | "superseded" | "failed" | "academic" | "emerging";
lineage: "industry" | "standards" | "academic" | "opensource";
desc: string; // 1–3 sentence factual summary — what it is, who made it,
// its design choice, its current footprint. Self-referential
// only (no "better than X", no competitor framing).
desc: string; // 1–3 sentence factual summary — what it is, the specification
// that defines it, its design choice, its current footprint.
// Self-referential only: no comparisons, no company, product
// or person names.
wai?: WaiLink[]; // OPTIONAL: present only on an entry whose bitstream a
// registered WAI capability carries (SPEC §5)
}
WaiLink = {
capability: string; // a payload or companion capability SPEC §5 registers
licence: "royalty-free" | "patent-pool" | "proprietary" | "non-commercial";
replicate: "standard-defined" | "standard-tolerance" | "standard-unclassified" | "neural-int-exact"
| "neural-entropy-exact" | "generative-exact" | "neural-float" | "behavioral";
}
```
### Field values
- **`name`** identifies the format. A format's own name, its FourCC, its file extension and its standard designation are the identifiers a reader recognises the bitstream by, and an entry is titled by one of them. Where the format's name embeds the name of a company, of a company's product line, or of the product it shipped in (a player, a framework, a platform, a capture tool), the entry is titled by its standard designation where it has one (for example "AC-3 (ATSC A/52)", "VC-1 (SMPTE 421M)", "SMPTE RDD 36", "ETSI TS 103 491"), otherwise by its FourCC or file-format name ("BIK video (.bik)", "RV10–RV40 (FourCC)"), otherwise by what it is ("Layered height-channel cinema audio (2010)"). A research codec is titled by its paper's method, not its authors ("Scale hyperprior (2018)").
- **No entry names a company, a product or a person**, in `name` or `desc`. A `desc` describes the platform a format shipped in by its function ("a desktop video framework", "a 2000s browser-plugin multimedia platform", "the FLV container"), and cites a paper by its title and venue. The public-name check (`scripts/check-public-names.sh`, with the private pattern list) scans this file, so an entry that breaks the rule fails CI; JSON has no comments, so no line here can carry an `allow-name` opt-out.
- **`body`** is a value from the allowlist below, or `null`. It is never a company or a person.
- **`modality`** is the media class. `text` is general-purpose byte compression; `volumetric` is meshes, point clouds and Gaussian splats; `haptic` is haptic signals; `metadata` is data applied with another payload (HDR dynamic metadata). A linked entry's modality is the registry's media class of the capability it links (`speech` counts as `audio`).
- **`status`** is descriptive, not normative:
- `success` — the codec carries non-trivial production traffic today.
- `niche` — alive in specific contexts (icons, pipelines, debug tools, particular hardware).
@ -46,24 +65,88 @@ Mirrored as a typed Astro module at [`wai-transaction-science-web/src/data/codec
- `academic` — published only in research; no production deployment.
- `emerging` — recent enough that adoption trajectory is undetermined.
- **`lineage`** captures origin culture (affects how the codec evolves):
- `industry` — vendor-driven (a company shipping a product).
- `standards` — formal body (ISO, ITU-T, IETF, IEEE, SMPTE).
- `industry` — developed by a company for its product.
- `standards` — a formal standards body (ISO, ITU-T, IETF, IEEE, SMPTE).
- `academic` — research lab.
- `opensource` — community-led from inception.
- **`era`** marks the discontinuity: `classical` = handcrafted DCT/wavelet/predictive, `neural` = end-to-end learned bitstream.
### `body` allowlist
- `ISO/IEC JTC 1/SC 29 (MPEG)`
- `ISO/IEC JTC 1/SC 29 (JPEG)`
- `ITU-T`
- `ITU-T and ISO/IEC JTC 1/SC 29 (MPEG)`
- `ITU-T and ISO/IEC JTC 1/SC 29 (JPEG)`
- `ISO`
- `IEC`
- `IEEE`
- `IETF`
- `W3C`
- `ETSI`
- `3GPP`
- `3GPP2`
- `TIA`
- `ATSC`
- `SMPTE`
- `Alliance for Open Media`
- `Xiph.Org Foundation`
- `WebM Project`
- `Khronos Group`
- `Bluetooth SIG`
- `IAU FITS Working Group`
- `Interactive Multimedia Association`
- `PNG Development Group`
- `FS-1015`
- `MIL-STD-3005`
- `T/AI 109.2-2021`
- `T/AI 109.3-2023`
- `X Window System`
- `Netpbm`
- `OpenEXR`
- `Radiance`
- `QOI`
- `MozJPEG`
- `FLIF`
- `FFmpeg`
- `Xvid`
- `Ut Video`
- `ScummVM`
- `Musepack`
- `WavPack`
- `TTA`
- `lossyWAV`
- `Codec 2`
- `LPCNet`
- `Lyra`
- `EnCodec`
- `SNAC`
- `Draco`
- `SPZ`
- `XZ Utils`
A value is a standards body, a standard designation (for a format whose issuing body was not read from a primary source, the designation stands in for it), or an open-source project. Adding one is a change to this list and to nothing else.
### `wai` — the link to the registry
The registry of capabilities is SPEC §5 and the reference implementation's `wai-rs/src/codecs`. An entry's `wai` field records which registered capability decodes the codec the entry describes, and the two classes the registry gives that capability.
- `licence` and `replicate` are the registry's classes for `capability`, spelled as `wai capabilities` prints them and as [`../registry/capabilities.json`](../registry/capabilities.json) records them (`codecs::capability_license` with `LicenseClass::label`; `codecs::capability_replicate_class` with `ReplicateClass::label`). The registry is the source; the glossary repeats the classes so a reader of this file sees them, and a test holds the two equal.
- An entry links a capability when that capability's payload is the entry's bitstream, in the framing SPEC §5 pins for it. Several entries may link one capability (AAC, HE-AAC and HE-AAC v2 all link `wai.audio.aac`), and one entry may link several (EVC links `wai.video.evc` and `wai.video.evc.baseline`). An entry SPEC §5 excludes from a capability does not link it: xHE-AAC and MPEG-D USAC do not link `wai.audio.aac`.
- The links are a reviewed table in the test (`GLOSSARY_LINKS`), so a reviewer judges each mapping once; the file must match it exactly.
- Every payload and companion capability is linked from at least one entry, or is listed in the test's `GLOSSARY_NO_ENTRY` table with the reason (a WAI-native class, a candidate, a model with no entry yet).
`glossary_links_agree_with_the_registry`, a unit test in `wai-rs/src/codecs/glossary_check.rs`, checks this file against this schema, the allowlist and the registry: the schema version, the fields and values above, unique names, the body allowlist, the link table, the classes, the media class, coverage, the excluded pairs, and the no-comparison rule for `desc`. Each failure names the entry and the rule it breaks.
## Contributing
Two operations:
1. **Add a codec** — append a new entry to `entries[]`. Keep entries roughly chronological within their modality block.
2. **Correct a description** — edit `desc`. Aim for factual density over flair: spec year, the spec body that owns it, the architectural choice that defines it, its current deployment footprint. No comparisons; no opinions about quality unless backed by a citation in the description.
2. **Correct a description** — edit `desc`. Aim for factual density over flair: spec year, the specification or body that defines it, the architectural choice that defines it, its current deployment footprint. No comparisons, no company names, no opinions about quality unless backed by a citation in the description.
A patch that touches only `codecs.json` requires no test run. A patch that adds a new field or status enum value should:
1. Update `SCHEMA.md` first (define the field).
2. Update the typed module at `wai-transaction-science-web/src/data/codecs.ts` and the glossary page that consumes it (`wai-transaction-science-web/src/pages/glossary.astro`).
A patch that touches `codecs.json` must pass `glossary_links_agree_with_the_registry` (from `wai-rs/`: `cargo test --lib --no-default-features glossary`); the conformance workflow runs it on every change under `wai/glossary/`. A patch that adds a field, an enum value or a `body` value updates this file first, then the test's tables (`GLOSSARY_FIELDS`, `GLOSSARY_ENUMS`), then the site's copy.
## Relationship to the WAI spec
A codec listed here becomes a WAI capability (e.g., `wai.image.jxl`) only when the spec registers it in `SPEC.md §5`. Registration carries normative requirements: a canonical reference library, a payload format declaration, and conformance via bit-exact decode-equivalence. The glossary documents the full landscape; the spec selects from it.
A codec listed here becomes a WAI capability (e.g., `wai.image.jxl`) only when the spec registers it in `SPEC.md §5`. A registration fixes the capability string, its payload format and its role (SPEC §5, §8), and the registry gives the capability a licence class and a replicate class. How a decode of it conforms depends on that class (SPEC §7): bit-exact decode-equivalence for some classes, the named standard's own criterion for others. The glossary documents the wider landscape; the spec selects from it, and an entry the spec has selected says so in its `wai` field.

1999
wai/glossary/codecs.json
View file

File diff suppressed because it is too large Load diff

6
wai/neural-conformance/README.md
View file

@ -16,9 +16,9 @@ byte-equality corpus.
Because the decoder pulls in no float runtime, it lives at the crate root
behind the light `neural_int` feature (no `ort`), so this corpus is verified
in the same no-system-deps CI matrix as the reconstruct classes — proving
the integer-exact contract on x86-64 and arm64 across Linux, macOS, and
Windows, every push.
in the same no-system-deps CI matrix as the reconstruct classes, which checks
the integer-exact contract on x86-64 and arm64 (Linux x86_64, Linux arm64 and
macOS arm64) on every push that touches it.
## Layout

6
wai/quantum-ops-conformance/expected_hashes.json
View file

@ -4,18 +4,18 @@
"vectors": {
"cal_q3": {
"kind": "calibration",
"receipt_hash": "d2d878b6ccb1ef98ffa1c1134bbdcfafd933d12d5077c0f74f2250a994e78b3a",
"receipt_hash": "7c53298929a1261959315c91f9540781b4bff0669b5f4b783becee93dbf5f8fa",
"verifies": true
},
"cal_cz": {
"kind": "calibration",
"receipt_hash": "a0f46ce355da35268c9897c7bb78d021bba903c8e8c68670d03c8dcc14096aaa",
"receipt_hash": "4c6374ee05c956f928455f86326357f10d08f3466e88d569a6ef49975001f526",
"verifies": true,
"parent": "cal_q3"
},
"job_bell": {
"kind": "job",
"receipt_hash": "e4167062b34974276c7172832f35960328e5d3843e5b48580fb6bd980ac96b41",
"receipt_hash": "08b142d29dadc466b294b89d2127f39ee9fa1aa0b6b5a4c6c5d226baf343b9e3",
"verifies": true,
"calibration_ref": "cal_q3"
}

2
wai/quantum-ops-conformance/vectors/cal_cz.json
View file

@ -1 +1 @@
{"kind":"quantum-calibration","config_hash":"4579a9a6bd54aee3be094201b99e45bc7e6c22bb61f0b09a09f170d9c487a0ec","device_id":"qm:opx1000:lab3","evidence":[{"blob_hash":"1a03735f5c166c618d0184eb75a27597052ba584bebee1a3bb950a8715dc64e2","kind":"interleaved_rb","summary":"cz gate fidelity 0.9871"}],"grant":{"capability":"quantum.calibrate","funds_ceiling":null,"grant_hash":"65e3c147b00bf7159cf77ea73683a8278a594e879020b8d1acea970d1b65530d","joule_ceiling_micro":8000000},"joules_micro":6800000,"merkle_root":"5705b7eb3288eca42b2472844acd411cf97af5550ae02f1bfb25889bcaf4d8ae","parent_receipt_hash":"d2d878b6ccb1ef98ffa1c1134bbdcfafd933d12d5077c0f74f2250a994e78b3a","receipt_hash":"a0f46ce355da35268c9897c7bb78d021bba903c8e8c68670d03c8dcc14096aaa","sig":"e9384f54078052e71c32126a799f56541179726eef7cb651c4f93128db809796b8b3b8596c332f1bb8db0f13f29b83cfc490af578e478387ebb115f535f5f704","signer_id":"did:key:demo-lab","signer_pubkey":"d04ab232742bb4ab3a1368bd4615e4e6d0224ab71a016baf8520a332c9778737","target":"cz(q3,q4)"}
{"kind":"quantum-calibration","config_hash":"4579a9a6bd54aee3be094201b99e45bc7e6c22bb61f0b09a09f170d9c487a0ec","device_id":"example:lab3-device","evidence":[{"blob_hash":"1a03735f5c166c618d0184eb75a27597052ba584bebee1a3bb950a8715dc64e2","kind":"interleaved_rb","summary":"cz gate fidelity 0.9871"}],"grant":{"capability":"quantum.calibrate","funds_ceiling":null,"grant_hash":"65e3c147b00bf7159cf77ea73683a8278a594e879020b8d1acea970d1b65530d","joule_ceiling_micro":8000000},"joules_micro":6800000,"merkle_root":"5705b7eb3288eca42b2472844acd411cf97af5550ae02f1bfb25889bcaf4d8ae","parent_receipt_hash":"7c53298929a1261959315c91f9540781b4bff0669b5f4b783becee93dbf5f8fa","receipt_hash":"4c6374ee05c956f928455f86326357f10d08f3466e88d569a6ef49975001f526","sig":"005ed3258c569afeaeb3f96062e60c2743cbf93d5c89982efef3dbb401325b7b0722d18b405dfeadefa396b4027e20af7a4851803f42f9d472142617d66dcc0a","signer_id":"did:key:demo-lab","signer_pubkey":"d04ab232742bb4ab3a1368bd4615e4e6d0224ab71a016baf8520a332c9778737","target":"cz(q3,q4)"}

2
wai/quantum-ops-conformance/vectors/cal_q3.json
View file

@ -1 +1 @@
{"kind":"quantum-calibration","config_hash":"5105f919eddea0c18dba494abf6c5ea71654c291517cf5262275f15acfc46e79","device_id":"qm:opx1000:lab3","evidence":[{"blob_hash":"226f5f5166a539c2f865b4deb26c3fa110c5a741403abf60def2d50d1b84e6f0","kind":"randomized_benchmarking","summary":"1q RB fidelity 0.9992"}],"grant":{"capability":"quantum.calibrate","funds_ceiling":null,"grant_hash":"65e3c147b00bf7159cf77ea73683a8278a594e879020b8d1acea970d1b65530d","joule_ceiling_micro":5000000},"joules_micro":4200000,"merkle_root":"57f9b6a130abc017a67d5d6ac948e972b4e1ec22d78c4a99d9bca9c3245753ed","parent_receipt_hash":null,"receipt_hash":"d2d878b6ccb1ef98ffa1c1134bbdcfafd933d12d5077c0f74f2250a994e78b3a","sig":"aaa04ba04d27a078d57ded5795f20c1c0bef26178a90a0bdc84b73a06a692ce63b89f654f8a4f2577c46a9d3d25c8562db30d510a6a40150b045fec6d4de3f0a","signer_id":"did:key:demo-lab","signer_pubkey":"d04ab232742bb4ab3a1368bd4615e4e6d0224ab71a016baf8520a332c9778737","target":"q3"}
{"kind":"quantum-calibration","config_hash":"5105f919eddea0c18dba494abf6c5ea71654c291517cf5262275f15acfc46e79","device_id":"example:lab3-device","evidence":[{"blob_hash":"226f5f5166a539c2f865b4deb26c3fa110c5a741403abf60def2d50d1b84e6f0","kind":"randomized_benchmarking","summary":"1q RB fidelity 0.9992"}],"grant":{"capability":"quantum.calibrate","funds_ceiling":null,"grant_hash":"65e3c147b00bf7159cf77ea73683a8278a594e879020b8d1acea970d1b65530d","joule_ceiling_micro":5000000},"joules_micro":4200000,"merkle_root":"57f9b6a130abc017a67d5d6ac948e972b4e1ec22d78c4a99d9bca9c3245753ed","parent_receipt_hash":null,"receipt_hash":"7c53298929a1261959315c91f9540781b4bff0669b5f4b783becee93dbf5f8fa","sig":"c5505b2dea07ad9c939f10551c3b563204156d8025b9f38f180f70605e1a0460c99f8de0cb224b9b55b5060e0c97885227a9d3670a76def114da1f73944ebf0d","signer_id":"did:key:demo-lab","signer_pubkey":"d04ab232742bb4ab3a1368bd4615e4e6d0224ab71a016baf8520a332c9778737","target":"q3"}

2
wai/quantum-ops-conformance/vectors/job_bell.json
View file

@ -1 +1 @@
{"kind":"quantum-job","backend_id":"ionq:aria-1","calibration_ref":"d2d878b6ccb1ef98ffa1c1134bbdcfafd933d12d5077c0f74f2250a994e78b3a","circuit_hash":"3071b48c3078c1fa737e7a94360a0fdcf5242f1f9785315f4ab94e50bb8e3a17","grant":{"capability":"quantum.job.run","funds_ceiling":50,"grant_hash":"883c52e9a16898a65e6ae3b5b896a8a5ff4686d927b411e91c9a2a4c92ee6fc9","joule_ceiling_micro":1000000},"joules_micro":250,"merkle_root":"66df81244d578d4b78e66ff69c553815ff5490f1db59764233b03a15f99c022a","parent_receipt_hash":null,"receipt_hash":"e4167062b34974276c7172832f35960328e5d3843e5b48580fb6bd980ac96b41","result_hash":"88dcf1cff7a0733a50578ed6e9d3073f21cc8644daa3b48f0d71d87bc1d4592e","shots":1000,"sig":"acf9d2b1019c466e4d7c642a58cc56f4845e1fc79e365f7d10ec229a8297ddc5d64837d22743ea9e71046901219fe719da1342446fbdfb39dde4ffeea3c6b208","signer_id":"did:key:demo-lab","signer_pubkey":"a09aa5f47a6759802ff955f8dc2d2a14a5c99d23be97f864127ff9383455a4f0"}
{"kind":"quantum-job","backend_id":"example:qpu-a","calibration_ref":"7c53298929a1261959315c91f9540781b4bff0669b5f4b783becee93dbf5f8fa","circuit_hash":"3071b48c3078c1fa737e7a94360a0fdcf5242f1f9785315f4ab94e50bb8e3a17","grant":{"capability":"quantum.job.run","funds_ceiling":50,"grant_hash":"883c52e9a16898a65e6ae3b5b896a8a5ff4686d927b411e91c9a2a4c92ee6fc9","joule_ceiling_micro":1000000},"joules_micro":250,"merkle_root":"fb000c7a1aea53bc465c9168082a061ff692ab7375b0a498a1ffb95a974b9fec","parent_receipt_hash":null,"receipt_hash":"08b142d29dadc466b294b89d2127f39ee9fa1aa0b6b5a4c6c5d226baf343b9e3","result_hash":"88dcf1cff7a0733a50578ed6e9d3073f21cc8644daa3b48f0d71d87bc1d4592e","shots":1000,"sig":"01fe297c5be82134cf725edb3eaf33de34fa19ce1b3b6fa4142ca844cf480821c785265087c1897b16029c5758d180707ed5507ef757f4db80ec1310f34b4e0f","signer_id":"did:key:demo-lab","signer_pubkey":"a09aa5f47a6759802ff955f8dc2d2a14a5c99d23be97f864127ff9383455a4f0"}

2
wai/registry/.gitattributes vendored Normal file
View file

@ -0,0 +1,2 @@
# Generated by `wai registry export`: compared byte for byte, so no line-ending conversion.
* -text

1408
wai/registry/capabilities.json Normal file
View file

File diff suppressed because it is too large Load diff

181
wai/registry/capabilities.schema.json Normal file
View file

@ -0,0 +1,181 @@
{
"$schema": "https://json-schema.org/draft/2020-12/schema",
"$id": "https://git.transaction.science/Transaction-Science/open-standards/src/branch/main/wai/registry/capabilities.schema.json",
"title": "WAI capability registry",
"description": "Every string SPEC §5 registers, generated by `wai registry export` from the reference implementation's registry (wai-rs/src/codecs/registry.rs). REGISTRY.md states the policy.",
"type": "object",
"additionalProperties": false,
"required": [
"schema",
"capabilities",
"identifiers",
"unregistered_labels"
],
"properties": {
"schema": {
"type": "string"
},
"capabilities": {
"type": "array",
"items": {
"type": "object",
"additionalProperties": false,
"required": [
"capability",
"kind",
"status",
"media",
"role",
"tier",
"licence",
"replicate",
"sets",
"spec_section",
"edition",
"registered"
],
"properties": {
"capability": {
"type": "string",
"pattern": "^wai(\\.[a-z0-9_]+)+$"
},
"kind": {
"enum": [
"payload",
"companion",
"derivation",
"record"
]
},
"status": {
"enum": [
"registered",
"candidate"
]
},
"media": {
"enum": [
"image",
"audio",
"video",
"text",
"volumetric",
"haptic",
"metadata",
"interactive",
"binary",
null
]
},
"role": {
"enum": [
"standalone",
"companion",
"enhancement",
null
]
},
"tier": {
"enum": [
"lossless",
"lossy",
"perceptual",
null
]
},
"licence": {
"enum": [
"royalty-free",
"patent-pool",
"proprietary",
"non-commercial",
null
]
},
"replicate": {
"enum": [
"standard-defined",
"standard-tolerance",
"standard-unclassified",
"neural-int-exact",
"neural-entropy-exact",
"generative-exact",
"neural-float",
"behavioral",
null
]
},
"sets": {
"type": "array",
"uniqueItems": true,
"items": {
"enum": [
"floor",
"recommended"
]
}
},
"spec_section": {
"type": "string"
},
"edition": {
"type": [
"string",
"null"
],
"description": "The edition the registration names; null means the edition of the named standard current on the date in `registered` (SPEC §8)."
},
"registered": {
"type": "string",
"pattern": "^[0-9]{4}-[0-9]{2}-[0-9]{2}$",
"description": "The date of the first commit of this repository that registers the string; the specification entered the repository on 2026-06-12."
}
}
}
},
"identifiers": {
"type": "array",
"items": {
"type": "object",
"additionalProperties": false,
"required": [
"identifier",
"spec_section",
"meaning"
],
"properties": {
"identifier": {
"type": "string",
"pattern": "^wai(\\.[a-z0-9_]+)+$"
},
"spec_section": {
"type": "string"
},
"meaning": {
"type": "string"
}
}
}
},
"unregistered_labels": {
"type": "array",
"items": {
"type": "object",
"additionalProperties": false,
"required": [
"label",
"reason"
],
"properties": {
"label": {
"type": "string",
"pattern": "^wai(\\.[a-z0-9_]+)+$"
},
"reason": {
"type": "string"
}
}
}
}
}
}

40
wai/research/2026-09-global-sweep.md
View file

@ -70,7 +70,7 @@
> the page's URL or document identifier is given as the citation, because the
> row cannot be checked without it. A company whose material was not read is
> neither named nor cited. Interface and platform names (RAPL, NVML, IOReport,
> Apple Silicon, ONNX Runtime) are used as identifiers, the way the tree's own
> [name removed 2026-09-30], ONNX Runtime) are used as identifiers, the way the tree's own
> texts use them, and so are file paths in the tree. Quotations from the tree and
> from sources are verbatim, except that a company name inside one is elided
> with "…".
@ -164,8 +164,8 @@ observation.
| — its part number, ISO/IEC 23888-6 (MPEG-AI Part 6), and later stages | the meeting page names the CfP but not the part number. CD at MPEG 158; FDIS early 2028 | — | [`mpeg-ai-part6-mapping.md`](mpeg-ai-part6-mapping.md); 2030 §6.5 | tree |
| Beyond-VVC Joint Call for Proposals (JVET, SC 29/WG 5 with ITU-T) | issued at JVET's 43rd meeting. Registration by early September 2026; main package due **end of October 2026**, a supplemental package mid-December; evaluation at the JVET meeting in Brisbane, **January 2027**; first test model in the second half of 2027; first version targeted **end of 2029**. **Welcomes technology based on neural networks and AI** alongside conventional tools | — | https://www.mpeg.org/155th-meeting-of-mpeg/ | P✓ |
| Gaussian splat coding — Exploration 45 | exploration. The page lists documents from **MPEG 150** (draft use cases and requirements, 2025-04-26) through MPEG 152 and 154 (common test conditions) to **MPEG 155** (use cases, 2026-07-18; a **call for content**, 2026-07-31). It names **no working group** | MPEG 150 onward | https://www.mpeg.org/standards/Explorations/45/ (read 2026-09-27) | P✓ |
| — its tracks and timeline | intra (I-3DGS) and inter (A-3DGS) tracks; 83 inputs at MPEG 155; codec selection 2027; standard targeted Oct 2029. **None of this is on the part page** | MPEG 155 | — | **R** ⚠ |
| V-PCC (ISO/IEC 23090-5) Gaussian-splat amendment | CDAM | MPEG 153 | https://www.mpeg.org/153rd-meeting-of-mpeg/ | P✓ |
| — its tracks and timeline | intra (I-3DGS) and inter (A-3DGS) tracks *(2026-09-30: WG 4 N 837 and WG 2 N 531 define I-3DGS and A-3DGS as representations — I-3DGS "point clouds represented in the “INRIA” format", A-3DGS "alternative Gaussian splat representations" — not intra and inter tracks)*; 83 inputs at MPEG 155; codec selection 2027; standard targeted Oct 2029. **None of this is on the part page** | MPEG 155 | — | **R** ⚠ |
| V-PCC (ISO/IEC 23090-5) Gaussian-splat amendment | CDAM *(2026-09-30: Draft Amendment since MPEG 154, per WG 4 N 837)* | MPEG 153 | https://www.mpeg.org/153rd-meeting-of-mpeg/ | P✓ |
| G-PCC (ISO/IEC 23090-9) Gaussian-splat Amd 1 | **CDAM** | MPEG 154 | https://www.mpeg.org/154th-meeting-of-mpeg/ | P✓ |
| Enhanced G-PCC, ISO/IEC 23090-38 | FDIS | MPEG 155 | https://www.mpeg.org/155th-meeting-of-mpeg/ | P✓ |
@ -328,7 +328,7 @@ registers `wai.video.vvc`, `wai.video.lcevc` (companion-only) and
|---|---|---|
| Germany DVB-I launch | planned for 2026; **sources give conflicting dates** | **C** ⚠ |
| Spain national multiplex RGE2 | moves to DVB-T2 + HEVC UHD from 5 Nov 2026 | **R** ⚠ |
| Fraunhofer FOKUS, *Green Streaming* whitepaper (Oct 2024) | kWh/GB is an inadequate metric; end devices account for 70–80% of chain energy | **R** ⚠ |
| [name removed 2026-09-30], *Green Streaming* whitepaper (Oct 2024) | kWh/GB is an inadequate metric; end devices account for 70–80% of chain energy | **R** ⚠ |
| FAMIUM Green hardware-in-the-loop energy rig | *Computers* 15(10):653, 25 Sep 2026. **Body not read** | P (citation; not re-read) / ⚠ content |
| France RGESN | 17 May 2024; 78 criteria; mandatory for public bodies | **R** ⚠ |
| France ARCEP decision 2025-2428 | declarative; **excludes streaming platforms** | **R** ⚠ |
@ -355,10 +355,10 @@ The claim is only as strong as that one source.
## 4. Industry deployment reality (2026)
- **Codec mix.** An encoder vendor's survey (March 2026; vendor-run; **sample
size not stated**; not read directly, and not named under this record's
naming rule) reports production use of H.264 84%, HEVC 65%,
AV1 17%, VP9 15% and VVC 4%. **S** ⚠. The sweep's reading is that VVC ships
- **Codec mix.** An encoder vendor's survey (March 2026; vendor-run; not read
directly, and not named under this record's naming rule) reports production
use of H.264, HEVC, AV1, VP9 and VVC [figures removed 2026-09-30: a
vendor-run survey's numbers are not cited by this record]. **S** ⚠. The sweep's reading is that VVC ships
at scale only where a national mandate requires it (Brazil, §3.3). **R** ⚠.
That is an inference from the sources found, not a census.
- **Media over QUIC.** The CDN relay service found by the sweep publishes
@ -366,11 +366,14 @@ The claim is only as strong as that one source.
under the naming rule). It lists relays for
**draft-14** and **draft-16** of MoQ Transport and a **draft-18** test relay,
and describes the service as a **beta** ("free to use during the beta
period"). P✓ (https://developers.cloudflare.com/moq/). The current transport
period"). P✓ ([URL removed 2026-09-30]). The current transport
draft is **-21** (§2.3). There are niche production deployments and many
trade-show demonstrations (**R** ⚠). **No top consumer platform was found
delivering to viewers over MoQ** (**R** ⚠). WebTransport is reportedly
baseline across the major browsers (**S** ⚠). *Bearing:* any interop claim for
baseline across the major browsers (**S** ⚠) *(2026-09-30: confirmed from a
primary source — the W3C WebTransport Candidate Recommendation Snapshot of
2026-07-30, and Baseline status since 2026-03-24 in the web-platform status
data)*. *Bearing:* any interop claim for
[`moq-streaming-format.md`](../extensions/moq-streaming-format.md) must name
the transport draft it was tested against. The relay found serves -14 and -16
and is testing -18, while the WG text is at -21.
@ -548,8 +551,8 @@ and it tests devices, not streams.
### 5.3 How the monorepo's standards class an IOReport reading
IOReport is the SoC energy interface the reference meters read on Apple
Silicon. This table records, per standard, the class an IOReport reading
IOReport is the SoC energy interface the reference meters read on arm64 macOS
[name removed 2026-09-30]. This table records, per standard, the class an IOReport reading
actually gets, from the normative text and from the reference code. It was made
by reading every tracked line that mentions IOReport (`git grep -n -i ioreport`,
outside this record), with its surrounding lines wherever a class name was near,
@ -563,12 +566,12 @@ from code, it is the tag the code assigns, not an observed receipt.
|---|---|---|---|
| **JCR-1** (the shared receipt envelope) | `jouleclaw/jouleclaw-rs/crates/jouleclaw-jcr1/docs/JCR-1-spec.md`:69–72 (§2.3): `ModelBased` — "vendor model estimate from freq/voltage/utilization (… IOReport, NVML `power.draw`)" | conformance vector 6 (`conformance/vectors.json`:148, `conformance/README.md`:31): `ModelBased` | **`ModelBased`** |
| **JouleClaw** | `jouleclaw/SPEC.md`:73–74 (provenance table), :88 ("Model-based, not measured"): `ModelBased`. The same table puts ROCm SMI under `ModelBased` | `jouleclaw-energy`: `src/apple_ioreport.rs`:1, :11 and test `provenance_is_model_based` (:431–433); `conformance/vendors/apple-silicon-ioreport.json`:12; `MEASURED_ENERGY.md`:23 — `ModelBased`. Its load-average fallback, `src/apple.rs`:1, :89–91, is `Estimator`. **`jouleclaw-compliance`** `src/passport.rs`:132–133 (and test :701–704) maps `MeasurementMethod::IoReport` to **`Estimator`**, "until the FFI is wired". `jouleclaw-energy/conformance/README.md`:26 lists the ROCm SMI counter as `HwShunt`, against the SPEC | **`ModelBased`**; **`Estimator`** in `jouleclaw-compliance` |
| **JCP** | `jcp/SPEC.md` §5.1 (:395–402): RAPL and NVML → `HwShunt`, time × power → `ModelBased`. **No IOReport rung** | `jcp-energy` `src/lib.rs`:15–17, :435–438, :481 (`Meter::IoReport => Provenance::ModelBased`): `ModelBased`. **`jcp-gateway`** `src/lib.rs`:429–436 seals its metered evaluation as `HwShunt` whenever the Sandbox `PowerMeter` reads hardware, which on Apple Silicon is IOReport (`jcp/README.md`:49) | **split**: `ModelBased` (`jcp-energy`), `HwShunt` (`jcp-gateway`) |
| **JCP** | `jcp/SPEC.md` §5.1 (:395–402): RAPL and NVML → `HwShunt`, time × power → `ModelBased`. **No IOReport rung** | `jcp-energy` `src/lib.rs`:15–17, :435–438, :481 (`Meter::IoReport => Provenance::ModelBased`): `ModelBased`. **`jcp-gateway`** `src/lib.rs`:429–436 seals its metered evaluation as `HwShunt` whenever the Sandbox `PowerMeter` reads hardware, which on arm64 macOS [name removed 2026-09-30] is IOReport (`jcp/README.md`:49) | **split**: `ModelBased` (`jcp-energy`), `HwShunt` (`jcp-gateway`) |
| **Joule Code** | `joule-code/spec/receipt.md`:32 (normative sub-spec): `HwShunt` — "read from a hardware energy interface (RAPL, … SMC/IOReport, an external shunt)". Its signed form is a JCR-1 envelope (:38–40) | `joule-code-remediate` `src/lib.rs`:635–657 seals an attributed share of a Sandbox-meter reading as `ModelBased` (its source string: "attributed(ModelBased over IOReport HwShunt)"). `examples/l3-jouleclaw/src/bin/run.rs`:29–33 uses JouleClaw's `ModelBased` counter; that example's `README.md`:44–45 still says `Estimator` "until IOReport is wired" | **split**: `HwShunt` (normative text); `ModelBased` in both code paths |
| **Sandbox** | `sandbox/SPEC.md`:217 (§6.3) lists RAPL/powercap, NVML, ROCm SMI, Level Zero and "an SMC/shunt otherwise" under `HwShunt`; **IOReport is not named**. `sandbox/README.md`:9: the reference meter reads IOReport, RAPL and NVML, "each tagged `HwShunt`" | `sandbox-meter` `src/probe/mod.rs`:8–10, :247 and `src/probe/ioreport.rs`:3: `HwShunt`; `sandbox-proof` `src/lib.rs`:1153 likewise. `sandbox-tee` `src/energy.rs` gives IOReport a `CounterClass` of its own (`IoReport`), not independently rooted | **`HwShunt`** (receipt); own class in `sandbox-tee` |
| **Proof** | `proof/SPEC.md`:89 and `proof-schema` `src/lib.rs`:104: `HwShunt` = RAPL/NVML/SMC/shunt; **IOReport not named** | `proof-cascade` `src/meter.rs`:1–2, :23 calls itself "the `HwShunt` backend" and reads IOReport on Apple Silicon; `proof-cli` `src/main.rs`:90–93, :117 seals the reading with `Energy::measured`, which sets `HwShunt` (`proof-schema` `src/lib.rs`:153–158). The same module doc, `meter.rs`:9–10, says "a macOS reading is `ModelBased`" | **split**: `HwShunt` (code); `ModelBased` (its own module doc) |
| **Proof** | `proof/SPEC.md`:89 and `proof-schema` `src/lib.rs`:104: `HwShunt` = RAPL/NVML/SMC/shunt; **IOReport not named** | `proof-cascade` `src/meter.rs`:1–2, :23 calls itself "the `HwShunt` backend" and reads IOReport on arm64 macOS [name removed 2026-09-30]; `proof-cli` `src/main.rs`:90–93, :117 seals the reading with `Energy::measured`, which sets `HwShunt` (`proof-schema` `src/lib.rs`:153–158). The same module doc, `meter.rs`:9–10, says "a macOS reading is `ModelBased`" | **split**: `HwShunt` (code); `ModelBased` (its own module doc) |
| **Sense** | `sense/SPEC.md`:166–170: the reference meter is Sandbox's `PowerMeter` ("real RAPL/IOReport") | `sense-schema` `src/lib.rs`:214–215: `HwShunt` — "A hardware energy reading (RAPL, IOReport, NVML)" | **`HwShunt`** |
| **Joule UI** | `joule-ui/SPEC.md`:73: the reference meter is Sandbox's `PowerMeter` ("real RAPL/IOReport") | `joule-ui-serve` `Cargo.toml`:21–23 and `src/bin/joule_ui_serve.rs`:29–31: an Apple Silicon IOReport reading makes "the receipt's `HwShunt`" true | **`HwShunt`** |
| **Joule UI** | `joule-ui/SPEC.md`:73: the reference meter is Sandbox's `PowerMeter` ("real RAPL/IOReport") | `joule-ui-serve` `Cargo.toml`:21–23 and `src/bin/joule_ui_serve.rs`:29–31: an arm64 macOS [name removed 2026-09-30] IOReport reading makes "the receipt's `HwShunt`" true | **`HwShunt`** |
| **Mesh** | `mesh-schema` `src/lib.rs`:41–44: `HwShunt` = RAPL/NVML/SMC/shunt; IOReport not named | `mesh-meter` `Cargo.toml`:22 (`hw-meter-apple` enables JouleClaw's IOReport counter); `src/lib.rs`:222–233 passes JouleClaw's provenance through unchanged | **`ModelBased`** (inherited from JouleClaw) |
| **JouleDesk** | — | `jouledesk-github` `Cargo.toml`:20–21, `src/lib.rs`:34–44: JouleClaw's `detect_meter` with the IOReport feature; records the meter's own provenance | **`ModelBased`** (inherited from JouleClaw) |
| **EOC** | `eoc/vocab/v1.ttl`:34–35 carries the three names; IOReport is not mapped | `eoc-core` `src/joule_cost.rs`:7–12 has two classes, `Measured` and `Estimated`; `eoc-meter` `src/lib.rs`:56–61, :145 reads IOReport as `Measured` ("Measured-grade") | **`Measured`** (a two-class vocabulary) |
@ -646,7 +649,7 @@ lands. It is not a statement that the finding needs nothing further.
| Brazil lawful-basis declaration (ABNT NBR 25608 §8.8) | 2030 landscape §7.2; at link 10 also `jwp-receipts.md` §2.4.6, `wai-rs/src/claim_receipt.rs` (feature `receipt_claims`), `claim-conformance/` | *`main`:* analysed, open ingest. *Link 10:* specified and implemented as a claim class (PR #112) | *`main`:* no extension or code carries it (grep for `lawful`, `PRRD`, `privacy receipt` and `25608` outside the landscapes). *Link 10:* a `lawful-basis` claim is a declaration of purposes, each with its own basis and data categories from a declared vocabulary, or a decision recorded against one; its shape is the one jwp-receipts §2.4.6 attributes to the standard's public operational guidelines. It is not a consent mechanism, and nothing supplies an operator's key (jwp-receipts §2.4.6). The clause-number caveat of §3.3 stands. 2030 §8 item 4 |
| MPEG-AI Part 6 CfP | [`mpeg-ai-part6-mapping.md`](mpeg-ai-part6-mapping.md) | analysed | requirement mapping; membership blocker stated |
| the C2PA environmental-sustainability assertion | `energy-binding.md`; `wai-rs/src/energy_binding.rs` (`C2PA_STANDARD_LABEL`); `wai-rs/src/c2pa_emit.rs` | implemented | emitted as a companion alongside WAI's own assertion |
| MPEG's splat and point-cloud work: Exploration 45; the V-PCC and G-PCC Gaussian-splat amendments; Enhanced G-PCC, ISO/IEC 23090-38 (§2.1) | [`media-codec-registry.md`](media-codec-registry.md), *Volumetric / 3D geometry compression*; `wai-rs/src/codecs/mod.rs` (`CAP_PC_GPCC`, `CAP_PC_VPCC`, `CAP_SPLAT_SOG`, `CAP_SPLAT_INT_CODEC`); `wai-rs/src/int_splat.rs`; [`volumetric.md`](../extensions/volumetric.md) (`wai.splat.4d`); [`3d-neural-coding-landscape.md`](3d-neural-coding-landscape.md); 2030 §6.4 | `wai.splat.int_codec` **implemented**. `wai.pc.gpcc`, `wai.pc.vpcc` and `wai.splat.sog` are **registered**: a capability string, a licence class and a tier, and no decoder inside `wai-rs` (grep). Their decoders are sink-supplied. `media-codec-registry.md` records a demonstrated end-to-end decode of `wai.pc.gpcc` through the MPEG reference tool built from source, and a real encode plus structural decode of `wai.pc.vpcc`, both via `tools/wai_carriage_demo.sh` (not re-run here). CI's `tests/carriage.rs` covers byte-identical carriage and known-capability dispatch for the registry, not these decodes | Registry: `wai.pc.gpcc` (23090-9) and `wai.pc.vpcc` (23090-5) are POOL. `wai.splat.sog` is "MPEG WG4 WIP", "RF (open work)", a **candidate** "registered so a capability exists ahead of an MPEG freeze". *`main`:* `capability_license` gives `PatentPool` for the two PCC capabilities and, through its default arm, `RoyaltyFree` for `wai.splat.sog`; `capability_replicate_class` gives all three `StandardDefined` through its default arm. *Link 10:* all three are `StandardUnclassified` (link 02, PR #103; SPEC §7's table), and `capability_license` lists every capability with no default class, `wai.splat.sog` as royalty-free "SOG open work" (link 04, PR #105). `3d-neural-coding-landscape.md` says splat coding applies NNC to the splat parameters; 2030 §6.4 records an NNC amendment covering Gaussian splats, unverified there (G11) |
| MPEG's splat and point-cloud work: Exploration 45; the V-PCC and G-PCC Gaussian-splat amendments; Enhanced G-PCC, ISO/IEC 23090-38 (§2.1) | [`media-codec-registry.md`](media-codec-registry.md), *Volumetric / 3D geometry compression*; `wai-rs/src/codecs/mod.rs` (`CAP_PC_GPCC`, `CAP_PC_VPCC`, `CAP_SPLAT_SOG`, `CAP_SPLAT_INT_CODEC`); `wai-rs/src/int_splat.rs`; [`volumetric.md`](../extensions/volumetric.md) (`wai.splat.4d`); [`3d-neural-coding-landscape.md`](3d-neural-coding-landscape.md); 2030 §6.4 | `wai.splat.int_codec` **implemented**. `wai.pc.gpcc`, `wai.pc.vpcc` and `wai.splat.sog` are **registered**: a capability string, a licence class and a tier, and no decoder inside `wai-rs` (grep). Their decoders are sink-supplied. `media-codec-registry.md` records a demonstrated end-to-end decode of `wai.pc.gpcc` through the MPEG reference tool built from source, and a real encode plus structural decode of `wai.pc.vpcc`, both via `tools/wai_carriage_demo.sh` (not re-run here). CI's `tests/carriage.rs` covers byte-identical carriage and known-capability dispatch for the registry, not these decodes | Registry: `wai.pc.gpcc` (23090-9) and `wai.pc.vpcc` (23090-5) are POOL. `wai.splat.sog` is "MPEG WG4 WIP", "RF (open work)", a **candidate** "registered so a capability exists ahead of an MPEG freeze". *`main`:* `capability_license` gives `PatentPool` for the two PCC capabilities and, through its default arm, `RoyaltyFree` for `wai.splat.sog`; `capability_replicate_class` gives all three `StandardDefined` through its default arm. *Link 10:* all three are `StandardUnclassified` (link 02, PR #103; SPEC §7's table), and `capability_license` lists every capability with no default class, `wai.splat.sog` as royalty-free "SOG open work" (link 04, PR #105). `3d-neural-coding-landscape.md` says splat coding applies NNC to the splat parameters; 2030 §6.4 records an NNC amendment covering Gaussian splats, unverified there (G11) *(2026-09-30: 2030 §6.4 verifies it against MPEG's MPEG 154 press release; see its "Verified (Sept 2026)" paragraph)* |
| a royalty class per capability | [`media-codec-registry.md`](media-codec-registry.md), *License classification*; `codecs::capability_license()` / `LicenseClass` | implemented | the mechanism any new pool-licensed registration uses. At link 04 the function is exhaustive with no default class, so a registered capability cannot become royalty-free by omission (G5) |
### 6.1 The 2030 energy-square falsifier, leg by leg
@ -682,7 +685,7 @@ review.
| # | gap (on `main`) | evidence on `main` | status at link 10 |
|---|---|---|---|
| G1 | **The integer decoders are scalar and single-threaded, and no throughput figure is published.** A determinism tier with no speed figure cannot be weighed against an approach that pays in bitrate rather than arithmetic (G7). | no `rayon`, `std::thread`, `core::arch`, `std::simd` or `target_feature` in `wai-rs/src/int_*.rs`, `rans.rs` or `jpegai_*.rs`; no throughput figure in `README.md`, `SPEC.md` or `determinism-tiers.md` (grep) | **Closed at link 07 (PR #108).** `wai-rs/src/int_kernels.rs` adds register-blocked convolution, transposed-convolution and IGDN kernels (SIMD widening multiply-accumulate on aarch64, portable Rust elsewhere) and scoped worker threads (none on wasm), selected by a run-time `Policy` whose default is the fast kernels on all cores. The scalar reference kernels are kept and selectable. The claim is that every policy gives the same integer for every output element; it is argued by construction and tested, not machine-proved (SPEC Appendix A). [`decode-throughput.md`](decode-throughput.md) §4 publishes whole-decoder and kernel figures from `wai_decode_bench`, measured on one macOS arm64 machine, best of four runs under a stated load (§3). Its limits (§5): one machine and one architecture; x86_64 compiled but not executed; the largest rows use synthetic weights. determinism-tiers §5 (*What `decode-equivalence` costs*) described the decoders as single-threaded with no explicit SIMD and no throughput figure until link 11, which describes the fast kernels and points to the note |
| G2 | **The SOTA-quality and JPEG-AI whole-decode tests skip-as-pass when their inputs are absent, and no CI job runs them.** A skipped test reports as passed. | `wai-rs/tests/int_hyper_synth.rs` and `int_video_sota_keyframe.rs` (`return` when the artifacts are not in `WAI_SOTA_INT_DIR`, default `/tmp/sota_int`); `jpegai_e2e_full.rs` (`return` without `WAI_JPEGAI_CODESTREAM` / `_WEIGHTS_DIR` / `_NORM`). The WAI workflows run `cargo test --lib`, `--test carriage`, `--test neural_native_decode`, the `byte-exact-conformance` stage goldens, the corpus verifiers and `wai_oer_vectors`; none runs the three test targets above | **Addressed at link 06 (PR #107); both workflows pass on the forge.** With `WAI_REQUIRE_SOTA_VECTORS=1` a missing or incomplete input fails the test and names the tool that produces it; without the switch the skip is written to the process's real stderr (`tests/support/require_vectors.rs`; `tests/require_vectors_rule.rs` checks the rule in child processes). `wai-sota-vectors.yml` regenerates the `int_hyper` vectors from pinned inputs and runs `int_hyper_synth`, `int_video_sota_keyframe` and `require_vectors_rule`. `wai-jpegai-e2e.yml` fetches the pinned JPEG AI inputs, checks them against `tools/jpegai-e2e-v1.sha256`, and runs `jpegai_e2e_full`, `jpegai_reconstruction_binding` and `jpegai_cross_platform`. Both set the switch, require each test by name, fail on a skip banner, and also run weekly. Both have run on the forge's Linux x86_64 runner and passed on every run since 2026-09-28 (runs read 2026-09-30); determinism-tiers §6 says so from link 11. Still in no workflow: `jpegai_live` and the dispatch test `jpegai_exact_bundle_decodes`, which need the `neural` feature and local assets |
| G2 | **The SOTA-quality and JPEG-AI whole-decode tests skip-as-pass when their inputs are absent, and no CI job runs them.** A skipped test reports as passed. | `wai-rs/tests/int_hyper_synth.rs` and `int_video_sota_keyframe.rs` (`return` when the artifacts are not in `WAI_SOTA_INT_DIR`, default `/tmp/sota_int`); `jpegai_e2e_full.rs` (`return` without `WAI_JPEGAI_CODESTREAM` / `_WEIGHTS_DIR` / `_NORM`). The WAI workflows run `cargo test --lib`, `--test carriage`, `--test neural_native_decode`, the `byte-exact-conformance` stage goldens, the corpus verifiers and `wai_oer_vectors`; none runs the three test targets above | **Addressed at link 06 (PR #107); both workflows pass on the forge.** With `WAI_REQUIRE_SOTA_VECTORS=1` a missing or incomplete input fails the test and names the tool that produces it; without the switch the skip is written to the process's real stderr (`tests/support/require_vectors.rs`; `tests/require_vectors_rule.rs` checks the rule in child processes). `wai-sota-vectors.yml` regenerates the `int_hyper` vectors from pinned inputs and runs `int_hyper_synth`, `int_video_sota_keyframe` and `require_vectors_rule`. `wai-jpegai-e2e.yml` fetches the pinned JPEG AI inputs, checks them against `tools/jpegai-e2e-v1.sha256`, and runs `jpegai_e2e_full`, `jpegai_reconstruction_binding` and `jpegai_cross_platform`. Both set the switch, require each test by name, fail on a skip banner, and also run weekly. Both have run on the forge's Linux x86_64 runner and passed on every run since 2026-09-28 (runs read 2026-09-30); determinism-tiers §6 says so from link 11. Still in no workflow: `jpegai_live` and the dispatch test `jpegai_exact_bundle_decodes`, which need the `neural` feature and local assets | <!-- allow-phrase: dated record; names the vector tests by their file names -->
| G3 | **SyfAI's AI update-framework thread, opened at MPEG 155 jointly with WG 7 (S ⚠), covers parameter formats, manifests, repositories, integrity checking and bit exactness for NN decoders. The artifact a WAI `Prior` names has no normative wire form.** `Prior { id, sha256, determinism }` pins identity: `sha256` hashes the decoder artifact the sink runs. No wire format for that artifact is specified (2030 §6.4). | `wai-rs/src/container.rs` `struct Prior`; 2030 landscape §6.4; no SyfAI or Exploration 52 mention in the tree (grep) | **Addressed at link 05 (PR #106), with limits.** [`prior-carriage.md`](../extensions/prior-carriage.md) §2: a `Prior` MAY declare `form`; when it does, the digest covers the coded bitstream, and a sink that cannot read the form MUST refuse the prior. §3 registers `wai.prior.nnc` (ISO/IEC 15938-17:2024) as the one form, optional and not recommended, because its royalty status and cross-decoder exactness are not established. `Prior::is_cross_hardware_exact` requires a declared form to be established exact, none is, so a `replicate` resting on a declared form is refused (§5). The reference sink reads no form (`READABLE_PRIOR_FORMS` is empty), and `wai-rs` has no NNC reader or writer. SPEC's opening section now says a model's identity MAY be pinned by digest (a `Prior`). [`syfai-input.md`](syfai-input.md), committed with this record, maps WAI onto the update framework; its G1 records what SPEC still does not define |
| G4 | **WAI's `HwShunt` takes in IOReport, a reading the monorepo classes five ways; and neither WAI nor the family's receipt ladder separates platform-relayed counters from independently rooted instruments. Only `sandbox-tee` does.** Three layers. **(a) The family's receipt ladder.** Every standard that lists interfaces under `HwShunt` puts on-die counters and dedicated shunts in it together (JCR-1 §2.3, JCP §5.1, Sandbox §6.3, `mesh-schema`, `proof-schema`), and IOReport is classed inconsistently (§5.3). **(b) `sandbox-tee`'s `CounterClass`** reserves `HwShunt` for a dedicated shunt and gives RAPL, NVML, IOReport and Redfish classes of their own; its `AttestTrust` ladder is derived from evidence, and `efficiency/1` commits to the attestation (§5.1). **(c) WAI** has `HwShunt` cover "RAPL, NVML, SMC, IOReport, or an external shunt", and `wai_video_meter` stamps IOReport readings `HwShunt`. That places WAI with Joule Code's text and the Sandbox meter, and against JCR-1 §2.3. WAI's texts present the ladder as the family's own without noting the split: `video_receipt.rs` says it is "reused verbatim from `mesh-schema` / `proof-schema`", which list no IOReport; `energy_binding.rs` calls it "the family's ladder"; `energy-measurement.md` §2 says WAI reuses "the same three names the receipt families elsewhere in the ecosystem already use". At every layer but (b), a verifier cannot tell a platform-relayed counter, which may be deliberately filtered (§5.1), from an independently rooted instrument. | §5.3 (the per-standard survey); `wai/extensions/energy-measurement.md` §2; `wai-rs/src/energy_binding.rs`:30, :47–51 (`Acquisition`); `wai-rs/src/video_receipt.rs` `Provenance` doc comment; `wai-rs/src/bin/wai_video_meter.rs`:184–190; `jouleclaw/jouleclaw-rs/crates/jouleclaw-jcr1/docs/JCR-1-spec.md` §2.3; `jcp/SPEC.md` §5.1; `sandbox/SPEC.md` §6.3–§6.4; `mesh/mesh-rs/crates/mesh-schema/src/lib.rs` and `proof/proof-rs/crates/proof-schema/src/lib.rs` (`Provenance`); `sandbox/sandbox-rs/crates/sandbox-tee/src/energy.rs` (`AttestTrust`, `CounterClass`, `is_independently_rooted`); `jcp/jcp-rs/crates/jcp-efficiency/MAPPING.md` §4 and `src/training.rs` (`energy_attestation`). No mention of `sandbox-tee`, `CounterClass`, `AttestTrust`, `energy_attestation`, `PhysicallyRooted` or `IntegrityAttested` under `wai/` (grep) | **WAI's side closed at link 08 (PR #109); the family's side open.** energy-measurement §2 separates `CalibratedInstrument` (an external instrument with a current calibration) from `OnChipCounter` (a counter the measured silicon keeps, or any instrument whose uncertainty is not traceable). Each carries a declared uncertainty (§2.3); the counter carries its filtering state (§2.4), the instrument a calibration reference (§2.5). `HwShunt` is legacy, accepted and never emitted (§2.6), and IOReport is `OnChipCounter` (§2.9). `energy_class.rs` is the one definition, and the reference meters seal `OnChipCounter`. The "reused verbatim" sentence is gone; `video_receipt.rs` and `energy_binding.rs` still call the ladder the family's, and §2 says the three-name vocabulary is extended, not replaced, with both refinements mapping to `HwShunt`. §2.9's list of differing documents is partial (§5.3). No WAI text references `sandbox-tee`'s ladder (grep). The family's IOReport split and whether its receipt-level ladder should separate the two kinds of hardware evidence are unassigned (below) |
| G5 | **The codec registry has no AV2, VVC, HEVC, LCEVC, MPEG-H Audio, AVS3 or Audio Vivid capability.** Brazil mandates three of these (§3.3). Two are Chinese standards: AVS3 (a group standard, with its national standard pending) and Audio Vivid (GB/T 46271-2025). 2030 §6.1 (*Honest scope*) records that WAI registers no VSEI-family codec. **The tree's codec survey is a second surface to reconcile.** `wai/glossary/codecs.json`, which `wai/README.md` describes as the "Reference catalog of media codecs surveyed for capability registration", has entries for HEVC / H.265 and VVC / H.266 (both `niche`), LCEVC (`niche`) and ECM (`academic`, JVET's exploration model beyond VVC), and none for AV2, MPEG-H Audio, AVS3 or Audio Vivid. `glossary/SCHEMA.md` defines `success` as "carries non-trivial production traffic today" and `niche` as "alive in specific contexts". §4 reports HEVC in production use at 65% in a vendor survey (**S** ⚠), §3.3 reports VVC as a national broadcast system's main codec (**R** ⚠), and the HEVC entry's own description calls it universal in broadcast UHD. The grades are descriptive and the contrary evidence is unverified, so this is a reconciliation item, not a correction. | capability strings in `wai-rs/src/codecs/*.rs`: none for these codecs. Other mentions, from a grep over `*.rs`, `*.md`, `*.json`, `*.toml`, `*.yml`, `*.yaml`, `*.html`, `*.ts` and `*.js` under `wai/`: the `glossary/codecs.json` entries above; the H.264/H.265 royalty sentence in `SPEC.md` and `CHANGELOG.md`; the BD-rate note that names VVC in `wai-rs/src/codecs/mod.rs` and `wai-rs/src/neural/mlicpp.rs`; `HEVC` as an example name in `glossary/SCHEMA.md`; the 2030 landscape. Royalty class per codec **not established by this sweep**: only AV2's royalty-free patent policy was seen at the issuing body (the AOMedia press release, §2.4) | **Closed in the registry at link 04 (PR #105); the glossary is not reconciled.** SPEC §5 *Broadcast and streaming codecs* registers `wai.video.{av2,vvc,hevc,avc,vp9,evc,evc.baseline,avs3,lcevc}` and `wai.audio.{mpegh,avs3,aac}`, each with one payload framing and a licence class whose basis is recorded in [`media-codec-registry.md`](media-codec-registry.md). `wai.audio.avs3` names AVS3 audio / Audio Vivid (GB/T 46271-2025). All are sink-supplied, none is a default, and `wai.video.lcevc` is companion-only. SPEC §7 classes AVC, HEVC, VVC and VP9 `StandardDefined` on the conformance clauses read, AV2 `StandardTolerance`, and EVC, AVS3, LCEVC, MPEG-H 3D Audio and AAC `StandardUnclassified`. The licence classes rest on the registry's recorded bases, not on this sweep. `glossary/codecs.json` is unchanged, so its grades still stand against §3.3 and §4 |
@ -691,7 +694,7 @@ review.
| G8 | **Standards inputs not tracked in the tree:** SyfAI; 3GPP TR 26.942; DVB S100 and CM-EE; the beyond-VVC CfP; Gaussian Splat Coding Exploration 45; IEEE 1857.11 / P1857.11.1; UWA 042; India D2M; the JPEG AI core experiments on energy consumption and bit-exact reconstruction. | a case-insensitive, word-bounded `git grep -P` over tracked `*.md`, `*.rs`, `*.json`, `*.toml`, `*.yml`, `*.yaml`, `*.html`, `*.ts` and `*.js` files finds none of these designations outside this record. The one other hit for `uwa` is the Hausa word in `aac/aac-rs/crates/aac-core/src/translations_data.json` (`"ha":"uwa"`), which is not the organisation. MPEG's splat work *is* tracked, though not Exploration 45 by name: `media-codec-registry.md` and `codecs/mod.rs` register `wai.splat.sog` against "MPEG WG4" work in progress, `3d-neural-coding-landscape.md` records splat coding under standardisation, and 2030 §6.4 records an NNC amendment covering Gaussian splats (G11) | **Partly closed by the two notes committed with this record.** [`syfai-input.md`](syfai-input.md) tracks SyfAI, IEEE 1857.11 and UWA 042; [`energy-exposure-mapping.md`](energy-exposure-mapping.md) tracks TR 26.942, DVB S100 and CM-EE. The same grep over the tree at link 10 finds the beyond-VVC CfP, Exploration 45, India D2M and the JPEG AI core experiments in no document but this record. Those four stay open |
| G9 | **WAI's texts and code class `wai.image.jpegai` in ways that contradict one another, and SPEC contradicts itself on it.** SPEC §5, the capability's registration (`SPEC.md`:197), describes the decode as "integer entropy front end (mANS) + integer dequant/value path + fixed-order synthesis and post-filters" and declares "**decode-equivalence**: one pinned reconstruction … reproducible on every machine (a portable decoded-pixel hash)". `reconstruction-binding.md` §1 (:30–31) says the decode to RGB is "byte-identical on every machine", and §2 (:72) sets the binding's criterion at `decode-equivalence`; `determinism-tiers.md` §2–§3 list the capability there. SPEC §7 lets a neural capability back `replicate` only "with no float operation anywhere" (:445–448). `capability_replicate_class` returns `StandardDefined` for it through the default arm, and SPEC §7 defines that class as "reconstruction is fixed by the named codec/standard … the neural float hazard does not apply"; neither half holds for JPEG AI (§2.2; its decode runs the model). The strict `replicate` check reads only the envelope's prior (`neural/mod.rs`:282–292). **Loose ends:** (i) `jpegai_synthesis.rs` and `jpegai_mans.rs` say they back `wai.neural.jpeg_ai`, a string registered nowhere; (ii) the decode has f32 steps between its integer stages — its dequantisation among them, although SPEC §5 calls it integer; (iii) the byte-for-byte claim is made normatively, in SPEC §5 and `reconstruction-binding.md` §1, and rests on an `exp()` from the platform maths library and on no measurement on a second platform; (iv) the reference sink's dispatch (`neural/mod.rs`:205–216) uses the exact decoder only when the integer weight bundle is present, and otherwise falls back to a float ONNX Runtime decode. | `SPEC.md` §5 (:197) and §7 (:445–448, the `StandardDefined` bullet at :476–478, :492–495); `extensions/reconstruction-binding.md` §1 (:21–35) and §2 (:72); `extensions/determinism-tiers.md` §2–§3; `wai-rs/src/codecs/mod.rs` (the default arm and the test asserting `StandardDefined`); `wai-rs/src/jpegai_decode.rs`, `jpegai_synthesis.rs`, `jpegai_mans.rs`, `jpegai_pipeline.rs`; `wai-rs/src/neural/mod.rs`:205–216, :282–292; `git grep -n decode-equivalence` over `wai/` | **Closed at link 02 (PR #103), with the maths library removed at link 03 (PR #104).** SPEC §5's row registers `entropy-consistency` (registry class `NeuralEntropyExact`), describes dequantisation, latent refinement, post-filters and colour conversion as IEEE-754 float with a fixed evaluation order, and says "**Downgraded in v0.4.0:** this row previously claimed decode-equivalence". `reconstruction-binding.md` §1–§2 make the pinned reconstruction a property of the exact decoder, reproducible under that decoder's build and floating-point conditions. SPEC §7 states the new class and the reason. Loose ends: (i) fixed — the module docs name `wai.image.jpegai`, and `wai.neural.jpeg_ai` appears nowhere else in the tree (grep); (ii) the float steps remain and are now stated as float; (iii) link 03 builds `exp` and rounding from IEEE-754 basic operations (`float_repro`), guards the decode path with clippy's `disallowed_methods` and a linked-symbol check (`tests/jpegai_decode_symbols.rs`), and pins nine hashes for one codestream; determinism-tiers §6 records them identical on macOS arm64, wasm32-wasip2, Linux arm64 and Linux x86_64 under user-mode emulation, and not measured on Windows, macOS x86_64 or x86_64 hardware; (iv) the float fallback remains, and SPEC §5, determinism-tiers §5 and reconstruction-binding §1 now state it, and that the result does not record which path ran. Still true: a `norm.json` with a missing or mistyped key panics in the loaders instead of falling back, as the `try_jpegai_exact` doc comment says |
| G10 | **WAI's texts give three conditions for a neural decode to back `replicate` or a portable reconstruction hash, and SPEC states all three.** (1) **All-integer:** SPEC §7 (`SPEC.md`:445–448, :492–495), enforced in code on the envelope's declared prior (`replicate_is_sound` requires `DeterminismClass::IntegerExact`, a substring heuristic over `Prior.determinism`). (2) **`decode-equivalence` with fixed-order float admitted:** `determinism-tiers.md` §2 defines the tier over "integer / fixed-order arithmetic end to end" (:57–58), and SPEC §5 registers `wai.image.jpegai`, whose float stages run in a fixed order, at that tier. (3) **`entropy-consistency` as the floor:** SPEC §7's closing paragraph (:497–505) calls it "the floor for a sound neural `replicate`", and `determinism-tiers.md` §2–§3 repeat it; §3 also says "a float neural decode meets neither tier", which does not square with the explicit-parameter route. A decode with integer entropy and fixed-order float synthesis meets (2) and (3) but not (1). | `SPEC.md` §5 (:197) and §7 (:445–448, :492–495, :497–505); `extensions/determinism-tiers.md` §1–§3; `extensions/reconstruction-binding.md` §1–§2; `wai-rs/src/container.rs` (`Prior::determinism_class` at :108–117, `is_cross_hardware_exact`, `replicate_is_sound`) | **Closed at link 02 (PR #103), on condition (1).** SPEC §7: a neural capability backs `replicate` only if its entire decode path is integer / fixed-point with no float operation, "the only construction by which a neural capability is registered at `decode-equivalence`"; `entropy-consistency` ("necessary for a sound neural `replicate` … and not sufficient") and a reconstruction pinned by one implementation's fixed-order float stages MUST NOT back it. determinism-tiers §2 defines `decode-equivalence` over "integer / fixed-point arithmetic end to end", and §3 says a decode that derives its entropy parameters in float meets neither tier while one that reads them from the codestream, or derives them in integer arithmetic, and runs later stages in float meets `entropy-consistency` only. The strict check reads the registry class before the prior. Remaining: the prior check still reads a sender-declared string by substring, which determinism-tiers §3 states |
| G11 | **Three volumetric registrations sit against MPEG's 2026 splat and point-cloud work, and none of the sweep's rows was mapped to them.** `wai.splat.sog` is a candidate "registered so a capability exists ahead of an MPEG freeze" (registry: "MPEG WG4 WIP", "RF (open work)"). `wai.pc.gpcc` (23090-9) and `wai.pc.vpcc` (23090-5) are registered POOL. None has a decoder inside `wai-rs`; the registry records demonstrated sink-side decodes via `tools/wai_carriage_demo.sh` (not re-run here, and not in CI). The sweep found a V-PCC Gaussian-splat CDAM (MPEG 153), a G-PCC Gaussian-splat Amd 1 at CDAM (MPEG 154), Enhanced G-PCC ISO/IEC 23090-38 at FDIS (MPEG 155), and Exploration 45's document series up to a call for content at MPEG 155, all P✓; its tracks and dates are **R**. Questions: **(a)** whether what MPEG standardises is what `wai.splat.sog` names, which cites Self-Organizing Gaussians / SOGS while the part page names no candidate technology; **(b)** whether its royalty class holds (`capability_license` returns `RoyaltyFree` through the default arm; nothing the sweep read establishes the licensing of a codec MPEG selects); **(c)** whether 23090-38 is a payload `wai.pc.gpcc` admits or needs its own capability, and whether the splat amendments change what the two PCC capabilities admit; **(d)** the replicate class: all three fall through to `StandardDefined`, although `wai.splat.sog` has no frozen standard. The tree's "WG4" attribution was not checked (the Exploration 45 page names no working group), and `3d-neural-coding-landscape.md` describes a "future `wai.pointcloud.*`" without mentioning the registered `wai.pc.*` strings. | `research/media-codec-registry.md`; `wai-rs/src/codecs/mod.rs` (`CAP_PC_GPCC`, `CAP_PC_VPCC`, `CAP_SPLAT_SOG` and their arms); `tools/wai_carriage_demo.sh`; `wai-rs/tests/carriage.rs`; `research/3d-neural-coding-landscape.md`; 2030 §6.4 | **(d) closed at link 02 (PR #103); (b) made explicit at link 04 (PR #105); (a), (c), the "WG4" attribution and the `wai.pointcloud.*` sentence open.** All three are `StandardUnclassified`, the class for a standard whose decoder-conformance clauses have not been checked (SPEC §7 notes that `wai.splat.sog` names no frozen standard yet). `capability_license` lists `wai.splat.sog` explicitly as royalty-free ("SOG open work"), which makes the class explicit and does not establish it. `codecs/mod.rs` and `media-codec-registry.md` still say "MPEG WG4", and `3d-neural-coding-landscape.md` still describes a "future `wai.pointcloud.*`". Exploration 45 is untracked (G8) |
| G11 | **Three volumetric registrations sit against MPEG's 2026 splat and point-cloud work, and none of the sweep's rows was mapped to them.** `wai.splat.sog` is a candidate "registered so a capability exists ahead of an MPEG freeze" (registry: "MPEG WG4 WIP", "RF (open work)"). `wai.pc.gpcc` (23090-9) and `wai.pc.vpcc` (23090-5) are registered POOL. None has a decoder inside `wai-rs`; the registry records demonstrated sink-side decodes via `tools/wai_carriage_demo.sh` (not re-run here, and not in CI). The sweep found a V-PCC Gaussian-splat CDAM (MPEG 153) *(2026-09-30: at Draft Amendment since MPEG 154, WG 4 N 837)*, a G-PCC Gaussian-splat Amd 1 at CDAM (MPEG 154), Enhanced G-PCC ISO/IEC 23090-38 at FDIS (MPEG 155), and Exploration 45's document series up to a call for content at MPEG 155, all P✓; its tracks and dates are **R**. Questions: **(a)** whether what MPEG standardises is what `wai.splat.sog` names, which cites Self-Organizing Gaussians / SOGS while the part page names no candidate technology; **(b)** whether its royalty class holds (`capability_license` returns `RoyaltyFree` through the default arm; nothing the sweep read establishes the licensing of a codec MPEG selects); **(c)** whether 23090-38 is a payload `wai.pc.gpcc` admits or needs its own capability, and whether the splat amendments change what the two PCC capabilities admit; **(d)** the replicate class: all three fall through to `StandardDefined`, although `wai.splat.sog` has no frozen standard. The tree's "WG4" attribution was not checked (the Exploration 45 page names no working group), and `3d-neural-coding-landscape.md` describes a "future `wai.pointcloud.*`" without mentioning the registered `wai.pc.*` strings. | `research/media-codec-registry.md`; `wai-rs/src/codecs/mod.rs` (`CAP_PC_GPCC`, `CAP_PC_VPCC`, `CAP_SPLAT_SOG` and their arms); `tools/wai_carriage_demo.sh`; `wai-rs/tests/carriage.rs`; `research/3d-neural-coding-landscape.md`; 2030 §6.4 | **(d) closed at link 02 (PR #103); (b) made explicit at link 04 (PR #105); (a), (c), the "WG4" attribution and the `wai.pointcloud.*` sentence open.** All three are `StandardUnclassified`, the class for a standard whose decoder-conformance clauses have not been checked (SPEC §7 notes that `wai.splat.sog` names no frozen standard yet). `capability_license` lists `wai.splat.sog` explicitly as royalty-free ("SOG open work"), which makes the class explicit and does not establish it. *(2026-09-30: `capability_license` now classifies `wai.splat.sog` patent-pool, conservatively and provisionally while it is a candidate; SPEC §5, Licence classes)* `codecs/mod.rs` and `media-codec-registry.md` still say "MPEG WG4", and `3d-neural-coding-landscape.md` still describes a "future `wai.pointcloud.*`". Exploration 45 is untracked (G8) |
**Found but assigned to no link.** These are recorded so they are not lost:
@ -782,6 +785,7 @@ when that document is next revised.
| Mar 2027 | LOC, the streaming format and Privacy Pass authentication to IESG (WG milestone) | P✓ |
| 19–23 Apr 2027 | MPEG 158, Geneva | Part 6 CD target per `mpeg-ai-part6-mapping.md` (tree). Dates P✓ |
| 15 Aug 2027 | EU data-centre label generated from this date (C(2026) 3472 Art. 3) | P✓ (§5.2) |
| before 15 Oct 2026 | Gaussian splat coding call for content: test material wanted for MPEG 156 *(2026-09-30 addition, WG 5 N 422)* | P✓ |
| 2027 | Gaussian Splat Coding codec selection | **R** ⚠ |
| 31 Dec 2028 | EU review: ICT-equipment energy indicators; (partial) certification or auditing of reported data (C(2026) 3472 Art. 6) | P✓ (§5.2) |
| end of 2029 | beyond-VVC first version targeted | P✓ |

20
wai/research/2030-transport-landscape.md
View file

@ -73,7 +73,7 @@ network payload — and, in the 01/2026 edition, an in-band **prompt** and seed
into that network's input tensor. Beside the pixels: MPEG-AI Part 2 codes video
*and descriptors* jointly, and Part 4 codes no pixels at all, shipping an
intermediate feature tensor from the middle of a network. Instruction-carriage
is not a differentiator any more. What remains unclaimed is the *contract over
is specified at each of those layers. What WAI specifies is the *contract over
the instruction*.
**1.2 "Correct" is forking into several incompatible contracts.** Byte
@ -313,15 +313,14 @@ taken from inside a machine-vision network: the origin runs the first N layers,
ships the tensor, the sink runs the remainder. No pixels are reconstructed at any
point.
**Vendor-defined "ML-Safe" compression.** Origin: Beamr Imaging Ltd.; **stage:
**Vendor-defined "ML-Safe" compression.** Origin: [name removed 2026-09-30]; **stage:
vendor framing, no standards body, no designation** (blog post, 29 July 2026,
scoped in the source to autonomous-vehicle camera and synthetic footage).
Compression is called ML-Safe when the change it induces in a downstream model's
behaviour is statistically indistinguishable from the variance that model
already tolerates. What is *deployed* is the underlying content-adaptive rate
control; what is *proposed* is the fidelity framing — the source itself states
the industry "lacks a shared framework," which is the strongest available
evidence that the seam is unclaimed.
the industry "lacks a shared framework."
**What all three do to WAI.** `determinism-tiers.md` admits only
byte-comparable tiers, and `ReplicateClass::Behavioral` is a disclaimer rather
@ -584,7 +583,9 @@ precedent and the register discipline, not novel machinery.
### 4.4 The streaming-format slot, and how CMSF proves it is open
MoQ separates **transport** (`draft-ietf-moq-transport`, rev 18, approaching WG
Last Call) from **streaming format** (`draft-ietf-moq-msf`). MOQT moves objects;
Last Call) *(2026-09-30: the current revision is draft-ietf-moq-transport-21,
published 2026-09-08, a WG Document; the working group's milestone for sending it
to the IESG is 2026-12-31)* from **streaming format** (`draft-ietf-moq-msf`). MOQT moves objects;
MSF says what the objects *mean* — it fragments a bitstream into objects, uses a
**catalog** to declare what a publisher emits, and sets prioritisation.
@ -628,8 +629,7 @@ transport rather than a normative extension that pins it.
`draft-ietf-moq-msf` and `draft-ietf-moq-cmsf-00`. The automated survey did not
surface the streaming-format layer at all — it found MoQ only through the
confidentiality binding WAI already has — which is itself worth noting about the
method: a sweep keyed to *what is new* can miss a slot that is old, open, and
unclaimed.
method: a sweep keyed to *what is new* can miss a slot that is old and open.
### 4.5 What must NOT be bound
@ -816,7 +816,7 @@ integrated — so cite it as *"the NNC amendment (CDAM, MPEG 154)"*, not "Ed. 3"
Reference software: specified by a separate part, ISO/IEC 15938-18 (*Conformance
and reference software for compression of neural networks*; 1st ed. 2023, 2nd
ed. 2025), which also carries the conformance bitstreams and the decoder and
bitstream test procedures. NNCodec (Fraunhofer HHI) is a separate open
bitstream test procedures. NNCodec ([name removed 2026-09-30]) is a separate open
implementation: its README calls it "an efficient implementation of NNC" and
states that 2.0-beta is compliant with 15938-17:2024, and its licence file states
that no patent rights are granted. Describe NNC as the **first** international
@ -861,7 +861,7 @@ The **base standard is confirmed**: ISO/IEC 15938-17 (1st ed. 2022, 2nd ed.
2024), *Compression of neural networks for multimedia content description and
analysis* — DeepCABAC entropy coding, pruning/sparsification/low-rank
preprocessing, incremental-update support, with reference software in
ISO/IEC 15938-18 and a separate open implementation (Fraunhofer HHI, NNCodec).
ISO/IEC 15938-18 and a separate open implementation (NNCodec) [name removed 2026-09-30].
The claim that needs no amendment at all is the carriage one: NNC is a ratified
wire format for the artifact a `Prior` names, which WAI currently leaves as an
informal directory. The signing feature is a second surface. It concerns how
@ -1057,7 +1057,7 @@ Items 1–2 are internal defects that need no external dependency at all.
| 11 | **NMOS constraint-set grammar over the capability registry** (§4.3) | a sink-side precondition on §4 dispatch; author register entries for the non-broadcast families | BCP-004-01/-02 v1.0.0 (ratified) | makes §7.1 rendition selection evaluable rather than policy-deferred |
| 12 | **Register the Part 16 derivation operations with a per-operation determinism tier** (§3.1) | capability strings + tiers; do not adopt compositing | ISO/IEC 23001-16:2021 (published) | prevents restating a 2021 ratification as a WAI invention |
| 13 | **MPEG-AI Part 6 requirement mapping + SC 29 membership path** (§6.5) | map WAI's measured-energy binding and determinism tiers onto the CfP requirements | CfP responses due **13 January 2027** | the only square in this survey with a deadline |
| 13b | **MoQ streaming-format binding** — a sibling I-D declaring `packaging: "wai"`, plus a per-object MSF Property carrying the JWP content commitment and measured joules (§4.4) | one draft; inherit MSF; **no IANA action** (CMSF precedent) | `draft-ietf-moq-msf` / `-cmsf`; MOQT pre-Last-Call, so track rather than pin | the per-object property slot is empty, the receipt is already per-object, and the transport is the one Akamai-class delivery is converging on |
| 13b | **MoQ streaming-format binding** — a sibling I-D declaring `packaging: "wai"`, plus a per-object MSF Property carrying the JWP content commitment and measured joules (§4.4) | one draft; inherit MSF; **no IANA action** (CMSF precedent) | `draft-ietf-moq-msf` / `-cmsf`; MOQT pre-Last-Call, so track rather than pin | the per-object property slot is empty, the receipt is already per-object, and the transport is the one [name removed 2026-09-30]-class delivery is converging on |
| 14 | **Watch list** | relay geocode (unadopted I-D, data model only); FCM cut-point receipt; the VCM conformance suite when it issues; ISO/IEC 23001-11 4th ed. and 23001-19 catalogue confirmation | — | none is a stable surface yet |
---

84
wai/research/3d-neural-coding-landscape.md
View file

@ -13,18 +13,35 @@
The learned image-codec machinery is being reused one dimension up:
- **JPEG Pleno Learning-based Point Cloud Coding** is finalized, and it **reuses
JPEG AI** to code the point cloud's colour attributes — the same learned-2D
transform, applied to attribute images.
- **Gaussian Splat Coding** is under standardization (MPEG WG working groups,
JPEG), applying **Neural Network Coding (NNC)** to the splat parameters —
spherical-harmonic coefficients, positions, scales.
- **JPEG Pleno Learning-based Point Cloud Coding** is reported to reuse JPEG AI to
code the point cloud's colour attributes — the same learned 2-D transform,
applied to attribute images. *(Not verified here: the tree records no source for
this, and none was read for this note's 2026-09-30 revision.)*
- **Gaussian splat coding** is being standardized in MPEG along two paths, per
MPEG's pages and public output documents read on 2026-09-30 (listed in
[`media-codec-registry.md`](media-codec-registry.md), *MPEG's Gaussian-splat and
point-cloud work*): amendments that add Gaussian splats to the two point-cloud
standards, V-PCC (ISO/IEC 23090-5, at Draft Amendment) and G-PCC (ISO/IEC
23090-9, at Committee Draft Amendment); and a joint exploration (Exploration 45),
coordinated across WG 4, WG 5/JVET and WG 7 of ISO/IEC JTC 1/SC 29, that is
preparing a call for proposals for content in the I-3DGS representation.
Separately, a Committee Draft amendment of **Neural Network Coding (NNC)**
extends NNC to "parameters of 3D and 4D scene representations such as Gaussian
splats" ([2030 landscape](2030-transport-landscape.md) §6.4). No source read here
establishes a JPEG activity on splat coding.
- **AI-based point cloud coding** is MPEG-AI Part 5 (ISO/IEC 23888-5); its part
page lists common test conditions at MPEG 155
([2026-09 sweep](2026-09-global-sweep.md) §2.1).
Both inherit the property that made the flat learned codecs hard to deploy: a
learned decode is not reproducible across hardware. A float attribute/coefficient
decode drifts by round-off, the entropy stage desyncs, and — exactly as for 2D —
the reconstruction is reproducible only on the same build. The determinism
problem, and its fix, are the same shape in 3D.
Where these paths use a learned decode, they inherit the property that made the
flat learned codecs hard to deploy: a float decode drifts by round-off, the
entropy stage desyncs, and the reconstruction is reproducible only on the same
build. Whether a given path's decode is learned is not established here: the
amendment texts are not public, and the call for proposals has selected no
technology. The V-PCC amendment specifies a reconstruction profile because
"without a clearly specified reconstruction profile, different implementations
could decode the same bitstream into different Gaussian Splat parameters"
(ISO/IEC JTC 1/SC 29/WG 4 N 837).
## What WAI already has
@ -39,7 +56,7 @@ the contract.
That is the 3D instance of the same thesis WAI applies to its integer image
codecs (`wai.neural.int_hyper`): the learned decode to the medium's canonical
representation is pinned and integer, so the reconstruction has a portable hash. `wai.splat.int_codec` already sits at the
**`decode-equivalence`** tier (see [`extensions/determinism-tiers.md`]) — the
**`decode-equivalence`** tier (see [`extensions/determinism-tiers.md`](../extensions/determinism-tiers.md)) — the
attribute tensor is the "final medium" whose bytes reproduce.
## Where it connects — and where it doesn't
@ -52,26 +69,33 @@ attribute tensor is the "final medium" whose bytes reproduce.
provenance binding neural-coded media could not carry becomes available for 3D
too. As the 3D standards adopt learned coding, the determinism-for-provenance gap
is the same one WAI closes for 2D.
- **Honest scope — WAI does not decode the ISO point-cloud bitstream.**
`wai.splat.int_codec` is a WAI-native integer codec, not the JPEG-Pleno PCC or
the MPEG splat-coding stream. The shared thing is the *problem and the approach*
(a learned 3D decode that needs to be reproducible), not the wire format. A
future `wai.pointcloud.*` that byte-exactly decodes a published 3D standard's
codestream would be the 3D analogue of what WAI's exact JPEG AI decoder is for
JPEG AI — and, because JPEG-Pleno PCC reuses JPEG AI for its colour attributes,
that decoder (integer entropy decode and synthesis networks;
`wai.image.jpegai` itself is registered at `entropy-consistency`, because its
later stages are float) is a component that path could build on. Not built;
recorded as the natural next capability if the 3D neural-coding standards land
in scope.
- **Scope — WAI does not decode the ISO/IEC point-cloud bitstreams.**
`wai.splat.int_codec` is a WAI-native integer codec, not a JPEG Pleno or MPEG
bitstream; the shared thing is the *problem and the approach*, not the wire
format. WAI already registers MPEG's point-cloud standards as sink-supplied
capabilities — `wai.pc.gpcc` (G-PCC, ISO/IEC 23090-9) and `wai.pc.vpcc` (V-PCC,
ISO/IEC 23090-5), replicate class `StandardUnclassified`, with no decoder in
`wai-rs` — and the research registry records a sink-side G-PCC decode and a V-PCC
encode with a structural V3C parse through the MPEG reference tools. A decoder in
WAI that reproduces a registered standard's reconstruction byte for byte would
decode that capability's existing payload, as WAI's exact JPEG AI decoder decodes
`wai.image.jpegai`; it adds no capability string. A 3D standard WAI does not
register yet would need a string of its own (SPEC §5, §8); which editions and
amendments the two `wai.pc.*` strings carry follows SPEC §8's edition rule, and
what remains open is recorded in [`media-codec-registry.md`](media-codec-registry.md),
open item (i). If JPEG Pleno's learning-based coding reuses JPEG AI for colour
attributes (unverified, above), WAI's JPEG AI decoder — integer entropy decode
and synthesis networks, with `wai.image.jpegai` registered at
`entropy-consistency` because its later stages are float — is a component that
path could build on. Not built.
## Takeaway
The determinism contract is not image-specific. WAI's byte-exact family already
spans image (`int_synth`, `int_hyper`), audio (`int_codec`), and 3D
(`splat.int_codec`), with `jpegai` at `entropy-consistency` beside them; the
reconstruction binding and the tier vocabulary apply to all of them. The emerging 3D neural-coding standards face the same
non-determinism WAI addresses for 2D, and the JPEG-AI decode WAI has is a
component of the point-cloud path — so the ingest is: keep the determinism
The determinism contract is not image-specific. WAI's byte-exact family — the
registry's `NeuralIntegerExact` class (`codecs::capability_replicate_class`) —
already spans image, video, audio and 3D, with `jpegai` at `entropy-consistency`
beside it; the reconstruction binding and the tier vocabulary apply to all of
them. The emerging 3D coding standards that adopt a learned decode face the same
non-determinism WAI addresses for 2D, so the ingest is: keep the determinism
contract media-general, and treat a byte-exact decode of a published 3D standard
as the natural next capability, not a separate track.

41
wai/research/cargo-roadmap.md
View file

@ -4,12 +4,11 @@
> state-feeds, avatars, and volumetric/4D splats all shipped, promoted to
> `extensions/`, and live. This document is now the archived plan of
> record — see `CHANGELOG.md` for current state and `content-taxonomy.md`
> for what remains white space (e.g. music-as-score).
> for each content class's status.
>
> The build plan for the white-space content classes in
> `content-taxonomy.md`. Every phase follows the **same proven recipe**
> the interactive-worlds extension established — so this is execution,
> not invention.
> The build plan for the content classes phased in `content-taxonomy.md`.
> Every phase follows the **same recipe** the interactive-worlds extension
> established.
## The recipe (every cargo class follows it)
@ -61,8 +60,8 @@ assets · PBR + EWA-splat renderers · native energy meter. Siblings:
## Phase 1 — Object-based spatial audio · `wai.audio.scene`
**Why first:** the cleanest technical analog to interactive-worlds.
Ship sound *objects* + a scene; the sink renders to its own speaker
**Why first:** it is interactive-worlds with the tick replaced by the
audio sample. Ship sound *objects* + a scene; the sink renders to its own speaker
layout. Mixdown-equivalence is the audio counterpart of
replay-equivalence, and it composes the asset path (each source is a
content-addressed `wai.audio.*` object) + receipts + determinism.
@ -161,8 +160,8 @@ a special case of replay-equivalence.
## Phase 5 — Volumetric / light-field · `wai.splat.4d` (temporal Gaussians)
**Why last:** the frontier of "video"; no clean envelope exists. Extends
the splat path to time.
**Why last:** it is the heaviest payload of the five. Extends the splat
path to time.
**Payload:** a base Gaussian cloud + per-frame deltas, OR per-Gaussian
trajectories (position/rotation/scale/opacity keyframes). Sink
@ -171,23 +170,25 @@ interpolates and renders with the existing EWA pipeline.
**Conformance:** frame-state-equivalence — the Gaussian set at frame N
hashes identically (presentation render stays the renderer's business).
**Effort:** high. **Risk:** size (volumetric is heavy — the win is
content-addressing + dedup + receipts, *not* payload reduction; state
that honestly, as with static splats).
**Effort:** high. **Risk:** size (volumetric is heavy — what the class
carries is content-addressing + dedup + receipts, *not* payload
reduction; state that plainly, as with static splats).
---
## Sequencing rationale
1. **Spatial audio** — cleanest analog, highest "proves the thesis
generalizes" value, composes the asset path.
2. **Haptics** — fast win, completes the "every sense" story (sight via
worlds/splats, hearing via audio-scene, touch via haptics).
3. **State-feeds** — biggest commercial wedge, mostly already built.
4. **Digital humans** — high value, composes worlds + mesh.
5. **Volumetric** — frontier, heaviest, do last.
1. **Spatial audio** — reuses the worlds floor, seal and receipt profile
verbatim with a mixer as the sim, and composes the asset path.
2. **Haptics** — a small payload and a pure recipe; with it, sight
(worlds/splats), hearing (audio-scene) and touch (haptics) each have a
class.
3. **State-feeds** — the broadcast profile of `wai.world.live`; most of it
was already built.
4. **Digital humans** — composes worlds + mesh.
5. **Volumetric** — the heaviest payload; built last.
Cross-cutting, anytime: surface **content-addressing + JWP receipts +
measured joules** on the captured-reality classes (photos, recordings,
telemetry) — the lever-4 win that needs no reconstruction and applies to
telemetry) — lever 4, which needs no reconstruction and applies to
the content that can't be reduced to instructions.

42
wai/research/content-taxonomy.md
View file

@ -15,16 +15,16 @@ captured reality ──► synthesizable ──► executable ──► sy
```
**WAI's leverage is proportional to how much of the content the sink can
reconstruct instead of receiving pre-rendered.** That's the whole map.
reconstruct instead of receiving pre-rendered.**
The discipline it enforces: do *not* claim "everything becomes
instructions" — a real photograph is irreducibly samples. Different
position on the axis → different lever → different size of win.
position on the axis → a different lever, and a different gain.
## 2. The four levers
| # | Lever | What it does | Where it applies |
|---|---|---|---|
| 1 | **Codec dispatch** | name the SOTA royalty-free codec; sink has it or falls back | all media |
| 1 | **Codec dispatch** | name a registered codec; the sink decodes it or falls back | all media |
| 2 | **Neural shared-prior** | ship conditioning, regenerate against a requirement-of-existence model | perceptual media a model can reconstruct |
| 3 | **Instructions-at-the-sink** | ship the recipe / state / intent; the sink renders | anything whose payload is secretly a rendered result |
| 4 | **Determinism + receipts** | replay/decode/mixdown-equivalence, content-addressing, JWP signatures, joules | **cross-cutting** — applies even to content that can't be reconstructed |
@ -60,7 +60,7 @@ Status: ✅ shipped · 🟡 partial/composable from shipped pieces · ⬜ white
| Procedural / game audio, soundscapes | instructions-at-sink | spatial-audio (`wai.audio.scene`) | 🟡 Phase 1 |
| Text-to-speech | neural prior | ship text + a voice prior (`wai.neural.*` speech) | 🟡 |
| Data visualizations / charts | instructions-at-sink | ship data + spec (joule-ui) | 🟡 |
| Diagrams, vector art, **fonts** | instructions-at-sink | the precedents that already won the web; envelope only | ✅ (precedent) |
| Diagrams, vector art, **fonts** | instructions-at-sink | formats the web already carries as instructions; envelope only | ✅ (precedent) |
| **Maps / geospatial** | instructions-at-sink | `map` standard | ✅ (sibling) |
### C. Interactive / executable — it IS a program (levers 3, 4)
@ -86,16 +86,18 @@ Status: ✅ shipped · 🟡 partial/composable from shipped pieces · ⬜ white
| Datasets / tabular / scientific data / knowledge graphs | content-addressing + receipts | `wai.text.*` + JWP; `JouleDB` lineage | 🟡 |
| Notifications / messages / event-&-state feeds | instructions + receipts | `wai.feed.state` (broadcast profile, replay-equivalence) | ✅ |
### The under-the-radar classes (flagged white space)
### The cargo-roadmap phases
| Content | Why it matters | Plan |
Five classes were built as phases of `cargo-roadmap.md`; each has shipped.
| Content | What it shares with the thesis | Phase and status |
|---|---|---|
| **Object-based / spatial audio** | the *exact audio analog* of the video thesis: ship sound objects + a scene, render to the listener's speaker config — the cleanest technical fit | **Phase 1** (`wai.audio.scene`) |
| **Haptics** | instruction streams to actuators; tiny payloads, pure recipe, deterministic | **Phase 2** (`wai.haptic.signal`) |
| **Live state-feeds** | sports/events as *state*, not pixels — the multiplayer-games precedent applied to broadcast; a top commercial wedge | **Phase 3** (broadcast profile of `wai.world.live` + `wai.feed.state`) |
| **Digital humans / avatars** | the highest-bandwidth capture today, the most reducible to rig + stream — "personal at broadcast scale" | **Phase 4** |
| **Volumetric / light fields** | where "video" goes next; no clean envelope exists for it yet | **Phase 5** |
| **Full-sensory / olfactory** | the holodeck endgame | on the axis; not scheduled |
| **Object-based / spatial audio** | the audio form of the video thesis: ship sound objects + a scene, render to the listener's speaker config; mixdown-equivalence is the audio counterpart of replay-equivalence | **Phase 1**, shipped (`wai.audio.scene`) |
| **Haptics** | instruction streams to actuators; tiny payloads, pure recipe, deterministic | **Phase 2**, shipped (`wai.haptic.signal`) |
| **Live state-feeds** | sports/events as *state*, not pixels — the multiplayer-games precedent applied to broadcast | **Phase 3**, shipped (`wai.feed.state`, the broadcast profile of `wai.world.live`) |
| **Digital humans / avatars** | a high-bandwidth capture that reduces to a rig plus a pose stream | **Phase 4**, shipped (`wai.avatar.pose`) |
| **Volumetric / light fields** | the splat path extended in time; frame-state-equivalence | **Phase 5**, shipped (`wai.splat.4d`) |
| **Full-sensory / olfactory** | the far end of the axis | on the axis; not scheduled |
## 4. The one sentence to navigate by
@ -107,13 +109,13 @@ Status: ✅ shipped · 🟡 partial/composable from shipped pieces · ⬜ white
## 5. Reading the map
- The **biggest commercial wedges**: animation-film (a sealed
camera-on-rails world) and live state-feeds.
- The **cleanest technical analog** to what's already proven:
object-based spatial audio (mixdown-equivalence is the audio
counterpart of replay-equivalence).
- The **cross-cutting win** that needs no reconstruction at all:
content-addressing + JWP receipts + measured joules on *every* class,
including raw captured reality.
- Lever 3 applies most directly to animation film (a sealed
camera-on-rails world) and to live state-feeds.
- Object-based spatial audio carries the same equivalence rule to
audio: mixdown-equivalence is the audio counterpart of
replay-equivalence.
- Lever 4 needs no reconstruction at all: content-addressing, JWP
receipts and measured joules apply to *every* class, including raw
captured reality.
See `cargo-roadmap.md` for the phased build plan.

2
wai/research/decode-throughput.md
View file

@ -289,7 +289,7 @@ What was checked (the claim rests on these, not on the argument alone):
## 3. Method
- **Machine:** Apple M5 Max, 18 logical cores, macOS 27.0; rustc 1.98.0. macOS
- **Machine:** an arm64 macOS laptop [model removed 2026-09-30], 18 logical cores, macOS 27.0; rustc 1.98.0. macOS
reports the cores as two performance levels (`hw.perflevelN.name`): 6 "Super"
and 12 "Performance". It reports no efficiency cores. The harness prints the
levels as the OS names them.

132
wai/research/media-codec-registry.md
View file

@ -31,8 +31,8 @@ registry — deliberately **not** a per-file manifest claim, because a codec's
licence can't be honestly asserted in an envelope's own header. A deployer
audits a title or a whole catalogue in code: "encode only to royalty-free
capabilities", or "flag every pool/proprietary capability in this library."
That is the *per-title toll-road* choice made into a property of the format
rather than a slide. `wai capabilities` prints the class per row (the registry
That makes the per-title licensing choice a property of the format, checkable
in code. `wai capabilities` prints the class per row (the registry
reports the royalty-free count outright).
The classification is **exhaustive, with no default class**. Every registered
@ -83,8 +83,8 @@ given, or a source outside `wai.audio.*`). It covers the output the standard or
codec the cropped decoded pictures, before colour conversion — not a conversion a
sink applies afterwards.
**Measured** with `tools/wai_replicate_probe.sh` on an Apple M5 Max (arm64,
macOS 27.0) with ffmpeg 9.0.2, September 2026. Each format was encoded from a
**Measured** with `tools/wai_replicate_probe.sh` on an arm64 macOS laptop (18
logical cores, macOS 27.0) with ffmpeg 9.0.2, September 2026. Each format was encoded from a
synthetic source: 20 frames of 320×240 `testsrc2` video, a 1 s 440 Hz 48 kHz
mono tone, and a 320×240 grayscale still, so that no colour conversion enters
the comparison. Each was decoded by every independent decoder present, and
@ -122,26 +122,25 @@ say whether that difference is within its criterion.
| capability | standard | royalty | reference | tier |
|---|---|---|---|---|
| `wai.audio.iamf` | IAMF / "Eclipsa Audio" (AOMedia Immersive Audio Model & Formats) | **RF** | libiamf (Apache-2.0) | Lossless¹ |
| `wai.audio.ac4` | Dolby AC-4 (codec under the Atmos umbrella) | **PROP** | Dolby | Lossy |
| `wai.audio.iamf` | IAMF (AOMedia Immersive Audio Model & Formats) | **RF** | libiamf (Apache-2.0) | Lossless¹ |
| `wai.audio.ac4` | AC-4 (ETSI TS 103 190) | **PROP** | single licensor | Lossy |
| `wai.audio.ivas` | 3GPP IVAS (Immersive Voice & Audio Services, EVS-based) | **POOL**² | 3GPP | Lossy |
The floor (`wai.audio.{opus,flac}`) + the deterministic object mix
(`wai.audio.scene`) cover stereo and the bit-exact mixdown; these add the
field's standard *immersive carriers*. **IAMF is the pick** — open,
royalty-free, AV1-lineage, and codec-agnostic (it carries Opus / FLAC / LPCM
for channel, scene/ambisonic, and binaural audio), which fits "name the
capability" exactly. AC-4 is registered as a license-flagged slot so marquee
object-audio content can ride *inside* a WAI envelope without forcing the brand
out. IVAS is the conversational/telepresence immersive codec.
standard *immersive carriers*. IAMF is open, royalty-free, from the Alliance
for Open Media, and codec-agnostic (it carries Opus / FLAC / LPCM for channel,
scene/ambisonic, and binaural audio), which fits "name the capability"
directly. AC-4 is registered as a licence-flagged slot so object-audio content
already encoded in it can ride inside a WAI envelope. IVAS is the
conversational/telepresence immersive codec.
¹ IAMF can carry lossless codecs (FLAC/LPCM) → lossless ceiling. **Verified
RF**: published under the AOMedia Patent License 1.0 ("no-charge, royalty-free,
irrevocable"); reference under BSD-3-Clause-Clear. (Adopted by YouTube +
Samsung 2025.)
² **Verified POOL**: IVAS is in Via Licensing Alliance's Voice Codec pool
(launched Dec 2024, alongside EVS; Fraunhofer / NTT / Ericsson / Panasonic et
al.) — **not** royalty-free.
irrevocable"); reference under BSD-3-Clause-Clear. (Deployed in streaming and
consumer devices in 2025.)
² **Verified POOL**: IVAS is in a voice-codec patent pool (launched Dec 2024,
alongside EVS) — **not** royalty-free.
## Cinema mastering / intra-frame mezzanine
@ -150,9 +149,9 @@ al.) — **not** royalty-free.
| `wai.image.jp2` | JPEG 2000 still (ISO/IEC 15444-1; DCP via SMPTE ST 429 / MXF) | RF core³ | OpenJPEG (BSD-2) | Lossless |
| `wai.video.j2k` | Intra-frame JPEG 2000 video (each frame an independent codestream) | RF core³ | OpenJPEG | Lossless |
The theatrical interchange path: a DCP is JPEG 2000 in MXF. ProRes / DNxHR are
the production mezzanines but proprietary, so out of scope for a royalty-free
floor (a license-flagged slot could be added if a workflow needs it).
The theatrical interchange path: a DCP is JPEG 2000 in MXF. The proprietary
production mezzanine formats are out of scope for a royalty-free floor (a
licence-flagged slot could be added if a workflow needs one).
³ **Verified RF (Part 1)**: JPEG 2000 Part 1 (the core coding system) is
licensable fee-free under ITU-T patent policy 2.1 — 20+ organisations agreed to
@ -165,20 +164,70 @@ the DCP/mezzanine Part-1 path; don't extend the RF claim to Part 2+.
| capability | standard | royalty | reference | tier |
|---|---|---|---|---|
| `wai.mesh.draco` | Draco (Google) mesh + point-cloud geometry | **RF** (Apache-2.0) | draco | Lossless⁴ |
| `wai.pc.gpcc` | MPEG G-PCC (ISO/IEC 23090-9), sparse / LiDAR clouds | **POOL** | mpeg-pcc-tmc13 | Lossy |
| `wai.pc.vpcc` | MPEG V-PCC (ISO/IEC 23090-5), dense / dynamic clouds | **POOL** | mpeg-pcc-tmc2 | Lossy |
| `wai.splat.sog` | Gaussian-splat compression (Self-Organizing Gaussians; MPEG WG4 WIP) | RF (open work) | SOG / SOGS | Lossy |
| `wai.mesh.draco` | Draco (open-source library) mesh + point-cloud geometry | **RF** (Apache-2.0) | draco | Lossless⁴ |
| `wai.pc.gpcc` | MPEG G-PCC (ISO/IEC 23090-9), sparse / LiDAR clouds | **POOL**¹⁴ | mpeg-pcc-tmc13 | Lossy |
| `wai.pc.vpcc` | MPEG V-PCC (ISO/IEC 23090-5), dense / dynamic clouds | **POOL**¹⁴ | mpeg-pcc-tmc2 | Lossy |
| `wai.splat.sog` | Gaussian-splat compression — a **candidate**: no payload format pinned (registered for the Self-Organizing Gaussians family, SOG / SOGS) | POOL, provisional: classified conservatively, since the licensing of a format not yet pinned is not established | none cited | Lossy |
These compress the heavy asset clouds the existing capabilities already ship
(`wai.mesh.glb`, `wai.splat.spz`, `wai.splat.4d`). `wai.splat.sog` is a
**candidate** — registered so a capability exists ahead of an MPEG freeze; its
payload format pins when an open reference firms up (the same "track until the
license/anchor firms" discipline used for the codec newcomers).
**candidate**. The string was registered in June 2026 for the Self-Organizing
Gaussians family of splat formats before any payload format was pinned, and none
is pinned yet, so SPEC §5 (*Candidate capabilities*) makes it unemittable: an
encoder MUST NOT emit it and a sink treats it as unsupported. It pins when a
revision cites a published specification of the format and states its framing.
It was registered royalty-free, but no licence of a format specification or of
reference code was read, so that class was never established; it is now
classified patent-pool, conservatively and provisionally, until a revision pins
its payload format and the format's licensing is read. No
MPEG page or document read for this registry names Self-Organizing Gaussians, SOG
or SOGS (*MPEG's Gaussian-splat and point-cloud work*, below).
⁴ Draco mesh geometry has a lossless mode (exact-geometry ceiling, like
`wai.mesh.glb`); its quantization knobs are the encoder's choice.
¹⁴ No licensing programme for G-PCC or V-PCC was read. Both are MPEG standards
with patent declarations, and the class is a conservative classification, as
footnote ⁵ records for MIHS: not asserted royalty-free.
### MPEG's Gaussian-splat and point-cloud work (read 2026-09-30)
What MPEG's public pages and public output documents said on 2026-09-30, for the
three volumetric registrations above. The last column says what was read. The
meeting pages were read through a tool that returns an extraction of each page;
the output documents are the public files the Exploration 45 page links,
downloaded from mpeg.org and read in full. The draft call for proposals (ISO/IEC
JTC 1/SC 29/WG 4 N 836) and the amendment texts (WG 7 N 1453 and N 1454) are not
public (HTTP 404), and iso.org refused automated reads (HTTP 403), so no ISO
catalogue stage or edition is confirmed here.
| item | stage and what the source states | read |
|---|---|---|
| Exploration 45, *Gaussian splat coding* (edition 1, "Implicit Neural Visual Representation") | "ongoing"; documents run from MPEG 150 to MPEG 155; the page names no working group and no coding technology beyond "GSC" | https://www.mpeg.org/standards/Explorations/45/ |
| Who works on it | "Gaussian Splat Coding standardization is coordinated across WG 4 (Video Coding), WG 5 / JVET, and WG 7 (3D Graphics and Haptics)"; documents for the joint exploration at the next meeting "are to be managed by WG 5" | ISO/IEC JTC 1/SC 29/WG 4 N 837, *FAQ on Gaussian splat coding* (MPEG 154), https://www.mpeg.org/wp-content/uploads/mpeg_meetings/154_SantaEularia/w26333.zip |
| The call for proposals | N 837: "MPEG approved the first draft Call for Proposals (CfP) for Gaussian Splat Coding". N 422: "MPEG WGs are currently preparing a Call for Proposals (CfP) covering coding technology for static and dynamic objects and scenes represented as I-3DGS, where I-3DGS denotes per-frame 3DGS content in the original INRIA format"; test material is wanted "before October 15th 2026", for MPEG 156 | WG 4 N 837; WG 5 N 422, *Call for content for Gaussian splats coding* (MPEG 155), https://www.mpeg.org/wp-content/uploads/mpeg_meetings/155_Geneva/w26655.zip |
| I-3DGS and A-3DGS | I-3DGS "Refers to point clouds represented in the “INRIA” format"; A-3DGS "Encompasses alternative Gaussian splat representations" | WG 4 N 837; WG 2 N 531, *Gaussian splat coding use cases* (MPEG 155), https://www.mpeg.org/wp-content/uploads/mpeg_meetings/155_Geneva/w26708.zip |
| Anchors | a G-PCC anchor coded with the test model of "ISO/IEC 23090-9:2023"; a V-PCC anchor on "the V-PCC GS toolset and RecGS reconstruction profile of ISO/IEC 23090-5"; a video-based anchor that "uses packing of Gaussian splat positions and attributes to video planes that are encoded as separate videos using HEVC (HM)". An anchor "does not predetermine the architecture of a future GSC standard" (N 837) | WG 4 N 826 (MPEG 154), https://www.mpeg.org/wp-content/uploads/mpeg_meetings/154_SantaEularia/w26322.zip, and WG 4 N 751 (MPEG 152), https://www.mpeg.org/wp-content/uploads/mpeg_meetings/152_Geneva/w25782.zip, both titled *Common test conditions for Gaussian splat coding*; WG 4 N 837 |
| V-PCC amendment (ISO/IEC 23090-5) | Committee Draft Amendment at MPEG 153, promoted by "MPEG 3D Graphics and Haptics Coding (WG 7)", defining "a V-PCC profile specialized for Gaussian Splats"; Draft Amendment at MPEG 154 | https://www.mpeg.org/153rd-meeting-of-mpeg/; WG 4 N 837 |
| G-PCC amendment (ISO/IEC 23090-9) | "the first amendment to Geometry-based Point Cloud Compression, ISO/IEC 23090-9", at Committee Draft Amendment at MPEG 154 | https://www.mpeg.org/154th-meeting-of-mpeg/; WG 4 N 837 |
| Both amendments | "The AMD1 amendments to both standards introduce explicit support for Gaussian Splat parameters in their respective syntaxes and reconstruction processes." The V-PCC amendment adds a reconstruction profile because "Without a clearly specified reconstruction profile, different implementations could decode the same bitstream into different Gaussian Splat parameters" | WG 4 N 837 |
| NNC amendment (ISO/IEC 15938-17) | a Committee Draft of an amendment at MPEG 154, extending NNC to "parameters of 3D and 4D scene representations such as Gaussian splats" | https://www.mpeg.org/154th-meeting-of-mpeg/; [2030 landscape](2030-transport-landscape.md) §6.4 |
| Enhanced G-PCC, ISO/IEC 23090-38 | FDIS at MPEG 155; "a new generation of geometry-based point cloud compression specifically designed for dynamic point cloud sequences"; "While the original G-PCC standard (MPEG-I Part 9) primarily compresses point clouds on a frame-by-frame basis, the Enhanced G-PCC introduces a temporal coding framework". Neither page says whether a 23090-9 decoder decodes it | https://www.mpeg.org/155th-meeting-of-mpeg/; https://www.mpeg.org/standards/MPEG-I/38/ |
| MPEG-I Part 40, conformance and reference software for Enhanced G-PCC | Committee Draft at MPEG 152, for "the enhanced edition of the Geometry-based Point Cloud Compression (E-G-PCC) standard" | https://www.mpeg.org/152nd-meeting-of-mpeg/ |
| ISO/IEC 23090-41, G-PCC for solid objects | Committee Draft at MPEG 154; it "complements the original G-PCC standard, ISO/IEC 23090-9, which was primarily designed for sparse point clouds" | https://www.mpeg.org/154th-meeting-of-mpeg/ |
| ISO/IEC 23090-5, fourth edition | FDIS at MPEG 152, "allowing V3C to also serve as the structural foundation for V-DMC"; before it, "V3C could be instantiated for V-PCC (Video-based Point Cloud Compression) and MIV (MPEG Immersive Video)" | https://www.mpeg.org/152nd-meeting-of-mpeg/ |
*What this settles.* (a) No page or document above names Self-Organizing
Gaussians, SOG or SOGS, and MPEG's call concerns content in a representation
(I-3DGS), not a chosen coding technology: `wai.splat.sog` names no MPEG work item.
Whatever MPEG standardises for Gaussian splats takes a capability string of its
own when it is published; `wai.splat.sog` cannot be repurposed for it (SPEC §8).
(c) Enhanced G-PCC is a separate part with its own conformance part, and no source
read says a 23090-9 decoder decodes it, so it is not a `wai.pc.gpcc` payload
(SPEC §5, *Point clouds*); a MINOR revision may register it under its own string
once 23090-38 is published and its framing is pinned, and likewise 23090-41. What
`wai.pc.gpcc` and `wai.pc.vpcc` carry is open item (i).
## Standardized haptics
| capability | standard | royalty | reference | tier |
@ -203,7 +252,7 @@ confirms otherwise — do not assert royalty-free.
| capability | standard | royalty | tier |
|---|---|---|---|
| `wai.meta.hdr10plus` | HDR10+ dynamic metadata (SMPTE ST 2094-40) | **RF** (content) | Lossless |
| `wai.meta.dovi` | Dolby Vision dynamic metadata (SMPTE ST 2094-10, RPU) | **PROP** | Lossless |
| `wai.meta.dovi` | SMPTE ST 2094-10 dynamic metadata (RPU) | **PROP** | Lossless |
**Architectural note.** Unlike the codecs above, HDR metadata is *not* a
standalone media payload — it is color / dynamic-range metadata that
@ -248,7 +297,7 @@ a standard whose conformance text was not read stays `StandardUnclassified`.
| `wai.video.evc`, `wai.video.evc.baseline`, `wai.video.avs3`, `wai.audio.mpegh`, `wai.audio.avs3` | `StandardUnclassified` | not checked | nothing read for this purpose: ISO/IEC 23094-1 and 23008-3 and their conformance parts are not freely available, and T/AI 109.2 / 109.3 were not obtained |
| `wai.video.lcevc` | `StandardUnclassified` | not checked; its decoding process includes a signalled dither step (clause 8.11, "Decoding process for dither filter"), so identical output across decoders is not presumed | ISO/IEC 23094-2:2021 public preview sample (front matter to clause 5.8): table of contents and clause 3 only. The conformance part (ISO/IEC 23094-3) not read |
| `wai.audio.aac` | `StandardUnclassified` | not checked. Three decoders on one host gave three different PCM streams from one ADTS stream (measured above), so deployed decoders do not agree byte for byte; whether ISO/IEC 14496-26 admits that, or makes some of them non-conforming, was not established | nothing read for this purpose |
| `wai.audio.ac4`, `wai.mesh.draco`, `wai.pc.gpcc`, `wai.pc.vpcc`, `wai.splat.sog`, `wai.haptic.mihs`, `wai.meta.hdr10plus`, `wai.meta.dovi` | `StandardUnclassified` | not checked | nothing read for this purpose. (`wai.haptic.mihs`'s registry comment and SPEC §5 say ISO/IEC 23090-33 defines bit-exact decoder verification; that rests on a summary, not the standard text, and was not checked here) |
| `wai.audio.ac4`, `wai.mesh.draco`, `wai.pc.gpcc`, `wai.pc.vpcc`, `wai.splat.sog`, `wai.haptic.mihs`, `wai.meta.hdr10plus`, `wai.meta.dovi` | `StandardUnclassified` | not checked | nothing read for this purpose. (`wai.haptic.mihs`'s registry comment and SPEC §5 say ISO/IEC 23090-33 defines bit-exact decoder verification; that rests on a summary, not the standard text, and was not checked here); `wai.splat.sog` is a candidate with no pinned payload, so there is no conformance text for it to read |
| `wai.mesh.glb` | `StandardUnclassified` | the payload bytes pass through intact; what a sink decodes from them is not classified. A glTF binary carries more than raw buffers: glTF 2.0 images may be PNG or JPEG ("JPEG images MUST be compatible with JPEG File Interchange Format"), embedded through a `bufferView`, and geometry may be compressed through extensions (`KHR_draco_mesh_compression`; the vendor extension `EXT_meshopt_compression`). JPEG is `StandardTolerance` and Draco `StandardUnclassified` above, so a scene built from those can differ between conforming sinks; and glTF specifies an asset format, not a decoder conformance over rendered output (edge appearance, for one, "MAY be subject to implementation-specific techniques") | glTF 2.0.1 specification (Khronos registry HTML), text search; Khronos glTF extension registry README |
| `wai.splat.spz` | `StandardUnclassified` | the payload bytes pass through intact; SPZ stores quantised attributes, and its decode is not classified | not read |
@ -528,8 +577,10 @@ floor codecs are *wrapped* reference libraries, not re-implemented). So
reconstruct step stays sink-supplied while the encode + V3C-conformance +
carriage are real. Patent-pool, like G-PCC.
- **Decoder sink-supplied** (open reference exists, not bundled / not installed
here): `wai.video.j2k`, `wai.splat.sog`,
`wai.haptic.mihs`, `wai.meta.hdr10plus`.
here): `wai.video.j2k`, `wai.haptic.mihs`, `wai.meta.hdr10plus`.
- **Candidate, refused** (SPEC §5, *Candidate capabilities*): `wai.splat.sog` has
no pinned payload format, so the demo checks that `wai wrap` refuses it and
writes nothing.
- **Proprietary** (no open decoder, by design — carried, sink-supplied):
`wai.audio.ac4`, `wai.meta.dovi`.
- **Companion-only** (`wai.meta.hdr10plus`, `wai.meta.dovi`,
@ -602,8 +653,9 @@ point, and the reason WAI does not need to re-implement any of them.
be spoofable; the codec's licence is intrinsic to the capability).
3. **Royalty footnotes — VERIFIED (June 2026).** IAMF (AOMedia PL 1.0), JPEG
2000 Part 1 (ITU-T 2.1), Draco (Apache-2.0), HDR10+ (content+device RF) →
confirmed royalty-free; IVAS (Via-LA Voice Codec pool), G-PCC/V-PCC (MPEG) →
confirmed pool; AC-4 / Dolby Vision → proprietary. Residual: JPEG 2000
confirmed royalty-free; IVAS (a voice-codec pool) → confirmed pool; G-PCC/V-PCC
→ pool, a conservative classification (¹⁴); AC-4 / ST 2094-10 dynamic metadata →
proprietary. Residual: JPEG 2000
extensions/submarine-patent caveat (³), MPEG-I Haptics pool status
unconfirmed (⁵).
4. **Broadcast and streaming codecs — REGISTERED (September 2026).** AV2, VVC,
@ -635,7 +687,7 @@ point, and the reason WAI does not need to re-implement any of them.
CC-BY-NC 4.0, which would make `wai.neural.encodec32` non-commercial if it
applies. (d) E-AC-3 / AC-3, Extended HE-AAC and HDR Vivid (see above).
(e) The `StandardUnclassified` rows (EVC, AVS3 video and audio, LCEVC,
MPEG-H 3D Audio, AAC, AC-4, G-PCC, V-PCC, Draco, SOG, MIHS, the HDR
MPEG-H 3D Audio, AAC, AC-4, G-PCC, V-PCC, Draco, SOG (which first needs a pinned payload), MIHS, the HDR
descriptors, glb, SPZ) are waiting on each standard's conformance text; any
whose clause requires identical decoded output moves to `StandardDefined`,
with that clause recorded, and any with a stated criterion to
@ -650,3 +702,15 @@ point, and the reason WAI does not need to re-implement any of them.
bitstream of T/AI 109.2 and T/AI 109.3, which were not obtained. Before §8
fixes the strings, confirm from those texts that each defines one form of
its bitstream, or name the form the capability carries.
(i) What `wai.pc.gpcc` and `wai.pc.vpcc` carry. Under SPEC §8 (*Editions*), a
registration that names no edition carries the edition of its standard current
when it was registered, here 2026-06-17, and no later amendment, so the
Gaussian-splat amendments (V-PCC at Draft Amendment, G-PCC at Committee Draft
Amendment), a later edition, and V-DMC (a V3C application from the fourth
edition of ISO/IEC 23090-5) need strings of their own. Still open: which
edition of each part was current on 2026-06-17 (iso.org refused automated
reads; the common test conditions name a 2023 edition of 23090-9), and whether
`wai.pc.vpcc` carries the V3C bitstream of MIV.
(j) The video-based anchor of both common-test-conditions documents installs a
package named PLAS, which neither document describes. Whether it relates to the
family of formats `wai.splat.sog` was registered for was not established.

10
wai/research/mpeg-ai-part6-mapping.md
View file

@ -45,11 +45,11 @@ Two positions in this tree are absent from every provenance standard surveyed in
inside a lifecycle-scoped authenticity specification:
**Per-object measured energy with a declared acquisition class.** Provenance
standards record *what was done and by whom*. None records *what it cost, as
measured, with the provenance of the measurement stated*. WAI's binding is
per-object, carries `HwShunt` / `ModelBased` / `Estimator`, and refuses to
publish a figure from a failed measurement. Nothing else in the surveyed field
binds a measured physical quantity to a media object under signature.
standards record *what was done and by whom*. WAI's binding records *what it
cost, as measured, with the provenance of the measurement stated*: it is
per-object, carries `HwShunt` / `ModelBased` / `Estimator`, refuses to publish a
figure from a failed measurement, and binds the measured quantity to the media
object under signature.
**Determinism tiers separating byte-exact reconstruction from approximate
re-rendering.** Authenticity over generated or reconstructed media needs a

7
wai/research/spatial-audio-draft.md
View file

@ -136,11 +136,10 @@ A render session receipts as a JWP group (jwp-receipts.md):
`joules_micro` = the native meter's measured CPU energy for the mix
(the `wai_world_meter` pattern, applied to the mixer).
## Appendix — why this is the cleanest first cargo class
## Appendix — why this is the first cargo class
Mixdown-equivalence is replay-equivalence with the tick replaced by the
audio sample; the `Fx` floor, the content-addressed assets, the seal,
the receipt profile, and the fallback floor are all reused verbatim. A
spatial-audio scene is, structurally, a world whose "sim" is a mixer —
which is exactly why it proves the interactive-worlds thesis
generalizes to a second sense.
spatial-audio scene is, structurally, a world whose "sim" is a mixer,
so the interactive-worlds rules carry over to a second sense.

9
wai/research/syfai-input.md
View file

@ -530,7 +530,7 @@ there: `Conditioning::prior` in a `WAI1` manifest and `RenditionMeta::prior` on
each `WAI2` rendition. The digest check and mismatch refusal exist only in
`neural/mod.rs` (`resolve_checked`), and apply to the `WAI1` pin only (G12).
On `main` the specification says less, and in one place the opposite: SPEC.md's
On `main` the specification says less, and in one place the opposite: SPEC.md's <!-- allow-phrase: dated record, plain English -->
opening section says the model is "never transported and never hash-pinned".
In the tree at link 10 (link 05, PR #106) that sentence reads that the model is
never transported in the envelope that decodes against it, and that "its
@ -696,7 +696,10 @@ In the tree at link 10:
They are still outside the byte-exact golden suite, and the comparisons with
the Python reference in `int_audio.rs` and `int_splat.rs` still run in no
job. determinism-tiers §6 said until link 11 that no CI job enables their
features; link 11 corrects it.
features; link 11 corrects it. *(2026-09-30: the conformance workflow now also
names `tests/int_audio.rs` and `tests/int_splat.rs`, declared with their
features, so the comparisons with the Python reference's hashes run on every
leg; `wai.splat.sog` is now a candidate that no encoder may emit, SPEC §5.)*
- **PR #101 (independent of the chain).** Makes the neural end-to-end job link
and fail on a missing fixture, and fixes the MLIC++ fixture mismatch (§2). At
link 10 the job is unchanged and still cannot link.
@ -874,7 +877,7 @@ these together:
every machine as a property of the capability, or checked end to end in CI.
In the tree at link 10 its texts say the same (G11, G8).
- It does not claim that WAI specifies a fallback for a parameter-set
mismatch. SPEC.md's dispatch rule requires the opposite (UC-2, G12). The
mismatch. SPEC.md's dispatch rule requires the opposite (UC-2, G12). The <!-- allow-phrase: dated record, plain English -->
reference implementation's per-rendition pin is data that nothing checks.
- It does not claim a four-platform CI pass for `main`. The `windows` leg has
not run since about 2026-08-28, and the float decode tests have never run in

2
wai/tools/build_reference_decoders.sh
View file

@ -22,7 +22,7 @@
# On success it prints the exact `export PATH=… TMC2_CFG=…` line to source
# before running ./tools/wai_carriage_demo.sh.
#
# Toolchain notes (verified on macOS arm64, Apple clang 21, cmake 4.3.3):
# Toolchain notes (verified on macOS arm64, the platform clang 21, cmake 4.3.3):
# * tmc2/tmc13 ship a build.sh that rejects cmake major != 3; we invoke cmake
# directly with -DCMAKE_POLICY_VERSION_MINIMUM=3.5 so cmake 4.x accepts the
# pre-3.5 minimums the reference CMakeLists still declare.

2
wai/tools/requirements-sota-int.txt
View file

@ -5,7 +5,7 @@
# change here, not silently on the next CI run.
#
# torch/torchvision: install from the CPU wheel index first on Linux, or pip
# resolves the CUDA builds (several GB). `==2.14.0` matches `2.14.0+cpu`.
# resolves the GPU builds (several GB). `==2.14.0` matches `2.14.0+cpu`.
# pip install torch==2.14.0 torchvision==0.29.0 --index-url https://download.pytorch.org/whl/cpu
# pip install -r tools/requirements-sota-int.txt
torch==2.14.0

5
wai/tools/wai_carriage_demo.sh
View file

@ -26,6 +26,7 @@
# carriage = representative payload carried byte-exact; decoder
# sink-supplied (no open encoder installed for the format).
# proprietary = carried byte-exact; decoder is proprietary (sink-supplied).
# candidate = a registered capability with no pinned payload format (SPEC §5); `wai wrap` must refuse it and write nothing.
#
# ./tools/wai_carriage_demo.sh
#
@ -178,7 +179,9 @@ PY
demo wai.pc.vpcc "$TMP/vpcc" "real+parse" "PccAppParser $TMP/p.out >$TMP/v3c.txt 2>&1; grep -q V3C_VPS $TMP/v3c.txt"
else rep "$TMP/vpcc" 4096; demo wai.pc.vpcc "$TMP/vpcc" "carriage"; fi
else rep "$TMP/vpcc" 4096; demo wai.pc.vpcc "$TMP/vpcc" "carriage"; fi
rep "$TMP/sog" 4096; demo wai.splat.sog "$TMP/sog" "carriage"
rep "$TMP/sog" 4096; rm -f "$TMP/sog.wai"
if "$WAI" wrap wai.splat.sog "$TMP/sog" -o "$TMP/sog.wai" >/dev/null 2>&1 || [ -e "$TMP/sog.wai" ]; then sog_rt="✗ wrap ACCEPTED a candidate"; else sog_rt="✓ refused, nothing written"; fi
printf '%-22s %-7s %-13s %s\n' "wai.splat.sog" "$("$WAI" capabilities 2>/dev/null | awk '$1=="wai.splat.sog"{print $2}')" "candidate" "$sog_rt"
echo '{"version":1,"avatars":[]}' > "$TMP/h.hjif"
demo wai.haptic.mihs "$TMP/h.hjif" "carriage"

7
wai/tools/wai_dac_encode.py
View file

@ -1,9 +1,8 @@
"""WAI source-side neural encoder for `wai.neural.dac`.
DAC (Descript Audio Codec) at 44.1 kHz, 9 codebooks. The model is the
DAC (improved RVQGAN, arXiv 2306.06546) at 44.1 kHz, 9 codebooks. The model is the
sink-installed decoder; only the quantized codes ship in the WAI
envelope (~6 kbps for near-transparent music quality, well below Opus
at the same quality target).
envelope: 9 codebooks of 10 bits at about 86 frames a second, about 8 kbps.
Wire format (uniform across all wai.neural.<audio> capabilities):
<III> q t n_samples (little-endian u32 × 3)
@ -27,7 +26,7 @@ from pathlib import Path
import numpy as np
MODEL_ID = "descript/dac_44khz"
MODEL_ID = "descript/dac_44khz" # allow-name: model-hub id the tool defaults to
WAI_MAGIC = b"WAI1"
SR = 44_100

4
wai/tools/wai_dac_export_onnx.py
View file

@ -43,14 +43,14 @@ class WaiDacDecoder(torch.nn.Module):
if wav.dim() == 2:
wav = wav.unsqueeze(1) # → [B, 1, samples]
# Touch audio_scales so ONNX keeps it in the input signature for ABI
# parity with EnCodec/WavTokenizer. The expression collapses to 1.0.
# consistency with the EnCodec/WavTokenizer exports. The expression collapses to 1.0.
keep_scales = audio_scales.sum() / audio_scales.sum().clamp(min=1e-9)
return wav * keep_scales
def main() -> int:
ap = argparse.ArgumentParser()
ap.add_argument("--model-id", default="descript/dac_44khz",
ap.add_argument("--model-id", default="descript/dac_44khz", # allow-name: model-hub id the tool defaults to
help="HF model id or local path.")
ap.add_argument("--out", required=True, help="Output .onnx path.")
ap.add_argument("--opset", type=int, default=17)

6
wai/tools/wai_int_splat_export.py
View file

@ -1,9 +1,9 @@
#!/usr/bin/env python3
"""Train a learned INTEGER-exact Gaussian-splat ATTRIBUTE codec — "the int_hyper of
3D". A Gaussian splat is N Gaussians × 14 attributes (pos[3], scale[3], rot[4],
color[3], opacity) — already a fixed-point (Fx i64) tensor in WAI. The SOTA splat
codecs (SOG / self-organizing Gaussians) decode through float, so the reconstructed
cloud — and its hash — drift between machines. Here the synthesis is integer (the
color[3], opacity) — already a fixed-point (Fx i64) tensor in WAI. A decode that runs
float arithmetic can round differently on different machines, and then the
reconstructed cloud — and its hash — differ between them. Here the synthesis is integer (the
int_hyper machinery, 14 channels instead of RGB's 3): the decoded attribute tensor is
byte-identical on every machine — **splat-equivalence**, a portable cloud hash. The
float rasterizer is sink-supplied presentation; the *attributes* are the contract.

10
wai/tools/wai_int_world_export.py
View file

@ -1,8 +1,8 @@
#!/usr/bin/env python3
"""Train a small action-conditioned INTERACTIVE WORLD MODEL and export it as an
INTEGER-exact decoder — the conformance-is-a-hash answer to Genie / Oasis / Matrix /
fal-WMA. Those generate the next frame from an action through a FLOAT network, so the
generated world is non-reproducible and ephemeral. Here the world model is integer:
INTEGER-exact decoder, so that conformance is a hash. A world model that generates
the next frame from an action through a FLOAT network gives a world that is not
reproducible from one platform to another. Here the world model is integer:
frame_0 = g_s(z_0); z_{t+1} = clip( T( z_t ‖ action_t ) ); frame_{t+1} = g_s(z_{t+1})
@ -218,7 +218,7 @@ def main(outdir):
frames = torch.tensor(np.stack([d[0] for d in data]))
acts = [d[1] for d in data]
# GPU when available (Apple MPS) — the free-roll recurrent backprop DEPTH is the
# GPU when available (torch's `mps` backend) — the free-roll recurrent backprop DEPTH is the
# training cost, so the GPU is what unblocks a longer horizon (bigger ROLL).
dev = 'mps' if torch.backends.mps.is_available() else ('cuda' if torch.cuda.is_available() else 'cpu')
print(f' training on {dev} (ROLL={ROLL}, C={C}, steps={STEPS})')
@ -282,7 +282,7 @@ def main(outdir):
if __name__ == '__main__':
out = sys.argv[1] if len(sys.argv) > 1 else '/tmp/int_world'
# optional SOTA-scale overrides: FRAME LAT C ROLL WORLD (FRAME must = LAT·8)
# optional full-scale overrides: FRAME LAT C ROLL WORLD (FRAME must = LAT·8)
if len(sys.argv) > 2:
FRAME = int(sys.argv[2])
if len(sys.argv) > 3:

9
wai/tools/wai_mimi_encode.py
View file

@ -1,9 +1,8 @@
"""WAI source-side neural encoder for `wai.neural.mimi`.
Mimi (Kyutai) at 24 kHz, 12.5 Hz frame rate, 8 codebooks of 2048.
Designed for real-time speech LLM use cases. Token rate is among the
lowest of any high-quality neural audio codec (~1.1 kbps), which makes
it the right pick when the wire-byte budget dominates.
Mimi (arXiv 2410.00037) at 24 kHz, 12.5 Hz frame rate, 8 codebooks of 2048.
Designed for real-time speech use. The token rate is 8 codebooks of 11 bits at
12.5 Hz, about 1.1 kbps on the wire.
Wire format (uniform with all wai.neural.<audio> capabilities):
<III> q t n_samples (little-endian u32 × 3)
@ -24,7 +23,7 @@ from pathlib import Path
import numpy as np
MODEL_ID = "kyutai/mimi"
MODEL_ID = "kyutai/mimi" # allow-name: model-hub id the tool defaults to
WAI_MAGIC = b"WAI1"
SR = 24_000

2
wai/tools/wai_mimi_export_onnx.py
View file

@ -55,7 +55,7 @@ class WaiMimiDecoder(torch.nn.Module):
def main() -> int:
ap = argparse.ArgumentParser()
ap.add_argument("--model-id", default="kyutai/mimi",
ap.add_argument("--model-id", default="kyutai/mimi", # allow-name: model-hub id the tool defaults to
help="HF model id or local path.")
ap.add_argument("--out", required=True)
ap.add_argument("--opset", type=int, default=17)

4
wai/tools/wai_replicate_probe.sh
View file

@ -25,9 +25,9 @@
# not failed.
#
# Decoders used when present: ffmpeg's own decoders; libdav1d, libvpx and
# libopus through ffmpeg; Apple VideoToolbox hardware decode (forced onto
# libopus through ffmpeg; macOS VideoToolbox hardware decode (forced onto
# hardware surfaces, so it fails rather than falling back to software);
# Apple AudioToolbox AAC; libjpeg-turbo `djpeg`; OpenJPEG `opj_decompress`;
# macOS AudioToolbox AAC; libjpeg-turbo `djpeg`; OpenJPEG `opj_decompress`;
# libFLAC `flac`; Core Audio `afconvert`.
#
# ./tools/wai_replicate_probe.sh

2
wai/tools/wai_sota_int_synth_export.py
View file

@ -193,7 +193,7 @@ def main(imgpath, outdir):
# The ENCODER side (g_a, h_a, the float h_s used for the reference) is float, so
# its latent can depend on kernel choice and reduction order. Both are pinned
# to make two runs of this tool on one machine emit identical vectors. That
# held for repeated runs on an Apple Silicon Mac, but two runs on an x86_64
# held for repeated runs on an arm64 macOS machine, but two runs on an x86_64
# Linux runner wrote different cond.json and int_synth_sota.json, most likely
# in psnr_float (`base` below), which the Rust tests read only as a quality
# threshold; the latent and both rgb_hash values matched. The decode being

2
wai/tools/wai_sota_vectors.sh
View file

@ -21,7 +21,7 @@
# The output is not portable, and part of it is not repeatable run to run on
# every machine. Step 2 runs the float model (torch, CPU) to encode the
# photograph, on the pinned stack, which each platform installs as its own build.
# - Between an Apple Silicon Mac and an x86_64 Linux runner,
# - Between an arm64 macOS machine and an x86_64 Linux runner,
# int_synth_sota_input.json (the synthesis-only latent, which the float
# encoder and hyper-decoder compute) differed, and with it the rgb_hash in
# int_synth_sota.json. Each side wrote the same latent in all of its runs.

4
wai/tools/wai_video_bmshj2018_encode.py
View file

@ -6,11 +6,11 @@ payload format as `wai.neural.bmshj2018`), and the per-frame payloads
are bundled into a single WAI envelope with a small frame-table header.
This is browser-decodable — bmshj2018 runs in onnxruntime-web (WebGPU
or WASM). Full DCVC-RT / DCVC-FM (the 2025-26 SOTA for inter-frame
or WASM). Full DCVC-RT / DCVC-FM (2025-26 learned inter-frame
neural video) is NOT encoded by this per-frame tool: it carries a
temporal net + reference-feature propagation between frames. Its decode
route is implemented for native sinks (`neural/video_dcvc.rs`, CPU-capable
via `ort` — not CUDA-only), but it is too heavy for a browser sink, so
via `ort` — no GPU required), but it is too heavy for a browser sink, so
this per-frame path is what browser sinks decode. SPEC.md §5 documents
the split.

2
wai/tools/wai_wavtokenizer_encode.py
View file

@ -106,7 +106,7 @@ if __name__ == "__main__":
p.add_argument("input")
p.add_argument("-o", "--output", required=True)
p.add_argument("--wavtokenizer-src", default="/tmp/wavtokenizer",
help="Path to the cloned WavTokenizer GitHub repo.")
help="Path to a clone of the WavTokenizer source repository.") # allow-name: open-source project the tool imports
p.add_argument("--config", required=True, help="YAML model config")
p.add_argument("--ckpt", required=True, help=".ckpt checkpoint")
a = p.parse_args()

2
wai/vocab/.gitattributes vendored Normal file
View file

@ -0,0 +1,2 @@
# v2.* are generated by `wai registry export` and compared byte for byte.
v2.* -text

50
wai/vocab/README.md
View file

@ -1,24 +1,38 @@
# WAI vocabulary (v1)
# WAI vocabulary
Stable IRIs for the Web AI Media Transport & Execution open
standard. Covers `Manifest`, `Capability`, the two paths
(`wai:NeuralPath` — shared-prior regeneration; `wai:ZerothPath` —
registered codec menu), the six modalities (Image / Audio / Video
/ Volume / Mesh / Compressed), and the nine codecs in the v1 menu
(PNG / JPEG / AVIF / JPEG-XL / FLAC / Opus / AV1 / zstd / XZ).
Stable IRIs for the Web AI Media Transport & Execution open standard, in two
versions. A WAI manifest that carries one of the JSON-LD contexts below expands
into typed RDF triples for any JSON-LD or SPARQL consumer without changing its
wire form. Licence Apache-2.0, matching the reference implementation.
Namespace: `https://wai.transaction.science/vocab/v1#` (prefix
`wai:` by convention).
## v2 — generated from the registry
Namespace: `https://wai.transaction.science/vocab/v2#`.
| File | Format | What |
|------|--------|------|
| [`v1.jsonld`](v1.jsonld) | JSON-LD `@context` | Term → IRI mapping. |
| [`v1.ttl`](v1.ttl) | Turtle (RDF) | Classes, enumerations, properties; modality + path + codec taxonomies. |
| [`v1.shapes.ttl`](v1.shapes.ttl) | SHACL Core | Shape graph — validates `Manifest` and `NeuralPrior` locally before file open. Enforces the modality + path + codec enums against the v1 menu. SHACL Core only; consumable by any conformant validator. |
| [`v2.ttl`](v2.ttl) | Turtle (RDF) | One IRI per string SPEC §5 registers, with its kind, status, media class, role, fidelity tier, licence class, replicate class, registering section, edition and registration date (`registeredOn`); the enumerations as individuals named `<class prefix>-<label>` (`kind-companion`, `role-companion`, `media-image`, …), so no two terms share an IRI; and the manifest terms the context maps. |
| [`v2.jsonld`](v2.jsonld) | JSON-LD `@context` | For a WAI manifest, `WAI1` or `WAI2`, as written on the wire: `capability`, `fallback` and a component's `companion_of` expand to capability IRIs, `media` to a media-class individual; `kind` and a component's `role` are plain strings under terms of their own (`payloadKind`, `componentRole`), and `target` is a JSON literal. |
| [`v2.shapes.ttl`](v2.shapes.ttl) | SHACL Core | Applies to a manifest with or without an `@type` (it targets every node with a `wai` or `intent` field, and every component of `renditions`). Checks that a `WAI1` names a capability and a `WAI2` a component list, never both; the `capability` and every component's capability against the registered payload and companion capabilities that are not candidates; the `fallback` and a companion's base against the registered payload capabilities that are not candidates. `media` is informational and a reader accepts a value it does not know (SPEC §3, §8), so only its form is checked. The shapes check a manifest against this revision's registry: a violation for a capability or fallback this revision does not register means the string is not registered in this revision, not that the envelope fails to conform or cannot be dispatched, since a later MINOR revision may register it (SPEC §4, §8). |
A WAI manifest that carries
`@context: "https://wai.transaction.science/vocab/v1.jsonld"`
expands into typed RDF triples for any JSON-LD or SPARQL consumer
without changing its wire form. Mirrors the pattern shipped at
`jouleclaw/vocab/v1`. Licence Apache-2.0 to match the reference
implementation.
The three files are generated by `wai registry export` from the reference
implementation's registry (`wai-rs/src/codecs/registry.rs`), together with
[`../registry/capabilities.json`](../registry/capabilities.json); a library test
and a CI step fail when they drift. [`check_shapes.py`](check_shapes.py) expands
sample manifests, typed and untyped, through the context and validates them
against the shapes, each with the result it must give; CI runs it
(`.github/workflows/wai-conformance.yml`). Do not edit the generated files by hand —
[`../REGISTRY.md`](../REGISTRY.md) says how a string is registered. The v2 IRIs
resolve only once the website publishes the files.
## v1 — stable
Namespace: `https://wai.transaction.science/vocab/v1#`. The v1 files
([`v1.jsonld`](v1.jsonld), [`v1.ttl`](v1.ttl), [`v1.shapes.ttl`](v1.shapes.ttl))
are stable in this sense: no v1 IRI is removed, and no term changes what it
means, so nothing that already names them breaks. A label or comment may be
corrected, and a term may be added. They cover the first codec menu only (PNG, JPEG, AVIF,
JPEG XL, FLAC, Opus, AV1, zstd, XZ) and model the neural and zeroth paths as
classes; they are not the registry. v1's SHACL shape rejects a manifest that
names any other registered capability, so a validator of current manifests uses
v2.

104
wai/vocab/check_shapes.py Normal file
View file

@ -0,0 +1,104 @@
#!/usr/bin/env python3
"""Validate sample WAI manifests against the generated v2 vocabulary.
Each manifest is expanded through vocab/v2.jsonld exactly as written on the
wire, with no @type added, and validated against vocab/v2.shapes.ttl with
v2.ttl as the ontology graph. Every case states whether it must conform. Exit
status 0 when every case behaves as stated, 1 otherwise.
Needs rdflib, pyld and pyshacl (CI pins their versions). Run from anywhere:
python3 wai/vocab/check_shapes.py
"""
import copy
import json
import pathlib
import sys
import rdflib
from pyld import jsonld
from pyshacl import validate
VOCAB = pathlib.Path(__file__).resolve().parent
def wai1(capability, fallback=None, media="image", intent="replicate", **extra):
m = {
"wai": "1.0",
"media": media,
"intent": intent,
"model_requirement": {"capability": capability},
"conditioning": {"kind": "png"},
"target": {"note": "opaque JSON, carried as a literal"},
}
if fallback is not None:
m["model_requirement"]["fallback"] = fallback
m.update(extra)
return m
def wai2(*components, intent="improve"):
renditions = []
for c in components:
if isinstance(c, tuple):
cap, base = c
renditions.append({"capability": cap, "kind": "bytes", "role": "companion", "companion_of": base})
else:
renditions.append({"capability": c, "kind": "bytes"})
return {"wai": "1.1", "media": "video", "intent": intent, "renditions": renditions, "target": None}
# (name, manifest, must conform)
CASES = [
("WAI1 png, untyped", wai1("wai.image.png"), True),
("WAI1 png with a png fallback", wai1("wai.image.jxl", "wai.image.png"), True),
("WAI1 media value no revision lists (a reader accepts it)", wai1("wai.mesh.glb", media="model"), True),
("WAI1 candidate capability", wai1("wai.splat.sog", media="volumetric"), False),
("WAI1 candidate capability, the other candidate", wai1("wai.tensor.nnc", media="binary"), False),
("WAI1 candidate fallback", wai1("wai.image.png", "wai.tensor.nnc"), False),
("WAI1 companion-only fallback", wai1("wai.video.av1", "wai.meta.hdr10plus", media="video"), False),
("WAI1 unregistered capability", wai1("wai.image.nope"), False),
("WAI1 record as capability", wai1("wai.video.manifest", media="video"), False),
("WAI1 with no intent", {k: v for k, v in wai1("wai.image.png").items() if k != "intent"}, False),
("WAI2 rendition + companion, untyped", wai2("wai.video.av1", ("wai.meta.hdr10plus", "wai.video.av1")), True),
("WAI2 candidate rendition", wai2("wai.splat.sog", "wai.image.png"), False),
("WAI2 candidate companion", wai2("wai.video.av1", ("wai.splat.sog", "wai.video.av1")), False),
("WAI2 companion bound to a candidate base", wai2("wai.video.av1", ("wai.meta.hdr10plus", "wai.tensor.nnc")), False),
("WAI1 and WAI2 at once", dict(wai1("wai.image.png"), renditions=[{"capability": "wai.image.png", "kind": "png"}]), False),
]
def main():
context = json.loads((VOCAB / "v2.jsonld").read_text())["@context"]
shapes = rdflib.Graph().parse(VOCAB / "v2.shapes.ttl", format="turtle")
ontology = rdflib.Graph().parse(VOCAB / "v2.ttl", format="turtle")
failures = 0
for name, manifest, want in CASES:
doc = copy.deepcopy(manifest)
doc["@context"] = context
data = rdflib.Graph().parse(data=json.dumps(jsonld.expand(doc)), format="json-ld")
conforms, _, text = validate(data, shacl_graph=shapes, ont_graph=ontology, inference="none")
ok = conforms == want
failures += not ok
print(f"{'ok ' if ok else 'FAIL'} {name}: conforms={conforms}, expected {want}")
if not ok:
print(text)
# The typed form validates the same way as the untyped one.
typed = wai1("wai.splat.sog")
typed["@type"] = "Manifest"
typed["@context"] = context
data = rdflib.Graph().parse(data=json.dumps(jsonld.expand(typed)), format="json-ld")
conforms, _, _ = validate(data, shacl_graph=shapes, ont_graph=ontology, inference="none")
ok = conforms is False
failures += not ok
print(f"{'ok ' if ok else 'FAIL'} WAI1 candidate capability, typed: conforms={conforms}, expected False")
if failures:
print(f"{failures} case(s) did not behave as stated")
return 1
print("v2 shapes: every case behaves as stated")
return 0
if __name__ == "__main__":
sys.exit(main())

2
wai/vocab/v1.ttl
View file

@ -38,7 +38,7 @@ wai:Path a owl:Class ; rdfs:label "Decode path" .
wai:NeuralPath a wai:Path ; rdfs:label "Neural shared-prior path" ;
rdfs:comment "Sink regenerates the media from a shared ambient prior referenced by URI + hash." .
wai:ZerothPath a wai:Path ; rdfs:label "Zeroth-condition path" ;
rdfs:comment "Registered SOTA codec menu — PNG/FLAC/zstd mandatory floor + the recommended modern set." .
rdfs:comment "Registered codec menu — PNG/FLAC/zstd mandatory floor + the recommended modern set." .
# ─── Registered codecs ───────────────────────────────────────────────

48
wai/vocab/v2.jsonld Normal file
View file

@ -0,0 +1,48 @@
{
"@context": {
"@version": 1.1,
"@vocab": "https://wai.transaction.science/vocab/v2#",
"xsd": "http://www.w3.org/2001/XMLSchema#",
"Manifest": "https://wai.transaction.science/vocab/v2#Manifest",
"Component": "https://wai.transaction.science/vocab/v2#Component",
"model_requirement": "@nest",
"capability": {
"@id": "https://wai.transaction.science/vocab/v2#capability",
"@type": "@vocab"
},
"fallback": {
"@id": "https://wai.transaction.science/vocab/v2#fallback",
"@type": "@vocab"
},
"companion_of": {
"@id": "https://wai.transaction.science/vocab/v2#companionOf",
"@type": "@vocab"
},
"media": {
"@id": "https://wai.transaction.science/vocab/v2#media",
"@type": "@vocab",
"@context": {
"@vocab": "https://wai.transaction.science/vocab/v2#media-"
}
},
"intent": {
"@id": "https://wai.transaction.science/vocab/v2#intent",
"@type": "xsd:string"
},
"kind": {
"@id": "https://wai.transaction.science/vocab/v2#payloadKind",
"@type": "xsd:string"
},
"role": {
"@id": "https://wai.transaction.science/vocab/v2#componentRole",
"@type": "xsd:string"
},
"renditions": {
"@id": "https://wai.transaction.science/vocab/v2#renditions"
},
"target": {
"@id": "https://wai.transaction.science/vocab/v2#target",
"@type": "@json"
}
}
}

339
wai/vocab/v2.shapes.ttl Normal file
View file

@ -0,0 +1,339 @@
# Generated by `wai registry export` from wai-rs/src/codecs/registry.rs. Do not edit.
@prefix wai: <https://wai.transaction.science/vocab/v2#> .
@prefix sh: <http://www.w3.org/ns/shacl#> .
@prefix xsd: <http://www.w3.org/2001/XMLSchema#> .
# A WAI manifest, typed or not: every manifest has a wai (version) field. A WAI1 manifest names a
# capability, a WAI2 manifest a renditions list, and never both. The capability is a
# registered payload or companion capability that is not a candidate; the fallback a
# registered payload capability that is not a candidate. (A WAI1 naming a companion-only
# capability parses but does not conform; the reference implementation's role check
# reports it.) media is informational, and a reader accepts a value it does not know
# (SPEC sections 3 and 8), so only its form is checked.
wai:ManifestShape a sh:NodeShape ;
sh:targetClass wai:Manifest ;
sh:targetSubjectsOf wai:wai ;
sh:targetSubjectsOf wai:intent ;
sh:property [ sh:path wai:capability ; sh:nodeKind sh:IRI ; sh:maxCount 1 ;
sh:in (
wai:wai.audio.aac
wai:wai.audio.ac4
wai:wai.audio.avs3
wai:wai.audio.flac
wai:wai.audio.iamf
wai:wai.audio.int_codec
wai:wai.audio.ivas
wai:wai.audio.mpegh
wai:wai.audio.opus
wai:wai.audio.scene
wai:wai.audio.score
wai:wai.avatar.pose
wai:wai.feed.state
wai:wai.film.linear
wai:wai.film.reel
wai:wai.generate.spectral
wai:wai.haptic.mihs
wai:wai.haptic.signal
wai:wai.image.avif
wai:wai.image.jp2
wai:wai.image.jpeg
wai:wai.image.jpegai
wai:wai.image.jxl
wai:wai.image.png
wai:wai.mesh.draco
wai:wai.mesh.glb
wai:wai.meta.dovi
wai:wai.meta.hdr10plus
wai:wai.neural.bmshj2018
wai:wai.neural.cmic
wai:wai.neural.dac
wai:wai.neural.dcvc_fm
wai:wai.neural.dcvc_rt
wai:wai.neural.elic
wai:wai.neural.encodec32
wai:wai.neural.glc
wai:wai.neural.int_hyper
wai:wai.neural.int_mlicpp
wai:wai.neural.int_synth
wai:wai.neural.mbt2018_mean
wai:wai.neural.mimi
wai:wai.neural.mlicpp
wai:wai.neural.mlicv2
wai:wai.neural.snac
wai:wai.neural.video_bmshj2018
wai:wai.neural.wavtokenizer
wai:wai.pc.gpcc
wai:wai.pc.vpcc
wai:wai.prior.nnc
wai:wai.quantum.circuit
wai:wai.semantic.clip
wai:wai.semantic.perco2
wai:wai.splat.4d
wai:wai.splat.int_codec
wai:wai.splat.spz
wai:wai.ternary.tt
wai:wai.text.xz
wai:wai.text.zstd
wai:wai.video.av1
wai:wai.video.av1.lossless
wai:wai.video.av2
wai:wai.video.avc
wai:wai.video.avs3
wai:wai.video.evc
wai:wai.video.evc.baseline
wai:wai.video.hevc
wai:wai.video.int_hyper
wai:wai.video.int_motion
wai:wai.video.j2k
wai:wai.video.lcevc
wai:wai.video.vp9
wai:wai.video.vvc
wai:wai.world.live
wai:wai.world.model
wai:wai.world.replay
wai:wai.world.scene
) ] ;
sh:property [ sh:path wai:fallback ; sh:nodeKind sh:IRI ; sh:maxCount 1 ;
sh:in (
wai:wai.audio.aac
wai:wai.audio.ac4
wai:wai.audio.avs3
wai:wai.audio.flac
wai:wai.audio.iamf
wai:wai.audio.int_codec
wai:wai.audio.ivas
wai:wai.audio.mpegh
wai:wai.audio.opus
wai:wai.audio.scene
wai:wai.audio.score
wai:wai.avatar.pose
wai:wai.feed.state
wai:wai.film.linear
wai:wai.film.reel
wai:wai.generate.spectral
wai:wai.haptic.mihs
wai:wai.haptic.signal
wai:wai.image.avif
wai:wai.image.jp2
wai:wai.image.jpeg
wai:wai.image.jpegai
wai:wai.image.jxl
wai:wai.image.png
wai:wai.mesh.draco
wai:wai.mesh.glb
wai:wai.neural.bmshj2018
wai:wai.neural.cmic
wai:wai.neural.dac
wai:wai.neural.dcvc_fm
wai:wai.neural.dcvc_rt
wai:wai.neural.elic
wai:wai.neural.encodec32
wai:wai.neural.glc
wai:wai.neural.int_hyper
wai:wai.neural.int_mlicpp
wai:wai.neural.int_synth
wai:wai.neural.mbt2018_mean
wai:wai.neural.mimi
wai:wai.neural.mlicpp
wai:wai.neural.mlicv2
wai:wai.neural.snac
wai:wai.neural.video_bmshj2018
wai:wai.neural.wavtokenizer
wai:wai.pc.gpcc
wai:wai.pc.vpcc
wai:wai.prior.nnc
wai:wai.quantum.circuit
wai:wai.semantic.clip
wai:wai.semantic.perco2
wai:wai.splat.4d
wai:wai.splat.int_codec
wai:wai.splat.spz
wai:wai.ternary.tt
wai:wai.text.xz
wai:wai.text.zstd
wai:wai.video.av1
wai:wai.video.av1.lossless
wai:wai.video.av2
wai:wai.video.avc
wai:wai.video.avs3
wai:wai.video.evc
wai:wai.video.evc.baseline
wai:wai.video.hevc
wai:wai.video.int_hyper
wai:wai.video.int_motion
wai:wai.video.j2k
wai:wai.video.vp9
wai:wai.video.vvc
wai:wai.world.live
wai:wai.world.model
wai:wai.world.replay
wai:wai.world.scene
) ] ;
sh:property [ sh:path wai:media ; sh:nodeKind sh:IRI ; sh:maxCount 1 ] ;
sh:property [ sh:path wai:intent ; sh:datatype xsd:string ; sh:minCount 1 ; sh:maxCount 1 ] ;
sh:xone (
[ sh:property [ sh:path wai:capability ; sh:minCount 1 ] ]
[ sh:property [ sh:path wai:renditions ; sh:minCount 1 ] ]
) .
# A WAI2 component, in either role: its capability is a registered payload or companion
# capability that is not a candidate, and the base a companion names is a registered
# payload capability that is not a candidate.
wai:ComponentShape a sh:NodeShape ;
sh:targetClass wai:Component ;
sh:targetObjectsOf wai:renditions ;
sh:property [ sh:path wai:capability ; sh:nodeKind sh:IRI ; sh:minCount 1 ; sh:maxCount 1 ;
sh:in (
wai:wai.audio.aac
wai:wai.audio.ac4
wai:wai.audio.avs3
wai:wai.audio.flac
wai:wai.audio.iamf
wai:wai.audio.int_codec
wai:wai.audio.ivas
wai:wai.audio.mpegh
wai:wai.audio.opus
wai:wai.audio.scene
wai:wai.audio.score
wai:wai.avatar.pose
wai:wai.feed.state
wai:wai.film.linear
wai:wai.film.reel
wai:wai.generate.spectral
wai:wai.haptic.mihs
wai:wai.haptic.signal
wai:wai.image.avif
wai:wai.image.jp2
wai:wai.image.jpeg
wai:wai.image.jpegai
wai:wai.image.jxl
wai:wai.image.png
wai:wai.mesh.draco
wai:wai.mesh.glb
wai:wai.meta.dovi
wai:wai.meta.hdr10plus
wai:wai.neural.bmshj2018
wai:wai.neural.cmic
wai:wai.neural.dac
wai:wai.neural.dcvc_fm
wai:wai.neural.dcvc_rt
wai:wai.neural.elic
wai:wai.neural.encodec32
wai:wai.neural.glc
wai:wai.neural.int_hyper
wai:wai.neural.int_mlicpp
wai:wai.neural.int_synth
wai:wai.neural.mbt2018_mean
wai:wai.neural.mimi
wai:wai.neural.mlicpp
wai:wai.neural.mlicv2
wai:wai.neural.snac
wai:wai.neural.video_bmshj2018
wai:wai.neural.wavtokenizer
wai:wai.pc.gpcc
wai:wai.pc.vpcc
wai:wai.prior.nnc
wai:wai.quantum.circuit
wai:wai.semantic.clip
wai:wai.semantic.perco2
wai:wai.splat.4d
wai:wai.splat.int_codec
wai:wai.splat.spz
wai:wai.ternary.tt
wai:wai.text.xz
wai:wai.text.zstd
wai:wai.video.av1
wai:wai.video.av1.lossless
wai:wai.video.av2
wai:wai.video.avc
wai:wai.video.avs3
wai:wai.video.evc
wai:wai.video.evc.baseline
wai:wai.video.hevc
wai:wai.video.int_hyper
wai:wai.video.int_motion
wai:wai.video.j2k
wai:wai.video.lcevc
wai:wai.video.vp9
wai:wai.video.vvc
wai:wai.world.live
wai:wai.world.model
wai:wai.world.replay
wai:wai.world.scene
) ] ;
sh:property [ sh:path wai:companionOf ; sh:nodeKind sh:IRI ; sh:maxCount 1 ;
sh:in (
wai:wai.audio.aac
wai:wai.audio.ac4
wai:wai.audio.avs3
wai:wai.audio.flac
wai:wai.audio.iamf
wai:wai.audio.int_codec
wai:wai.audio.ivas
wai:wai.audio.mpegh
wai:wai.audio.opus
wai:wai.audio.scene
wai:wai.audio.score
wai:wai.avatar.pose
wai:wai.feed.state
wai:wai.film.linear
wai:wai.film.reel
wai:wai.generate.spectral
wai:wai.haptic.mihs
wai:wai.haptic.signal
wai:wai.image.avif
wai:wai.image.jp2
wai:wai.image.jpeg
wai:wai.image.jpegai
wai:wai.image.jxl
wai:wai.image.png
wai:wai.mesh.draco
wai:wai.mesh.glb
wai:wai.neural.bmshj2018
wai:wai.neural.cmic
wai:wai.neural.dac
wai:wai.neural.dcvc_fm
wai:wai.neural.dcvc_rt
wai:wai.neural.elic
wai:wai.neural.encodec32
wai:wai.neural.glc
wai:wai.neural.int_hyper
wai:wai.neural.int_mlicpp
wai:wai.neural.int_synth
wai:wai.neural.mbt2018_mean
wai:wai.neural.mimi
wai:wai.neural.mlicpp
wai:wai.neural.mlicv2
wai:wai.neural.snac
wai:wai.neural.video_bmshj2018
wai:wai.neural.wavtokenizer
wai:wai.pc.gpcc
wai:wai.pc.vpcc
wai:wai.prior.nnc
wai:wai.quantum.circuit
wai:wai.semantic.clip
wai:wai.semantic.perco2
wai:wai.splat.4d
wai:wai.splat.int_codec
wai:wai.splat.spz
wai:wai.ternary.tt
wai:wai.text.xz
wai:wai.text.zstd
wai:wai.video.av1
wai:wai.video.av1.lossless
wai:wai.video.av2
wai:wai.video.avc
wai:wai.video.avs3
wai:wai.video.evc
wai:wai.video.evc.baseline
wai:wai.video.hevc
wai:wai.video.int_hyper
wai:wai.video.int_motion
wai:wai.video.j2k
wai:wai.video.vp9
wai:wai.video.vvc
wai:wai.world.live
wai:wai.world.model
wai:wai.world.replay
wai:wai.world.scene
) ] .

1175
wai/vocab/v2.ttl Normal file
View file

File diff suppressed because it is too large Load diff

2
wai/wai-quantum/src/quantum.rs
View file

@ -1,7 +1,7 @@
//! Deterministic quantum-circuit **simulation** as transportable, receiptable
//! content — `wai.quantum.circuit` (extensions/quantum-sim).
//!
//! The honest framing first: WAI's whole moat is *determinism* — byte-identical
//! The framing first: WAI's core property is *determinism* — byte-identical
//! reconstruction on every machine, "conformance is a hash". Real quantum
//! HARDWARE is the natural enemy of that (NISQ noise + stochastic measurement are
//! not reproducible across devices). So quantum does NOT enter WAI as a hardware

29
wai/wai-quantum/src/quantum_cal.rs
View file

@ -1,11 +1,9 @@
//! Deterministic autonomous qubit CALIBRATION — `wai.quantum.cal.rabi`
//! (extensions/quantum-ops § Calibration engines).
//!
//! The quantum-control field (Q-CTRL, QuantrolOx, Qruise, Conductor …) sells one
//! thing above all: **autonomous gate tune-up** — sweep a control parameter,
//! measure the qubit, fit the response, set the optimum, no human in the loop.
//! Those are *algorithms that run on measured data*, and the vendors develop and
//! demo them against **simulators / digital twins** (Qruise ships one by name).
//! **Autonomous gate tune-up** — sweep a control parameter, measure the qubit, fit
//! the response, set the optimum, no human in the loop — is a set of *algorithms
//! that run on measured data*, commonly developed against **device simulators**.
//! This module is that algorithm — the canonical **Rabi π-pulse tune-up** — done
//! the WAI way: a deterministic fixed-point device simulator + an autonomous
//! calibrator, **byte-identical on every machine** (a portable tune-up hash), so
@ -17,7 +15,7 @@
//! model is a clean Rabi model, not a full transmon Hamiltonian. What is *not*
//! approximate is the reproducibility: no float anywhere in the run, so the
//! calibrated amplitude and the outcome hash are the same bytes everywhere — the
//! property the float vendor stacks cannot offer, on top of which the receipt
//! property a float run does not have, on top of which the receipt
//! makes every run auditable.
/// Fixed-point fractional bits. All amplitudes, probabilities and angles are i64
@ -683,8 +681,8 @@ pub fn calibrate_readout(dev: &ReadoutDevice, n_per_class: u32) -> ReadoutResult
}
// ===========================================================================
// Charge-stability tuning — silicon-spin double-dot autotuning
// (Conductor Quantum / QuantrolOx origin capability): find the gate-voltage
// Charge-stability tuning — silicon-spin double-dot charge-state autotuning:
// find the gate-voltage
// operating point for a target charge configuration from a 2D charge-stability
// scan. The honeycomb is a clean integer function of the *hidden* capacitance
// model; the tuner discovers the operating point from the scan, byte-exactly —
@ -810,7 +808,7 @@ pub fn tune_charge_state(
// ===========================================================================
// Two-qubit cross-resonance gate calibration
// (Q-CTRL / Qruise / Quantum Machines two-qubit tune-up). The target qubit's
// (a standard two-qubit tune-up). The target qubit's
// Rabi rate depends on the CONTROL qubit's state; calibrate both conditional
// π-amplitudes, whose difference is the ZX interaction. Model-fit, byte-exact.
// ===========================================================================
@ -938,7 +936,7 @@ pub fn calibrate_cr(dev: &CrDevice, cfg: &CrConfig) -> CrResult {
// ===========================================================================
// Crosstalk characterization — the N×N drive-response matrix
// (Q-CTRL / Conductor crosstalk mitigation, step one: measure the matrix).
// (crosstalk mitigation, step one: measure the matrix).
// ===========================================================================
/// An `n`-qubit device with a hidden crosstalk matrix: driving qubit `i` at
@ -1045,7 +1043,7 @@ pub fn characterize_crosstalk(dev: &CrosstalkDevice, probe_a_fx: i64, shots: u32
// ===========================================================================
// DRAG / leakage pulse calibration — the pulse-shaping (optimal-control) primitive
// (Conductor / Q-CTRL leakage suppression). Optimize the DRAG coefficient β that
// (leakage suppression). Optimize the DRAG coefficient β that
// minimizes leakage to |2⟩. A parabolic minimization — the simplest honest
// representative of the control-optimization layer (NOT full robust control).
// ===========================================================================
@ -1150,7 +1148,7 @@ pub fn calibrate_drag(dev: &DragDevice, beta_min_fx: i64, beta_max_fx: i64, n: u
}
// ===========================================================================
// Robust optimal control — composite-pulse design (Q-CTRL Boulder Opal class).
// Robust optimal control — composite-pulse design.
// Numerically OPTIMIZE a control pulse to implement a π-gate that is robust to
// amplitude noise: minimize the worst-case gate error over an amplitude-error
// range. Bloch-sphere evolution in fixed-point, no float — the optimizer
@ -1333,9 +1331,8 @@ pub use seal_any::seal_artifacts;
// ===========================================================================
// Calibration-graph orchestration — a bring-up campaign as a signed lineage
// (Quantum Machines' QUAlibrate differentiator, made superior: their node graph
// is unsigned; here each node's receipt CHAINS to the previous via
// parent_receipt_hash, so the whole campaign is one portable, verifiable lineage).
// (a node graph of calibrations; here each node's receipt CHAINS to the previous
// via parent_receipt_hash, so the whole campaign is one portable, verifiable lineage).
// ===========================================================================
#[cfg(feature = "quantum_ops")]
@ -1505,7 +1502,7 @@ mod tests {
assert!(receipt.verify());
// the evidence content-addresses the exact sweep the calibrator produced
assert!(receipt.evidence_matches(0, &sweep_bytes(&r)));
// and a re-run reproduces that sweep byte-for-byte (the superiority claim)
// and a re-run reproduces that sweep byte-for-byte (the reproducibility claim)
let r2 = calibrate_rabi(&device, &cfg);
assert!(receipt.evidence_matches(0, &sweep_bytes(&r2)));
}

2
wai/wai-quantum/src/quantum_control.rs
View file

@ -4,7 +4,7 @@
//!
//! The composite-pulse robust control already in [`crate::quantum_cal`] hardens a
//! gate against a *static* amplitude error. Real devices are limited by
//! *correlated, low-frequency* noise — 1/f dephasing above all — and the SOTA
//! *correlated, low-frequency* noise — 1/f dephasing above all — and current
//! tools address it in the frequency domain via the **filter function**: a
//! control's first-order sensitivity `F(ω)` to dephasing noise, so the coherence
//! decay is the overlap `χ = (1/π) ∫ S(ω) F(ω)/ω² dω` of the noise PSD `S(ω)`

8
wai/wai-quantum/src/quantum_mitigate.rs
View file

@ -6,10 +6,8 @@
//! amplified noise, calibrating a readout matrix, taking randomized snapshots),
//! and **(B)** a deterministic numerical back-end — the extrapolation, the matrix
//! inversion, the shadow estimator — that turns the raw counts into the reported
//! number. Half (B) is the entire value of the field, and today it runs in
//! nondeterministic float Python (Mitiq, Boulder Opal) with **no reproducibility
//! guarantee and no provenance over the step**. Nobody signs the classical layer
//! between the QPU and the reported number.
//! number. Run in float, half (B) is reproducible only on the same build, and
//! nothing records the step between the QPU and the reported number.
//!
//! This module is half (B), done the WAI way: pure fixed-point integer numerics,
//! byte-identical on every machine, each producing a signed, content-addressed,
@ -30,7 +28,7 @@
//! are developed against. What is not approximate is the reproducibility of the
//! post-processing and the receipt over it — the property the float stacks lack.
//!
//! For the SOTA ceiling (PEC/PEA, tensor-network mitigation) the receipt's
//! For the heavier estimators (PEC/PEA, tensor-network mitigation) the receipt's
//! `noise_model_hash` is the hook: those estimators consume an externally-learned
//! noise model, and the receipt pins *which* model produced the number without
//! WAI claiming to have learned it.

12
wai/wai-quantum/src/quantum_mps.rs
View file

@ -1,8 +1,7 @@
//! Matrix Product State backend — the classical referee (`wai.quantum.mps`).
//!
//! The 2024–2026 literature's central lesson is that the method which actually
//! beats quantum hardware on "utility-scale" circuits is the **tensor network**:
//! for a state whose entanglement stays bounded, an MPS represents `2ⁿ` amplitudes
//! Published classical simulations of "utility-scale" circuits (2024–2026) use
//! **tensor networks**: for a state whose entanglement stays bounded, an MPS represents `2ⁿ` amplitudes
//! in `O(n · χ²)` numbers and evolves them gate-by-gate, truncating the bond
//! dimension `χ` at each two-qubit gate. This is the backend the dense simulator's
//! cheapest-sufficient cascade hands large, low-entanglement circuits to.
@ -10,9 +9,10 @@
//! Contract, stated honestly and differently from the dense simulator: an MPS with
//! bounded `χ` is **approximate** — the whole point is to discard small Schmidt
//! weight — so byte-exact-vs-analytic is not the promise here. What IS preserved is
//! **determinism**: every arithmetic step uses only IEEE-754 `+ − × ÷ √` (no FMA,
//! no transcendentals), so the result is bit-reproducible on every machine, and the
//! engine reports the **max bond dimension** it used and the **retained weight**
//! **determinism**: the contraction and truncation use only IEEE-754 `+ − × ÷ √`
//! (no FMA). The phase gate's angle is computed with the platform `cos` and `sin`,
//! so the result is bit-reproducible only where the math library returns the same
//! bits for those calls. The engine reports the **max bond dimension** it used and the **retained weight**
//! (the standard truncation-fidelity heuristic). Verified against the byte-exact dense
//! simulator: exact agreement when `χ` is unbounded, graceful, measured degradation
//! when it is squeezed.

47
wai/wai-quantum/src/quantum_ops.rs
View file

@ -2,16 +2,12 @@
//! (extensions/quantum-ops). The verify-half of WAI applied to the quantum
//! processing industry.
//!
//! Every quantum-control stack in the field emits the same three unsigned things
//! — an HDF5 dataset, a JSON state snapshot, and a dashboard/log — and not one
//! signs, content-addresses, ports, or meters the joules of a single operation.
//! The industry already reaches for reproducibility (QM's `__package_versions__`,
//! QBOM bills-of-materials, QCIVET hash-chained audit) but stops exactly short of
//! two axes: **cryptographic signature/portability**, and **measured joules**.
//! A calibration run or a QPU job leaves a dataset, a state snapshot and a log.
//! This module adds a receipt over each operation with two properties:
//! **cryptographic signature/portability** and **measured joules**.
//!
//! This module is those two axes. It does NOT reconstruct anything and does NOT
//! run hardware — the calibration/job is performed by the sink's control system
//! (Quantify / QUAlibrate / Juice / Qubex / a real QPU), exactly as WAI dispatches
//! It does NOT reconstruct anything and does NOT run hardware — the calibration/job
//! is performed by the sink's control system (or a real QPU), exactly as WAI dispatches
//! a decode to a sink's codec. WAI owns the **receipt**: a signed, content-
//! addressed, joule-metered, capability-granted record of what ran, under what
//! device state, at what energy cost, authorized by whom.
@ -203,11 +199,11 @@ impl Evidence {
// ---------------------------------------------------------------------------
/// A signed, joule-metered, grant-authorized record of one calibration/tune-up
/// run. Chains to the prior device state via `parent_receipt_hash` — the signed
/// "digital twin" lineage the vendors keep only as mutable, unsigned data.
/// run. Chains to the prior device state via `parent_receipt_hash`: a signed
/// lineage of device states.
#[derive(Clone, Debug, PartialEq)]
pub struct CalibrationReceipt {
/// Device / QPU identity (e.g. `"qm:opx1000:lab3"`, a wafer/die id).
/// Device / QPU identity (e.g. `"example:lab3-device"`, a wafer/die id).
pub device_id: String,
/// What was calibrated: `"q3"`, `"cz(q3,q4)"`, `"readout:q3"`.
pub target: String,
@ -424,7 +420,7 @@ impl CalibrationReceipt {
/// bound to the exact calibration state the device was in when it ran.
#[derive(Clone, Debug, PartialEq)]
pub struct JobReceipt {
/// Which backend actually ran it: `"ionq:aria-1"`, `"wai.quantum.circuit"`.
/// Which backend actually ran it: `"example:qpu-a"`, `"wai.quantum.circuit"`.
pub backend_id: String,
/// Content hash of the dispatched circuit (e.g. a WQC container / QASM).
pub circuit_hash: [u8; 32],
@ -556,8 +552,7 @@ impl JobReceipt {
/// The cross-link check: this job claims it ran under `cal`'s device state.
/// Confirms `cal` verifies and its id equals this job's `calibration_ref`.
/// A third party can now assert "this result was produced on a device in
/// this attested calibration state" — the property QCIVET names and nobody
/// signs.
/// this attested calibration state" — the property QCIVET names, here signed.
pub fn verify_against_calibration(&self, cal: &CalibrationReceipt) -> bool {
self.calibration_ref == Some(cal.receipt_hash()) && cal.verify() && self.verify()
}
@ -638,11 +633,10 @@ impl JobReceipt {
/// (optional, hash-pinned) noise model that went IN to the mitigated estimate
/// and its error bar that came OUT, plus the method and seed that produced it.
///
/// This is the layer nobody signs. Zero-noise extrapolation, readout inversion,
/// classical shadows, PEC/PEA aggregation — all are deterministic numerics that
/// today run in float Python (Mitiq, Boulder Opal) with no reproducibility
/// guarantee and no provenance over the step between the QPU and the reported
/// number. A `MitigationReceipt` makes `(raw counts, noise-model hash, method,
/// Zero-noise extrapolation, readout inversion, classical shadows, PEC/PEA
/// aggregation — all are deterministic numerics; run in float, their result is
/// reproducible only on the same build, and nothing records the step between the
/// QPU and the reported number. A `MitigationReceipt` makes `(raw counts, noise-model hash, method,
/// seed) → (mitigated estimate ± error, joules)` a byte-exact, content-addressed,
/// budget-gated, auditable object: a quantum result you can take to an auditor.
///
@ -919,9 +913,8 @@ impl MitigationReceipt {
///
/// A real-time QEC decoder is the classical bottleneck of a fault-tolerant
/// machine: it must return a correction for every syndrome round inside the code
/// cycle, and its energy budget is a hard system constraint. Yet no decoder today
/// reports **joules-per-decode**, and none signs the correction it emitted. This
/// receipt does both — `(check-matrix hash, syndrome hash, decoder params) →
/// cycle, and its energy budget is a hard system constraint. This receipt reports
/// **joules-per-decode** and signs the correction the decoder emitted — `(check-matrix hash, syndrome hash, decoder params) →
/// (correction hash, converged, work, joules)`, content-addressed and budget-
/// gated. The decode is byte-exact (a deterministic fixed-point decoder), so the
/// correction reproduces from the receipt and verifies with a public key.
@ -1371,7 +1364,7 @@ impl CompileReceipt {
/// traps — a random, ~half-full pattern — then uses mobile tweezers to rearrange
/// them into a **defect-free** target register before the computation. The plan
/// that does it (which atom moves to which site, in what order) is produced by a
/// combinatorial solver whose optimality and validity nobody signs. This receipt
/// combinatorial solver. This receipt
/// binds the initial occupancy, the target sites, and the move plan to a `success`
/// verdict (every target site filled) and the plan's cost — produced by a
/// **deterministic** solver (Hungarian / LSAP, minimizing total move distance),
@ -1591,7 +1584,7 @@ mod tests {
funds_ceiling: Some(100),
};
let r = CalibrationReceipt::seal(
&key(1), "did:key:lab", "qm:opx1000:lab3", "q3",
&key(1), "did:key:lab", "example:lab3-device", "q3",
content_hash(b"pulse-config-v7"), evid(), 4_200_000, grant, None,
);
assert!(r.verify());
@ -1638,7 +1631,7 @@ mod tests {
100, GrantRef::unbounded("quantum.calibrate"), None,
);
let job = JobReceipt::seal(
&key(4), "lab", "ionq:aria-1",
&key(4), "lab", "example:qpu-a",
content_hash(b"<QASM circuit>"), content_hash(b"{counts:{00:512,11:488}}"),
Some(cal.receipt_hash()), 1000, 250, GrantRef::unbounded("quantum.job.run"), None,
);
@ -1679,7 +1672,7 @@ mod tests {
assert_eq!(CalibrationReceipt::from_json(&cal.to_json()).unwrap(), cal);
let job = JobReceipt::seal(
&key(6), "did:key:z", "ionq:aria-1", content_hash(b"c"), content_hash(b"r"),
&key(6), "did:key:z", "example:qpu-a", content_hash(b"c"), content_hash(b"r"),
Some(cal.receipt_hash()), 2048, 42, GrantRef::unbounded("quantum.job.run"), Some(cal.receipt_hash()),
);
let back = JobReceipt::from_json(&job.to_json()).unwrap();

4
wai/wai-quantum/src/quantum_pauli.rs
View file

@ -1,8 +1,8 @@
//! Sparse-Pauli / Heisenberg propagation — the observable referee
//! (`wai.quantum.pauli`).
//!
//! The method that reproduced IBM's 127-qubit "utility" experiment better and
//! faster on a laptop (Begušić–Chan sparse Pauli dynamics): instead of evolving the
//! Sparse Pauli dynamics (Begušić and Chan, published as a classical simulation
//! of a 127-qubit kicked-Ising "utility" experiment on a laptop): instead of evolving the
//! `2ⁿ`-amplitude *state*, evolve the *observable* in the Heisenberg picture. To
//! compute `⟨0|U†OU|0⟩`, propagate `O` backward through the circuit as a sum of
//! Pauli strings, `O = Σ_P c_P P`. A Clifford gate maps each Pauli to a single

3
wai/wai-quantum/src/quantum_qdata.rs
View file

@ -19,8 +19,7 @@
//!
//! This is an unconditional, exponential separation in *sample* complexity, and it
//! is what makes [[crate::quantum_noise]]'s Bell / Cycle-Benchmarking sampling
//! (`wai.quantum.cal.noise_learn`) exponentially data-efficient — the real moat,
//! surfaced here as a first-class quantum-ML capability. Both estimators are
//! (`wai.quantum.cal.noise_learn`) exponentially data-efficient, surfaced here as a first-class quantum-ML capability. Both estimators are
//! simulated exactly; deterministic given `seed` (reproducible f64).
fn splitmix64(s: &mut u64) -> u64 {

7
wai/wai-quantum/src/quantum_qec.rs
View file

@ -7,7 +7,7 @@
//! code cycle. For quantum LDPC (qLDPC) codes — the high-rate codes that make
//! fault tolerance affordable — the decoder of record is **belief propagation**,
//! and plain BP *fails* on quantum codes: the stabilizer degeneracy and the short
//! cycles in the Tanner graph trap it in symmetric non-solutions. The SOTA fix is
//! cycles in the Tanner graph trap it in symmetric non-solutions. A published fix is
//! **Relay-BP**: an ensemble of BP passes with disordered per-node memory
//! strengths, relayed (warm-started) from one another, which breaks the symmetry.
//!
@ -15,8 +15,7 @@
//! passing in pure fixed-point integer arithmetic — no float, no `tanh`/`atanh` —
//! so a decode is **byte-identical on every machine** and the emitted correction
//! is a portable hash. Every decode seals a signed, joule-metered
//! [`crate::quantum_ops::DecodeReceipt`] carrying **joules-per-decode**, the metric
//! no decoder reports and none signs.
//! [`crate::quantum_ops::DecodeReceipt`] carrying **joules-per-decode**.
//!
//! Min-sum on a single uniform channel is scale-invariant (check node = min,
//! variable node = sum, both homogeneous), so the channel prior is a fixed unit
@ -2472,7 +2471,7 @@ mod tests {
}
}
/// A stabilizer is the opposite case: also undetectable, but it must NOT flip
/// A stabilizer is the converse case: also undetectable, but it must NOT flip
/// the observable. Getting only the first half right would still be wrong.
#[test]
fn dem_says_stabilizers_are_harmless() {

8
wai/wai-quantum/src/quantum_toolchain.rs
View file

@ -1,12 +1,10 @@
//! WAI Quantum Toolchain — the compute core of a classical-hardware quantum IDE
//! (extensions/quantum-sim § Toolchain, `wai.quantum.toolchain`).
//!
//! Everyone teaches quantum circuits-first, inside one vendor's SDK, attached to a
//! cloud QPU, behind a wall of linear algebra. This is the opposite: a
//! framework-neutral toolchain over the byte-exact fixed-point simulator
//! A framework-neutral toolchain over the byte-exact fixed-point simulator
//! ([`crate::quantum`]) that runs entirely on the hardware you already have, with
//! no cloud, no account, no vendor lock-in — and whose every run is reproducible
//! to the bit and sealable into a signed receipt.
//! no cloud service or account — and whose every run is reproducible to the bit
//! and sealable into a signed receipt.
//!
//! Four layers sit on the simulator:
//! - **Algorithm library** — the canonical algorithms (Bell, GHZ, QFT, Grover,

13
wai/wai-quantum/src/quantum_vml.rs
View file

@ -1,12 +1,9 @@
//! Variational quantum machine learning — the interference the thermodynamic
//! crowd left behind (`wai.quantum.vml`).
//! Variational quantum machine learning — interference on classical hardware
//! (`wai.quantum.vml`).
//!
//! The people who built quantum ML and then pivoted to thermodynamic hardware
//! (Coles's barren-plateaus lineage → Normal Computing; Verdon's TensorFlow Quantum
//! → Extropic) moved the *sampling* workload to classical physics — because a
//! Boltzmann/thermodynamic sampler draws from a real, non-negative Gibbs
//! distribution `P(x) ∝ e^{-E(x)}`. What that hardware physically cannot do is
//! **interference**: represent a *complex* amplitude `ψ(x)` whose wrong answers
//! A Boltzmann/thermodynamic sampler draws from a real, non-negative Gibbs
//! distribution `P(x) ∝ e^{-E(x)}`. What such a sampler cannot represent is
//! **interference**: a *complex* amplitude `ψ(x)` whose wrong answers
//! **destructively cancel**. That is the genuinely-quantum resource, and it runs on
//! classical hardware via a state-vector — exactly what this module does.
//!

14
wai/wai-quantum/src/receipt_pins.rs
View file

@ -61,12 +61,16 @@ mod ops {
#[test]
fn calibration_legacy_bytes() {
let r = CalibrationReceipt::seal(
&key(1), "did:key:pin", "qm:opx1000:lab3", "q3", content_hash(b"cfg"), evid(),
&key(1), "did:key:pin", "example:lab3-device", "q3", content_hash(b"cfg"), evid(),
4_200_000, grant("quantum.calibrate"), None,
);
assert!(r.verify());
assert_eq!(hx(&r.receipt_hash()), "17e5f4e95928251c01735db5390ca4ce9d0970ce78132b426112de41bfe42c7a");
assert_eq!(json_hash(&r.to_json()), "5d30fcc5c28a9a6f208dfdcd2e8cb6674927e6c73220d69cd32aaaa002d63185");
// Recomputed 2026-09-30 when the device id became the neutral
// "example:lab3-device": this receipt sealed by the code at 1c2af088 (the
// commit before the acquisition class, d009db90), with that literal. The same
// code with the previous literal reproduced the previous pins exactly.
assert_eq!(hx(&r.receipt_hash()), "43326592236b428cb98124d94987b7c59c7ab09b2984ad3a4f622384992c2d03");
assert_eq!(json_hash(&r.to_json()), "6803e8c6b38d2ce866f88a34313bd551272ad247646e0f258fa61142e87ee9f5");
}
#[test]
@ -323,7 +327,7 @@ mod ops_labels {
#[test]
fn calibration_label() {
let r = CalibrationReceipt::seal(
&key(1), "did:key:pin", "qm:opx1000:lab3", "q3", content_hash(b"cfg"),
&key(1), "did:key:pin", "example:lab3-device", "q3", content_hash(b"cfg"),
vec![Evidence { kind: "rb".into(), blob_hash: content_hash(b"<rb>"), summary: "s".into() }],
4_200_000, grant("quantum.calibrate"), None,
);
@ -383,7 +387,7 @@ mod ops_labels {
fn a_job_cross_links_to_a_calibration_in_either_form() {
use crate::quantum_energy::{Labelled, MaybeLabelled};
let cal = CalibrationReceipt::seal(
&key(1), "did:key:pin", "qm:opx1000:lab3", "q3", content_hash(b"cfg"),
&key(1), "did:key:pin", "example:lab3-device", "q3", content_hash(b"cfg"),
vec![Evidence { kind: "rb".into(), blob_hash: content_hash(b"<rb>"), summary: "s".into() }],
4_200_000, grant("quantum.calibrate"), None,
);

52
wai/wai-rs/Cargo.toml
View file

@ -4,7 +4,7 @@ version = "0.3.0"
edition = "2024"
rust-version = "1.97"
license = "Apache-2.0"
description = "WAI — Web AI media transport + execution, and a deterministic QUANTUM stack: byte-exact circuit simulation with statevector / stabilizer / tensor-network / sparse-Pauli backends, error mitigation, qLDPC decoding, noise learning, circuit-equivalence proofs and signed energy-accounted receipts — identical results native and in cloud WASM. Plus a capability-as-requirement envelope and zeroth-condition wrappers around SOTA codecs (AVIF / JPEG-XL / PNG / Opus / FLAC / AV1 / zstd / LZMA)."
description = "WAI — Web AI media transport + execution, and a deterministic QUANTUM stack: byte-exact circuit simulation with statevector / stabilizer / tensor-network / sparse-Pauli backends, error mitigation, qLDPC decoding, noise learning, circuit-equivalence proofs and signed energy-accounted receipts — identical results native and in cloud WASM. Plus a capability-as-requirement envelope and zeroth-condition wrappers around standard codec libraries (AVIF / JPEG-XL / PNG / Opus / FLAC / AV1 / zstd / LZMA)."
repository = "https://git.transaction.science/Transaction-Science/open-standards"
homepage = "https://wai.transaction.science"
documentation = "https://wai.transaction.science/quantum-ide"
@ -59,7 +59,7 @@ xz2 = { version = "0.1", optional = true } # liblzma bindings (XZ/LZM
byteorder = { version = "1", optional = true }
# ---- neural runtime (optional, behind the `neural` feature) ----
# `ort` wraps Microsoft's onnxruntime — same ORT version (1.26) the
# `ort` wraps ONNX Runtime — same ORT version (1.26) the
# browser-side wai-web demo uses, so models built with our exporters
# load identically here. The `download-binaries` feature fetches the
# right libonnxruntime build at compile time; no manual brew install.
@ -259,20 +259,19 @@ ternary_receipt = ["ternary", "world"]
quantum_ops = ["dep:wai-quantum", "wai-quantum/quantum_ops"]
# Deterministic autonomous CALIBRATION engines (extensions/quantum-ops § Calibration):
# a fixed-point qubit-device simulator + autonomous tune-up algorithms (Rabi π-pulse
# to start) — the software capability the quantum-control vendors sell, byte-exact and
# reproducible, sealing a wai.quantum.calibration receipt per run. Pulls quantum_ops.
# to start), byte-exact and reproducible, sealing a wai.quantum.calibration receipt per run. Pulls quantum_ops.
quantum_cal = ["quantum_ops", "wai-quantum/quantum_cal"]
# Deterministic quantum ERROR MITIGATION (extensions/quantum-ops § Mitigation,
# `wai.quantum.mitigate`): the classical post-processing layer between raw QPU
# counts and the reported number — zero-noise extrapolation, tensored readout
# inversion, classical shadows — in pure fixed-point, byte-exact, each sealing a
# signed, joule-metered MitigationReceipt. The layer nobody signs. Pulls quantum_ops.
# signed, joule-metered MitigationReceipt over the step. Pulls quantum_ops.
quantum_mitigate = ["quantum_ops", "wai-quantum/quantum_mitigate"]
# Deterministic quantum LDPC DECODING (extensions/quantum-ops § Decoding,
# `wai.quantum.decode`): a fixed-point Relay-BP (normalized min-sum) decoder for
# qLDPC codes (toric + bivariate-bicycle), byte-exact on every machine, each
# decode sealing a signed, joule-metered DecodeReceipt carrying joules-per-decode
# — the metric no decoder reports and none signs. Pulls quantum_ops.
# signed with the correction. Pulls quantum_ops.
quantum_qec = ["quantum_ops", "wai-quantum/quantum_qec"]
# Deterministic FILTER-FUNCTION control + NOISE SPECTROSCOPY (extensions/quantum-ops
# § Calibration, frequency domain): the filter-function overlap χ = ∫ S(ω)F(ω)/ω²
@ -318,8 +317,8 @@ quantum_mps = ["quantum", "wai-quantum/quantum_mps"]
quantum_stabilizer = ["quantum", "wai-quantum/quantum_stabilizer"]
# Sparse-Pauli / Heisenberg propagation (extensions/quantum-sim § Pauli,
# `wai.quantum.pauli`): the observable referee — propagate an observable backward
# through the circuit as a truncated sum of Pauli strings (the Begušić–Chan method
# that reproduced IBM's utility experiment on a laptop). Computes ⟨O⟩ on large
# through the circuit as a truncated sum of Pauli strings (Begušić and Chan's sparse
# Pauli dynamics). Computes ⟨O⟩ on large
# low-magic circuits where no state vector exists; grows with magic, not qubits.
quantum_pauli = ["quantum", "wai-quantum/quantum_pauli"]
quantum_qir = ["quantum_toolchain", "wai-quantum/quantum_qir"]
@ -428,10 +427,12 @@ name = "jpeg_test"
path = "examples/jpeg_test.rs"
required-features = ["codecs"]
# `encode` and `decode` need the `codecs` feature and are compiled out without it;
# every other command (wrap, unwrap, inspect, capabilities, registry export) builds
# with --no-default-features, which is how CI runs it.
[[bin]]
name = "wai"
path = "src/bin/wai.rs"
required-features = ["codecs"]
# The quantum stack as a CLI. Pure Rust, no host bindings — the same source builds
# for native AND `wasm32-wasip2`, producing byte-identical output under wasmtime.
@ -560,6 +561,39 @@ path = "tests/jpegai_decode_symbols.rs"
harness = false
required-features = ["neural_int"]
# Feature-gated integration tests that CI names with `--test`. Declared with their
# features so that naming one without them stops cargo with an error, instead of
# compiling the `#![cfg(feature)]` file to an empty binary that reports ok.
[[test]]
name = "int_audio"
path = "tests/int_audio.rs"
required-features = ["audio_int"]
[[test]]
name = "int_splat"
path = "tests/int_splat.rs"
required-features = ["splat_int"]
[[test]]
name = "int_kernels_vectors"
path = "tests/int_kernels_vectors.rs"
required-features = ["neural_int", "video_int", "audio_int", "world_int", "splat_int"]
[[test]]
name = "int_kernels_geometry"
path = "tests/int_kernels_geometry.rs"
required-features = ["neural_int"]
[[test]]
name = "int_mlicpp"
path = "tests/int_mlicpp.rs"
required-features = ["neural_int"]
[[test]]
name = "int_mlicpp_preflight"
path = "tests/int_mlicpp_preflight.rs"
required-features = ["neural_int"]
[dev-dependencies]
serde_json = { version = "1", features = ["preserve_order"] } # construct manifests in tests

4
wai/wai-rs/build.rs
View file

@ -66,8 +66,8 @@ fn link_libflac() {
#[cfg(target_os = "macos")]
{
let candidates = [
"/opt/homebrew/opt/flac/lib", // Apple Silicon
"/usr/local/opt/flac/lib", // Intel macOS
"/opt/homebrew/opt/flac/lib", // arm64 macOS (Homebrew)
"/usr/local/opt/flac/lib", // x86_64 macOS (Homebrew)
];
for p in candidates {
if std::path::Path::new(p).exists() {

7
wai/wai-rs/include/wai.h
View file

@ -470,6 +470,9 @@ int wai_video_av1_decode(const uint8_t *bytes,
* Builds a complete WAI file from a codec-produced payload + manifest
* fields. `manifest_json` is a UTF-8 JSON object overriding/supplying
* any fields; `payload` is the bytes from one of the codec encoders.
* Refuses (`WAI_ERR_INVALID_ARG`) a manifest that names a candidate capability
* as its capability or fallback, which an encoder MUST NOT emit (SPEC §5,
* *Candidate capabilities*): it emits through `Wai::to_emit_bytes`.
*/
int wai_envelope_pack(const char *manifest_json,
@ -491,6 +494,10 @@ int wai_envelope_pack(const char *manifest_json,
* - `payload_lens` is the matching array of `n` lengths.
* - The i-th payload pairs with `manifest.renditions[i]`.
*
* Refuses (`WAI_ERR_INVALID_ARG`) a table with a component, in either role,
* whose capability is a candidate (SPEC §5, *Candidate capabilities*): it emits
* through `WaiMulti::to_emit_bytes`.
*
* # Safety
* All array pointers must point to valid memory of the declared length.
*/

209
wai/wai-rs/src/bin/wai.rs
View file

@ -1,14 +1,21 @@
//! WAI CLI — encode/decode through registered zeroth-condition codecs
//! (AVIF/JPEG-XL/PNG/Opus/FLAC/AV1/zstd/XZ). Designed for scripting and
//! quick verification; production paths go through the Rust API or the
//! C FFI.
//! WAI CLI — carry any registered capability's bytes in an envelope (`wrap`,
//! `unwrap`, `inspect`), list and export the registry (`capabilities`,
//! `registry export`), and, in a build with the `codecs` feature, encode and
//! decode through the natively decoded standard codecs
//! (AVIF/JPEG-XL/PNG/Opus/FLAC/AV1/zstd/XZ). Designed for scripting and quick
//! verification; production paths go through the Rust API or the C FFI.
//!
//! Every command except `encode` and `decode` builds without the `codecs`
//! feature (no system codec libraries), which is how CI runs it.
use std::env;
use std::fs;
use std::process::ExitCode;
#[cfg(feature = "codecs")]
use wai::codecs::{audio, image, text, video};
use wai::{Conditioning, Manifest, ModelRequirement, Wai};
#[cfg(feature = "codecs")]
use wai::*;
fn die(msg: impl std::fmt::Display) -> ExitCode {
@ -24,7 +31,11 @@ fn usage() {
eprintln!(" [--companion <cap> <payload>]... WAI2: companions bound to the payload");
eprintln!("wai unwrap <in.wai> [-o payload] [--component <i>] carry/extract ANY registered codec's bytes");
eprintln!("wai capabilities list registered capabilities + tier + license + replicate class + role");
eprintln!("wai registry export [--root <wai-dir>] [--check]");
eprintln!(" write (or, with --check, compare) registry/capabilities.json, its schema,");
eprintln!(" vocab/v2.* and the generated table of REGISTRY.md");
eprintln!();
eprintln!("encode and decode need a build with the `codecs` feature.");
eprintln!("codecs: png jpeg avif jxl jxl-lossless opus flac av1 av1-lossless zstd xz");
eprintln!("inputs:");
eprintln!(" image codecs: .rgb (with .meta containing 'H W' ASCII) or PNG/JPEG file");
@ -33,6 +44,20 @@ fn usage() {
eprintln!(" text codecs: any file");
}
/// The capabilities this binary decodes natively: the crate's sink set
/// (`wai::sink_capabilities`) in a build with the `codecs` feature, and none
/// without it, where `decode` is unavailable.
fn native_capabilities() -> &'static [&'static str] {
#[cfg(feature = "codecs")]
{
wai::sink_capabilities()
}
#[cfg(not(feature = "codecs"))]
{
&[]
}
}
fn opt<'a>(args: &'a [String], flag: &str) -> Option<&'a str> {
args.iter().position(|a| a == flag)
.and_then(|i| args.get(i + 1).map(|s| s.as_str()))
@ -56,6 +81,7 @@ fn companion_args(args: &[String]) -> Result<Vec<(String, String)>, String> {
Ok(pairs)
}
#[cfg(feature = "codecs")]
fn read_meta(path: &str) -> Result<Vec<u32>, String> {
let s = fs::read_to_string(format!("{path}.meta"))
.map_err(|e| format!("missing {path}.meta: {e}"))?;
@ -63,6 +89,17 @@ fn read_meta(path: &str) -> Result<Vec<u32>, String> {
.collect()
}
#[cfg(not(feature = "codecs"))]
fn encode(_codec: &str, _input: &str, _out: &str, _q: u8) -> Result<(), String> {
Err("this `wai` was built without the codecs feature: encode is unavailable".into())
}
#[cfg(not(feature = "codecs"))]
fn decode(_input: &str, _out: &str) -> Result<(), String> {
Err("this `wai` was built without the codecs feature: decode is unavailable".into())
}
#[cfg(feature = "codecs")]
fn encode(codec: &str, input: &str, out: &str, q: u8) -> Result<(), String> {
let (cap, kind, payload): (&str, &str, Vec<u8>) = match codec {
"png" => {
@ -161,16 +198,9 @@ fn encode(codec: &str, input: &str, out: &str, q: u8) -> Result<(), String> {
c => return Err(format!("unknown codec '{c}'")),
};
let media = match cap {
c if c.starts_with("wai.image.") => "image",
c if c.starts_with("wai.audio.") => "audio",
c if c.starts_with("wai.video.") => "video",
c if c.starts_with("wai.text.") => "text",
_ => "binary",
};
let manifest = Manifest {
wai: "1.0".into(),
media: media.into(),
media: media_for(cap).into(),
intent: "replicate".into(),
model_requirement: ModelRequirement {
capability: cap.into(),
@ -186,6 +216,7 @@ fn encode(codec: &str, input: &str, out: &str, q: u8) -> Result<(), String> {
Ok(())
}
#[cfg(feature = "codecs")]
fn decode(input: &str, out: &str) -> Result<(), String> {
let wai = Wai::read(input).map_err(|e| e.to_string())?;
// SPEC §4 dispatch, shared with `inspect` so the two agree: the primary
@ -268,7 +299,7 @@ fn inspect(path: &str) -> Result<(), String> {
let bytes = std::fs::read(path).map_err(|e| e.to_string())?;
let version = wai::container::detect_version(&bytes).map_err(|e| e.to_string())?;
let known: std::collections::HashSet<&str> =
wai::sink_capabilities().iter().copied().collect();
native_capabilities().iter().copied().collect();
let size = bytes.len();
println!("wai inspect — {path}");
println!(" envelope size: {size:>10} B");
@ -280,10 +311,11 @@ fn inspect(path: &str) -> Result<(), String> {
let w = wai::container::Wai::from_bytes(&bytes).map_err(|e| e.to_string())?;
let cap = w.capability();
let fb = w.fallback();
// A companion-only capability is never dispatched standalone (SPEC §5),
// so it resolves neither as the capability nor as the fallback. The
// verdict comes from the same dispatch `decode` runs.
let usable = |c: &str| known.contains(c) && !wai::codecs::is_companion_only(c);
// A companion-only capability is never dispatched standalone, and a
// candidate is unsupported (SPEC §5), so neither resolves as the
// capability or the fallback. The verdict comes from the same dispatch
// `decode` runs.
let usable = |c: &str| known.contains(c) && wai::codecs::is_dispatchable(c);
let cap_ok = usable(cap);
let fb_ok = fb.map(usable);
let chosen = w.dispatch_capability(|c| known.contains(c));
@ -302,6 +334,7 @@ fn inspect(path: &str) -> Result<(), String> {
println!(" resolution: {}",
if fb_ok.unwrap() { "\u{2713} this sink advertises it" }
else if wai::codecs::is_companion_only(f) { "\u{2717} companion-only, never dispatched standalone" }
else if wai::codecs::is_candidate(f) { "\u{2717} a candidate, unsupported until its payload is pinned" }
else { "\u{2717} NOT installed either" });
}
let verdict = match chosen {
@ -322,6 +355,9 @@ fn inspect(path: &str) -> Result<(), String> {
Ok(()) => println!(" component roles: \u{2713} conform (SPEC \u{a7}5)"),
Err(e) => println!(" component roles: \u{2717} {e}"),
}
if let Err(e) = w.check_candidates() {
println!(" candidates: \u{2717} {e}");
}
} else {
// WAI2 inspection
let m = wai::container::WaiMulti::from_bytes(&bytes).map_err(|e| e.to_string())?;
@ -358,52 +394,38 @@ fn inspect(path: &str) -> Result<(), String> {
Ok(()) => println!(" component roles: \u{2713} conform (SPEC \u{a7}5, \u{a7}7.1)"),
Err(e) => println!(" component roles: \u{2717} {e}"),
}
if let Err(e) = m.check_candidates() {
println!(" candidates: \u{2717} {e}");
}
}
println!();
println!(" this sink advertises {} capabilities:", known.len());
let mut ks: Vec<&&str> = known.iter().collect(); ks.sort();
for k in ks { println!(" - {k}"); }
if known.is_empty() {
println!(" this `wai` was built without the codecs feature: it decodes nothing natively,");
println!(" so every verdict above is for a sink that advertises nothing");
} else {
println!(" this sink advertises {} capabilities:", known.len());
let mut ks: Vec<&&str> = known.iter().collect(); ks.sort();
for k in ks { println!(" - {k}"); }
}
Ok(())
}
/// The media class for a capability prefix (manifest `media` field).
/// The manifest `media` value for a capability: the registry's media class
/// (`codecs::capability_media`), or `binary` for a string it does not class.
fn media_for(cap: &str) -> &'static str {
match cap {
c if c.starts_with("wai.image.") => "image",
c if c.starts_with("wai.audio.") => "audio",
c if c.starts_with("wai.video.") => "video",
c if c.starts_with("wai.text.") => "text",
c if c.starts_with("wai.mesh.") || c.starts_with("wai.pc.") || c.starts_with("wai.splat.") => "model",
c if c.starts_with("wai.haptic.") => "haptic",
c if c.starts_with("wai.meta.") => "metadata",
c if c.starts_with("wai.world.") || c.starts_with("wai.film.")
|| c.starts_with("wai.avatar.") || c.starts_with("wai.feed.") => "interactive",
_ => "binary",
}
wai::codecs::capability_media(cap).map_or("binary", wai::codecs::Media::label)
}
/// How dispatch treats a capability's payload: a standalone payload, a
/// companion applied with a base rendition, or an enhancement layer bound to
/// exactly one base encode (SPEC §5, §7.1).
fn role_str(cap: &str) -> &'static str {
if wai::codecs::is_enhancement_layer(cap) {
"enhancement"
} else if wai::codecs::is_companion_only(cap) {
"companion"
} else {
"standalone"
}
wai::codecs::capability_role(cap).unwrap_or("standalone")
}
fn license_str(cap: &str) -> &'static str {
match wai::codecs::capability_license(cap) {
Some(wai::codecs::LicenseClass::RoyaltyFree) => "royalty-free",
Some(wai::codecs::LicenseClass::PatentPool) => "patent-pool",
Some(wai::codecs::LicenseClass::Proprietary) => "proprietary",
Some(wai::codecs::LicenseClass::NonCommercial) => "non-commercial",
None => "?",
}
wai::codecs::capability_license(cap).map_or("?", wai::codecs::LicenseClass::label)
}
/// `wai wrap <capability> <input> [-o out.wai]` — carry ANY registered
@ -427,12 +449,23 @@ fn license_str(cap: &str) -> &'static str {
/// each companion's. It takes no `--fallback`. Either way the
/// envelope must pass the role check before it is written: a companion-only
/// capability is refused as a `WAI1` capability or fallback, and a `WAI2` table
/// that breaks the binding rules is refused.
/// that breaks the binding rules is refused. A candidate capability (SPEC §5,
/// *Candidate capabilities*) is refused in every place — capability, fallback or
/// companion — and nothing is written.
fn wrap(cap: &str, input: &str, out: &str, fallback: Option<&str>,
intent: Option<&str>, companions: &[(String, String)]) -> Result<(), String> {
if !wai::codecs::is_known_capability(cap) {
return Err(format!("'{cap}' is not a registered capability (see `wai capabilities`)"));
}
// A candidate is refused before any other rule is applied, in every place the
// envelope would name it (SPEC §5, Candidate capabilities).
if companions.is_empty() {
wai::container::check_candidates_v1(cap, fallback)
} else {
wai::container::check_component_candidates(
std::iter::once(cap).chain(companions.iter().map(|(c, _)| c.as_str())))
}
.map_err(|e| e.to_string())?;
let intent = wai::codecs::unpinned_envelope_intent(cap, intent).map_err(|why| format!(
"not writing intent=replicate over {cap}: {why} (SPEC \u{a7}7). `wai wrap` pins no \
prior; pass --intent <intent> to carry the payload without a reconstruction claim \
@ -466,7 +499,9 @@ fn wrap(cap: &str, input: &str, out: &str, fallback: Option<&str>,
target: serde_json::Value::Null,
};
let envelope = Wai::new(manifest, payload);
// The reference tool does not write an envelope that breaks the role rules.
// The reference tool does not write an envelope that names a candidate, or
// one that breaks the role rules.
envelope.check_candidates().map_err(|e| e.to_string())?;
envelope.check_component_roles().map_err(|e| match e {
wai::container::RoleError::CompanionOnlyAsCapability { .. } => format!(
"{e} (SPEC \u{a7}5): `wai wrap <base-cap> <base-payload> --companion {cap} <payload>`"),
@ -510,6 +545,7 @@ fn wrap_with_companions(cap: &str, payload: Vec<u8>, out: &str, fallback: Option
}
let envelope = wai::container::WaiMulti::new(
media_for(cap), intent, serde_json::Value::Null, components);
envelope.check_candidates().map_err(|e| e.to_string())?;
envelope.check_component_roles()
.map_err(|e| format!("{e} (SPEC \u{a7}5, \u{a7}7.1)"))?;
let n = envelope.write(out).map_err(|e| e.to_string())?;
@ -549,7 +585,7 @@ fn unwrap(input: &str, out: Option<&str>, component: Option<&str>) -> Result<(),
let w = Wai::from_bytes(&bytes).map_err(|e| e.to_string())?;
let cap = w.capability().to_string();
let tier = wai::codecs::capability_tier(&cap).map(|t| format!("{t:?}")).unwrap_or_else(|| "?".into());
let native = wai::codecs::sink_capabilities().contains(&cap.as_str());
let native = native_capabilities().contains(&cap.as_str());
println!("envelope: {input}");
println!(" capability {cap}");
println!(" media {}", w.media());
@ -562,6 +598,9 @@ fn unwrap(input: &str, out: Option<&str>, component: Option<&str>) -> Result<(),
Ok(()) => println!(" conformance \u{2713} component roles (SPEC \u{a7}5)"),
Err(e) => println!(" conformance \u{2717} {e}"),
}
if let Err(e) = w.check_candidates() {
println!(" conformance \u{2717} {e}");
}
println!(" payload {} bytes", w.payload.len());
if let Some(out) = out {
std::fs::write(out, &w.payload).map_err(|e| format!("write {out}: {e}"))?;
@ -587,6 +626,9 @@ fn unwrap_component(input: &str, bytes: &[u8], out: Option<&str>,
Ok(()) => println!(" conformance \u{2713} component roles (SPEC \u{a7}5, \u{a7}7.1)"),
Err(e) => println!(" conformance \u{2717} {e}"),
}
if let Err(e) = m.check_candidates() {
println!(" conformance \u{2717} {e}");
}
println!(" component [{idx}] {} · {} bytes", meta.capability, payload.len());
if let Some(out) = out {
std::fs::write(out, payload).map_err(|e| format!("write {out}: {e}"))?;
@ -597,7 +639,7 @@ fn unwrap_component(input: &str, bytes: &[u8], out: Option<&str>,
fn capabilities() -> Result<(), String> {
let native: std::collections::HashSet<&str> =
wai::codecs::sink_capabilities().iter().copied().collect();
native_capabilities().iter().copied().collect();
let caps = wai::codecs::known_capabilities();
let rf = caps.iter().filter(|c| wai::codecs::is_royalty_free(c)).count();
let nie = caps.iter().filter(|c| wai::codecs::is_neural_integer_exact(c)).count();
@ -606,19 +648,16 @@ fn capabilities() -> Result<(), String> {
"wai capabilities — {} registered ({rf} royalty-free; {nie}/{neural} neural integer-exact)",
caps.len()
);
if native.is_empty() {
println!(" (built without the codecs feature: this binary decodes nothing natively)");
}
println!(" {:<26} {:<11} {:<13} {:<14} {:<21} {}", "capability", "fidelity", "decode", "license", "replicate", "role");
println!(" {:-<26} {:-<11} {:-<13} {:-<14} {:-<21} {:-<11}", "", "", "", "", "", "");
for &c in caps {
let tier = wai::codecs::capability_tier(c)
.map(|t| format!("{t:?}")).unwrap_or_else(|| "?".into());
let decode = if native.contains(c) { "native" } else { "sink-supplied" };
let license = match wai::codecs::capability_license(c) {
Some(wai::codecs::LicenseClass::RoyaltyFree) => "royalty-free",
Some(wai::codecs::LicenseClass::PatentPool) => "patent-pool",
Some(wai::codecs::LicenseClass::Proprietary) => "proprietary",
Some(wai::codecs::LicenseClass::NonCommercial) => "non-commercial",
None => "?",
};
let license = license_str(c);
let replicate = wai::codecs::capability_replicate_class(c)
.map(wai::codecs::ReplicateClass::label)
.unwrap_or("?");
@ -627,6 +666,55 @@ fn capabilities() -> Result<(), String> {
Ok(())
}
/// `wai registry export [--root <wai-dir>] [--check]`: write the registry export
/// (`codecs::registry::export_files`) and the generated table of `REGISTRY.md`
/// under `<wai-dir>` (default: the `wai/` directory this crate was built from).
/// With `--check`, write nothing: compare, and report every file that differs.
/// `Ok(true)` when everything matches or was written.
fn registry(args: &[String]) -> Result<bool, String> {
use std::path::PathBuf;
if args.first().map(String::as_str) != Some("export") {
return Err("usage: wai registry export [--root <wai-dir>] [--check]".into());
}
let check = args.iter().any(|a| a == "--check");
let root = opt(args, "--root").map(PathBuf::from)
.unwrap_or_else(|| PathBuf::from(env!("CARGO_MANIFEST_DIR")).join(".."));
let mut files: Vec<(String, String)> = wai::codecs::registry::export_files()
.into_iter().map(|(p, c)| (p.to_string(), c)).collect();
let md_path = root.join("REGISTRY.md");
let md = fs::read_to_string(&md_path).map_err(|e| format!("read {}: {e}", md_path.display()))?;
let spliced = wai::codecs::registry::splice_registry_md(&md).ok_or_else(|| format!(
"{} lacks the generated-table markers", md_path.display()))?;
files.push(("REGISTRY.md".into(), spliced));
let mut stale = Vec::new();
for (rel, want) in &files {
let path = root.join(rel);
let have = fs::read_to_string(&path).ok();
if have.as_deref() == Some(want.as_str()) {
continue;
}
if check {
stale.push(rel.clone());
} else {
if let Some(dir) = path.parent() {
fs::create_dir_all(dir).map_err(|e| format!("create {}: {e}", dir.display()))?;
}
fs::write(&path, want).map_err(|e| format!("write {}: {e}", path.display()))?;
println!("wrote {}", path.display());
}
}
if check && !stale.is_empty() {
eprintln!("the registry export is stale: {} differ from the registry in wai-rs/src/codecs/registry.rs",
stale.join(", "));
eprintln!("regenerate: cd wai/wai-rs && cargo run --no-default-features --bin wai -- registry export");
return Ok(false);
}
if check {
println!("registry export: up to date");
}
Ok(true)
}
fn main() -> ExitCode {
let args: Vec<String> = env::args().skip(1).collect();
if args.is_empty() {
@ -652,6 +740,11 @@ fn main() -> ExitCode {
opt(&args[2..], "--component")),
"inspect" if args.len() >= 2 => inspect(&args[1]),
"capabilities" => capabilities(),
"registry" => match registry(&args[1..]) {
Ok(true) => Ok(()),
Ok(false) => return ExitCode::from(1),
Err(e) => Err(e),
},
_ => { usage(); return ExitCode::from(2); }
};
match r {

4
wai/wai-rs/src/bin/wai_int_synth_vectors.rs
View file

@ -14,8 +14,8 @@
//! Each vector is a realistic multi-layer integer decoder: a committed
//! `<name>.int_synth.json` (the model) + a `<name>.bin` payload (header +
//! int8 latents) + a frozen BLAKE3 of the decoded RGB. `verify` re-decodes
//! and checks the hash — on ubuntu x86-64, ubuntu arm64, macOS arm64, and
//! Windows, every push.
//! and checks the hash — in CI on Linux x86_64, Linux arm64 and macOS arm64,
//! on every push that touches it.
//!
//! wai_int_synth_vectors gen <dir> | verify <dir>

2
wai/wai-rs/src/bin/wai_quantum_meter.rs
View file

@ -2,7 +2,7 @@
//! `wai.quantum.circuit` reconstruction's energy (extensions/quantum-sim § Receipt).
//!
//! Measures the **real CPU energy** a classical sink spends reconstructing a WQC
//! circuit's statevector, with no elevated privileges. On Apple Silicon the SoC
//! circuit's statevector, with no elevated privileges. On arm64 macOS the SoC
//! energy counters are readable through IOReport — the same source `macmon`/
//! `asitop` use — so this meter samples power via `macmon` (the installed,
//! audited IOReport reader), brackets a tight reconstruction burst against an

4
wai/wai-rs/src/bin/wai_quantum_ops_vectors.rs
View file

@ -34,7 +34,7 @@ const CIRCUIT: &[u8] = b"OPENQASM 3.0; qubit[2] q; h q[0]; cx q[0], q[1];";
const RESULT: &[u8] = br#"{"counts":{"00":512,"11":488}}"#;
const LAB: &str = "did:key:demo-lab";
const DEVICE: &str = "qm:opx1000:lab3";
const DEVICE: &str = "example:lab3-device";
fn cal_q3() -> CalibrationReceipt {
let ev = vec![Evidence {
@ -79,7 +79,7 @@ fn job_bell(cal_ref: [u8; 32]) -> JobReceipt {
funds_ceiling: Some(50),
};
JobReceipt::seal(
&key(0x22), LAB, "ionq:aria-1", content_hash(CIRCUIT), content_hash(RESULT),
&key(0x22), LAB, "example:qpu-a", content_hash(CIRCUIT), content_hash(RESULT),
Some(cal_ref), 1000, 250, grant, None,
)
}

2
wai/wai-rs/src/bin/wai_video_meter.rs
View file

@ -2,7 +2,7 @@
//! the integer inter-frame video codec.
//!
//! Measures the **real CPU energy** a sink spends decoding an integer video
//! codestream to frames, with no elevated privileges. On Apple Silicon the SoC
//! codestream to frames, with no elevated privileges. On arm64 macOS the SoC
//! energy counters are readable through IOReport — the same source `macmon`/
//! `asitop` use — so this meter samples power via `macmon`, brackets a tight
//! decode burst against an idle baseline, and reports the **marginal** CPU

2
wai/wai-rs/src/bin/wai_world_meter.rs
View file

@ -1,7 +1,7 @@
//! `wai_world_meter` — the native sink-metering path for §8 joules.
//!
//! Measures the **real CPU energy** a sink spends replaying a sealed
//! `wai.world.replay`, with no elevated privileges. On Apple Silicon
//! `wai.world.replay`, with no elevated privileges. On arm64 macOS
//! the SoC energy counters are readable through IOReport — the same
//! source `macmon`/`asitop` use — so this meter samples power via
//! `macmon` (the installed, audited IOReport reader), brackets a

2
wai/wai-rs/src/claim_receipt.rs
View file

@ -2319,7 +2319,7 @@ mod corpus {
// on the choices step 3 fixes for a key or R of small order.
//
// `under_key` is the exception. Nothing can sign for the large-order point it uses, so
// it asserts the opposite — its signature fails the equation under any decoding — and
// it asserts the reverse — its signature fails the equation under any decoding — and
// its vector pins only the reason step 3 gives.
/// The identity point's encoding, which is also the encoding of R = [0]B.

4
wai/wai-rs/src/codecs/audio.rs
View file

@ -1,4 +1,4 @@
//! Audio codec wrappers. Opus (lossy, modern SOTA) + FLAC (lossless,
//! Audio codec wrappers. Opus (lossy) + FLAC (lossless,
//! universal). Mono only for v1; multichannel is a follow-up.
//!
//! Sample format: f32 in [-1, 1] for the WAI-facing API; codecs may
@ -18,7 +18,7 @@ impl<E: std::fmt::Display> From<E> for AudioError {
pub type Result<T> = std::result::Result<T, AudioError>;
// ---- Opus (lossy, modern SOTA) --------------------------------------
// ---- Opus (lossy) ---------------------------------------------------
// Opus only supports specific sample rates (8/12/16/24/48 kHz) and
// fixed frame sizes. We frame at 20 ms (the Opus default) and resample
// upstream if needed (caller's job).

375
wai/wai-rs/src/codecs/glossary_check.rs Normal file
View file

@ -0,0 +1,375 @@
//! Holds the codec glossary (`wai/glossary/codecs.json`) to its schema
//! (`wai/glossary/SCHEMA.md`) and to the registry. Test-only.
//!
//! Two tables here are the reviewed decisions; the test checks the JSON against
//! them and against the registry functions:
//!
//! - [`GLOSSARY_LINKS`]: which glossary entry's bitstream each linked capability
//! carries. The `wai` link means "this capability decodes the codec this entry
//! describes". A reviewer judges each row once.
//! - [`GLOSSARY_NO_ENTRY`]: every payload or companion capability that no entry
//! links, with the reason.
use super::registry::Kind;
use super::*;
use serde_json::Value;
/// The `schema_version` the file declares (SCHEMA.md, *Top-level shape*).
const GLOSSARY_SCHEMA_VERSION: u64 = 2;
/// The fields every entry has, in schema order. `wai` is the one optional field.
const GLOSSARY_FIELDS: &[&str] = &["name", "year", "body", "modality", "era", "status", "lineage", "desc"];
const GLOSSARY_ENUMS: &[(&str, &[&str])] = &[
("modality", &["image", "video", "audio", "speech", "text", "volumetric", "haptic", "metadata"]),
("era", &["classical", "neural"]),
("status", &["success", "niche", "superseded", "failed", "academic", "emerging"]),
("lineage", &["industry", "standards", "academic", "opensource"]),
];
/// `(entry name, capability)`: the committed link table.
const GLOSSARY_LINKS: &[(&str, &str)] = &[
("JPEG", CAP_IMAGE_JPEG),
("JPEG 2000", CAP_IMAGE_JP2),
("AVIF", CAP_IMAGE_AVIF),
("JPEG XL", CAP_IMAGE_JXL),
("PNG", CAP_IMAGE_PNG),
("JPEG AI", CAP_IMAGE_JPEGAI),
("H.264 / AVC", CAP_VIDEO_AVC),
("HEVC / H.265", CAP_VIDEO_HEVC),
("VVC / H.266", CAP_VIDEO_VVC),
("LCEVC", CAP_VIDEO_LCEVC),
("EVC", CAP_VIDEO_EVC),
("EVC", CAP_VIDEO_EVC_BASELINE),
("AVS3 video", CAP_VIDEO_AVS3),
("VP9", CAP_VIDEO_VP9),
("AV1", CAP_VIDEO_AV1),
("AV1", CAP_VIDEO_AV1_LOSSLESS),
("AV2", CAP_VIDEO_AV2),
("JPEG 2000 (DCI cinema)", CAP_VIDEO_J2K),
("DCVC-FM", CAP_NEURAL_DCVC_FM),
("AAC", CAP_AUDIO_AAC),
("HE-AAC", CAP_AUDIO_AAC),
("HE-AAC v2", CAP_AUDIO_AAC),
("MPEG-H 3D Audio", CAP_AUDIO_MPEGH),
("AVS3 audio (Audio Vivid)", CAP_AUDIO_AVS3),
("IAMF", CAP_AUDIO_IAMF),
("FLAC", CAP_AUDIO_FLAC),
("Opus", CAP_AUDIO_OPUS),
("AC-4 (ETSI TS 103 190)", CAP_AUDIO_AC4),
("IVAS", CAP_AUDIO_IVAS),
("EnCodec", CAP_NEURAL_ENCODEC32),
("DAC", CAP_NEURAL_DAC),
("SNAC", CAP_NEURAL_SNAC),
("Mimi", CAP_NEURAL_MIMI),
("WavTokenizer", CAP_NEURAL_WAVTOKENIZER),
("Zstandard", CAP_TEXT_ZSTD),
("XZ (LZMA2)", CAP_TEXT_XZ),
("glTF 2.0 Binary (GLB)", CAP_MESH_GLB),
("Draco", CAP_MESH_DRACO),
("G-PCC", CAP_PC_GPCC),
("V-PCC", CAP_PC_VPCC),
("SPZ", CAP_SPLAT_SPZ),
("MPEG-I Haptics", CAP_HAPTIC_MIHS),
("HDR10+ (SMPTE ST 2094-40)", CAP_META_HDR10PLUS),
("SMPTE ST 2094-10 dynamic metadata", CAP_META_DOVI),
];
const WAI_NATIVE: &str = "a WAI-native capability class: its payload format is WAI's own, so no glossary codec describes it";
const NO_GLOSSARY_ENTRY: &str = "the glossary has no entry for this model yet";
/// `(capability, reason)`: every payload or companion capability no entry links.
const GLOSSARY_NO_ENTRY: &[(&str, &str)] = &[
(CAP_AUDIO_INT_CODEC, WAI_NATIVE),
(CAP_AUDIO_SCENE, WAI_NATIVE),
(CAP_AUDIO_SCORE, WAI_NATIVE),
(CAP_AVATAR_POSE, WAI_NATIVE),
(CAP_FEED_STATE, WAI_NATIVE),
(CAP_FILM_LINEAR, WAI_NATIVE),
(CAP_FILM_REEL, WAI_NATIVE),
(CAP_GENERATE_SPECTRAL, WAI_NATIVE),
(CAP_HAPTIC_SIGNAL, WAI_NATIVE),
(CAP_NEURAL_INT_HYPER, WAI_NATIVE),
(CAP_NEURAL_INT_MLICPP, WAI_NATIVE),
(CAP_NEURAL_INT_SYNTH, WAI_NATIVE),
(CAP_QUANTUM_CIRCUIT, WAI_NATIVE),
(CAP_SPLAT_4D, WAI_NATIVE),
(CAP_SPLAT_INT_CODEC, WAI_NATIVE),
(CAP_TERNARY_TT, WAI_NATIVE),
(CAP_VIDEO_INT_HYPER, WAI_NATIVE),
(CAP_VIDEO_INT_MOTION, WAI_NATIVE),
(CAP_WORLD_LIVE, WAI_NATIVE),
(CAP_WORLD_MODEL, WAI_NATIVE),
(CAP_WORLD_REPLAY, WAI_NATIVE),
(CAP_WORLD_SCENE, WAI_NATIVE),
(CAP_NEURAL_BMSHJ2018, "the 2018 scale-hyperprior entry describes that paper's hyperprior model, not its factorized-prior baseline, which this capability carries"),
(CAP_NEURAL_MBT2018_MEAN, "the entry for the 2018 joint-prior paper describes its autoregressive context model, not the mean-scale hyperprior this capability carries"),
(CAP_NEURAL_VIDEO_BMSHJ2018, "per-frame use of the factorized-prior image model; no glossary codec describes it"),
(CAP_NEURAL_CMIC, NO_GLOSSARY_ENTRY),
(CAP_NEURAL_DCVC_RT, NO_GLOSSARY_ENTRY),
(CAP_NEURAL_ELIC, NO_GLOSSARY_ENTRY),
(CAP_NEURAL_GLC, NO_GLOSSARY_ENTRY),
(CAP_NEURAL_MLICPP, NO_GLOSSARY_ENTRY),
(CAP_NEURAL_MLICV2, NO_GLOSSARY_ENTRY),
(CAP_SEMANTIC_CLIP, "a generative-semantic profile, not a codec"),
(CAP_SEMANTIC_PERCO2, "PerCoV2 is a different model from the PerCo entry"),
(CAP_PRIOR_NNC, "neural-network coding codes model parameters, not media"),
(CAP_TENSOR_NNC, "a candidate with no payload format pinned; neural-network coding codes tensors, not media"),
(CAP_SPLAT_SOG, "a candidate with no payload format pinned"),
];
/// Pairs SPEC §5 excludes from a capability: such an entry must not link it.
const GLOSSARY_EXCLUDED: &[(&str, &str, &str)] = &[
("xHE-AAC", CAP_AUDIO_AAC, "ADTS cannot carry Extended HE-AAC"),
("MPEG-D USAC", CAP_AUDIO_AAC, "ADTS cannot carry Extended HE-AAC"),
];
/// Phrases no `desc` may contain: the glossary's no-comparison rule (SCHEMA.md),
/// the same forms `scripts/check-comparative-prose.sh` rejects.
const COMPARATIVE_PHRASES: &[&str] = &[
"SOTA", "win vs", "moat", "nobody signs", "best-in-class", "superior", "% better", // allow-phrase: the rejected phrases
"strictly worse", "toll road", "outperform", "underperform", "fiasco", "market chose", // allow-phrase: the rejected phrases
];
/// The `body` allowlist, read from the bulleted list under the heading
/// "### `body` allowlist" in SCHEMA.md (each bullet one backticked value).
fn bodies_from_schema(schema: &str) -> Vec<String> {
let mut out = Vec::new();
let mut inside = false;
for line in schema.lines() {
if line.starts_with("### ") {
inside = line.contains("`body` allowlist");
continue;
}
if inside {
if let Some(rest) = line.strip_prefix("- `") {
if let Some(end) = rest.find('`') {
out.push(rest[..end].to_string());
}
}
}
}
out
}
fn media_of_modality(m: &str) -> &str {
if m == "speech" { "audio" } else { m }
}
/// Every problem with the glossary `doc`, one line each, against the given link
/// and no-entry tables, the body allowlist and the registry.
fn glossary_problems(doc: &Value, links: &[(&str, &str)], no_entry: &[(&str, &str)], bodies: &[String]) -> Vec<String> {
let mut p = Vec::new();
let Some(obj) = doc.as_object() else { return vec!["glossary: not a JSON object".into()] };
for k in obj.keys() {
if k != "schema" && k != "schema_version" && k != "entries" {
p.push(format!("glossary: top-level key `{k}` is not defined in SCHEMA.md"));
}
}
if obj.get("schema_version").and_then(Value::as_u64) != Some(GLOSSARY_SCHEMA_VERSION) {
p.push(format!("glossary: `schema_version` is not {GLOSSARY_SCHEMA_VERSION}"));
}
let Some(entries) = obj.get("entries").and_then(Value::as_array) else {
p.push("glossary: `entries` is missing or not an array".into());
return p;
};
let mut names = std::collections::HashSet::new();
let mut found_links: Vec<(String, String)> = Vec::new();
for e in entries {
let name = e.get("name").and_then(Value::as_str).unwrap_or("<unnamed>").to_string();
let Some(eo) = e.as_object() else { p.push(format!("glossary entry {name}: not an object")); continue };
// R2: fields.
for f in GLOSSARY_FIELDS {
if !eo.contains_key(*f) { p.push(format!("glossary entry {name}: missing field `{f}`")); }
}
for k in eo.keys() {
if !GLOSSARY_FIELDS.contains(&k.as_str()) && k != "wai" {
p.push(format!("glossary entry {name}: field `{k}` is not defined in SCHEMA.md"));
}
}
if eo.get("year").is_some_and(|y| !y.is_u64()) {
p.push(format!("glossary entry {name}: `year` is not a whole number"));
}
for f in ["name", "desc"] {
if eo.get(f).is_some_and(|v| v.as_str().is_none_or(str::is_empty)) {
p.push(format!("glossary entry {name}: `{f}` is not a non-empty string"));
}
}
// R13: body from the allowlist, or null.
match eo.get("body") {
Some(Value::Null) | None => {}
Some(Value::String(b)) if bodies.iter().any(|x| x == b) => {}
Some(b) => p.push(format!("glossary entry {name}: body {b} is not in the SCHEMA.md allowlist")),
}
// R3: enums.
for (f, allowed) in GLOSSARY_ENUMS {
if let Some(v) = eo.get(*f) {
if !v.as_str().is_some_and(|s| allowed.contains(&s)) {
p.push(format!("glossary entry {name}: {f} {v} is not a SCHEMA.md value"));
}
}
}
// R4: unique names.
if !names.insert(name.clone()) {
p.push(format!("glossary: two entries are named {name}"));
}
// R14: no comparative phrasing.
if let Some(desc) = eo.get("desc").and_then(Value::as_str) {
for ph in COMPARATIVE_PHRASES {
if desc.contains(ph) {
p.push(format!("glossary entry {name}: desc contains the comparative phrase `{ph}`"));
}
}
}
// R5–R10, R12: links.
let Some(w) = eo.get("wai") else { continue };
let Some(arr) = w.as_array().filter(|a| !a.is_empty()) else {
p.push(format!("glossary entry {name}: `wai` must be a non-empty array of links (SCHEMA.md)"));
continue;
};
let modality = eo.get("modality").and_then(Value::as_str).unwrap_or("");
let mut seen = std::collections::HashSet::new();
for l in arr {
let ok_shape = l.as_object().is_some_and(|o| {
o.len() == 3 && ["capability", "licence", "replicate"].iter().all(|k| o.get(*k).is_some_and(Value::is_string))
});
if !ok_shape {
p.push(format!("glossary entry {name}: a `wai` link must have exactly the string fields capability, licence and replicate"));
continue;
}
let cap = l["capability"].as_str().unwrap();
if !seen.insert(cap.to_string()) {
p.push(format!("glossary entry {name} links {cap} twice"));
}
found_links.push((name.clone(), cap.to_string()));
if !matches!(capability_kind(cap), Some(Kind::Payload | Kind::Companion)) {
p.push(format!("glossary entry {name} links {cap}, which SPEC §5 does not register as a payload or companion capability"));
continue;
}
let lic = capability_license(cap).map_or("?", LicenseClass::label);
if l["licence"].as_str() != Some(lic) {
p.push(format!("glossary entry {name}: the registry's licence class for {cap} is {lic}, not {}", l["licence"]));
}
let rep = capability_replicate_class(cap).map_or("?", ReplicateClass::label);
if l["replicate"].as_str() != Some(rep) {
p.push(format!("glossary entry {name}: the registry's replicate class for {cap} is {rep}, not {}", l["replicate"]));
}
let media = capability_media(cap).map_or("?", Media::label);
if media_of_modality(modality) != media {
p.push(format!("glossary entry {name} ({modality}) links {cap}, whose media class ({media}) it is not"));
}
for (ex_name, ex_cap, why) in GLOSSARY_EXCLUDED {
if *ex_name == name && *ex_cap == cap {
p.push(format!("glossary entry {name} links {cap}, which SPEC §5 excludes: {why}"));
}
}
}
}
// R15: the links are exactly the committed table.
for (n, c) in &found_links {
if !links.iter().any(|(ln, lc)| ln == n && lc == c) {
p.push(format!("glossary entry {n} links {c}, which GLOSSARY_LINKS does not list"));
}
}
for (n, c) in links {
if !found_links.iter().any(|(fn_, fc)| fn_ == n && fc == c) {
p.push(format!("GLOSSARY_LINKS lists {n} → {c}, which the glossary does not link"));
}
}
// R11: every payload or companion capability is linked or explained.
for &cap in known_capabilities() {
let linked = found_links.iter().any(|(_, c)| c == cap);
let listed = no_entry.iter().any(|(c, _)| *c == cap);
if !linked && !listed {
p.push(format!("{cap} is registered (SPEC §5) but no glossary entry links it and GLOSSARY_NO_ENTRY does not list it"));
}
if linked && listed {
p.push(format!("{cap} is linked and also listed in GLOSSARY_NO_ENTRY"));
}
}
p
}
fn read(rel: &str) -> String {
let path = std::path::PathBuf::from(env!("CARGO_MANIFEST_DIR")).join("..").join(rel);
std::fs::read_to_string(&path).unwrap_or_else(|e| panic!("read {}: {e}", path.display()))
}
#[test]
fn glossary_links_agree_with_the_registry() {
let doc: Value = serde_json::from_str(&read("glossary/codecs.json")).expect("codecs.json parses");
let bodies = bodies_from_schema(&read("glossary/SCHEMA.md"));
assert!(!bodies.is_empty(), "SCHEMA.md has no `body` allowlist");
let p = glossary_problems(&doc, GLOSSARY_LINKS, GLOSSARY_NO_ENTRY, &bodies);
assert!(p.is_empty(), "the codec glossary disagrees with its schema or the registry:\n{}", p.join("\n"));
for (_, why) in GLOSSARY_NO_ENTRY {
assert!(!why.is_empty());
}
}
/// A valid fixture, then one mutation per rule: removing any rule from
/// `glossary_problems` fails its row.
#[test]
fn glossary_check_names_each_broken_rule() {
use serde_json::json;
let bodies = vec!["ITU-T".to_string()];
let caps: Vec<&str> = known_capabilities().iter().copied()
.filter(|c| !matches!(capability_media(c), Some(Media::Interactive | Media::Binary)))
.collect();
let links: Vec<(&str, &str)> = caps.iter().map(|c| (*c, *c)).collect();
let no_entry: Vec<(&str, &str)> = known_capabilities().iter().copied()
.filter(|c| !caps.contains(c)).map(|c| (c, "fixture")).collect();
let entry = |cap: &str| json!({
"name": cap, "year": 2020, "body": null, "modality": capability_media(cap).unwrap().label(),
"era": "classical", "status": "niche", "lineage": "standards", "desc": "A codec.",
"wai": [{ "capability": cap, "licence": capability_license(cap).unwrap().label(),
"replicate": capability_replicate_class(cap).unwrap().label() }]
});
let good = json!({ "schema": "x", "schema_version": GLOSSARY_SCHEMA_VERSION, "entries": caps.iter().map(|c| entry(c)).collect::<Vec<_>>() });
let ok = glossary_problems(&good, &links, &no_entry, &bodies);
assert!(ok.is_empty(), "{ok:?}");
let idx = |cap: &str| caps.iter().position(|c| *c == cap).unwrap();
let hevc = idx(CAP_VIDEO_HEVC);
let aac = idx(CAP_AUDIO_AAC);
type Mutation = Box<dyn Fn(&mut Value)>;
let cases: Vec<(Mutation, &str)> = vec![
(Box::new(|d| { d["extra"] = json!(1); }), "top-level key `extra`"),
(Box::new(|d| { d.as_object_mut().unwrap().remove("schema_version"); }), "`schema_version` is not 2"),
(Box::new(|d| { d["schema_version"] = json!(1); }), "`schema_version` is not 2"),
(Box::new(|d| { d["entries"][0].as_object_mut().unwrap().remove("desc"); }), "missing field `desc`"),
(Box::new(|d| { d["entries"][0]["codec"] = json!("x"); }), "field `codec` is not defined"),
(Box::new(|d| { d["entries"][0]["year"] = json!("2020"); }), "`year` is not a whole number"),
(Box::new(|d| { d["entries"][0]["status"] = json!("popular"); }), "is not a SCHEMA.md value"),
(Box::new(|d| { let n = d["entries"][0]["name"].clone(); d["entries"][1]["name"] = n; }), "two entries are named"),
(Box::new(|d| { d["entries"][0]["wai"] = json!([]); }), "non-empty array of links"),
(Box::new(|d| { let l = d["entries"][0]["wai"][0].as_object_mut().unwrap(); let v = l.remove("licence").unwrap(); l.insert("license".into(), v); }), "exactly the string fields"),
(Box::new(|d| { d["entries"][0]["wai"][0]["capability"] = json!("wai.video.h266"); }), "does not register"),
(Box::new(move |d| { d["entries"][hevc]["wai"][0]["licence"] = json!("royalty-free"); }), "licence class for wai.video.hevc is patent-pool"),
(Box::new(move |d| { d["entries"][aac]["wai"][0]["replicate"] = json!("standard-defined"); }), "replicate class for wai.audio.aac is standard-unclassified"),
(Box::new(move |d| { d["entries"][hevc]["modality"] = json!("audio"); }), "whose media class"),
(Box::new(move |d| { let l = d["entries"][hevc]["wai"][0].clone(); d["entries"][hevc]["wai"].as_array_mut().unwrap().push(l); }), "links wai.video.hevc twice"),
(Box::new(move |d| { d["entries"][hevc].as_object_mut().unwrap().remove("wai"); }), "wai.video.hevc is registered (SPEC §5) but no glossary entry links it"),
(Box::new(move |d| { d["entries"][aac]["name"] = json!("xHE-AAC"); }), "which SPEC §5 excludes"),
(Box::new(|d| { d["entries"][0]["body"] = json!("A Company Inc."); }), "not in the SCHEMA.md allowlist"),
(Box::new(|d| { d["entries"][0]["desc"] = json!("Roughly 50% better than its predecessor."); }), "comparative phrase"), // allow-phrase: test fixture
];
for (mutate, want) in cases {
let mut d = good.clone();
mutate(&mut d);
let p = glossary_problems(&d, &links, &no_entry, &bodies);
assert!(p.iter().any(|m| m.contains(want)), "expected a problem containing {want:?}, got {p:?}");
}
// The link table is binding in both directions.
let mut fewer = links.clone();
fewer.pop();
assert!(glossary_problems(&good, &fewer, &no_entry, &bodies).iter().any(|m| m.contains("GLOSSARY_LINKS does not list")));
let mut more = links.clone();
more.push(("nonexistent entry", CAP_IMAGE_PNG));
assert!(glossary_problems(&good, &more, &no_entry, &bodies).iter().any(|m| m.contains("which the glossary does not link")));
}
#[test]
fn the_body_allowlist_is_read_from_the_schema() {
let md = "intro\n### `body` allowlist\n\n- `ITU-T` — x\n- `IETF`\n\n### next\n- `not this`\n";
assert_eq!(bodies_from_schema(md), vec!["ITU-T".to_string(), "IETF".to_string()]);
}

167
wai/wai-rs/src/codecs/mod.rs
View file

@ -1,11 +1,14 @@
//! Registered zeroth-condition codec capabilities.
//! The capability registry: every capability string SPEC §5 registers, and the
//! classes the registry gives each one (fidelity tier, licence class, replicate
//! class, role, kind, status, media).
//!
//! The WAI thesis: the **envelope** + **capability dispatch** is the
//! standard; the codecs themselves are the SOTA libraries every modern
//! media system already uses. The zeroth condition is a *menu* of
//! always-implementable standard capabilities — pick any one and a sink
//! with the corresponding library can open the file. No hand-rolled
//! codec lives in this crate.
//! WAI dispatches registered standard codecs, which a sink supplies from the
//! libraries it already has, and also defines WAI-native capability classes (the
//! integer codecs, worlds, scores, films and the other extensions of SPEC §5).
//! The zeroth condition is the menu of standard capabilities: a sink with the
//! corresponding library opens the file. The standard codecs this crate decodes
//! natively are thin wrappers around their reference libraries (the `codecs`
//! feature); the registry itself needs no feature.
#[cfg(feature = "codecs")]
pub mod audio;
@ -21,6 +24,15 @@ pub mod video;
// the wire forms a `Prior` may declare (extensions/prior-carriage).
pub mod derivation;
pub mod prior_form;
// The registry as one table, and its export (`wai registry export`).
pub mod registry;
// The codec glossary (wai/glossary) held to its schema and the registry. Test-only.
#[cfg(test)]
mod glossary_check;
pub use registry::{
capability_kind, capability_media, capability_role, capability_status, is_candidate,
is_dispatchable, Kind, Media, Registration, RegistrationStatus, registrations,
};
// ---- Capability strings (the values that go in manifest.model_requirement.capability)
// Image
@ -35,30 +47,30 @@ pub const CAP_AUDIO_FLAC: &str = "wai.audio.flac"; // lossless
// Video
pub const CAP_VIDEO_AV1: &str = "wai.video.av1"; // lossy AV1
pub const CAP_VIDEO_AV1_LOSSLESS: &str = "wai.video.av1.lossless"; // lossless AV1
pub const CAP_VIDEO_INT_MOTION: &str = "wai.video.int_motion"; // DETERMINISTIC INTEGER inter-frame codec (int_video.rs): I-frame + integer block motion-compensated P-frames + integer-quantized residual over rANS. Decoder IMPLEMENTED, pure i64, no float/ONNX — every frame is byte-identical on every machine (frame-state-equivalence; per-frame portable hash). The temporal extension of the integer contract; cf. the float DCVC family (wai.neural.dcvc_*) which desyncs down the GOP. Bitstream from tools/wai_int_video_encode.py
pub const CAP_VIDEO_INT_MOTION: &str = "wai.video.int_motion"; // DETERMINISTIC INTEGER inter-frame codec (int_video.rs): I-frame + integer block motion-compensated P-frames + integer-quantized residual over rANS. Decoder IMPLEMENTED, pure i64, no float/ONNX — every frame is byte-identical on every machine (frame-state-equivalence; per-frame portable hash). The temporal extension of the integer contract. Bitstream from tools/wai_int_video_encode.py
pub const CAP_VIDEO_INT_HYPER: &str = "wai.video.int_hyper"; // NEURAL-KEYFRAME inter-frame codec: the I-frame is the LEARNED integer codec int_hyper (wai.neural.int_hyper) decoded byte-exact, the P-frames are integer block-MC residuals on top (int_video::decode_pframes). A learned keyframe you can put in a receipt + integer inter-frames — the whole decode integer, byte-identical everywhere. Composes int_hyper_synth + int_video; pure i64
// Text
pub const CAP_TEXT_ZSTD: &str = "wai.text.zstd"; // general-purpose
pub const CAP_TEXT_XZ: &str = "wai.text.xz"; // maximum classical ratio
// Neural (declared here so the dispatcher sees them; impls live at the sink).
// See SPEC.md §5 "Neural capabilities" for the SOTA selection rationale.
pub const CAP_NEURAL_ENCODEC32: &str = "wai.neural.encodec32"; // Meta EnCodec, 32 kHz
pub const CAP_NEURAL_DAC: &str = "wai.neural.dac"; // Descript Audio Codec, 44.1 kHz
pub const CAP_NEURAL_MIMI: &str = "wai.neural.mimi"; // Kyutai Mimi, real-time speech
// See SPEC.md §5 "Neural capabilities" for each model and its published operating point.
pub const CAP_NEURAL_ENCODEC32: &str = "wai.neural.encodec32"; // EnCodec, 32 kHz
pub const CAP_NEURAL_DAC: &str = "wai.neural.dac"; // DAC (improved RVQGAN, arXiv 2306.06546), 44.1 kHz
pub const CAP_NEURAL_MIMI: &str = "wai.neural.mimi"; // Mimi, real-time speech
pub const CAP_NEURAL_WAVTOKENIZER: &str = "wai.neural.wavtokenizer"; // ultra-low bitrate audio
pub const CAP_NEURAL_BMSHJ2018: &str = "wai.neural.bmshj2018"; // bmshj2018-factorized image codec (2018 baseline)
pub const CAP_NEURAL_MBT2018_MEAN: &str = "wai.neural.mbt2018_mean"; // mean-scale HYPERPRIOR image codec (Gaussian-conditional entropy) — decoder IMPLEMENTED in this crate (neural/hyperprior.rs); the conditional-entropy family, stronger than factorized bmshj2018; sink-supplied ONNX (hyper_synthesis + synthesis)
pub const CAP_NEURAL_MBT2018_MEAN: &str = "wai.neural.mbt2018_mean"; // mean-scale HYPERPRIOR image codec (Gaussian-conditional entropy) — decoder IMPLEMENTED in this crate (neural/hyperprior.rs); the conditional-entropy family: a mean-scale hyperprior conditions each latent's Gaussian; sink-supplied ONNX (hyper_synthesis + synthesis)
pub const CAP_NEURAL_INT_SYNTH: &str = "wai.neural.int_synth"; // INTEGER fixed-point synthesis (neural/int_synth.rs): all-i64 transposed-conv decode, no float/ONNX — bit-identical on every machine BY CONSTRUCTION. The first capability that can soundly back intent=replicate cross-hardware (determinism IS integer-exact). Weights are sink-supplied (int_synth.json from a QAT model)
pub const CAP_NEURAL_INT_HYPER: &str = "wai.neural.int_hyper"; // INTEGER fixed-point CONDITIONAL codec (int_hyper_synth.rs): the full Gaussian-conditional decode integer end-to-end — factorized rANS-decode z (z_q = s·2^afrac + median_q) → integer hyper-decoder h_s → σ,μ → integer-σ bucket (int_entropy) → integer rANS-decode residual → y_hat = r·2^afrac + μ → integer synthesis g_s. Both bitstreams entropy-coded, no raw latents. No float/ONNX anywhere, so the WHOLE bitstream→image decode is bit-identical on every machine BY CONSTRUCTION — int_synth's sibling that adds the conditional entropy. Quality: PTQ of the pretrained mbt2018_mean (quality 6); CI (wai-sota-vectors.yml) requires the integer decode within 1 dB of the same model's float decode on a pinned test photograph — measured cost there 0.19 dB (35.985 vs 36.175 dB, Apple Silicon Mac); image-dependent, see int_hyper_synth.rs. Tables sink-supplied
pub const CAP_NEURAL_INT_HYPER: &str = "wai.neural.int_hyper"; // INTEGER fixed-point CONDITIONAL codec (int_hyper_synth.rs): the full Gaussian-conditional decode integer end-to-end — factorized rANS-decode z (z_q = s·2^afrac + median_q) → integer hyper-decoder h_s → σ,μ → integer-σ bucket (int_entropy) → integer rANS-decode residual → y_hat = r·2^afrac + μ → integer synthesis g_s. Both bitstreams entropy-coded, no raw latents. No float/ONNX anywhere, so the WHOLE bitstream→image decode is bit-identical on every machine BY CONSTRUCTION — int_synth's sibling that adds the conditional entropy. Quality: PTQ of the pretrained mbt2018_mean (quality 6); CI (wai-sota-vectors.yml) requires the integer decode within 1 dB of the same model's float decode on a pinned test photograph — measured cost there 0.19 dB (35.985 vs 36.175 dB, arm64 macOS); image-dependent, see int_hyper_synth.rs. Tables sink-supplied
pub const CAP_NEURAL_INT_MLICPP: &str = "wai.neural.int_mlicpp"; // INTEGER MLIC++-topology codec (int_mlicpp.rs): the channel-autoregressive × checkerboard entropy model — K channel slices, each decoded as an anchor pass then a non-anchor pass, with an entropy network over concat[y decoded so far, hyper] giving σ‖μ before every pass — with every decode-side network integer (int_hyper_synth networks, int_entropy bucketing, integer rANS). The encoder and every decoder compute the same σ, so the same bucket, at any thread count: no float/ONNX anywhere, byte-identical on every machine BY CONSTRUCTION. Refuses any stream value outside the exact integer range. Payload (WIM1) and one-file model bundle (WMB1, so a prior pins every network and table) specified in int_mlicpp.rs. Conformance vector tests/vectors/int_mlicpp/ has SYNTHETIC weights (seeded, quantised like a trained model): it shows decode-equivalence, not compression quality. Model sink-supplied
pub const CAP_NEURAL_ELIC: &str = "wai.neural.elic"; // checkerboard CONTEXT-MODEL image codec (neural/context.rs) — the JPEG-AI/MLIC++ entropy class: hyperprior + 2-pass spatial context (anchor → context net → non-anchor). Decoder IMPLEMENTED; sink-supplied ONNX (hyper_synthesis + context + synthesis). WAI-native, NOT the ISO JPEG AI bitstream
pub const CAP_NEURAL_MLICPP: &str = "wai.neural.mlicpp"; // MLIC++ (−13.39% BD-rate vs VVC/VTM-17): the multi-reference entropy model — channel-autoregressive slices + checkerboard + global attention. Decoder IMPLEMENTED (neural/mlicpp.rs): the slice/checkerboard decode orchestration; the attention nets are sink-supplied ONNX (hyper_synthesis + entropy + synthesis)
pub const CAP_NEURAL_MLICV2: &str = "wai.neural.mlicv2"; // MLICv2 / MLICv2+ (2025): CURRENT SOTA learned image, −20.46% BD-rate vs VTM-17 (vs MLIC++'s −13.39%). SAME multi-reference channel-autoregressive decode orchestration as mlicpp — the gains are in the MetaFormer transform + enhanced entropy/global-context nets, all sink-supplied ONNX. Decoder route IMPLEMENTED (shares neural/mlicpp.rs)
pub const CAP_NEURAL_CMIC: &str = "wai.neural.cmic"; // Content-Adaptive Mamba (CMIC, ICLR 2026, arXiv 2508.02192): a state-space-model transform at the learned-image frontier. Its entropy model is SCCTX (space-channel context = channel slices + checkerboard) = the mlicpp orchestration; the Mamba lives in the sink-supplied transform. Decoder route IMPLEMENTED (shares neural/mlicpp.rs)
pub const CAP_NEURAL_MLICPP: &str = "wai.neural.mlicpp"; // MLIC++: the multi-reference entropy model — channel-autoregressive slices + checkerboard + global attention. Decoder IMPLEMENTED (neural/mlicpp.rs): the slice/checkerboard decode orchestration; the attention nets are sink-supplied ONNX (hyper_synthesis + entropy + synthesis)
pub const CAP_NEURAL_MLICV2: &str = "wai.neural.mlicv2"; // MLICv2 / MLICv2+ (2025): a learned image codec with the SAME multi-reference channel-autoregressive decode orchestration as mlicpp — the gains are in the MetaFormer transform + enhanced entropy/global-context nets, all sink-supplied ONNX. Decoder route IMPLEMENTED (shares neural/mlicpp.rs)
pub const CAP_NEURAL_CMIC: &str = "wai.neural.cmic"; // Content-Adaptive Mamba (CMIC, ICLR 2026, arXiv 2508.02192): a state-space-model transform for learned image coding. Its entropy model is SCCTX (space-channel context = channel slices + checkerboard) = the mlicpp orchestration; the Mamba lives in the sink-supplied transform. Decoder route IMPLEMENTED (shares neural/mlicpp.rs)
pub const CAP_NEURAL_VIDEO_BMSHJ2018: &str = "wai.neural.video_bmshj2018"; // per-frame bmshj2018 video
pub const CAP_NEURAL_GLC: &str = "wai.neural.glc"; // GLC ultra-low-bpp images: CATEGORICAL/VQ entropy (codebook indices), not Gaussian. Decoder IMPLEMENTED (neural/glc.rs): per-position softmax → CompressAI-exact pmf_to_quantized_cdf → bypass-free categorical rANS over a checkerboard → VQ synthesis. Sink-supplied ONNX
pub const CAP_NEURAL_DCVC_RT: &str = "wai.neural.dcvc_rt"; // DCVC-RT (2025): first 100+fps-1080p / 4K-real-time neural video, −21% vs VTM. INTER-frame decoder IMPLEMENTED (neural/video_dcvc.rs): frame loop + reference-feature propagation, CPU-capable via ort (NOT CUDA-only — that was a speed claim). Sink-supplied temporal net
pub const CAP_NEURAL_DCVC_FM: &str = "wai.neural.dcvc_fm"; // DCVC-FM (feature modulation): SOTA-RD neural video, wide single-model quality range, RGB+YUV. Shares the inter-frame decode orchestration (neural/video_dcvc.rs); sink-supplied temporal net
pub const CAP_NEURAL_DCVC_RT: &str = "wai.neural.dcvc_rt"; // DCVC-RT (2025): real-time neural video (the paper reports 1080p and 4K real-time decode on a GPU). INTER-frame decoder IMPLEMENTED (neural/video_dcvc.rs): frame loop + reference-feature propagation, CPU-capable via ort (a GPU is needed for its real-time figures, not for the decode). Sink-supplied temporal net
pub const CAP_NEURAL_DCVC_FM: &str = "wai.neural.dcvc_fm"; // DCVC-FM (feature modulation): neural video with a wide single-model quality range, RGB+YUV. Shares the inter-frame decode orchestration (neural/video_dcvc.rs); sink-supplied temporal net
pub const CAP_NEURAL_SNAC: &str = "wai.neural.snac"; // SNAC: multi-resolution RVQ audio (hierarchical code streams, e.g. lengths [12,24,48,96]), ubiquitous in low-bitrate TTS. Decoder IMPLEMENTED (neural/snac.rs): N variable-length code levels → dynamic-input ONNX (codes_0..codes_{N-1}) → waveform; carries its own sample rate. Sink-supplied ONNX
// Perceptual / generative-semantic (the `wai.semantic.*` path): the wire
// carries an "understanding" (e.g. a CLIP embedding) and ANY conforming sink
@ -87,13 +99,13 @@ pub const CAP_WORLD_LIVE: &str = "wai.world.live"; // joinable world; ops ar
// total order, any arrival — to one portable session hash (replay-equivalence); the
// renderer is a declared model_id (sink-supplied, or the int_* family for byte-exact
// frames), the generated frames vouched for by a signed, joules-accounted provenance
// receipt. The neural sibling of wai.world.replay; vs the field (Genie/Oasis/Matrix/
// Hunyuan-World) which streams non-reproducible, unmetered pixels. world_model.rs
// receipt. The neural sibling of wai.world.replay: the session is reproducible and
// metered as an action trace, whatever renderer draws it. world_model.rs
pub const CAP_WORLD_MODEL: &str = "wai.world.model";
// Asset media for worlds (and any other consumer): same zeroth-condition
// philosophy as the rest of §5 — dispatch to the field's SOTA royalty-free
// standard, don't invent. Payload is canonical glTF 2.0 Binary (.glb,
// approach as the rest of §5 — dispatch to a published royalty-free standard.
// Payload is canonical glTF 2.0 Binary (.glb,
// Khronos, royalty-free): meshes, materials, textures, skeletal armatures
// in one self-contained object. Decoder is sink/library-supplied (every
// engine and DCC tool ships one); this crate treats the bytes as opaque.
@ -111,9 +123,9 @@ pub const CAP_MESH_GLB: &str = "wai.mesh.glb";
pub const CAP_SPLAT_4D: &str = "wai.splat.4d";
// Gaussian-splat scene asset: the same asset-path role as wai.mesh.glb,
// a different presentation representation. Payload is SPZ (Niantic's
// open-source splat interchange, ~10x smaller than the PLY 3DGS shipped
// with originally). PRESENTATION ONLY: a splat is bound to a node the
// a different presentation representation. Payload is SPZ, an open-source
// splat interchange format that stores quantised attributes. PRESENTATION
// ONLY: a splat is bound to a node the
// sim moves; it never enters the state hash. Heavy (MB-scale) — the win
// is content-addressing + dedup + one envelope, NOT payload size.
// Decoder/renderer is sink-supplied. WAI dispatches; it does not
@ -123,9 +135,8 @@ pub const CAP_SPLAT_SPZ: &str = "wai.splat.spz";
// transform over the [14,S,S] attribute image (pos/scale/rot/color/opacity), integer
// synthesis (int_hyper machinery, 14 channels) over a factorized-rANS latent →
// byte-exact attribute tensor on every machine (splat-equivalence; a portable cloud
// hash). The int_hyper of 3D — the decoded attributes you can put in a receipt, vs
// the float SOG / self-organizing-Gaussian codecs whose decode desyncs. Decoder
// IMPLEMENTED, pure i64, no float/ONNX. Model + bitstream from
// hash). The int_hyper of 3D — the decoded attributes you can put in a receipt.
// Decoder IMPLEMENTED, pure i64, no float/ONNX. Model + bitstream from
// tools/wai_int_splat_export.py. The float rasterizer stays sink-supplied.
pub const CAP_SPLAT_INT_CODEC: &str = "wai.splat.int_codec";
@ -149,10 +160,9 @@ pub const CAP_AUDIO_SCORE: &str = "wai.audio.score";
// whose SYNTHESIS is a stack of integer transposed convolutions + integer IGDN,
// decoding the per-channel factorized-rANS latent to int16 PCM byte-identically on
// every machine (sample-equivalence; a portable PCM hash). The audio analog of
// int_hyper — a neural audio codec whose reconstruction you can put in a receipt,
// vs the float SoundStream/EnCodec family (wai.neural.{encodec32,dac,mimi,...})
// whose decode desyncs on ~1e-5 round-off. Decoder IMPLEMENTED, pure i64, no
// float/ONNX. Model + bitstream from tools/wai_int_audio_export.py
// int_hyper — a neural audio codec whose reconstruction you can put in a receipt.
// Decoder IMPLEMENTED, pure i64, no float/ONNX. Model + bitstream from
// tools/wai_int_audio_export.py
pub const CAP_AUDIO_INT_CODEC: &str = "wai.audio.int_codec";
// Haptics (extensions/haptics): per-channel intensity/sharpness envelope,
@ -196,9 +206,9 @@ pub const CAP_QUANTUM_CIRCUIT: &str = "wai.quantum.circuit";
// quantum-INSPIRED (tensor-network) LOSSY compressor whose whole decode is
// integer (per-column integer scale, i64/i128), so the reconstruction is
// byte-identical on every machine (reconstruction-equivalence) — receiptable.
// Engine behind the `ternary` feature (crate::ternary). Honest scope: a
// PROVENANCE transform (auditable, deterministic), NOT a rate-competitive codec
// (measured RD is mediocre; see the extension's §Verdict). Tier = Lossy;
// Engine behind the `ternary` feature (crate::ternary). Scope: a PROVENANCE
// transform (auditable, deterministic); rate reduction is not its purpose
// (measured rate-distortion: the extension's §5). Tier = Lossy;
// determinism = WAI's deterministic floor (StandardDefined replicate class).
pub const CAP_TERNARY_TT: &str = "wai.ternary.tt";
@ -211,10 +221,10 @@ pub const CAP_TERNARY_TT: &str = "wai.ternary.tt";
// tests are specified by ISO/IEC 15938-18.
pub const CAP_PRIOR_NNC: &str = "wai.prior.nnc";
// wai.tensor.nnc: general tensorial data (split-inference feature maps, 3D/4D
// scene parameters) coded with NNC. A CANDIDATE, like wai.splat.sog: general-tensor
// coding is in an NNC amendment at committee-draft stage, so the payload format is
// not fixed and an encoder MUST NOT emit it until it is pinned to a published
// edition. For splat attributes, wai.splat.int_codec is the byte-exact path; this
// scene parameters) coded with NNC. A CANDIDATE (is_candidate; SPEC §5, Candidate
// capabilities): general-tensor coding is in an NNC amendment at committee-draft
// stage, so the payload format is not fixed and an encoder MUST NOT emit it until
// it is pinned to a published edition. For splat attributes, wai.splat.int_codec is the byte-exact path; this
// is an interchange form with no determinism guarantee.
pub const CAP_TENSOR_NNC: &str = "wai.tensor.nnc";
@ -241,18 +251,18 @@ pub const CAP_FILM_REEL: &str = "wai.film.reel";
// decoder/renderer is sink-supplied, exactly as for jxl/av1.
// ---------------------------------------------------------------------------
// IAMF / "Eclipsa Audio" (AOMedia Immersive Audio Model and Formats) — the
// ROYALTY-FREE immersive answer, same consortium/lineage as AV1. NOT a codec
// IAMF (AOMedia Immersive Audio Model and Formats) — ROYALTY-FREE, from the
// Alliance for Open Media, which also publishes AV1. NOT a codec
// but a codec-agnostic 3D-audio description that carries open codecs (Opus,
// FLAC, LPCM): channel-based (5.1.2 / 7.1.4 per ITU-R BS.2051), scene-based
// ambisonics, and binaural rendering, up to 28 input channels. Reference:
// libiamf (Apache-2.0). Payload = the canonical IAMF bitstream (opaque to WAI).
pub const CAP_AUDIO_IAMF: &str = "wai.audio.iamf";
// Dolby AC-4 (the codec under the Atmos umbrella for broadcast/streaming) —
// PROPRIETARY, per-device/per-title licensing. Registered as a license-flagged
// optional slot (like wai.semantic.perco2) so marquee content can ride INSIDE
// a WAI envelope without forcing the brand out; never a default, never a floor.
// AC-4 (ETSI TS 103 190) — PROPRIETARY, licensed per device and per title.
// Registered as a licence-flagged optional slot (like wai.semantic.perco2) so
// content already encoded in it can ride inside a WAI envelope; never a
// default, never a floor.
pub const CAP_AUDIO_AC4: &str = "wai.audio.ac4";
// 3GPP IVAS (Immersive Voice and Audio Services, 2024; EVS-based) — the
@ -281,8 +291,8 @@ pub const CAP_VIDEO_J2K: &str = "wai.video.j2k";
// codecs the field uses; decoder is sink-supplied.
// ---------------------------------------------------------------------------
// Draco (Google) — ROYALTY-FREE (Apache-2.0) mesh + point-cloud geometry
// compression. The safe interoperable baseline; pairs with glTF.
// Draco — ROYALTY-FREE (Apache-2.0) mesh + point-cloud geometry compression,
// an open-source library; glTF carries it through an extension.
pub const CAP_MESH_DRACO: &str = "wai.mesh.draco";
// MPEG G-PCC (ISO/IEC 23090-9) — geometry-based point-cloud coding for
@ -294,11 +304,14 @@ pub const CAP_PC_GPCC: &str = "wai.pc.gpcc";
// dynamic volumetric video (the offline distribution path). MPEG pool.
pub const CAP_PC_VPCC: &str = "wai.pc.vpcc";
// Gaussian-splat compression — the live frontier (Self-Organizing Gaussians /
// SOG/SOGS; MPEG WG4 work-in-progress, NO frozen standard yet). Registered as
// a CANDIDATE so a capability exists ahead of the freeze; payload format will
// be pinned when an open reference firms up. Compresses the wai.splat.spz /
// wai.splat.4d clouds the heavy asset path ships.
// Gaussian-splat compression, registered (June 2026) as a CANDIDATE for the
// Self-Organizing Gaussians (SOG / SOGS) family of splat formats. No payload
// format is pinned and no specification of the format is cited, so an encoder
// MUST NOT emit it and a sink treats it as unsupported until a revision pins
// one (is_candidate; SPEC §5, Candidate capabilities). It names no MPEG work
// item: the MPEG Gaussian-splat work read for research/media-codec-registry.md
// names no such format. Would compress the wai.splat.spz / wai.splat.4d
// clouds the heavy asset path ships.
pub const CAP_SPLAT_SOG: &str = "wai.splat.sog";
// ---------------------------------------------------------------------------
@ -321,12 +334,12 @@ pub const CAP_HAPTIC_MIHS: &str = "wai.haptic.mihs";
// is the multi-component envelope's job (see the registry roadmap).
// ---------------------------------------------------------------------------
// HDR10+ dynamic metadata (SMPTE ST 2094-40) — ROYALTY-FREE for content; the
// open dynamic-metadata pick.
// HDR10+ dynamic metadata (SMPTE ST 2094-40) — ROYALTY-FREE for content and
// devices (only a logo is licensed).
pub const CAP_META_HDR10PLUS: &str = "wai.meta.hdr10plus";
// Dolby Vision dynamic metadata (SMPTE ST 2094-10, RPU) — PROPRIETARY,
// license-flagged premium slot (preserve the brand for marquee content).
// SMPTE ST 2094-10 dynamic metadata (the RPU form) — PROPRIETARY, a
// licence-flagged slot. The capability string is a functional identifier.
pub const CAP_META_DOVI: &str = "wai.meta.dovi";
// ---------------------------------------------------------------------------
@ -725,7 +738,7 @@ CAP_WORLD_SCENE | CAP_WORLD_REPLAY | CAP_WORLD_LIVE | CAP_WORLD_MODEL | CAP_MESH
/// claim (a codec's license can't be honestly asserted in an envelope's
/// header). Tooling uses it to audit a title or a whole library: "use only
/// royalty-free capabilities", or "flag every pool/proprietary capability in
/// this catalog" — the per-title toll-road choice, enforceable in code.
/// this catalog", in code.
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub enum LicenseClass {
/// No royalty or patent fee to implement: an open standard or an open
@ -764,10 +777,15 @@ pub fn capability_license(cap: &str) -> Option<LicenseClass> {
// is licensed with no patent rights granted, and no royalty-free
// commitment was found. Classified conservatively, as MPEG-I Haptics is.
CAP_PRIOR_NNC | CAP_TENSOR_NNC => PatentPool,
// Single-vendor proprietary (Dolby).
// wai.splat.sog: a candidate with no pinned payload format, so the
// licensing of whatever format is pinned is not established.
// Classified conservatively and provisionally (SPEC §5, Licence classes).
CAP_SPLAT_SOG => PatentPool,
// Single-licensor proprietary: AC-4 and ST 2094-10 dynamic metadata.
CAP_AUDIO_AC4 | CAP_META_DOVI => Proprietary,
// Patent-pool / SEP royalties: IVAS (Via-LA Voice Codec pool, with
// EVS); MPEG point clouds (G-PCC/V-PCC); MPEG-I Haptics (ISO patent
// Patent-pool / SEP royalties: IVAS (a voice-codec pool, with EVS); MPEG
// point clouds (G-PCC/V-PCC: classified conservatively, as MIHS is, see
// research/media-codec-registry.md); MPEG-I Haptics (ISO patent
// declarations apply — pool status unconfirmed, classified
// conservatively).
CAP_AUDIO_IVAS | CAP_PC_GPCC | CAP_PC_VPCC | CAP_HAPTIC_MIHS => PatentPool,
@ -814,9 +832,9 @@ pub fn capability_license(cap: &str) -> Option<LicenseClass> {
| CAP_MESH_GLB | CAP_SPLAT_SPZ
// JPEG 2000 Part 1 is fee-free under ITU-T policy 2.1 (extensions /
// submarine-patent caveats noted); IAMF under the AOMedia Patent
// License 1.0; Draco Apache-2.0; SOG open work; HDR10+ royalty-free for
// content + devices (only the logo is licensed).
| CAP_IMAGE_JP2 | CAP_VIDEO_J2K | CAP_AUDIO_IAMF | CAP_MESH_DRACO | CAP_SPLAT_SOG
// License 1.0; Draco Apache-2.0; HDR10+ royalty-free for content +
// devices (only the logo is licensed).
| CAP_IMAGE_JP2 | CAP_VIDEO_J2K | CAP_AUDIO_IAMF | CAP_MESH_DRACO
| CAP_META_HDR10PLUS
// AV2 under the AOMedia Patent License 1.0; VP9 under its BSD source +
// patent grant; EVC Baseline royalty-free by design (caveat recorded).
@ -936,7 +954,7 @@ pub enum ReplicateClass {
/// one host give three different PCM streams from one ADTS stream; whether its
/// conformance admits that, or makes some of them non-conforming, was not
/// established), `wai.audio.ac4`; `wai.mesh.draco`, `wai.pc.gpcc`,
/// `wai.pc.vpcc`, `wai.splat.sog` (no frozen standard yet), `wai.haptic.mihs`,
/// `wai.pc.vpcc`, `wai.splat.sog` (a candidate: no payload format is pinned), `wai.haptic.mihs`,
/// and the HDR metadata descriptors `wai.meta.hdr10plus` / `wai.meta.dovi`
/// (what they reconstruct is a display-mapped picture); and the asset
/// capabilities `wai.mesh.glb` and `wai.splat.spz`, whose payload bytes pass
@ -1392,7 +1410,7 @@ mod tests {
#[test]
fn capability_registry_is_coherent() {
// The new SOTA capabilities are registered.
// Sink-supplied learned capabilities are registered.
assert!(is_known_capability(CAP_NEURAL_MLICPP));
assert!(is_known_capability(CAP_IMAGE_JPEGAI));
@ -1459,12 +1477,12 @@ mod tests {
assert_eq!(capability_license(CAP_IMAGE_JP2), Some(RoyaltyFree)); // ITU-T 2.1 Part 1
assert_eq!(capability_license(CAP_MESH_DRACO), Some(RoyaltyFree)); // Apache-2.0
assert_eq!(capability_license(CAP_META_HDR10PLUS), Some(RoyaltyFree));
assert_eq!(capability_license(CAP_AUDIO_IVAS), Some(PatentPool)); // Via-LA Voice Codec pool
assert_eq!(capability_license(CAP_AUDIO_IVAS), Some(PatentPool)); // voice-codec pool
assert_eq!(capability_license(CAP_PC_GPCC), Some(PatentPool));
assert_eq!(capability_license(CAP_PC_VPCC), Some(PatentPool));
assert_eq!(capability_license(CAP_HAPTIC_MIHS), Some(PatentPool)); // conservative (MPEG)
assert_eq!(capability_license(CAP_AUDIO_AC4), Some(Proprietary)); // Dolby
assert_eq!(capability_license(CAP_META_DOVI), Some(Proprietary)); // Dolby Vision
assert_eq!(capability_license(CAP_AUDIO_AC4), Some(Proprietary)); // single licensor
assert_eq!(capability_license(CAP_META_DOVI), Some(Proprietary)); // single licensor
assert_eq!(capability_license(CAP_SEMANTIC_PERCO2), Some(NonCommercial)); // CC-BY-NC-SA
// Unknown → no class.
assert_eq!(capability_license("wai.image.nope"), None);
@ -1541,9 +1559,18 @@ mod tests {
CAP_VIDEO_AVS3, CAP_AUDIO_MPEGH, CAP_AUDIO_AVS3, CAP_AUDIO_AAC,
// NNC (ISO/IEC 15938-17): royalty status not established, so pooled.
CAP_PRIOR_NNC, CAP_TENSOR_NNC,
// A candidate whose format is not pinned: provisional, conservative.
CAP_SPLAT_SOG,
];
want.sort_unstable();
assert_eq!(non_rf, want);
// No candidate is royalty-free: the licensing of a format no revision has
// pinned is not established (SPEC §5, Licence classes).
for &c in known_capabilities() {
if registry::is_candidate(c) {
assert!(!is_royalty_free(c), "{c} is a candidate, so its class cannot be royalty-free");
}
}
}
#[test]
@ -1699,7 +1726,7 @@ mod tests {
CAP_NEURAL_INT_MLICPP];
want.sort_unstable();
assert_eq!(nie, want);
// The SOTA learned codecs run float inference → not cross-hardware sound.
// The learned codecs that run float inference are not cross-hardware sound.
for &c in &[CAP_NEURAL_MLICPP, CAP_NEURAL_MLICV2, CAP_NEURAL_CMIC, CAP_NEURAL_ELIC, CAP_NEURAL_GLC,
CAP_NEURAL_MBT2018_MEAN, CAP_NEURAL_DCVC_FM, CAP_NEURAL_SNAC] {
assert_eq!(capability_replicate_class(c), Some(NeuralFloat), "{c}");

1217
wai/wai-rs/src/codecs/registry.rs Normal file
View file

File diff suppressed because it is too large Load diff

274
wai/wai-rs/src/container.rs
View file

@ -438,14 +438,25 @@ impl Wai {
/// payload, or `None` if the envelope is inert there. Step 3 takes the
/// primary capability whenever the sink advertises it, whatever the fallback
/// names; step 5 takes the fallback only if the primary is not advertised.
/// A companion-only capability ([`crate::codecs::is_companion_only`]) is
/// never dispatched standalone (SPEC §5): named as the primary, the payload
/// is companion data and the envelope is inert; named as the fallback, step 5
/// is skipped, and step 3 is unaffected. So an envelope that
/// [`Self::check_component_roles`] reports for its fallback alone still
/// decodes at a sink that advertises its primary capability.
/// Only a capability [`crate::codecs::is_dispatchable`] accepts is ever
/// taken, whatever `advertises` says:
///
/// - a companion-only capability ([`crate::codecs::is_companion_only`]) is
/// never dispatched standalone (SPEC §5): named as the primary, the payload
/// is companion data and the envelope is inert; named as the fallback, step
/// 5 is skipped, and step 3 is unaffected. So an envelope that
/// [`Self::check_component_roles`] reports for its fallback alone still
/// decodes at a sink that advertises its primary capability;
/// - a candidate ([`crate::codecs::is_candidate`]) is treated as unsupported
/// (SPEC §5, *Candidate capabilities*): as the primary, step 5 may still
/// take a usable fallback; as the fallback, it is never taken;
/// - a record or a derivation names no payload format, so it is never taken;
/// as the primary, step 5 may still take a usable fallback;
/// - a string this revision does not register is left to `advertises`: a
/// later MINOR revision may register it (SPEC §8), and a sink that has a
/// codec for it says so by advertising it (SPEC §4, step 3).
pub fn dispatch_capability(&self, advertises: impl Fn(&str) -> bool) -> Option<&str> {
let usable = |c: &str| advertises(c) && !crate::codecs::is_companion_only(c);
let usable = |c: &str| advertises(c) && crate::codecs::is_dispatchable(c);
let cap = self.capability();
if crate::codecs::is_companion_only(cap) {
return None;
@ -523,6 +534,23 @@ impl Wai {
Ok(())
}
/// Check that this envelope names no candidate capability
/// ([`crate::codecs::is_candidate`]), which an encoder MUST NOT emit (SPEC §5,
/// *Candidate capabilities*): first the `capability`, then the `fallback`.
/// Parsing stays format-agnostic, so such an envelope still parses; this is
/// what reports it.
pub fn check_candidates(&self) -> Result<(), CandidateError> {
check_candidates_v1(self.capability(), self.fallback())
}
/// The envelope's bytes, for an encoder to emit: [`Self::check_candidates`],
/// then [`Self::to_bytes`]. The C packer `wai_envelope_pack` emits through
/// this, so an encoder that uses it cannot emit a candidate (SPEC §5).
pub fn to_emit_bytes(&self) -> Result<Vec<u8>, EmitError> {
self.check_candidates().map_err(EmitError::Candidate)?;
self.to_bytes().map_err(EmitError::Serialize)
}
/// Why this envelope's `replicate` claim is unsound, or `None` if
/// [`Self::replicate_is_sound`]. See [`replicate_prior_refusal`].
pub fn replicate_prior_refusal(&self) -> Option<PriorRefusal> {
@ -572,13 +600,14 @@ impl RenditionMeta {
}
/// Whether `pick`/`primary` may select this component: it is not marked a
/// companion **and** its capability is not registered companion-only
/// ([`crate::codecs::is_companion_only`]). An HDR metadata block or an
/// LCEVC enhancement mislabelled as a rendition is still never selected —
/// it has no picture or sound of its own. [`WaiMulti::check_component_roles`]
/// reports the mislabelling.
/// companion **and** [`crate::codecs::is_dispatchable`] accepts its
/// capability — so not a companion-only capability, a candidate, a record or
/// a derivation. An HDR metadata block or an LCEVC enhancement mislabelled as
/// a rendition is still never selected — it has no picture or sound of its
/// own. [`WaiMulti::check_component_roles`] reports the mislabelling, and
/// [`WaiMulti::check_candidates`] a candidate.
pub fn is_selectable(&self) -> bool {
!self.is_companion() && !crate::codecs::is_companion_only(&self.capability)
!self.is_companion() && crate::codecs::is_dispatchable(&self.capability)
}
}
@ -661,6 +690,79 @@ impl std::fmt::Display for RoleError {
impl std::error::Error for RoleError {}
/// An envelope that names a candidate capability ([`crate::codecs::is_candidate`]),
/// which an encoder MUST NOT emit (SPEC §5, *Candidate capabilities*). Returned by
/// [`Wai::check_candidates`] and [`WaiMulti::check_candidates`]. The envelope still
/// parses.
#[derive(Debug, Clone, PartialEq, Eq)]
#[non_exhaustive]
pub enum CandidateError {
/// v1.0: a candidate named as the `WAI1` `capability`.
AsCapability { capability: String },
/// v1.0: a candidate named as the `WAI1` `fallback`.
AsFallback { capability: String },
/// v1.1: a `WAI2` component, in either role, whose capability is a candidate.
AsComponent { index: usize, capability: String },
}
const CANDIDATE_TAIL: &str = "is a candidate capability: no payload format is pinned for it, so an encoder MUST NOT emit it (SPEC \u{a7}5, Candidate capabilities)";
impl std::fmt::Display for CandidateError {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
match self {
CandidateError::AsCapability { capability } => write!(f, "the capability {capability} {CANDIDATE_TAIL}"),
CandidateError::AsFallback { capability } => write!(f, "the fallback {capability} {CANDIDATE_TAIL}"),
CandidateError::AsComponent { index, capability } => write!(f, "component {index} ({capability}) {CANDIDATE_TAIL}"),
}
}
}
impl std::error::Error for CandidateError {}
/// [`Wai::check_candidates`] over a `WAI1` manifest's two fields: the
/// `capability`, then the `fallback`. For a writer that holds the manifest fields
/// rather than a [`Wai`] (the wasm `pack`).
pub fn check_candidates_v1(capability: &str, fallback: Option<&str>) -> Result<(), CandidateError> {
if crate::codecs::is_candidate(capability) {
return Err(CandidateError::AsCapability { capability: capability.to_string() });
}
match fallback {
Some(fb) if crate::codecs::is_candidate(fb) => Err(CandidateError::AsFallback { capability: fb.to_string() }),
_ => Ok(()),
}
}
/// [`WaiMulti::check_candidates`] over a `WAI2` component table's capabilities,
/// in table order. For a writer that holds the table rather than a [`WaiMulti`]
/// (the wasm `packMulti`).
pub fn check_component_candidates<'a>(capabilities: impl IntoIterator<Item = &'a str>) -> Result<(), CandidateError> {
match capabilities.into_iter().enumerate().find(|(_, c)| crate::codecs::is_candidate(c)) {
Some((index, capability)) => Err(CandidateError::AsComponent { index, capability: capability.to_string() }),
None => Ok(()),
}
}
/// Why [`Wai::to_emit_bytes`] or [`WaiMulti::to_emit_bytes`] wrote nothing.
#[derive(Debug)]
#[non_exhaustive]
pub enum EmitError {
/// The envelope names a candidate capability (SPEC §5).
Candidate(CandidateError),
/// The envelope did not serialise.
Serialize(std::io::Error),
}
impl std::fmt::Display for EmitError {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
match self {
EmitError::Candidate(e) => e.fmt(f),
EmitError::Serialize(e) => write!(f, "envelope serialize: {e}"),
}
}
}
impl std::error::Error for EmitError {}
#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct ManifestV2 {
pub wai: String, // "1.1"
@ -845,9 +947,10 @@ impl WaiMulti {
Ok(Self { manifest, renditions })
}
/// Pick the first *selectable rendition* (never a companion, never a
/// companion-only capability — see [`RenditionMeta::is_selectable`]) whose
/// capability matches `pred`.
/// Pick the first *selectable rendition* ([`RenditionMeta::is_selectable`]:
/// never a companion, a companion-only capability, a candidate, a record or a
/// derivation) whose capability matches `pred`. A string this revision does
/// not register is selected when `pred` accepts it (SPEC §4, §8).
pub fn pick<F: Fn(&str) -> bool>(&self, pred: F) -> Option<(&RenditionMeta, &[u8])> {
for (meta, payload) in self.manifest.renditions.iter().zip(self.renditions.iter()) {
if meta.is_selectable() && pred(&meta.capability) {
@ -857,15 +960,31 @@ impl WaiMulti {
None
}
/// First selectable rendition in deployer-preferred order (the canonical
/// default pick when the sink has no further policy). Skips companions
/// and companion-only capabilities.
/// First selectable rendition ([`RenditionMeta::is_selectable`]) in
/// deployer-preferred order: the canonical default pick when the sink has no
/// further policy. Never a companion, a companion-only capability, a
/// candidate, a record or a derivation.
pub fn primary(&self) -> Option<(&RenditionMeta, &[u8])> {
self.manifest.renditions.iter().zip(self.renditions.iter())
.find(|(m, _)| m.is_selectable())
.map(|(m, p)| (m, p.as_slice()))
}
/// Check that no component, in either role, names a candidate capability
/// ([`crate::codecs::is_candidate`]), which an encoder MUST NOT emit (SPEC §5,
/// *Candidate capabilities*). Reports the first such component in table order.
pub fn check_candidates(&self) -> Result<(), CandidateError> {
check_component_candidates(self.manifest.renditions.iter().map(|m| m.capability.as_str()))
}
/// The envelope's bytes, for an encoder to emit: [`Self::check_candidates`],
/// then [`Self::to_bytes`]. The C packer `wai_envelope_pack_multi` emits
/// through this, so an encoder that uses it cannot emit a candidate (SPEC §5).
pub fn to_emit_bytes(&self) -> Result<Vec<u8>, EmitError> {
self.check_candidates().map_err(EmitError::Candidate)?;
self.to_bytes().map_err(EmitError::Serialize)
}
/// Check the component table against the role rules of SPEC §5 / §7.1,
/// using the capability registry. Parsing ([`Self::from_bytes`]) stays
/// format-agnostic — an unknown capability is still carried — so a sink
@ -1311,7 +1430,7 @@ mod tests {
#[test]
fn companion_descriptors_bind_to_a_primary() {
// one HDR video payload (two codec renditions of it) + two HDR
// dynamic-metadata companions (HDR10+ and Dolby Vision) bound to the AV1
// dynamic-metadata companions (HDR10+ and ST 2094-10) bound to the AV1
// rendition. One envelope, intrinsic binding.
let m = WaiMulti::new(
"video",
@ -1705,6 +1824,121 @@ mod tests {
assert_eq!(pmeta.capability, "wai.neural.encodec32");
}
/// SPEC §5, *Candidate capabilities*: a candidate is never dispatched, picked
/// or treated as a primary, and the candidate check reports every place one
/// can be named; such an envelope still parses.
#[test]
fn a_candidate_is_never_dispatched_picked_or_emitted() {
use crate::codecs::{is_candidate, known_capabilities};
let v1 = |cap: &str, fb: Option<&str>| Wai::new(Manifest {
wai: "1.0".into(), media: "binary".into(), intent: "improve".into(),
model_requirement: ModelRequirement { capability: cap.into(), fallback: fb.map(String::from),
min_sim: None, sim_metric: None },
conditioning: Conditioning { kind: "bytes".into(), prior: None },
target: serde_json::Value::Null,
}, b"payload".to_vec());
let all = |_: &str| true;
let candidates: Vec<&str> = known_capabilities().iter().copied().filter(|c| is_candidate(c)).collect();
assert!(!candidates.is_empty());
for &cand in &candidates {
// A candidate primary is unsupported: step 5 takes a usable fallback …
assert_eq!(v1(cand, Some("wai.image.png")).dispatch_capability(all), Some("wai.image.png"), "{cand}");
// … and without one the envelope is inert.
assert_eq!(v1(cand, None).dispatch_capability(all), None, "{cand}");
// A candidate fallback is never taken, even advertised alone.
assert_eq!(v1("wai.image.png", Some(cand)).dispatch_capability(|c| c == cand), None, "{cand}");
// The check names each place, and every message says why.
let as_cap = v1(cand, None).check_candidates().unwrap_err();
assert_eq!(as_cap, CandidateError::AsCapability { capability: cand.into() });
let as_fb = v1("wai.image.png", Some(cand)).check_candidates().unwrap_err();
assert_eq!(as_fb, CandidateError::AsFallback { capability: cand.into() });
for e in [&as_cap, &as_fb] {
assert!(e.to_string().contains("MUST NOT emit"), "{e}");
}
// Such an envelope still parses, byte for byte.
let env = v1(cand, Some("wai.image.png"));
let back = Wai::from_bytes(&env.to_bytes().unwrap()).unwrap();
assert_eq!(back.capability(), cand);
assert!(back.check_candidates().is_err());
// WAI2: a candidate at index 0 is skipped by primary and pick, and reported.
let m = WaiMulti::new("binary", "improve", serde_json::Value::Null, vec![
Rendition::media(cand, "bytes", b"c".to_vec()),
Rendition::media("wai.image.png", "png", b"p".to_vec()),
]);
let back = WaiMulti::from_bytes(&m.to_bytes().unwrap()).unwrap();
assert_eq!(back.primary().unwrap().0.capability, "wai.image.png", "{cand}");
assert_eq!(back.pick(all).unwrap().0.capability, "wai.image.png", "{cand}");
assert!(back.pick(|c| c == cand).is_none(), "{cand}");
assert_eq!(back.check_candidates(),
Err(CandidateError::AsComponent { index: 0, capability: cand.into() }));
}
// A registered, pinned capability is unaffected.
assert_eq!(v1("wai.pc.gpcc", None).dispatch_capability(all), Some("wai.pc.gpcc"));
assert_eq!(v1("wai.pc.gpcc", None).check_candidates(), Ok(()));
// A string this revision does not register is the sink's to advertise
// (SPEC §4, §8): taken when advertised, the fallback otherwise.
assert_eq!(v1("wai.image.nope", Some("wai.image.png")).dispatch_capability(all), Some("wai.image.nope"));
assert_eq!(v1("wai.image.nope", Some("wai.image.png")).dispatch_capability(|c| c == "wai.image.png"),
Some("wai.image.png"));
// A record or derivation names no payload: never taken, whatever is advertised.
assert_eq!(v1("wai.video.manifest", Some("wai.image.png")).dispatch_capability(all), Some("wai.image.png"));
assert_eq!(v1("wai.derive.crop", None).dispatch_capability(all), None);
let m = WaiMulti::new("binary", "improve", serde_json::Value::Null, vec![
Rendition::media("wai.video.manifest", "bytes", b"r".to_vec()),
Rendition::media("wai.image.nope", "bytes", b"n".to_vec()),
]);
assert!(!m.manifest.renditions[0].is_selectable());
assert_eq!(m.primary().unwrap().0.capability, "wai.image.nope");
assert_eq!(m.pick(|c| c == "wai.image.nope").unwrap().0.capability, "wai.image.nope");
}
/// The emit path of the C packers (`wai_envelope_pack`,
/// `wai_envelope_pack_multi`), driven exactly as they drive it: the manifest
/// JSON parsed into the manifest type, the envelope built, and the bytes taken
/// from `to_emit_bytes`, which refuses a candidate in every place one can be
/// named and writes nothing. No codec feature is needed.
#[test]
fn the_emit_path_refuses_a_candidate() {
let v1 = |cap: &str, fb: &str| {
let json = format!(r#"{{"wai":"1.0","media":"binary","intent":"improve","model_requirement":{{"capability":"{cap}","fallback":{fb}}},"conditioning":{{"kind":"bytes"}},"target":null}}"#);
let manifest: Manifest = serde_json::from_str(&json).unwrap();
Wai::new(manifest, b"bytes".to_vec()).to_emit_bytes()
};
let refused = v1("wai.splat.sog", "null").unwrap_err();
assert!(matches!(refused, EmitError::Candidate(CandidateError::AsCapability { .. })), "{refused}");
let refused = v1("wai.image.png", "\"wai.tensor.nnc\"").unwrap_err();
assert!(matches!(refused, EmitError::Candidate(CandidateError::AsFallback { .. })), "{refused}");
assert!(refused.to_string().contains("MUST NOT emit"), "{refused}");
let ok = v1("wai.image.png", "null").unwrap();
assert_eq!(Wai::from_bytes(&ok).unwrap().capability(), "wai.image.png");
let v2 = |caps: &[(&str, Option<&str>)]| {
let comps: Vec<String> = caps.iter().map(|(c, of)| match of {
Some(of) => format!(r#"{{"capability":"{c}","kind":"bytes","role":"companion","companion_of":"{of}"}}"#),
None => format!(r#"{{"capability":"{c}","kind":"bytes"}}"#),
}).collect();
let json = format!(r#"{{"wai":"1.1","media":"binary","intent":"improve","renditions":[{}],"target":null}}"#, comps.join(","));
let manifest: ManifestV2 = serde_json::from_str(&json).unwrap();
let renditions = manifest.renditions.iter().map(|m| Rendition {
capability: m.capability.clone(), kind: m.kind.clone(), payload: b"p".to_vec(),
prior: m.prior.clone(), role: m.role.clone(), companion_of: m.companion_of.clone(),
}).collect();
WaiMulti::new(manifest.media, manifest.intent, manifest.target, renditions).to_emit_bytes()
};
// A candidate as a rendition, and as a companion.
let refused = v2(&[("wai.splat.sog", None)]).unwrap_err();
assert!(matches!(refused, EmitError::Candidate(CandidateError::AsComponent { index: 0, .. })), "{refused}");
let refused = v2(&[("wai.video.av1", None), ("wai.tensor.nnc", Some("wai.video.av1"))]).unwrap_err();
assert!(matches!(refused, EmitError::Candidate(CandidateError::AsComponent { index: 1, .. })), "{refused}");
assert!(v2(&[("wai.video.av1", None), ("wai.meta.hdr10plus", Some("wai.video.av1"))]).is_ok());
// The free functions the wasm writers use agree with the methods.
assert_eq!(check_candidates_v1("wai.splat.sog", None),
Err(CandidateError::AsCapability { capability: "wai.splat.sog".into() }));
assert_eq!(check_component_candidates(["wai.image.png", "wai.splat.sog"]),
Err(CandidateError::AsComponent { index: 1, capability: "wai.splat.sog".into() }));
assert_eq!(check_component_candidates(["wai.image.png"]), Ok(()));
}
#[test]
fn version_detection_and_cross_rejection() {
let v1 = Wai::new(

6
wai/wai-rs/src/energy_class.rs
View file

@ -41,7 +41,7 @@
//! a real sensor reading, not an estimate. What it cannot offer is a
//! calibration traceable to a reference: its uncertainty is whatever its
//! vendor states or the implementation can evaluate, and some vendors add
//! deliberate noise to it (Intel's RAPL energy filtering, INTEL-SA-00389 —
//! deliberate noise to it (RAPL energy filtering, security advisory INTEL-SA-00389 —
//! typically under 2 % standard deviation over 1 s, 5–15 % over 100 ms).
//! - **An external calibrated instrument** is a separate meter whose error has
//! been established against a reference and written down in a certificate.
@ -165,8 +165,8 @@ impl Uncertainty {
/// Whether an on-chip counter was adding deliberate noise to its readings.
///
/// Some counters filter their output as a side-channel defence. On Intel
/// processors (the technical guidance for INTEL-SA-00389) filtered energy is
/// Some counters filter their output as a side-channel defence. On processors
/// covered by security advisory INTEL-SA-00389 (its technical guidance) filtered energy is
/// reported when SGX is enabled or system software has enabled filtering, and
/// TDX uses the filtered energy information; software with MSR access can
/// read it as `IA32_MISC_PACKAGE_CTLS[0]` (`ENERGY_FILTERING_ENABLE`, which

2
wai/wai-rs/src/energy_meter.rs
View file

@ -179,7 +179,7 @@ impl CounterMeter {
///
/// The filtering state is left [`CounterFiltering::Unknown`]: powercap does
/// not expose it, and an unprivileged reader cannot read the MSR that does.
/// Where filtering may be active, Intel states the added noise as typically
/// Where filtering may be active, the advisory's guidance states the added noise as typically
/// under 2 % standard deviation over 1 s and 5–15 % over 100 ms
/// (INTEL-SA-00389); `relative_ppm` should cover it at the windows used.
#[cfg(not(target_arch = "wasm32"))]

Some files were not shown because too many files have changed in this diff Show more