open-standards/wai/research/syfai-input.md
Transaction Science 22928147f3 wai: the parameter-set pin, component identity, pin-aware selection
SPEC §3.1 defines the pin a manifest carries (conditioning.prior, and a
WAI2 rendition's prior): id, sha256, determinism, and the optional form,
tier and supersedes. The digest covers the whole parameter set. A set of
several files is pinned through its listing, the new registered prior
form wai.prior.set. A sink verifies the pin before it decodes, reports a
refusal with one of an ordered set of snake_case reasons, and decodes
only from the set it verified. Tier is a closed vocabulary; supersedes is
informative.

A pin that does not resolve makes its capability unsupported for that
envelope: SPEC §4 dispatch continues at the fallback, §7.1 selection at
the next rendition, and an inert envelope's error names the reasons. A
rendition entry may carry id, sha256 and, on a companion,
companion_of_id. SPEC §7 gains a Digests subsection, and §8 says how a
MINOR revision adds an optional member.

In wai-rs: the manifest types are #[non_exhaustive] with builders; a new
pin module verifies pins in memory and on disk; Wai::dispatch_verified,
WaiMulti::pick and primary check pins and component digests; every
neural decoder reads through ParamSource; a codec-free `ffi` feature
carries the C envelope surface with wai_envelope_select and
wai_neural_decode_ex. wai-web wraps wai::container. New vectors: the
int_mlicpp set-form twin and the pin-conformance corpus with an
independent Python verifier, plus sha256.json sidecars for the vectors
of capabilities that decode against a parameter set.

Squashed from wai/stream-a-parameter-set-identity onto link 12. Conflicts
are resolved to keep both sides; the integration commits that follow
make the two streams agree.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-01 05:51:50 -04:00

982 lines
64 KiB
Markdown
Raw Permalink Blame History

This file contains ambiguous Unicode characters

This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.

# MPEG SyfAI (Exploration 52): WAI's input on the AI update framework
> **Status: draft input, not submitted.** A preparation document written in the
> shape of an input contribution toward MPEG SyfAI's next contribution window,
> MPEG 156, Hangzhou, 19–23 October 2026. It is not a normative extension.
> **An input to an MPEG meeting can only be registered by a delegate accredited
> through an ISO/IEC JTC 1/SC 29 national body. WAI has no such accreditation
> today**, so nothing here should be planned around an assumed submission. The
> posture is the same as [mpeg-ai-part6-mapping](mpeg-ai-part6-mapping.md):
> specify WAI *against* the work so it can become a profile of the eventual
> text.
>
> **Which tree.** WAI's mechanisms are described as they stand in the tree at
> link 10 of the landing chain: a linear chain of pull requests on the forge,
> each targeting `main`, merging in order. **As of 2026-09-30 none is merged
> into `main`.**
>
> | link | branch | PR |
> |---|---|---|
> | 01 | `feat/wai-video-workflow` | #82 |
> | 02 | `wai/land-02-determinism-tiers` | #103 |
> | 03 | `wai/land-03-jpegai-no-libm` | #104 |
> | 04 | `wai/land-04-codec-registry` | #105 |
> | 05 | `wai/land-05-prior-nnc` | #106 |
> | 06 | `wai/land-06-ci-headline-tests` | #107 |
> | 07 | `wai/land-07-int-decode-kernels` | #108 |
> | 08 | `wai/land-08-energy-classes` | #109 |
> | 09 | `wai/land-09-int-mlicpp` | #111 |
> | 10 | `wai/land-10-receipt-classes` | #112 |
>
> Each link contains the ones before it. Everything this note attributes to a
> link is proposed in that link's pull request and is not in `main`; where
> `main` differs, the text says so. The CI evidence in §2 is `main`'s, read from
> the forge on 2026-09-27, and the chain's, from the forge's runs on its pull
> requests, read on 2026-09-30.
>
> This note cross-references rather than repeats. The NNC carriage question is in
> [2030-transport-landscape](2030-transport-landscape.md) §6.4, and the form WAI
> registers for it is [prior-carriage](../extensions/prior-carriage.md). The
> post-decode processing hole is described in §3.2 of the landscape, and the rule
> that closes it is [determinism-tiers](../extensions/determinism-tiers.md) §2.
> Provenance over reconstructed output is in
> [reconstruction-binding](../extensions/reconstruction-binding.md). Where the
> landscape's text disagrees with a finding here, the gap that finds it says so
> (G1, G2). Where WAI's own normative text and its reference implementation
> disagree, the gap names both sides (G1, G12).
>
> Everything marked **⚠ unverified** was not seen in a primary source. The source
> list at the end says what was read and when.
---
## 1. The exploration, as the primary documents record it
| meeting | output document | title | status of this note's read |
|---|---|---|---|
| MPEG 151, Daejeon, 30 Jun – 4 Jul 2025 | none published | *Exploration on MPEG Systems support for Coding for Machine technologies* | title only, from the exploration page |
| MPEG 153, online, 19–23 Jan 2026 | WG 3 N1752 (w25953), dated 2026-02-09 | *Exploration on Systems technologies for coding for machine standards* | **read in full** |
| MPEG 154, Santa Eulària, 27 Apr – 1 May 2026 | WG 3 N1809 (w26201), dated 2026-05-02 | *Exploration on Systems technology for AI-based media standards (SyfAI)* | **read in full** |
| MPEG 155, Geneva, 13–17 Jul 2026 | listed, not published on mpeg.org as of 2026-09-27 | same SyfAI title | **not read**; substance known only from a secondary report |
| MPEG 156, Hangzhou, 19–23 Oct 2026 | — | next meeting | — |
| MPEG 157, Brisbane, 18–22 Jan 2027 | — | also evaluates MPEG-AI Part 6 CfP responses | — |
**When it began.** The exploration was not opened at MPEG 155. N1809's own
summary says the work "was established" at MPEG 153, where it concerned carriage
of Video Coding for Machines (VCM) NAL-unit video in ISOBMFF. It carries forward
an MPEG 151 exploration on systems support for coding for machines. The SyfAI
title first appears on the MPEG 154 output. The exploration's output documents
are **WG 3 (MPEG Systems)** documents. The secondary report on MPEG 155
(source 5) says the AI-update-framework thread was "opened jointly with WG 7"
(3D Graphics and Haptics). No WG document seen here records that. An input
should nonetheless be planned for both groups.
**What N1809 records from MPEG 154.** Most of the document is draft carriage text
for VCM in ISO/IEC 14496-15 (`vcm1`/`vcmi` sample entries, a
`VCMDecoderConfigurationRecord`). The part that concerns WAI is §3, the
discussion list. Its first item, *flexible AI codec model framework*, records
this need:
- unified terminology (modules, blocks, primitives);
- system-level support for model **discovery, validation, storage, delivery**;
- configuration update mechanisms;
- standardised signalling of **models, weights, metadata, training info**;
- a potential **model repository** infrastructure.
The comments recorded against it ask for a conceptual architecture
("when/where/how does system fetch"), for requirements covering **security,
authentication**, level of flexibility and repository, and for a defined scope.
One comment reads "menifest [sic] shall include inner functionality of codec,
delivery of model, capalibity [sic], etc." They name AI-based point-cloud coding
and Feature Coding for Machines (FCM) as starting points. The recommendation
reads: *"Noted. Bring architectural diagram and requirements."* A use case is
requested for the next meeting.
A second item proposes **a new MPEG standard for storage and carriage of
compressed neural networks as defined in ISO/IEC 15938-17 (NNC)**. The
recommendation was to bring a use case. This updates landscape §6.4: the
question of how the artifact a `Prior` names travels on the wire is being asked
at the systems layer in WG 3, not only in the NNC amendment.
**What the MPEG 155 output adds (⚠ secondary source only).** A July 2026 meeting
report lists five topics for an *AI update framework*: "(i) a format for AI
parameter data, (ii) a manifest for updating parameters, (iii) a repository to
serve them, (iv) integrity checking, and (v) bit exactness together with
conformance". It adds that "neural inference varies with library and hardware, so
deciding what conformance means and how to test it remains open". Topics (i) to
(iv) correspond to items N1809 §3 records: signalling of models and weights,
configuration update, a repository, and security and authentication. **Topic
(v), bit exactness with conformance, has no counterpart in N1809.** A search of
its text for exactness, conformance and integrity finds only a VCM HRD note. So
this note's topic (v), which §2 row (v), R2, R4 to R7 and G10 answer, rests on the
secondary report alone. Read the MPEG 155 output when it is published before
quoting any of the five topics.
**The shape an input should take.** N1809 asks for *use cases, an architecture
and requirements*. It does not ask for technology proposals yet. §3 and §4 below
are drafted in that shape.
---
## 2. Mapping: the five framework topics against what WAI specifies
"Specified" means normative text in this tree (SPEC.md or an extension).
"Implemented" means the reference implementation (`wai-rs`) does it.
"CI-checked" means a workflow on the forge is defined to exercise it. *CI
scope*, below the table, says which workflows do so and, for `main`, what their
runs showed. A row marked *reference implementation only* describes code, not
the standard, and an input cannot present it as WAI's normative position until
the text catches up (G1).
| framework topic | WAI mechanism | state | honest limit |
|---|---|---|---|
| (i) AI parameter-data format | **Registered forms, each optional.** A pin MAY declare that the set its digest covers is the listing of a set of files, `wai.prior.set` ([SPEC.md §3.1](../SPEC.md)), or an ISO/IEC 15938-17:2024 (NNC) bitstream, `wai.prior.nnc`; the digest then covers the listing or the coded bitstream ([prior-carriage](../extensions/prior-carriage.md) §2–§3). The model is still never transported in the envelope that decodes against it (SPEC.md's opening section). Informative formats exist: the integer `int_synth.json` model in [neural-conformance](../neural-conformance/README.md); the JPEG AI weight bundle (`weights/*.json` + `norm.json`) read by `try_jpegai_exact`, whose hyper-decoder, context-model and synthesis networks are integer but whose eICCI network and one `norm.json` table are `f32`; and the one-file `WMB1` bundle of `wai.neural.int_mlicpp` — its integer networks and the entropy tables in one file. | Listing form: specified, implemented, CI-checked. NNC form: specified, not implemented. Informative formats: implemented | `wai-rs` has no NNC reader or writer; the reference sink reads `wai.prior.set` and no coded form. NNC is optional and not recommended: its royalty status, and whether conforming decoders reconstruct identical tensors, are not established (prior-carriage §4–§5). |
| (ii) manifest | SPEC.md §3.1 defines the pin: `conditioning.prior = { id, sha256, determinism, form?, tier?, supersedes? }` in a single-payload (`WAI1`) manifest, and a rendition entry's `prior` in a multi-rendition (`WAI2`) one, beside the entry's `id` and payload `sha256` (§7.1). It names *which* parameter set, exactly — the SHA-256 of the whole set, a set of several files through its listing — and *how* its decode is made reproducible, with a tier from a closed vocabulary. | **specified (SPEC.md §3, §3.1, §7.1) and implemented** | The pin names identity, not location. Sinks of earlier revisions may ignore it (the pin then fails open) or reject the envelope (SPEC.md §3.1, §8). `id` is informative; `supersedes` is informative and never makes a successor satisfy its predecessor's pin. |
| (iii) repository | **None.** SPEC.md §1 scopes WAI as a container and capability-dispatch standard; nothing in the tree specifies a model store or service. Identity-by-digest makes any store content-addressable, but WAI specifies no store. | out of scope | A repository would be SyfAI's. WAI's input is the identity rule a repository must honour, not the repository. |
| (iv) integrity checking | SPEC.md §3.1 requires a sink to verify a pin before it decodes, and to refuse one its parameter sets do not satisfy with one of an ordered set of named reasons; a refused pin makes the capability unsupported for that envelope, and dispatch continues at the fallback (§4) or the next rendition (§7.1). A pinned decode reads only the set it verified. The reference implementation does this for `WAI1` and `WAI2` (`Wai::dispatch_verified`, `WaiMulti::pick`, `pin::verify_bytes`, `neural::decode_envelope`, `neural::decode_multi`), in the C ABI and in the browser crate. `decode_envelope_strict` checks the capability's registry class and then the pin before a `replicate` decode. Because the manifest sits inside the envelope bytes the JWP receipt hashes ([jwp-receipts](../extensions/jwp-receipts.md) §2.1), a signed receipt authenticates the pin as well as the payload. | **specified (SPEC.md §3.1, §4, §7.1) and implemented; the pin vectors are checked on the conformance legs** | A digest proves the bytes are the named bytes. It does not say who chose them. That comes from a signed JWP receipt over the envelope, and WAI has no key discovery (landscape §3.6). The lenient `decode_envelope` path decodes an unpinned neural capability without checking; this is honest for perceptual use and is not a replicate path. |
| (v) bit exactness + conformance | Two named tiers, `entropy-consistency` and `decode-equivalence`, and the rule that a tier covers what the sink **emits**, including post-decode processing ([determinism-tiers](../extensions/determinism-tiers.md) §2). Over a neural capability `replicate` promises `decode-equivalence`, which SPEC.md §7 admits only by an integer / fixed-point decode path with no float operation; `entropy-consistency` does not back it (link 02). A per-capability `ReplicateClass` in the registry: the `NeuralIntegerExact` capabilities at `decode-equivalence`, `wai.neural.int_mlicpp` among them (link 09), and `wai.image.jpegai` alone in `NeuralEntropyExact` at `entropy-consistency` (link 02). `codecs::DeterminismTier` carries the tiers in code (link 02). Frozen corpora of decoded-output digests. | tiers: specified, and a type in code at link 02 (on `main`, text only). Emitted-medium rule: **specified only**. Integer-entropy contract, `ReplicateClass` and the strict check: specified + implemented. Corpora and byte-equality goldens: implemented, and CI-checked on `main` for the capabilities listed under *CI scope*, on the legs that ran. | No manifest field names a tier; a capability's tier comes from its registration. `wai-rs` does not handle post-decode processing or operation-reduction hints. `wai.image.jpegai`'s exact decoder pins one reconstruction, reproducible only under its own build and floating-point conditions, and the reference sink can decode the same capability through a float path (G11). The float MLIC++ decoder agrees with its encoder only by convention between the two implementations; `wai.neural.int_mlicpp` makes that agreement a property of the arithmetic (*CI scope*). On `main`, a malformed stream's outcome depends on the build profile; link 01 fixes the known case (G6). Float capabilities have no corpus, deliberately. |
**CI scope, precisely.** Two states are described. For `main`, the workflow
files and the forge's run records read on 2026-09-27. For the tree at link 10,
the workflow files and the forge's runs on the chain's pull requests, read on
2026-09-30.
*On `main`.* Three workflows exercise WAI's decode paths. This is what each is
defined to run at commit 1b3e390, read from the workflow files (afc7ece adds the
OER vector step):
- **`wai-conformance.yml`** runs on any change under `wai/wai-rs/`, the
conformance corpora or `wai/byte-exact-conformance/`. Its matrix is linux
x86-64, linux arm64, macOS arm64 and windows x86-64. Each leg runs debug
builds of three steps:
- `cargo test --lib` over the pure-Rust features, including `neural_int`.
This covers the JPEG AI parsing, dequantisation and post-filter unit tests,
which check closed-form cases.
- The carriage test.
- Re-derivation of every cargo-class corpus, including the `int_synth`
neural corpus.
A second job runs the `byte-exact-conformance` crate, described next, in
release builds on the same four legs.
- **`wai-byte-equality.yml`** runs only on changes to the listed integer-decode
sources and vectors. It uses the same four legs and runs
`byte-exact-conformance` in release builds. It checks:
- the integer rANS goldens;
- the `int_synth` output digest;
- the JPEG AI entropy-stage (mANS) goldens;
- the `wai.video.int_motion` clip goldens;
- the `wai.video.int_hyper` clip, and its learned keyframe on its own, which
is decoded through a small committed `wai.neural.int_hyper` model.
- **`wai-neural-e2e.yml`** runs only on changes to `wai-rs/src/neural/`,
`tests/neural_native_decode.rs`, the fixture generator or the workflow
itself. It has one leg, linux x86-64. `tools/wai_validate_synthetic.py`
builds minimal synthetic models and bitstreams in the job. The job then runs
the float `_matches_reference` tests in `tests/neural_native_decode.rs`.
Those tests check the ELIC, MLIC++, SNAC, DCVC and GLC decoders against a
reference decode within ±4. So the job is defined to test whether each float
decoder is correct end to end on one machine. It makes no cross-architecture
claim, and its models are synthetic, not trained. No run of it has yet
reached those tests (below).
Four limits apply when any of these is read as evidence. They come from the
forge's run listing, and from each run's per-job status and job logs, all read
on 2026-09-27 (source 11).
- **An offline runner leaves a cancelled job behind, not a gap.** The host of
the `windows` runner has been offline since about 2026-08-28. The two matrix
workflows still create its jobs, and no runner claims them. Each such job
created from 2026-08-28 on was later cancelled, in one of two ways: a newer
push to the same branch started a new run, or a day or more passed after the
job was created. A cancelled job marks the whole run as cancelled, even when
every other job in it passed. The run on `main` at `afc7ece` was still
waiting on its two `windows` jobs.
- **Since then, every job on the other three platforms that finished has
passed.** In the runs of `wai-conformance.yml` and `wai-byte-equality.yml`
created from 2026-08-28 to 2026-09-27, no linux x86-64, linux arm64 or
macOS arm64 job failed. Some were cancelled when a newer push superseded
their run, and no `windows` job ran. The last runs in which the `windows`
leg also passed date from 2026-08-27 (`wai-conformance.yml`) and 2026-08-26
(`wai-byte-equality.yml`).
- No run exists at 1b3e390 itself. The nearest is the run on `main` at
`afc7ece`. In `wai-rs` that commit adds only the `oer` module, behind its
own feature, with its vector binary and the workflow steps that run them.
- In that run, on linux x86-64, linux arm64 and macOS arm64, both jobs
passed: the corpus job (library tests, carriage test and corpus
re-derivation) and the `byte-exact-conformance` job. Their logs record no
failed test.
- So the cross-architecture evidence for `main` covers three of the four
platforms. The `windows` leg's latest evidence predates 2026-08-28.
- **Tests that need model assets pass when the assets are absent (G8).** In
`wai-neural-e2e.yml` this also covers a partial fixture set. On `main` the
generator exits 0 if any one of its five harnesses succeeds, and the Rust
tests whose fixtures are missing then report as passed. PR #101, independent
of the chain, makes both cases fail the job.
- **`wai-neural-e2e.yml` has never reached its tests.** The listing holds 21
runs of it, from 2026-07-23 to 2026-09-26. Eighteen failed and three were
cancelled.
- The logs of all eighteen failures show the fixture step generating all
five synthetic fixtures. The job then stopped before any Rust test ran.
- Two early runs (2026-07-23 and 2026-07-26) failed at the Rust toolchain
step with `rustup: command not found`.
- The other sixteen, which include every run from 2026-07-31 on, failed to
link the test binary. The prebuilt ONNX Runtime static library that the
`ort` crate links references C-library symbols (`__isoc23_strtol`,
`__isoc23_strtoll`, `__isoc23_strtoull`) and, from 2026-08-26 on, a C++
standard-library symbol (`_M_replace_cold`) that the job's default
Ubuntu 22.04 image does not provide.
- No log contains a test result. The float decode tests are *defined* to run
in CI, and they have never run there.
**What the MLIC++ comparison exposes once the job links.** The float
`wai.neural.mlicpp` comparison then fails on the fixtures as generated, and the
cause is established. The fixture generator ran ONNX Runtime at its default
intra-op thread count, while the Rust decoder loads every session
single-threaded (`ort_session::load_deterministic`). Convolution reductions
therefore summed in a different order, and one scale landed on the other side of
a scale-table edge at symbol 416 of 12,288, which desynchronised the rANS decode
from there. PR #101 (independent of the chain) moves the job to an Ubuntu 24.04
image, in which the test binary links, and makes the generator load every session
with the decoder's execution settings. `wai-neural-e2e.yml` passed on the forge on
#101's head (2026-09-29), and its record reports the MLIC++ comparison going from
max |Δ| 255 to 0. That fix
is a convention between two implementations.
[`wai.neural.int_mlicpp`](../SPEC.md) (link 09, PR #111) makes the agreement a
property of the arithmetic: the scales come from integer networks, so every
conforming decoder computes the encoder's bucket for every element at any thread
count (`wai-rs/src/int_mlicpp.rs`, module documentation).
*At link 10*, the chain changes what CI runs:
- `wai-conformance.yml` adds, on every leg and in both build profiles, the
kernel-equivalence tests of link 07 over every committed integer-decoder
vector — among them the `wai.audio.int_codec` and `wai.splat.int_codec`
vectors, each decoded against its frozen hash under every kernel policy — and
the `wai.neural.int_mlicpp` tests of link 09. On its unix legs it adds two
checks that keep platform maths-library calls off the JPEG AI exact decode
(link 03). Its library tests add the `oer` and `receipt_claims` features.
- `wai-byte-equality.yml` adds the `wai.neural.int_mlicpp` golden (link 09).
- `wai-neural-gates.yml` (link 02) runs, in an Ubuntu 24.04 image, the
model-free neural-sink tests: the prior-hash refusal, the strict `replicate`
gate and, at link 09, the `wai.neural.int_mlicpp` dispatch.
- `wai-sota-vectors.yml` and `wai-jpegai-e2e.yml` (link 06) supply the inputs of
the full-size integer-model tests and of the JPEG AI whole-decode tests, and
fail on a missing input (G8).
- `wai-energy-receipts.yml` (link 08) and `wai-formal-verification.yml`
(link 01) cover the energy-class receipts and the bounded model-checking
harnesses.
- `wai-neural-e2e.yml` is unchanged: at link 10 it still runs in the default
image and still generates fixtures at the default thread count.
On the forge (runs read 2026-09-30), the chain's workflows have run since
2026-09-28. `wai-sota-vectors.yml`, `wai-jpegai-e2e.yml` and
`wai-neural-gates.yml` have passed on every run. The conformance and
byte-equality workflows have run on every unix leg, with a few Linux arm64
failures; each link's current result is on its pull request.
`wai-neural-e2e.yml` fails on every link, as on `main`, and passes only with
#101. The Windows legs have not run: their runner has been offline since late
August. [determinism-tiers](../extensions/determinism-tiers.md) §6 says this
from link 11.
A statement that a capability "is CI-checked" therefore means that a workflow
checks it whenever the workflow runs, on the legs that ran. For `main`, that
means linux x86-64, linux arm64 and macOS arm64 for the byte-exact workflows,
and nothing yet for `wai-neural-e2e.yml`. For the chain, it means the same three unix
legs, and `wai-neural-e2e.yml` only once #101 is merged.
---
## 3. Use cases, in the form N1809 asked for
**UC-1: Replicate-grade reconstruction across heterogeneous decoders.** A publisher
encodes with a learned codec and declares that any conforming decoder reproduces
the same output bytes. The use case needs a manifest that says which parameter
set, which reproducibility tier, and what a conformance test compares. The
tier can be the entropy-stage symbols or the reconstructed medium. WAI's
integer capabilities are built to reproduce the same output bytes. On `main`,
the CI workflows check that on well-formed streams for `wai.neural.int_synth`,
`wai.neural.int_hyper` (through a small committed model), `wai.video.int_motion`,
`wai.video.int_hyper` and the JPEG AI entropy stage, on the legs that ran (§2):
since 2026-08-28, linux x86-64, linux arm64 and macOS arm64, where every job
that finished passed. At link 10 the workflows are also defined to check
`wai.neural.int_mlicpp` on the byte-exact matrix, and the committed
`wai.audio.int_codec` and `wai.splat.int_codec` vectors on every conformance leg
(§2). [determinism-tiers](../extensions/determinism-tiers.md) §6 records
`wai.neural.int_mlicpp`'s committed vector decoding to its pinned bytes on macOS
arm64 and on wasm32-wasip2, measured on the development machine. The manifest
says less than this use case needs. In the reference implementation it names the
parameter set by a digest over one file unless a form is declared (G1, G2). No
manifest field names the tier (R2, R4). What a conformance test compares is
fixed by the capability's registration and corpus, not stated in the manifest.
**UC-2: A parameter update that must not decode silently.** A service moves a stream
from parameter set A to parameter set B. A sink that still holds A must refuse
B-coded content, or fall back to a declared alternative. Decoding it against A
is not acceptable, because a learned entropy decode against the wrong parameters
yields plausible garbage, not an error.
What WAI's text says about this case today:
- **SPEC.md requires the decode this use case rules out.** Dispatch (SPEC.md §4)
is keyed on the capability string alone. A sink holding A advertises the
capability, so step 3 applies: "decode `payload` via the codec registered for
`cap` (§5) and return". Nothing in §4 looks at the parameter set.
- **`model_requirement.fallback` does not help.** SPEC.md §3 and §4 steps 4–5
reach it only when the sink does *not* advertise the primary capability, and
§7.1 calls it "declarative" in v1.0.
- **Nor do renditions, as specified.** A §7.1 rendition is chosen by
capability advertisement or deployer policy. SPEC.md gives a rendition entry
a `capability` and a `kind`, plus an optional `role` and `companion_of` for
companion descriptors. It defines no prior field, so the specification
cannot mark one entry as the one for sinks that hold A.
- **The reference implementation refuses a mismatch only in a single-payload
envelope.** When a `WAI1` manifest pins a prior, `resolve_checked` refuses a
mismatched model with `DecodeError::PriorMismatch`, a hard error. Nothing
then tries the fallback capability or another rendition. On `main`,
`fallback` is only printed by the CLI (`inspect`, `unwrap`). At link 04
(PR #105), `Wai::dispatch_capability` applies SPEC.md §4 steps 3 and 5 and
`wai decode` and `wai inspect` use it, so the CLI decodes through the fallback
when the sink lacks the primary capability; it still never does so after a
`PriorMismatch`. With no pinned prior, the decode runs against whatever model
is registered.
- **The reference implementation carries a prior per rendition, and nothing
uses it.**
- `RenditionMeta` and `Rendition` each have `prior: Option<Prior>`
(`container.rs`), with the same meaning as the `WAI1` pin.
- It is serialised in the `WAI2` manifest, and the C FFI's pack call passes
it through (`ffi.rs`).
- The library test `multi_rendition_round_trip` checks that it survives the
wire. `wai-conformance.yml` runs that test, and it passed in the `afc7ece`
run on the three platforms that ran (§2).
- Selection ignores it. `pick()` and `primary()` choose by capability and
role alone. `resolve_checked` is called only from the `WAI1` decode, so no
path checks a `WAI2` rendition's pin.
- The WebAssembly binding's rendition metadata (`renditionsMeta` in
`wai-web`) drops the field. Its raw manifest string keeps it.
- So an envelope can carry one rendition pinned to A and another pinned to B
under the same capability, and `pick()` returns whichever comes first,
whatever the sink holds.
So WAI's normative text has no answer to this use case, and its dispatch rule
points the other way. The reference implementation refuses a pinned mismatch
in a single-payload envelope, but offers no declared alternative. For
renditions it already carries the data a declared alternative would need, a
prior per rendition, but no selection rule or check uses it. The gap is G12,
together with G1 and G5.
**UC-3: Provenance over the reconstructed output.** A provenance manifest binds a
hash of the *decoded* pixels, which is portable only when the reconstruction is
pinned. This connects SyfAI's bit-exactness topic to MPEG-AI Part 6. In the tree
at link 10, such a binding is checkable by every conforming sink over a
`decode-equivalence` capability; over `wai.image.jpegai` it holds only for the
exact decoder's output, under that decoder's conditions
([reconstruction-binding](../extensions/reconstruction-binding.md) §1–§2). See
also [mpeg-ai-part6-mapping](mpeg-ai-part6-mapping.md).
**UC-4: Energy attributed to a reproducible decode.** The energy of a decode
can be reported against a work unit that a third party recomputes from the
codestream. A work unit such as pixel operations needs only the codestream's
geometry, and WAI's video receipt derives it that way. What needs a
reproducible decode is binding the figure to a hash of the output that a third
party can regenerate. See [energy-exposure-mapping](energy-exposure-mapping.md).
---
## 4. Candidate requirements
*Status in this tree.* R1, R2, R3, R7 and R9 are now specified: R1 by SPEC.md §3.1
(the pin covers the whole parameter set; several files through the listing form
`wai.prior.set`); R2 by §3.1's closed `tier` vocabulary; R3 by §3.1's ordered
refusal reasons and §4 / §7.1's dispatch rule; R7 by §7 *Digests* (SHA-256 for
parameter-set identity, component identity and the decoded output of capabilities
that decode against a parameter set, with `sha256.json` sidecars on the committed
vectors); R9 by `supersedes`, which names the predecessor and changes only the
reason a sink that holds it reports — predecessor content decodes exactly while the
sink still holds the predecessor. The entries below record the state they were
written against.
Drafted as an input would phrase them. After each one: what WAI does in the tree
at link 10, or the gap.
**R1: Digest over the complete parameter set.** A manifest entry for an AI
parameter set shall identify it by a cryptographic digest over a canonical
serialisation of *every* artifact that affects decoder output: networks,
entropy tables and normative constants. A digest over one file of a multi-file
set does not meet this.
*WAI: partial.* The pin covers the one registered file (G2). Two routes to R1
exist at link 10: a declared form, whose digest covers the whole coded bitstream
(prior-carriage §2; no form is readable by the reference sink), and a one-file
model bundle, as `wai.neural.int_mlicpp` uses.
**R2: A declared reproducibility tier from a registered vocabulary.** The
manifest shall state the tier of the decode path the parameter set enables. At
minimum it shall distinguish entropy-stage reproducibility,
full-reconstruction reproducibility, and no cross-implementation guarantee.
*WAI: the vocabulary exists, in the capability's registration, not in the
manifest.* determinism-tiers names the two tiers, and at link 02 the registry
carries them in code (`ReplicateClass::neural_tier`, `codecs::DeterminismTier`),
with `NeuralFloat` claiming neither. The per-envelope contract string is still
free text (G3), and no manifest field states a tier.
**R3: Refuse on mismatch.** A decoder whose resolved parameter set does not match
the manifest digest shall refuse to decode under that manifest's claims. It
shall not degrade silently. *WAI: implemented in the reference implementation
only, and only for single-payload envelopes* (`PriorMismatch`). A `WAI2`
rendition's pin is carried but not checked. prior-carriage §2 requires refusal
of an unreadable declared form, but no normative text requires refusal on a
digest mismatch, and SPEC.md §4 step 3 requires a sink that advertises the
capability to decode (G1, G12).
**R4: Reproducibility claims are admissible only for a qualifying path.** A claim
that output is reproducible across implementations shall be admissible, for
full-reconstruction tiers, only when the decode path is integer or fixed-point
end to end. For entropy-stage tiers, the path may instead be integer at the
entropy stage or rely on explicit parameter transmission. A reconstruction that
one implementation reproduces by evaluating float stages in a fixed order is a
property of that implementation, not of the parameter set. The manifest shall say
which route a path takes.
*WAI: specified and implemented at link 02 (PR #103), except the manifest field.*
SPEC.md §7 lets a neural capability back `replicate` only if its entire decode
path is integer / fixed-point with no float operation, "the only construction by
which a neural capability is registered at `decode-equivalence`". Two weaker
properties MUST NOT back it: `entropy-consistency`, and a reconstruction pinned by
one implementation's fixed-order float stages. determinism-tiers §3 says a decode
that derives its entropy parameters in float meets neither tier, and one that
reads them from the codestream, or derives them in integer arithmetic, and runs
its later stages in float meets `entropy-consistency` only.
`decode_envelope_strict` checks the registry class, then the prior. No manifest
field says which route a path takes. On `main` the texts can be read two ways
(G10).
**R5: The tier covers the emitted medium.** Processing a decoder applies after
decoding, such as a post-filter or an operation-reduction hint, is part of the
path the tier describes. A decoder that alters the medium after decode shall not
claim the tier for what it emits. *WAI: specified, not implemented*
(determinism-tiers §2, "What a tier covers"). The reference implementation
does not handle post-decode processing or operation-reduction hints.
**R6: Conformance includes non-conforming input.** For a bit-exact tier, the
decoder's behaviour on a malformed or truncated stream shall be specified,
normally as rejection with an error. Otherwise build configuration, overflow
semantics or undefined behaviour change the decode outcome, and "bit exact"
holds only for well-formed streams. *WAI: the known defect is fixed at link 01;
the requirement is not normative text* (G6).
**R7: Frozen, architecture-independent conformance digests.** Conformance
material for a bit-exact tier shall be frozen digests of decoded output, with
the digest algorithm and the canonical output buffer named. It shall be
re-derived on more than one instruction-set architecture.
*WAI: implemented, but with three digest algorithms, where a contribution should
name one per purpose (G7).* `expected_hashes.json` per corpus. Canonical `RGB8`
in reconstruction-binding §2.
**R8: Integrity and authenticity are separate requirements.** Digest match
proves which bytes. Binding the manifest under a signature proves who named
them. A framework that specifies only the first cannot say whether an update
came from the content's publisher. *WAI:* in the reference implementation the
pin sits in the manifest, inside the envelope bytes a JWP receipt commits to
(jwp-receipts §2.1), so a signed receipt authenticates it. The pin itself is not
yet specified in SPEC.md (G1), and key discovery is absent (landscape §3.6;
transparency §6 disclaims it).
**R9: Updates name their predecessor.** A parameter update shall carry the digest
of the set it replaces and state whether content coded against the predecessor
remains decodable under the successor. *WAI: gap* (G5).
---
## 5. Two routes to cross-device consistency
A SyfAI conformance notion has to admit both routes the field uses, and it has
to say what each one reaches. Neither is a defect.
- **Integer arithmetic.** In JPEG AI (ISO/IEC 6048), "the entropy decoding
pipeline employs integer arithmetic neural network operations that can be
implemented with 8-bit multipliers and 32-bit registers, which allows bitexact
replication" across devices. JPEG AI "defines a bit-exact conformance point
only at the end of the entropy decoding pipeline", and its overview states
that "bit-exact picture reconstruction and lossless coding were deliberately
excluded from the first version" (arXiv 2510.13867, §IV, §III and §XIII
respectively). The capabilities WAI's registry classes as
`NeuralIntegerExact` take the same route through synthesis as well, which is
what `decode-equivalence` means. `wai.neural.int_mlicpp` (link 09) takes it
through a channel-autoregressive × checkerboard entropy model, re-deriving its
scales from integer networks before each pass. WAI's own JPEG AI decoder takes
it for its entropy decode and its integer networks, but not for its
dequantisation, latent refinement, post-filters or colour conversion, which
run in float; in the tree at link 10, SPEC.md §5 says so and registers the
capability at `entropy-consistency` (G11).
- **Explicit parameter transmission.** A 2026 learned video codec transmits
scale parameters through the hyperprior. This "guarantees entropy coding
consistency across devices without requiring bit-exact arithmetic" at a
bitrate cost (arXiv 2606.28027, ECCV 2026; abstract read, body not read). The
abstract does not say whether reconstructions match across devices. The route
guarantees the entropy stage, so identical reconstructions are not guaranteed
by it; that is an inference, not something the paper was seen to state. WAI's
tier text counts this route as reaching `entropy-consistency`, and at link 02
determinism-tiers §2 says why it stops there and §3 no longer reads against
that. **No WAI capability is registered as using it** (determinism-tiers §5
lists none). A contribution should say that plainly rather than imply
coverage.
- **Reproducibility by cross-check.** The IEEE 1857.11 call for proposals came
out of the working subgroup's meeting on 9 Dec 2021, and its timeline lists
the announcement on 21 Dec 2021. It states that "every proponent who submits
a complete proposal would be requested to participate in the reproducibility
check, i.e., to cross-check another proposal". The submitted decoder "can be
binary executable without source code". The call states no bit-exactness
requirement. IEEE 1857.11-2024 was published on 2024-12-20.
The value WAI can add at this layer is naming the tiers and tying each claim to
one of them. It does not need to prefer one route.
---
## 6. Gaps WAI must close before any of this is contributed
*Status in this tree.* G1 is closed by SPEC.md §3.1 (the pin is normative text,
for `WAI1` and per rendition); G2 by the parameter-set shapes of §5 and the
`wai.prior.set` listing; G3 by the closed `tier` vocabulary, with the tier-less
reading frozen as the legacy rule; G5 by `supersedes`; G7 by §7 *Digests*; G12 by
§4 / §7.1's dispatch rule and its implementation for `WAI2`. The entries below
record the state they were written against.
Each gap was found on `main`. Its status says what the tree at link 10 holds.
**G1: The prior pin is in the reference implementation, not in the
specification.** `Prior { id, sha256, determinism }` is defined in `wai-rs`
(`container.rs`), with an optional `form` at link 05. It is carried in two places
there: `Conditioning::prior` in a `WAI1` manifest and `RenditionMeta::prior` on
each `WAI2` rendition. The digest check and mismatch refusal exist only in
`neural/mod.rs` (`resolve_checked`), and apply to the `WAI1` pin only (G12).
On `main` the specification says less, and in one place the opposite: SPEC.md's <!-- allow-phrase: dated record, plain English -->
opening section says the model is "never transported and never hash-pinned".
In the tree at link 10 (link 05, PR #106) that sentence reads that the model is
never transported in the envelope that decodes against it, and that "its
identity MAY be pinned by digest (a `Prior`), and the pinned artifact's wire form
MAY be declared". prior-carriage §2 (a Draft extension) shows the `prior` object
with its members and fixes SHA-256 as the digest; §5 and SPEC.md §7 describe the
strict check on the pinned prior. What the specification still does not do:
- SPEC.md §3's manifest schema defines `conditioning` as `{ "kind" }` and has no
`prior` field, and SPEC.md §7.1 defines no prior on a rendition entry;
- no normative text requires a sink to refuse on a digest mismatch;
prior-carriage §2 says a sink refuses an unreadable form "as it refuses a
digest mismatch", which presupposes that refusal without stating it;
- SPEC.md §4 step 3 requires a sink that advertises the capability to decode
and return, so the only normative dispatch rule leads to a decode on a
mismatch, not a refusal (UC-2, G12).
Landscape §6.4 says "WAI *does* pin model **identity** normatively", and asks
for SPEC.md's opening sentence to be reconciled with it. The opening sentence is
reconciled at link 05. The landscape's sentence still goes further than the
text: until SPEC.md defines the pin (fields, digest, refusal on mismatch), in
both its single-payload and its per-rendition form, table rows (ii) and (iv) and
R3 describe code and a Draft extension, not the core standard. An input cannot
rest on them as WAI's normative position. G2 and G3 should be settled in the same
change, so that the specified pin is not the one-file, free-text form.
*Status: narrowed at link 05; open.*
**G2: Without a declared form, the prior pin covers one file of a multi-file
parameter set.** `resolve_checked` hashes the single registered path. Several
decoders then read sibling files the pin never covers:
- the hyperprior, context and GLC paths load `hyper_synthesis.onnx`,
`synthesis.onnx`, `context.onnx` and CDF tables (`*_cdfs_q<q>.json`) from the
model's directory (`neural/hyperprior.rs`, `context.rs`, `glc.rs`);
- the DCVC path loads `temporal_decoder.onnx` (`neural/video_dcvc.rs`);
- `wai.image.jpegai` registers a **directory**, so a pinned prior on it cannot
pass verification at all. Hashing reads the path as a file, which fails on a
directory. This was checked on the dev machine (macOS arm64, rustc 1.98.0),
which returns `Is a directory (os error 21)`.
The CDF tables determine the entropy decode, so this is not only a float-path
concern. The fix is a digest over a canonical manifest of the whole set, which
is R1. `wai.neural.int_mlicpp` (link 09) shows one form of it: its model is one
`WMB1` file holding three integer networks and the entropy tables, so a pinned
prior's SHA-256 covers everything its reconstruction depends on. A declared form
(link 05) is the other: its digest covers the whole coded bitstream, but the
reference sink reads no form. Landscape §8 queue item 9 says "`Prior.sha256`
currently covers an informal directory". As shown above, a pin on a directory
cannot verify at all, and where the registered path is a file, `sha256` covers
that one file. *Status: open for every multi-file decoder.*
**G3: The determinism contract is free text classified by substring.**
`Prior::determinism_class` returns `IntegerExact` when the string contains
`int`, `fixed` or `integer` and no `f32`, `f64` or `float` token
(`container.rs`). A contract naming a float runtime without a float token
classifies as integer-exact. At link 02 the registry class is checked first, so
such a contract can no longer admit a `replicate` over a float or entropy-only
capability; for a `NeuralIntegerExact` capability, the prior check still
verifies the declaration, not the arithmetic (determinism-tiers §3). A
registered vocabulary belongs in the manifest, which is R2. *Status: narrowed at
link 02; open.*
**G4: No normative parameter carriage format.** This is deliberate, and it is the
question N1809's NNC carriage proposal puts to WG 3. At link 05, prior-carriage
registers `wai.prior.nnc` (ISO/IEC 15938-17:2024) as the one form a `Prior` MAY
declare, optional and not recommended, and registers `wai.tensor.nnc` as a
candidate payload whose coding is in an unpublished amendment. prior-carriage §4
tracks that amendment from MPEG's 154th-meeting press release; its text,
including the data-unit signing it adds, was not read (⚠ unverified). *Status:
a form registered; carriage remains sink-supplied.*
**G5: No update semantics.** WAI has no field relating one parameter set to
another (R9). prior-carriage §2 forbids pinning an NNC incremental-update
bitstream on its own and registers no form for a base-plus-updates chain.
*Status: open.*
**G6: On `main`, malformed-stream behaviour depends on the build profile.** The
rANS bypass escape in `wai-rs/src/rans.rs` runs this loop:
`raw_val |= v << (j * BYPASS_PRECISION)`. It shifts a `u32` by a
stream-controlled amount. The shift panics in a debug build and is masked in a
release build. The `wai-rs` release profile does not enable `overflow-checks`.
- **Probe:** a scratch harness `#[path]`-included `rans.rs` at commit 1b3e390
and decoded 20,000 random 64-byte streams (SplitMix64, seeds 0–19,999)
against a three-entry CDF chosen to force the escape, four symbols each. It
was built in both profiles on the dev machine (macOS arm64, rustc 1.98.0,
release with opt-level 3, LTO and one codegen unit, matching `wai-rs`).
- **Result:** 18,015 streams returned values in release and panicked with
"shift left with overflow" in debug. 1,984 decoded identically, and one
panicked in both for different reasons. No stream produced two different
values.
- **Scope:** random input, not conformant streams. The well-formed corpora are
unaffected. The point is R6: without a specified error behaviour, "bit exact"
stops at well-formed input.
- **Status: fixed at link 01 (PR #82) for this escape.** `rans.rs` bounds the
run of bypass groups at what a 32-bit value can hold, so a malformed stream
decodes one way in both profiles, and every clamp is a no-op on a well-formed
stream. SPEC.md §7 (*Machine-checked totality*, informative) records four
defects in the escape found by bounded model checking and fixed, with the
conformance vectors unchanged. `wai.neural.int_mlicpp` (link 09) refuses every
stream value it cannot hold exactly. The fix clamps rather than rejects, and
no normative text specifies the outcome of a malformed stream, so R6 is not
met as written.
**G7: At least three digest algorithms for decoded output.**
- The `int_synth` corpus uses a domain-prefixed BLAKE3
(`wai_int_synth_vectors`).
- The `int_synth` byte-equality golden uses SHA-256 over the RGB output
(`byte-exact-conformance/src/golden.rs`). Reconstruction-binding also uses
SHA-256, because C2PA validators support it.
- The `wai.video.int_motion` and `wai.video.int_hyper` goldens, including the
learned keyframe, use a 64-bit FNV-1a. It is not a cryptographic hash
(`int_video_golden.rs`, `neural_keyframe_golden.rs`).
- At link 09, the `wai.neural.int_mlicpp` vector pins its RGB under both FNV-1a
and SHA-256, and its bucket and symbol sequences under FNV-1a.
A CI golden need not be cryptographic to detect an accidental divergence. A
conformance digest that a receipt or a third party relies on should be. A
contribution must name one algorithm per purpose (R7), and for conformance
material that others rely on, the one named should not be FNV-1a. *Status:
open.*
**G8: Tests that need model assets pass when the assets are absent, and some
capabilities have no CI.** On `main` these tests print a skip message and return
success when their assets are absent. What CI supplies differs between them:
- **Not run, or run without their assets, in CI on `main`:**
- the whole-decode tests of WAI's JPEG AI decoder, including
`jpegai_exact_bundle_decodes` in `neural/mod.rs`, which needs the `neural`
feature;
- the full-size integer-model tests (`tests/int_hyper_synth.rs`,
`tests/int_video_sota_keyframe.rs`), which no workflow invokes;
- the JPEG AI synthesis tests on real weights (`jpegai_synthesis.rs`). They
run in `cargo test --lib` and skip there, because no weights are supplied;
- the rest of `tests/neural_native_decode.rs`, including its tests on
exported or trained models. CI runs only the tests the job's
`_matches_reference` filter selects. `snac_real_model_matches_reference`
matches that filter but has no generated fixture, so it skips.
- **Defined in `wai-neural-e2e.yml` with synthetic fixtures, but never run
there:** the float `_matches_reference` tests for ELIC, MLIC++, SNAC, DCVC
and GLC in `tests/neural_native_decode.rs` (§2). They would check
correctness at ±4 against minimal synthetic models on one leg, not at
realistic scale and not across architectures.
- **No CI coverage of any kind on `main`:** `wai.audio.int_codec` and
`wai.splat.int_codec`. Their features (`audio_int`, `splat_int`) appear in no
workflow, and `byte-exact-conformance` does not include their sources. Yet
the registry classes both `NeuralIntegerExact`.
In the tree at link 10:
- **Link 06 (PR #107).** With `WAI_REQUIRE_SOTA_VECTORS=1` a missing or
incomplete input fails the headline tests instead of passing; without it the
skip is written to stderr. `wai-sota-vectors.yml` regenerates the full-size
`int_hyper` vectors from pinned inputs and runs both full-size tests.
`wai-jpegai-e2e.yml` fetches pinned JPEG AI inputs, checks them by SHA-256, and
runs the whole-decode, reconstruction-binding and cross-platform tests. Both
fail on a skip banner. Still in no workflow: `jpegai_exact_bundle_decodes`,
`jpegai_live`, and the synthesis tests on real weights.
- **Link 07 (PR #108).** `wai-conformance.yml` decodes the committed
`wai.audio.int_codec` and `wai.splat.int_codec` vectors against their frozen
hashes on every leg, under every kernel policy (`tests/int_kernels_vectors.rs`).
They are still outside the byte-exact golden suite, and the comparisons with
the Python reference in `int_audio.rs` and `int_splat.rs` still run in no
job. determinism-tiers §6 said until link 11 that no CI job enables their
features; link 11 corrects it. *(2026-09-30: the conformance workflow now also
names `tests/int_audio.rs` and `tests/int_splat.rs`, declared with their
features, so the comparisons with the Python reference's hashes run on every
leg; `wai.splat.sog` is now a candidate that no encoder may emit, SPEC §5.)*
- **PR #101 (independent of the chain).** Makes the neural end-to-end job link
and fail on a missing fixture, and fixes the MLIC++ fixture mismatch (§2). At
link 10 the job is unchanged and still cannot link.
What does run without skipping, on `main`, is the small committed integer
models: the `int_synth` corpus, and the `int_hyper` keyframe through a small
committed model (§2). For JPEG AI on `main`, the entropy stage is the only part
with a cross-architecture golden; its parsing, dequantisation and post-filter
kernels have closed-form unit tests in `cargo test --lib` on every leg of the
matrix, which since 2026-08-28 means three of the four. A conformance claim must
cite only the tests that run, on the legs that ran. *Status: addressed at links
06 and 07. On the forge, `wai-sota-vectors.yml` and `wai-jpegai-e2e.yml` have
passed on every run since 2026-09-28, and the conformance workflow runs the
link-07 vectors on its unix legs (runs read 2026-09-30). The neural end-to-end
job passes only with PR #101.*
**G9: No throughput figure.** On `main` the integer decoders are scalar and
single-threaded, and no decode-speed figure has been measured and published.
An argument for integer conformance invites a cost question. *Status: closed at
link 07 (PR #108).* `wai-rs/src/int_kernels.rs` adds fast integer kernels (SIMD on
aarch64, portable Rust elsewhere, worker threads except on wasm) that give the
reference's integer for every output element, and
[decode-throughput](decode-throughput.md) §4 publishes whole-decoder and kernel
figures measured on one macOS arm64 machine. Its §5 states the limits: one
machine and architecture, x86_64 not executed, synthetic weights for the largest
rows, and an equivalence that is argued and tested, not machine-proved.
**G10: WAI's texts word the `replicate` requirement ambiguously, one sentence
reads against its own section, and the tiers exist only as text.** This is the
gap closest to topic (v). On `main`:
- SPEC.md §7 lets a neural capability back `intent = replicate` "only if its
entire decode path — entropy parameterization *and* synthesis — is integer /
fixed-point, with no float operation anywhere".
- SPEC.md §7's closing paragraph calls `entropy-consistency` "the floor for a
sound neural `replicate`". determinism-tiers §3 says a `replicate` envelope
"MUST pin a prior that meets at least `entropy-consistency`".
- Read strictly these compose to the stricter rule, but the wording invites the
other reading: under determinism-tiers §2, integer entropy arithmetic with
float synthesis reaches `entropy-consistency`, and a reader who takes "at
least" as sufficient would admit a `replicate` claim that SPEC.md §7 forbids.
- determinism-tiers §3 says "a float neural decode meets neither tier", which,
read literally, excludes the explicit-parameter-transmission route that §2
counts as reaching `entropy-consistency` (§5 here).
- `wai-rs` has no type for either tier. `ReplicateClass` is a different
classification with no entropy-stage member.
*Status: closed at link 02 (PR #103), on the strict reading.* SPEC.md §7 and
determinism-tiers §2–§3 state one position: over a neural capability `replicate`
promises `decode-equivalence`, admitted only by the integer construction;
`entropy-consistency` is "necessary … and not sufficient" and MUST NOT back it.
determinism-tiers §3's sentence now separates a decode that derives its entropy
parameters in float (neither tier) from one that reads them from the codestream
or derives them in integer arithmetic (`entropy-consistency`). The registry gains
an entropy-stage class, `NeuralEntropyExact`, and a tier type,
`codecs::DeterminismTier`. Still open: R5 (post-decode processing) is specified
and not implemented.
**G11: `wai.image.jpegai`'s normative registration, its registry class and
its decode paths disagree.** On `main`:
- SPEC.md §5 registers `wai.image.jpegai` as an "integer entropy front end
(mANS) + integer dequant/value path + fixed-order synthesis and post-filters",
at **decode-equivalence**, "reproducible on every machine", and says "entropy
and dequant are integer-exact". determinism-tiers §2–§3 and
reconstruction-binding §1 say the same.
- `capability_replicate_class` has no arm for it, so it falls to
`StandardDefined`, the class for a reconstruction "fixed by the named
codec/standard", and a registry test asserts this. JPEG AI fixes a bit-exact
conformance point only at the end of entropy decoding (§5), so the standard
does not fix this reconstruction.
- The preferred path's dequantisation is floating point (`jpegai_pipeline.rs`):
the residual-variance stage multiplies and scales in `f64` and rounds to
`f32`, and the gain stage multiplies by the reciprocal of a gain computed with
an `f64` `exp()` from the platform maths library, which has no cross-platform
rounding guarantee. LSBS refinement, the conversion into the integer networks,
the post-filter chain (including the `f32` eICCI network) and colour
conversion are float too. The whole decode does not run in CI.
- `try_jpegai_exact` returns `None`, and the decode falls back to float ONNX
inference, when the integer bundle's directory or `norm.json` is absent, a
weight file cannot be read or parsed, or the codestream is outside the
implemented configuration. Both paths return the same `Decoded::Image`.
*Status: closed at link 02 (PR #103) and link 03 (PR #104), by bringing the texts
and the class down to what the code does.* SPEC.md §5 registers the capability
at `entropy-consistency` (registry class `NeuralEntropyExact`), describes
dequantisation, latent refinement, post-filters and colour conversion as float
in a fixed evaluation order, and records the downgrade. determinism-tiers §5 and
reconstruction-binding §1–§2 make the pinned reconstruction a property of the
exact decoder, reproducible under that decoder's build and floating-point
conditions, which it checks before decoding. Link 03 replaces the platform
`exp` and roundings with `float_repro`, built from IEEE-754 basic operations,
guards the decode path against maths-library calls, and pins nine hashes for one
codestream; determinism-tiers §6 records them identical on four platforms and not
measured on Windows, macOS x86_64 or x86_64 hardware. `wai-jpegai-e2e.yml`
defines the whole decode in CI (G8). What remains is stated in the texts, not
contradicted by them: the float fallback exists and the result does not record
which path ran, and a `norm.json` with a missing or mistyped key panics in the
loaders instead of falling back (the `try_jpegai_exact` doc comment says so).
**G12: No normative behaviour for a parameter-set mismatch, and a
per-rendition pin the code carries but never checks.** UC-2 sets out the
detail.
- *Specification:* SPEC.md §4 dispatches on the capability string alone, so a
sink holding the wrong parameter set must decode. `model_requirement.fallback`
fires only for a missing capability. SPEC.md §7.1 defines no prior on a
rendition entry.
- *Reference implementation:* a `WAI1` pin is checked, and a mismatch is
refused (`PriorMismatch`) with no declared alternative. A `WAI2` rendition
already carries its own pin (`RenditionMeta.prior`), on the wire and through
the C FFI. But `pick()` and `primary()` ignore it, and no path verifies it.
So the data for "a rendition keyed to a prior" exists in code. What is missing
is the text that defines it, and any behaviour that uses it. Closing this needs
these together:
- the pin defined in SPEC.md, including the per-rendition form (G1);
- dispatch and rendition selection that consult the pin: a sink picks the
rendition whose pin matches a parameter set it holds, and verifies that pin
before decoding, as `resolve_checked` does for `WAI1`;
- a refusal rule for when no pin matches;
- the predecessor link R9 asks for (G5), so that a sink holding A can tell
that B replaces it.
*Status: open; unchanged at link 10.*
**G13: Membership.** See the note at the top.
---
## 7. Adjacent work at the same layer
- **MPEG-AI Part 6** (ISO/IEC 23888-6): CfP responses are due 13 Jan 2027.
Model identity under provenance is shared ground; see
[mpeg-ai-part6-mapping](mpeg-ai-part6-mapping.md).
- **NNC** (ISO/IEC 15938-17): a carriage proposal is in SyfAI (N1809 §3). At
link 05 WAI registers the 2024 edition as the one form a `Prior` MAY declare,
and fixes how an in-band data-unit signature relates to WAI's digest
([prior-carriage](../extensions/prior-carriage.md) §3, §6). Landscape §6.4
covers the rest.
- **VCM carriage** (the main body of N1809): WAI registers no VCM capability,
so this is not a mapping target.
- **UWA〔2026〕042**, 《AI超高清媒体封装与传输协议》 ("AI ultra-high-definition
media encapsulation and transport protocol"): the project was approved on
18 Jun 2026 and is at pre-draft. It sits at the same layer. Watch it; there
is nothing to map yet.
- **JPEG AI Parts 3 and 4** (reference software, conformance): a web search on
2026-09-27 returned a catalogue snippet placing Part 4 at stage 60.00, under
publication. ⚠ The ISO catalogue refused automated access, so neither part's
stage was confirmed from the catalogue itself. Part 4 defines the conformance
tests an entropy-stage claim is checked against, and SPEC.md §7 at link 02
records that WAI's classing of `wai.image.jpegai` waits on a reading of it.
- **IEEE 1857.11-2024**: published. Its conformance approach is described in §5.
---
## 8. What this document does not claim
- It does not claim that a contribution will be made, or that WAI holds
SC 29 accreditation. It is a draft, not submitted.
- It does not treat the MPEG 155 five-topic list as WG 3 text. That list comes
from a secondary report.
- It does not describe any WAI mechanism as adopted by, aligned with, or
endorsed by WG 3.
- It does not present anything attributed to a link of the landing chain as
being in `main`. None of the chain is merged as of 2026-09-30.
- It does not present the reference implementation's prior pin, digest check or
mismatch refusal as core normative WAI text. They are code and a Draft
extension (G1).
- It does not present the emitted-medium rule as implemented. It is
specification text, and the code does not handle post-decode processing
(R5, G10).
- It does not claim that WAI's JPEG AI decode is float-free, reproducible on
every machine as a property of the capability, or checked end to end in CI.
In the tree at link 10 its texts say the same (G11, G8).
- It does not claim that WAI specifies a fallback for a parameter-set
mismatch. SPEC.md's dispatch rule requires the opposite (UC-2, G12). The <!-- allow-phrase: dated record, plain English -->
reference implementation's per-rendition pin is data that nothing checks.
- It does not claim a four-platform CI pass for `main`. The `windows` leg has
not run since about 2026-08-28, and the float decode tests have never run in
CI (§2). It does not claim any CI result for the chain's pull requests.
- It does not claim that WAI's integer route is the only sound one. §5 states
what each route reaches.
---
## Sources
Read on 2026-09-27 unless noted.
1. MPEG Explorations, Part 52: document table and titles.
<https://www.mpeg.org/standards/Explorations/52/>
2. ISO/IEC JTC 1/SC 29/WG 3 **N1809** (w26201), *Exploration on Systems
technology for AI-based media standards (SyfAI)*, 2026-05-02. Read in full
from <https://www.mpeg.org/wp-content/uploads/mpeg_meetings/154_SantaEularia/w26201.zip>.
3. ISO/IEC JTC 1/SC 29/WG 3 **N1752** (w25953), *Exploration on Systems
technologies for coding for machine standards*, 2026-02-09. Read in full from
<https://www.mpeg.org/wp-content/uploads/mpeg_meetings/153_OnLine/w25953.zip>.
4. MPEG meeting schedule. <https://www.mpeg.org/meetings/>
5. *MPEG news: a report from the 155th meeting*, 31 Jul 2026 (secondary; the
AI update framework wording, and that the thread was opened jointly with
WG 7).
<https://multimediacommunication.blogspot.com/2026/07/mpeg-news-report-from-155th-meeting.html>
6. *An Overview of the JPEG AI Learning-Based Image Coding Standard*, arXiv
2510.13867. The PDF was read for the conformance-point, integer-pipeline and
excluded-features passages. <https://arxiv.org/abs/2510.13867>
7. *MLVC: Multi-platform Learned Video Codec for Real-World Deployment*, arXiv
2606.28027, ECCV 2026. Abstract page only.
<https://arxiv.org/abs/2606.28027>
8. IEEE 1857.11 call for proposals, the outcome of the 9 Dec 2021 working
subgroup meeting; its timeline lists the announcement on 21 Dec 2021 (the
URL is dated 24 Dec 2021). Read through a text rendering, because the page
serves an interstitial to scripted clients.
<https://sagroups.ieee.org/dcsc/2021/12/24/call-for-proposals-on-neural-network-based-image-coding/>
IEEE 1857.11-2024 record (Board approval 2024-09-26, published 2024-12-20):
<https://standards.ieee.org/ieee/1857.11/10691/>
9. UWA project approval notice, UWA〔2026〕042. <https://theuwa.com/new-detail/899>
10. In this tree, at link 10 of the landing chain (2026-09-30), and on `main`
where a statement says so: `SPEC.md` opening section, §1, §3, §4, §5 (the
`wai.image.jpegai`, `wai.neural.int_mlicpp` and `wai.prior.nnc` rows), §7
(including *Replicate-soundness*, *`wai.image.jpegai` and the standard
classes* and *Machine-checked totality*), §7.1 (including *Companion
components*) and Appendix A; `extensions/determinism-tiers.md` §1–§6;
`extensions/prior-carriage.md` §1–§7; `extensions/reconstruction-binding.md`
§1–§2; `extensions/jwp-receipts.md` §2.1; `extensions/transparency.md` §6;
`research/2030-transport-landscape.md` §3.2, §6.4 and §8 item 9;
`research/decode-throughput.md` §3–§5; `wai-rs/src/container.rs` (including
`Prior`, `RenditionMeta`, `Rendition`, `pick`, `primary`,
`dispatch_capability` and the `multi_rendition_round_trip` test);
`wai-rs/src/ffi.rs` (the `WAI2` pack and unpack calls); `wai-web/src/lib.rs`
(its `WAI2` binding); `wai-rs/src/codecs/mod.rs` (`ReplicateClass`,
`DeterminismTier`, `capability_replicate_class`); `wai-rs/src/int_entropy.rs`;
`wai-rs/src/int_kernels.rs`; `wai-rs/src/int_mlicpp.rs` (module
documentation); `wai-rs/src/ort_session.rs`; `wai-rs/src/neural/mod.rs`,
`hyperprior.rs`, `context.rs`, `glc.rs`, `video_dcvc.rs`;
`wai-rs/src/jpegai_decode.rs`, `jpegai_synthesis.rs`, `jpegai_pipeline.rs`,
`jpegai_icci.rs`; `wai-rs/src/rans.rs`; `wai-rs/src/bin/wai.rs`;
`wai-rs/tests/neural_native_decode.rs`, `int_kernels_vectors.rs`;
`tools/wai_validate_synthetic.py`; `byte-exact-conformance/src/`; the WAI
workflow definitions. The G6 probe ran against `rans.rs` at
commit 1b3e390.
11. The forge's Actions records for this repository, read on 2026-09-27:
- the run listing (`/api/v1/repos/{owner}/{repo}/actions/runs`), giving
every run of the three WAI workflows from 2026-07-23 to 2026-09-27 with
status, commit, branch and times;
- the per-job status of each of those runs, from the web interface's job
endpoint;
- the job logs: every failed `wai-neural-e2e.yml` run, the three
non-`windows` legs of both jobs in the `afc7ece` run, and the
byte-equality legs of one 2026-09-21 run. A job that was never claimed
has no log.
- the private network's peer status, read from the dev machine: the
`windows` runner's host was offline and last seen 30 days before
2026-09-27.
12. PR #101 (independent of the landing chain), read from its commit message
on 2026-09-30: the neural end-to-end job's link failure, the fixture
generator's thread settings, and the MLIC++ trace (the diverging symbol and
the max |Δ| before and after); and its forge runs, read on 2026-09-30:
`wai-neural-e2e.yml` passed on its head on 2026-09-29.
13. The forge's CI runs for the chain's pull requests (#82, #103–#109, #111,
#112) since 2026-09-28, read on 2026-09-30 through the forge's task list.