SPEC §3.1 defines the pin a manifest carries (conditioning.prior, and a WAI2 rendition's prior): id, sha256, determinism, and the optional form, tier and supersedes. The digest covers the whole parameter set. A set of several files is pinned through its listing, the new registered prior form wai.prior.set. A sink verifies the pin before it decodes, reports a refusal with one of an ordered set of snake_case reasons, and decodes only from the set it verified. Tier is a closed vocabulary; supersedes is informative. A pin that does not resolve makes its capability unsupported for that envelope: SPEC §4 dispatch continues at the fallback, §7.1 selection at the next rendition, and an inert envelope's error names the reasons. A rendition entry may carry id, sha256 and, on a companion, companion_of_id. SPEC §7 gains a Digests subsection, and §8 says how a MINOR revision adds an optional member. In wai-rs: the manifest types are #[non_exhaustive] with builders; a new pin module verifies pins in memory and on disk; Wai::dispatch_verified, WaiMulti::pick and primary check pins and component digests; every neural decoder reads through ParamSource; a codec-free `ffi` feature carries the C envelope surface with wai_envelope_select and wai_neural_decode_ex. wai-web wraps wai::container. New vectors: the int_mlicpp set-form twin and the pin-conformance corpus with an independent Python verifier, plus sha256.json sidecars for the vectors of capabilities that decode against a parameter set. Squashed from wai/stream-a-parameter-set-identity onto link 12. Conflicts are resolved to keep both sides; the integration commits that follow make the two streams agree. Co-Authored-By: Claude Opus 5.5 <[email protected]>
982 lines
64 KiB
Markdown
982 lines
64 KiB
Markdown
# MPEG SyfAI (Exploration 52): WAI's input on the AI update framework
|
||
|
||
> **Status: draft input, not submitted.** A preparation document written in the
|
||
> shape of an input contribution toward MPEG SyfAI's next contribution window,
|
||
> MPEG 156, Hangzhou, 19–23 October 2026. It is not a normative extension.
|
||
> **An input to an MPEG meeting can only be registered by a delegate accredited
|
||
> through an ISO/IEC JTC 1/SC 29 national body. WAI has no such accreditation
|
||
> today**, so nothing here should be planned around an assumed submission. The
|
||
> posture is the same as [mpeg-ai-part6-mapping](mpeg-ai-part6-mapping.md):
|
||
> specify WAI *against* the work so it can become a profile of the eventual
|
||
> text.
|
||
>
|
||
> **Which tree.** WAI's mechanisms are described as they stand in the tree at
|
||
> link 10 of the landing chain: a linear chain of pull requests on the forge,
|
||
> each targeting `main`, merging in order. **As of 2026-09-30 none is merged
|
||
> into `main`.**
|
||
>
|
||
> | link | branch | PR |
|
||
> |---|---|---|
|
||
> | 01 | `feat/wai-video-workflow` | #82 |
|
||
> | 02 | `wai/land-02-determinism-tiers` | #103 |
|
||
> | 03 | `wai/land-03-jpegai-no-libm` | #104 |
|
||
> | 04 | `wai/land-04-codec-registry` | #105 |
|
||
> | 05 | `wai/land-05-prior-nnc` | #106 |
|
||
> | 06 | `wai/land-06-ci-headline-tests` | #107 |
|
||
> | 07 | `wai/land-07-int-decode-kernels` | #108 |
|
||
> | 08 | `wai/land-08-energy-classes` | #109 |
|
||
> | 09 | `wai/land-09-int-mlicpp` | #111 |
|
||
> | 10 | `wai/land-10-receipt-classes` | #112 |
|
||
>
|
||
> Each link contains the ones before it. Everything this note attributes to a
|
||
> link is proposed in that link's pull request and is not in `main`; where
|
||
> `main` differs, the text says so. The CI evidence in §2 is `main`'s, read from
|
||
> the forge on 2026-09-27, and the chain's, from the forge's runs on its pull
|
||
> requests, read on 2026-09-30.
|
||
>
|
||
> This note cross-references rather than repeats. The NNC carriage question is in
|
||
> [2030-transport-landscape](2030-transport-landscape.md) §6.4, and the form WAI
|
||
> registers for it is [prior-carriage](../extensions/prior-carriage.md). The
|
||
> post-decode processing hole is described in §3.2 of the landscape, and the rule
|
||
> that closes it is [determinism-tiers](../extensions/determinism-tiers.md) §2.
|
||
> Provenance over reconstructed output is in
|
||
> [reconstruction-binding](../extensions/reconstruction-binding.md). Where the
|
||
> landscape's text disagrees with a finding here, the gap that finds it says so
|
||
> (G1, G2). Where WAI's own normative text and its reference implementation
|
||
> disagree, the gap names both sides (G1, G12).
|
||
>
|
||
> Everything marked **⚠ unverified** was not seen in a primary source. The source
|
||
> list at the end says what was read and when.
|
||
|
||
---
|
||
|
||
## 1. The exploration, as the primary documents record it
|
||
|
||
| meeting | output document | title | status of this note's read |
|
||
|---|---|---|---|
|
||
| MPEG 151, Daejeon, 30 Jun – 4 Jul 2025 | none published | *Exploration on MPEG Systems support for Coding for Machine technologies* | title only, from the exploration page |
|
||
| MPEG 153, online, 19–23 Jan 2026 | WG 3 N1752 (w25953), dated 2026-02-09 | *Exploration on Systems technologies for coding for machine standards* | **read in full** |
|
||
| MPEG 154, Santa Eulària, 27 Apr – 1 May 2026 | WG 3 N1809 (w26201), dated 2026-05-02 | *Exploration on Systems technology for AI-based media standards (SyfAI)* | **read in full** |
|
||
| MPEG 155, Geneva, 13–17 Jul 2026 | listed, not published on mpeg.org as of 2026-09-27 | same SyfAI title | **not read**; substance known only from a secondary report |
|
||
| MPEG 156, Hangzhou, 19–23 Oct 2026 | — | next meeting | — |
|
||
| MPEG 157, Brisbane, 18–22 Jan 2027 | — | also evaluates MPEG-AI Part 6 CfP responses | — |
|
||
|
||
**When it began.** The exploration was not opened at MPEG 155. N1809's own
|
||
summary says the work "was established" at MPEG 153, where it concerned carriage
|
||
of Video Coding for Machines (VCM) NAL-unit video in ISOBMFF. It carries forward
|
||
an MPEG 151 exploration on systems support for coding for machines. The SyfAI
|
||
title first appears on the MPEG 154 output. The exploration's output documents
|
||
are **WG 3 (MPEG Systems)** documents. The secondary report on MPEG 155
|
||
(source 5) says the AI-update-framework thread was "opened jointly with WG 7"
|
||
(3D Graphics and Haptics). No WG document seen here records that. An input
|
||
should nonetheless be planned for both groups.
|
||
|
||
**What N1809 records from MPEG 154.** Most of the document is draft carriage text
|
||
for VCM in ISO/IEC 14496-15 (`vcm1`/`vcmi` sample entries, a
|
||
`VCMDecoderConfigurationRecord`). The part that concerns WAI is §3, the
|
||
discussion list. Its first item, *flexible AI codec model framework*, records
|
||
this need:
|
||
|
||
- unified terminology (modules, blocks, primitives);
|
||
- system-level support for model **discovery, validation, storage, delivery**;
|
||
- configuration update mechanisms;
|
||
- standardised signalling of **models, weights, metadata, training info**;
|
||
- a potential **model repository** infrastructure.
|
||
|
||
The comments recorded against it ask for a conceptual architecture
|
||
("when/where/how does system fetch"), for requirements covering **security,
|
||
authentication**, level of flexibility and repository, and for a defined scope.
|
||
One comment reads "menifest [sic] shall include inner functionality of codec,
|
||
delivery of model, capalibity [sic], etc." They name AI-based point-cloud coding
|
||
and Feature Coding for Machines (FCM) as starting points. The recommendation
|
||
reads: *"Noted. Bring architectural diagram and requirements."* A use case is
|
||
requested for the next meeting.
|
||
|
||
A second item proposes **a new MPEG standard for storage and carriage of
|
||
compressed neural networks as defined in ISO/IEC 15938-17 (NNC)**. The
|
||
recommendation was to bring a use case. This updates landscape §6.4: the
|
||
question of how the artifact a `Prior` names travels on the wire is being asked
|
||
at the systems layer in WG 3, not only in the NNC amendment.
|
||
|
||
**What the MPEG 155 output adds (⚠ secondary source only).** A July 2026 meeting
|
||
report lists five topics for an *AI update framework*: "(i) a format for AI
|
||
parameter data, (ii) a manifest for updating parameters, (iii) a repository to
|
||
serve them, (iv) integrity checking, and (v) bit exactness together with
|
||
conformance". It adds that "neural inference varies with library and hardware, so
|
||
deciding what conformance means and how to test it remains open". Topics (i) to
|
||
(iv) correspond to items N1809 §3 records: signalling of models and weights,
|
||
configuration update, a repository, and security and authentication. **Topic
|
||
(v), bit exactness with conformance, has no counterpart in N1809.** A search of
|
||
its text for exactness, conformance and integrity finds only a VCM HRD note. So
|
||
this note's topic (v), which §2 row (v), R2, R4 to R7 and G10 answer, rests on the
|
||
secondary report alone. Read the MPEG 155 output when it is published before
|
||
quoting any of the five topics.
|
||
|
||
**The shape an input should take.** N1809 asks for *use cases, an architecture
|
||
and requirements*. It does not ask for technology proposals yet. §3 and §4 below
|
||
are drafted in that shape.
|
||
|
||
---
|
||
|
||
## 2. Mapping: the five framework topics against what WAI specifies
|
||
|
||
"Specified" means normative text in this tree (SPEC.md or an extension).
|
||
"Implemented" means the reference implementation (`wai-rs`) does it.
|
||
"CI-checked" means a workflow on the forge is defined to exercise it. *CI
|
||
scope*, below the table, says which workflows do so and, for `main`, what their
|
||
runs showed. A row marked *reference implementation only* describes code, not
|
||
the standard, and an input cannot present it as WAI's normative position until
|
||
the text catches up (G1).
|
||
|
||
| framework topic | WAI mechanism | state | honest limit |
|
||
|---|---|---|---|
|
||
| (i) AI parameter-data format | **Registered forms, each optional.** A pin MAY declare that the set its digest covers is the listing of a set of files, `wai.prior.set` ([SPEC.md §3.1](../SPEC.md)), or an ISO/IEC 15938-17:2024 (NNC) bitstream, `wai.prior.nnc`; the digest then covers the listing or the coded bitstream ([prior-carriage](../extensions/prior-carriage.md) §2–§3). The model is still never transported in the envelope that decodes against it (SPEC.md's opening section). Informative formats exist: the integer `int_synth.json` model in [neural-conformance](../neural-conformance/README.md); the JPEG AI weight bundle (`weights/*.json` + `norm.json`) read by `try_jpegai_exact`, whose hyper-decoder, context-model and synthesis networks are integer but whose eICCI network and one `norm.json` table are `f32`; and the one-file `WMB1` bundle of `wai.neural.int_mlicpp` — its integer networks and the entropy tables in one file. | Listing form: specified, implemented, CI-checked. NNC form: specified, not implemented. Informative formats: implemented | `wai-rs` has no NNC reader or writer; the reference sink reads `wai.prior.set` and no coded form. NNC is optional and not recommended: its royalty status, and whether conforming decoders reconstruct identical tensors, are not established (prior-carriage §4–§5). |
|
||
| (ii) manifest | SPEC.md §3.1 defines the pin: `conditioning.prior = { id, sha256, determinism, form?, tier?, supersedes? }` in a single-payload (`WAI1`) manifest, and a rendition entry's `prior` in a multi-rendition (`WAI2`) one, beside the entry's `id` and payload `sha256` (§7.1). It names *which* parameter set, exactly — the SHA-256 of the whole set, a set of several files through its listing — and *how* its decode is made reproducible, with a tier from a closed vocabulary. | **specified (SPEC.md §3, §3.1, §7.1) and implemented** | The pin names identity, not location. Sinks of earlier revisions may ignore it (the pin then fails open) or reject the envelope (SPEC.md §3.1, §8). `id` is informative; `supersedes` is informative and never makes a successor satisfy its predecessor's pin. |
|
||
| (iii) repository | **None.** SPEC.md §1 scopes WAI as a container and capability-dispatch standard; nothing in the tree specifies a model store or service. Identity-by-digest makes any store content-addressable, but WAI specifies no store. | out of scope | A repository would be SyfAI's. WAI's input is the identity rule a repository must honour, not the repository. |
|
||
| (iv) integrity checking | SPEC.md §3.1 requires a sink to verify a pin before it decodes, and to refuse one its parameter sets do not satisfy with one of an ordered set of named reasons; a refused pin makes the capability unsupported for that envelope, and dispatch continues at the fallback (§4) or the next rendition (§7.1). A pinned decode reads only the set it verified. The reference implementation does this for `WAI1` and `WAI2` (`Wai::dispatch_verified`, `WaiMulti::pick`, `pin::verify_bytes`, `neural::decode_envelope`, `neural::decode_multi`), in the C ABI and in the browser crate. `decode_envelope_strict` checks the capability's registry class and then the pin before a `replicate` decode. Because the manifest sits inside the envelope bytes the JWP receipt hashes ([jwp-receipts](../extensions/jwp-receipts.md) §2.1), a signed receipt authenticates the pin as well as the payload. | **specified (SPEC.md §3.1, §4, §7.1) and implemented; the pin vectors are checked on the conformance legs** | A digest proves the bytes are the named bytes. It does not say who chose them. That comes from a signed JWP receipt over the envelope, and WAI has no key discovery (landscape §3.6). The lenient `decode_envelope` path decodes an unpinned neural capability without checking; this is honest for perceptual use and is not a replicate path. |
|
||
| (v) bit exactness + conformance | Two named tiers, `entropy-consistency` and `decode-equivalence`, and the rule that a tier covers what the sink **emits**, including post-decode processing ([determinism-tiers](../extensions/determinism-tiers.md) §2). Over a neural capability `replicate` promises `decode-equivalence`, which SPEC.md §7 admits only by an integer / fixed-point decode path with no float operation; `entropy-consistency` does not back it (link 02). A per-capability `ReplicateClass` in the registry: the `NeuralIntegerExact` capabilities at `decode-equivalence`, `wai.neural.int_mlicpp` among them (link 09), and `wai.image.jpegai` alone in `NeuralEntropyExact` at `entropy-consistency` (link 02). `codecs::DeterminismTier` carries the tiers in code (link 02). Frozen corpora of decoded-output digests. | tiers: specified, and a type in code at link 02 (on `main`, text only). Emitted-medium rule: **specified only**. Integer-entropy contract, `ReplicateClass` and the strict check: specified + implemented. Corpora and byte-equality goldens: implemented, and CI-checked on `main` for the capabilities listed under *CI scope*, on the legs that ran. | No manifest field names a tier; a capability's tier comes from its registration. `wai-rs` does not handle post-decode processing or operation-reduction hints. `wai.image.jpegai`'s exact decoder pins one reconstruction, reproducible only under its own build and floating-point conditions, and the reference sink can decode the same capability through a float path (G11). The float MLIC++ decoder agrees with its encoder only by convention between the two implementations; `wai.neural.int_mlicpp` makes that agreement a property of the arithmetic (*CI scope*). On `main`, a malformed stream's outcome depends on the build profile; link 01 fixes the known case (G6). Float capabilities have no corpus, deliberately. |
|
||
|
||
**CI scope, precisely.** Two states are described. For `main`, the workflow
|
||
files and the forge's run records read on 2026-09-27. For the tree at link 10,
|
||
the workflow files and the forge's runs on the chain's pull requests, read on
|
||
2026-09-30.
|
||
|
||
*On `main`.* Three workflows exercise WAI's decode paths. This is what each is
|
||
defined to run at commit 1b3e390, read from the workflow files (afc7ece adds the
|
||
OER vector step):
|
||
|
||
- **`wai-conformance.yml`** runs on any change under `wai/wai-rs/`, the
|
||
conformance corpora or `wai/byte-exact-conformance/`. Its matrix is linux
|
||
x86-64, linux arm64, macOS arm64 and windows x86-64. Each leg runs debug
|
||
builds of three steps:
|
||
- `cargo test --lib` over the pure-Rust features, including `neural_int`.
|
||
This covers the JPEG AI parsing, dequantisation and post-filter unit tests,
|
||
which check closed-form cases.
|
||
- The carriage test.
|
||
- Re-derivation of every cargo-class corpus, including the `int_synth`
|
||
neural corpus.
|
||
|
||
A second job runs the `byte-exact-conformance` crate, described next, in
|
||
release builds on the same four legs.
|
||
- **`wai-byte-equality.yml`** runs only on changes to the listed integer-decode
|
||
sources and vectors. It uses the same four legs and runs
|
||
`byte-exact-conformance` in release builds. It checks:
|
||
- the integer rANS goldens;
|
||
- the `int_synth` output digest;
|
||
- the JPEG AI entropy-stage (mANS) goldens;
|
||
- the `wai.video.int_motion` clip goldens;
|
||
- the `wai.video.int_hyper` clip, and its learned keyframe on its own, which
|
||
is decoded through a small committed `wai.neural.int_hyper` model.
|
||
- **`wai-neural-e2e.yml`** runs only on changes to `wai-rs/src/neural/`,
|
||
`tests/neural_native_decode.rs`, the fixture generator or the workflow
|
||
itself. It has one leg, linux x86-64. `tools/wai_validate_synthetic.py`
|
||
builds minimal synthetic models and bitstreams in the job. The job then runs
|
||
the float `_matches_reference` tests in `tests/neural_native_decode.rs`.
|
||
Those tests check the ELIC, MLIC++, SNAC, DCVC and GLC decoders against a
|
||
reference decode within ±4. So the job is defined to test whether each float
|
||
decoder is correct end to end on one machine. It makes no cross-architecture
|
||
claim, and its models are synthetic, not trained. No run of it has yet
|
||
reached those tests (below).
|
||
|
||
Four limits apply when any of these is read as evidence. They come from the
|
||
forge's run listing, and from each run's per-job status and job logs, all read
|
||
on 2026-09-27 (source 11).
|
||
|
||
- **An offline runner leaves a cancelled job behind, not a gap.** The host of
|
||
the `windows` runner has been offline since about 2026-08-28. The two matrix
|
||
workflows still create its jobs, and no runner claims them. Each such job
|
||
created from 2026-08-28 on was later cancelled, in one of two ways: a newer
|
||
push to the same branch started a new run, or a day or more passed after the
|
||
job was created. A cancelled job marks the whole run as cancelled, even when
|
||
every other job in it passed. The run on `main` at `afc7ece` was still
|
||
waiting on its two `windows` jobs.
|
||
- **Since then, every job on the other three platforms that finished has
|
||
passed.** In the runs of `wai-conformance.yml` and `wai-byte-equality.yml`
|
||
created from 2026-08-28 to 2026-09-27, no linux x86-64, linux arm64 or
|
||
macOS arm64 job failed. Some were cancelled when a newer push superseded
|
||
their run, and no `windows` job ran. The last runs in which the `windows`
|
||
leg also passed date from 2026-08-27 (`wai-conformance.yml`) and 2026-08-26
|
||
(`wai-byte-equality.yml`).
|
||
- No run exists at 1b3e390 itself. The nearest is the run on `main` at
|
||
`afc7ece`. In `wai-rs` that commit adds only the `oer` module, behind its
|
||
own feature, with its vector binary and the workflow steps that run them.
|
||
- In that run, on linux x86-64, linux arm64 and macOS arm64, both jobs
|
||
passed: the corpus job (library tests, carriage test and corpus
|
||
re-derivation) and the `byte-exact-conformance` job. Their logs record no
|
||
failed test.
|
||
- So the cross-architecture evidence for `main` covers three of the four
|
||
platforms. The `windows` leg's latest evidence predates 2026-08-28.
|
||
- **Tests that need model assets pass when the assets are absent (G8).** In
|
||
`wai-neural-e2e.yml` this also covers a partial fixture set. On `main` the
|
||
generator exits 0 if any one of its five harnesses succeeds, and the Rust
|
||
tests whose fixtures are missing then report as passed. PR #101, independent
|
||
of the chain, makes both cases fail the job.
|
||
- **`wai-neural-e2e.yml` has never reached its tests.** The listing holds 21
|
||
runs of it, from 2026-07-23 to 2026-09-26. Eighteen failed and three were
|
||
cancelled.
|
||
- The logs of all eighteen failures show the fixture step generating all
|
||
five synthetic fixtures. The job then stopped before any Rust test ran.
|
||
- Two early runs (2026-07-23 and 2026-07-26) failed at the Rust toolchain
|
||
step with `rustup: command not found`.
|
||
- The other sixteen, which include every run from 2026-07-31 on, failed to
|
||
link the test binary. The prebuilt ONNX Runtime static library that the
|
||
`ort` crate links references C-library symbols (`__isoc23_strtol`,
|
||
`__isoc23_strtoll`, `__isoc23_strtoull`) and, from 2026-08-26 on, a C++
|
||
standard-library symbol (`_M_replace_cold`) that the job's default
|
||
Ubuntu 22.04 image does not provide.
|
||
- No log contains a test result. The float decode tests are *defined* to run
|
||
in CI, and they have never run there.
|
||
|
||
**What the MLIC++ comparison exposes once the job links.** The float
|
||
`wai.neural.mlicpp` comparison then fails on the fixtures as generated, and the
|
||
cause is established. The fixture generator ran ONNX Runtime at its default
|
||
intra-op thread count, while the Rust decoder loads every session
|
||
single-threaded (`ort_session::load_deterministic`). Convolution reductions
|
||
therefore summed in a different order, and one scale landed on the other side of
|
||
a scale-table edge at symbol 416 of 12,288, which desynchronised the rANS decode
|
||
from there. PR #101 (independent of the chain) moves the job to an Ubuntu 24.04
|
||
image, in which the test binary links, and makes the generator load every session
|
||
with the decoder's execution settings. `wai-neural-e2e.yml` passed on the forge on
|
||
#101's head (2026-09-29), and its record reports the MLIC++ comparison going from
|
||
max |Δ| 255 to 0. That fix
|
||
is a convention between two implementations.
|
||
[`wai.neural.int_mlicpp`](../SPEC.md) (link 09, PR #111) makes the agreement a
|
||
property of the arithmetic: the scales come from integer networks, so every
|
||
conforming decoder computes the encoder's bucket for every element at any thread
|
||
count (`wai-rs/src/int_mlicpp.rs`, module documentation).
|
||
|
||
*At link 10*, the chain changes what CI runs:
|
||
|
||
- `wai-conformance.yml` adds, on every leg and in both build profiles, the
|
||
kernel-equivalence tests of link 07 over every committed integer-decoder
|
||
vector — among them the `wai.audio.int_codec` and `wai.splat.int_codec`
|
||
vectors, each decoded against its frozen hash under every kernel policy — and
|
||
the `wai.neural.int_mlicpp` tests of link 09. On its unix legs it adds two
|
||
checks that keep platform maths-library calls off the JPEG AI exact decode
|
||
(link 03). Its library tests add the `oer` and `receipt_claims` features.
|
||
- `wai-byte-equality.yml` adds the `wai.neural.int_mlicpp` golden (link 09).
|
||
- `wai-neural-gates.yml` (link 02) runs, in an Ubuntu 24.04 image, the
|
||
model-free neural-sink tests: the prior-hash refusal, the strict `replicate`
|
||
gate and, at link 09, the `wai.neural.int_mlicpp` dispatch.
|
||
- `wai-sota-vectors.yml` and `wai-jpegai-e2e.yml` (link 06) supply the inputs of
|
||
the full-size integer-model tests and of the JPEG AI whole-decode tests, and
|
||
fail on a missing input (G8).
|
||
- `wai-energy-receipts.yml` (link 08) and `wai-formal-verification.yml`
|
||
(link 01) cover the energy-class receipts and the bounded model-checking
|
||
harnesses.
|
||
- `wai-neural-e2e.yml` is unchanged: at link 10 it still runs in the default
|
||
image and still generates fixtures at the default thread count.
|
||
|
||
On the forge (runs read 2026-09-30), the chain's workflows have run since
|
||
2026-09-28. `wai-sota-vectors.yml`, `wai-jpegai-e2e.yml` and
|
||
`wai-neural-gates.yml` have passed on every run. The conformance and
|
||
byte-equality workflows have run on every unix leg, with a few Linux arm64
|
||
failures; each link's current result is on its pull request.
|
||
`wai-neural-e2e.yml` fails on every link, as on `main`, and passes only with
|
||
#101. The Windows legs have not run: their runner has been offline since late
|
||
August. [determinism-tiers](../extensions/determinism-tiers.md) §6 says this
|
||
from link 11.
|
||
|
||
A statement that a capability "is CI-checked" therefore means that a workflow
|
||
checks it whenever the workflow runs, on the legs that ran. For `main`, that
|
||
means linux x86-64, linux arm64 and macOS arm64 for the byte-exact workflows,
|
||
and nothing yet for `wai-neural-e2e.yml`. For the chain, it means the same three unix
|
||
legs, and `wai-neural-e2e.yml` only once #101 is merged.
|
||
|
||
---
|
||
|
||
## 3. Use cases, in the form N1809 asked for
|
||
|
||
**UC-1: Replicate-grade reconstruction across heterogeneous decoders.** A publisher
|
||
encodes with a learned codec and declares that any conforming decoder reproduces
|
||
the same output bytes. The use case needs a manifest that says which parameter
|
||
set, which reproducibility tier, and what a conformance test compares. The
|
||
tier can be the entropy-stage symbols or the reconstructed medium. WAI's
|
||
integer capabilities are built to reproduce the same output bytes. On `main`,
|
||
the CI workflows check that on well-formed streams for `wai.neural.int_synth`,
|
||
`wai.neural.int_hyper` (through a small committed model), `wai.video.int_motion`,
|
||
`wai.video.int_hyper` and the JPEG AI entropy stage, on the legs that ran (§2):
|
||
since 2026-08-28, linux x86-64, linux arm64 and macOS arm64, where every job
|
||
that finished passed. At link 10 the workflows are also defined to check
|
||
`wai.neural.int_mlicpp` on the byte-exact matrix, and the committed
|
||
`wai.audio.int_codec` and `wai.splat.int_codec` vectors on every conformance leg
|
||
(§2). [determinism-tiers](../extensions/determinism-tiers.md) §6 records
|
||
`wai.neural.int_mlicpp`'s committed vector decoding to its pinned bytes on macOS
|
||
arm64 and on wasm32-wasip2, measured on the development machine. The manifest
|
||
says less than this use case needs. In the reference implementation it names the
|
||
parameter set by a digest over one file unless a form is declared (G1, G2). No
|
||
manifest field names the tier (R2, R4). What a conformance test compares is
|
||
fixed by the capability's registration and corpus, not stated in the manifest.
|
||
|
||
**UC-2: A parameter update that must not decode silently.** A service moves a stream
|
||
from parameter set A to parameter set B. A sink that still holds A must refuse
|
||
B-coded content, or fall back to a declared alternative. Decoding it against A
|
||
is not acceptable, because a learned entropy decode against the wrong parameters
|
||
yields plausible garbage, not an error.
|
||
|
||
What WAI's text says about this case today:
|
||
|
||
- **SPEC.md requires the decode this use case rules out.** Dispatch (SPEC.md §4)
|
||
is keyed on the capability string alone. A sink holding A advertises the
|
||
capability, so step 3 applies: "decode `payload` via the codec registered for
|
||
`cap` (§5) and return". Nothing in §4 looks at the parameter set.
|
||
- **`model_requirement.fallback` does not help.** SPEC.md §3 and §4 steps 4–5
|
||
reach it only when the sink does *not* advertise the primary capability, and
|
||
§7.1 calls it "declarative" in v1.0.
|
||
- **Nor do renditions, as specified.** A §7.1 rendition is chosen by
|
||
capability advertisement or deployer policy. SPEC.md gives a rendition entry
|
||
a `capability` and a `kind`, plus an optional `role` and `companion_of` for
|
||
companion descriptors. It defines no prior field, so the specification
|
||
cannot mark one entry as the one for sinks that hold A.
|
||
- **The reference implementation refuses a mismatch only in a single-payload
|
||
envelope.** When a `WAI1` manifest pins a prior, `resolve_checked` refuses a
|
||
mismatched model with `DecodeError::PriorMismatch`, a hard error. Nothing
|
||
then tries the fallback capability or another rendition. On `main`,
|
||
`fallback` is only printed by the CLI (`inspect`, `unwrap`). At link 04
|
||
(PR #105), `Wai::dispatch_capability` applies SPEC.md §4 steps 3 and 5 and
|
||
`wai decode` and `wai inspect` use it, so the CLI decodes through the fallback
|
||
when the sink lacks the primary capability; it still never does so after a
|
||
`PriorMismatch`. With no pinned prior, the decode runs against whatever model
|
||
is registered.
|
||
- **The reference implementation carries a prior per rendition, and nothing
|
||
uses it.**
|
||
- `RenditionMeta` and `Rendition` each have `prior: Option<Prior>`
|
||
(`container.rs`), with the same meaning as the `WAI1` pin.
|
||
- It is serialised in the `WAI2` manifest, and the C FFI's pack call passes
|
||
it through (`ffi.rs`).
|
||
- The library test `multi_rendition_round_trip` checks that it survives the
|
||
wire. `wai-conformance.yml` runs that test, and it passed in the `afc7ece`
|
||
run on the three platforms that ran (§2).
|
||
- Selection ignores it. `pick()` and `primary()` choose by capability and
|
||
role alone. `resolve_checked` is called only from the `WAI1` decode, so no
|
||
path checks a `WAI2` rendition's pin.
|
||
- The WebAssembly binding's rendition metadata (`renditionsMeta` in
|
||
`wai-web`) drops the field. Its raw manifest string keeps it.
|
||
- So an envelope can carry one rendition pinned to A and another pinned to B
|
||
under the same capability, and `pick()` returns whichever comes first,
|
||
whatever the sink holds.
|
||
|
||
So WAI's normative text has no answer to this use case, and its dispatch rule
|
||
points the other way. The reference implementation refuses a pinned mismatch
|
||
in a single-payload envelope, but offers no declared alternative. For
|
||
renditions it already carries the data a declared alternative would need, a
|
||
prior per rendition, but no selection rule or check uses it. The gap is G12,
|
||
together with G1 and G5.
|
||
|
||
**UC-3: Provenance over the reconstructed output.** A provenance manifest binds a
|
||
hash of the *decoded* pixels, which is portable only when the reconstruction is
|
||
pinned. This connects SyfAI's bit-exactness topic to MPEG-AI Part 6. In the tree
|
||
at link 10, such a binding is checkable by every conforming sink over a
|
||
`decode-equivalence` capability; over `wai.image.jpegai` it holds only for the
|
||
exact decoder's output, under that decoder's conditions
|
||
([reconstruction-binding](../extensions/reconstruction-binding.md) §1–§2). See
|
||
also [mpeg-ai-part6-mapping](mpeg-ai-part6-mapping.md).
|
||
|
||
**UC-4: Energy attributed to a reproducible decode.** The energy of a decode
|
||
can be reported against a work unit that a third party recomputes from the
|
||
codestream. A work unit such as pixel operations needs only the codestream's
|
||
geometry, and WAI's video receipt derives it that way. What needs a
|
||
reproducible decode is binding the figure to a hash of the output that a third
|
||
party can regenerate. See [energy-exposure-mapping](energy-exposure-mapping.md).
|
||
|
||
---
|
||
|
||
## 4. Candidate requirements
|
||
|
||
*Status in this tree.* R1, R2, R3, R7 and R9 are now specified: R1 by SPEC.md §3.1
|
||
(the pin covers the whole parameter set; several files through the listing form
|
||
`wai.prior.set`); R2 by §3.1's closed `tier` vocabulary; R3 by §3.1's ordered
|
||
refusal reasons and §4 / §7.1's dispatch rule; R7 by §7 *Digests* (SHA-256 for
|
||
parameter-set identity, component identity and the decoded output of capabilities
|
||
that decode against a parameter set, with `sha256.json` sidecars on the committed
|
||
vectors); R9 by `supersedes`, which names the predecessor and changes only the
|
||
reason a sink that holds it reports — predecessor content decodes exactly while the
|
||
sink still holds the predecessor. The entries below record the state they were
|
||
written against.
|
||
|
||
Drafted as an input would phrase them. After each one: what WAI does in the tree
|
||
at link 10, or the gap.
|
||
|
||
**R1: Digest over the complete parameter set.** A manifest entry for an AI
|
||
parameter set shall identify it by a cryptographic digest over a canonical
|
||
serialisation of *every* artifact that affects decoder output: networks,
|
||
entropy tables and normative constants. A digest over one file of a multi-file
|
||
set does not meet this.
|
||
*WAI: partial.* The pin covers the one registered file (G2). Two routes to R1
|
||
exist at link 10: a declared form, whose digest covers the whole coded bitstream
|
||
(prior-carriage §2; no form is readable by the reference sink), and a one-file
|
||
model bundle, as `wai.neural.int_mlicpp` uses.
|
||
|
||
**R2: A declared reproducibility tier from a registered vocabulary.** The
|
||
manifest shall state the tier of the decode path the parameter set enables. At
|
||
minimum it shall distinguish entropy-stage reproducibility,
|
||
full-reconstruction reproducibility, and no cross-implementation guarantee.
|
||
*WAI: the vocabulary exists, in the capability's registration, not in the
|
||
manifest.* determinism-tiers names the two tiers, and at link 02 the registry
|
||
carries them in code (`ReplicateClass::neural_tier`, `codecs::DeterminismTier`),
|
||
with `NeuralFloat` claiming neither. The per-envelope contract string is still
|
||
free text (G3), and no manifest field states a tier.
|
||
|
||
**R3: Refuse on mismatch.** A decoder whose resolved parameter set does not match
|
||
the manifest digest shall refuse to decode under that manifest's claims. It
|
||
shall not degrade silently. *WAI: implemented in the reference implementation
|
||
only, and only for single-payload envelopes* (`PriorMismatch`). A `WAI2`
|
||
rendition's pin is carried but not checked. prior-carriage §2 requires refusal
|
||
of an unreadable declared form, but no normative text requires refusal on a
|
||
digest mismatch, and SPEC.md §4 step 3 requires a sink that advertises the
|
||
capability to decode (G1, G12).
|
||
|
||
**R4: Reproducibility claims are admissible only for a qualifying path.** A claim
|
||
that output is reproducible across implementations shall be admissible, for
|
||
full-reconstruction tiers, only when the decode path is integer or fixed-point
|
||
end to end. For entropy-stage tiers, the path may instead be integer at the
|
||
entropy stage or rely on explicit parameter transmission. A reconstruction that
|
||
one implementation reproduces by evaluating float stages in a fixed order is a
|
||
property of that implementation, not of the parameter set. The manifest shall say
|
||
which route a path takes.
|
||
*WAI: specified and implemented at link 02 (PR #103), except the manifest field.*
|
||
SPEC.md §7 lets a neural capability back `replicate` only if its entire decode
|
||
path is integer / fixed-point with no float operation, "the only construction by
|
||
which a neural capability is registered at `decode-equivalence`". Two weaker
|
||
properties MUST NOT back it: `entropy-consistency`, and a reconstruction pinned by
|
||
one implementation's fixed-order float stages. determinism-tiers §3 says a decode
|
||
that derives its entropy parameters in float meets neither tier, and one that
|
||
reads them from the codestream, or derives them in integer arithmetic, and runs
|
||
its later stages in float meets `entropy-consistency` only.
|
||
`decode_envelope_strict` checks the registry class, then the prior. No manifest
|
||
field says which route a path takes. On `main` the texts can be read two ways
|
||
(G10).
|
||
|
||
**R5: The tier covers the emitted medium.** Processing a decoder applies after
|
||
decoding, such as a post-filter or an operation-reduction hint, is part of the
|
||
path the tier describes. A decoder that alters the medium after decode shall not
|
||
claim the tier for what it emits. *WAI: specified, not implemented*
|
||
(determinism-tiers §2, "What a tier covers"). The reference implementation
|
||
does not handle post-decode processing or operation-reduction hints.
|
||
|
||
**R6: Conformance includes non-conforming input.** For a bit-exact tier, the
|
||
decoder's behaviour on a malformed or truncated stream shall be specified,
|
||
normally as rejection with an error. Otherwise build configuration, overflow
|
||
semantics or undefined behaviour change the decode outcome, and "bit exact"
|
||
holds only for well-formed streams. *WAI: the known defect is fixed at link 01;
|
||
the requirement is not normative text* (G6).
|
||
|
||
**R7: Frozen, architecture-independent conformance digests.** Conformance
|
||
material for a bit-exact tier shall be frozen digests of decoded output, with
|
||
the digest algorithm and the canonical output buffer named. It shall be
|
||
re-derived on more than one instruction-set architecture.
|
||
*WAI: implemented, but with three digest algorithms, where a contribution should
|
||
name one per purpose (G7).* `expected_hashes.json` per corpus. Canonical `RGB8`
|
||
in reconstruction-binding §2.
|
||
|
||
**R8: Integrity and authenticity are separate requirements.** Digest match
|
||
proves which bytes. Binding the manifest under a signature proves who named
|
||
them. A framework that specifies only the first cannot say whether an update
|
||
came from the content's publisher. *WAI:* in the reference implementation the
|
||
pin sits in the manifest, inside the envelope bytes a JWP receipt commits to
|
||
(jwp-receipts §2.1), so a signed receipt authenticates it. The pin itself is not
|
||
yet specified in SPEC.md (G1), and key discovery is absent (landscape §3.6;
|
||
transparency §6 disclaims it).
|
||
|
||
**R9: Updates name their predecessor.** A parameter update shall carry the digest
|
||
of the set it replaces and state whether content coded against the predecessor
|
||
remains decodable under the successor. *WAI: gap* (G5).
|
||
|
||
---
|
||
|
||
## 5. Two routes to cross-device consistency
|
||
|
||
A SyfAI conformance notion has to admit both routes the field uses, and it has
|
||
to say what each one reaches. Neither is a defect.
|
||
|
||
- **Integer arithmetic.** In JPEG AI (ISO/IEC 6048), "the entropy decoding
|
||
pipeline employs integer arithmetic neural network operations that can be
|
||
implemented with 8-bit multipliers and 32-bit registers, which allows bitexact
|
||
replication" across devices. JPEG AI "defines a bit-exact conformance point
|
||
only at the end of the entropy decoding pipeline", and its overview states
|
||
that "bit-exact picture reconstruction and lossless coding were deliberately
|
||
excluded from the first version" (arXiv 2510.13867, §IV, §III and §XIII
|
||
respectively). The capabilities WAI's registry classes as
|
||
`NeuralIntegerExact` take the same route through synthesis as well, which is
|
||
what `decode-equivalence` means. `wai.neural.int_mlicpp` (link 09) takes it
|
||
through a channel-autoregressive × checkerboard entropy model, re-deriving its
|
||
scales from integer networks before each pass. WAI's own JPEG AI decoder takes
|
||
it for its entropy decode and its integer networks, but not for its
|
||
dequantisation, latent refinement, post-filters or colour conversion, which
|
||
run in float; in the tree at link 10, SPEC.md §5 says so and registers the
|
||
capability at `entropy-consistency` (G11).
|
||
- **Explicit parameter transmission.** A 2026 learned video codec transmits
|
||
scale parameters through the hyperprior. This "guarantees entropy coding
|
||
consistency across devices without requiring bit-exact arithmetic" at a
|
||
bitrate cost (arXiv 2606.28027, ECCV 2026; abstract read, body not read). The
|
||
abstract does not say whether reconstructions match across devices. The route
|
||
guarantees the entropy stage, so identical reconstructions are not guaranteed
|
||
by it; that is an inference, not something the paper was seen to state. WAI's
|
||
tier text counts this route as reaching `entropy-consistency`, and at link 02
|
||
determinism-tiers §2 says why it stops there and §3 no longer reads against
|
||
that. **No WAI capability is registered as using it** (determinism-tiers §5
|
||
lists none). A contribution should say that plainly rather than imply
|
||
coverage.
|
||
- **Reproducibility by cross-check.** The IEEE 1857.11 call for proposals came
|
||
out of the working subgroup's meeting on 9 Dec 2021, and its timeline lists
|
||
the announcement on 21 Dec 2021. It states that "every proponent who submits
|
||
a complete proposal would be requested to participate in the reproducibility
|
||
check, i.e., to cross-check another proposal". The submitted decoder "can be
|
||
binary executable without source code". The call states no bit-exactness
|
||
requirement. IEEE 1857.11-2024 was published on 2024-12-20.
|
||
|
||
The value WAI can add at this layer is naming the tiers and tying each claim to
|
||
one of them. It does not need to prefer one route.
|
||
|
||
---
|
||
|
||
## 6. Gaps WAI must close before any of this is contributed
|
||
|
||
*Status in this tree.* G1 is closed by SPEC.md §3.1 (the pin is normative text,
|
||
for `WAI1` and per rendition); G2 by the parameter-set shapes of §5 and the
|
||
`wai.prior.set` listing; G3 by the closed `tier` vocabulary, with the tier-less
|
||
reading frozen as the legacy rule; G5 by `supersedes`; G7 by §7 *Digests*; G12 by
|
||
§4 / §7.1's dispatch rule and its implementation for `WAI2`. The entries below
|
||
record the state they were written against.
|
||
|
||
Each gap was found on `main`. Its status says what the tree at link 10 holds.
|
||
|
||
**G1: The prior pin is in the reference implementation, not in the
|
||
specification.** `Prior { id, sha256, determinism }` is defined in `wai-rs`
|
||
(`container.rs`), with an optional `form` at link 05. It is carried in two places
|
||
there: `Conditioning::prior` in a `WAI1` manifest and `RenditionMeta::prior` on
|
||
each `WAI2` rendition. The digest check and mismatch refusal exist only in
|
||
`neural/mod.rs` (`resolve_checked`), and apply to the `WAI1` pin only (G12).
|
||
|
||
On `main` the specification says less, and in one place the opposite: SPEC.md's <!-- allow-phrase: dated record, plain English -->
|
||
opening section says the model is "never transported and never hash-pinned".
|
||
In the tree at link 10 (link 05, PR #106) that sentence reads that the model is
|
||
never transported in the envelope that decodes against it, and that "its
|
||
identity MAY be pinned by digest (a `Prior`), and the pinned artifact's wire form
|
||
MAY be declared". prior-carriage §2 (a Draft extension) shows the `prior` object
|
||
with its members and fixes SHA-256 as the digest; §5 and SPEC.md §7 describe the
|
||
strict check on the pinned prior. What the specification still does not do:
|
||
|
||
- SPEC.md §3's manifest schema defines `conditioning` as `{ "kind" }` and has no
|
||
`prior` field, and SPEC.md §7.1 defines no prior on a rendition entry;
|
||
- no normative text requires a sink to refuse on a digest mismatch;
|
||
prior-carriage §2 says a sink refuses an unreadable form "as it refuses a
|
||
digest mismatch", which presupposes that refusal without stating it;
|
||
- SPEC.md §4 step 3 requires a sink that advertises the capability to decode
|
||
and return, so the only normative dispatch rule leads to a decode on a
|
||
mismatch, not a refusal (UC-2, G12).
|
||
|
||
Landscape §6.4 says "WAI *does* pin model **identity** normatively", and asks
|
||
for SPEC.md's opening sentence to be reconciled with it. The opening sentence is
|
||
reconciled at link 05. The landscape's sentence still goes further than the
|
||
text: until SPEC.md defines the pin (fields, digest, refusal on mismatch), in
|
||
both its single-payload and its per-rendition form, table rows (ii) and (iv) and
|
||
R3 describe code and a Draft extension, not the core standard. An input cannot
|
||
rest on them as WAI's normative position. G2 and G3 should be settled in the same
|
||
change, so that the specified pin is not the one-file, free-text form.
|
||
*Status: narrowed at link 05; open.*
|
||
|
||
**G2: Without a declared form, the prior pin covers one file of a multi-file
|
||
parameter set.** `resolve_checked` hashes the single registered path. Several
|
||
decoders then read sibling files the pin never covers:
|
||
|
||
- the hyperprior, context and GLC paths load `hyper_synthesis.onnx`,
|
||
`synthesis.onnx`, `context.onnx` and CDF tables (`*_cdfs_q<q>.json`) from the
|
||
model's directory (`neural/hyperprior.rs`, `context.rs`, `glc.rs`);
|
||
- the DCVC path loads `temporal_decoder.onnx` (`neural/video_dcvc.rs`);
|
||
- `wai.image.jpegai` registers a **directory**, so a pinned prior on it cannot
|
||
pass verification at all. Hashing reads the path as a file, which fails on a
|
||
directory. This was checked on the dev machine (macOS arm64, rustc 1.98.0),
|
||
which returns `Is a directory (os error 21)`.
|
||
|
||
The CDF tables determine the entropy decode, so this is not only a float-path
|
||
concern. The fix is a digest over a canonical manifest of the whole set, which
|
||
is R1. `wai.neural.int_mlicpp` (link 09) shows one form of it: its model is one
|
||
`WMB1` file holding three integer networks and the entropy tables, so a pinned
|
||
prior's SHA-256 covers everything its reconstruction depends on. A declared form
|
||
(link 05) is the other: its digest covers the whole coded bitstream, but the
|
||
reference sink reads no form. Landscape §8 queue item 9 says "`Prior.sha256`
|
||
currently covers an informal directory". As shown above, a pin on a directory
|
||
cannot verify at all, and where the registered path is a file, `sha256` covers
|
||
that one file. *Status: open for every multi-file decoder.*
|
||
|
||
**G3: The determinism contract is free text classified by substring.**
|
||
`Prior::determinism_class` returns `IntegerExact` when the string contains
|
||
`int`, `fixed` or `integer` and no `f32`, `f64` or `float` token
|
||
(`container.rs`). A contract naming a float runtime without a float token
|
||
classifies as integer-exact. At link 02 the registry class is checked first, so
|
||
such a contract can no longer admit a `replicate` over a float or entropy-only
|
||
capability; for a `NeuralIntegerExact` capability, the prior check still
|
||
verifies the declaration, not the arithmetic (determinism-tiers §3). A
|
||
registered vocabulary belongs in the manifest, which is R2. *Status: narrowed at
|
||
link 02; open.*
|
||
|
||
**G4: No normative parameter carriage format.** This is deliberate, and it is the
|
||
question N1809's NNC carriage proposal puts to WG 3. At link 05, prior-carriage
|
||
registers `wai.prior.nnc` (ISO/IEC 15938-17:2024) as the one form a `Prior` MAY
|
||
declare, optional and not recommended, and registers `wai.tensor.nnc` as a
|
||
candidate payload whose coding is in an unpublished amendment. prior-carriage §4
|
||
tracks that amendment from MPEG's 154th-meeting press release; its text,
|
||
including the data-unit signing it adds, was not read (⚠ unverified). *Status:
|
||
a form registered; carriage remains sink-supplied.*
|
||
|
||
**G5: No update semantics.** WAI has no field relating one parameter set to
|
||
another (R9). prior-carriage §2 forbids pinning an NNC incremental-update
|
||
bitstream on its own and registers no form for a base-plus-updates chain.
|
||
*Status: open.*
|
||
|
||
**G6: On `main`, malformed-stream behaviour depends on the build profile.** The
|
||
rANS bypass escape in `wai-rs/src/rans.rs` runs this loop:
|
||
`raw_val |= v << (j * BYPASS_PRECISION)`. It shifts a `u32` by a
|
||
stream-controlled amount. The shift panics in a debug build and is masked in a
|
||
release build. The `wai-rs` release profile does not enable `overflow-checks`.
|
||
|
||
- **Probe:** a scratch harness `#[path]`-included `rans.rs` at commit 1b3e390
|
||
and decoded 20,000 random 64-byte streams (SplitMix64, seeds 0–19,999)
|
||
against a three-entry CDF chosen to force the escape, four symbols each. It
|
||
was built in both profiles on the dev machine (macOS arm64, rustc 1.98.0,
|
||
release with opt-level 3, LTO and one codegen unit, matching `wai-rs`).
|
||
- **Result:** 18,015 streams returned values in release and panicked with
|
||
"shift left with overflow" in debug. 1,984 decoded identically, and one
|
||
panicked in both for different reasons. No stream produced two different
|
||
values.
|
||
- **Scope:** random input, not conformant streams. The well-formed corpora are
|
||
unaffected. The point is R6: without a specified error behaviour, "bit exact"
|
||
stops at well-formed input.
|
||
- **Status: fixed at link 01 (PR #82) for this escape.** `rans.rs` bounds the
|
||
run of bypass groups at what a 32-bit value can hold, so a malformed stream
|
||
decodes one way in both profiles, and every clamp is a no-op on a well-formed
|
||
stream. SPEC.md §7 (*Machine-checked totality*, informative) records four
|
||
defects in the escape found by bounded model checking and fixed, with the
|
||
conformance vectors unchanged. `wai.neural.int_mlicpp` (link 09) refuses every
|
||
stream value it cannot hold exactly. The fix clamps rather than rejects, and
|
||
no normative text specifies the outcome of a malformed stream, so R6 is not
|
||
met as written.
|
||
|
||
**G7: At least three digest algorithms for decoded output.**
|
||
|
||
- The `int_synth` corpus uses a domain-prefixed BLAKE3
|
||
(`wai_int_synth_vectors`).
|
||
- The `int_synth` byte-equality golden uses SHA-256 over the RGB output
|
||
(`byte-exact-conformance/src/golden.rs`). Reconstruction-binding also uses
|
||
SHA-256, because C2PA validators support it.
|
||
- The `wai.video.int_motion` and `wai.video.int_hyper` goldens, including the
|
||
learned keyframe, use a 64-bit FNV-1a. It is not a cryptographic hash
|
||
(`int_video_golden.rs`, `neural_keyframe_golden.rs`).
|
||
- At link 09, the `wai.neural.int_mlicpp` vector pins its RGB under both FNV-1a
|
||
and SHA-256, and its bucket and symbol sequences under FNV-1a.
|
||
|
||
A CI golden need not be cryptographic to detect an accidental divergence. A
|
||
conformance digest that a receipt or a third party relies on should be. A
|
||
contribution must name one algorithm per purpose (R7), and for conformance
|
||
material that others rely on, the one named should not be FNV-1a. *Status:
|
||
open.*
|
||
|
||
**G8: Tests that need model assets pass when the assets are absent, and some
|
||
capabilities have no CI.** On `main` these tests print a skip message and return
|
||
success when their assets are absent. What CI supplies differs between them:
|
||
|
||
- **Not run, or run without their assets, in CI on `main`:**
|
||
- the whole-decode tests of WAI's JPEG AI decoder, including
|
||
`jpegai_exact_bundle_decodes` in `neural/mod.rs`, which needs the `neural`
|
||
feature;
|
||
- the full-size integer-model tests (`tests/int_hyper_synth.rs`,
|
||
`tests/int_video_sota_keyframe.rs`), which no workflow invokes;
|
||
- the JPEG AI synthesis tests on real weights (`jpegai_synthesis.rs`). They
|
||
run in `cargo test --lib` and skip there, because no weights are supplied;
|
||
- the rest of `tests/neural_native_decode.rs`, including its tests on
|
||
exported or trained models. CI runs only the tests the job's
|
||
`_matches_reference` filter selects. `snac_real_model_matches_reference`
|
||
matches that filter but has no generated fixture, so it skips.
|
||
- **Defined in `wai-neural-e2e.yml` with synthetic fixtures, but never run
|
||
there:** the float `_matches_reference` tests for ELIC, MLIC++, SNAC, DCVC
|
||
and GLC in `tests/neural_native_decode.rs` (§2). They would check
|
||
correctness at ±4 against minimal synthetic models on one leg, not at
|
||
realistic scale and not across architectures.
|
||
- **No CI coverage of any kind on `main`:** `wai.audio.int_codec` and
|
||
`wai.splat.int_codec`. Their features (`audio_int`, `splat_int`) appear in no
|
||
workflow, and `byte-exact-conformance` does not include their sources. Yet
|
||
the registry classes both `NeuralIntegerExact`.
|
||
|
||
In the tree at link 10:
|
||
|
||
- **Link 06 (PR #107).** With `WAI_REQUIRE_SOTA_VECTORS=1` a missing or
|
||
incomplete input fails the headline tests instead of passing; without it the
|
||
skip is written to stderr. `wai-sota-vectors.yml` regenerates the full-size
|
||
`int_hyper` vectors from pinned inputs and runs both full-size tests.
|
||
`wai-jpegai-e2e.yml` fetches pinned JPEG AI inputs, checks them by SHA-256, and
|
||
runs the whole-decode, reconstruction-binding and cross-platform tests. Both
|
||
fail on a skip banner. Still in no workflow: `jpegai_exact_bundle_decodes`,
|
||
`jpegai_live`, and the synthesis tests on real weights.
|
||
- **Link 07 (PR #108).** `wai-conformance.yml` decodes the committed
|
||
`wai.audio.int_codec` and `wai.splat.int_codec` vectors against their frozen
|
||
hashes on every leg, under every kernel policy (`tests/int_kernels_vectors.rs`).
|
||
They are still outside the byte-exact golden suite, and the comparisons with
|
||
the Python reference in `int_audio.rs` and `int_splat.rs` still run in no
|
||
job. determinism-tiers §6 said until link 11 that no CI job enables their
|
||
features; link 11 corrects it. *(2026-09-30: the conformance workflow now also
|
||
names `tests/int_audio.rs` and `tests/int_splat.rs`, declared with their
|
||
features, so the comparisons with the Python reference's hashes run on every
|
||
leg; `wai.splat.sog` is now a candidate that no encoder may emit, SPEC §5.)*
|
||
- **PR #101 (independent of the chain).** Makes the neural end-to-end job link
|
||
and fail on a missing fixture, and fixes the MLIC++ fixture mismatch (§2). At
|
||
link 10 the job is unchanged and still cannot link.
|
||
|
||
What does run without skipping, on `main`, is the small committed integer
|
||
models: the `int_synth` corpus, and the `int_hyper` keyframe through a small
|
||
committed model (§2). For JPEG AI on `main`, the entropy stage is the only part
|
||
with a cross-architecture golden; its parsing, dequantisation and post-filter
|
||
kernels have closed-form unit tests in `cargo test --lib` on every leg of the
|
||
matrix, which since 2026-08-28 means three of the four. A conformance claim must
|
||
cite only the tests that run, on the legs that ran. *Status: addressed at links
|
||
06 and 07. On the forge, `wai-sota-vectors.yml` and `wai-jpegai-e2e.yml` have
|
||
passed on every run since 2026-09-28, and the conformance workflow runs the
|
||
link-07 vectors on its unix legs (runs read 2026-09-30). The neural end-to-end
|
||
job passes only with PR #101.*
|
||
|
||
**G9: No throughput figure.** On `main` the integer decoders are scalar and
|
||
single-threaded, and no decode-speed figure has been measured and published.
|
||
An argument for integer conformance invites a cost question. *Status: closed at
|
||
link 07 (PR #108).* `wai-rs/src/int_kernels.rs` adds fast integer kernels (SIMD on
|
||
aarch64, portable Rust elsewhere, worker threads except on wasm) that give the
|
||
reference's integer for every output element, and
|
||
[decode-throughput](decode-throughput.md) §4 publishes whole-decoder and kernel
|
||
figures measured on one macOS arm64 machine. Its §5 states the limits: one
|
||
machine and architecture, x86_64 not executed, synthetic weights for the largest
|
||
rows, and an equivalence that is argued and tested, not machine-proved.
|
||
|
||
**G10: WAI's texts word the `replicate` requirement ambiguously, one sentence
|
||
reads against its own section, and the tiers exist only as text.** This is the
|
||
gap closest to topic (v). On `main`:
|
||
|
||
- SPEC.md §7 lets a neural capability back `intent = replicate` "only if its
|
||
entire decode path — entropy parameterization *and* synthesis — is integer /
|
||
fixed-point, with no float operation anywhere".
|
||
- SPEC.md §7's closing paragraph calls `entropy-consistency` "the floor for a
|
||
sound neural `replicate`". determinism-tiers §3 says a `replicate` envelope
|
||
"MUST pin a prior that meets at least `entropy-consistency`".
|
||
- Read strictly these compose to the stricter rule, but the wording invites the
|
||
other reading: under determinism-tiers §2, integer entropy arithmetic with
|
||
float synthesis reaches `entropy-consistency`, and a reader who takes "at
|
||
least" as sufficient would admit a `replicate` claim that SPEC.md §7 forbids.
|
||
- determinism-tiers §3 says "a float neural decode meets neither tier", which,
|
||
read literally, excludes the explicit-parameter-transmission route that §2
|
||
counts as reaching `entropy-consistency` (§5 here).
|
||
- `wai-rs` has no type for either tier. `ReplicateClass` is a different
|
||
classification with no entropy-stage member.
|
||
|
||
*Status: closed at link 02 (PR #103), on the strict reading.* SPEC.md §7 and
|
||
determinism-tiers §2–§3 state one position: over a neural capability `replicate`
|
||
promises `decode-equivalence`, admitted only by the integer construction;
|
||
`entropy-consistency` is "necessary … and not sufficient" and MUST NOT back it.
|
||
determinism-tiers §3's sentence now separates a decode that derives its entropy
|
||
parameters in float (neither tier) from one that reads them from the codestream
|
||
or derives them in integer arithmetic (`entropy-consistency`). The registry gains
|
||
an entropy-stage class, `NeuralEntropyExact`, and a tier type,
|
||
`codecs::DeterminismTier`. Still open: R5 (post-decode processing) is specified
|
||
and not implemented.
|
||
|
||
**G11: `wai.image.jpegai`'s normative registration, its registry class and
|
||
its decode paths disagree.** On `main`:
|
||
|
||
- SPEC.md §5 registers `wai.image.jpegai` as an "integer entropy front end
|
||
(mANS) + integer dequant/value path + fixed-order synthesis and post-filters",
|
||
at **decode-equivalence**, "reproducible on every machine", and says "entropy
|
||
and dequant are integer-exact". determinism-tiers §2–§3 and
|
||
reconstruction-binding §1 say the same.
|
||
- `capability_replicate_class` has no arm for it, so it falls to
|
||
`StandardDefined`, the class for a reconstruction "fixed by the named
|
||
codec/standard", and a registry test asserts this. JPEG AI fixes a bit-exact
|
||
conformance point only at the end of entropy decoding (§5), so the standard
|
||
does not fix this reconstruction.
|
||
- The preferred path's dequantisation is floating point (`jpegai_pipeline.rs`):
|
||
the residual-variance stage multiplies and scales in `f64` and rounds to
|
||
`f32`, and the gain stage multiplies by the reciprocal of a gain computed with
|
||
an `f64` `exp()` from the platform maths library, which has no cross-platform
|
||
rounding guarantee. LSBS refinement, the conversion into the integer networks,
|
||
the post-filter chain (including the `f32` eICCI network) and colour
|
||
conversion are float too. The whole decode does not run in CI.
|
||
- `try_jpegai_exact` returns `None`, and the decode falls back to float ONNX
|
||
inference, when the integer bundle's directory or `norm.json` is absent, a
|
||
weight file cannot be read or parsed, or the codestream is outside the
|
||
implemented configuration. Both paths return the same `Decoded::Image`.
|
||
|
||
*Status: closed at link 02 (PR #103) and link 03 (PR #104), by bringing the texts
|
||
and the class down to what the code does.* SPEC.md §5 registers the capability
|
||
at `entropy-consistency` (registry class `NeuralEntropyExact`), describes
|
||
dequantisation, latent refinement, post-filters and colour conversion as float
|
||
in a fixed evaluation order, and records the downgrade. determinism-tiers §5 and
|
||
reconstruction-binding §1–§2 make the pinned reconstruction a property of the
|
||
exact decoder, reproducible under that decoder's build and floating-point
|
||
conditions, which it checks before decoding. Link 03 replaces the platform
|
||
`exp` and roundings with `float_repro`, built from IEEE-754 basic operations,
|
||
guards the decode path against maths-library calls, and pins nine hashes for one
|
||
codestream; determinism-tiers §6 records them identical on four platforms and not
|
||
measured on Windows, macOS x86_64 or x86_64 hardware. `wai-jpegai-e2e.yml`
|
||
defines the whole decode in CI (G8). What remains is stated in the texts, not
|
||
contradicted by them: the float fallback exists and the result does not record
|
||
which path ran, and a `norm.json` with a missing or mistyped key panics in the
|
||
loaders instead of falling back (the `try_jpegai_exact` doc comment says so).
|
||
|
||
**G12: No normative behaviour for a parameter-set mismatch, and a
|
||
per-rendition pin the code carries but never checks.** UC-2 sets out the
|
||
detail.
|
||
|
||
- *Specification:* SPEC.md §4 dispatches on the capability string alone, so a
|
||
sink holding the wrong parameter set must decode. `model_requirement.fallback`
|
||
fires only for a missing capability. SPEC.md §7.1 defines no prior on a
|
||
rendition entry.
|
||
- *Reference implementation:* a `WAI1` pin is checked, and a mismatch is
|
||
refused (`PriorMismatch`) with no declared alternative. A `WAI2` rendition
|
||
already carries its own pin (`RenditionMeta.prior`), on the wire and through
|
||
the C FFI. But `pick()` and `primary()` ignore it, and no path verifies it.
|
||
|
||
So the data for "a rendition keyed to a prior" exists in code. What is missing
|
||
is the text that defines it, and any behaviour that uses it. Closing this needs
|
||
these together:
|
||
|
||
- the pin defined in SPEC.md, including the per-rendition form (G1);
|
||
- dispatch and rendition selection that consult the pin: a sink picks the
|
||
rendition whose pin matches a parameter set it holds, and verifies that pin
|
||
before decoding, as `resolve_checked` does for `WAI1`;
|
||
- a refusal rule for when no pin matches;
|
||
- the predecessor link R9 asks for (G5), so that a sink holding A can tell
|
||
that B replaces it.
|
||
|
||
*Status: open; unchanged at link 10.*
|
||
|
||
**G13: Membership.** See the note at the top.
|
||
|
||
---
|
||
|
||
## 7. Adjacent work at the same layer
|
||
|
||
- **MPEG-AI Part 6** (ISO/IEC 23888-6): CfP responses are due 13 Jan 2027.
|
||
Model identity under provenance is shared ground; see
|
||
[mpeg-ai-part6-mapping](mpeg-ai-part6-mapping.md).
|
||
- **NNC** (ISO/IEC 15938-17): a carriage proposal is in SyfAI (N1809 §3). At
|
||
link 05 WAI registers the 2024 edition as the one form a `Prior` MAY declare,
|
||
and fixes how an in-band data-unit signature relates to WAI's digest
|
||
([prior-carriage](../extensions/prior-carriage.md) §3, §6). Landscape §6.4
|
||
covers the rest.
|
||
- **VCM carriage** (the main body of N1809): WAI registers no VCM capability,
|
||
so this is not a mapping target.
|
||
- **UWA〔2026〕042**, 《AI超高清媒体封装与传输协议》 ("AI ultra-high-definition
|
||
media encapsulation and transport protocol"): the project was approved on
|
||
18 Jun 2026 and is at pre-draft. It sits at the same layer. Watch it; there
|
||
is nothing to map yet.
|
||
- **JPEG AI Parts 3 and 4** (reference software, conformance): a web search on
|
||
2026-09-27 returned a catalogue snippet placing Part 4 at stage 60.00, under
|
||
publication. ⚠ The ISO catalogue refused automated access, so neither part's
|
||
stage was confirmed from the catalogue itself. Part 4 defines the conformance
|
||
tests an entropy-stage claim is checked against, and SPEC.md §7 at link 02
|
||
records that WAI's classing of `wai.image.jpegai` waits on a reading of it.
|
||
- **IEEE 1857.11-2024**: published. Its conformance approach is described in §5.
|
||
|
||
---
|
||
|
||
## 8. What this document does not claim
|
||
|
||
- It does not claim that a contribution will be made, or that WAI holds
|
||
SC 29 accreditation. It is a draft, not submitted.
|
||
- It does not treat the MPEG 155 five-topic list as WG 3 text. That list comes
|
||
from a secondary report.
|
||
- It does not describe any WAI mechanism as adopted by, aligned with, or
|
||
endorsed by WG 3.
|
||
- It does not present anything attributed to a link of the landing chain as
|
||
being in `main`. None of the chain is merged as of 2026-09-30.
|
||
- It does not present the reference implementation's prior pin, digest check or
|
||
mismatch refusal as core normative WAI text. They are code and a Draft
|
||
extension (G1).
|
||
- It does not present the emitted-medium rule as implemented. It is
|
||
specification text, and the code does not handle post-decode processing
|
||
(R5, G10).
|
||
- It does not claim that WAI's JPEG AI decode is float-free, reproducible on
|
||
every machine as a property of the capability, or checked end to end in CI.
|
||
In the tree at link 10 its texts say the same (G11, G8).
|
||
- It does not claim that WAI specifies a fallback for a parameter-set
|
||
mismatch. SPEC.md's dispatch rule requires the opposite (UC-2, G12). The <!-- allow-phrase: dated record, plain English -->
|
||
reference implementation's per-rendition pin is data that nothing checks.
|
||
- It does not claim a four-platform CI pass for `main`. The `windows` leg has
|
||
not run since about 2026-08-28, and the float decode tests have never run in
|
||
CI (§2). It does not claim any CI result for the chain's pull requests.
|
||
- It does not claim that WAI's integer route is the only sound one. §5 states
|
||
what each route reaches.
|
||
|
||
---
|
||
|
||
## Sources
|
||
|
||
Read on 2026-09-27 unless noted.
|
||
|
||
1. MPEG Explorations, Part 52: document table and titles.
|
||
<https://www.mpeg.org/standards/Explorations/52/>
|
||
2. ISO/IEC JTC 1/SC 29/WG 3 **N1809** (w26201), *Exploration on Systems
|
||
technology for AI-based media standards (SyfAI)*, 2026-05-02. Read in full
|
||
from <https://www.mpeg.org/wp-content/uploads/mpeg_meetings/154_SantaEularia/w26201.zip>.
|
||
3. ISO/IEC JTC 1/SC 29/WG 3 **N1752** (w25953), *Exploration on Systems
|
||
technologies for coding for machine standards*, 2026-02-09. Read in full from
|
||
<https://www.mpeg.org/wp-content/uploads/mpeg_meetings/153_OnLine/w25953.zip>.
|
||
4. MPEG meeting schedule. <https://www.mpeg.org/meetings/>
|
||
5. *MPEG news: a report from the 155th meeting*, 31 Jul 2026 (secondary; the
|
||
AI update framework wording, and that the thread was opened jointly with
|
||
WG 7).
|
||
<https://multimediacommunication.blogspot.com/2026/07/mpeg-news-report-from-155th-meeting.html>
|
||
6. *An Overview of the JPEG AI Learning-Based Image Coding Standard*, arXiv
|
||
2510.13867. The PDF was read for the conformance-point, integer-pipeline and
|
||
excluded-features passages. <https://arxiv.org/abs/2510.13867>
|
||
7. *MLVC: Multi-platform Learned Video Codec for Real-World Deployment*, arXiv
|
||
2606.28027, ECCV 2026. Abstract page only.
|
||
<https://arxiv.org/abs/2606.28027>
|
||
8. IEEE 1857.11 call for proposals, the outcome of the 9 Dec 2021 working
|
||
subgroup meeting; its timeline lists the announcement on 21 Dec 2021 (the
|
||
URL is dated 24 Dec 2021). Read through a text rendering, because the page
|
||
serves an interstitial to scripted clients.
|
||
<https://sagroups.ieee.org/dcsc/2021/12/24/call-for-proposals-on-neural-network-based-image-coding/>
|
||
IEEE 1857.11-2024 record (Board approval 2024-09-26, published 2024-12-20):
|
||
<https://standards.ieee.org/ieee/1857.11/10691/>
|
||
9. UWA project approval notice, UWA〔2026〕042. <https://theuwa.com/new-detail/899>
|
||
10. In this tree, at link 10 of the landing chain (2026-09-30), and on `main`
|
||
where a statement says so: `SPEC.md` opening section, §1, §3, §4, §5 (the
|
||
`wai.image.jpegai`, `wai.neural.int_mlicpp` and `wai.prior.nnc` rows), §7
|
||
(including *Replicate-soundness*, *`wai.image.jpegai` and the standard
|
||
classes* and *Machine-checked totality*), §7.1 (including *Companion
|
||
components*) and Appendix A; `extensions/determinism-tiers.md` §1–§6;
|
||
`extensions/prior-carriage.md` §1–§7; `extensions/reconstruction-binding.md`
|
||
§1–§2; `extensions/jwp-receipts.md` §2.1; `extensions/transparency.md` §6;
|
||
`research/2030-transport-landscape.md` §3.2, §6.4 and §8 item 9;
|
||
`research/decode-throughput.md` §3–§5; `wai-rs/src/container.rs` (including
|
||
`Prior`, `RenditionMeta`, `Rendition`, `pick`, `primary`,
|
||
`dispatch_capability` and the `multi_rendition_round_trip` test);
|
||
`wai-rs/src/ffi.rs` (the `WAI2` pack and unpack calls); `wai-web/src/lib.rs`
|
||
(its `WAI2` binding); `wai-rs/src/codecs/mod.rs` (`ReplicateClass`,
|
||
`DeterminismTier`, `capability_replicate_class`); `wai-rs/src/int_entropy.rs`;
|
||
`wai-rs/src/int_kernels.rs`; `wai-rs/src/int_mlicpp.rs` (module
|
||
documentation); `wai-rs/src/ort_session.rs`; `wai-rs/src/neural/mod.rs`,
|
||
`hyperprior.rs`, `context.rs`, `glc.rs`, `video_dcvc.rs`;
|
||
`wai-rs/src/jpegai_decode.rs`, `jpegai_synthesis.rs`, `jpegai_pipeline.rs`,
|
||
`jpegai_icci.rs`; `wai-rs/src/rans.rs`; `wai-rs/src/bin/wai.rs`;
|
||
`wai-rs/tests/neural_native_decode.rs`, `int_kernels_vectors.rs`;
|
||
`tools/wai_validate_synthetic.py`; `byte-exact-conformance/src/`; the WAI
|
||
workflow definitions. The G6 probe ran against `rans.rs` at
|
||
commit 1b3e390.
|
||
11. The forge's Actions records for this repository, read on 2026-09-27:
|
||
- the run listing (`/api/v1/repos/{owner}/{repo}/actions/runs`), giving
|
||
every run of the three WAI workflows from 2026-07-23 to 2026-09-27 with
|
||
status, commit, branch and times;
|
||
- the per-job status of each of those runs, from the web interface's job
|
||
endpoint;
|
||
- the job logs: every failed `wai-neural-e2e.yml` run, the three
|
||
non-`windows` legs of both jobs in the `afc7ece` run, and the
|
||
byte-equality legs of one 2026-09-21 run. A job that was never claimed
|
||
has no log.
|
||
- the private network's peer status, read from the dev machine: the
|
||
`windows` runner's host was offline and last seen 30 days before
|
||
2026-09-27.
|
||
12. PR #101 (independent of the landing chain), read from its commit message
|
||
on 2026-09-30: the neural end-to-end job's link failure, the fixture
|
||
generator's thread settings, and the MLIC++ trace (the diverging symbol and
|
||
the max |Δ| before and after); and its forge runs, read on 2026-09-30:
|
||
`wai-neural-e2e.yml` passed on its head on 2026-09-29.
|
||
13. The forge's CI runs for the chain's pull requests (#82, #103–#109, #111,
|
||
#112) since 2026-09-28, read on 2026-09-30 through the forge's task list.
|