wai: research notes at the chain's final state; determinism-tiers and energy-measurement match the tree #113
6 changed files with 2494 additions and 22 deletions
|
|
@ -445,13 +445,21 @@ same ones for its integer networks:
|
|||
max|Δ| = 1 on 4.446% of samples (macOS arm64, release build, 2026-09-27). CI
|
||||
jobs re-check both bars: `wai-sota-vectors.yml` requires the `int_hyper` cost to
|
||||
stay within 1 dB on that photograph, and `wai-jpegai-e2e.yml` runs the JPEG AI
|
||||
comparison and requires the decoded pixels to hash to a pinned value. Neither
|
||||
workflow has run through the forge yet (§6).
|
||||
- **Compute.** The reference decoders are single-threaded CPU code with no
|
||||
explicit SIMD, in integer arithmetic (i64, with i128 intermediates in places;
|
||||
plus fixed-order float for the JPEG AI exact decoder's float stages). They
|
||||
cannot use the float GPU / NPU paths a float learned decoder runs on. This
|
||||
document states no throughput figure for them.
|
||||
comparison and requires the decoded pixels to hash to a pinned value. Both
|
||||
have run on the forge's Linux runner and passed on every run since 2026-09-28
|
||||
(§6).
|
||||
- **Compute.** The decoders are CPU code in integer arithmetic (i64, with i128
|
||||
intermediates in places; plus fixed-order float for the JPEG AI exact
|
||||
decoder's float stages, which stay single-threaded). Their convolutions, IGDN
|
||||
and the JPEG AI synthesis's attention and element-wise steps run through
|
||||
`int_kernels`: register-blocked tiles, NEON widening multiply-accumulate on
|
||||
aarch64 (portable Rust elsewhere), and worker threads on native targets (one
|
||||
worker on wasm). Each fast kernel gives the same integer for every output as
|
||||
the scalar reference kernel kept beside it, which stays selectable at run
|
||||
time (`Policy::reference()`). They cannot use the float GPU / NPU paths a
|
||||
float learned decoder runs on. Measured throughput, on one machine and with
|
||||
its limits, is in [`research/decode-throughput.md`](../research/decode-throughput.md)
|
||||
§4–§5; no claim in this document depends on those figures.
|
||||
- **Model preparation.** A model is offered at this tier only once exported into
|
||||
the decoder's pinned form, sink-supplied: quantised weights and requantisation
|
||||
constants for the integer networks (and, for the JPEG AI exact decoder's eICCI
|
||||
|
|
@ -485,14 +493,14 @@ day.
|
|||
| Cross-architecture identity of the rANS coder, `int_synth`, and the JPEG AI symbol decode (a reference-encoder codestream, given its distribution indices) | golden-byte assertions passed on Linux x86_64, Linux arm64 and macOS arm64 on main at `afc7ece` (2026-09-27, `wai-conformance.yml`; `wai-byte-equality.yml` last ran on main at `3f50097`, 2026-08-29, all three legs passing), and on Windows x86_64 last on main at `3e7811f` (2026-08-27). Between `3e7811f` and `afc7ece` the suite only gained the video and keyframe goldens; these goldens, the sources they decode and the suite's lockfile are unchanged | `byte-exact-conformance/`; `.github/workflows/wai-conformance.yml`, `wai-byte-equality.yml` |
|
||||
| Cross-architecture identity of integer inter-frame video, and of `int_hyper` on a small learned keyframe | the same suite, passing on Linux x86_64, Linux arm64 and macOS arm64 on main at `afc7ece` (2026-09-27). Windows x86_64 never decoded the keyframe clip: it was added in `d4c922b` (2026-08-28), whose Windows legs were cancelled. Windows decoded the integer video goldens only on the pre-merge commits `c9be2ad`, `256e5dd`, `7dfca26` and `3d34860` (2026-08-25/26), whose video sources, vectors and golden file match main but whose `byte-exact-conformance/Cargo.lock` differs; the change reached main at `3f50097`, after the last Windows run on main | as above |
|
||||
| `wai.image.jpegai` exact decoder: the integer derivation of the distribution indices matches the reference decoder exactly | a test on one machine that skips unless its fixtures (sink-supplied weights, normative tables, codestream) are provided; CI does not provide them | `wai-rs/tests/jpegai_hyperscale_calibration.rs` |
|
||||
| `wai.image.jpegai` exact decoder, whole decode through the one public `decode()` call: it stays within a stated bound of the reference decoder's output (max\|Δ\| ≤ 2, ≤ 5% of samples); two decodes in one process are byte-identical, and a reconstruction binding taken over one verifies the other | `wai-jpegai-e2e.yml` fetches the pinned `jpegai/v1` inputs (an extraction of the reference-software assets, published under WAI's models root), checks each file against the SHA-256 manifest `tools/jpegai-e2e-v1.sha256`, and runs both tests with `WAI_REQUIRE_SOTA_VECTORS=1`, under which a missing input fails instead of skipping. The job also fails on any skip banner in the test log, and requires the decoded pixels to hash to a pinned SHA-256. One 256×256 codestream. The workflow has not run through the forge. Its job passed under `forgejo-runner exec` on the Linux runner's host (x86_64 hardware, 2026-09-27), with the exact decoder as it stood on main at `afc7ece`, which calls the platform `exp`; there the decoded pixels matched the pinned hash, the value measured on macOS arm64. The job's steps have otherwise run on macOS arm64 only, outside a runner | `wai-rs/tests/jpegai_e2e_full.rs`, `jpegai_reconstruction_binding.rs`; `.github/workflows/wai-jpegai-e2e.yml` |
|
||||
| `wai.image.jpegai` exact decoder across platforms: the `jpegai/v1` codestream decodes to the same RGB8 bytes, and to the same bits in the float values the decoder reports after each stage (values inside a stage are not hashed), on each platform measured | measured 2026-09-28 with rustc 1.98.0, release builds, inputs checked against their SHA-256, all four on the source of the change that added this row: macOS arm64 (native, on the development machine); wasm32-wasip2 (wasmtime 49, on the same machine); Linux arm64 (glibc 2.36, a container in an arm64 Linux VM on that machine); Linux x86_64 (glibc 2.36, the x86_64 build run in that container under user-mode emulation, QEMU 7.0, not on x86_64 hardware). All nine pinned hashes matched on each platform. On all four they also matched on an earlier state of that change, before the decode's `exp` and rounding moved to `float_repro`. Before the move the decode called the platform `exp` on every target, and its roundings compiled to calls to the symbols `round`, `roundf` and `rint` on x86_64 and `round` and `roundf` on wasm32, which the link resolves: an x86_64 Linux test executable inspected after the move, whose other code still rounds this way, took them from Rust's compiler builtins, not the C library (for wasm32 this was not checked). Not measured: macOS x86_64 (no x86_64 translation layer is installed on the development machine), Windows, x86_64 hardware, 32-bit targets. The test skips unless the inputs are supplied. `wai-jpegai-e2e.yml` supplies them and requires this test to run and pass on its Linux runner; that workflow has not yet run through the forge | `wai-rs/tests/jpegai_cross_platform.rs` |
|
||||
| `float_repro`, the decode's `exp` and rounding: the same bits over a fixed grid of inputs; `exp` within 2 ULPs of the host math library (1 ULP measured); rounding equal to the standard library's | library unit tests under the library-test step of the conformance workflow (`neural_int`), which has not yet run them on a runner. The pinned-bits and rounding tests never skip. The host comparison asserts the host library's accuracy as much as this module's. Its gate selects by target, not by host: it runs on macOS arm64, on Linux x86_64 and arm64 with glibc, and on WASI, and is reported as ignored, with the reason, elsewhere, Windows included. On the source of the change that added this row it was run against these hosts' `exp`, with a worst case of 1 ULP on each: Apple's math library on macOS arm64; the WASI C library under wasmtime; glibc 2.36 on Linux arm64, and glibc 2.35 in the Ubuntu 22.04 image the workflow's Linux arm64 runner used on 2026-09-28; glibc 2.36 on Linux x86_64 under user-mode emulation, against three of the four `exp` variants glibc chooses among by processor feature (the variant each run used follows from the features glibc's loader reported active): SSE2 (QEMU 7.0, whose emulated processor reports none of AVX, FMA, AVX2 and FMA4), AVX (QEMU 7.2, `-cpu max` with FMA and AVX2 masked by glibc's `hwcaps` tunable, which leaves AVX active) and FMA (QEMU 7.2, `-cpu max`). On a separate probe of 20 million inputs, the SSE2 and AVX variants returned the same bits, and the FMA variant different bits for some of them. Not compared: the FMA4 variant, which glibc chooses on a processor that has FMA4 but not both FMA and AVX2, and which neither emulator offers. Not run on x86_64 hardware or on other glibc versions, where the gate still runs it; a failure there would say that the host's `exp` and this one are more than 2 ULPs apart, not that `float_repro`'s bits changed | `wai-rs/src/float_repro.rs` tests |
|
||||
| The exact decode path calls no platform math library | two checks. Clippy's `disallowed_methods` (`wai-rs/clippy.toml`), denied in each module on the decode path, rejects any use of a listed float method, including `f64::exp(x)` and `.map(f64::exp)`. `tests/jpegai_decode_symbols.rs` lists the symbols of an executable that links only the decoder's entry points and weight loaders. It rejects any C math-library function the executable imports, and any it defines other than the round-to-integral functions (`floor`, `round`, `rint` and the like), whose results IEEE 754 fixes. It allows those definitions because a definition shows what the linker kept, not what the decode calls: linked with GNU ld 2.40 for x86_64 Linux, the debug executable defined the compiler builtins' `floor`, though nothing in it referred to `floor`. On the source of the change that added this row, both checks pass on macOS arm64, the symbol check in debug and release builds. The symbol check also passes, in debug builds, on Linux arm64 (the container above) and on Linux x86_64 under user-mode emulation, linked with GNU ld (reporting that `floor` as allowed) and with LLD. With a platform `exp` put back as `f64::exp(…)`, both fail on macOS arm64 while the decoder's unit tests still pass, and the symbol check fails on Linux x86_64 with GNU ld. With a float `%` put into the decode, the symbol check fails on macOS arm64 and clippy, which does not see operators, passes. The conformance workflow runs both on its unix legs; neither has run on a runner yet | `wai-rs/clippy.toml`, `wai-rs/tests/jpegai_decode_symbols.rs`, `.github/workflows/wai-conformance.yml` |
|
||||
| `wai.image.jpegai` exact decoder, whole decode through the one public `decode()` call: it stays within a stated bound of the reference decoder's output (max\|Δ\| ≤ 2, ≤ 5% of samples); two decodes in one process are byte-identical, and a reconstruction binding taken over one verifies the other | `wai-jpegai-e2e.yml` fetches the pinned `jpegai/v1` inputs (an extraction of the reference-software assets, published under WAI's models root), checks each file against the SHA-256 manifest `tools/jpegai-e2e-v1.sha256`, and runs both tests with `WAI_REQUIRE_SOTA_VECTORS=1`, under which a missing input fails instead of skipping. The job also fails on any skip banner in the test log, and requires the decoded pixels to hash to a pinned SHA-256. One 256×256 codestream. The workflow has run on the forge's Linux runner (x86_64 hardware) and passed on every run since 2026-09-28. Before that, its job passed under `forgejo-runner exec` on the same host (2026-09-27) with the exact decoder as it stood on main at `afc7ece`, which calls the platform `exp`; there too the decoded pixels matched the pinned hash, the value measured on macOS arm64 | `wai-rs/tests/jpegai_e2e_full.rs`, `jpegai_reconstruction_binding.rs`; `.github/workflows/wai-jpegai-e2e.yml` |
|
||||
| `wai.image.jpegai` exact decoder across platforms: the `jpegai/v1` codestream decodes to the same RGB8 bytes, and to the same bits in the float values the decoder reports after each stage (values inside a stage are not hashed), on each platform measured | measured 2026-09-28 with rustc 1.98.0, release builds, inputs checked against their SHA-256, all four on the source of the change that added this row: macOS arm64 (native, on the development machine); wasm32-wasip2 (wasmtime 49, on the same machine); Linux arm64 (glibc 2.36, a container in an arm64 Linux VM on that machine); Linux x86_64 (glibc 2.36, the x86_64 build run in that container under user-mode emulation, QEMU 7.0, not on x86_64 hardware). All nine pinned hashes matched on each platform. On all four they also matched on an earlier state of that change, before the decode's `exp` and rounding moved to `float_repro`. Before the move the decode called the platform `exp` on every target, and its roundings compiled to calls to the symbols `round`, `roundf` and `rint` on x86_64 and `round` and `roundf` on wasm32, which the link resolves: an x86_64 Linux test executable inspected after the move, whose other code still rounds this way, took them from Rust's compiler builtins, not the C library (for wasm32 this was not checked). Not measured: macOS x86_64 (no x86_64 translation layer is installed on the development machine), Windows, x86_64 hardware, 32-bit targets. The test skips unless the inputs are supplied. `wai-jpegai-e2e.yml` supplies them and requires this test to run and pass on its Linux runner; that workflow has run on the forge's Linux x86_64 runner and passed on every run since 2026-09-28 | `wai-rs/tests/jpegai_cross_platform.rs` |
|
||||
| `float_repro`, the decode's `exp` and rounding: the same bits over a fixed grid of inputs; `exp` within 2 ULPs of the host math library (1 ULP measured); rounding equal to the standard library's | library unit tests under the library-test step of the conformance workflow (`neural_int`), which has run them on its macOS, Linux x86_64 and Linux arm64 runners since 2026-09-28. The pinned-bits and rounding tests never skip. The host comparison asserts the host library's accuracy as much as this module's. Its gate selects by target, not by host: it runs on macOS arm64, on Linux x86_64 and arm64 with glibc, and on WASI, and is reported as ignored, with the reason, elsewhere, Windows included. On the source of the change that added this row it was run against these hosts' `exp`, with a worst case of 1 ULP on each: Apple's math library on macOS arm64; the WASI C library under wasmtime; glibc 2.36 on Linux arm64, and glibc 2.35 in the Ubuntu 22.04 image the workflow's Linux arm64 runner used on 2026-09-28; glibc 2.36 on Linux x86_64 under user-mode emulation, against three of the four `exp` variants glibc chooses among by processor feature (the variant each run used follows from the features glibc's loader reported active): SSE2 (QEMU 7.0, whose emulated processor reports none of AVX, FMA, AVX2 and FMA4), AVX (QEMU 7.2, `-cpu max` with FMA and AVX2 masked by glibc's `hwcaps` tunable, which leaves AVX active) and FMA (QEMU 7.2, `-cpu max`). On a separate probe of 20 million inputs, the SSE2 and AVX variants returned the same bits, and the FMA variant different bits for some of them. Not compared: the FMA4 variant, which glibc chooses on a processor that has FMA4 but not both FMA and AVX2, and which neither emulator offers. Not run on x86_64 hardware or on other glibc versions, where the gate still runs it; a failure there would say that the host's `exp` and this one are more than 2 ULPs apart, not that `float_repro`'s bits changed | `wai-rs/src/float_repro.rs` tests |
|
||||
| The exact decode path calls no platform math library | two checks. Clippy's `disallowed_methods` (`wai-rs/clippy.toml`), denied in each module on the decode path, rejects any use of a listed float method, including `f64::exp(x)` and `.map(f64::exp)`. `tests/jpegai_decode_symbols.rs` lists the symbols of an executable that links only the decoder's entry points and weight loaders. It rejects any C math-library function the executable imports, and any it defines other than the round-to-integral functions (`floor`, `round`, `rint` and the like), whose results IEEE 754 fixes. It allows those definitions because a definition shows what the linker kept, not what the decode calls: linked with GNU ld 2.40 for x86_64 Linux, the debug executable defined the compiler builtins' `floor`, though nothing in it referred to `floor`. On the source of the change that added this row, both checks pass on macOS arm64, the symbol check in debug and release builds. The symbol check also passes, in debug builds, on Linux arm64 (the container above) and on Linux x86_64 under user-mode emulation, linked with GNU ld (reporting that `floor` as allowed) and with LLD. With a platform `exp` put back as `f64::exp(…)`, both fail on macOS arm64 while the decoder's unit tests still pass, and the symbol check fails on Linux x86_64 with GNU ld. With a float `%` put into the decode, the symbol check fails on macOS arm64 and clippy, which does not see operators, passes. The conformance workflow runs both on its unix legs, on the forge's runners since 2026-09-28 | `wai-rs/clippy.toml`, `wai-rs/tests/jpegai_decode_symbols.rs`, `.github/workflows/wai-conformance.yml` |
|
||||
| `wai.image.jpegai` scope: the reference codestream is accepted; copies with β, `model_id`, the colour transform or an LSBS flag changed are refused; a flush-to-zero or round-toward-zero environment is detected. The float ONNX path calls the same check | library unit tests over a committed copy of the reference codestream, which never skip and fall under the library-test step of the conformance workflow (`neural_int`); the environment test runs on arm64 only. Run on the four platforms above, on the source of the change that added the cross-platform row (the environment test on macOS arm64 and Linux arm64). The float path's refusal (β and `model_id` edits) is tested only with its ONNX bundle supplied (`jpegai_scope_ort.rs`, skips otherwise) | `wai-rs/src/jpegai_decode.rs` tests, `wai-rs/tests/vectors/jpegai/` |
|
||||
| SPEC §7's replicate rule: the registry class decides first, so an integer-declared prior over `wai.image.jpegai` or a float codec is refused; `wai.image.jpegai` is the only `NeuralEntropyExact` capability; the standard codecs sit in the classes SPEC §7 lists, and the tolerance and unclassified ones are not refused; an encoder that pins no prior (`wai wrap`) writes `replicate` only over the standard classes, for the capability and for any declared fallback, or, in the `WAI2` companion form, for the base payload's capability and each companion's | library unit tests (`codecs`, `container`) that never skip and fall under the same library-test step. The strict-decode test in `tests/neural_native_decode.rs` needs the `neural` feature; `wai-neural-gates.yml` runs it on the Linux runner, in an Ubuntu 24.04 image because the `neural` test binary does not link in that runner's default 22.04 image. The workflow has not run through the forge; its job passed under `forgejo-runner exec` on the Linux runner's host (Linux x86_64, 2026-09-27). The test that runs the `wai wrap` binary needs the `codecs` feature, which no CI job builds. Otherwise run so far on macOS arm64 only | `wai-rs/src/codecs/mod.rs`, `container.rs` tests; `wai-rs/tests/neural_native_decode.rs`, `cli_wrap_intent.rs` |
|
||||
| `wai.neural.int_hyper` and `wai.video.int_hyper` at full model scale (the pretrained `mbt2018_mean`, exported to integers): the Rust decode reproduces the export tool's integer reference byte for byte (synthesis, hyper-decoder, the whole conditional bitstream, an eight-frame clip), and costs at most 1 dB against the same model's float decode on one pinned photograph | `wai-sota-vectors.yml` regenerates the vectors on the runner from pinned inputs (the photograph's pixels and the checkpoint's bytes by SHA-256, the Python stack by version) and runs both tests with `WAI_REQUIRE_SOTA_VECTORS=1`, under which a missing vector fails instead of skipping. The job also fails on any skip banner in the test log, and runs `require_vectors_rule.rs`, which checks that without the switch a variable naming absent or incomplete inputs fails as well. Rust is checked against the export of the same run on the same machine: the vectors come from float computation, and some of them differed between machines and between runs, so none of their hashes is pinned and none is compared across machines. The workflow has not run through the forge. Its job passed under `forgejo-runner exec` on the Linux runner's host (x86_64 hardware, 2026-09-27) in an earlier form that did not yet run `require_vectors_rule.rs`; its current steps have run on macOS arm64 only, outside a runner | `wai-rs/tests/int_hyper_synth.rs`, `int_video_sota_keyframe.rs`, `require_vectors_rule.rs`; `.github/workflows/wai-sota-vectors.yml` |
|
||||
| `wai.audio.int_codec`, `wai.splat.int_codec` | committed vectors that never skip, matched against an independent Python integer reference; their cargo features (`audio_int`, `splat_int`) are enabled by no CI job | `wai-rs/tests/int_audio.rs`, `int_splat.rs` |
|
||||
| SPEC §7's replicate rule: the registry class decides first, so an integer-declared prior over `wai.image.jpegai` or a float codec is refused; `wai.image.jpegai` is the only `NeuralEntropyExact` capability; the standard codecs sit in the classes SPEC §7 lists, and the tolerance and unclassified ones are not refused; an encoder that pins no prior (`wai wrap`) writes `replicate` only over the standard classes, for the capability and for any declared fallback, or, in the `WAI2` companion form, for the base payload's capability and each companion's | library unit tests (`codecs`, `container`) that never skip and fall under the same library-test step. The strict-decode test in `tests/neural_native_decode.rs` needs the `neural` feature; `wai-neural-gates.yml` runs it on the Linux runner, in an Ubuntu 24.04 image because the `neural` test binary does not link in that runner's default 22.04 image. The workflow has run on the forge and passed on every run since 2026-09-28; before that its job passed under `forgejo-runner exec` on the Linux runner's host (Linux x86_64, 2026-09-27). The test that runs the `wai wrap` binary needs the `codecs` feature, which no CI job builds. Otherwise run so far on macOS arm64 only | `wai-rs/src/codecs/mod.rs`, `container.rs` tests; `wai-rs/tests/neural_native_decode.rs`, `cli_wrap_intent.rs` |
|
||||
| `wai.neural.int_hyper` and `wai.video.int_hyper` at full model scale (the pretrained `mbt2018_mean`, exported to integers): the Rust decode reproduces the export tool's integer reference byte for byte (synthesis, hyper-decoder, the whole conditional bitstream, an eight-frame clip), and costs at most 1 dB against the same model's float decode on one pinned photograph | `wai-sota-vectors.yml` regenerates the vectors on the runner from pinned inputs (the photograph's pixels and the checkpoint's bytes by SHA-256, the Python stack by version) and runs both tests with `WAI_REQUIRE_SOTA_VECTORS=1`, under which a missing vector fails instead of skipping. The job also fails on any skip banner in the test log, and runs `require_vectors_rule.rs`, which checks that without the switch a variable naming absent or incomplete inputs fails as well. Rust is checked against the export of the same run on the same machine: the vectors come from float computation, and some of them differed between machines and between runs, so none of their hashes is pinned and none is compared across machines. The workflow has run on the forge's Linux runner (x86_64 hardware) and passed on every run since 2026-09-28 | `wai-rs/tests/int_hyper_synth.rs`, `int_video_sota_keyframe.rs`, `require_vectors_rule.rs`; `.github/workflows/wai-sota-vectors.yml` |
|
||||
| `wai.audio.int_codec`, `wai.splat.int_codec` | committed vectors that never skip, matched against an independent Python integer reference; the conformance workflow's fast-kernel steps enable their cargo features (`audio_int`, `splat_int`) and decode both vectors against their frozen hashes under every kernel policy (`tests/int_kernels_vectors.rs`); the comparisons with the Python reference in `int_audio.rs` and `int_splat.rs` run in no CI job | `wai-rs/tests/int_audio.rs`, `int_splat.rs` |
|
||||
| `wai.neural.int_mlicpp` on its committed vector (synthetic seeded weights; a 64×64 image, 8 latent channels in 4 slices): the decode reproduces the export tool's independent integer reference byte for byte — the RGB, and every scale bucket and symbol of the latent stream in coding order (512 symbols over all 16 buckets of the scale table, 47 of the symbols coded by the rANS escape) — under the reference kernels and under the fast kernels on 1, 2, 3, 5 and all available threads; one integer unit changed in each of four chosen parameters, one on each path that feeds a value (a hyper-decoder weight, an entropy-network weight, a mean-head bias, a synthesis weight), changes the image; a header whose shapes the model cannot produce, or on which a network would form a buffer over the cap, is refused before anything is allocated (`tests/int_mlicpp_preflight.rs`, which counts allocations); damaged payloads are refused or decode, and never panic in an overflow-checked build | measured 2026-09-29 with rustc 1.98.0 on the source of the change that added this row: macOS arm64 native (the wai-rs tests, with overflow checks on and off; the `byte-exact-conformance` golden, release) and wasm32-wasip2 under wasmtime 49.0.1 (the same tests and golden, release; on wasm the fast kernels run single-threaded as portable scalar code, and `usize` is 32-bit). All pinned hashes matched on both. Not measured: x86_64 — the development machine has no x86_64 translation layer installed, and an x86_64-apple-darwin build of the golden compiles but cannot execute there — nor Linux or Windows. The golden runs on every leg of the byte-exact matrix (`wai-conformance.yml`, `wai-byte-equality.yml`) and `tests/int_mlicpp.rs` on every leg of `wai-conformance.yml` in both profiles; neither has run through the forge yet. The dispatch through `decode_envelope_strict` (the prior pinning the one-file model bundle, the strict gate) runs in `wai-neural-gates.yml`. The weights are synthetic, so nothing here measures compression quality | `wai-rs/tests/int_mlicpp.rs`, `wai-rs/tests/int_mlicpp_preflight.rs`, `byte-exact-conformance/src/int_mlicpp_golden.rs`, `wai-rs/tests/neural_native_decode.rs`; `tools/wai_int_mlicpp_export.py` |
|
||||
|
||||
So at `entropy-consistency`, WAI's JPEG AI symbol decode is demonstrated across
|
||||
|
|
|
|||
|
|
@ -413,12 +413,15 @@ uncertainty.
|
|||
|
||||
### 2.4 Counter filtering state (`OnChipCounter`)
|
||||
|
||||
Some on-chip counters filter their output as a side-channel defence. Intel's
|
||||
advisory INTEL-SA-00389 (CVE-2020-8694, CVE-2020-8695) describes RAPL energy
|
||||
filtering: unpredictable noise added to the energy reported, typically under
|
||||
2 % standard deviation in readings at 1 s intervals, growing to 5–15 % at
|
||||
100 ms, with larger variance possible at thermal design power. In most cases
|
||||
enabling SGX enables it, and TDX uses the filtered energy information. Software
|
||||
Some on-chip counters filter their output as a side-channel defence. The
|
||||
technical guidance published with advisory INTEL-SA-00389 (*Running Average
|
||||
Power Limit Energy Reporting*, CVE-2020-8694, CVE-2020-8695) describes RAPL
|
||||
energy filtering: noise added to the energy reported, typically under 2 %
|
||||
standard deviation in readings at 1 s intervals, growing to 5–15 % at 100 ms,
|
||||
with larger variance possible at thermal design power. Filtered energy is
|
||||
reported when SGX is enabled or system software has enabled filtering (on some
|
||||
implementations the processor sets it during boot to protect SGX), and TDX uses
|
||||
the filtered energy information. Software
|
||||
that can read model-specific registers sees it as `IA32_MISC_PACKAGE_CTLS[0]`
|
||||
(`ENERGY_FILTERING_ENABLE`), supported where `IA32_ARCH_CAPABILITIES[11]` is
|
||||
set; once set, that bit cannot be cleared by software.
|
||||
|
|
|
|||
887
wai/research/2026-09-global-sweep.md
Normal file
887
wai/research/2026-09-global-sweep.md
Normal file
|
|
@ -0,0 +1,887 @@
|
|||
# 2026-09 global sweep — record of a multi-language landscape scan
|
||||
|
||||
> **Provenance note.** This is a record, not a normative extension and not a
|
||||
> conformance statement. It captures what a multi-language sweep of media
|
||||
> standards, deployments, energy measurement and regulation found on
|
||||
> **2026-09-26**, what the WAI tree holds and in what form, and which gaps the
|
||||
> findings expose. Nothing in it has been implemented by writing it down.
|
||||
>
|
||||
> It is a companion to [`2030-transport-landscape.md`](2030-transport-landscape.md)
|
||||
> (cited below as *the 2030 landscape*), which remains the analysis and the ranked
|
||||
> ingest queue. This record does not restate that document. Where an item is
|
||||
> already there it is cited by section, and §8 lists the places where this sweep
|
||||
> updates or corrects it. Two notes committed with this record take up standards
|
||||
> inputs it found untracked: [`syfai-input.md`](syfai-input.md) and
|
||||
> [`energy-exposure-mapping.md`](energy-exposure-mapping.md).
|
||||
>
|
||||
> **How much to trust each line.** The sweep gathered the external claims on
|
||||
> 2026-09-26. Every P row but one was then read again against its source on
|
||||
> 2026-09-27; §1.1 says how. The record was re-checked against its sources and
|
||||
> against the tree before publication. Every row carries a grade:
|
||||
>
|
||||
> | grade | meaning |
|
||||
> |---|---|
|
||||
> | **P✓** | the cited issuing-body page, datatracker entry or paper was **read or re-read on 2026-09-27**, or on the later date the row gives, and contains the fact as stated |
|
||||
> | **P** | an issuing-body source is recorded by link or identifier but was **not re-read** after the sweep |
|
||||
> | **tree** | the fact comes from another document in this tree, cited by section, and carries that document's own sourcing |
|
||||
> | **R** ⚠ | reported by the sweep from its gathered sources, but no link was kept. Re-verify before quoting outward |
|
||||
> | **S** ⚠ | secondary source only (blog, trade press, vendor material, a co-organiser's page) |
|
||||
> | **N** ⚠ | seen only in search-result snippets, body not read |
|
||||
> | **C** ⚠ | sources conflict |
|
||||
>
|
||||
> Anything graded R, S, N or C is unverified.
|
||||
>
|
||||
> **Which tree.** The repo-side claims (§5.3, §6, §7) describe two states.
|
||||
> `main` is the published tree. Every file under `wai/` that those sections cite
|
||||
> by line is unchanged on `main` since afc7ece (2026-09-27), so a line number
|
||||
> given for `main` holds at that commit. The WAI work that answers most of the
|
||||
> gaps below lands as a linear chain of pull requests on the forge. Each targets
|
||||
> `main`, and they merge in order. **As of 2026-09-30 none is merged into
|
||||
> `main`.**
|
||||
>
|
||||
> | link | branch | PR |
|
||||
> |---|---|---|
|
||||
> | 01 | `feat/wai-video-workflow` | #82 |
|
||||
> | 02 | `wai/land-02-determinism-tiers` | #103 |
|
||||
> | 03 | `wai/land-03-jpegai-no-libm` | #104 |
|
||||
> | 04 | `wai/land-04-codec-registry` | #105 |
|
||||
> | 05 | `wai/land-05-prior-nnc` | #106 |
|
||||
> | 06 | `wai/land-06-ci-headline-tests` | #107 |
|
||||
> | 07 | `wai/land-07-int-decode-kernels` | #108 |
|
||||
> | 08 | `wai/land-08-energy-classes` | #109 |
|
||||
> | 09 | `wai/land-09-int-mlicpp` | #111 |
|
||||
> | 10 | `wai/land-10-receipt-classes` | #112 |
|
||||
>
|
||||
> Each link contains the links before it, so the tree at link 10 holds all ten.
|
||||
> What a row says a link does was read from the files of that tree. Everything
|
||||
> attributed to a link is proposed in that link's pull request and is not in
|
||||
> `main`. The forge's CI runs for the chain's pull requests were read on
|
||||
> 2026-09-30. The workflows the chain adds or changes have run on the forge's
|
||||
> runners since 2026-09-28; `wai-sota-vectors.yml`, `wai-jpegai-e2e.yml` and
|
||||
> `wai-neural-gates.yml` have passed on every run. `wai-neural-e2e.yml` fails on
|
||||
> every link and passes on the head of PR #101, the MLIC++ fixture fix, which is
|
||||
> independent of the chain. Link 11, which carries this record, also corrects
|
||||
> determinism-tiers §5–§6 and energy-measurement §2.4 where this record found
|
||||
> them out of date.
|
||||
>
|
||||
> **Naming.** Standards bodies, public institutions, research institutes, papers
|
||||
> and standards are named where a source names them. Companies are not named in
|
||||
> this record's prose. Where a row rests on a company's own page that was read,
|
||||
> the page's URL or document identifier is given as the citation, because the
|
||||
> row cannot be checked without it. A company whose material was not read is
|
||||
> neither named nor cited. Interface and platform names (RAPL, NVML, IOReport,
|
||||
> Apple Silicon, ONNX Runtime) are used as identifiers, the way the tree's own
|
||||
> texts use them, and so are file paths in the tree. Quotations from the tree and
|
||||
> from sources are verbatim, except that a company name inside one is elided
|
||||
> with "…".
|
||||
>
|
||||
> Posture throughout: record facts about the field; say what they mean for
|
||||
> WAI's own text. Nothing here is framed as a contest with any other effort.
|
||||
|
||||
---
|
||||
|
||||
## 1. Method and coverage limits
|
||||
|
||||
**Scans.** Seven scans ran on 2026-09-26. Three were organised by language and
|
||||
four by topic:
|
||||
|
||||
1. Chinese-language sources.
|
||||
2. Japanese and Korean sources.
|
||||
3. European languages and Brazil.
|
||||
4. Standards bodies and the state of the art in neural codecs.
|
||||
5. Media energy measurement.
|
||||
6. Rest of world and industry deployment.
|
||||
7. Regulation.
|
||||
|
||||
**The search budget ran out.** The shared web-search budget was exhausted
|
||||
partway through the scans. Reports were written from sources already gathered,
|
||||
so coverage is uneven. **"None found" in this record means none found in what
|
||||
was searched. It does not mean none exists.** The areas the sweep did not reach:
|
||||
|
||||
- energy figures for codecs (joules per decode, per encode, per stream);
|
||||
- 2026 hardware media engines and NPU decode support, including for AV2 and
|
||||
learned codecs;
|
||||
- the current C2PA specification version and how manifests behave across
|
||||
re-encode;
|
||||
- the literature on deterministic neural decoding, beyond the papers the sweep
|
||||
read;
|
||||
- ISO catalogue stage codes (iso.org returned HTTP 403 to automated fetch, as it
|
||||
did for the 2030 landscape);
|
||||
- Nordic and Dutch sources;
|
||||
- Korean ATSC 3.0 developments in 2025–26;
|
||||
- which institutions are behind the Chinese integer learned-codec work.
|
||||
|
||||
**Verification.** No adversarial verification pass ran during the sweep. The
|
||||
2030 landscape used two lenses; this record grades by source quality instead.
|
||||
The repo-side claims were checked against the tree by reading the named files
|
||||
and by the greps each row states.
|
||||
|
||||
### 1.1 How the sources were read
|
||||
|
||||
- *Fetched as raw text and searched:* the IETF drafts (ietf.org archive), their
|
||||
revisions, states and dates (datatracker API), RFC 9943 (rfc-editor.org), the
|
||||
arXiv abstracts (arXiv API), and DVB BlueBook S100 (PDF converted to text).
|
||||
- *Read through a web-fetch tool that returns a model's extraction of the page,
|
||||
with verbatim quotes requested:* the mpeg.org pages (meetings; the 153rd, 154th
|
||||
and 155th meeting pages; Exploration 52; MPEG-B Part 11), the JPEG
|
||||
111th-meeting press release, the AOMedia AV2 landing page and press release,
|
||||
two dvb.org news items, the 5G-MAG workshop page, the UWA project page, and one
|
||||
CDN operator's MoQ documentation. Each was read once. The quoted phrases below
|
||||
are what the tool returned, not a verbatim archive.
|
||||
|
||||
The pages read later on 2026-09-27 (Exploration 45, the MPEG-B Part 19 and
|
||||
MPEG-AI Part 5 pages, and EU C(2026) 3472 with its annexes) say so in their
|
||||
rows. The one P row not re-read on 2026-09-27 is the
|
||||
FAMIUM citation (§3.3).
|
||||
|
||||
---
|
||||
|
||||
## 2. Standards status
|
||||
|
||||
Stages as observed on the dates given. Stages move, so treat every row as a dated
|
||||
observation.
|
||||
|
||||
### 2.1 ISO/IEC JTC 1/SC 29 (MPEG) and ITU-T/JVET
|
||||
|
||||
| item | stage | date | source | grade |
|
||||
|---|---|---|---|---|
|
||||
| MPEG 155, Geneva | held | 13–17 Jul 2026 | https://www.mpeg.org/meetings/ | P✓ |
|
||||
| MPEG 156, Hangzhou | scheduled | 19–23 Oct 2026 | same | P✓ |
|
||||
| MPEG 157, Brisbane | scheduled | 18–22 Jan 2027 | same | P✓ |
|
||||
| MPEG 158, Geneva | scheduled | 19–23 Apr 2027 | same | P✓ |
|
||||
| **SyfAI — Exploration 52**, *Systems technology for AI-based media standards*, an MPEG Systems exploration (MPEG Systems is SC 29/WG 3, as 2030 §6.5 records) | exploration. The page lists documents at **MPEG 151** (*MPEG Systems support for Coding for Machine technologies*), **MPEG 153** (*Systems technologies for coding for machine standards*, dated 2026-02-12), **MPEG 154** (first under the SyfAI title, dated 2026-07-14) and **MPEG 155** (SyfAI title, listed without a date) | MPEG 151 onward | https://www.mpeg.org/standards/Explorations/52/ | P✓ |
|
||||
| — when "SyfAI" began | the secondary report says SyfAI "started at the 153rd MPEG meeting"; the part page shows the SyfAI title first at MPEG 154, on a line of documents going back to MPEG 151 | — | the part page, and https://multimediacommunication.blogspot.com/2026/07/mpeg-news-report-from-155th-meeting.html | **C** ⚠ |
|
||||
| — its **AI update-framework thread**, opened **jointly with WG 7**. Five topics: a format for AI parameter data, a manifest for updating parameters, a repository to serve them, integrity checking, and "bit exactness together with conformance" | opened | MPEG 155 | the secondary report above. The MPEG 155 Exploration 52 document and any joint WG 3 / WG 7 output were **not read** | **S** ⚠ |
|
||||
| ISO/IEC 23001-11 Green Metadata, 4th edition | WD dated 2026-03-02 (MPEG 153); **CD text** listed under MPEG 155 with the date 2026-09-21. The text is WG 3 N1864, and its cover is dated 2026-08-28 ([`energy-exposure-mapping.md`](energy-exposure-mapping.md) §1) | — | https://www.mpeg.org/standards/MPEG-B/11/ | P✓ / tree (document number, cover date) |
|
||||
| — earlier editions | 3rd ed. 2023; Amd 1:2024 (EVC); Amd 2:2025 (display power reduction). The part page shows no publication years | — | 2030 landscape §5.1 | tree |
|
||||
| — its content | the part page describes it as metadata "enabling a decoder to consume less energy while providing a usable video". Tool list (decoder complexity metrics, operation-reduction metadata, quality metrics, display power reduction) from 2030 §5.1. The CD text was not read for this record. [`energy-exposure-mapping.md`](energy-exposure-mapping.md) §4 searched it for *signature*, *hash*, *attest*, *certif*, *joule*, *watt*, *measured* and *measurement*, found none, and describes its metadata as predictive | — | part page; 2030 §5.1; energy-exposure-mapping §4 | P✓ (description) / tree (tools; no measured or signed energy in the CD) |
|
||||
| ISO/IEC 23001-19 Carriage of Green Metadata | **advanced to publication (IS) at MPEG 153** by WG 3. The part page is consistent with this: edition status "ongoing", newest document at MPEG 152 (2025-10-24), titled *Potential improvement of ISO/IEC DIS 23001-19* — the DIS stage before the MPEG 153 promotion | MPEG 153 | https://www.mpeg.org/153rd-meeting-of-mpeg/ ; https://www.mpeg.org/standards/MPEG-B/19/ (both read 2026-09-27) | P✓ |
|
||||
| MPEG-2 Systems 10th ed. WD: adds *MPEG-Green data transport* | WD | MPEG 155 | — | **R** ⚠ |
|
||||
| ISO/IEC 23888-2 VCM | promoted to DIS at MPEG 153; FDIS expected late 2026 | — | https://www.mpeg.org/153rd-meeting-of-mpeg/ | P✓ |
|
||||
| ISO/IEC 23888-4 FCM | CD issued at MPEG 154 | — | https://www.mpeg.org/154th-meeting-of-mpeg/ | P✓ |
|
||||
| ISO/IEC 23888-5 (MPEG-AI Part 5), AI-based point cloud coding | edition 1, status "ongoing". The part page lists one document, *CTC on AI-based point cloud coding*, at MPEG 155, dated 2026-09-06. The sweep's "common test conditions **finalised**" is not established by the page, and the item is not on the MPEG 153, 154 or 155 meeting pages | MPEG 155 | https://www.mpeg.org/standards/MPEG-AI/5/ (read 2026-09-27) | P✓ (CTC listed) / **R** ⚠ ("finalised") |
|
||||
| Call for Proposals on media authenticity and provenance indication with MPEG Systems technologies | CfP issued at MPEG 155. Proposals due **13 Jan 2027**, evaluated at MPEG 157 | — | https://www.mpeg.org/155th-meeting-of-mpeg/ | P✓ |
|
||||
| — its part number, ISO/IEC 23888-6 (MPEG-AI Part 6), and later stages | the meeting page names the CfP but not the part number. CD at MPEG 158; FDIS early 2028 | — | [`mpeg-ai-part6-mapping.md`](mpeg-ai-part6-mapping.md); 2030 §6.5 | tree |
|
||||
| Beyond-VVC Joint Call for Proposals (JVET, SC 29/WG 5 with ITU-T) | issued at JVET's 43rd meeting. Registration by early September 2026; main package due **end of October 2026**, a supplemental package mid-December; evaluation at the JVET meeting in Brisbane, **January 2027**; first test model in the second half of 2027; first version targeted **end of 2029**. **Welcomes technology based on neural networks and AI** alongside conventional tools | — | https://www.mpeg.org/155th-meeting-of-mpeg/ | P✓ |
|
||||
| Gaussian splat coding — Exploration 45 | exploration. The page lists documents from **MPEG 150** (draft use cases and requirements, 2025-04-26) through MPEG 152 and 154 (common test conditions) to **MPEG 155** (use cases, 2026-07-18; a **call for content**, 2026-07-31). It names **no working group** | MPEG 150 onward | https://www.mpeg.org/standards/Explorations/45/ (read 2026-09-27) | P✓ |
|
||||
| — its tracks and timeline | intra (I-3DGS) and inter (A-3DGS) tracks; 83 inputs at MPEG 155; codec selection 2027; standard targeted Oct 2029. **None of this is on the part page** | MPEG 155 | — | **R** ⚠ |
|
||||
| V-PCC (ISO/IEC 23090-5) Gaussian-splat amendment | CDAM | MPEG 153 | https://www.mpeg.org/153rd-meeting-of-mpeg/ | P✓ |
|
||||
| G-PCC (ISO/IEC 23090-9) Gaussian-splat Amd 1 | **CDAM** | MPEG 154 | https://www.mpeg.org/154th-meeting-of-mpeg/ | P✓ |
|
||||
| Enhanced G-PCC, ISO/IEC 23090-38 | FDIS | MPEG 155 | https://www.mpeg.org/155th-meeting-of-mpeg/ | P✓ |
|
||||
|
||||
*Bearing on the registry.* The last five rows bear directly on three registered
|
||||
capabilities. `wai.pc.vpcc` is ISO/IEC 23090-5 and `wai.pc.gpcc` is 23090-9,
|
||||
both with a patent-pool licence class. `wai.splat.sog` is a candidate registered
|
||||
"ahead of an MPEG freeze". None of the three has a decoder inside `wai-rs`;
|
||||
their decoders are sink-supplied.
|
||||
[`media-codec-registry.md`](media-codec-registry.md) records a demonstrated
|
||||
end-to-end decode of `wai.pc.gpcc` through the MPEG reference tool built from
|
||||
source, and an encode plus structural decode of `wai.pc.vpcc`, both through
|
||||
`tools/wai_carriage_demo.sh` (not re-run for this record). §6 maps the rows, and
|
||||
G11 lists the questions they raise.
|
||||
|
||||
### 2.2 JPEG (ISO/IEC JTC 1/SC 29/WG 1)
|
||||
|
||||
| item | stage | date | source | grade |
|
||||
|---|---|---|---|---|
|
||||
| JPEG AI, ISO/IEC 6048 | Part 1 published 2025; Parts 2 and 5 published 2026; Part 3 (reference software) and Part 4 (conformance) at stage 60.00, under publication | — | — | **R** ⚠ |
|
||||
| — its determinism design, as published | integer entropy pipeline (8-bit multipliers, 32-bit registers, overflow-free) is bit-exact; the normative conformance point is the **entropy-decoder output**, and reconstruction may diverge | — | overview paper arXiv 2510.13867 (its abstract does not state this; body not re-read for this record; [`syfai-input.md`](syfai-input.md) §5 quotes the passages from its body) | S (consistent with [`determinism-tiers.md`](../extensions/determinism-tiers.md) §1) |
|
||||
| JPEG AI **core experiments on energy-consumption analysis and on bit-exact reconstruction** | reviewed at the 111th meeting (virtual, 13–17 Apr 2026). The release reports results across CPU, GPU and FPGA, with FPGA the most energy-efficient and GPU the lowest-latency. **New core experiments on energy consumption, bit-exact reconstruction** and evaluation are to continue | press release published 8 Jun 2026 | https://jpeg.org/items/20260608_press.html | P✓ |
|
||||
| JPEG Trust | Part 1 (*Core Foundation*) 2nd edition published. Part 2 (*Trust profiles and reports*) "has now entered the DIS stage"; Part 3 (*Media asset watermarking*) "released for DIS balloting"; the Part 4 (*Reference Software*) DIS "also released for balloting". "A first JPEG Trust Summit is planned on 12-13 November 2026 in London." The standard number the sweep gave, ISO/IEC 21617, is not on the release | press release published 8 Jun 2026 | https://jpeg.org/items/20260608_press.html (read 2026-09-27) | P✓ (stages, summit) / **R** ⚠ (number) |
|
||||
|
||||
*Bearing.* The bit-exact reconstruction work is in the base standard's own core
|
||||
experiments. If it produces a reconstruction conformance point, the published
|
||||
Part 1 design in the row above stops describing later editions (§6, F11).
|
||||
|
||||
On `main`, WAI's texts do not agree on the capability. SPEC §5 registers
|
||||
`wai.image.jpegai` at `decode-equivalence`, "reproducible on every machine";
|
||||
the code classes it `StandardDefined`, which SPEC §7 defines as reconstruction
|
||||
fixed by the standard; and SPEC §7 admits no float operation into `replicate`
|
||||
(G9). In the tree at link 10 (link 02, PR #103), SPEC §5 registers the
|
||||
capability at `entropy-consistency`, its registry class is `NeuralEntropyExact`,
|
||||
and it does not back `replicate`. SPEC §7 (*`wai.image.jpegai` and the standard
|
||||
classes*) records that its conformance part, ISO/IEC 6048-4, has not been read,
|
||||
and that the capability is a candidate for `StandardTolerance` if Part 4 sets a
|
||||
criterion on the reconstruction.
|
||||
|
||||
### 2.3 IETF (datatracker, read 2026-09-26; revisions, states and dates re-read 2026-09-27)
|
||||
|
||||
| item | stage | date | grade |
|
||||
|---|---|---|---|
|
||||
| `draft-ietf-moq-transport-21` | WG Document. Milestones: transport and C4M (Common Access Tokens) to IESG **Dec 2026**. LOC, the streaming format (the milestone names it the WARP Streaming Format; `draft-ietf-moq-warp` is in state *Replaced*, by `draft-ietf-moq-msf`) and Privacy Pass authentication to IESG **Mar 2027** | 2026-09-08 | P✓ |
|
||||
| `draft-ietf-moq-loc-04`, `-msf-01`, `-cmsf-01`, `-c4m-01`, `-secure-objects-01`, `-privacy-pass-auth-03` | WG drafts. **None of these, nor `-transport-21`, contains the terms energy, joule, watt or power consumption** (text search) | — | P✓ |
|
||||
| WebTransport over HTTP/3, `draft-ietf-webtrans-http3-16` | In WG Last Call | 2026-07-06 | P✓ |
|
||||
| RTP over QUIC, `draft-ietf-avtcore-rtp-over-quic-14` | **expired**. Revision -14 is dated 20 March 2025 in its header (datatracker: published 2025-03-21) and expired in September 2025 (header: 21 September; datatracker: 2025-09-22). No later revision | 2025-03-20 | P✓ |
|
||||
| SCONE protocol, `draft-ietf-scone-protocol-09` | submitted to the IESG; IESG state *IESG Evaluation*. Bearing on WAI not assessed | 2026-09-25 | P✓ |
|
||||
| GREEN WG: `framework-02` (2026-07-06, per its header), `power-and-energy-yang-04` (2026-09-09), `terminology-02` (2026-06-30), `use-cases-02` (2026-08-07) | WG drafts. Device and sub-component power telemetry. `-04` keeps the graded `data-source-accuracy` hierarchy (measured bronze/silver/gold/red/ones; estimated static/historic/learned). No signature over energy values in any of the four. `framework-02` does carry certification and calibrated-sensor material and asks for origin authentication of monitored data used for accounting (§6.1) | as listed | P✓ |
|
||||
| `draft-ietf-avtcore-rtcp-green-metadata-17` | TSRR/TSRN hint messages; IESG state **RFC Ed Queue** | 2026-09-09 | P✓ |
|
||||
|
||||
### 2.4 Other bodies
|
||||
|
||||
| item | stage | date | source | grade |
|
||||
|---|---|---|---|---|
|
||||
| AOMedia AV2: specification and `libavm` reference software | specification and reference code v1.0.0 dated 28 May 2026 (landing page). Release announced 9 Jun 2026, "developed under AOMedia's royalty-free patent policy"; `libavm` remains the reference software; commercial conformance-stream packages from two vendors "are already available" (press release) | 2026-05-28 / 2026-06-09 | https://av2.aomedia.org/ ; http://aomedia.org/press%20releases/Alliance-for-Open-Media-Releases-AV2-Codec/ | P✓ |
|
||||
| — its content | neither page mentions neural tools, energy or determinism. **The specification text was not read for this record**, so absence from the specification is not established | — | both pages | ⚠ |
|
||||
| — shipping hardware decode | **unknown**: the hardware area was not reached | — | — | ⚠ |
|
||||
| 3GPP SA4 WI 1080050, *Study on Media Energy Consumption Exposure and Evaluation Framework phase 2* | study; outcomes documented in **TR 26.942** | — | named on the workshop page below; the 3GPP portal was not read for this record ([`energy-exposure-mapping.md`](energy-exposure-mapping.md) §3 reads TR 26.942 V20.0.0) | **S** ⚠ |
|
||||
| — joint workshop of 3GPP SA4, Greening of Streaming and 5G-MAG | quoted findings: "the lack of granular, shareable energy data across devices, networks and services"; "there is no standard that will give you the measurement… in an end-to-end use case". The page mentions no signing or attestation layer | 19 Mar 2026 | https://www.5g-mag.com/workshops/media-energy-consumption/ | P✓ |
|
||||
| DVB BlueBook S100, *Study Mission on Energy-Aware service Delivery and Consumption* | study-mission report | Nov 2023 | https://dvb.org/wp-content/uploads/2023/11/S100_Study-Mission-on-Energy-Aware-Delivery-and-Consumption_Nov-2023.pdf | P✓ |
|
||||
| — its content | four use cases: common measurement (§7.1), exposure (§7.2), energy as a source- or quality-selection criterion (§7.3), modernisation (§7.4). Existing coverage stated as "None in DVB" for §7.1 and §7.3. For §7.2, existing DVB APIs are said to be not designed to report energy. Draft requirements include a common reporting framework, open APIs per component, energy efficiency of a media instance in DVB-I, and DASH representation selection by energy efficiency | — | same | P✓ |
|
||||
| DVB CM-EE subgroup | announced as a new Commercial Module subgroup on 17 Nov 2023 | 2023-11-17 | https://dvb.org/news/dvb-to-consider-energy-efficiency-when-creating-commercial-requirements-for-specs/ | P✓ |
|
||||
| — the DVB chair's *TV 2030* article | says "in 2024 we launched a working group on energy efficiency" and "so far we have struggled to bring this group and topic to life", and invites participation. **The article does not name CM-EE.** That this working group is CM-EE is the record's inference | 2025-09-10 | https://dvb.org/news/tv-2030-preparing-for-a-future-thats-already-in-sight/ (re-read 2026-09-27) | P✓ (quotes; a year old) / ⚠ (identification with CM-EE) |
|
||||
| DVB-I package | approved, adds a security framework (TS 103 770); no energy items | 30 Jun 2026 | — | **R** ⚠ |
|
||||
| IEEE 1857.11-2024, neural-network-based image coding (AVS_1857_WG) | published | 2024-12-20 | — | **R** ⚠ |
|
||||
| — its call for proposals | required **reproducibility cross-checks**, not bit-exact decode | — | — | **R** ⚠ |
|
||||
| IEEE P1857.11.1 (large generative models) | PAR approved | 2025-06-13 | — | **R** ⚠ |
|
||||
|
||||
[`syfai-input.md`](syfai-input.md) §5 and its sources read the IEEE 1857.11 call
|
||||
for proposals and the standard's record page; this record's two IEEE 1857.11
|
||||
rows keep the sweep's own grade.
|
||||
|
||||
---
|
||||
|
||||
## 3. Regional findings
|
||||
|
||||
### 3.1 China
|
||||
|
||||
| item | status | grade |
|
||||
|---|---|---|
|
||||
| AVS4 | announced Mar 2026; combines conventional and AI coding tools; **no public draft** | **R** ⚠ |
|
||||
| AVS3 video | group standard since 2020; national standard **still awaiting issuance** | **R** ⚠ |
|
||||
| Audio Vivid, GB/T 46271-2025 | in force 2026-03-01; includes a neural component; also ITU-R BS.2493 | **R** ⚠ |
|
||||
| HDR Vivid, GB/T 46269.1/.2-2025 | national standard | **R** ⚠ |
|
||||
| AVS point cloud, T/AI 128.2-2024 | group standard | **R** ⚠ |
|
||||
| Perceptually lossless coding, T/AI 129.1-2024 | group standard | **R** ⚠ |
|
||||
| Smart Media Transport, GB/T 33475.6-2024 | national standard | **R** ⚠ |
|
||||
| UWA〔2026〕042, *AI Ultra High Definition Media Encapsulation and Transmission Protocol* (《AI超高清媒体封装与传输协议》) | project approved 2026-06-18; recruiting drafting participants, so **pre-draft**; the proposer the project page names is an operator's media subsidiary. https://theuwa.com/new-detail/899 | P✓ (project page) |
|
||||
| Low-latency live delivery | converged on WebRTC. **No MoQ work found at a Chinese standards body** | **R** ⚠ |
|
||||
| Media energy | **no media-energy standard found**. Device-level only: GB 24850-2020, GB 21520-2023 | **R** ⚠ |
|
||||
|
||||
*Relation to the 2030 landscape.* §7.3 of that document rejected an unnamed
|
||||
Chinese intelligent-media transport standard: no ingestible surface, and an
|
||||
opaque WAI envelope rides such a bearer unchanged. **⚠ Whether that item was
|
||||
GB/T 33475.6 was not checked.** If it was, the same reasoning applies here.
|
||||
UWA 042 is a different item, an AI-specific encapsulation still at pre-draft,
|
||||
and is a watch item ([`syfai-input.md`](syfai-input.md) §7).
|
||||
|
||||
*Unreached:* which institutions are behind the Chinese integer learned-codec
|
||||
work.
|
||||
|
||||
### 3.2 Japan and Korea
|
||||
|
||||
**Japan.**
|
||||
|
||||
| item | status | grade |
|
||||
|---|---|---|
|
||||
| ARIB STD-B80 (ISDB-T3) | v1.1 (2026-04-16) | **R** ⚠ |
|
||||
| ARIB STD-B60 (MMT) | v2.0 (2025-03-25), the main deployed MMT profile | **R** ⚠ |
|
||||
| NHK STRL Open House 2026 | C2PA provenance across capture, edit and distribution, with hardened in-browser verification. Cloud-unified broadcast/online production on ST 2110 + CMAF low-latency (**no MoQ or QUIC**). Live 8K 360-degree coding on VVC | **R** ⚠ |
|
||||
| a learned image coding line with quantised arithmetic since 2020 | arXiv 2007.04684 (*fixed point weight quantization*), 2111.09348 (*quantized weights and activations*). arXiv metadata carries no affiliation, so the institution the sweep attached to this line is **not verified** | P✓ (titles) / ⚠ (affiliation) |
|
||||
| arXiv 2608.00483 (2026-08; GLOBECOM 2026), by authors overlapping the line above | mixed-precision cross-platform neural video coding. P-frames run in FP16. I-frames and periodic feature-adapter resets run in IEEE-754 FP32, keeping the arithmetic entropy coder synchronised across GPUs. Evaluated across 12 GPUs of four architecture generations. Claims "deterministic synchronization at critical boundaries" and cross-generation decodability "for recent architectures". **The abstract makes no bit-exactness or identical-reconstruction claim** | P✓ (abstract) |
|
||||
| METI Top Runner, TV target FY2026 | per-device annual kWh | **S** ⚠ |
|
||||
|
||||
The 2030 landscape (§7.3) rejected a clause-level claim about a Japanese
|
||||
MMT-over-TLV standard as a committee proposal rather than adopted text. Nothing
|
||||
in this sweep reopens it.
|
||||
|
||||
**Korea.**
|
||||
|
||||
| item | status | grade |
|
||||
|---|---|---|
|
||||
| Low-complexity feature transform adopted into MPEG FCM (ISO/IEC 23888-4) | MPEG 155. No secondary source was kept, and the MPEG 155 report cited in §2.1 does not mention it | **R** ⚠ |
|
||||
| ISO/IEC 23003-8 biomedical and general waveform signal coding | FDIS at MPEG 155 (https://www.mpeg.org/155th-meeting-of-mpeg/). Described by the sweep as ETRI technology | P✓ (stage) / **R** ⚠ (attribution) |
|
||||
| TV energy efficiency | per-device grades | **R** ⚠ |
|
||||
|
||||
*Unreached:* Korean ATSC 3.0 developments in 2025–26.
|
||||
|
||||
### 3.3 Europe and Brazil
|
||||
|
||||
**Brazil — TV 3.0 / DTV+.** Established by **Decree 12.595/2025** (27 Aug 2025).
|
||||
The system uses:
|
||||
|
||||
- the ATSC 3.0 physical layer;
|
||||
- ROUTE/DASH transport, with HLS optional for internet-delivered alternative content;
|
||||
- **VVC** as the main video codec with **LCEVC** enhancement;
|
||||
- **MPEG-H Audio**;
|
||||
- the ABNT NBR 25601–25609 series.
|
||||
|
||||
Anatel Ato 2705 (24 Feb 2026) sets provisional conformity requirements.
|
||||
Broadcasts have been on air since June 2026 in Rio de Janeiro, São Paulo and
|
||||
Brasília, with 15 capitals targeted by 2030. **R** ⚠ for all of these. The 2030
|
||||
landscape (§7.2) analyses the part of this system with a direct WAI bearing, the
|
||||
lawful-basis declaration and privacy receipt in ABNT NBR 25608:2025 (§8.8 there).
|
||||
⚠ That clause number comes from public companions: the 2030 landscape (§9
|
||||
item 5) records that the paywalled standard itself was not read, and asks that
|
||||
it be bought before the clause number is quoted outward. On `main` no extension
|
||||
or code carries that declaration. In the tree at link 10 (PR #112), a
|
||||
`lawful-basis` claim class ([`jwp-receipts.md`](../extensions/jwp-receipts.md)
|
||||
§2.4.6) records a declaration of purposes, each with its own basis, and decisions
|
||||
taken against it, in the shape that section attributes to the standard's public
|
||||
operational guidelines (§6). The bearing on the registry: on `main` all three
|
||||
mandated media codecs are unregistered in WAI; at link 04 (PR #105), SPEC §5
|
||||
registers `wai.video.vvc`, `wai.video.lcevc` (companion-only) and
|
||||
`wai.audio.mpegh` (G5).
|
||||
|
||||
**Europe.**
|
||||
|
||||
| item | status | grade |
|
||||
|---|---|---|
|
||||
| Germany DVB-I launch | planned for 2026; **sources give conflicting dates** | **C** ⚠ |
|
||||
| Spain national multiplex RGE2 | moves to DVB-T2 + HEVC UHD from 5 Nov 2026 | **R** ⚠ |
|
||||
| Fraunhofer FOKUS, *Green Streaming* whitepaper (Oct 2024) | kWh/GB is an inadequate metric; end devices account for 70–80% of chain energy | **R** ⚠ |
|
||||
| FAMIUM Green hardware-in-the-loop energy rig | *Computers* 15(10):653, 25 Sep 2026. **Body not read** | P (citation; not re-read) / ⚠ content |
|
||||
| France RGESN | 17 May 2024; 78 criteria; mandatory for public bodies | **R** ⚠ |
|
||||
| France ARCEP decision 2025-2428 | declarative; **excludes streaming platforms** | **R** ⚠ |
|
||||
| Blue Angel DE-UZ 215 (resource- and energy-efficient software) | ecolabel with a measure-and-publish requirement | **R** ⚠ |
|
||||
|
||||
*Bearing.* If end devices account for most of the chain's energy (a single
|
||||
whitepaper, unverified), the sink-side decode is where a per-object figure
|
||||
carries the most weight. That is the scope
|
||||
[`energy-measurement.md`](../extensions/energy-measurement.md) already meters.
|
||||
The claim is only as strong as that one source.
|
||||
|
||||
*Unreached:* Nordic and Dutch sources.
|
||||
|
||||
### 3.4 Rest of world
|
||||
|
||||
| item | status | grade |
|
||||
|---|---|---|
|
||||
| India, Direct-to-Mobile (D2M) on ATSC 3.0 in 470–582 MHz | Phase I authorised 2026 | **S** ⚠ |
|
||||
| India, BIS IS 18112:2025 receiver standard | mandatory from 26 Jan 2026 | **R** ⚠ |
|
||||
| Russia | no domestic codec programme found. The largest platform defaults to H.264, uses AV1 only for popular creators, and delivers over HTTP/3 | **R** ⚠ |
|
||||
| Turkey, Indonesia | on DVB-T2 (Indonesia's DVB-T2 early-warning mechanism: 2030 landscape §7.1) | **R** ⚠ |
|
||||
|
||||
---
|
||||
|
||||
## 4. Industry deployment reality (2026)
|
||||
|
||||
- **Codec mix.** An encoder vendor's survey (March 2026; vendor-run; **sample
|
||||
size not stated**; not read directly, and not named under this record's
|
||||
naming rule) reports production use of H.264 84%, HEVC 65%,
|
||||
AV1 17%, VP9 15% and VVC 4%. **S** ⚠. The sweep's reading is that VVC ships
|
||||
at scale only where a national mandate requires it (Brazil, §3.3). **R** ⚠.
|
||||
That is an inference from the sources found, not a census.
|
||||
- **Media over QUIC.** The CDN relay service found by the sweep publishes
|
||||
developer documentation (last updated 31 Jul 2026; read, so cited by URL
|
||||
under the naming rule). It lists relays for
|
||||
**draft-14** and **draft-16** of MoQ Transport and a **draft-18** test relay,
|
||||
and describes the service as a **beta** ("free to use during the beta
|
||||
period"). P✓ (https://developers.cloudflare.com/moq/). The current transport
|
||||
draft is **-21** (§2.3). There are niche production deployments and many
|
||||
trade-show demonstrations (**R** ⚠). **No top consumer platform was found
|
||||
delivering to viewers over MoQ** (**R** ⚠). WebTransport is reportedly
|
||||
baseline across the major browsers (**S** ⚠). *Bearing:* any interop claim for
|
||||
[`moq-streaming-format.md`](../extensions/moq-streaming-format.md) must name
|
||||
the transport draft it was tested against. The relay found serves -14 and -16
|
||||
and is testing -18, while the WG text is at -21.
|
||||
- **Neural codecs.** **No neural codec was found in production at large
|
||||
platforms.** **R** ⚠. The hardware and NPU area, where such a deployment
|
||||
would most likely show up, was not reached (§1).
|
||||
- **Cross-platform learned video without bit-exact arithmetic.** arXiv
|
||||
2606.28027 (MLVC, ECCV 2026) *explicitly transmits scale parameters through
|
||||
the hyperprior*. Its abstract says this "guarantees entropy coding consistency
|
||||
across devices without requiring bit-exact arithmetic", at a bitrate cost
|
||||
(**magnitude not recorded**). It reports real-time encode and decode on
|
||||
commodity NPUs from three vendors. P✓ (abstract). The abstract claims
|
||||
entropy-coding consistency, not identical reconstructions. **That the
|
||||
reconstructions differ across platforms is inferred, not read** ⚠. This is a
|
||||
published instance of the explicit-parameter-transmission approach. On
|
||||
`main`, [`determinism-tiers.md`](../extensions/determinism-tiers.md) §1–§2
|
||||
names that approach only generically. In the tree at link 10 (link 02,
|
||||
PR #103), §1 names arXiv 2606.28027 as its instance and says the side
|
||||
information costs bitrate, and §2 says why the route stops at
|
||||
`entropy-consistency` (G7).
|
||||
- **Energy disclosure.** **No platform was found publishing per-stream or
|
||||
per-view energy.** **R** ⚠.
|
||||
|
||||
---
|
||||
|
||||
## 5. Energy verifiability and regulation
|
||||
|
||||
### 5.1 Who can check an energy figure today
|
||||
|
||||
The question asked here is narrower than "who measures energy". It is
|
||||
whether a party that was not present can check a measured figure, and whether
|
||||
that figure is bound to a specific media outcome.
|
||||
|
||||
**No standard or product was found that binds measured energy to a specific
|
||||
media outcome in a way a third party can verify.** The closest precedents each
|
||||
provide part of that:
|
||||
|
||||
| precedent | what is signed or checked | what is not | grade |
|
||||
|---|---|---|---|
|
||||
| **OCMF** signed meter values (EV charging, under German calibration law) | a component of the metering installation signs session start and stop readings, and a third party can check them independently. [`energy-exposure-mapping.md`](energy-exposure-mapping.md) L2 reads the OCMF text: the signing component need not be the meter itself | not media; no binding to a reproducible output | **R** ⚠ / tree (which component signs: energy-exposure-mapping L2) |
|
||||
| An ML-benchmark power methodology | wall-plug instrument plus peer review of the method | **unsigned** | **R** ⚠ |
|
||||
| RAPL under confidential computing | — | the counter is **deliberately filtered** where an enclave may be running. The technical guidance published with advisory INTEL-SA-00389 (CVE-2020-8694, CVE-2020-8695; https://www.intel.com/content/www/us/en/developer/articles/technical/software-security-guidance/advisory-guidance/running-average-power-limit-energy-reporting.html) gives the added noise as typically under 2 % standard deviation in readings at 1 s intervals and 5–15 % at 100 ms, with larger variance at thermal design power; filtered energy is reported when SGX is enabled or system software has enabled filtering, and TDX uses the filtered energy information. The advisory bulletin itself carries none of this, which is why the first reading found neither. energy-measurement §2.4 cites the guidance from link 11 | **P✓** (the guidance, read 2026-09-30) |
|
||||
| Confidential-computing GPU attestation; SPDM / Caliptra | firmware and configuration | **not energy** | **R** ⚠ |
|
||||
| Cloud per-customer carbon reporting | — | explicitly **not assured** (one hyperscaler's methodology page, updated 2026-09-24) | **R** ⚠ |
|
||||
| SCITT, RFC 9943 (Standards Track, June 2026) | signed statements plus transparency receipts | **defines no energy claims** (the RFC text contains neither "energy" nor "joule") | P✓; profiled in [`transparency.md`](../extensions/transparency.md) |
|
||||
| *Emission Impossible*, arXiv 2506.16347 | zero-knowledge proofs over carbon-emissions claims across a supply chain | a **proposed** methodology ("This paper proposes…"), not a deployed system | P✓ (abstract); already cited in `jcp/jcp-rs/crates/jcp-efficiency/MAPPING.md` §4 |
|
||||
| 3GPP TR 26.942 / SA4 WI 1080050 | an exposure and evaluation framework | the workshop page mentions **no signing or attestation layer**; [`energy-exposure-mapping.md`](energy-exposure-mapping.md) §3 finds no *signature*, *attest*, *integrity* or *tamper* in the V20.0.0 text | **S** ⚠ (§2.4) / tree |
|
||||
| DVB S100 draft requirements (2023) | a common reporting framework; per-component APIs; per-media-instance efficiency; energy-based DASH representation selection | S100 stated existing DVB coverage as "None in DVB" for two of its four use cases. **No CM-EE output was found by the sweep**, and the DVB chair wrote in Sep 2025 that a working group on energy efficiency launched in 2024 had "struggled to bring this group and topic to life" (the article does not name CM-EE) | P✓ (§2.4) |
|
||||
|
||||
**In-tree precedent, outside WAI.** The Sandbox standard's `sandbox-tee` energy
|
||||
module (`sandbox/sandbox-rs/crates/sandbox-tee/src/energy.rs`) binds an energy
|
||||
statement into a TEE attestation report and **derives** a trust level from the
|
||||
evidence:
|
||||
|
||||
- `Unattested`: a signed figure over a counter of unproven provenance.
|
||||
- `IntegrityAttested`: a verified report over a counter the platform reports
|
||||
about itself.
|
||||
- `PhysicallyRooted`: a dedicated sensor signed at the source.
|
||||
|
||||
Its `CounterClass` keeps `HwShunt` for a dedicated shunt and gives RAPL, NVML,
|
||||
IOReport and Redfish classes of their own. Only `HwShunt` is
|
||||
`is_independently_rooted()`.
|
||||
|
||||
JCP's `efficiency/1` carries an optional `energy_attestation`, a commitment to
|
||||
that report (`jcp/jcp-rs/crates/jcp-efficiency/src/training.rs`). The rationale
|
||||
is in that crate's `MAPPING.md` §4, which already cites PLATYPUS and arXiv
|
||||
2506.16347. For TEE-attested readings this is the family's own answer to *who
|
||||
can check the figure*. WAI's receipts do not reference it, on `main` or at
|
||||
link 10 (§7, G4, G6).
|
||||
|
||||
**The family's receipt ladder draws no such line.** JCR-1 is the signed-receipt
|
||||
format the family shares. JCP's SPEC says its receipts share the JCR-1 envelope,
|
||||
and Sandbox §6.4 calls JCR-1 "the signed-receipt format shared across the …
|
||||
family". Its normative §2.3
|
||||
(`jouleclaw/jouleclaw-rs/crates/jouleclaw-jcr1/docs/JCR-1-spec.md`:69–72) puts a
|
||||
RAPL MSR, the NVML cumulative-energy counter and a board shunt in
|
||||
`HwShunt` alike. The other standards that list interfaces under `HwShunt` do
|
||||
the same with RAPL, NVML, ROCm SMI, Level Zero, SMC or a shunt: JCP §5.1,
|
||||
Sandbox §6.3, Proof, Mesh. So at the receipt level, on-die counters and
|
||||
dedicated shunts share one class across the family. Only `sandbox-tee`'s
|
||||
counter classes separate them, and, in the tree at link 10, WAI's own classes
|
||||
(below).
|
||||
|
||||
Where the family puts **IOReport** is not uniform. §5.3 records each standard's
|
||||
actual class for an IOReport reading, with file references.
|
||||
|
||||
**In WAI.** [`oer-receipts.md`](../extensions/oer-receipts.md) gives every
|
||||
figure an **evidential grade**: `projected`, `simulated`, `derived` or
|
||||
`measured`. It also makes receipts canonical, content-addressed, hash-chained
|
||||
and signed. Its header says "a third party can check every figure without
|
||||
trusting the producer". The checks in its §5–§6 cover the following:
|
||||
|
||||
- canonical form and identity;
|
||||
- the chain and the signature under the key the vector file publishes;
|
||||
- that each breakdown sums;
|
||||
- the Landauer-floor arithmetic;
|
||||
- the bundle roots and record hashes;
|
||||
- that the receipts sum to a ledger's `meter_joules`.
|
||||
|
||||
These establish the figures' internal consistency and who signed them. They do
|
||||
not establish that a figure matches a physical measurement:
|
||||
|
||||
- **The ledger value.** `meter_joules` is a plain value in the vector file, and
|
||||
the extension does not say who attests it.
|
||||
- **The `measured` grade.** It covers "an instrument, a datasheet, or a
|
||||
published measurement" alike.
|
||||
- **No acquisition-class member.** §3's member list is closed and has no member
|
||||
for the class. §2 says which grade each class supports and forbids a producer
|
||||
to grade `measured` a figure whose class is `Estimator`, and says that no
|
||||
reason code checks the rule, because a receipt carries no class. The
|
||||
`provenance.device` and `energy.method` members are free text; the corpus
|
||||
values are `hardware`, `emulation` and `simulation`, and `hardware_meter` and
|
||||
`model_estimated`.
|
||||
|
||||
In the tree at link 10, oer-receipts §2 (link 08, PR #109) extends that
|
||||
correspondence to the refined hardware classes and says that neither
|
||||
`hardware` nor `hardware_meter` names a class, and
|
||||
[`energy-measurement.md`](../extensions/energy-measurement.md) §1 places OER/2
|
||||
receipts outside that extension's rules. An `energy` claim
|
||||
([`jwp-receipts.md`](../extensions/jwp-receipts.md) §2.4.5; link 10, PR #112)
|
||||
carries both axes for one object's figure — the class with its evidence, and a
|
||||
grade it refuses when the grade ranks above what the class supports — and may
|
||||
name the OER receipt that records the figure. A verifier holding both checks
|
||||
that they agree on grade and figure, which makes the producer rule checkable
|
||||
against the class the claim's signer states. The OER receipt itself still
|
||||
carries no class.
|
||||
|
||||
So an OER receipt does not say whether a `measured` figure came from a counter
|
||||
the platform relays or from an independently rooted instrument. It also does not
|
||||
say who, other than the signer, stands behind the magnitude (§7, G6). OER is
|
||||
WAI's own extension. It does not bear on this section's conclusion about the
|
||||
field outside WAI (F1).
|
||||
|
||||
*Bearing on WAI.* [`energy-measurement.md`](../extensions/energy-measurement.md)
|
||||
§5 states that a figure "is not verifiable by re-execution" and that, where
|
||||
reconstruction is byte-exact, energy is "the single attested variable". The
|
||||
OCMF precedent (**R** ⚠) shows a property that would make that variable
|
||||
checkable by a third party: a signature made inside the metering installation,
|
||||
not by the producer. The filtered-RAPL row (**P✓**) shows the
|
||||
opposite case, an on-chip counter that may carry deliberate noise. On `main`,
|
||||
WAI's `HwShunt` puts both in one class, as the family's receipt-level `HwShunt`
|
||||
does, and takes in IOReport. In the tree at link 10 (link 08, PR #109),
|
||||
energy-measurement §2 splits WAI's hardware class into `CalibratedInstrument`
|
||||
and `OnChipCounter`. Each carries a declared uncertainty (§2.3); the counter
|
||||
records its filtering state (§2.4) and the instrument a calibration reference
|
||||
(§2.5). `HwShunt` becomes a legacy class, accepted and never emitted (§2.6), and
|
||||
IOReport is `OnChipCounter` (§2.9). The class is the producer's signed
|
||||
declaration: a calibration reference makes the calibration claim checkable, not
|
||||
true (§2.5).
|
||||
|
||||
### 5.2 Regulation
|
||||
|
||||
**No jurisdiction was found to mandate per-stream, per-view or per-decode
|
||||
energy figures.** The rules found are device-scoped, facility-scoped,
|
||||
declarative, or allow estimates:
|
||||
|
||||
| jurisdiction / instrument | what it requires | grade |
|
||||
|---|---|---|
|
||||
| EU, Commission Delegated Regulation **C(2026) 3472** (dated Brussels, 21 Sep 2026; transmitted to the Council the same day as ST 13444/26) | a common Union **rating scheme and label for data centres**. From **15 Aug 2027** the European database generates an electronic label for data centres that have reported (Art. 3). The amendments to Regulation 2024/1364 remove "some indicators (such as the data traffic indicators)". Its **Annex III** adds to point 1 of Annex I of Regulation 2024/1364 a point (f), *Certification of the reported data*: a scheme "in the framework of which a qualified or accredited third party validates the monitoring, data validation and reporting procedures", with "a distinction between self-certification and third-party certification". It defines an item; the text read does not require certification. Annex I of 2024/1364 itself was not read. A **review by 31 Dec 2028** (Art. 6) is to consider "indicators to measure the performance of the ICT equipment energy use" and "(partial) certification or auditing of the reported data" | P✓ (read 2026-09-27: the act, https://energy.ec.europa.eu/document/download/b073ba9b-82a8-45cb-b8ba-78dad149e941_en?filename=C_2026_3472_1_EN_ACT_part1_v9.pdf ; its annexes, https://data.consilium.europa.eu/doc/document/ST-13444-2026-ADD-1/en/pdf) |
|
||||
| EU ESPR working plan 2025–2030 | ecodesign/label revision for displays, indicatively 2027; **no ICT-specific act** | **R** ⚠ |
|
||||
| EU CSRD | scope narrowed by the Omnibus I package | **R** ⚠ |
|
||||
| EU AI Act Art. 53 | general-purpose AI model providers must document known or **estimated** training energy (estimation allowed) | **R** ⚠ |
|
||||
| France, RGESN / ARCEP 2025-2428 | §3.3: mandatory for public bodies / declarative, excluding streaming platforms | **R** ⚠ |
|
||||
| Germany, Blue Angel DE-UZ 215 | voluntary ecolabel: measure and publish | **R** ⚠ |
|
||||
| US | DOE withdrew set-top box coverage. The voluntary set-top box agreement (Tier 4 from 2026) uses **third-party lab testing and audits** | **R** ⚠ |
|
||||
| Japan (METI Top Runner), Korea (TV grades), China (GB 24850-2020, GB 21520-2023) | per-device annual figures or grades | **S / R** ⚠ |
|
||||
|
||||
Two EU texts touch **who checks reported data**, both for data centres, not
|
||||
streams. C(2026) 3472's Annex III defines *Certification of the reported data*
|
||||
and requires self-certification to be distinguished from third-party
|
||||
certification; its 2028 review is chartered to consider (partial) certification
|
||||
or auditing. The only regime found that relies
|
||||
on **third-party testing** is the US voluntary set-top box agreement (**R** ⚠),
|
||||
and it tests devices, not streams.
|
||||
|
||||
### 5.3 How the monorepo's standards class an IOReport reading
|
||||
|
||||
IOReport is the SoC energy interface the reference meters read on Apple
|
||||
Silicon. This table records, per standard, the class an IOReport reading
|
||||
actually gets, from the normative text and from the reference code. It was made
|
||||
by reading every tracked line that mentions IOReport (`git grep -n -i ioreport`,
|
||||
outside this record), with its surrounding lines wherever a class name was near,
|
||||
together with each standard's `HwShunt` definition and the code that assigns the
|
||||
tag. None of the files cited in the rows other than WAI's differs between
|
||||
afc7ece and the tree at link 10, so their line numbers hold on `main` and in
|
||||
that tree. The WAI row gives both states. Nothing was run: where a class comes
|
||||
from code, it is the tag the code assigns, not an observed receipt.
|
||||
|
||||
| standard | normative text | reference code | class of an IOReport reading |
|
||||
|---|---|---|---|
|
||||
| **JCR-1** (the shared receipt envelope) | `jouleclaw/jouleclaw-rs/crates/jouleclaw-jcr1/docs/JCR-1-spec.md`:69–72 (§2.3): `ModelBased` — "vendor model estimate from freq/voltage/utilization (… IOReport, NVML `power.draw`)" | conformance vector 6 (`conformance/vectors.json`:148, `conformance/README.md`:31): `ModelBased` | **`ModelBased`** |
|
||||
| **JouleClaw** | `jouleclaw/SPEC.md`:73–74 (provenance table), :88 ("Model-based, not measured"): `ModelBased`. The same table puts ROCm SMI under `ModelBased` | `jouleclaw-energy`: `src/apple_ioreport.rs`:1, :11 and test `provenance_is_model_based` (:431–433); `conformance/vendors/apple-silicon-ioreport.json`:12; `MEASURED_ENERGY.md`:23 — `ModelBased`. Its load-average fallback, `src/apple.rs`:1, :89–91, is `Estimator`. **`jouleclaw-compliance`** `src/passport.rs`:132–133 (and test :701–704) maps `MeasurementMethod::IoReport` to **`Estimator`**, "until the FFI is wired". `jouleclaw-energy/conformance/README.md`:26 lists the ROCm SMI counter as `HwShunt`, against the SPEC | **`ModelBased`**; **`Estimator`** in `jouleclaw-compliance` |
|
||||
| **JCP** | `jcp/SPEC.md` §5.1 (:395–402): RAPL and NVML → `HwShunt`, time × power → `ModelBased`. **No IOReport rung** | `jcp-energy` `src/lib.rs`:15–17, :435–438, :481 (`Meter::IoReport => Provenance::ModelBased`): `ModelBased`. **`jcp-gateway`** `src/lib.rs`:429–436 seals its metered evaluation as `HwShunt` whenever the Sandbox `PowerMeter` reads hardware, which on Apple Silicon is IOReport (`jcp/README.md`:49) | **split**: `ModelBased` (`jcp-energy`), `HwShunt` (`jcp-gateway`) |
|
||||
| **Joule Code** | `joule-code/spec/receipt.md`:32 (normative sub-spec): `HwShunt` — "read from a hardware energy interface (RAPL, … SMC/IOReport, an external shunt)". Its signed form is a JCR-1 envelope (:38–40) | `joule-code-remediate` `src/lib.rs`:635–657 seals an attributed share of a Sandbox-meter reading as `ModelBased` (its source string: "attributed(ModelBased over IOReport HwShunt)"). `examples/l3-jouleclaw/src/bin/run.rs`:29–33 uses JouleClaw's `ModelBased` counter; that example's `README.md`:44–45 still says `Estimator` "until IOReport is wired" | **split**: `HwShunt` (normative text); `ModelBased` in both code paths |
|
||||
| **Sandbox** | `sandbox/SPEC.md`:217 (§6.3) lists RAPL/powercap, NVML, ROCm SMI, Level Zero and "an SMC/shunt otherwise" under `HwShunt`; **IOReport is not named**. `sandbox/README.md`:9: the reference meter reads IOReport, RAPL and NVML, "each tagged `HwShunt`" | `sandbox-meter` `src/probe/mod.rs`:8–10, :247 and `src/probe/ioreport.rs`:3: `HwShunt`; `sandbox-proof` `src/lib.rs`:1153 likewise. `sandbox-tee` `src/energy.rs` gives IOReport a `CounterClass` of its own (`IoReport`), not independently rooted | **`HwShunt`** (receipt); own class in `sandbox-tee` |
|
||||
| **Proof** | `proof/SPEC.md`:89 and `proof-schema` `src/lib.rs`:104: `HwShunt` = RAPL/NVML/SMC/shunt; **IOReport not named** | `proof-cascade` `src/meter.rs`:1–2, :23 calls itself "the `HwShunt` backend" and reads IOReport on Apple Silicon; `proof-cli` `src/main.rs`:90–93, :117 seals the reading with `Energy::measured`, which sets `HwShunt` (`proof-schema` `src/lib.rs`:153–158). The same module doc, `meter.rs`:9–10, says "a macOS reading is `ModelBased`" | **split**: `HwShunt` (code); `ModelBased` (its own module doc) |
|
||||
| **Sense** | `sense/SPEC.md`:166–170: the reference meter is Sandbox's `PowerMeter` ("real RAPL/IOReport") | `sense-schema` `src/lib.rs`:214–215: `HwShunt` — "A hardware energy reading (RAPL, IOReport, NVML)" | **`HwShunt`** |
|
||||
| **Joule UI** | `joule-ui/SPEC.md`:73: the reference meter is Sandbox's `PowerMeter` ("real RAPL/IOReport") | `joule-ui-serve` `Cargo.toml`:21–23 and `src/bin/joule_ui_serve.rs`:29–31: an Apple Silicon IOReport reading makes "the receipt's `HwShunt`" true | **`HwShunt`** |
|
||||
| **Mesh** | `mesh-schema` `src/lib.rs`:41–44: `HwShunt` = RAPL/NVML/SMC/shunt; IOReport not named | `mesh-meter` `Cargo.toml`:22 (`hw-meter-apple` enables JouleClaw's IOReport counter); `src/lib.rs`:222–233 passes JouleClaw's provenance through unchanged | **`ModelBased`** (inherited from JouleClaw) |
|
||||
| **JouleDesk** | — | `jouledesk-github` `Cargo.toml`:20–21, `src/lib.rs`:34–44: JouleClaw's `detect_meter` with the IOReport feature; records the meter's own provenance | **`ModelBased`** (inherited from JouleClaw) |
|
||||
| **EOC** | `eoc/vocab/v1.ttl`:34–35 carries the three names; IOReport is not mapped | `eoc-core` `src/joule_cost.rs`:7–12 has two classes, `Measured` and `Estimated`; `eoc-meter` `src/lib.rs`:56–61, :145 reads IOReport as `Measured` ("Measured-grade") | **`Measured`** (a two-class vocabulary) |
|
||||
| **WAI** | *`main`:* `extensions/energy-measurement.md`:34, :42–43: `HwShunt` = "RAPL, NVML, SMC, IOReport, or an external shunt"; "RAPL, NVML and IOReport are sensor values". *Link 10 (link 08, PR #109):* §2's table puts RAPL, NVML, SMC and IOReport under `OnChipCounter`, §2.9 classes IOReport `OnChipCounter`, and §2 requires a producer that carries a WAI figure into a three-name vocabulary to map both refined hardware classes to `HwShunt` | *`main`:* `bin/wai_video_meter.rs`:184–190 stamps an IOReport reading `HwShunt`, and the other receipt families that seal a `joules_micro` figure (quantum, ternary, world, film, haptic, score, splat4d, spatial audio, provenance) carry no class. *Link 10:* the reference meters (`wai_video_meter`, `wai_meter`, `wai_world_meter`, `wai_quantum_meter`, `wai_prov`) read IOReport through a non-privileged reader (`energy_burst`) and seal `OnChipCounter`, with filtering state `Unknown` and a declared uncertainty, or seal unmetered; every `wai-rs` and `wai-quantum` receipt family that seals a figure can carry the class (§6) | *`main`:* **`HwShunt`**, or unlabelled. *Link 10:* **`OnChipCounter`** (`HwShunt` at the three-name level), or unlabelled in a receipt of the legacy form |
|
||||
|
||||
Map, Access (`aac`), JouleContract, JouleHook and Smart Byte either carry the
|
||||
three names or none, and mention IOReport nowhere.
|
||||
|
||||
**What this shows.**
|
||||
|
||||
- **The monorepo is split on IOReport**, not uniform with WAI departing. The
|
||||
normative JCR-1 text and JouleClaw class it `ModelBased`, and Mesh and
|
||||
JouleDesk inherit that. Joule Code's normative receipt text, the Sandbox
|
||||
reference meter, Sense, Joule UI, `jcp-gateway`, Proof's code and WAI on
|
||||
`main` class it `HwShunt`. WAI at link 10 classes it `OnChipCounter`, a
|
||||
refinement that maps to `HwShunt` at the three-name level.
|
||||
`jouleclaw-compliance` classes it `Estimator`, EOC `Measured`, and
|
||||
`sandbox-tee` gives it a class of its own.
|
||||
- **Four standards are split internally:** JCP (`jcp-energy` against
|
||||
`jcp-gateway`), JouleClaw (`jouleclaw-energy` against `jouleclaw-compliance`,
|
||||
and ROCm SMI between its SPEC and its conformance table), Joule Code (its
|
||||
normative text against its code) and Proof (its code against its own module
|
||||
doc).
|
||||
- **JCR-1 envelopes can carry both.** Several of the `HwShunt` emitters seal
|
||||
JCR-1 envelopes (Sandbox SPEC §6, `sandbox/SPEC.md`:225; Proof's
|
||||
`proof-receipt`; `sense-receipt/src/jcr1.rs`; Joule Code `receipt.md` §3), so
|
||||
a JCR-1 envelope in this tree can carry an IOReport reading as `HwShunt`
|
||||
while JCR-1 §2.3 classes it `ModelBased`. This follows from the code paths;
|
||||
no such envelope was produced for this record.
|
||||
- **ROCm SMI is split the same way:** `ModelBased` in `jouleclaw/SPEC.md`:74,
|
||||
`HwShunt` in `sandbox/SPEC.md`:217 and in `jouleclaw-energy`'s conformance
|
||||
table.
|
||||
|
||||
**energy-measurement §2.9 at link 10** lists where other documents class
|
||||
IOReport differently: `jouleclaw/SPEC.md`, `jouleclaw-energy`, the JCR-1
|
||||
conformance files, `jouleclaw-compliance`, `jcp-energy` and the Joule Code
|
||||
example. It names `sandbox/README.md`, `proof/README.md`,
|
||||
`joule-code/spec/receipt.md` and `eoc-meter` as agreeing with it at the
|
||||
three-name level. It does not list the JCR-1 §2.3 text itself, `jcp-gateway`,
|
||||
Sense, Joule UI, the Sandbox meter code, Proof's code and module doc,
|
||||
`joule-code-remediate`, Mesh or JouleDesk. It changes none of those documents.
|
||||
|
||||
---
|
||||
|
||||
## 6. What the tree holds, and in what form
|
||||
|
||||
Checked by reading the named files and grepping `wai/` for code, in the tree at
|
||||
link 10 and on `main`. The **state** column says how far each item goes:
|
||||
|
||||
- **implemented**: code in `wai-rs` (or `wai-quantum`) carries it, in the file
|
||||
named;
|
||||
- **specified**: extension or SPEC text only, and no code carries it (grep
|
||||
stated);
|
||||
- **analysed**: a research document only, and the ingest is open.
|
||||
|
||||
Where the two trees differ, a row gives `main` first and then link 10, naming
|
||||
the link and pull request. Only *implemented* rows are carried by code.
|
||||
*Specified* and *analysed* rows are still work, and much of it is in the 2030
|
||||
ingest queue (2030 §8, item cited per row). This section maps where each finding
|
||||
lands. It is not a statement that the finding needs nothing further.
|
||||
|
||||
| finding | where in the tree | state | what the tree holds |
|
||||
|---|---|---|---|
|
||||
| JPEG AI's published conformance point is the entropy-decoder output; reconstruction may diverge | `SPEC.md` §5 (the `wai.image.jpegai` row) and §7; [`determinism-tiers.md`](../extensions/determinism-tiers.md) §1–§6; [`reconstruction-binding.md`](../extensions/reconstruction-binding.md) §1–§2; `wai-rs/src/codecs/mod.rs`; `wai-rs/src/container.rs`; `wai-rs/src/jpegai_decode.rs`; `wai-rs/src/reconstruction_binding.rs` | *`main`:* tier names specified; a coarser classification and the reconstruction hash implemented; SPEC §5 registers `wai.image.jpegai` at `decode-equivalence` while the code classes it `StandardDefined` (G9). *Link 10:* specified and implemented, with the tiers as a type in code | *`main`:* `capability_replicate_class` puts the integer neural capabilities in `NeuralIntegerExact` (`wai.neural.int_synth`, `wai.neural.int_hyper`, `wai.video.int_hyper`, `wai.audio.int_codec`, `wai.splat.int_codec`); `wai.image.jpegai` falls through a default arm to `StandardDefined`; the two tier names appear in no type. *Link 10:* determinism-tiers §1 states the base standard's conformance point and quotes an overview's account of its conformance part; §2 defines the tiers; §5 lists the capabilities at each (link 02, PR #103). `NeuralIntegerExact` also holds `wai.neural.int_mlicpp` (link 09, PR #111). `wai.image.jpegai` is the one `NeuralEntropyExact` capability, registered at `entropy-consistency` (link 02). `codecs::DeterminismTier` and `ReplicateClass::neural_tier` carry the tiers in code (link 02). Every registered capability is listed under one class with no default class: an unlisted one returns `None`, which the tests reject, and `StandardDefined` needs a recorded basis pinned in the tests (link 04, PR #105). Per `Prior`, a substring heuristic over the free-text `determinism` string still gives `IntegerExact`, `FloatReproducible` or `Unknown`; since link 02 the registry class is checked before it (`codecs::replicate_refusal`). ⚠ JPEG AI's own core experiments on bit-exact reconstruction are continuing (§2.2), so this row describes the published Part 1, not necessarily later editions (F11) |
|
||||
| the floor a neural decode must meet to back `intent = replicate` | `SPEC.md` §3, §7 (*Replicate-soundness*); `determinism-tiers.md` §2–§3; `reconstruction-binding.md` §1–§2; `wai-rs/src/codecs/mod.rs` (`replicate_refusal`); `wai-rs/src/container.rs` (`replicate_is_sound`, `determinism_class`); `wai-rs/src/neural/mod.rs` (`decode_envelope_strict`) | *`main`:* specified three ways (G10); the code follows SPEC §7's all-integer rule, applied to the prior's declared string. *Link 10:* specified once and implemented (link 02, PR #103) | *Link 10:* over a neural capability `replicate` promises `decode-equivalence`, which SPEC §7 admits only by an integer / fixed-point decode path with no float operation. `entropy-consistency` MUST NOT back `replicate` ("necessary … and not sufficient"), and neither may a reconstruction that one implementation pins through fixed-order float stages. determinism-tiers §2 defines `decode-equivalence` over "integer / fixed-point arithmetic end to end". `decode_envelope_strict` and `Wai::replicate_is_sound` check the capability's registry class first, then the pinned prior. The free function `container::replicate_is_sound` still checks only the prior |
|
||||
| explicit parameter transmission as a route to a reproducible entropy stage | `determinism-tiers.md` §1–§2 | specified | *`main`:* named with no instance or cost (G7). *Link 10:* §1 names arXiv 2606.28027 as the instance and says the side information costs bitrate, and §2 says why transmitting parameters stops at `entropy-consistency` (link 02, PR #103) |
|
||||
| decode followed by processing that changes the output (post-filters, operation-reduction hints) | `determinism-tiers.md` §2, *What a tier covers* | specified | a tier covers the whole path to the medium the sink emits; 2030 §8 item 1 landed as this text. At link 02 the section also accounts for every metadata set the Introduction of the 23001-11 4th-edition CD text names, read from that text. No code enforces the rule (grep over `codecs/`, `container.rs` and `neural/`); the post-filter matches in `wai-rs/src/jpegai_*.rs` are JPEG AI's own decoder stages |
|
||||
| Green Metadata (23001-11 / -19), the RTCP green-metadata draft, IETF GREEN | [`2030-transport-landscape.md`](2030-transport-landscape.md) §3.5, §5; [`energy-exposure-mapping.md`](energy-exposure-mapping.md) §1, §4–§5; at link 10 also `determinism-tiers.md` §2 and [`energy-binding.md`](../extensions/energy-binding.md) §7.1–§7.2 | analysed; the tier rule above is specified | complementary predictive or device-scoped work. At link 02, energy-binding §7.1–§7.2 cite 23001-11's decoding operation reduction request and response (DOR-Req / DOR-Resp) as the request/commitment case |
|
||||
| a per-figure acquisition class | [`energy-measurement.md`](../extensions/energy-measurement.md) §1–§2.9; `wai-rs/src/energy_class.rs`; `wai-quantum/src/quantum_energy.rs`; `wai-rs/src/claim_receipt.rs` | *`main`:* implemented in the energy binding and the video receipt only. *Link 10:* implemented in every `wai-rs` and `wai-quantum` receipt family that seals a figure (link 08, PR #109), and in an `energy` claim (link 10, PR #112); absent from the JWP group and object receipts and from OER | *`main`:* `HwShunt` / `ModelBased` / `Estimator`. [`jwp-receipts.md`](../extensions/jwp-receipts.md) §2.3–§2.4 and its implementation (`joule-code/joule-code-rs/crates/joule-moq/src/receipt.rs`) have no class field; the other receipt families carry none. *Link 10:* `CalibratedInstrument`, `OnChipCounter`, `ModelBased`, `Estimator` and legacy `HwShunt` (§2), one definition in `energy_class`, mirrored by `quantum_energy`. The class is always present in the energy binding, its C2PA companion assertion, the `wai.video.int_motion` receipt and the `wai.video.*` OTT receipts, and an optional signed label in the world-session, film and reel, haptic, score, 4D-splat, audio-scene, ternary and provenance-step receipts and in every `wai-quantum` receipt (`Labelled`, and `LabelledQbom` per entry). Every `wai-quantum` type with a `joules_micro` field is labellable (grep). Every `wai-quantum` receipt type with a `joules_micro` field implements the label, and the bill of materials is labelled per entry (grep). An `energy` claim (jwp-receipts §2.4.5) gives a JWP object's figure a signed place for its class. The JWP object and group receipts still have no member for it, the `WAI_RECEIPT` property carries it unsigned, and an OER/2 receipt has none (energy-measurement §1; G6) |
|
||||
| acquisition class derived from network-layer identities, never improved, with a **declared tolerance** | `energy-measurement.md` §2.1, §2.3 | *`main`:* specified, not implemented. *Link 10:* the declared uncertainty implemented for the hardware classes; the derivation specified, not implemented | *`main`:* §2.1 makes the tolerance a MUST, and no receipt has a field for one. *Link 10 (link 08, PR #109):* §2.3 requires `CalibratedInstrument` and `OnChipCounter` to carry a standard relative uncertainty with its integration window, and `energy_class` refuses a hardware class without one or below the figure's own resolution. §2.1 says a derivation from a network-layer identity SHOULD be made and MUST NOT improve the class, that a graded band MUST NOT be declared as an uncertainty, and that a declared uncertainty MUST NOT select a band. `ModelBased` and `Estimator` still carry no uncertainty (§2.1). No code maps `data-source-accuracy` identities (grep). 2030 §8 item 8 |
|
||||
| an evidential grade per figure; content-addressed, chained, signed receipts; a ledger identity tying the chain to a meter | [`oer-receipts.md`](../extensions/oer-receipts.md); `wai-rs/src/oer.rs` (feature `oer`); `oer-conformance/` | **implemented** (on `main`; CI runs `wai_oer_vectors`) | Four grades, weakest first: `projected`, `simulated`, `derived`, `measured`. The grade is a separate axis from the acquisition class, and §2 states which grade each class supports, extended at link 08 to the refined hardware classes. Its kinds include `reconstruction` and `decode`, but the conformance corpus holds only decision, bundle and anchor receipts, plus a `teleport` negative for `unknown_kind`. What its checks establish, and what they do not, is in §5.1 (G4, G6) |
|
||||
| request / commitment / delivery and the residual | [`energy-binding.md`](../extensions/energy-binding.md) §7 | **specified, not implemented** | `op_requested` / `op_committed` / `op_delivered` appear only in the extension text (grep over `wai-rs` and `wai-quantum`). 2030 §8 item 3 |
|
||||
| SCITT / COSE Receipts | [`transparency.md`](../extensions/transparency.md) (Status: Draft); at link 10 also `jwp-receipts.md` §2.4.7 | *`main`:* specified, not implemented. *Link 10:* a `log-inclusion` claim implemented; emission and inclusion checking specified only | *`main`:* an emission profile; no `scitt` or `cose_sign1` in any `*.rs` under `wai/`. *Link 10 (PR #112):* a registration MAY be recorded as a `log-inclusion` claim that carries the log's receipt verbatim with its registration profile; accepting that claim is not verifying the inclusion, and checking the log's receipt under its profile is not specified. Still no `scitt` or `cose_sign1` in code (grep). 2030 §8 item 7 |
|
||||
| MoQ MSF / LOC / CMSF | [`moq-streaming-format.md`](../extensions/moq-streaming-format.md) (Status: Draft) | **specified, not implemented** | a `packaging` value `"wai"` plus a per-object property; at link 08 the property gains `acquisition_evidence`. No MSF packaging code under `wai/` (grep for `msf`; the `packaging` hits are `wai.video.package`, a different layer). 2030 §8 item 13b |
|
||||
| Brazil lawful-basis declaration (ABNT NBR 25608 §8.8) | 2030 landscape §7.2; at link 10 also `jwp-receipts.md` §2.4.6, `wai-rs/src/claim_receipt.rs` (feature `receipt_claims`), `claim-conformance/` | *`main`:* analysed, open ingest. *Link 10:* specified and implemented as a claim class (PR #112) | *`main`:* no extension or code carries it (grep for `lawful`, `PRRD`, `privacy receipt` and `25608` outside the landscapes). *Link 10:* a `lawful-basis` claim is a declaration of purposes, each with its own basis and data categories from a declared vocabulary, or a decision recorded against one; its shape is the one jwp-receipts §2.4.6 attributes to the standard's public operational guidelines. It is not a consent mechanism, and nothing supplies an operator's key (jwp-receipts §2.4.6). The clause-number caveat of §3.3 stands. 2030 §8 item 4 |
|
||||
| MPEG-AI Part 6 CfP | [`mpeg-ai-part6-mapping.md`](mpeg-ai-part6-mapping.md) | analysed | requirement mapping; membership blocker stated |
|
||||
| the C2PA environmental-sustainability assertion | `energy-binding.md`; `wai-rs/src/energy_binding.rs` (`C2PA_STANDARD_LABEL`); `wai-rs/src/c2pa_emit.rs` | implemented | emitted as a companion alongside WAI's own assertion |
|
||||
| MPEG's splat and point-cloud work: Exploration 45; the V-PCC and G-PCC Gaussian-splat amendments; Enhanced G-PCC, ISO/IEC 23090-38 (§2.1) | [`media-codec-registry.md`](media-codec-registry.md), *Volumetric / 3D geometry compression*; `wai-rs/src/codecs/mod.rs` (`CAP_PC_GPCC`, `CAP_PC_VPCC`, `CAP_SPLAT_SOG`, `CAP_SPLAT_INT_CODEC`); `wai-rs/src/int_splat.rs`; [`volumetric.md`](../extensions/volumetric.md) (`wai.splat.4d`); [`3d-neural-coding-landscape.md`](3d-neural-coding-landscape.md); 2030 §6.4 | `wai.splat.int_codec` **implemented**. `wai.pc.gpcc`, `wai.pc.vpcc` and `wai.splat.sog` are **registered**: a capability string, a licence class and a tier, and no decoder inside `wai-rs` (grep). Their decoders are sink-supplied. `media-codec-registry.md` records a demonstrated end-to-end decode of `wai.pc.gpcc` through the MPEG reference tool built from source, and a real encode plus structural decode of `wai.pc.vpcc`, both via `tools/wai_carriage_demo.sh` (not re-run here). CI's `tests/carriage.rs` covers byte-identical carriage and known-capability dispatch for the registry, not these decodes | Registry: `wai.pc.gpcc` (23090-9) and `wai.pc.vpcc` (23090-5) are POOL. `wai.splat.sog` is "MPEG WG4 WIP", "RF (open work)", a **candidate** "registered so a capability exists ahead of an MPEG freeze". *`main`:* `capability_license` gives `PatentPool` for the two PCC capabilities and, through its default arm, `RoyaltyFree` for `wai.splat.sog`; `capability_replicate_class` gives all three `StandardDefined` through its default arm. *Link 10:* all three are `StandardUnclassified` (link 02, PR #103; SPEC §7's table), and `capability_license` lists every capability with no default class, `wai.splat.sog` as royalty-free "SOG open work" (link 04, PR #105). `3d-neural-coding-landscape.md` says splat coding applies NNC to the splat parameters; 2030 §6.4 records an NNC amendment covering Gaussian splats, unverified there (G11) |
|
||||
| a royalty class per capability | [`media-codec-registry.md`](media-codec-registry.md), *License classification*; `codecs::capability_license()` / `LicenseClass` | implemented | the mechanism any new pool-licensed registration uses. At link 04 the function is exhaustive with no default class, so a registered capability cannot become royalty-free by omission (G5) |
|
||||
|
||||
### 6.1 The 2030 energy-square falsifier, leg by leg
|
||||
|
||||
2030 §9 item 1 reads: *"If any of ISO/IEC 23001-11's 4th edition, MPEG-AI Part
|
||||
6's eventual working draft, or a GREEN WG document adds a measured per-object
|
||||
figure with an acquisition class bound to a reproducible output under a
|
||||
signature, then §2's 'unoccupied conjunction' is false … Check at each MPEG
|
||||
meeting and at each GREEN WG revision."*
|
||||
|
||||
| leg | what was checked | result |
|
||||
|---|---|---|
|
||||
| ISO/IEC 23001-11 4th edition | the part page's one-line description and the 2030 §5.1 tool list, read for this record. The CD text (WG 3 N1864) was not read for this record; [`energy-exposure-mapping.md`](energy-exposure-mapping.md) §4 searched it for *signature*, *hash*, *attest*, *certif*, *joule*, *watt*, *measured* and *measurement* and found none | **not triggered at CD** (tree: energy-exposure-mapping §4). Recheck at DIS |
|
||||
| MPEG-AI Part 6 working draft | none exists: the CfP closes 13 Jan 2027 (§2.1) | not applicable yet |
|
||||
| GREEN WG documents | the four WG drafts at their current revisions (§2.3), fetched as text. Searched for signing, provenance, attestation, certification and per-object or per-stream terms; the security sections, `framework-02` §4.1.4–§4.1.6 (read 2026-09-27) and the video-streaming use case (`use-cases-02` §2.8) were read | **not triggered.** Energy is scoped per hardware component (the Energy Object). `-04` carries a measured/estimated accuracy hierarchy, already recorded at `-03` in 2030 §3.5. No draft signs energy values. `framework-02` §8 asks for authentication, authorisation and integrity protection of control actions, and protection of monitored data; it adds that monitored data "may be used as input to control, accounting, and other actions, so integrity of transmitted information and authentication of the origin may be needed". Its §4.1.4 requires every reported metric to carry an accuracy indication; §4.1.5 supports reporting industry-standard efficiency **certifications** per Energy Object, as a device-level reference; §4.1.6 gives an example accuracy identity, "Vendor-specific calibrated sensor with certificate ID XYZ". These are the GREEN material closest to F1 and to §5.1's calibrated-instrument point: a certificate identity attached to a sensor class, and origin authentication for accounting. None of them signs a per-object figure or binds one to a reproducible output. The video use case concerns traffic volume, multicast and caching. The individual draft `draft-madpr-green-provenance-00` (2030 §3.5) is not a WG document and had no newer revision on 2026-09-27 |
|
||||
|
||||
**2030 falsifier #1 is not shown to be triggered.** That holds on the GREEN leg
|
||||
(checked here), on the 23001-11 leg at CD stage (by the term search in
|
||||
energy-exposure-mapping §4), and is moot for now on the Part 6 leg. The 23001-11
|
||||
leg must be re-checked at DIS.
|
||||
|
||||
---
|
||||
|
||||
## 7. Gaps, and their status in the landing chain
|
||||
|
||||
G1–G11 were found on `main`, by the method in the *evidence* column; every file
|
||||
that column cites by line is unchanged on `main` since afc7ece. The **status at
|
||||
link 10** column says what the tree at link 10 holds, and which link and pull
|
||||
request carries it. A closed gap names where it closed. An open gap stays open.
|
||||
None of the chain is in `main` until it merges, and each pull request has its own
|
||||
review.
|
||||
|
||||
| # | gap (on `main`) | evidence on `main` | status at link 10 |
|
||||
|---|---|---|---|
|
||||
| G1 | **The integer decoders are scalar and single-threaded, and no throughput figure is published.** A determinism tier with no speed figure cannot be weighed against an approach that pays in bitrate rather than arithmetic (G7). | no `rayon`, `std::thread`, `core::arch`, `std::simd` or `target_feature` in `wai-rs/src/int_*.rs`, `rans.rs` or `jpegai_*.rs`; no throughput figure in `README.md`, `SPEC.md` or `determinism-tiers.md` (grep) | **Closed at link 07 (PR #108).** `wai-rs/src/int_kernels.rs` adds register-blocked convolution, transposed-convolution and IGDN kernels (SIMD widening multiply-accumulate on aarch64, portable Rust elsewhere) and scoped worker threads (none on wasm), selected by a run-time `Policy` whose default is the fast kernels on all cores. The scalar reference kernels are kept and selectable. The claim is that every policy gives the same integer for every output element; it is argued by construction and tested, not machine-proved (SPEC Appendix A). [`decode-throughput.md`](decode-throughput.md) §4 publishes whole-decoder and kernel figures from `wai_decode_bench`, measured on one macOS arm64 machine, best of four runs under a stated load (§3). Its limits (§5): one machine and one architecture; x86_64 compiled but not executed; the largest rows use synthetic weights. determinism-tiers §5 (*What `decode-equivalence` costs*) described the decoders as single-threaded with no explicit SIMD and no throughput figure until link 11, which describes the fast kernels and points to the note |
|
||||
| G2 | **The SOTA-quality and JPEG-AI whole-decode tests skip-as-pass when their inputs are absent, and no CI job runs them.** A skipped test reports as passed. | `wai-rs/tests/int_hyper_synth.rs` and `int_video_sota_keyframe.rs` (`return` when the artifacts are not in `WAI_SOTA_INT_DIR`, default `/tmp/sota_int`); `jpegai_e2e_full.rs` (`return` without `WAI_JPEGAI_CODESTREAM` / `_WEIGHTS_DIR` / `_NORM`). The WAI workflows run `cargo test --lib`, `--test carriage`, `--test neural_native_decode`, the `byte-exact-conformance` stage goldens, the corpus verifiers and `wai_oer_vectors`; none runs the three test targets above | **Addressed at link 06 (PR #107); both workflows pass on the forge.** With `WAI_REQUIRE_SOTA_VECTORS=1` a missing or incomplete input fails the test and names the tool that produces it; without the switch the skip is written to the process's real stderr (`tests/support/require_vectors.rs`; `tests/require_vectors_rule.rs` checks the rule in child processes). `wai-sota-vectors.yml` regenerates the `int_hyper` vectors from pinned inputs and runs `int_hyper_synth`, `int_video_sota_keyframe` and `require_vectors_rule`. `wai-jpegai-e2e.yml` fetches the pinned JPEG AI inputs, checks them against `tools/jpegai-e2e-v1.sha256`, and runs `jpegai_e2e_full`, `jpegai_reconstruction_binding` and `jpegai_cross_platform`. Both set the switch, require each test by name, fail on a skip banner, and also run weekly. Both have run on the forge's Linux x86_64 runner and passed on every run since 2026-09-28 (runs read 2026-09-30); determinism-tiers §6 says so from link 11. Still in no workflow: `jpegai_live` and the dispatch test `jpegai_exact_bundle_decodes`, which need the `neural` feature and local assets |
|
||||
| G3 | **SyfAI's AI update-framework thread, opened at MPEG 155 jointly with WG 7 (S ⚠), covers parameter formats, manifests, repositories, integrity checking and bit exactness for NN decoders. The artifact a WAI `Prior` names has no normative wire form.** `Prior { id, sha256, determinism }` pins identity: `sha256` hashes the decoder artifact the sink runs. No wire format for that artifact is specified (2030 §6.4). | `wai-rs/src/container.rs` `struct Prior`; 2030 landscape §6.4; no SyfAI or Exploration 52 mention in the tree (grep) | **Addressed at link 05 (PR #106), with limits.** [`prior-carriage.md`](../extensions/prior-carriage.md) §2: a `Prior` MAY declare `form`; when it does, the digest covers the coded bitstream, and a sink that cannot read the form MUST refuse the prior. §3 registers `wai.prior.nnc` (ISO/IEC 15938-17:2024) as the one form, optional and not recommended, because its royalty status and cross-decoder exactness are not established. `Prior::is_cross_hardware_exact` requires a declared form to be established exact, none is, so a `replicate` resting on a declared form is refused (§5). The reference sink reads no form (`READABLE_PRIOR_FORMS` is empty), and `wai-rs` has no NNC reader or writer. SPEC's opening section now says a model's identity MAY be pinned by digest (a `Prior`). [`syfai-input.md`](syfai-input.md), committed with this record, maps WAI onto the update framework; its G1 records what SPEC still does not define |
|
||||
| G4 | **WAI's `HwShunt` takes in IOReport, a reading the monorepo classes five ways; and neither WAI nor the family's receipt ladder separates platform-relayed counters from independently rooted instruments. Only `sandbox-tee` does.** Three layers. **(a) The family's receipt ladder.** Every standard that lists interfaces under `HwShunt` puts on-die counters and dedicated shunts in it together (JCR-1 §2.3, JCP §5.1, Sandbox §6.3, `mesh-schema`, `proof-schema`), and IOReport is classed inconsistently (§5.3). **(b) `sandbox-tee`'s `CounterClass`** reserves `HwShunt` for a dedicated shunt and gives RAPL, NVML, IOReport and Redfish classes of their own; its `AttestTrust` ladder is derived from evidence, and `efficiency/1` commits to the attestation (§5.1). **(c) WAI** has `HwShunt` cover "RAPL, NVML, SMC, IOReport, or an external shunt", and `wai_video_meter` stamps IOReport readings `HwShunt`. That places WAI with Joule Code's text and the Sandbox meter, and against JCR-1 §2.3. WAI's texts present the ladder as the family's own without noting the split: `video_receipt.rs` says it is "reused verbatim from `mesh-schema` / `proof-schema`", which list no IOReport; `energy_binding.rs` calls it "the family's ladder"; `energy-measurement.md` §2 says WAI reuses "the same three names the receipt families elsewhere in the ecosystem already use". At every layer but (b), a verifier cannot tell a platform-relayed counter, which may be deliberately filtered (§5.1), from an independently rooted instrument. | §5.3 (the per-standard survey); `wai/extensions/energy-measurement.md` §2; `wai-rs/src/energy_binding.rs`:30, :47–51 (`Acquisition`); `wai-rs/src/video_receipt.rs` `Provenance` doc comment; `wai-rs/src/bin/wai_video_meter.rs`:184–190; `jouleclaw/jouleclaw-rs/crates/jouleclaw-jcr1/docs/JCR-1-spec.md` §2.3; `jcp/SPEC.md` §5.1; `sandbox/SPEC.md` §6.3–§6.4; `mesh/mesh-rs/crates/mesh-schema/src/lib.rs` and `proof/proof-rs/crates/proof-schema/src/lib.rs` (`Provenance`); `sandbox/sandbox-rs/crates/sandbox-tee/src/energy.rs` (`AttestTrust`, `CounterClass`, `is_independently_rooted`); `jcp/jcp-rs/crates/jcp-efficiency/MAPPING.md` §4 and `src/training.rs` (`energy_attestation`). No mention of `sandbox-tee`, `CounterClass`, `AttestTrust`, `energy_attestation`, `PhysicallyRooted` or `IntegrityAttested` under `wai/` (grep) | **WAI's side closed at link 08 (PR #109); the family's side open.** energy-measurement §2 separates `CalibratedInstrument` (an external instrument with a current calibration) from `OnChipCounter` (a counter the measured silicon keeps, or any instrument whose uncertainty is not traceable). Each carries a declared uncertainty (§2.3); the counter carries its filtering state (§2.4), the instrument a calibration reference (§2.5). `HwShunt` is legacy, accepted and never emitted (§2.6), and IOReport is `OnChipCounter` (§2.9). `energy_class.rs` is the one definition, and the reference meters seal `OnChipCounter`. The "reused verbatim" sentence is gone; `video_receipt.rs` and `energy_binding.rs` still call the ladder the family's, and §2 says the three-name vocabulary is extended, not replaced, with both refinements mapping to `HwShunt`. §2.9's list of differing documents is partial (§5.3). No WAI text references `sandbox-tee`'s ladder (grep). The family's IOReport split and whether its receipt-level ladder should separate the two kinds of hardware evidence are unassigned (below) |
|
||||
| G5 | **The codec registry has no AV2, VVC, HEVC, LCEVC, MPEG-H Audio, AVS3 or Audio Vivid capability.** Brazil mandates three of these (§3.3). Two are Chinese standards: AVS3 (a group standard, with its national standard pending) and Audio Vivid (GB/T 46271-2025). 2030 §6.1 (*Honest scope*) records that WAI registers no VSEI-family codec. **The tree's codec survey is a second surface to reconcile.** `wai/glossary/codecs.json`, which `wai/README.md` describes as the "Reference catalog of media codecs surveyed for capability registration", has entries for HEVC / H.265 and VVC / H.266 (both `niche`), LCEVC (`niche`) and ECM (`academic`, JVET's exploration model beyond VVC), and none for AV2, MPEG-H Audio, AVS3 or Audio Vivid. `glossary/SCHEMA.md` defines `success` as "carries non-trivial production traffic today" and `niche` as "alive in specific contexts". §4 reports HEVC in production use at 65% in a vendor survey (**S** ⚠), §3.3 reports VVC as a national broadcast system's main codec (**R** ⚠), and the HEVC entry's own description calls it universal in broadcast UHD. The grades are descriptive and the contrary evidence is unverified, so this is a reconciliation item, not a correction. | capability strings in `wai-rs/src/codecs/*.rs`: none for these codecs. Other mentions, from a grep over `*.rs`, `*.md`, `*.json`, `*.toml`, `*.yml`, `*.yaml`, `*.html`, `*.ts` and `*.js` under `wai/`: the `glossary/codecs.json` entries above; the H.264/H.265 royalty sentence in `SPEC.md` and `CHANGELOG.md`; the BD-rate note that names VVC in `wai-rs/src/codecs/mod.rs` and `wai-rs/src/neural/mlicpp.rs`; `HEVC` as an example name in `glossary/SCHEMA.md`; the 2030 landscape. Royalty class per codec **not established by this sweep**: only AV2's royalty-free patent policy was seen at the issuing body (the AOMedia press release, §2.4) | **Closed in the registry at link 04 (PR #105); the glossary is not reconciled.** SPEC §5 *Broadcast and streaming codecs* registers `wai.video.{av2,vvc,hevc,avc,vp9,evc,evc.baseline,avs3,lcevc}` and `wai.audio.{mpegh,avs3,aac}`, each with one payload framing and a licence class whose basis is recorded in [`media-codec-registry.md`](media-codec-registry.md). `wai.audio.avs3` names AVS3 audio / Audio Vivid (GB/T 46271-2025). All are sink-supplied, none is a default, and `wai.video.lcevc` is companion-only. SPEC §7 classes AVC, HEVC, VVC and VP9 `StandardDefined` on the conformance clauses read, AV2 `StandardTolerance`, and EVC, AVS3, LCEVC, MPEG-H 3D Audio and AAC `StandardUnclassified`. The licence classes rest on the registry's recorded bases, not on this sweep. `glossary/codecs.json` is unchanged, so its grades still stand against §3.3 and §4 |
|
||||
| G6 | **No WAI receipt says who can check the energy figure, and the base JWP receipt does not carry even the acquisition class that `energy-measurement.md` requires.** The precedents in §5.1 differ exactly on who can check: instrument-signed (OCMF), peer-reviewed but unsigned, log-registered (SCITT), TEE-attested with a derived trust level (the family's `sandbox-tee` `AttestTrust`), or producer-asserted. **The base JWP receipt** has no acquisition-class field; its group signature covers `total_joules_micro` and no class, and the per-object `joules_micro` is in neither the Merkle leaf nor the signature. `energy-measurement.md` §2 requires a class inside the signature for any non-zero figure, while its header says it adds "no new wire format". **The video receipt** signs its class. **The energy binding** carries its class in a C2PA companion assertion signed as part of whatever manifest carries it. **The OER receipt** carries an evidential grade and a ledger identity, and no class. 2030 §8 item 4 queues typed receipt classes. | `extensions/energy-measurement.md` header, §1, §2 and §5; `extensions/jwp-receipts.md` §2.1–§2.4; `joule-code/joule-code-rs/crates/joule-moq/src/receipt.rs` (`ObjectReceipt`, `GroupReceipt::signing_payload`: no class); `wai-rs/src/energy_binding.rs`, `wai-rs/src/video_receipt.rs`; `extensions/oer-receipts.md` §2–§3, `wai-rs/src/oer.rs`; no reference to the family ladder under `wai/` (grep, as in G4) | **Partly closed at link 10 (PR #112); the rest open.** Every receipt within energy-measurement's scope that `wai-rs` and `wai-quantum` seal can carry the class (link 08, PR #109). jwp-receipts §2.4.3–§2.4.7 add a `claim` kind with typed classes. An `energy` claim (§2.4.5) restates a JWP object's figure beside its class, the class's evidence, a coverage statement and a grade; a verifier holding the object receipt rejects the pair if the figures differ, and one holding the OER receipt the claim names checks that grade and figure agree. `claim_receipt.rs` (feature `receipt_claims`) implements it, with vectors in `claim-conformance/`. Open, as energy-measurement §1 records: the JWP object receipt has no member of its own for the class; the group total has none, and nothing carries one for it; the `WAI_RECEIPT` property's class and evidence sit outside any signature; an OER/2 receipt has no member for the class. On who can check: a claim carries its signer's authority and no more (§2.4.4); key discovery and an authority class for who may sign which class are not done (CHANGELOG, v0.4.0 entry for typed receipt classes); no WAI receipt references `sandbox-tee`'s ladder or `efficiency/1`'s attestation (grep) |
|
||||
| G7 | **Explicit parameter transmission appears in `determinism-tiers.md` only as a generic route. Its first published cross-NPU instance (arXiv 2606.28027) and its cost are not recorded.** Also unrecorded: the mixed-precision entropy-synchronisation line (arXiv 2608.00483: FP16/FP32 across GPUs, no bit-exactness claim); the quantised learned-coding line (arXiv 2007.04684, 2111.09348); and IEEE 1857.11's use of *reproducibility cross-checks* rather than bit-exact decode (**R** ⚠), a reproducibility notion the tier vocabulary does not name. Both 2606.28027 and 2608.00483 target a synchronised entropy stage, so at most `entropy-consistency`, and 2608.00483 only over the GPUs it tested. On `main`, whether such a decode may back `replicate` depends on which of G10's conditions is read. | `extensions/determinism-tiers.md` §1–§3; `SPEC.md` §7 (read) | **Partly closed at link 02 (PR #103).** determinism-tiers §1 names arXiv 2606.28027 as the explicit-parameter instance and says the side information costs bitrate (no magnitude), and names a quantised decoder, arXiv 2312.11209, as the route to a reproducible reconstruction. Such a decode reaches `entropy-consistency` and does not back `replicate` (G10). Not in the tree at link 10 (grep): arXiv 2608.00483, the quantised line 2007.04684 / 2111.09348, and IEEE 1857.11's reproducibility cross-checks; the last is described in [`syfai-input.md`](syfai-input.md) §5, committed with this record |
|
||||
| G8 | **Standards inputs not tracked in the tree:** SyfAI; 3GPP TR 26.942; DVB S100 and CM-EE; the beyond-VVC CfP; Gaussian Splat Coding Exploration 45; IEEE 1857.11 / P1857.11.1; UWA 042; India D2M; the JPEG AI core experiments on energy consumption and bit-exact reconstruction. | a case-insensitive, word-bounded `git grep -P` over tracked `*.md`, `*.rs`, `*.json`, `*.toml`, `*.yml`, `*.yaml`, `*.html`, `*.ts` and `*.js` files finds none of these designations outside this record. The one other hit for `uwa` is the Hausa word in `aac/aac-rs/crates/aac-core/src/translations_data.json` (`"ha":"uwa"`), which is not the organisation. MPEG's splat work *is* tracked, though not Exploration 45 by name: `media-codec-registry.md` and `codecs/mod.rs` register `wai.splat.sog` against "MPEG WG4" work in progress, `3d-neural-coding-landscape.md` records splat coding under standardisation, and 2030 §6.4 records an NNC amendment covering Gaussian splats (G11) | **Partly closed by the two notes committed with this record.** [`syfai-input.md`](syfai-input.md) tracks SyfAI, IEEE 1857.11 and UWA 042; [`energy-exposure-mapping.md`](energy-exposure-mapping.md) tracks TR 26.942, DVB S100 and CM-EE. The same grep over the tree at link 10 finds the beyond-VVC CfP, Exploration 45, India D2M and the JPEG AI core experiments in no document but this record. Those four stay open |
|
||||
| G9 | **WAI's texts and code class `wai.image.jpegai` in ways that contradict one another, and SPEC contradicts itself on it.** SPEC §5, the capability's registration (`SPEC.md`:197), describes the decode as "integer entropy front end (mANS) + integer dequant/value path + fixed-order synthesis and post-filters" and declares "**decode-equivalence**: one pinned reconstruction … reproducible on every machine (a portable decoded-pixel hash)". `reconstruction-binding.md` §1 (:30–31) says the decode to RGB is "byte-identical on every machine", and §2 (:72) sets the binding's criterion at `decode-equivalence`; `determinism-tiers.md` §2–§3 list the capability there. SPEC §7 lets a neural capability back `replicate` only "with no float operation anywhere" (:445–448). `capability_replicate_class` returns `StandardDefined` for it through the default arm, and SPEC §7 defines that class as "reconstruction is fixed by the named codec/standard … the neural float hazard does not apply"; neither half holds for JPEG AI (§2.2; its decode runs the model). The strict `replicate` check reads only the envelope's prior (`neural/mod.rs`:282–292). **Loose ends:** (i) `jpegai_synthesis.rs` and `jpegai_mans.rs` say they back `wai.neural.jpeg_ai`, a string registered nowhere; (ii) the decode has f32 steps between its integer stages — its dequantisation among them, although SPEC §5 calls it integer; (iii) the byte-for-byte claim is made normatively, in SPEC §5 and `reconstruction-binding.md` §1, and rests on an `exp()` from the platform maths library and on no measurement on a second platform; (iv) the reference sink's dispatch (`neural/mod.rs`:205–216) uses the exact decoder only when the integer weight bundle is present, and otherwise falls back to a float ONNX Runtime decode. | `SPEC.md` §5 (:197) and §7 (:445–448, the `StandardDefined` bullet at :476–478, :492–495); `extensions/reconstruction-binding.md` §1 (:21–35) and §2 (:72); `extensions/determinism-tiers.md` §2–§3; `wai-rs/src/codecs/mod.rs` (the default arm and the test asserting `StandardDefined`); `wai-rs/src/jpegai_decode.rs`, `jpegai_synthesis.rs`, `jpegai_mans.rs`, `jpegai_pipeline.rs`; `wai-rs/src/neural/mod.rs`:205–216, :282–292; `git grep -n decode-equivalence` over `wai/` | **Closed at link 02 (PR #103), with the maths library removed at link 03 (PR #104).** SPEC §5's row registers `entropy-consistency` (registry class `NeuralEntropyExact`), describes dequantisation, latent refinement, post-filters and colour conversion as IEEE-754 float with a fixed evaluation order, and says "**Downgraded in v0.4.0:** this row previously claimed decode-equivalence". `reconstruction-binding.md` §1–§2 make the pinned reconstruction a property of the exact decoder, reproducible under that decoder's build and floating-point conditions. SPEC §7 states the new class and the reason. Loose ends: (i) fixed — the module docs name `wai.image.jpegai`, and `wai.neural.jpeg_ai` appears nowhere else in the tree (grep); (ii) the float steps remain and are now stated as float; (iii) link 03 builds `exp` and rounding from IEEE-754 basic operations (`float_repro`), guards the decode path with clippy's `disallowed_methods` and a linked-symbol check (`tests/jpegai_decode_symbols.rs`), and pins nine hashes for one codestream; determinism-tiers §6 records them identical on macOS arm64, wasm32-wasip2, Linux arm64 and Linux x86_64 under user-mode emulation, and not measured on Windows, macOS x86_64 or x86_64 hardware; (iv) the float fallback remains, and SPEC §5, determinism-tiers §5 and reconstruction-binding §1 now state it, and that the result does not record which path ran. Still true: a `norm.json` with a missing or mistyped key panics in the loaders instead of falling back, as the `try_jpegai_exact` doc comment says |
|
||||
| G10 | **WAI's texts give three conditions for a neural decode to back `replicate` or a portable reconstruction hash, and SPEC states all three.** (1) **All-integer:** SPEC §7 (`SPEC.md`:445–448, :492–495), enforced in code on the envelope's declared prior (`replicate_is_sound` requires `DeterminismClass::IntegerExact`, a substring heuristic over `Prior.determinism`). (2) **`decode-equivalence` with fixed-order float admitted:** `determinism-tiers.md` §2 defines the tier over "integer / fixed-order arithmetic end to end" (:57–58), and SPEC §5 registers `wai.image.jpegai`, whose float stages run in a fixed order, at that tier. (3) **`entropy-consistency` as the floor:** SPEC §7's closing paragraph (:497–505) calls it "the floor for a sound neural `replicate`", and `determinism-tiers.md` §2–§3 repeat it; §3 also says "a float neural decode meets neither tier", which does not square with the explicit-parameter route. A decode with integer entropy and fixed-order float synthesis meets (2) and (3) but not (1). | `SPEC.md` §5 (:197) and §7 (:445–448, :492–495, :497–505); `extensions/determinism-tiers.md` §1–§3; `extensions/reconstruction-binding.md` §1–§2; `wai-rs/src/container.rs` (`Prior::determinism_class` at :108–117, `is_cross_hardware_exact`, `replicate_is_sound`) | **Closed at link 02 (PR #103), on condition (1).** SPEC §7: a neural capability backs `replicate` only if its entire decode path is integer / fixed-point with no float operation, "the only construction by which a neural capability is registered at `decode-equivalence`"; `entropy-consistency` ("necessary for a sound neural `replicate` … and not sufficient") and a reconstruction pinned by one implementation's fixed-order float stages MUST NOT back it. determinism-tiers §2 defines `decode-equivalence` over "integer / fixed-point arithmetic end to end", and §3 says a decode that derives its entropy parameters in float meets neither tier while one that reads them from the codestream, or derives them in integer arithmetic, and runs later stages in float meets `entropy-consistency` only. The strict check reads the registry class before the prior. Remaining: the prior check still reads a sender-declared string by substring, which determinism-tiers §3 states |
|
||||
| G11 | **Three volumetric registrations sit against MPEG's 2026 splat and point-cloud work, and none of the sweep's rows was mapped to them.** `wai.splat.sog` is a candidate "registered so a capability exists ahead of an MPEG freeze" (registry: "MPEG WG4 WIP", "RF (open work)"). `wai.pc.gpcc` (23090-9) and `wai.pc.vpcc` (23090-5) are registered POOL. None has a decoder inside `wai-rs`; the registry records demonstrated sink-side decodes via `tools/wai_carriage_demo.sh` (not re-run here, and not in CI). The sweep found a V-PCC Gaussian-splat CDAM (MPEG 153), a G-PCC Gaussian-splat Amd 1 at CDAM (MPEG 154), Enhanced G-PCC ISO/IEC 23090-38 at FDIS (MPEG 155), and Exploration 45's document series up to a call for content at MPEG 155, all P✓; its tracks and dates are **R**. Questions: **(a)** whether what MPEG standardises is what `wai.splat.sog` names, which cites Self-Organizing Gaussians / SOGS while the part page names no candidate technology; **(b)** whether its royalty class holds (`capability_license` returns `RoyaltyFree` through the default arm; nothing the sweep read establishes the licensing of a codec MPEG selects); **(c)** whether 23090-38 is a payload `wai.pc.gpcc` admits or needs its own capability, and whether the splat amendments change what the two PCC capabilities admit; **(d)** the replicate class: all three fall through to `StandardDefined`, although `wai.splat.sog` has no frozen standard. The tree's "WG4" attribution was not checked (the Exploration 45 page names no working group), and `3d-neural-coding-landscape.md` describes a "future `wai.pointcloud.*`" without mentioning the registered `wai.pc.*` strings. | `research/media-codec-registry.md`; `wai-rs/src/codecs/mod.rs` (`CAP_PC_GPCC`, `CAP_PC_VPCC`, `CAP_SPLAT_SOG` and their arms); `tools/wai_carriage_demo.sh`; `wai-rs/tests/carriage.rs`; `research/3d-neural-coding-landscape.md`; 2030 §6.4 | **(d) closed at link 02 (PR #103); (b) made explicit at link 04 (PR #105); (a), (c), the "WG4" attribution and the `wai.pointcloud.*` sentence open.** All three are `StandardUnclassified`, the class for a standard whose decoder-conformance clauses have not been checked (SPEC §7 notes that `wai.splat.sog` names no frozen standard yet). `capability_license` lists `wai.splat.sog` explicitly as royalty-free ("SOG open work"), which makes the class explicit and does not establish it. `codecs/mod.rs` and `media-codec-registry.md` still say "MPEG WG4", and `3d-neural-coding-landscape.md` still describes a "future `wai.pointcloud.*`". Exploration 45 is untracked (G8) |
|
||||
|
||||
**Found but assigned to no link.** These are recorded so they are not lost:
|
||||
|
||||
- **The 23001-11 4th edition at DIS.** The CD text was searched for
|
||||
[`energy-exposure-mapping.md`](energy-exposure-mapping.md) §4; the DIS text
|
||||
needs the same search (§6.1, F1).
|
||||
- **JPEG AI's core experiments on bit-exact reconstruction** (§2.2). If JPEG AI
|
||||
adopts a bit-exact reconstruction conformance point, the tier split between
|
||||
what the base standard fixes and what WAI adds moves for JPEG AI (F11). In the
|
||||
tree at link 10, determinism-tiers §4 calls `entropy-consistency` "the tier at
|
||||
which JPEG AI sets its bit-exact conformance point", and SPEC §7 records that
|
||||
the conformance part has not been read. Neither text tracks the core
|
||||
experiments.
|
||||
- **The family's IOReport and `HwShunt` vocabulary** (G4, §5.3) raises two
|
||||
questions beyond WAI.
|
||||
- *Which class IOReport gets.* The monorepo classes an IOReport reading five
|
||||
ways, and JCP, JouleClaw, Joule Code and Proof are each split internally.
|
||||
JCR-1 envelopes can therefore carry the same reading as `ModelBased` or
|
||||
`HwShunt` depending on which standard sealed them. Settling it spans JCR-1
|
||||
§2.3 and every standard in §5.3; ROCm SMI needs the same decision.
|
||||
energy-measurement §2.9 states WAI's own choice (`OnChipCounter`) and lists
|
||||
some of the differences; it does not, and cannot, settle the others.
|
||||
- *Whether the receipt-level ladder should separate platform-relayed counters
|
||||
from independently rooted instruments,* as `sandbox-tee`'s counter classes
|
||||
and WAI's `OnChipCounter` / `CalibratedInstrument` split do. A family
|
||||
question, unassigned.
|
||||
- *Stale texts found on the way:* `joule-code/examples/l3-jouleclaw/README.md`:44–45
|
||||
(says `Estimator` "until IOReport is wired", while its `run.rs` uses the
|
||||
`ModelBased` counter); `jouleclaw-compliance` `passport.rs`:117, :133
|
||||
(`Estimator`, "or estimator until FFI ships", while `jouleclaw-energy` has an
|
||||
IOReport reader); `proof-cascade` `meter.rs`:9–10 against the same file's
|
||||
:1–2.
|
||||
- **The open parts of G6.** A signed place for a JWP group total's class, a
|
||||
member of its own for the JWP object receipt, a signature over
|
||||
`WAI_RECEIPT`, and key discovery for claim signers.
|
||||
- **G7's other lines** (arXiv 2608.00483, the quantised learned-coding line,
|
||||
IEEE 1857.11's reproducibility cross-checks) and **G8's other inputs** (the
|
||||
beyond-VVC CfP, Exploration 45, India D2M, the JPEG AI core experiments) are
|
||||
not in the tree at link 10.
|
||||
- **MoQ secure objects.** [`confidentiality.md`](../extensions/confidentiality.md)
|
||||
is built on the `draft-jennings-moq-e2ee-mls` binding (MLS + SFrame), and the
|
||||
WG draft `draft-ietf-moq-secure-objects-01` exists. **The relationship between the
|
||||
two was not examined.**
|
||||
- **`draft-ietf-avtcore-rtcp-green-metadata` is in the RFC Editor queue**
|
||||
(§2.3). When it publishes, 2030 §5.1's reference becomes an RFC number.
|
||||
- **JPEG Trust Part 3 (media asset watermarking), released for DIS balloting**
|
||||
(§2.2), bears on the second binding class (2030 §3.4), which link 10 adds as
|
||||
a `soft-binding` claim with an ancestor rule (jwp-receipts §2.6.2, §3.2;
|
||||
PR #112). It is a watch item.
|
||||
- **MPEG-Green data transport** in the MPEG-2 Systems 10th edition WD is
|
||||
carriage, which 2030 §4.5 says WAI does not bind. It is a watch item only.
|
||||
- **EU C(2026) 3472** (§5.2): its Annex III item on certification of reported
|
||||
data, with self- and third-party certification kept distinct, and its 2028
|
||||
review of (partial) certification or auditing. Both concern data centres. A
|
||||
watch item for F2 and for G6's who-can-check question.
|
||||
|
||||
---
|
||||
|
||||
## 8. Updates and corrections to the 2030 landscape
|
||||
|
||||
These are recorded here and are not applied to that file. The 2030 landscape in
|
||||
the tree at link 10 still carries each reading in the second column. Apply them
|
||||
when that document is next revised.
|
||||
|
||||
| 2030 section | it says | this sweep observed | grade |
|
||||
|---|---|---|---|
|
||||
| §3.3 (FCM paragraph) | "MPEG 155 (August 2026)" | MPEG 155 met **13–17 Jul 2026** (mpeg.org/meetings; the 2030 landscape's own §6.5 agrees). The mpeg.org 155th-meeting press page is posted 12 Aug 2026, a likely origin of "August" | P✓ |
|
||||
| §3.3 (FCM paragraph and its verification note) | "no published CD/DIS/FDIS milestone on mpeg.org"; the catalogue's CD is "one milestone further than mpeg.org's silence implied" | mpeg.org is not silent: the MPEG 154 page says "a CD of ISO/IEC 23888-4 was issued at the 23rd ISO/IEC JTC 1/SC 29/WG 4 MPEG Video coding meeting" (re-read 2026-09-27). The catalogue's CD agrees with it | P✓ |
|
||||
| §3.5, §5.1 | `draft-ietf-green-power-and-energy-yang-03` | now `-04` (2026-09-09): same accuracy hierarchy; still per-component and unsigned | P✓ |
|
||||
| §4.4 | `draft-ietf-moq-transport` rev 18, approaching WG Last Call | now `-21` (2026-09-08), state WG Document; transport and C4M to IESG targeted Dec 2026 | P✓ |
|
||||
| §5.1 | `draft-ietf-avtcore-rtcp-green-metadata-16`, IESG Evaluation | now `-17` (2026-09-09); IESG state **RFC Ed Queue** | P✓ |
|
||||
| §5.1 | 23001-11 4th ed. at CD (N01742) | dated on the part page: WD 2026-03-02 (MPEG 153), CD text 2026-09-21 (MPEG 155). The CD text is WG 3 N1864 ([`energy-exposure-mapping.md`](energy-exposure-mapping.md) §1) | P✓ / tree |
|
||||
| §9 item 1 | three legs (23001-11 4th ed., Part 6 WD, a GREEN WG document); "check at each MPEG meeting and at each GREEN WG revision" | GREEN leg checked and not triggered; Part 6 leg not applicable yet; 23001-11 leg not triggered at CD on energy-exposure-mapping §4's search (§6.1) | P✓ / tree |
|
||||
|
||||
---
|
||||
|
||||
## 9. Dated windows
|
||||
|
||||
| when | what | note |
|
||||
|---|---|---|
|
||||
| 19–23 Oct 2026 | MPEG 156, Hangzhou (dates P✓). That SyfAI (Exploration 52) and its AI update-framework thread continue there is the record's expectation from their document series (§2.1), **not a published agenda** ⚠ | input contributions need ISO/IEC JTC 1/SC 29 national-body membership (the blocker already stated in 2030 §6.5). [`syfai-input.md`](syfai-input.md) is WAI's draft input toward this window; it is not submitted |
|
||||
| end of Oct 2026 | beyond-VVC CfP: main response package due; a supplemental package due mid-Dec 2026 | P✓ for the dates and for "technology based on neural networks and artificial intelligence will be considered". That the next video codec will then face the reproducibility question is **the record's inference** ⚠ |
|
||||
| 5 Nov 2026 | Spain RGE2 → DVB-T2 + HEVC UHD | **R** ⚠ |
|
||||
| 12–13 Nov 2026 | JPEG Trust Summit, London | P✓ (JPEG 111 release, §2.2) |
|
||||
| Dec 2026 | MOQT and C4M to IESG (WG milestone) | P✓ |
|
||||
| 13 Jan 2027 | MPEG-AI Part 6 CfP responses due, evaluated at MPEG 157 | [`mpeg-ai-part6-mapping.md`](mpeg-ai-part6-mapping.md); SC 29 membership. P✓ |
|
||||
| 18–22 Jan 2027 | MPEG 157, Brisbane; beyond-VVC evaluation at the JVET meeting there | P✓ |
|
||||
| Mar 2027 | LOC, the streaming format and Privacy Pass authentication to IESG (WG milestone) | P✓ |
|
||||
| 19–23 Apr 2027 | MPEG 158, Geneva | Part 6 CD target per `mpeg-ai-part6-mapping.md` (tree). Dates P✓ |
|
||||
| 15 Aug 2027 | EU data-centre label generated from this date (C(2026) 3472 Art. 3) | P✓ (§5.2) |
|
||||
| 2027 | Gaussian Splat Coding codec selection | **R** ⚠ |
|
||||
| 31 Dec 2028 | EU review: ICT-equipment energy indicators; (partial) certification or auditing of reported data (C(2026) 3472 Art. 6) | P✓ (§5.2) |
|
||||
| end of 2029 | beyond-VVC first version targeted | P✓ |
|
||||
| Oct 2029 | Gaussian Splat Coding standard targeted | **R** ⚠ |
|
||||
| 2030 | Brazil TV 3.0 in 15 capitals | **R** ⚠ |
|
||||
|
||||
---
|
||||
|
||||
## 10. What would falsify this
|
||||
|
||||
Each item names an observation that would invalidate a specific conclusion
|
||||
above, so the record can be checked rather than believed. Every "none found"
|
||||
conclusion is also exposed to the unreached areas in §1.
|
||||
|
||||
- **F1. "No standard or product binds measured energy to a media outcome in a
|
||||
way a third party can verify" (§5.1).** Falsified if any of the following
|
||||
signs a *measured* figure bound to a reproducible output that a party who
|
||||
was not present can check:
|
||||
- the 23001-11 4th edition at DIS or later (the CD text was searched and does
|
||||
not, §6.1);
|
||||
- an MPEG-AI Part 6 working draft;
|
||||
- a GREEN WG document. The current revisions of the four WG drafts were
|
||||
checked and do not (§6.1); re-check at each revision;
|
||||
- a revision of 3GPP TR 26.942;
|
||||
- a DVB CM-EE output.
|
||||
|
||||
If F1 fires through 23001-11, Part 6 or a GREEN WG document, the 2030
|
||||
falsifier #1 fires with it.
|
||||
- **F2. "No jurisdiction mandates per-stream, per-view or per-decode energy"
|
||||
(§5.2).** Falsified by such a rule in any jurisdiction, and Nordic and Dutch
|
||||
sources were not reached. It is also falsified if the ESPR displays revision
|
||||
or the EU 2028 review adds per-stream reporting.
|
||||
- **F3. "No WAI receipt lets a verifier tell a platform-relayed counter from an
|
||||
independently rooted instrument" (G4, G6).** This holds on `main`, for all four
|
||||
WAI receipt forms: the energy binding and the video receipt put both kinds of
|
||||
source under `HwShunt`, the JWP receipt carries no class, and the OER
|
||||
receipt's `measured` grade covers "an instrument, a datasheet, or a published
|
||||
measurement" while its `provenance.device` and `energy.method` members are free
|
||||
text. In the tree at link 10 (link 08, PR #109), `OnChipCounter` and
|
||||
`CalibratedInstrument` are signed classes that make exactly this distinction,
|
||||
so F3 no longer describes that tree for any receipt that carries one of them,
|
||||
including an `energy` claim for a JWP object (link 10, PR #112). It still
|
||||
describes a JWP group total, an OER/2 receipt, and any figure in an optional-label
|
||||
family sealed without its label. The distinction is the producer's signed
|
||||
declaration: a verifier gates on the class and its evidence, and can check a
|
||||
calibration reference against its certificate, but the class is not an
|
||||
attestation by the instrument (energy-measurement §2.5). Across the monorepo the
|
||||
distinction also exists in `sandbox-tee`'s `CounterClass::is_independently_rooted()`
|
||||
and `AttestTrust`, with `efficiency/1`'s `energy_attestation`. The family's
|
||||
receipt-level ladder in JCR-1 §2.3 does not make it (§5.1), and the family does
|
||||
not agree on where IOReport sits in it (§5.3).
|
||||
- **F4. "The codec registry lacks AV2, VVC, HEVC, LCEVC, MPEG-H, AVS3 and Audio
|
||||
Vivid" (G5).** This holds on `main`: no capability string in
|
||||
`wai-rs/src/codecs/*.rs` admits these payloads, and the glossary survey entries
|
||||
register none. At link 04 (PR #105) the registry registers all of them, so F4
|
||||
does not describe the tree at link 10.
|
||||
- **F5. "Parameter transmission reaches a reproducible entropy stage but not a
|
||||
reproducible reconstruction" (§4, G7).** Falsified if arXiv 2606.28027's
|
||||
reconstructions turn out bit-identical across the tested NPUs. The boundary
|
||||
`determinism-tiers.md` §2 draws between `entropy-consistency` and
|
||||
`decode-equivalence` would then be misplaced for that approach. arXiv
|
||||
2608.00483 does not bear on this: it synchronises the entropy stage by mixed
|
||||
precision and claims no identical reconstruction. At link 10 (link 02,
|
||||
PR #103) the consequence for `replicate` is settled either way: a decode that
|
||||
is not integer / fixed-point end to end does not back it, whatever its
|
||||
reconstructions do on the devices tested.
|
||||
- **F6. "SyfAI's AI update-framework thread covers the properties WAI's `Prior`
|
||||
and determinism tiers specify" (§2.1, G3).** This rests on a **secondary**
|
||||
report. It is falsified or narrowed if either of the following scopes bit
|
||||
exactness out, or defines it as reference-software reproducibility only: the
|
||||
MPEG 155 Exploration 52 document, or the joint WG 3 / WG 7 output on the
|
||||
update framework. The part page lists the MPEG 155 document without a date,
|
||||
and it was not read. This is the most load-bearing unverified item in this
|
||||
record, and the one to read first before MPEG 156.
|
||||
- **F7. "No neural codec in production at large platforms" and "no top consumer
|
||||
platform delivers over MoQ" (§4).** Falsified by a disclosed deployment. The
|
||||
hardware and NPU area, where the first would most likely appear, was not
|
||||
reached.
|
||||
- **F8. "VVC ships at scale only under national mandate" (§4).** Falsified by a
|
||||
disclosed large non-mandated VVC deployment. The supporting survey is
|
||||
vendor-run with no stated sample size, so this conclusion is weak to begin
|
||||
with.
|
||||
- **F9. "DVB has written energy requirements and no CM-EE output was found"
|
||||
(§2.4, §5.1).** Falsified if CM-EE has published output since the DVB chair's
|
||||
September 2025 article, which is a year old and describes "a working group on
|
||||
energy efficiency" without naming CM-EE. That would also change the DVB item
|
||||
of [`energy-exposure-mapping.md`](energy-exposure-mapping.md) §8.
|
||||
- **F10. Mis-staging.** The 2030 landscape named mis-staging and
|
||||
misattribution among the dominant failure modes of its refuted candidates
|
||||
(its §9, item 7). For that reason every P row but one was re-read before
|
||||
publication (§1.1). Many **R**-graded stage claims (§2) remain unverified.
|
||||
Re-verify stage and issuing body before quoting any row outside the tree, and
|
||||
re-check the tree before quoting a §5.3, §6 or §7 row: the chain's pull
|
||||
requests may change before they merge.
|
||||
- **F11. "JPEG AI's normative conformance point is the entropy-decoder output;
|
||||
reconstruction may diverge" (§2.2, §6).** JPEG AI's core experiments on
|
||||
bit-exact reconstruction are continuing (P✓). If a JPEG AI edition or
|
||||
amendment adds a bit-exact reconstruction conformance point, this finding is
|
||||
false for that edition onward. In the tree at link 10, determinism-tiers §4
|
||||
and SPEC §7 would then no longer describe JPEG AI's own guarantee as
|
||||
`entropy-consistency`, and the `NeuralEntropyExact` class of
|
||||
`wai.image.jpegai`, which SPEC §7 calls a conservative choice pending a
|
||||
reading of Part 4, would need revisiting. Check at each JPEG meeting.
|
||||
612
wai/research/energy-exposure-mapping.md
Normal file
612
wai/research/energy-exposure-mapping.md
Normal file
|
|
@ -0,0 +1,612 @@
|
|||
# Media energy exposure: where WAI's per-object energy figure sits
|
||||
|
||||
> A preparation document and not a submission. It maps WAI's energy measurement
|
||||
> and binding onto the four bodies now specifying media energy: DVB, 3GPP SA4,
|
||||
> MPEG (ISO/IEC 23001-11) and the IETF. It also notes one regulatory review
|
||||
> window. **Inputs to DVB require DVB membership. Inputs to 3GPP SA4 require
|
||||
> membership of a 3GPP Organisational Partner. Comments on an ISO/IEC CD go
|
||||
> through an SC 29 national body. WAI holds none of these today.** The IETF is
|
||||
> the one venue here open to any participant without membership.
|
||||
>
|
||||
> This note cross-references rather than repeats. For the table of existing
|
||||
> instruments and the reasoning that predictive metadata is not prior art against
|
||||
> a measured binding, see [2030-transport-landscape](2030-transport-landscape.md)
|
||||
> §5. For the GREEN accuracy identities, see §3.5 there. The §1 table below
|
||||
> updates four of the landscape's rows, because their stages have moved. The
|
||||
> mechanisms mapped are [energy-measurement](../extensions/energy-measurement.md),
|
||||
> [energy-binding](../extensions/energy-binding.md),
|
||||
> [jwp-receipts](../extensions/jwp-receipts.md) and
|
||||
> [moq-streaming-format](../extensions/moq-streaming-format.md) §5. The companion
|
||||
> record [2026-09-global-sweep](2026-09-global-sweep.md) grades the wider field.
|
||||
>
|
||||
> **Which tree.** WAI's side is described as it stands in the tree at link 10 of
|
||||
> the landing chain: a linear chain of pull requests on the forge, each
|
||||
> targeting `main`, merging in order. **As of 2026-09-30 none is merged into
|
||||
> `main`.** The links this note relies on are link 01 (`feat/wai-video-workflow`,
|
||||
> PR #82), link 02 (`wai/land-02-determinism-tiers`, PR #103), link 08
|
||||
> (`wai/land-08-energy-classes`, PR #109) and link 10
|
||||
> (`wai/land-10-receipt-classes`, PR #112); each link contains the ones before
|
||||
> it. Everything attributed to a link is proposed in that link's pull request and
|
||||
> is not in `main`. Where `main` differs, the text says so.
|
||||
>
|
||||
> **⚠ unverified** marks anything not seen in a primary source.
|
||||
|
||||
---
|
||||
|
||||
## 1. The bodies and what each one carries
|
||||
|
||||
| body / instrument | stage (verified) | granularity | measured or predicted | signed |
|
||||
|---|---|---|---|---|
|
||||
| **DVB BlueBook S100**, *Study Mission report on Energy Aware service Delivery and Consumption* | study report, Nov 2023. A CM-EE (Commercial Module) subgroup was announced on 17 Nov 2023 ([2026-09-global-sweep](2026-09-global-sweep.md) §2.4). On 10 Sep 2025 the DVB chair wrote of "a working group on energy efficiency" launched in 2024 that "so far we have struggled to bring this group and topic to life", and invited participation; the article does not name CM-EE. | use cases and draft requirements; no specification | — | — |
|
||||
| **3GPP TR 26.942**, *Study on Media Energy Consumption Exposure and Evaluation Framework* | V20.0.0 (2026-06, Rel-20), under change control; V19.0.0 (Rel-19). Phase-1 study FS_MediaEnergyGREEN (UID 1030004); phase-2 WI 1080050. | per UE, slice, DNN, PDU session, Service Data Flow; Application Server reports add application session, configuration and service location | network-function energy in joules from the 5G Core Energy Information Function (EIF); UE side left non-normative | no integrity or attestation mechanism in the text |
|
||||
| **ISO/IEC 23001-11 4th edition**, *Energy-efficient media consumption (green metadata)* | CD text, WG 3 N1864, dated 2026-08-28, listed on mpeg.org 2026-09-21; CD ballot. The MPEG 153 output was a WD (mpeg.org lists it 2026-03-02), so landscape §5.1's "4th ed. at CD (WG 3 doc N01742)" is superseded. | per picture / per DASH Segment / per representation | predicted: complexity metrics, estimated decoding-operation ratios, display power indications | no |
|
||||
| **ISO/IEC 23001-19**, carriage of green metadata | **IS**: advanced to publication at MPEG 153 (landscape §5.1; the MPEG 153 meeting page, read for [2026-09-global-sweep](2026-09-global-sweep.md) §2.1). The mpeg.org part page's newest document, dated 2025-10-24, is titled *Potential improvement of ISO/IEC DIS 23001-19* — the DIS stage before that promotion — so the page and the meeting record are consistent. The N1864 CD text refers to it as "ISO/IEC 23001-19:2025". | ISOBMFF track / DASH adaptation set | carries 23001-11 | no |
|
||||
| **draft-ietf-avtcore-rtcp-green-metadata-17** | 2026-09-09, **RFC Editor queue**, Proposed Standard. This supersedes landscape §5.1's "IESG evaluation at -16". | RTP session | request and notification of resolution/frame rate | no integrity mechanism beyond SRTP/SAVPF |
|
||||
| **draft-ietf-green-power-and-energy-yang-04** | 2026-09-09, WG document (the landscape cites -03) | device and sub-component | measured or estimated, with graded accuracy identities | no (an individual provenance augmentation exists, `draft-madpr-green-provenance-00`) |
|
||||
| **WAI**: energy-measurement, energy-binding, JWP receipt profiles | Draft extensions in this tree | **per object** (one decoded clip, image or envelope) | **as specified at link 10** (link 08, PR #109): a figure of one acquisition class — read from a calibrated external instrument (`CalibratedInstrument`) or from a counter the measured silicon keeps (`OnChipCounter`), each with a declared uncertainty, or model-based, or a coarse estimate; legacy `HwShunt` is accepted and never emitted (energy-measurement §2) — published only with a clean bracket and its quality ratio (§4, §6 there). **As carried at link 10:** every `wai-rs` and `wai-quantum` receipt family can sign the class, with the declared uncertainty for a hardware class; no receipt carries the quality ratio or the other bracket values (§7, L1, L9). On `main`, only the video receipt and the C2PA energy binding sign a class. | **per profile.** At link 10 the class is always signed in the energy binding, the video receipts and an `energy` claim (link 10, PR #112), and is an optional signed label in the older receipt families. The JWP group receipt signs a total with no class; the JWP object receipt's figure is outside its signature; the MoQ `WAI_RECEIPT` property has no signature of its own (§7, L1). |
|
||||
|
||||
---
|
||||
|
||||
## 2. Against DVB S100's four use cases
|
||||
|
||||
Quoted requirement text is from S100 §7, read in full. S100 records "None in
|
||||
DVB" as existing coverage for use cases 1 and 3. For use case 2 it notes
|
||||
reporting APIs that were not designed for energy, and for use case 4 it lists
|
||||
existing codec, multicast-ABR and native-IP specifications.
|
||||
|
||||
**Use case 1: Common energy consumption measurement.** Draft requirement: *"a
|
||||
common reporting framework on energy consumption allowing to provide metrics from
|
||||
all components of the delivery chain and used by all industry players."* The
|
||||
service-provider view adds that third-party suppliers should use the same
|
||||
framework.
|
||||
|
||||
- *What WAI offers:* a reporting contract for one component, the decode (or
|
||||
production) of an object. A published figure must carry its acquisition class
|
||||
with the class's evidence, the idle mean and SD, the busy mean and SD, sample
|
||||
counts, the window, the completion count, the thread count and how the window
|
||||
opened, the attribution-quality ratio, the build profile and a recomputable
|
||||
work unit (energy-measurement §6). A figure below the publication bar may not
|
||||
be published as an absolute value, and a failed bracket is emitted as
|
||||
unmetered (§4 there). That is the record format a framework could carry for a
|
||||
decode component, including one run by a third party. In the tree at link 10
|
||||
the class and, for a hardware class, its declared uncertainty are signed in
|
||||
every `wai-rs` and `wai-quantum` receipt family that carries them (§7, L1); the
|
||||
bracket values themselves are in no receipt (L9). A signature binds the
|
||||
reporting party to the per-object figure and its class. It does not show that
|
||||
an instrument produced them (L2).
|
||||
- *What it is not:* a framework for the delivery chain. WAI measures nothing
|
||||
on the network, CDN, display or device idle, so it cannot answer the
|
||||
end-to-end question this use case asks.
|
||||
|
||||
**Use case 2: Energy consumption exposure.** Draft requirement: *"open APIs
|
||||
allowing to report energy consumption of each component in the delivery chain."*
|
||||
|
||||
- *What WAI offers:* a payload such an API could return, plus an in-band
|
||||
carrier for MoQ delivery. The per-object `WAI_RECEIPT` property
|
||||
(moq-streaming-format §5) carries `joules_micro`, `acquisition` and, at
|
||||
link 08, `acquisition_evidence`. None of these is signed (§7, L1). At link 10
|
||||
an `energy` claim can carry a signed copy of an object's figure and class
|
||||
(jwp-receipts §2.4.5).
|
||||
- *What it is not:* an API. SPEC.md §1 scopes WAI as a container and
|
||||
capability-dispatch standard, and nothing in the tree specifies a service.
|
||||
|
||||
**Use case 3: Energy as a criterion for source or quality selection.** Draft
|
||||
requirements: *"indicate energy efficiency of a media instance in DVB-I"* and
|
||||
*"select a video representation in DASH according to its energy efficiency like
|
||||
it is already done according to its bitrate."*
|
||||
|
||||
- *Already answered, predictively, by MPEG.* The 23001-11 4th-edition CD,
|
||||
clause 8 *Energy-efficient media selection*, defines decoder-power indication
|
||||
metadata per DASH representation: `dec_ops_reduction_ratio_from_max` (u(8))
|
||||
and `dec_ops_reduction_ratio_from_prev` (s(16)), computed from the
|
||||
"**estimated** number of decoding operations". It also defines
|
||||
display-power indication metadata. Both are carried per 23001-19. A DVB
|
||||
input on this use case should reference that clause, not restate it.
|
||||
- *What WAI's text adds:* the check on that prediction. energy-binding §7
|
||||
specifies sealing `op_requested`, `op_committed` and `op_delivered`, and
|
||||
requires the **residual** between a declared complexity descriptor and the
|
||||
measured energy of the same reconstruction. A representation chosen because
|
||||
it predicted fewer operations could then be scored against what it cost.
|
||||
This exists only as text: no receipt or binding in `wai-rs` or `wai-quantum`
|
||||
has these fields (§7, L10).
|
||||
- *What it lacks:* a binding to DASH or DVB-I. WAI's multi-rendition envelope
|
||||
(SPEC.md §7.1) could carry a measured figure per rendition, but no text says
|
||||
so.
|
||||
|
||||
**Use case 4: Modernisation.** This is not WAI's layer. The only link is that a
|
||||
per-work-unit measured figure is how an efficiency claim for a new technology
|
||||
gets evaluated. That is 3GPP's Key Issue #3 below.
|
||||
|
||||
---
|
||||
|
||||
## 3. Against 3GPP TR 26.942 (V20.0.0)
|
||||
|
||||
Read from the V20.0.0 text. The TR's conclusions (§8) and recommendations (§9)
|
||||
are what an input would respond to.
|
||||
|
||||
**Key Issue #1: Energy-related information exposure.** The TR recommends new
|
||||
stage 2 and stage 3 specifications for an Energy Information AF. Network energy
|
||||
is reported in joules in an `energyInfo` object (TS 29.122 clause
|
||||
5.3.2.3.20), at SDF / PDU-session / UE granularity (§7.14.5.3, §9.2.2).
|
||||
*WAI:* nothing at the network layer. Its figure sits one level below, as a
|
||||
per-object figure beside a per-flow aggregate.
|
||||
|
||||
**Key Issue #2: Monitoring and measurement.** The TR concludes that "because the
|
||||
metrics differ from one UE OS to another, there will not be normative work on UE
|
||||
energy-related information collection and metrics" (§8.3). For server-side
|
||||
interfaces (IPMI/DCMI, Redfish) it notes that "no formal end-to-end
|
||||
certification exists for the accuracy of these power metrics across vendors".
|
||||
Such data can be used "for operational estimates, comparative data … but not
|
||||
for the purpose of customer billing or legally binding reporting to regulators"
|
||||
(§4.2.9).
|
||||
|
||||
- *What WAI's text offers:* a way to *qualify* a figure without normalising
|
||||
the metric. The acquisition class says what kind of source produced it, and
|
||||
in the tree at link 10 it separates an on-chip counter, which carries no
|
||||
traceable uncertainty — the TR's point about server power metrics — from a
|
||||
calibrated external instrument (energy-measurement §2.2). The declared
|
||||
uncertainty says how far the figure can be leaned on (§2.3 there); the quality
|
||||
ratio says whether the bracket was clean enough to publish; and a failed
|
||||
measurement is emitted as unmetered rather than as a low number (§4 there).
|
||||
At link 10 the reference meters seal a hardware class only when the bracket
|
||||
passes §4's bar and the uncertainty can be declared, and seal unmetered
|
||||
otherwise (energy-measurement §2.8; the meters' module documentation). No
|
||||
receipt carries the quality ratio itself, which the measuring tool prints
|
||||
(L9). On `main` only two receipt profiles sign the class.
|
||||
- *Honest limit:* WAI's reference protocol is a **lab bracket**. It runs a
|
||||
saturating multi-threaded burst on an otherwise idle machine (§3 there),
|
||||
which is not an in-session measurement on a user's device. Qualifying a
|
||||
number is not the same as measuring the one KI #2 asks for.
|
||||
|
||||
**Key Issue #3: Evaluation framework.** The TR concludes that "no methodology has
|
||||
yet been identified allowing evaluation of the energy usage/savings of
|
||||
multimedia standards features and proposals". It makes "no specific
|
||||
recommendation at this time to initiate normative work" (§8.4). It frames the
|
||||
problem as characterising efficiency "in a repeatable and technology-agnostic
|
||||
way".
|
||||
|
||||
- *What WAI offers:* a candidate element for decode. The work unit is
|
||||
recomputable from the codestream. For WAI's integer capabilities the
|
||||
reconstruction is built to be byte-exact across architectures. That is checked
|
||||
only on well-formed streams, for the capabilities the CI goldens cover, on the
|
||||
CI legs that ran. On `main`, since about 2026-08-28, those legs are linux
|
||||
x86-64, linux arm64 and macOS arm64, where every job that finished passed; the
|
||||
`windows` leg has not run, because its runner host is offline
|
||||
([syfai-input](syfai-input.md) §2, G8; read 2026-09-27). Within those bounds,
|
||||
a third party can recompute what describes the output and the work: the
|
||||
reconstruction, its hash and the work unit. The joules rest on the reporter,
|
||||
and so does every field that says how they were measured: the acquisition
|
||||
class and its evidence, the idle and busy figures, sample counts, window,
|
||||
quality ratio and build profile (L8). A comparison between codec proposals is
|
||||
therefore repeatable in its outputs and work, and only as good as each
|
||||
reporter's bracket in its energy. Whether the figure and its class are inside
|
||||
a signature depends on the profile, and a signature shows who reported them,
|
||||
not that an instrument produced them (§7, L1, L2).
|
||||
- *Honest limit:* this holds only for WAI's byte-exact capabilities. It covers
|
||||
neither the network nor any float decoder.
|
||||
|
||||
**Solution #13: In-band signalling of energy to the player.** Energy reaches the
|
||||
Media Player through CMSD custom keys in HTTP response headers, and the TR notes
|
||||
it "could alternatively be achieved by embedding this information directly
|
||||
within the streaming presentation manifest" (§7.14.2). §9.2.3 says Solution
|
||||
#13's requirements "could be shared with the broader industry as formal
|
||||
requirements and serve as an invitation for collaboration", with metrics and
|
||||
in-band mechanisms from other organisations "for further study".
|
||||
|
||||
- *What WAI offers:* WAI's per-object MSF property is an in-band mechanism for a
|
||||
different transport (MoQ). This is the most direct point of contact in the
|
||||
TR, with a limit that has to be stated. The object's `content_hash` is
|
||||
committed under the group's signed Merkle root, and the property's
|
||||
`group_receipt` identifies that signed group. Its `joules_micro`,
|
||||
`acquisition` and `acquisition_evidence` fields are **not** signed: the group
|
||||
signature covers only the group's joule total (§7, L1). So the property is an
|
||||
in-band carrier for a per-object figure, not a signed one. In the tree at
|
||||
link 10 a player can check an object's figure and class only through an
|
||||
`energy` claim for that object (jwp-receipts §2.4.5), which the property
|
||||
neither carries nor references. An input could offer the property as a way
|
||||
for a player to check the figure only once it is signed or bound to such a
|
||||
claim.
|
||||
- *Where the contact ends:* CMSD headers and 5GMS are not WAI's layer, and WAI
|
||||
would not bind them.
|
||||
|
||||
**What the TR does not address.** A term search of the V20.0.0 text finds no
|
||||
occurrence of *signature*, *attest*, *integrity* or *tamper*. The TR specifies
|
||||
who reports energy, at what granularity and over which reference points. It
|
||||
does not specify how a recipient checks a reported figure. That question is
|
||||
open, and WAI's energy-binding §3 answers part of it: which output a figure
|
||||
describes, not whether the figure is accurate. That answer needs the figure
|
||||
inside a signature, and energy-measurement §2 needs the class there too. In the
|
||||
tree at link 10 that holds for every `wai-rs` and `wai-quantum` receipt that
|
||||
carries its class and for an `energy` claim; it does not hold for a JWP group
|
||||
total or the MoQ property (§7, L1). On `main` only the C2PA energy binding and
|
||||
the video receipt meet both.
|
||||
|
||||
---
|
||||
|
||||
## 4. ISO/IEC 23001-11 4th edition: complementary, and the falsifier check
|
||||
|
||||
**Landscape falsifier #1 is not triggered by the CD text.** The CD (WG 3 N1864)
|
||||
was searched for measured or signed energy. The terms *signature*, *hash*,
|
||||
*attest*, *certif*, *joule*, *watt*, *measured* and *measurement* do not
|
||||
occur. The metadata remains predictive: complexity metrics, estimated
|
||||
decoding-operation ratios, attenuation maps and quality levels. Recheck at DIS.
|
||||
|
||||
**Degradation is an explicit option in the standard.** The CD introduction says
|
||||
the metadata works "without any degradation in the quality of experience", and
|
||||
also that "it is also possible to use this metadata to get larger energy
|
||||
savings, but at the expense of some QoE degradation". WAI's rules cover both
|
||||
routes by which that happens (determinism-tiers §2, *What a tier covers*, at
|
||||
link 02, PR #103). A sink that alters the decoded medium, as attenuation maps
|
||||
and display adaptation do, may not claim a tier for what it emits. A tier claim,
|
||||
decoded-medium hash or energy figure over a payload that an encoder altered to
|
||||
save energy, at a receiver's request or on its own account, covers that payload
|
||||
and not the one it stands in for.
|
||||
|
||||
**A vocabulary note for reconciliation.** 23001-11 itself defines a
|
||||
request/response pair for point-to-point sessions, *decoding operation reduction
|
||||
request (DOR-Req) and response (DOR-Resp)*. In the tree at link 10,
|
||||
energy-binding §7.1–§7.2 (link 02) cite that pair as the 23001-11 case of the
|
||||
request/commitment pattern: the receiving device requests, and the remote
|
||||
encoder answers with how it decided to answer. Landscape §5.3 still attributes
|
||||
WAI's request/commit pattern to the RTCP TSRR/TSRN messages; DOR-Req/DOR-Resp is
|
||||
the earlier and closer precedent. `op_delivered` and the measured residual are
|
||||
what WAI's text adds. No receipt or binding carries any of the three fields yet
|
||||
(L10).
|
||||
|
||||
**One editorial observation on the CD text,** to raise only through a national
|
||||
body. The foreword says the 4th edition "cancels and replaces the third edition
|
||||
(ISO/IEC 23001-10:2025)", which appears to misstate the part number and year.
|
||||
|
||||
---
|
||||
|
||||
## 5. IETF GREEN
|
||||
|
||||
This is covered in landscape §3.5 and energy-measurement §2.1 and is not repeated
|
||||
here. Since the landscape was written:
|
||||
|
||||
- the YANG module has moved to `-04` (2026-09-09);
|
||||
- a power-state capability-discovery draft has appeared
|
||||
(`draft-claise-green-capability-discovery-02`, 2026-09-23).
|
||||
|
||||
The scope remains device and sub-component energy. It is the cheapest venue for
|
||||
the acquisition-identity derivation the landscape queues as item 8, because the
|
||||
IETF requires no membership. In the tree at link 10, energy-measurement §2.1
|
||||
(link 08) fixes the rules such a derivation would follow (§8).
|
||||
|
||||
---
|
||||
|
||||
## 6. EU data-centre rating scheme, C(2026) 3472
|
||||
|
||||
Commission Delegated Regulation C(2026) 3472 final, dated 21 Sep 2026, amends
|
||||
Delegated Regulation (EU) 2024/1364. Read from the Commission text:
|
||||
|
||||
- An electronic label is generated by the European database "by 15 August 2027
|
||||
and every year thereafter". Data centres below 500 kW installed IT power
|
||||
"may voluntarily participate".
|
||||
- The amendments "simplify it by removing some indicators (such as the data
|
||||
traffic indicators)".
|
||||
- Article 6 requires a review "by 31 December 2028 and every three years
|
||||
thereafter". The review includes the possibility to "(d) introduce indicators
|
||||
to measure the performance of the ICT equipment energy use" and "(e)
|
||||
introduce (partial) certification or auditing of the reported data".
|
||||
|
||||
Bearing: data-centre scope, not per object. The review asks, at facility scale,
|
||||
the question WAI's acquisition class asks at object scale: how can a reported
|
||||
figure be relied on? The recitals and articles read here (part 1) contain no
|
||||
per-stream, per-view or per-decode energy requirement. The annexes were not read
|
||||
for this note. The companion record read them
|
||||
([2026-09-global-sweep](2026-09-global-sweep.md) §5.2): Annex III adds a point
|
||||
(f), *Certification of the reported data*, to Annex I of Regulation 2024/1364,
|
||||
with self-certification and third-party certification kept distinct, and that
|
||||
record finds no per-stream, per-view or per-decode requirement in the act. ⚠
|
||||
Whether the act has completed scrutiny and been published in the Official
|
||||
Journal was not checked. It enters into force twenty days after publication.
|
||||
|
||||
---
|
||||
|
||||
## 7. Where WAI fits, and what it still lacks
|
||||
|
||||
**The fit, stated once.** Session and network aggregates answer *how much did
|
||||
this service, session or flow use?* That is 3GPP's EIF and AF and DVB's
|
||||
framework. Predictive metadata answers *how much should this cost?* That is
|
||||
23001-11. WAI's binding answers a third question: *what did producing or
|
||||
decoding this object cost, measured how, over which reproducible bytes?* The
|
||||
three are complementary. WAI's figure describes an output and a work unit that a
|
||||
third party can recompute. The joules, and how they were measured, rest on the
|
||||
reporter (L8).
|
||||
|
||||
**What WAI lacks.** Each item was checked against the extension text and the
|
||||
reference code in the tree at link 10; the state on `main` is given where it
|
||||
differs.
|
||||
|
||||
**L1: Which receipt profiles meet energy-measurement §2.** §2 requires a receipt
|
||||
it governs that carries a non-zero figure to carry an acquisition class, inside
|
||||
the signature, with the class's evidence, and a verifier to treat a receipt that
|
||||
breaks either coupling as invalid.
|
||||
|
||||
*On `main`* the profiles fall into three groups. They were sorted by reading
|
||||
every `signing_payload` in `wai-rs/src` and `wai-quantum/src`, and by searching
|
||||
both trees for a class field:
|
||||
|
||||
- **Figure and class both signed:** the video receipt
|
||||
(`wai-rs/src/video_receipt.rs`) and the C2PA energy binding.
|
||||
- **Figure signed, no class anywhere:** in `wai-rs`, the linear-film,
|
||||
ternary-tt, provenance, spatial-audio, volumetric, interactive-worlds (reused
|
||||
by avatars and state-feeds), haptics and score receipts; in `wai-quantum`, the
|
||||
quantum-sim receipt, the quantum-ops receipts, the phasor-meter receipt and
|
||||
the QBOM manifest.
|
||||
- **Per-object figure not signed:** the generic JWP receipt and the in-band
|
||||
`WAI_RECEIPT` MSF property.
|
||||
|
||||
*In the tree at link 10* (link 08, PR #109; link 10, PR #112):
|
||||
|
||||
- **Class always present and signed:** the energy binding and its C2PA companion
|
||||
assertion, the `wai.video.int_motion` receipt, the `wai.video.*` OTT receipts
|
||||
of link 01, and an `energy` claim (jwp-receipts §2.4.5). For the two hardware
|
||||
classes, the declared uncertainty and the filtering state or calibration
|
||||
reference are signed with the class.
|
||||
- **Class as an optional signed label** (energy-measurement §2.8): the
|
||||
world-session, film and reel, haptic, score, 4D-splat, audio-scene, ternary
|
||||
and provenance-step receipts in `wai-rs`, and every `wai-quantum` receipt
|
||||
(circuit, calibration, job, mitigation, decode, compile, rearrange, phasor,
|
||||
and the bill of materials per entry). A label changes the signing domain's
|
||||
version byte, so it can be neither added to nor stripped from a signed
|
||||
receipt. Without one, a receipt keeps its legacy bytes and its figure is
|
||||
*unlabelled*: accepted, and not read as any class. Labelling a new figure is a
|
||||
SHOULD that no verifier can check. The reference meters label every non-zero
|
||||
figure they seal; `world_model::frame_receipt` and the `wai-web` demos seal
|
||||
unlabelled figures, and no constructor in these families requires the class
|
||||
(§2.8 there).
|
||||
- **Per-object figure not signed:** the generic JWP group receipt's signing
|
||||
payload covers `total_joules_micro`, the group sum (jwp-receipts §2.3), and
|
||||
the Merkle leaf is `BLAKE3(domain ‖ content_hash)` (§2.2), so an object
|
||||
receipt's `joules_micro` and `origin` sit outside the signature, and the
|
||||
object receipt has no member for the class. An `energy` claim restates the
|
||||
object's figure beside its class under the claim's own signature, and a
|
||||
verifier holding both rejects the pair if the figures differ. Nothing carries
|
||||
a class for a group total. The `WAI_RECEIPT` property carries `content_hash`,
|
||||
`joules_micro`, `acquisition`, `acquisition_evidence` and `group_receipt` (a
|
||||
BLAKE3 of the group receipt), and has no signature of its own
|
||||
(moq-streaming-format §5).
|
||||
|
||||
energy-measurement §1 records the JWP group and object receipts and the
|
||||
`WAI_RECEIPT` property as falling short of §2, and places OER/2 receipts
|
||||
outside its scope: an OER receipt's members are fixed and include no class
|
||||
(oer-receipts §2–§3). energy-measurement §2 says the reference implementation
|
||||
enforces its rules in `verify()`; at link 10 that holds for the always-present
|
||||
profiles and for any label that is present, and §2.8 says the optional-label
|
||||
families do not hold producers to labelling.
|
||||
|
||||
So "signed energy with its acquisition class" may be said, at link 10, of the
|
||||
always-present profiles, of an `energy` claim, and of a labelled receipt in an
|
||||
optional-label family. It may not be said of an unlabelled receipt, of a JWP
|
||||
group total, or of the MoQ property. On `main` it may be said only of the video
|
||||
receipt and the C2PA energy binding. What remains is a signed place for a group
|
||||
total's class, a member of the object receipt's own (or `joules_micro` and the
|
||||
class bound into the leaf), and a signature over the property.
|
||||
|
||||
**L2: Calibrated instruments, on-chip counters, and who signs.** On `main`,
|
||||
`HwShunt` covers "RAPL, NVML, SMC, IOReport, or an external shunt"
|
||||
(energy-measurement §2). On-die counters carry no accuracy certification; the TR
|
||||
26.942 passage in §3 says the same of server power metrics. A reading from a
|
||||
calibrated instrument is a different claim. In the tree at link 10 (link 08),
|
||||
energy-measurement §2 makes them two classes: `CalibratedInstrument`, which
|
||||
requires a calibration reference that resolves to a certificate (§2.5), and
|
||||
`OnChipCounter`, which also covers any external meter whose uncertainty is not
|
||||
traceable to a current calibration (§2.2). `HwShunt` is legacy (§2.6). §2.1
|
||||
forbids using a declared uncertainty to select a GREEN accuracy band, and
|
||||
declaring a band as an uncertainty.
|
||||
|
||||
A precedent for signing such readings is OCMF (Open Charge Metering Format), a
|
||||
data format for charging-station meter readings "that are relevant under
|
||||
calibration law". In OCMF the signature is made by "the signature-generating
|
||||
component, which is referred here as the gateway", and which the spec says
|
||||
could also be called the "measuring capsule" or "the signing component". That
|
||||
component may be attributable to the energy meter or be a separate gateway. The
|
||||
spec identifies it separately from the meter, and meter identification "is
|
||||
optional if the signature generating component has already been identified".
|
||||
So the signing key sits in a component of the metering installation, which need
|
||||
not be the meter itself. The spec does not say where the private key is stored.
|
||||
⚠ How that component is type-approved or certified under calibration law is not
|
||||
in the spec and was not verified. energy-measurement §2.2 at link 10 says OCMF
|
||||
values are accepted because they come from "a conformity-assessed, legally
|
||||
verified measuring system — the meter together with the component that signs its
|
||||
values", and that legal verification is not the same statement as a calibration
|
||||
certificate's uncertainty. That agrees with the OCMF text read here on where the
|
||||
signature sits.
|
||||
|
||||
WAI's signing key is the publisher's, and a publisher's signature says who
|
||||
reported a number, not that an instrument produced it. The class now separates
|
||||
the two kinds of evidence, and the calibration reference makes the calibration
|
||||
claim checkable, not true (§2.5 there). Still open: a rule on which key signs —
|
||||
the publisher's, or one held by the metering component — and a mapping onto
|
||||
GREEN's accuracy identities.
|
||||
|
||||
**L3: Declared tolerances.** energy-measurement §2.1 requires an implementation
|
||||
to state how far its class lets the figure be leaned on. On `main` no tolerance
|
||||
figure appears in the extensions or in `energy_binding.rs` / `video_receipt.rs`.
|
||||
In the tree at link 10 (link 08), `CalibratedInstrument` and `OnChipCounter`
|
||||
must carry a declared standard uncertainty with its integration window (§2.3),
|
||||
and every verifier refuses one that is missing, zero, or smaller than the
|
||||
rounding of the whole-microjoule figure. `ModelBased` and `Estimator` still carry
|
||||
none, and §2.1 says so.
|
||||
|
||||
**L4: No publishable absolute figure in WAI's own text.** The worked example in
|
||||
energy-measurement §7 is a deliberately failing bracket at a ratio of 3.8,
|
||||
below the publication bar of 10, because a clean bracket was not reproducible on
|
||||
the authoring machine. Its receipt says `HwShunt`; §7 there says the reference
|
||||
meter now seals `OnChipCounter` for the same interface, and that the run's
|
||||
missing values cannot be reconstructed. An input that cites WAI's method has no
|
||||
example figure it may quote as absolute.
|
||||
|
||||
**L5: Scope.** The figure covers one component on one machine, in a lab bracket.
|
||||
The network, CDN, display, device idle and in-session measurement are all
|
||||
outside it (see §2 and §3).
|
||||
|
||||
**L6: No API and no DASH, DVB-I or 5GMS binding.** The only in-band carriage is
|
||||
the MoQ property.
|
||||
|
||||
**L7: No key discovery** (landscape §3.6). Every signature presumes the verifier
|
||||
already holds the key, and that includes the signer of an `energy` claim. The
|
||||
normative text says so too: transparency §6 says it does "not solve key
|
||||
discovery". It points to "jwp-receipts §7" for it, but jwp-receipts has no §7;
|
||||
its sections end at §6.8. OCMF leaves this out of band too: public keys reach the
|
||||
verifier "by means other than this protocol (out-of-band) … e.g. via a central
|
||||
register". WAI names no register or other mechanism.
|
||||
|
||||
**L8: Not re-measurable, by construction.** A third party reproduces the output
|
||||
and the work unit, never the joules (energy-measurement §5). The same holds for
|
||||
every field that describes the measurement: the acquisition class and its
|
||||
evidence, the idle and busy figures, sample counts, window, quality ratio and
|
||||
build profile. This is a stated limit, not a defect, and any input must repeat it
|
||||
in that form. energy-measurement §5 ("every field except the energy is
|
||||
independently checkable") and energy-binding §3 ("exactly one unverifiable
|
||||
term") read as if the joules were the only such term. That is true only if the
|
||||
measurement fields are counted as part of the joule figure. An input should use
|
||||
the wider wording, and those two sections should be reworded to match.
|
||||
|
||||
**L9: The reporting contract has no receipt fields.** energy-measurement §6
|
||||
requires a published figure to be accompanied by the idle mean and SD, busy
|
||||
mean and SD, sample counts, window, completion count, thread count, how the
|
||||
window opened, quality ratio and build profile. No receipt profile in `wai-rs`
|
||||
or `wai-quantum` has fields for them. At link 10 a hardware class's declared
|
||||
uncertainty and window are sealed, but not the values they were computed from.
|
||||
The energy binding carries a `method` that names the protocol, not the values,
|
||||
and an `energy` claim's `method` is a URI naming the protocol. The reference
|
||||
meters print the idle and busy figures, the marginal-to-idle ratio and the
|
||||
resolution term to their console. So the record DVB use case 1 would need (§2)
|
||||
is a requirement in the text, not yet a format any receipt carries.
|
||||
|
||||
**L10: The operating-point fields exist only as text.** energy-binding §7.2 and
|
||||
§7.3 specify `op_requested`, `op_committed`, `op_delivered` and the residual
|
||||
between a declared complexity descriptor and the measured energy. No receipt or
|
||||
binding in `wai-rs` or `wai-quantum` has these fields (grep). `energy_binding.rs`
|
||||
carries the reconstruction hash, the microjoules, the acquisition class and its
|
||||
evidence, coverage, work units and `method`. §2 use case 3 and §4 describe what
|
||||
the text adds, not what a receipt carries today.
|
||||
|
||||
**Unverified near-prior-art.** Search snippets seen during this work pointed to
|
||||
public repositories describing signed receipts for AI inference with
|
||||
NVML-measured joules. Their bodies were not read, so they are neither named nor
|
||||
cited here. As the snippets described them, their scope is inference, not media.
|
||||
Read such material before writing any statement about what is new in signed
|
||||
energy receipts.
|
||||
|
||||
---
|
||||
|
||||
## 8. Candidate input text, per venue
|
||||
|
||||
Each item needs the membership named at the top, except the IETF item.
|
||||
|
||||
- **DVB CM-EE (commercial requirements).** A requirement derived from S100
|
||||
§7.1.3 and §7.2.3: *"It shall be possible to bind a reported energy figure to
|
||||
the media object or work unit it describes, to state how the figure was
|
||||
acquired, and to identify the party that reported it."* This is the property a
|
||||
framework "used by all industry players" needs when figures come from third
|
||||
parties.
|
||||
- **3GPP SA4 (TR 26.942 further study / Solution #13 follow-up).** For KI #3,
|
||||
decode-energy evaluation against a work unit recomputable from the codestream.
|
||||
For the in-band path, the missing question of how a Media Player checks a
|
||||
figure it receives.
|
||||
- **ISO/IEC 23001-11 CD ballot.** No technical comment is required on the
|
||||
predictive semantics, which WAI's measured binding accompanies. The editorial
|
||||
point in §4 could be raised through a national body.
|
||||
- **IETF GREEN (open participation).** Derive WAI's acquisition class from
|
||||
`data-source-accuracy`, in one direction only, as energy-measurement §2.1 at
|
||||
link 10 sets it out: a derivation never improves the class, a measured
|
||||
identity yields `OnChipCounter` unless a calibration reference is held, and a
|
||||
graded band is never declared as a figure's uncertainty, nor selected by one.
|
||||
The per-figure uncertainty is stated separately (L2, L3). This is landscape
|
||||
queue item 8.
|
||||
|
||||
The open parts of L1 should be fixed, and any input should state L8 in its wider
|
||||
form, before any of these goes out.
|
||||
|
||||
---
|
||||
|
||||
## 9. What this document does not claim
|
||||
|
||||
- It does not claim that WAI measures delivery-chain, network or device energy.
|
||||
It measures an object's production or decode on one machine.
|
||||
- It does not claim that a signature attests a figure's accuracy (energy-binding
|
||||
§3), or that an acquisition class is an instrument's attestation
|
||||
(energy-measurement §2.5).
|
||||
- It does not claim that every WAI receipt profile meets energy-measurement §2.
|
||||
In the tree at link 10 the always-present profiles, an `energy` claim and a
|
||||
labelled receipt do; an unlabelled receipt, a JWP group total and the MoQ
|
||||
`WAI_RECEIPT` property do not (L1). On `main` only the video receipt and the
|
||||
C2PA energy binding do.
|
||||
- It does not present anything attributed to a link of the landing chain as
|
||||
being in `main`. None of the chain is merged as of 2026-09-30.
|
||||
- It does not claim that a third party can check anything beyond the output and
|
||||
the work unit. The joules, and every field describing how they were
|
||||
measured, rest on the reporter (L8).
|
||||
- It does not claim that any receipt carries the quality ratio or the other
|
||||
bracket values (L9), or the operating-point fields of energy-binding §7
|
||||
(L10). Both are requirements in the text only.
|
||||
- It does not claim membership of DVB, 3GPP or SC 29, or that any input will be
|
||||
made.
|
||||
- It does not describe any WAI mechanism as adopted or endorsed by any body
|
||||
named here.
|
||||
|
||||
---
|
||||
|
||||
## Sources
|
||||
|
||||
Read on 2026-09-27 unless noted.
|
||||
|
||||
1. DVB BlueBook **S100**, *Study Mission report on Energy Aware service Delivery
|
||||
and Consumption*, Nov 2023. Read in full (§7 use cases, §8–9).
|
||||
<https://dvb.org/wp-content/uploads/2023/11/S100_Study-Mission-on-Energy-Aware-Delivery-and-Consumption_Nov-2023.pdf>
|
||||
2. DVB, *TV 2030: preparing for a future that's already in sight*, 10 Sep 2025
|
||||
(the energy-efficiency working group's status).
|
||||
<https://dvb.org/news/tv-2030-preparing-for-a-future-thats-already-in-sight/>
|
||||
3. **3GPP TR 26.942 V20.0.0** (2026-06). Read from the archive:
|
||||
<https://www.3gpp.org/ftp/Specs/archive/26_series/26.942/26942-k00.zip>.
|
||||
Specification record:
|
||||
<https://portal.3gpp.org/desktopmodules/Specifications/SpecificationDetails.aspx?specificationId=4305>
|
||||
4. 5G-MAG / 3GPP SA4 / Greening of Streaming workshop, 19 Mar 2026 (WI 1080050;
|
||||
workshop statements).
|
||||
<https://www.5g-mag.com/workshops/media-energy-consumption/>
|
||||
5. **ISO/IEC CD 23001-11** 4th edition, WG 3 N1864 (w26533), cover dated
|
||||
2026-08-28. Clean text read from
|
||||
<https://www.mpeg.org/wp-content/uploads/mpeg_meetings/155_Geneva/w26533.zip>.
|
||||
Part page: <https://www.mpeg.org/standards/MPEG-B/11/>
|
||||
6. ISO/IEC 23001-19 part page.
|
||||
<https://www.mpeg.org/standards/MPEG-B/19/>
|
||||
7. IETF datatracker: GREEN WG documents,
|
||||
<https://datatracker.ietf.org/wg/green/documents/>. Also
|
||||
`draft-ietf-avtcore-rtcp-green-metadata`,
|
||||
<https://datatracker.ietf.org/doc/draft-ietf-avtcore-rtcp-green-metadata/>
|
||||
8. European Commission, **C(2026) 3472 final**, 21 Sep 2026, part 1 (recitals and
|
||||
articles; the annexes were not read for this note).
|
||||
<https://energy.ec.europa.eu/document/download/b073ba9b-82a8-45cb-b8ba-78dad149e941_en?filename=C_2026_3472_1_EN_ACT_part1_v9.pdf>
|
||||
9. OCMF, Open Charge Metering Format, `OCMF-en.md` at repository commit f6de139
|
||||
(2025-12-04). Read: *Goal*, *Transmission of Signed meter Readings*,
|
||||
*Signing and Verification Process*, *General Information* (the
|
||||
signature-generating component), *Meter Identification*, and *Relation of
|
||||
Serial Numbers, Charge Point and Public Key*. The spec does not describe how
|
||||
the signing component is certified under calibration law, and that was not
|
||||
checked elsewhere.
|
||||
<https://github.com/SAFE-eV/OCMF-Open-Charge-Metering-Format/blob/master/OCMF-en.md>
|
||||
10. In this tree, at link 10 of the landing chain (2026-09-30), and on `main`
|
||||
where a statement says so: `extensions/energy-measurement.md` §1–§8,
|
||||
`energy-binding.md` (§3, §4, §7), `jwp-receipts.md` §2.1–§3.1,
|
||||
`moq-streaming-format.md` §1 and §5, `determinism-tiers.md` §2,
|
||||
`oer-receipts.md` §2–§3, `transparency.md` §6, and the receipt sections of
|
||||
`linear-film.md`, `ternary-tt.md`, `provenance.md`, `spatial-audio.md`,
|
||||
`volumetric.md`, `interactive-worlds.md`, `avatars.md`, `state-feeds.md`,
|
||||
`haptics.md`, `score.md`, `quantum-sim.md` and `quantum-ops.md`. In code:
|
||||
`wai-rs/src/energy_class.rs`, `energy_burst.rs`, `video_receipt.rs`,
|
||||
`energy_binding.rs`, `c2pa_emit.rs`, `claim_receipt.rs`, the reference
|
||||
meters under `wai-rs/src/bin/`, and every `signing_payload` in `wai-rs/src`
|
||||
(`film/`, `ternary_receipt.rs`, `provenance.rs`, `audio_scene/`, `splat4d/`,
|
||||
`world/`, `haptic/`, `score/`, the `wai_video_*` modules) and
|
||||
`wai-quantum/src` (`quantum_receipt.rs`, `quantum_ops.rs`,
|
||||
`quantum_phasor_meter.rs`, `quantum_qbom.rs`, `quantum_energy.rs`). Also a
|
||||
search of both code trees for the energy-binding §7 fields (L10). The `main`
|
||||
groups of L1 were sorted at commit 1b3e390, and `main`'s `wai-rs` and
|
||||
`wai-quantum` receipt code is unchanged since.
|
||||
11. [2026-09-global-sweep](2026-09-global-sweep.md), for the CM-EE announcement
|
||||
(§2.4), the ISO/IEC 23001-19 stage (§2.1) and the annexes of C(2026) 3472
|
||||
(§5.2).
|
||||
961
wai/research/syfai-input.md
Normal file
961
wai/research/syfai-input.md
Normal file
|
|
@ -0,0 +1,961 @@
|
|||
# MPEG SyfAI (Exploration 52): WAI's input on the AI update framework
|
||||
|
||||
> **Status: draft input, not submitted.** A preparation document written in the
|
||||
> shape of an input contribution toward MPEG SyfAI's next contribution window,
|
||||
> MPEG 156, Hangzhou, 19–23 October 2026. It is not a normative extension.
|
||||
> **An input to an MPEG meeting can only be registered by a delegate accredited
|
||||
> through an ISO/IEC JTC 1/SC 29 national body. WAI has no such accreditation
|
||||
> today**, so nothing here should be planned around an assumed submission. The
|
||||
> posture is the same as [mpeg-ai-part6-mapping](mpeg-ai-part6-mapping.md):
|
||||
> specify WAI *against* the work so it can become a profile of the eventual
|
||||
> text.
|
||||
>
|
||||
> **Which tree.** WAI's mechanisms are described as they stand in the tree at
|
||||
> link 10 of the landing chain: a linear chain of pull requests on the forge,
|
||||
> each targeting `main`, merging in order. **As of 2026-09-30 none is merged
|
||||
> into `main`.**
|
||||
>
|
||||
> | link | branch | PR |
|
||||
> |---|---|---|
|
||||
> | 01 | `feat/wai-video-workflow` | #82 |
|
||||
> | 02 | `wai/land-02-determinism-tiers` | #103 |
|
||||
> | 03 | `wai/land-03-jpegai-no-libm` | #104 |
|
||||
> | 04 | `wai/land-04-codec-registry` | #105 |
|
||||
> | 05 | `wai/land-05-prior-nnc` | #106 |
|
||||
> | 06 | `wai/land-06-ci-headline-tests` | #107 |
|
||||
> | 07 | `wai/land-07-int-decode-kernels` | #108 |
|
||||
> | 08 | `wai/land-08-energy-classes` | #109 |
|
||||
> | 09 | `wai/land-09-int-mlicpp` | #111 |
|
||||
> | 10 | `wai/land-10-receipt-classes` | #112 |
|
||||
>
|
||||
> Each link contains the ones before it. Everything this note attributes to a
|
||||
> link is proposed in that link's pull request and is not in `main`; where
|
||||
> `main` differs, the text says so. The CI evidence in §2 is `main`'s, read from
|
||||
> the forge on 2026-09-27, and the chain's, from the forge's runs on its pull
|
||||
> requests, read on 2026-09-30.
|
||||
>
|
||||
> This note cross-references rather than repeats. The NNC carriage question is in
|
||||
> [2030-transport-landscape](2030-transport-landscape.md) §6.4, and the form WAI
|
||||
> registers for it is [prior-carriage](../extensions/prior-carriage.md). The
|
||||
> post-decode processing hole is described in §3.2 of the landscape, and the rule
|
||||
> that closes it is [determinism-tiers](../extensions/determinism-tiers.md) §2.
|
||||
> Provenance over reconstructed output is in
|
||||
> [reconstruction-binding](../extensions/reconstruction-binding.md). Where the
|
||||
> landscape's text disagrees with a finding here, the gap that finds it says so
|
||||
> (G1, G2). Where WAI's own normative text and its reference implementation
|
||||
> disagree, the gap names both sides (G1, G12).
|
||||
>
|
||||
> Everything marked **⚠ unverified** was not seen in a primary source. The source
|
||||
> list at the end says what was read and when.
|
||||
|
||||
---
|
||||
|
||||
## 1. The exploration, as the primary documents record it
|
||||
|
||||
| meeting | output document | title | status of this note's read |
|
||||
|---|---|---|---|
|
||||
| MPEG 151, Daejeon, 30 Jun – 4 Jul 2025 | none published | *Exploration on MPEG Systems support for Coding for Machine technologies* | title only, from the exploration page |
|
||||
| MPEG 153, online, 19–23 Jan 2026 | WG 3 N1752 (w25953), dated 2026-02-09 | *Exploration on Systems technologies for coding for machine standards* | **read in full** |
|
||||
| MPEG 154, Santa Eulària, 27 Apr – 1 May 2026 | WG 3 N1809 (w26201), dated 2026-05-02 | *Exploration on Systems technology for AI-based media standards (SyfAI)* | **read in full** |
|
||||
| MPEG 155, Geneva, 13–17 Jul 2026 | listed, not published on mpeg.org as of 2026-09-27 | same SyfAI title | **not read**; substance known only from a secondary report |
|
||||
| MPEG 156, Hangzhou, 19–23 Oct 2026 | — | next meeting | — |
|
||||
| MPEG 157, Brisbane, 18–22 Jan 2027 | — | also evaluates MPEG-AI Part 6 CfP responses | — |
|
||||
|
||||
**When it began.** The exploration was not opened at MPEG 155. N1809's own
|
||||
summary says the work "was established" at MPEG 153, where it concerned carriage
|
||||
of Video Coding for Machines (VCM) NAL-unit video in ISOBMFF. It carries forward
|
||||
an MPEG 151 exploration on systems support for coding for machines. The SyfAI
|
||||
title first appears on the MPEG 154 output. The exploration's output documents
|
||||
are **WG 3 (MPEG Systems)** documents. The secondary report on MPEG 155
|
||||
(source 5) says the AI-update-framework thread was "opened jointly with WG 7"
|
||||
(3D Graphics and Haptics). No WG document seen here records that. An input
|
||||
should nonetheless be planned for both groups.
|
||||
|
||||
**What N1809 records from MPEG 154.** Most of the document is draft carriage text
|
||||
for VCM in ISO/IEC 14496-15 (`vcm1`/`vcmi` sample entries, a
|
||||
`VCMDecoderConfigurationRecord`). The part that concerns WAI is §3, the
|
||||
discussion list. Its first item, *flexible AI codec model framework*, records
|
||||
this need:
|
||||
|
||||
- unified terminology (modules, blocks, primitives);
|
||||
- system-level support for model **discovery, validation, storage, delivery**;
|
||||
- configuration update mechanisms;
|
||||
- standardised signalling of **models, weights, metadata, training info**;
|
||||
- a potential **model repository** infrastructure.
|
||||
|
||||
The comments recorded against it ask for a conceptual architecture
|
||||
("when/where/how does system fetch"), for requirements covering **security,
|
||||
authentication**, level of flexibility and repository, and for a defined scope.
|
||||
One comment reads "menifest [sic] shall include inner functionality of codec,
|
||||
delivery of model, capalibity [sic], etc." They name AI-based point-cloud coding
|
||||
and Feature Coding for Machines (FCM) as starting points. The recommendation
|
||||
reads: *"Noted. Bring architectural diagram and requirements."* A use case is
|
||||
requested for the next meeting.
|
||||
|
||||
A second item proposes **a new MPEG standard for storage and carriage of
|
||||
compressed neural networks as defined in ISO/IEC 15938-17 (NNC)**. The
|
||||
recommendation was to bring a use case. This updates landscape §6.4: the
|
||||
question of how the artifact a `Prior` names travels on the wire is being asked
|
||||
at the systems layer in WG 3, not only in the NNC amendment.
|
||||
|
||||
**What the MPEG 155 output adds (⚠ secondary source only).** A July 2026 meeting
|
||||
report lists five topics for an *AI update framework*: "(i) a format for AI
|
||||
parameter data, (ii) a manifest for updating parameters, (iii) a repository to
|
||||
serve them, (iv) integrity checking, and (v) bit exactness together with
|
||||
conformance". It adds that "neural inference varies with library and hardware, so
|
||||
deciding what conformance means and how to test it remains open". Topics (i) to
|
||||
(iv) correspond to items N1809 §3 records: signalling of models and weights,
|
||||
configuration update, a repository, and security and authentication. **Topic
|
||||
(v), bit exactness with conformance, has no counterpart in N1809.** A search of
|
||||
its text for exactness, conformance and integrity finds only a VCM HRD note. So
|
||||
this note's topic (v), which §2 row (v), R2, R4 to R7 and G10 answer, rests on the
|
||||
secondary report alone. Read the MPEG 155 output when it is published before
|
||||
quoting any of the five topics.
|
||||
|
||||
**The shape an input should take.** N1809 asks for *use cases, an architecture
|
||||
and requirements*. It does not ask for technology proposals yet. §3 and §4 below
|
||||
are drafted in that shape.
|
||||
|
||||
---
|
||||
|
||||
## 2. Mapping: the five framework topics against what WAI specifies
|
||||
|
||||
"Specified" means normative text in this tree (SPEC.md or an extension).
|
||||
"Implemented" means the reference implementation (`wai-rs`) does it.
|
||||
"CI-checked" means a workflow on the forge is defined to exercise it. *CI
|
||||
scope*, below the table, says which workflows do so and, for `main`, what their
|
||||
runs showed. A row marked *reference implementation only* describes code, not
|
||||
the standard, and an input cannot present it as WAI's normative position until
|
||||
the text catches up (G1).
|
||||
|
||||
| framework topic | WAI mechanism | state | honest limit |
|
||||
|---|---|---|---|
|
||||
| (i) AI parameter-data format | **One registered form, optional.** A `Prior` MAY declare that the artifact its digest covers is an ISO/IEC 15938-17:2024 (NNC) bitstream, `wai.prior.nnc`; the digest then covers the coded bitstream ([prior-carriage](../extensions/prior-carriage.md) §2–§3; link 05, PR #106). The model is still never transported in the envelope that decodes against it (SPEC.md's opening section). Three informative formats exist: the integer `int_synth.json` model in [neural-conformance](../neural-conformance/README.md); the JPEG AI weight bundle (`weights/*.json` + `norm.json`) read by `try_jpegai_exact`, whose hyper-decoder, context-model and synthesis networks are integer but whose eICCI network and one `norm.json` table are `f32`; and the one-file `WMB1` bundle of `wai.neural.int_mlicpp` — three integer networks and the entropy tables in one file (link 09, PR #111). | NNC form: specified, not implemented. Informative formats: implemented | `wai-rs` has no NNC reader or writer, and the reference sink refuses any prior that declares a form (`READABLE_PRIOR_FORMS` is empty). NNC is optional and not recommended: its royalty status, and whether conforming decoders reconstruct identical tensors, are not established (prior-carriage §4–§5). On `main` no form exists at all. |
|
||||
| (ii) manifest | The reference implementation carries `conditioning.prior = { id, sha256, determinism }` in a single-payload (`WAI1`) manifest (`container.rs`, `Prior`), and at link 05 an optional `form`. A multi-rendition (`WAI2`) manifest carries the same `Prior` per rendition (`RenditionMeta.prior`). It names *which* model, exactly, and *how* its decode is made reproducible. prior-carriage §2 (a Draft extension) shows the object with its four members. | **implemented (reference implementation); shown in a Draft extension; not in SPEC.md §3** | SPEC.md §3's manifest schema defines `conditioning` as `{ "kind" }` only, and SPEC.md §7.1 defines no prior on a rendition entry (G1). The pin names identity, not location, and has no update or versioning semantics. Without a declared form, `sha256` covers **one file** (G2). `determinism` is free text (G3). The per-rendition pin is carried and never checked (G12). |
|
||||
| (iii) repository | **None.** SPEC.md §1 scopes WAI as a container and capability-dispatch standard; nothing in the tree specifies a model store or service. Identity-by-digest makes any store content-addressable, but WAI specifies no store. | out of scope | A repository would be SyfAI's. WAI's input is the identity rule a repository must honour, not the repository. |
|
||||
| (iv) integrity checking | `ModelRegistry::resolve_checked` recomputes SHA-256 over the registered artifact and refuses to decode on mismatch (`DecodeError::PriorMismatch`); at link 05 it also refuses, before hashing, a declared form the sink cannot read. The `wai.neural.int_mlicpp` decoder re-checks a pinned prior against the very bytes it decodes with (link 09). `resolve_checked` is called only from the `WAI1` decode (`decode_envelope`), so no code path checks a `WAI2` rendition's pin (G12). `decode_envelope_strict` checks the capability's registry class and then the pinned prior before a `replicate` decode (link 02, PR #103). Because the manifest sits inside the envelope bytes the JWP receipt hashes ([jwp-receipts](../extensions/jwp-receipts.md) §2.1), a signed receipt authenticates the pin as well as the payload. | **implemented (reference implementation); partly specified** | prior-carriage §2 requires a sink to refuse a prior whose declared form it cannot read, "as it refuses a digest mismatch", but no normative text states the refusal on a digest mismatch itself (G1). SPEC.md §7 requires a sink to refuse a `replicate` claim over a `NeuralEntropyExact`, `NeuralFloat`, `GenerativeDeterministic` or `Behavioral` capability (on `main`: over a `NeuralFloat` one). A digest proves the bytes are the named bytes. It does not say who chose them. That comes from a signed JWP receipt over the envelope, and WAI has no key discovery (landscape §3.6). The lenient `decode_envelope` path decodes unpinned neural capabilities without checking; this is honest for perceptual use and is not a replicate path. |
|
||||
| (v) bit exactness + conformance | Two named tiers, `entropy-consistency` and `decode-equivalence`, and the rule that a tier covers what the sink **emits**, including post-decode processing ([determinism-tiers](../extensions/determinism-tiers.md) §2). Over a neural capability `replicate` promises `decode-equivalence`, which SPEC.md §7 admits only by an integer / fixed-point decode path with no float operation; `entropy-consistency` does not back it (link 02). A per-capability `ReplicateClass` in the registry: the `NeuralIntegerExact` capabilities at `decode-equivalence`, `wai.neural.int_mlicpp` among them (link 09), and `wai.image.jpegai` alone in `NeuralEntropyExact` at `entropy-consistency` (link 02). `codecs::DeterminismTier` carries the tiers in code (link 02). Frozen corpora of decoded-output digests. | tiers: specified, and a type in code at link 02 (on `main`, text only). Emitted-medium rule: **specified only**. Integer-entropy contract, `ReplicateClass` and the strict check: specified + implemented. Corpora and byte-equality goldens: implemented, and CI-checked on `main` for the capabilities listed under *CI scope*, on the legs that ran. | No manifest field names a tier; a capability's tier comes from its registration. `wai-rs` does not handle post-decode processing or operation-reduction hints. `wai.image.jpegai`'s exact decoder pins one reconstruction, reproducible only under its own build and floating-point conditions, and the reference sink can decode the same capability through a float path (G11). The float MLIC++ decoder agrees with its encoder only by convention between the two implementations; `wai.neural.int_mlicpp` makes that agreement a property of the arithmetic (*CI scope*). On `main`, a malformed stream's outcome depends on the build profile; link 01 fixes the known case (G6). Float capabilities have no corpus, deliberately. |
|
||||
|
||||
**CI scope, precisely.** Two states are described. For `main`, the workflow
|
||||
files and the forge's run records read on 2026-09-27. For the tree at link 10,
|
||||
the workflow files and the forge's runs on the chain's pull requests, read on
|
||||
2026-09-30.
|
||||
|
||||
*On `main`.* Three workflows exercise WAI's decode paths. This is what each is
|
||||
defined to run at commit 1b3e390, read from the workflow files (afc7ece adds the
|
||||
OER vector step):
|
||||
|
||||
- **`wai-conformance.yml`** runs on any change under `wai/wai-rs/`, the
|
||||
conformance corpora or `wai/byte-exact-conformance/`. Its matrix is linux
|
||||
x86-64, linux arm64, macOS arm64 and windows x86-64. Each leg runs debug
|
||||
builds of three steps:
|
||||
- `cargo test --lib` over the pure-Rust features, including `neural_int`.
|
||||
This covers the JPEG AI parsing, dequantisation and post-filter unit tests,
|
||||
which check closed-form cases.
|
||||
- The carriage test.
|
||||
- Re-derivation of every cargo-class corpus, including the `int_synth`
|
||||
neural corpus.
|
||||
|
||||
A second job runs the `byte-exact-conformance` crate, described next, in
|
||||
release builds on the same four legs.
|
||||
- **`wai-byte-equality.yml`** runs only on changes to the listed integer-decode
|
||||
sources and vectors. It uses the same four legs and runs
|
||||
`byte-exact-conformance` in release builds. It checks:
|
||||
- the integer rANS goldens;
|
||||
- the `int_synth` output digest;
|
||||
- the JPEG AI entropy-stage (mANS) goldens;
|
||||
- the `wai.video.int_motion` clip goldens;
|
||||
- the `wai.video.int_hyper` clip, and its learned keyframe on its own, which
|
||||
is decoded through a small committed `wai.neural.int_hyper` model.
|
||||
- **`wai-neural-e2e.yml`** runs only on changes to `wai-rs/src/neural/`,
|
||||
`tests/neural_native_decode.rs`, the fixture generator or the workflow
|
||||
itself. It has one leg, linux x86-64. `tools/wai_validate_synthetic.py`
|
||||
builds minimal synthetic models and bitstreams in the job. The job then runs
|
||||
the float `_matches_reference` tests in `tests/neural_native_decode.rs`.
|
||||
Those tests check the ELIC, MLIC++, SNAC, DCVC and GLC decoders against a
|
||||
reference decode within ±4. So the job is defined to test whether each float
|
||||
decoder is correct end to end on one machine. It makes no cross-architecture
|
||||
claim, and its models are synthetic, not trained. No run of it has yet
|
||||
reached those tests (below).
|
||||
|
||||
Four limits apply when any of these is read as evidence. They come from the
|
||||
forge's run listing, and from each run's per-job status and job logs, all read
|
||||
on 2026-09-27 (source 11).
|
||||
|
||||
- **An offline runner leaves a cancelled job behind, not a gap.** The host of
|
||||
the `windows` runner has been offline since about 2026-08-28. The two matrix
|
||||
workflows still create its jobs, and no runner claims them. Each such job
|
||||
created from 2026-08-28 on was later cancelled, in one of two ways: a newer
|
||||
push to the same branch started a new run, or a day or more passed after the
|
||||
job was created. A cancelled job marks the whole run as cancelled, even when
|
||||
every other job in it passed. The run on `main` at `afc7ece` was still
|
||||
waiting on its two `windows` jobs.
|
||||
- **Since then, every job on the other three platforms that finished has
|
||||
passed.** In the runs of `wai-conformance.yml` and `wai-byte-equality.yml`
|
||||
created from 2026-08-28 to 2026-09-27, no linux x86-64, linux arm64 or
|
||||
macOS arm64 job failed. Some were cancelled when a newer push superseded
|
||||
their run, and no `windows` job ran. The last runs in which the `windows`
|
||||
leg also passed date from 2026-08-27 (`wai-conformance.yml`) and 2026-08-26
|
||||
(`wai-byte-equality.yml`).
|
||||
- No run exists at 1b3e390 itself. The nearest is the run on `main` at
|
||||
`afc7ece`. In `wai-rs` that commit adds only the `oer` module, behind its
|
||||
own feature, with its vector binary and the workflow steps that run them.
|
||||
- In that run, on linux x86-64, linux arm64 and macOS arm64, both jobs
|
||||
passed: the corpus job (library tests, carriage test and corpus
|
||||
re-derivation) and the `byte-exact-conformance` job. Their logs record no
|
||||
failed test.
|
||||
- So the cross-architecture evidence for `main` covers three of the four
|
||||
platforms. The `windows` leg's latest evidence predates 2026-08-28.
|
||||
- **Tests that need model assets pass when the assets are absent (G8).** In
|
||||
`wai-neural-e2e.yml` this also covers a partial fixture set. On `main` the
|
||||
generator exits 0 if any one of its five harnesses succeeds, and the Rust
|
||||
tests whose fixtures are missing then report as passed. PR #101, independent
|
||||
of the chain, makes both cases fail the job.
|
||||
- **`wai-neural-e2e.yml` has never reached its tests.** The listing holds 21
|
||||
runs of it, from 2026-07-23 to 2026-09-26. Eighteen failed and three were
|
||||
cancelled.
|
||||
- The logs of all eighteen failures show the fixture step generating all
|
||||
five synthetic fixtures. The job then stopped before any Rust test ran.
|
||||
- Two early runs (2026-07-23 and 2026-07-26) failed at the Rust toolchain
|
||||
step with `rustup: command not found`.
|
||||
- The other sixteen, which include every run from 2026-07-31 on, failed to
|
||||
link the test binary. The prebuilt ONNX Runtime static library that the
|
||||
`ort` crate links references C-library symbols (`__isoc23_strtol`,
|
||||
`__isoc23_strtoll`, `__isoc23_strtoull`) and, from 2026-08-26 on, a C++
|
||||
standard-library symbol (`_M_replace_cold`) that the job's default
|
||||
Ubuntu 22.04 image does not provide.
|
||||
- No log contains a test result. The float decode tests are *defined* to run
|
||||
in CI, and they have never run there.
|
||||
|
||||
**What the MLIC++ comparison exposes once the job links.** The float
|
||||
`wai.neural.mlicpp` comparison then fails on the fixtures as generated, and the
|
||||
cause is established. The fixture generator ran ONNX Runtime at its default
|
||||
intra-op thread count, while the Rust decoder loads every session
|
||||
single-threaded (`ort_session::load_deterministic`). Convolution reductions
|
||||
therefore summed in a different order, and one scale landed on the other side of
|
||||
a scale-table edge at symbol 416 of 12,288, which desynchronised the rANS decode
|
||||
from there. PR #101 (independent of the chain) moves the job to an Ubuntu 24.04
|
||||
image, in which the test binary links, and makes the generator load every session
|
||||
with the decoder's execution settings. `wai-neural-e2e.yml` passed on the forge on
|
||||
#101's head (2026-09-29), and its record reports the MLIC++ comparison going from
|
||||
max |Δ| 255 to 0. That fix
|
||||
is a convention between two implementations.
|
||||
[`wai.neural.int_mlicpp`](../SPEC.md) (link 09, PR #111) makes the agreement a
|
||||
property of the arithmetic: the scales come from integer networks, so every
|
||||
conforming decoder computes the encoder's bucket for every element at any thread
|
||||
count (`wai-rs/src/int_mlicpp.rs`, module documentation).
|
||||
|
||||
*At link 10*, the chain changes what CI runs:
|
||||
|
||||
- `wai-conformance.yml` adds, on every leg and in both build profiles, the
|
||||
kernel-equivalence tests of link 07 over every committed integer-decoder
|
||||
vector — among them the `wai.audio.int_codec` and `wai.splat.int_codec`
|
||||
vectors, each decoded against its frozen hash under every kernel policy — and
|
||||
the `wai.neural.int_mlicpp` tests of link 09. On its unix legs it adds two
|
||||
checks that keep platform maths-library calls off the JPEG AI exact decode
|
||||
(link 03). Its library tests add the `oer` and `receipt_claims` features.
|
||||
- `wai-byte-equality.yml` adds the `wai.neural.int_mlicpp` golden (link 09).
|
||||
- `wai-neural-gates.yml` (link 02) runs, in an Ubuntu 24.04 image, the
|
||||
model-free neural-sink tests: the prior-hash refusal, the strict `replicate`
|
||||
gate and, at link 09, the `wai.neural.int_mlicpp` dispatch.
|
||||
- `wai-sota-vectors.yml` and `wai-jpegai-e2e.yml` (link 06) supply the inputs of
|
||||
the full-size integer-model tests and of the JPEG AI whole-decode tests, and
|
||||
fail on a missing input (G8).
|
||||
- `wai-energy-receipts.yml` (link 08) and `wai-formal-verification.yml`
|
||||
(link 01) cover the energy-class receipts and the bounded model-checking
|
||||
harnesses.
|
||||
- `wai-neural-e2e.yml` is unchanged: at link 10 it still runs in the default
|
||||
image and still generates fixtures at the default thread count.
|
||||
|
||||
On the forge (runs read 2026-09-30), the chain's workflows have run since
|
||||
2026-09-28. `wai-sota-vectors.yml`, `wai-jpegai-e2e.yml` and
|
||||
`wai-neural-gates.yml` have passed on every run. The conformance and
|
||||
byte-equality workflows have run on every unix leg, with a few Linux arm64
|
||||
failures; each link's current result is on its pull request.
|
||||
`wai-neural-e2e.yml` fails on every link, as on `main`, and passes only with
|
||||
#101. The Windows legs have not run: their runner has been offline since late
|
||||
August. [determinism-tiers](../extensions/determinism-tiers.md) §6 says this
|
||||
from link 11.
|
||||
|
||||
A statement that a capability "is CI-checked" therefore means that a workflow
|
||||
checks it whenever the workflow runs, on the legs that ran. For `main`, that
|
||||
means linux x86-64, linux arm64 and macOS arm64 for the byte-exact workflows,
|
||||
and nothing yet for `wai-neural-e2e.yml`. For the chain, it means the same three unix
|
||||
legs, and `wai-neural-e2e.yml` only once #101 is merged.
|
||||
|
||||
---
|
||||
|
||||
## 3. Use cases, in the form N1809 asked for
|
||||
|
||||
**UC-1: Replicate-grade reconstruction across heterogeneous decoders.** A publisher
|
||||
encodes with a learned codec and declares that any conforming decoder reproduces
|
||||
the same output bytes. The use case needs a manifest that says which parameter
|
||||
set, which reproducibility tier, and what a conformance test compares. The
|
||||
tier can be the entropy-stage symbols or the reconstructed medium. WAI's
|
||||
integer capabilities are built to reproduce the same output bytes. On `main`,
|
||||
the CI workflows check that on well-formed streams for `wai.neural.int_synth`,
|
||||
`wai.neural.int_hyper` (through a small committed model), `wai.video.int_motion`,
|
||||
`wai.video.int_hyper` and the JPEG AI entropy stage, on the legs that ran (§2):
|
||||
since 2026-08-28, linux x86-64, linux arm64 and macOS arm64, where every job
|
||||
that finished passed. At link 10 the workflows are also defined to check
|
||||
`wai.neural.int_mlicpp` on the byte-exact matrix, and the committed
|
||||
`wai.audio.int_codec` and `wai.splat.int_codec` vectors on every conformance leg
|
||||
(§2). [determinism-tiers](../extensions/determinism-tiers.md) §6 records
|
||||
`wai.neural.int_mlicpp`'s committed vector decoding to its pinned bytes on macOS
|
||||
arm64 and on wasm32-wasip2, measured on the development machine. The manifest
|
||||
says less than this use case needs. In the reference implementation it names the
|
||||
parameter set by a digest over one file unless a form is declared (G1, G2). No
|
||||
manifest field names the tier (R2, R4). What a conformance test compares is
|
||||
fixed by the capability's registration and corpus, not stated in the manifest.
|
||||
|
||||
**UC-2: A parameter update that must not decode silently.** A service moves a stream
|
||||
from parameter set A to parameter set B. A sink that still holds A must refuse
|
||||
B-coded content, or fall back to a declared alternative. Decoding it against A
|
||||
is not acceptable, because a learned entropy decode against the wrong parameters
|
||||
yields plausible garbage, not an error.
|
||||
|
||||
What WAI's text says about this case today:
|
||||
|
||||
- **SPEC.md requires the decode this use case rules out.** Dispatch (SPEC.md §4)
|
||||
is keyed on the capability string alone. A sink holding A advertises the
|
||||
capability, so step 3 applies: "decode `payload` via the codec registered for
|
||||
`cap` (§5) and return". Nothing in §4 looks at the parameter set.
|
||||
- **`model_requirement.fallback` does not help.** SPEC.md §3 and §4 steps 4–5
|
||||
reach it only when the sink does *not* advertise the primary capability, and
|
||||
§7.1 calls it "declarative" in v1.0.
|
||||
- **Nor do renditions, as specified.** A §7.1 rendition is chosen by
|
||||
capability advertisement or deployer policy. SPEC.md gives a rendition entry
|
||||
a `capability` and a `kind`, plus an optional `role` and `companion_of` for
|
||||
companion descriptors. It defines no prior field, so the specification
|
||||
cannot mark one entry as the one for sinks that hold A.
|
||||
- **The reference implementation refuses a mismatch only in a single-payload
|
||||
envelope.** When a `WAI1` manifest pins a prior, `resolve_checked` refuses a
|
||||
mismatched model with `DecodeError::PriorMismatch`, a hard error. Nothing
|
||||
then tries the fallback capability or another rendition. On `main`,
|
||||
`fallback` is only printed by the CLI (`inspect`, `unwrap`). At link 04
|
||||
(PR #105), `Wai::dispatch_capability` applies SPEC.md §4 steps 3 and 5 and
|
||||
`wai decode` and `wai inspect` use it, so the CLI decodes through the fallback
|
||||
when the sink lacks the primary capability; it still never does so after a
|
||||
`PriorMismatch`. With no pinned prior, the decode runs against whatever model
|
||||
is registered.
|
||||
- **The reference implementation carries a prior per rendition, and nothing
|
||||
uses it.**
|
||||
- `RenditionMeta` and `Rendition` each have `prior: Option<Prior>`
|
||||
(`container.rs`), with the same meaning as the `WAI1` pin.
|
||||
- It is serialised in the `WAI2` manifest, and the C FFI's pack call passes
|
||||
it through (`ffi.rs`).
|
||||
- The library test `multi_rendition_round_trip` checks that it survives the
|
||||
wire. `wai-conformance.yml` runs that test, and it passed in the `afc7ece`
|
||||
run on the three platforms that ran (§2).
|
||||
- Selection ignores it. `pick()` and `primary()` choose by capability and
|
||||
role alone. `resolve_checked` is called only from the `WAI1` decode, so no
|
||||
path checks a `WAI2` rendition's pin.
|
||||
- The WebAssembly binding's rendition metadata (`renditionsMeta` in
|
||||
`wai-web`) drops the field. Its raw manifest string keeps it.
|
||||
- So an envelope can carry one rendition pinned to A and another pinned to B
|
||||
under the same capability, and `pick()` returns whichever comes first,
|
||||
whatever the sink holds.
|
||||
|
||||
So WAI's normative text has no answer to this use case, and its dispatch rule
|
||||
points the other way. The reference implementation refuses a pinned mismatch
|
||||
in a single-payload envelope, but offers no declared alternative. For
|
||||
renditions it already carries the data a declared alternative would need, a
|
||||
prior per rendition, but no selection rule or check uses it. The gap is G12,
|
||||
together with G1 and G5.
|
||||
|
||||
**UC-3: Provenance over the reconstructed output.** A provenance manifest binds a
|
||||
hash of the *decoded* pixels, which is portable only when the reconstruction is
|
||||
pinned. This connects SyfAI's bit-exactness topic to MPEG-AI Part 6. In the tree
|
||||
at link 10, such a binding is checkable by every conforming sink over a
|
||||
`decode-equivalence` capability; over `wai.image.jpegai` it holds only for the
|
||||
exact decoder's output, under that decoder's conditions
|
||||
([reconstruction-binding](../extensions/reconstruction-binding.md) §1–§2). See
|
||||
also [mpeg-ai-part6-mapping](mpeg-ai-part6-mapping.md).
|
||||
|
||||
**UC-4: Energy attributed to a reproducible decode.** The energy of a decode
|
||||
can be reported against a work unit that a third party recomputes from the
|
||||
codestream. A work unit such as pixel operations needs only the codestream's
|
||||
geometry, and WAI's video receipt derives it that way. What needs a
|
||||
reproducible decode is binding the figure to a hash of the output that a third
|
||||
party can regenerate. See [energy-exposure-mapping](energy-exposure-mapping.md).
|
||||
|
||||
---
|
||||
|
||||
## 4. Candidate requirements
|
||||
|
||||
Drafted as an input would phrase them. After each one: what WAI does in the tree
|
||||
at link 10, or the gap.
|
||||
|
||||
**R1: Digest over the complete parameter set.** A manifest entry for an AI
|
||||
parameter set shall identify it by a cryptographic digest over a canonical
|
||||
serialisation of *every* artifact that affects decoder output: networks,
|
||||
entropy tables and normative constants. A digest over one file of a multi-file
|
||||
set does not meet this.
|
||||
*WAI: partial.* The pin covers the one registered file (G2). Two routes to R1
|
||||
exist at link 10: a declared form, whose digest covers the whole coded bitstream
|
||||
(prior-carriage §2; no form is readable by the reference sink), and a one-file
|
||||
model bundle, as `wai.neural.int_mlicpp` uses.
|
||||
|
||||
**R2: A declared reproducibility tier from a registered vocabulary.** The
|
||||
manifest shall state the tier of the decode path the parameter set enables. At
|
||||
minimum it shall distinguish entropy-stage reproducibility,
|
||||
full-reconstruction reproducibility, and no cross-implementation guarantee.
|
||||
*WAI: the vocabulary exists, in the capability's registration, not in the
|
||||
manifest.* determinism-tiers names the two tiers, and at link 02 the registry
|
||||
carries them in code (`ReplicateClass::neural_tier`, `codecs::DeterminismTier`),
|
||||
with `NeuralFloat` claiming neither. The per-envelope contract string is still
|
||||
free text (G3), and no manifest field states a tier.
|
||||
|
||||
**R3: Refuse on mismatch.** A decoder whose resolved parameter set does not match
|
||||
the manifest digest shall refuse to decode under that manifest's claims. It
|
||||
shall not degrade silently. *WAI: implemented in the reference implementation
|
||||
only, and only for single-payload envelopes* (`PriorMismatch`). A `WAI2`
|
||||
rendition's pin is carried but not checked. prior-carriage §2 requires refusal
|
||||
of an unreadable declared form, but no normative text requires refusal on a
|
||||
digest mismatch, and SPEC.md §4 step 3 requires a sink that advertises the
|
||||
capability to decode (G1, G12).
|
||||
|
||||
**R4: Reproducibility claims are admissible only for a qualifying path.** A claim
|
||||
that output is reproducible across implementations shall be admissible, for
|
||||
full-reconstruction tiers, only when the decode path is integer or fixed-point
|
||||
end to end. For entropy-stage tiers, the path may instead be integer at the
|
||||
entropy stage or rely on explicit parameter transmission. A reconstruction that
|
||||
one implementation reproduces by evaluating float stages in a fixed order is a
|
||||
property of that implementation, not of the parameter set. The manifest shall say
|
||||
which route a path takes.
|
||||
*WAI: specified and implemented at link 02 (PR #103), except the manifest field.*
|
||||
SPEC.md §7 lets a neural capability back `replicate` only if its entire decode
|
||||
path is integer / fixed-point with no float operation, "the only construction by
|
||||
which a neural capability is registered at `decode-equivalence`". Two weaker
|
||||
properties MUST NOT back it: `entropy-consistency`, and a reconstruction pinned by
|
||||
one implementation's fixed-order float stages. determinism-tiers §3 says a decode
|
||||
that derives its entropy parameters in float meets neither tier, and one that
|
||||
reads them from the codestream, or derives them in integer arithmetic, and runs
|
||||
its later stages in float meets `entropy-consistency` only.
|
||||
`decode_envelope_strict` checks the registry class, then the prior. No manifest
|
||||
field says which route a path takes. On `main` the texts can be read two ways
|
||||
(G10).
|
||||
|
||||
**R5: The tier covers the emitted medium.** Processing a decoder applies after
|
||||
decoding, such as a post-filter or an operation-reduction hint, is part of the
|
||||
path the tier describes. A decoder that alters the medium after decode shall not
|
||||
claim the tier for what it emits. *WAI: specified, not implemented*
|
||||
(determinism-tiers §2, "What a tier covers"). The reference implementation
|
||||
does not handle post-decode processing or operation-reduction hints.
|
||||
|
||||
**R6: Conformance includes non-conforming input.** For a bit-exact tier, the
|
||||
decoder's behaviour on a malformed or truncated stream shall be specified,
|
||||
normally as rejection with an error. Otherwise build configuration, overflow
|
||||
semantics or undefined behaviour change the decode outcome, and "bit exact"
|
||||
holds only for well-formed streams. *WAI: the known defect is fixed at link 01;
|
||||
the requirement is not normative text* (G6).
|
||||
|
||||
**R7: Frozen, architecture-independent conformance digests.** Conformance
|
||||
material for a bit-exact tier shall be frozen digests of decoded output, with
|
||||
the digest algorithm and the canonical output buffer named. It shall be
|
||||
re-derived on more than one instruction-set architecture.
|
||||
*WAI: implemented, but with three digest algorithms, where a contribution should
|
||||
name one per purpose (G7).* `expected_hashes.json` per corpus. Canonical `RGB8`
|
||||
in reconstruction-binding §2.
|
||||
|
||||
**R8: Integrity and authenticity are separate requirements.** Digest match
|
||||
proves which bytes. Binding the manifest under a signature proves who named
|
||||
them. A framework that specifies only the first cannot say whether an update
|
||||
came from the content's publisher. *WAI:* in the reference implementation the
|
||||
pin sits in the manifest, inside the envelope bytes a JWP receipt commits to
|
||||
(jwp-receipts §2.1), so a signed receipt authenticates it. The pin itself is not
|
||||
yet specified in SPEC.md (G1), and key discovery is absent (landscape §3.6;
|
||||
transparency §6 disclaims it).
|
||||
|
||||
**R9: Updates name their predecessor.** A parameter update shall carry the digest
|
||||
of the set it replaces and state whether content coded against the predecessor
|
||||
remains decodable under the successor. *WAI: gap* (G5).
|
||||
|
||||
---
|
||||
|
||||
## 5. Two routes to cross-device consistency
|
||||
|
||||
A SyfAI conformance notion has to admit both routes the field uses, and it has
|
||||
to say what each one reaches. Neither is a defect.
|
||||
|
||||
- **Integer arithmetic.** In JPEG AI (ISO/IEC 6048), "the entropy decoding
|
||||
pipeline employs integer arithmetic neural network operations that can be
|
||||
implemented with 8-bit multipliers and 32-bit registers, which allows bitexact
|
||||
replication" across devices. JPEG AI "defines a bit-exact conformance point
|
||||
only at the end of the entropy decoding pipeline", and its overview states
|
||||
that "bit-exact picture reconstruction and lossless coding were deliberately
|
||||
excluded from the first version" (arXiv 2510.13867, §IV, §III and §XIII
|
||||
respectively). The capabilities WAI's registry classes as
|
||||
`NeuralIntegerExact` take the same route through synthesis as well, which is
|
||||
what `decode-equivalence` means. `wai.neural.int_mlicpp` (link 09) takes it
|
||||
through a channel-autoregressive × checkerboard entropy model, re-deriving its
|
||||
scales from integer networks before each pass. WAI's own JPEG AI decoder takes
|
||||
it for its entropy decode and its integer networks, but not for its
|
||||
dequantisation, latent refinement, post-filters or colour conversion, which
|
||||
run in float; in the tree at link 10, SPEC.md §5 says so and registers the
|
||||
capability at `entropy-consistency` (G11).
|
||||
- **Explicit parameter transmission.** A 2026 learned video codec transmits
|
||||
scale parameters through the hyperprior. This "guarantees entropy coding
|
||||
consistency across devices without requiring bit-exact arithmetic" at a
|
||||
bitrate cost (arXiv 2606.28027, ECCV 2026; abstract read, body not read). The
|
||||
abstract does not say whether reconstructions match across devices. The route
|
||||
guarantees the entropy stage, so identical reconstructions are not guaranteed
|
||||
by it; that is an inference, not something the paper was seen to state. WAI's
|
||||
tier text counts this route as reaching `entropy-consistency`, and at link 02
|
||||
determinism-tiers §2 says why it stops there and §3 no longer reads against
|
||||
that. **No WAI capability is registered as using it** (determinism-tiers §5
|
||||
lists none). A contribution should say that plainly rather than imply
|
||||
coverage.
|
||||
- **Reproducibility by cross-check.** The IEEE 1857.11 call for proposals came
|
||||
out of the working subgroup's meeting on 9 Dec 2021, and its timeline lists
|
||||
the announcement on 21 Dec 2021. It states that "every proponent who submits
|
||||
a complete proposal would be requested to participate in the reproducibility
|
||||
check, i.e., to cross-check another proposal". The submitted decoder "can be
|
||||
binary executable without source code". The call states no bit-exactness
|
||||
requirement. IEEE 1857.11-2024 was published on 2024-12-20.
|
||||
|
||||
The value WAI can add at this layer is naming the tiers and tying each claim to
|
||||
one of them. It does not need to prefer one route.
|
||||
|
||||
---
|
||||
|
||||
## 6. Gaps WAI must close before any of this is contributed
|
||||
|
||||
Each gap was found on `main`. Its status says what the tree at link 10 holds.
|
||||
|
||||
**G1: The prior pin is in the reference implementation, not in the
|
||||
specification.** `Prior { id, sha256, determinism }` is defined in `wai-rs`
|
||||
(`container.rs`), with an optional `form` at link 05. It is carried in two places
|
||||
there: `Conditioning::prior` in a `WAI1` manifest and `RenditionMeta::prior` on
|
||||
each `WAI2` rendition. The digest check and mismatch refusal exist only in
|
||||
`neural/mod.rs` (`resolve_checked`), and apply to the `WAI1` pin only (G12).
|
||||
|
||||
On `main` the specification says less, and in one place the opposite: SPEC.md's
|
||||
opening section says the model is "never transported and never hash-pinned".
|
||||
In the tree at link 10 (link 05, PR #106) that sentence reads that the model is
|
||||
never transported in the envelope that decodes against it, and that "its
|
||||
identity MAY be pinned by digest (a `Prior`), and the pinned artifact's wire form
|
||||
MAY be declared". prior-carriage §2 (a Draft extension) shows the `prior` object
|
||||
with its members and fixes SHA-256 as the digest; §5 and SPEC.md §7 describe the
|
||||
strict check on the pinned prior. What the specification still does not do:
|
||||
|
||||
- SPEC.md §3's manifest schema defines `conditioning` as `{ "kind" }` and has no
|
||||
`prior` field, and SPEC.md §7.1 defines no prior on a rendition entry;
|
||||
- no normative text requires a sink to refuse on a digest mismatch;
|
||||
prior-carriage §2 says a sink refuses an unreadable form "as it refuses a
|
||||
digest mismatch", which presupposes that refusal without stating it;
|
||||
- SPEC.md §4 step 3 requires a sink that advertises the capability to decode
|
||||
and return, so the only normative dispatch rule leads to a decode on a
|
||||
mismatch, not a refusal (UC-2, G12).
|
||||
|
||||
Landscape §6.4 says "WAI *does* pin model **identity** normatively", and asks
|
||||
for SPEC.md's opening sentence to be reconciled with it. The opening sentence is
|
||||
reconciled at link 05. The landscape's sentence still goes further than the
|
||||
text: until SPEC.md defines the pin (fields, digest, refusal on mismatch), in
|
||||
both its single-payload and its per-rendition form, table rows (ii) and (iv) and
|
||||
R3 describe code and a Draft extension, not the core standard. An input cannot
|
||||
rest on them as WAI's normative position. G2 and G3 should be settled in the same
|
||||
change, so that the specified pin is not the one-file, free-text form.
|
||||
*Status: narrowed at link 05; open.*
|
||||
|
||||
**G2: Without a declared form, the prior pin covers one file of a multi-file
|
||||
parameter set.** `resolve_checked` hashes the single registered path. Several
|
||||
decoders then read sibling files the pin never covers:
|
||||
|
||||
- the hyperprior, context and GLC paths load `hyper_synthesis.onnx`,
|
||||
`synthesis.onnx`, `context.onnx` and CDF tables (`*_cdfs_q<q>.json`) from the
|
||||
model's directory (`neural/hyperprior.rs`, `context.rs`, `glc.rs`);
|
||||
- the DCVC path loads `temporal_decoder.onnx` (`neural/video_dcvc.rs`);
|
||||
- `wai.image.jpegai` registers a **directory**, so a pinned prior on it cannot
|
||||
pass verification at all. Hashing reads the path as a file, which fails on a
|
||||
directory. This was checked on the dev machine (macOS arm64, rustc 1.98.0),
|
||||
which returns `Is a directory (os error 21)`.
|
||||
|
||||
The CDF tables determine the entropy decode, so this is not only a float-path
|
||||
concern. The fix is a digest over a canonical manifest of the whole set, which
|
||||
is R1. `wai.neural.int_mlicpp` (link 09) shows one form of it: its model is one
|
||||
`WMB1` file holding three integer networks and the entropy tables, so a pinned
|
||||
prior's SHA-256 covers everything its reconstruction depends on. A declared form
|
||||
(link 05) is the other: its digest covers the whole coded bitstream, but the
|
||||
reference sink reads no form. Landscape §8 queue item 9 says "`Prior.sha256`
|
||||
currently covers an informal directory". As shown above, a pin on a directory
|
||||
cannot verify at all, and where the registered path is a file, `sha256` covers
|
||||
that one file. *Status: open for every multi-file decoder.*
|
||||
|
||||
**G3: The determinism contract is free text classified by substring.**
|
||||
`Prior::determinism_class` returns `IntegerExact` when the string contains
|
||||
`int`, `fixed` or `integer` and no `f32`, `f64` or `float` token
|
||||
(`container.rs`). A contract naming a float runtime without a float token
|
||||
classifies as integer-exact. At link 02 the registry class is checked first, so
|
||||
such a contract can no longer admit a `replicate` over a float or entropy-only
|
||||
capability; for a `NeuralIntegerExact` capability, the prior check still
|
||||
verifies the declaration, not the arithmetic (determinism-tiers §3). A
|
||||
registered vocabulary belongs in the manifest, which is R2. *Status: narrowed at
|
||||
link 02; open.*
|
||||
|
||||
**G4: No normative parameter carriage format.** This is deliberate, and it is the
|
||||
question N1809's NNC carriage proposal puts to WG 3. At link 05, prior-carriage
|
||||
registers `wai.prior.nnc` (ISO/IEC 15938-17:2024) as the one form a `Prior` MAY
|
||||
declare, optional and not recommended, and registers `wai.tensor.nnc` as a
|
||||
candidate payload whose coding is in an unpublished amendment. prior-carriage §4
|
||||
tracks that amendment from MPEG's 154th-meeting press release; its text,
|
||||
including the data-unit signing it adds, was not read (⚠ unverified). *Status:
|
||||
a form registered; carriage remains sink-supplied.*
|
||||
|
||||
**G5: No update semantics.** WAI has no field relating one parameter set to
|
||||
another (R9). prior-carriage §2 forbids pinning an NNC incremental-update
|
||||
bitstream on its own and registers no form for a base-plus-updates chain.
|
||||
*Status: open.*
|
||||
|
||||
**G6: On `main`, malformed-stream behaviour depends on the build profile.** The
|
||||
rANS bypass escape in `wai-rs/src/rans.rs` runs this loop:
|
||||
`raw_val |= v << (j * BYPASS_PRECISION)`. It shifts a `u32` by a
|
||||
stream-controlled amount. The shift panics in a debug build and is masked in a
|
||||
release build. The `wai-rs` release profile does not enable `overflow-checks`.
|
||||
|
||||
- **Probe:** a scratch harness `#[path]`-included `rans.rs` at commit 1b3e390
|
||||
and decoded 20,000 random 64-byte streams (SplitMix64, seeds 0–19,999)
|
||||
against a three-entry CDF chosen to force the escape, four symbols each. It
|
||||
was built in both profiles on the dev machine (macOS arm64, rustc 1.98.0,
|
||||
release with opt-level 3, LTO and one codegen unit, matching `wai-rs`).
|
||||
- **Result:** 18,015 streams returned values in release and panicked with
|
||||
"shift left with overflow" in debug. 1,984 decoded identically, and one
|
||||
panicked in both for different reasons. No stream produced two different
|
||||
values.
|
||||
- **Scope:** random input, not conformant streams. The well-formed corpora are
|
||||
unaffected. The point is R6: without a specified error behaviour, "bit exact"
|
||||
stops at well-formed input.
|
||||
- **Status: fixed at link 01 (PR #82) for this escape.** `rans.rs` bounds the
|
||||
run of bypass groups at what a 32-bit value can hold, so a malformed stream
|
||||
decodes one way in both profiles, and every clamp is a no-op on a well-formed
|
||||
stream. SPEC.md §7 (*Machine-checked totality*, informative) records four
|
||||
defects in the escape found by bounded model checking and fixed, with the
|
||||
conformance vectors unchanged. `wai.neural.int_mlicpp` (link 09) refuses every
|
||||
stream value it cannot hold exactly. The fix clamps rather than rejects, and
|
||||
no normative text specifies the outcome of a malformed stream, so R6 is not
|
||||
met as written.
|
||||
|
||||
**G7: At least three digest algorithms for decoded output.**
|
||||
|
||||
- The `int_synth` corpus uses a domain-prefixed BLAKE3
|
||||
(`wai_int_synth_vectors`).
|
||||
- The `int_synth` byte-equality golden uses SHA-256 over the RGB output
|
||||
(`byte-exact-conformance/src/golden.rs`). Reconstruction-binding also uses
|
||||
SHA-256, because C2PA validators support it.
|
||||
- The `wai.video.int_motion` and `wai.video.int_hyper` goldens, including the
|
||||
learned keyframe, use a 64-bit FNV-1a. It is not a cryptographic hash
|
||||
(`int_video_golden.rs`, `neural_keyframe_golden.rs`).
|
||||
- At link 09, the `wai.neural.int_mlicpp` vector pins its RGB under both FNV-1a
|
||||
and SHA-256, and its bucket and symbol sequences under FNV-1a.
|
||||
|
||||
A CI golden need not be cryptographic to detect an accidental divergence. A
|
||||
conformance digest that a receipt or a third party relies on should be. A
|
||||
contribution must name one algorithm per purpose (R7), and for conformance
|
||||
material that others rely on, the one named should not be FNV-1a. *Status:
|
||||
open.*
|
||||
|
||||
**G8: Tests that need model assets pass when the assets are absent, and some
|
||||
capabilities have no CI.** On `main` these tests print a skip message and return
|
||||
success when their assets are absent. What CI supplies differs between them:
|
||||
|
||||
- **Not run, or run without their assets, in CI on `main`:**
|
||||
- the whole-decode tests of WAI's JPEG AI decoder, including
|
||||
`jpegai_exact_bundle_decodes` in `neural/mod.rs`, which needs the `neural`
|
||||
feature;
|
||||
- the full-size integer-model tests (`tests/int_hyper_synth.rs`,
|
||||
`tests/int_video_sota_keyframe.rs`), which no workflow invokes;
|
||||
- the JPEG AI synthesis tests on real weights (`jpegai_synthesis.rs`). They
|
||||
run in `cargo test --lib` and skip there, because no weights are supplied;
|
||||
- the rest of `tests/neural_native_decode.rs`, including its tests on
|
||||
exported or trained models. CI runs only the tests the job's
|
||||
`_matches_reference` filter selects. `snac_real_model_matches_reference`
|
||||
matches that filter but has no generated fixture, so it skips.
|
||||
- **Defined in `wai-neural-e2e.yml` with synthetic fixtures, but never run
|
||||
there:** the float `_matches_reference` tests for ELIC, MLIC++, SNAC, DCVC
|
||||
and GLC in `tests/neural_native_decode.rs` (§2). They would check
|
||||
correctness at ±4 against minimal synthetic models on one leg, not at
|
||||
realistic scale and not across architectures.
|
||||
- **No CI coverage of any kind on `main`:** `wai.audio.int_codec` and
|
||||
`wai.splat.int_codec`. Their features (`audio_int`, `splat_int`) appear in no
|
||||
workflow, and `byte-exact-conformance` does not include their sources. Yet
|
||||
the registry classes both `NeuralIntegerExact`.
|
||||
|
||||
In the tree at link 10:
|
||||
|
||||
- **Link 06 (PR #107).** With `WAI_REQUIRE_SOTA_VECTORS=1` a missing or
|
||||
incomplete input fails the headline tests instead of passing; without it the
|
||||
skip is written to stderr. `wai-sota-vectors.yml` regenerates the full-size
|
||||
`int_hyper` vectors from pinned inputs and runs both full-size tests.
|
||||
`wai-jpegai-e2e.yml` fetches pinned JPEG AI inputs, checks them by SHA-256, and
|
||||
runs the whole-decode, reconstruction-binding and cross-platform tests. Both
|
||||
fail on a skip banner. Still in no workflow: `jpegai_exact_bundle_decodes`,
|
||||
`jpegai_live`, and the synthesis tests on real weights.
|
||||
- **Link 07 (PR #108).** `wai-conformance.yml` decodes the committed
|
||||
`wai.audio.int_codec` and `wai.splat.int_codec` vectors against their frozen
|
||||
hashes on every leg, under every kernel policy (`tests/int_kernels_vectors.rs`).
|
||||
They are still outside the byte-exact golden suite, and the comparisons with
|
||||
the Python reference in `int_audio.rs` and `int_splat.rs` still run in no
|
||||
job. determinism-tiers §6 said until link 11 that no CI job enables their
|
||||
features; link 11 corrects it.
|
||||
- **PR #101 (independent of the chain).** Makes the neural end-to-end job link
|
||||
and fail on a missing fixture, and fixes the MLIC++ fixture mismatch (§2). At
|
||||
link 10 the job is unchanged and still cannot link.
|
||||
|
||||
What does run without skipping, on `main`, is the small committed integer
|
||||
models: the `int_synth` corpus, and the `int_hyper` keyframe through a small
|
||||
committed model (§2). For JPEG AI on `main`, the entropy stage is the only part
|
||||
with a cross-architecture golden; its parsing, dequantisation and post-filter
|
||||
kernels have closed-form unit tests in `cargo test --lib` on every leg of the
|
||||
matrix, which since 2026-08-28 means three of the four. A conformance claim must
|
||||
cite only the tests that run, on the legs that ran. *Status: addressed at links
|
||||
06 and 07. On the forge, `wai-sota-vectors.yml` and `wai-jpegai-e2e.yml` have
|
||||
passed on every run since 2026-09-28, and the conformance workflow runs the
|
||||
link-07 vectors on its unix legs (runs read 2026-09-30). The neural end-to-end
|
||||
job passes only with PR #101.*
|
||||
|
||||
**G9: No throughput figure.** On `main` the integer decoders are scalar and
|
||||
single-threaded, and no decode-speed figure has been measured and published.
|
||||
An argument for integer conformance invites a cost question. *Status: closed at
|
||||
link 07 (PR #108).* `wai-rs/src/int_kernels.rs` adds fast integer kernels (SIMD on
|
||||
aarch64, portable Rust elsewhere, worker threads except on wasm) that give the
|
||||
reference's integer for every output element, and
|
||||
[decode-throughput](decode-throughput.md) §4 publishes whole-decoder and kernel
|
||||
figures measured on one macOS arm64 machine. Its §5 states the limits: one
|
||||
machine and architecture, x86_64 not executed, synthetic weights for the largest
|
||||
rows, and an equivalence that is argued and tested, not machine-proved.
|
||||
|
||||
**G10: WAI's texts word the `replicate` requirement ambiguously, one sentence
|
||||
reads against its own section, and the tiers exist only as text.** This is the
|
||||
gap closest to topic (v). On `main`:
|
||||
|
||||
- SPEC.md §7 lets a neural capability back `intent = replicate` "only if its
|
||||
entire decode path — entropy parameterization *and* synthesis — is integer /
|
||||
fixed-point, with no float operation anywhere".
|
||||
- SPEC.md §7's closing paragraph calls `entropy-consistency` "the floor for a
|
||||
sound neural `replicate`". determinism-tiers §3 says a `replicate` envelope
|
||||
"MUST pin a prior that meets at least `entropy-consistency`".
|
||||
- Read strictly these compose to the stricter rule, but the wording invites the
|
||||
other reading: under determinism-tiers §2, integer entropy arithmetic with
|
||||
float synthesis reaches `entropy-consistency`, and a reader who takes "at
|
||||
least" as sufficient would admit a `replicate` claim that SPEC.md §7 forbids.
|
||||
- determinism-tiers §3 says "a float neural decode meets neither tier", which,
|
||||
read literally, excludes the explicit-parameter-transmission route that §2
|
||||
counts as reaching `entropy-consistency` (§5 here).
|
||||
- `wai-rs` has no type for either tier. `ReplicateClass` is a different
|
||||
classification with no entropy-stage member.
|
||||
|
||||
*Status: closed at link 02 (PR #103), on the strict reading.* SPEC.md §7 and
|
||||
determinism-tiers §2–§3 state one position: over a neural capability `replicate`
|
||||
promises `decode-equivalence`, admitted only by the integer construction;
|
||||
`entropy-consistency` is "necessary … and not sufficient" and MUST NOT back it.
|
||||
determinism-tiers §3's sentence now separates a decode that derives its entropy
|
||||
parameters in float (neither tier) from one that reads them from the codestream
|
||||
or derives them in integer arithmetic (`entropy-consistency`). The registry gains
|
||||
an entropy-stage class, `NeuralEntropyExact`, and a tier type,
|
||||
`codecs::DeterminismTier`. Still open: R5 (post-decode processing) is specified
|
||||
and not implemented.
|
||||
|
||||
**G11: `wai.image.jpegai`'s normative registration, its registry class and
|
||||
its decode paths disagree.** On `main`:
|
||||
|
||||
- SPEC.md §5 registers `wai.image.jpegai` as an "integer entropy front end
|
||||
(mANS) + integer dequant/value path + fixed-order synthesis and post-filters",
|
||||
at **decode-equivalence**, "reproducible on every machine", and says "entropy
|
||||
and dequant are integer-exact". determinism-tiers §2–§3 and
|
||||
reconstruction-binding §1 say the same.
|
||||
- `capability_replicate_class` has no arm for it, so it falls to
|
||||
`StandardDefined`, the class for a reconstruction "fixed by the named
|
||||
codec/standard", and a registry test asserts this. JPEG AI fixes a bit-exact
|
||||
conformance point only at the end of entropy decoding (§5), so the standard
|
||||
does not fix this reconstruction.
|
||||
- The preferred path's dequantisation is floating point (`jpegai_pipeline.rs`):
|
||||
the residual-variance stage multiplies and scales in `f64` and rounds to
|
||||
`f32`, and the gain stage multiplies by the reciprocal of a gain computed with
|
||||
an `f64` `exp()` from the platform maths library, which has no cross-platform
|
||||
rounding guarantee. LSBS refinement, the conversion into the integer networks,
|
||||
the post-filter chain (including the `f32` eICCI network) and colour
|
||||
conversion are float too. The whole decode does not run in CI.
|
||||
- `try_jpegai_exact` returns `None`, and the decode falls back to float ONNX
|
||||
inference, when the integer bundle's directory or `norm.json` is absent, a
|
||||
weight file cannot be read or parsed, or the codestream is outside the
|
||||
implemented configuration. Both paths return the same `Decoded::Image`.
|
||||
|
||||
*Status: closed at link 02 (PR #103) and link 03 (PR #104), by bringing the texts
|
||||
and the class down to what the code does.* SPEC.md §5 registers the capability
|
||||
at `entropy-consistency` (registry class `NeuralEntropyExact`), describes
|
||||
dequantisation, latent refinement, post-filters and colour conversion as float
|
||||
in a fixed evaluation order, and records the downgrade. determinism-tiers §5 and
|
||||
reconstruction-binding §1–§2 make the pinned reconstruction a property of the
|
||||
exact decoder, reproducible under that decoder's build and floating-point
|
||||
conditions, which it checks before decoding. Link 03 replaces the platform
|
||||
`exp` and roundings with `float_repro`, built from IEEE-754 basic operations,
|
||||
guards the decode path against maths-library calls, and pins nine hashes for one
|
||||
codestream; determinism-tiers §6 records them identical on four platforms and not
|
||||
measured on Windows, macOS x86_64 or x86_64 hardware. `wai-jpegai-e2e.yml`
|
||||
defines the whole decode in CI (G8). What remains is stated in the texts, not
|
||||
contradicted by them: the float fallback exists and the result does not record
|
||||
which path ran, and a `norm.json` with a missing or mistyped key panics in the
|
||||
loaders instead of falling back (the `try_jpegai_exact` doc comment says so).
|
||||
|
||||
**G12: No normative behaviour for a parameter-set mismatch, and a
|
||||
per-rendition pin the code carries but never checks.** UC-2 sets out the
|
||||
detail.
|
||||
|
||||
- *Specification:* SPEC.md §4 dispatches on the capability string alone, so a
|
||||
sink holding the wrong parameter set must decode. `model_requirement.fallback`
|
||||
fires only for a missing capability. SPEC.md §7.1 defines no prior on a
|
||||
rendition entry.
|
||||
- *Reference implementation:* a `WAI1` pin is checked, and a mismatch is
|
||||
refused (`PriorMismatch`) with no declared alternative. A `WAI2` rendition
|
||||
already carries its own pin (`RenditionMeta.prior`), on the wire and through
|
||||
the C FFI. But `pick()` and `primary()` ignore it, and no path verifies it.
|
||||
|
||||
So the data for "a rendition keyed to a prior" exists in code. What is missing
|
||||
is the text that defines it, and any behaviour that uses it. Closing this needs
|
||||
these together:
|
||||
|
||||
- the pin defined in SPEC.md, including the per-rendition form (G1);
|
||||
- dispatch and rendition selection that consult the pin: a sink picks the
|
||||
rendition whose pin matches a parameter set it holds, and verifies that pin
|
||||
before decoding, as `resolve_checked` does for `WAI1`;
|
||||
- a refusal rule for when no pin matches;
|
||||
- the predecessor link R9 asks for (G5), so that a sink holding A can tell
|
||||
that B replaces it.
|
||||
|
||||
*Status: open; unchanged at link 10.*
|
||||
|
||||
**G13: Membership.** See the note at the top.
|
||||
|
||||
---
|
||||
|
||||
## 7. Adjacent work at the same layer
|
||||
|
||||
- **MPEG-AI Part 6** (ISO/IEC 23888-6): CfP responses are due 13 Jan 2027.
|
||||
Model identity under provenance is shared ground; see
|
||||
[mpeg-ai-part6-mapping](mpeg-ai-part6-mapping.md).
|
||||
- **NNC** (ISO/IEC 15938-17): a carriage proposal is in SyfAI (N1809 §3). At
|
||||
link 05 WAI registers the 2024 edition as the one form a `Prior` MAY declare,
|
||||
and fixes how an in-band data-unit signature relates to WAI's digest
|
||||
([prior-carriage](../extensions/prior-carriage.md) §3, §6). Landscape §6.4
|
||||
covers the rest.
|
||||
- **VCM carriage** (the main body of N1809): WAI registers no VCM capability,
|
||||
so this is not a mapping target.
|
||||
- **UWA〔2026〕042**, 《AI超高清媒体封装与传输协议》 ("AI ultra-high-definition
|
||||
media encapsulation and transport protocol"): the project was approved on
|
||||
18 Jun 2026 and is at pre-draft. It sits at the same layer. Watch it; there
|
||||
is nothing to map yet.
|
||||
- **JPEG AI Parts 3 and 4** (reference software, conformance): a web search on
|
||||
2026-09-27 returned a catalogue snippet placing Part 4 at stage 60.00, under
|
||||
publication. ⚠ The ISO catalogue refused automated access, so neither part's
|
||||
stage was confirmed from the catalogue itself. Part 4 defines the conformance
|
||||
tests an entropy-stage claim is checked against, and SPEC.md §7 at link 02
|
||||
records that WAI's classing of `wai.image.jpegai` waits on a reading of it.
|
||||
- **IEEE 1857.11-2024**: published. Its conformance approach is described in §5.
|
||||
|
||||
---
|
||||
|
||||
## 8. What this document does not claim
|
||||
|
||||
- It does not claim that a contribution will be made, or that WAI holds
|
||||
SC 29 accreditation. It is a draft, not submitted.
|
||||
- It does not treat the MPEG 155 five-topic list as WG 3 text. That list comes
|
||||
from a secondary report.
|
||||
- It does not describe any WAI mechanism as adopted by, aligned with, or
|
||||
endorsed by WG 3.
|
||||
- It does not present anything attributed to a link of the landing chain as
|
||||
being in `main`. None of the chain is merged as of 2026-09-30.
|
||||
- It does not present the reference implementation's prior pin, digest check or
|
||||
mismatch refusal as core normative WAI text. They are code and a Draft
|
||||
extension (G1).
|
||||
- It does not present the emitted-medium rule as implemented. It is
|
||||
specification text, and the code does not handle post-decode processing
|
||||
(R5, G10).
|
||||
- It does not claim that WAI's JPEG AI decode is float-free, reproducible on
|
||||
every machine as a property of the capability, or checked end to end in CI.
|
||||
In the tree at link 10 its texts say the same (G11, G8).
|
||||
- It does not claim that WAI specifies a fallback for a parameter-set
|
||||
mismatch. SPEC.md's dispatch rule requires the opposite (UC-2, G12). The
|
||||
reference implementation's per-rendition pin is data that nothing checks.
|
||||
- It does not claim a four-platform CI pass for `main`. The `windows` leg has
|
||||
not run since about 2026-08-28, and the float decode tests have never run in
|
||||
CI (§2). It does not claim any CI result for the chain's pull requests.
|
||||
- It does not claim that WAI's integer route is the only sound one. §5 states
|
||||
what each route reaches.
|
||||
|
||||
---
|
||||
|
||||
## Sources
|
||||
|
||||
Read on 2026-09-27 unless noted.
|
||||
|
||||
1. MPEG Explorations, Part 52: document table and titles.
|
||||
<https://www.mpeg.org/standards/Explorations/52/>
|
||||
2. ISO/IEC JTC 1/SC 29/WG 3 **N1809** (w26201), *Exploration on Systems
|
||||
technology for AI-based media standards (SyfAI)*, 2026-05-02. Read in full
|
||||
from <https://www.mpeg.org/wp-content/uploads/mpeg_meetings/154_SantaEularia/w26201.zip>.
|
||||
3. ISO/IEC JTC 1/SC 29/WG 3 **N1752** (w25953), *Exploration on Systems
|
||||
technologies for coding for machine standards*, 2026-02-09. Read in full from
|
||||
<https://www.mpeg.org/wp-content/uploads/mpeg_meetings/153_OnLine/w25953.zip>.
|
||||
4. MPEG meeting schedule. <https://www.mpeg.org/meetings/>
|
||||
5. *MPEG news: a report from the 155th meeting*, 31 Jul 2026 (secondary; the
|
||||
AI update framework wording, and that the thread was opened jointly with
|
||||
WG 7).
|
||||
<https://multimediacommunication.blogspot.com/2026/07/mpeg-news-report-from-155th-meeting.html>
|
||||
6. *An Overview of the JPEG AI Learning-Based Image Coding Standard*, arXiv
|
||||
2510.13867. The PDF was read for the conformance-point, integer-pipeline and
|
||||
excluded-features passages. <https://arxiv.org/abs/2510.13867>
|
||||
7. *MLVC: Multi-platform Learned Video Codec for Real-World Deployment*, arXiv
|
||||
2606.28027, ECCV 2026. Abstract page only.
|
||||
<https://arxiv.org/abs/2606.28027>
|
||||
8. IEEE 1857.11 call for proposals, the outcome of the 9 Dec 2021 working
|
||||
subgroup meeting; its timeline lists the announcement on 21 Dec 2021 (the
|
||||
URL is dated 24 Dec 2021). Read through a text rendering, because the page
|
||||
serves an interstitial to scripted clients.
|
||||
<https://sagroups.ieee.org/dcsc/2021/12/24/call-for-proposals-on-neural-network-based-image-coding/>
|
||||
IEEE 1857.11-2024 record (Board approval 2024-09-26, published 2024-12-20):
|
||||
<https://standards.ieee.org/ieee/1857.11/10691/>
|
||||
9. UWA project approval notice, UWA〔2026〕042. <https://theuwa.com/new-detail/899>
|
||||
10. In this tree, at link 10 of the landing chain (2026-09-30), and on `main`
|
||||
where a statement says so: `SPEC.md` opening section, §1, §3, §4, §5 (the
|
||||
`wai.image.jpegai`, `wai.neural.int_mlicpp` and `wai.prior.nnc` rows), §7
|
||||
(including *Replicate-soundness*, *`wai.image.jpegai` and the standard
|
||||
classes* and *Machine-checked totality*), §7.1 (including *Companion
|
||||
components*) and Appendix A; `extensions/determinism-tiers.md` §1–§6;
|
||||
`extensions/prior-carriage.md` §1–§7; `extensions/reconstruction-binding.md`
|
||||
§1–§2; `extensions/jwp-receipts.md` §2.1; `extensions/transparency.md` §6;
|
||||
`research/2030-transport-landscape.md` §3.2, §6.4 and §8 item 9;
|
||||
`research/decode-throughput.md` §3–§5; `wai-rs/src/container.rs` (including
|
||||
`Prior`, `RenditionMeta`, `Rendition`, `pick`, `primary`,
|
||||
`dispatch_capability` and the `multi_rendition_round_trip` test);
|
||||
`wai-rs/src/ffi.rs` (the `WAI2` pack and unpack calls); `wai-web/src/lib.rs`
|
||||
(its `WAI2` binding); `wai-rs/src/codecs/mod.rs` (`ReplicateClass`,
|
||||
`DeterminismTier`, `capability_replicate_class`); `wai-rs/src/int_entropy.rs`;
|
||||
`wai-rs/src/int_kernels.rs`; `wai-rs/src/int_mlicpp.rs` (module
|
||||
documentation); `wai-rs/src/ort_session.rs`; `wai-rs/src/neural/mod.rs`,
|
||||
`hyperprior.rs`, `context.rs`, `glc.rs`, `video_dcvc.rs`;
|
||||
`wai-rs/src/jpegai_decode.rs`, `jpegai_synthesis.rs`, `jpegai_pipeline.rs`,
|
||||
`jpegai_icci.rs`; `wai-rs/src/rans.rs`; `wai-rs/src/bin/wai.rs`;
|
||||
`wai-rs/tests/neural_native_decode.rs`, `int_kernels_vectors.rs`;
|
||||
`tools/wai_validate_synthetic.py`; `byte-exact-conformance/src/`; the WAI
|
||||
workflow definitions. The G6 probe ran against `rans.rs` at
|
||||
commit 1b3e390.
|
||||
11. The forge's Actions records for this repository, read on 2026-09-27:
|
||||
- the run listing (`/api/v1/repos/{owner}/{repo}/actions/runs`), giving
|
||||
every run of the three WAI workflows from 2026-07-23 to 2026-09-27 with
|
||||
status, commit, branch and times;
|
||||
- the per-job status of each of those runs, from the web interface's job
|
||||
endpoint;
|
||||
- the job logs: every failed `wai-neural-e2e.yml` run, the three
|
||||
non-`windows` legs of both jobs in the `afc7ece` run, and the
|
||||
byte-equality legs of one 2026-09-21 run. A job that was never claimed
|
||||
has no log.
|
||||
- the private network's peer status, read from the dev machine: the
|
||||
`windows` runner's host was offline and last seen 30 days before
|
||||
2026-09-27.
|
||||
12. PR #101 (independent of the landing chain), read from its commit message
|
||||
on 2026-09-30: the neural end-to-end job's link failure, the fixture
|
||||
generator's thread settings, and the MLIC++ trace (the diverging symbol and
|
||||
the max |Δ| before and after); and its forge runs, read on 2026-09-30:
|
||||
`wai-neural-e2e.yml` passed on its head on 2026-09-29.
|
||||
13. The forge's CI runs for the chain's pull requests (#82, #103–#109, #111,
|
||||
#112) since 2026-09-28, read on 2026-09-30 through the forge's task list.
|
||||
|
|
@ -166,8 +166,9 @@ impl Uncertainty {
|
|||
/// Whether an on-chip counter was adding deliberate noise to its readings.
|
||||
///
|
||||
/// Some counters filter their output as a side-channel defence. On Intel
|
||||
/// processors (INTEL-SA-00389) enabling SGX enables the filter in most cases,
|
||||
/// and TDX uses the filtered energy information; software with MSR access can
|
||||
/// processors (the technical guidance for INTEL-SA-00389) filtered energy is
|
||||
/// reported when SGX is enabled or system software has enabled filtering, and
|
||||
/// TDX uses the filtered energy information; software with MSR access can
|
||||
/// read it as `IA32_MISC_PACKAGE_CTLS[0]` (`ENERGY_FILTERING_ENABLE`, which
|
||||
/// cannot be cleared once set), supported when `IA32_ARCH_CAPABILITIES[11]` is
|
||||
/// set. An unprivileged reader usually cannot tell, and says so.
|
||||
|
|
|
|||
Loading…
Reference in a new issue