wai: register the byte-exact JPEG-AI decoder + restore the byte-equality matrix #50

Merged
dcharlot merged 6 commits from wai-jpegai-integrate into main 2026-08-01 05:43:10 -04:00
14 changed files with 586 additions and 19 deletions

View file

@ -36,7 +36,7 @@ A container + capability-dispatch standard for media. WAI does not re-implement
- **Site:** [wai.transaction.science](https://wai.transaction.science)
- **Licence:** Apache-2.0 (see [`wai/LICENSE`](wai/LICENSE))
- **Status:** v1.0 draft standard. `wai-rs/` reference implementation (Rust lib + cdylib + staticlib, C FFI) passes 11 capability + envelope round-trip tests. Known consumer: CommunicationOS.
- **Status:** v1.0 draft standard. `wai-rs/` reference implementation (Rust lib + cdylib + staticlib, C FFI) round-trips every registered capability through the full envelope. Known consumer: CommunicationOS.
### [jouleclaw/](jouleclaw/) — Energy-optimised AI runtime

View file

@ -6,6 +6,52 @@ wire format may change and such changes are recorded here.
## v0.4.0 — current
### Reproducible-reconstruction binding — a C2PA/JPEG-Trust hard binding over pixels
- New `wai.*` provenance capability: `reconstruction_binding` (feature
`provenance`) + [`extensions/reconstruction-binding.md`]. A cryptographic hard
binding over the *deterministically decoded* pixels of a neural-coded asset —
which such media normally can't carry, because a neural decode isn't
reproducible (JPEG AI's normative conformance point is the entropy-decoder
output, not pixels; its multi-branch design reconstructs one codestream several
ways). WAI's byte-exact decode restores a portable decoded-pixel hash. SHA-256
(the C2PA default) over the canonical RGB8 buffer plus a descriptor pinning the
reproducible decode; carried as a custom C2PA/JPEG-Trust assertion
(`science.transaction.wai.hash.reconstruction`) beside the standard byte-binding.
A substrate for machine-readable AI-content marking (e.g. EU AI Act Art. 50).
### `wai.image.jpegai` now dispatches to the byte-exact decoder
- `decode_envelope` prefers the deterministic `jpegai_decode::decode` when the
model directory ships the integer weight bundle (`weights/*.json` + `norm.json`),
falling back to the ONNX-runtime decode otherwise — existing bundles keep
working. `wai.image.jpegai` is now registered in SPEC §5 (it was a live
capability missing from the normative menu).
- The cross-arch byte-equality matrix now covers the merged decoder's entropy
stage (`byte-exact-conformance`), and that crate's build — broken on a moved
`rans.rs` include — is restored.
### JPEG AI — a byte-exact codestream→RGB decoder (`jpegai_decode::decode`)
- `wai::jpegai_decode::decode(codestream, &DecoderWeights, &NormTables) -> Rgb`
reconstructs an ISO/IEC 6048 JPEG-AI codestream to RGB through a fully
integer entropy front end and a deterministic value/synthesis/post-filter
path — one pinned reconstruction, reproducible byte-for-byte on every machine,
so the decoded pixels have a portable hash. No per-image fixtures: every
per-image quantity is derived from the bitstream; only checkpoint constants
come from a sink-supplied model bundle.
- Determinism profile: the entropy decode (mANS), the dequant, and the derived
entropy inputs (skip mask, σ index, RVS band, dequant gain) are integer-exact
against the reference; the PTQ value/synthesis stages are deterministic and
land within 1 LSB of the ISO reference decoder. Distinct from the in-house
`wai.neural.int_*` codecs (all-i64, hash-portable): this decodes a published
external standard's codestream deterministically.
- Modules: `jpegai_decode` (the composed entry point) over `jpegai_bitstream`,
`jpegai_mans`, `jpegai_pipeline`, `jpegai_synthesis`, `jpegai_icci`
(feature `neural_int`). Inputs are regenerable from the shipped assets via
`tools/wai_jpegai_build_e2e_inputs.py`. The whole chain — codestream → RGB —
is exercised end to end against the reference PNG.
### JPEG AI — a live, self-sufficient `wai.image.jpegai` decode (2026-06-22)
- `neural::jpegai::decode(codestream, &JpegAiModel)` decodes a real ISO/IEC 6048

View file

@ -23,7 +23,7 @@ The capability a file requires is *named*, not supplied.
|------|----------|
| [`SPEC.md`](SPEC.md) | WAI v1.0 spec — envelope, manifest, capability menu |
| [`wai-rs/`](wai-rs/) | Rust reference implementation (lib + cdylib + staticlib + C FFI) |
| [`glossary/`](glossary/) | Reference catalog of media codecs surveyed for capability registration. [`codecs.json`](glossary/codecs.json) is the canonical 342-entry dataset; [`SCHEMA.md`](glossary/SCHEMA.md) defines the shape and contribution rules. Rendered at [wai.transaction.science/glossary](https://wai.transaction.science/glossary). |
| [`glossary/`](glossary/) | Reference catalog of media codecs surveyed for capability registration. [`codecs.json`](glossary/codecs.json) is the canonical survey dataset; [`SCHEMA.md`](glossary/SCHEMA.md) defines the shape and contribution rules. Rendered at [wai.transaction.science/glossary](https://wai.transaction.science/glossary). |
| [`corpus/`](corpus/) | Standard test material (Kodak/CLIC/Tecnick/SIPI images, Xiph Derf + UVG video, SQAM-class audio, enwik8 + Silesia text). Not committed; `CORPUS.md` documents reacquisition. |
## Quickstart

View file

@ -180,9 +180,9 @@ decode route for each is implemented in the reference sink (see the
| capability | model | win vs zeroth | source |
|---|---|---|---|
| `wai.neural.encodec32` | Meta EnCodec at 32 kHz, 4 codebooks | ~3 kbps acceptable music | `facebook/encodec_32khz` |
| `wai.neural.dac` | Descript Audio Codec at 44.1 kHz | 6 kbps near-transparent music | `descript/dac_44khz` |
| `wai.neural.mimi` | Kyutai Mimi, 12.5 Hz frame rate | ~1.1 kbps real-time speech | `kyutai/mimi` |
| `wai.neural.encodec32` | EnCodec at 32 kHz, 4 codebooks | ~3 kbps acceptable music | `facebook/encodec_32khz` |
| `wai.neural.dac` | DAC at 44.1 kHz | 6 kbps near-transparent music | `descript/dac_44khz` |
| `wai.neural.mimi` | Mimi, 12.5 Hz frame rate | ~1.1 kbps real-time speech | `kyutai/mimi` |
| `wai.neural.wavtokenizer` | WavTokenizer single-codebook VQ | 0.9 kbps beats DAC @ 9 kbps on UTMOS | `novateur/WavTokenizer-large-*` |
| `wai.neural.snac` | SNAC multi-resolution RVQ (hierarchical code levels, own sample rate) | low-bitrate TTS standard | `neural/snac.rs` |
@ -193,11 +193,12 @@ decode route for each is implemented in the reference sink (see the
| `wai.neural.bmshj2018` | bmshj2018-factorized + rANS (byte-exact CompressAI port) | ~80× vs raw RGB at q=3 (~0.26 bpp, 29 dB PSNR) | CompressAI |
| `wai.neural.mbt2018_mean` | mean-scale hyperprior (Gaussian-conditional entropy) | conditional-entropy family, stronger than factorized | `neural/hyperprior.rs` |
| `wai.neural.int_synth` | integer fixed-point synthesis (all-i64 transposed-conv, no float/ONNX) | bit-identical on every machine — soundly backs `intent = replicate` | `neural/int_synth.rs` (QAT weights sink-supplied) |
| `wai.neural.int_hyper` | integer fixed-point **conditional** codec — whole bitstream→image path integer (integer hyper-decoder σ → integer-σ entropy → integer synthesis) | the integer-entropy contract fully assembled; SOTA quality (PTQ of mbt2018_mean) bit-identical on every machine | `int_hyper_synth.rs` (tables sink-supplied) |
| `wai.neural.int_hyper` | integer fixed-point **conditional** codec — whole bitstream→image path integer (integer hyper-decoder σ → integer-σ entropy → integer synthesis) | the integer-entropy contract fully assembled; high quality (PTQ of mbt2018_mean) bit-identical on every machine | `int_hyper_synth.rs` (tables sink-supplied) |
| `wai.image.jpegai` | ISO/IEC 6048 JPEG-AI codestream → RGB — integer entropy front end (mANS) + integer dequant/value path + fixed-order synthesis and post-filters; every per-image quantity is derived from the bitstream, only checkpoint constants are sink-supplied | **decode-equivalence**: one pinned reconstruction of a published-standard codestream to RGB, reproducible on every machine (a portable decoded-pixel hash); entropy and dequant are integer-exact, synthesis reconstructs to within 1 LSB of the reference | `jpegai_decode.rs` (+ `jpegai_{bitstream,mans,pipeline,synthesis,icci}.rs`) |
| `wai.generate.spectral` | deterministic integer GENERATIVE synthesis (`intent=create`) — prompt+seed → integer-LUT spectral composition, no float/ONNX | byte-identical on every machine, pinned to (model, prompt, seed, steps); the create-side of "conformance is a hash" (generation-equivalence) | `generate.rs` (small pinned model bundle) |
| `wai.neural.elic` | checkerboard context model (hyperprior + 2-pass spatial context) | the JPEG-AI / MLIC++ entropy class | `neural/context.rs` |
| `wai.neural.mlicpp` | MLIC++ multi-reference entropy (channel-AR slices + checkerboard + global attention) | −13.39% BD-rate vs VVC/VTM-17 | research code, `neural/mlicpp.rs` |
| `wai.neural.mlicv2` | MLICv2 / MLICv2+ (2025), MetaFormer transform | −20.46% BD-rate vs VTM-17 — current SOTA learned image; shares the mlicpp route | research code, `neural/mlicpp.rs` |
| `wai.neural.mlicpp` | MLIC++ multi-reference entropy (channel-AR slices + checkerboard + global attention) | learned image, low-bitrate high-fidelity | research code, `neural/mlicpp.rs` |
| `wai.neural.mlicv2` | MLICv2 / MLICv2+ (2025), MetaFormer transform | learned-image fidelity frontier; shares the mlicpp route | research code, `neural/mlicpp.rs` |
| `wai.neural.cmic` | Content-Adaptive Mamba (ICLR 2026), SSM transform + SCCTX entropy | learned-image frontier; shares the mlicpp route | arXiv 2508.02192, `neural/mlicpp.rs` |
| `wai.neural.glc` | Generative Latent Coding — categorical / VQ entropy | <0.05 bpp where JPEG-XL collapses | research code, `neural/glc.rs` |
@ -206,9 +207,9 @@ decode route for each is implemented in the reference sink (see the
| capability | model | win vs zeroth | source |
|---|---|---|---|
| `wai.neural.video_bmshj2018` | per-frame bmshj2018 | 1000×+ vs raw RGB, browser-decodable on WebGPU | CompressAI |
| `wai.neural.dcvc_rt` | DCVC-RT (2025), inter-frame with reference-feature propagation | −21% vs VTM; real-time at 1080p/4K | research code, `neural/video_dcvc.rs` |
| `wai.neural.dcvc_rt` | DCVC-RT (2025), inter-frame with reference-feature propagation | real-time neural video at 1080p/4K | research code, `neural/video_dcvc.rs` |
| `wai.neural.dcvc_fm` | DCVC-FM (feature modulation), wide single-model quality range (RGB + YUV) | SOTA-RD neural video; shares the inter-frame route | research code, `neural/video_dcvc.rs` |
| `wai.video.int_motion` | DETERMINISTIC INTEGER inter-frame — I-frame + integer block motion-compensated P-frames + integer-quantized residual over rANS, no float/ONNX; motion is whole-pel or **half-pel** (`mv_precision`), the latter predicting at the half-grid by integer bilinear interpolation (`(a+b+1)>>1`, `(a+b+c+d+2)>>2`) so real footage — which pans a non-integer number of pixels per frame — costs fewer residual bits while staying byte-exact | byte-identical frame-by-frame on every machine (*frame-state-equivalence*, a per-frame portable hash); the temporal extension of the integer contract, vs the float DCVC family that desyncs down the GOP | `int_video.rs` (whole-pel `tools/wai_int_video_encode.py`; half-pel from real footage `tools/wai_int_video_real_encode.py`) |
| `wai.video.int_motion` | DETERMINISTIC INTEGER inter-frame — I-frame + integer block motion-compensated P-frames + integer-quantized residual over rANS, no float/ONNX; motion is whole-pel or **half-pel** (`mv_precision`), the latter predicting at the half-grid by integer bilinear interpolation (`(a+b+1)>>1`, `(a+b+c+d+2)>>2`) so real footage — which pans a non-integer number of pixels per frame — costs fewer residual bits while staying byte-exact | byte-identical frame-by-frame on every machine (*frame-state-equivalence*, a per-frame portable hash); the temporal extension of the integer contract, where a float inter-frame decode desyncs down the GOP | `int_video.rs` (whole-pel `tools/wai_int_video_encode.py`; half-pel from real footage `tools/wai_int_video_real_encode.py`) |
| `wai.video.int_hyper` | NEURAL-KEYFRAME inter-frame — the I-frame is the LEARNED integer codec `int_hyper` decoded byte-exact, the P-frames are integer block-MC residuals on top | a learned keyframe you can put in a receipt + integer inter-frames; whole decode integer, byte-identical everywhere | `int_video::decode_pframes` ∘ `int_hyper_synth::decode_conditional` |
**Sink-architecture note:** the inter-frame neural video codecs (DCVC-RT,
@ -243,6 +244,12 @@ Merkle roots, Ed25519 group signatures, and cross-hop preservation.
See [`extensions/confidentiality.md`](extensions/confidentiality.md) for the
private-channel sub-spec: MLS (RFC 9420) group keys + SFrame (RFC 9605)
object encryption, the byte-level framing behind §6.
See [`extensions/reconstruction-binding.md`](extensions/reconstruction-binding.md)
for the reproducible-reconstruction binding: a C2PA / JPEG-Trust hard binding
over the *deterministically decoded pixels* of a neural-coded asset (SHA-256 over
the canonical RGB plus the descriptor that pins the reconstruction) — the
reconstruct-side companion to `provenance`, applicable to `wai.image.jpegai` and
the byte-exact `wai.neural.int_*` codecs.
### Interactive, temporal & computed content (extensions)
@ -260,12 +267,12 @@ standard. See the linked extension for each.
| `wai.splat.spz` | SPZ (Niantic) | asset, content-addressed; presentation only |
| `wai.audio.scene` | WSCN scene (objects + trajectories) | mixdown-equivalence · [spatial-audio](extensions/spatial-audio.md) |
| `wai.audio.score` | WSCR container (note events) | music as instructions; sink synthesizes, mixdown-equivalence · [score](extensions/score.md) |
| `wai.audio.int_codec` | LEARNED 1-D transform codec — integer transposed-conv synthesis + integer IGDN over a factorized-rANS latent (model + bitstream from `tools/wai_int_audio_export.py`) | **sample-equivalence**: int16 PCM byte-identical on every machine (a portable PCM hash); the audio analog of `wai.neural.int_hyper`, vs the float SoundStream/EnCodec family (`wai.neural.{encodec32,dac,mimi,…}`) that desyncs on ~1e-5 round-off. Pure i64, no float/ONNX (`int_audio.rs`) |
| `wai.audio.int_codec` | LEARNED 1-D transform codec — integer transposed-conv synthesis + integer IGDN over a factorized-rANS latent (model + bitstream from `tools/wai_int_audio_export.py`) | **sample-equivalence**: int16 PCM byte-identical on every machine (a portable PCM hash); the audio analog of `wai.neural.int_hyper`, where a float learned-audio decode desyncs on ~1e-5 round-off. Pure i64, no float/ONNX (`int_audio.rs`) |
| `wai.haptic.signal` | WHAP envelope (intensity/sharpness keyframes) | sample-equivalence · [haptics](extensions/haptics.md) |
| `wai.feed.state` | WWLD, sim `wai.world.sim.feed0` | replay-equivalence · [state-feeds](extensions/state-feeds.md) |
| `wai.avatar.pose` | WWLD, sim `wai.world.sim.skeletal0` | pose-equivalence · [avatars](extensions/avatars.md) |
| `wai.splat.4d` | W4DS container (per-Gaussian keyframed tracks) | frame-state-equivalence · [volumetric](extensions/volumetric.md) |
| `wai.splat.int_codec` | LEARNED Gaussian-splat ATTRIBUTE codec — a learned transform over the `[14,S,S]` attribute image (pos/scale/rot/color/opacity), integer synthesis (int_hyper machinery, 14 channels) over a factorized-rANS latent (`int_splat.rs`; model + bitstream from `tools/wai_int_splat_export.py`) | **splat-equivalence**: the decoded attribute tensor is byte-identical on every machine (a portable cloud hash) — the int_hyper of 3D, vs the float SOG / self-organizing-Gaussian codecs whose decode desyncs. The float rasterizer stays sink-supplied presentation; the *attributes* are the contract. Pure i64, no float/ONNX |
| `wai.splat.int_codec` | LEARNED Gaussian-splat ATTRIBUTE codec — a learned transform over the `[14,S,S]` attribute image (pos/scale/rot/color/opacity), integer synthesis (int_hyper machinery, 14 channels) over a factorized-rANS latent (`int_splat.rs`; model + bitstream from `tools/wai_int_splat_export.py`) | **splat-equivalence**: the decoded attribute tensor is byte-identical on every machine (a portable cloud hash) — the int_hyper of 3D, where a float splat-attribute decode desyncs. The float rasterizer stays sink-supplied presentation; the *attributes* are the contract. Pure i64, no float/ONNX |
| `wai.film.linear` | WFLM container (sealed world + camera track + frame schedule) | film-equivalence · [linear-film](extensions/linear-film.md) |
| `wai.film.reel` | WREL container (sealed world + a sequence of shots = camera track + schedule each) | reel-equivalence (film-equivalence with cuts) · [linear-film](extensions/linear-film.md) |
| `wai.ternary.tt` | WQTT container (balanced-ternary MPS cores — packed trits + integer per-column scales) | a **quantum-inspired** (tensor-network) transform ported from the ternary/MPS fabric with ONE change: the per-column scale is an **integer**, so the whole decode is `i64`/`i128` and the reconstruction is byte-identical on every machine — **reconstruction-equivalence** (a portable recon hash), receiptable. Honest scope: a **provenance** transform, NOT a rate-competitive codec (measured RD is mediocre — `Lossy` tier; see the extension's §5 verdict). Encoder-side float SVD only · [ternary-tt](extensions/ternary-tt.md) |

View file

@ -100,3 +100,37 @@ fn jpegai_mans_decodes_reference_stream_identically() {
}
}
}
/// The SAME reference stream through the **merged byte-exact decoder's** mANS
/// (crate-root `jpegai_mans`, the one `wai.image.jpegai` reconstructs with) — a
/// different, later port with a flat-table / sliced-offset API. It must decode the
/// ISO stream to the identical symbols, so the decoder that actually ships is in the
/// cross-arch byte-equality matrix, not only the older `neural/` port.
#[test]
fn jpegai_exact_mans_decodes_reference_stream_identically() {
use crate::jpegai_mans_exact::{build_decode_transitions, cdf_from_pmf, MansDecoder};
let mut pmf: Vec<Vec<i64>> = Vec::new();
let mut off = 0usize;
for &n in &PMF_PREFIX_LENS {
let mut row = vec![0i64; PMF_WIDTH];
row[..n].copy_from_slice(&PMF_PREFIXES_FLAT[off..off + n]);
off += n;
pmf.push(row);
}
// The flat-table API: build [u32;256] tables from (pmf, cdf) and flatten them.
let tables = build_decode_transitions(&pmf, &cdf_from_pmf(&pmf));
let flat: Vec<u32> = tables.iter().flatten().copied().collect();
let mut dec = MansDecoder::new(&Y_MEMORY, &[Y_OFFSET]);
dec.set_sgm_transitions(&flat, &BOUNDS);
let masks: Vec<bool> = MASKS.iter().map(|&m| m != 0).collect();
let mut r = vec![0i16; EXPECTED_R.len()];
dec.decode_sgm(&SIGMA, &mut r, &masks);
for i in 0..r.len() {
if masks[i] {
assert_eq!(r[i], EXPECTED_R[i],
"byte-exact JPEG-AI mANS symbol {i} diverged on this platform");
}
}
}

View file

@ -17,5 +17,11 @@ pub mod neural;
#[path = "../../wai-rs/src/int_entropy.rs"]
pub mod int_entropy;
// The merged byte-exact JPEG-AI decoder's entropy stage (crate-root, self-contained —
// no shims needed). Included so the cross-arch byte-equality matrix covers the decoder
// that `wai.image.jpegai` actually reconstructs with, not only the older `neural/` port.
#[path = "../../wai-rs/src/jpegai_mans.rs"]
pub mod jpegai_mans_exact;
mod golden;
mod jpegai_golden;

View file

@ -0,0 +1,134 @@
# WAI Extension: Reproducible-Reconstruction Binding
> Status: Draft. A cryptographic **hard binding over the deterministically
> decoded pixels** of a neural-coded asset — the reconstruct-side analogue of
> the [Asset Provenance](provenance.md) extension, and a bridge from WAI's
> byte-exact decode into C2PA / JPEG Trust provenance. Reference impl: the
> `reconstruction_binding` module of `wai-rs` (feature `provenance`).
> Keywords **MUST**, **MUST NOT**, **SHOULD**, **MAY** are RFC 2119/8174.
## 1. Scope and model
A provenance manifest binds itself to an asset with a **hard binding**: a
cryptographic hash over the asset's bytes. For neural-coded media that hash
covers the *compressed* codestream, and it has been the only hard option
available — because a neural decode is normally not reproducible. Two conformant
decoders may produce different pixels, so a hash of the *decoded image* is not
portable across implementations, and provenance over the rendered result has had
to fall back to fuzzy **soft** bindings (perceptual hashes, watermarks), which
are approximate and spoofable.
This is not incidental to the codec. The JPEG AI standard (ISO/IEC 6048 /
ITU-T T.840) defines its normative bit-exact conformance point at the **output
of the entropy decoder**, not at reconstruction to pixels; and its multi-branch
design specifies **up to three synthesis transforms** (`decoderID` 0/1/2), any of
which produces a conformant reconstruction of the same codestream. Reconstruction
to pixels is therefore, by design, **not** required to be bit-exact, and one
codestream may legally decode to different images. A hash of the decoded pixels is
not portable under the base standard.
WAI decodes to RGB **deterministically** — one pinned reconstruction, byte-identical
on every machine (integer entropy and dequant; fixed-order, reproducible synthesis).
Pinning one reconstruction is a determinism guarantee **beyond what the base standard
requires**, and it restores exactly the precondition a hard binding needs: a stable,
portable hash of the decoded pixels. This extension expresses that hash as a
provenance assertion.
A `reconstruction-binding` is **not a content capability**. The asset declares an
ordinary capability (`wai.image.jpegai`, `wai.neural.int_hyper`, …) and is carried
and decoded normally; the binding is metadata *about* the decoded result, the way
[`provenance`](provenance.md) is metadata about the carried bytes.
## 2. The binding
A binding is a hash plus the descriptor that makes the decode reproducible:
```json
{
"alg": "sha256",
"hash": "<lowercase hex SHA-256 over the canonical decoded pixels>",
"reconstruction": {
"decoderId": 0, // the pinned synthesis branch (multi-branch codecs)
"profile": "<id>", // codec profile
"level": "<id>", // codec level
"width": 256,
"height": 256,
"format": "RGB8", // canonical: row-major, 3 bytes/pixel, no padding, no alpha
"colorSpace": "bt709"
}
}
```
- **Hash algorithm** — `sha256` (the C2PA default; every C2PA validator supports
it). MUST be an algorithm in the C2PA hash-algorithm registry.
- **Canonical buffer** — the hash is computed over the decoded image as a `RGB8`
buffer: row-major, three bytes per pixel, no row padding, no alpha, exactly
`width * height * 3` bytes. No other pre-processing.
- **The descriptor is part of the binding.** A multi-branch codec decodes one
codestream several ways; a verifier MUST decode under exactly the descriptor's
`decoderId`, `profile`, and `level` before hashing, or the comparison is
meaningless. The descriptor pins the one reconstruction the hash is over.
Conformance criterion: **decode-equivalence** — a verifier that decodes the
codestream with a WAI decoder under the binding's descriptor, hashes the canonical
`RGB8` buffer, and compares MUST obtain the recorded `hash`.
## 3. C2PA / JPEG Trust carriage
C2PA (and JPEG Trust, ISO/IEC 21617-1, which is built on the C2PA architecture)
has **no standard assertion that hashes decoded/rendered pixels** — every standard
hard binding (`c2pa.hash.data`, `c2pa.hash.boxes`, `c2pa.hash.bmff`) hashes the
compressed bytes. A reproducible-reconstruction binding is therefore expressed as a
**custom assertion**, using C2PA's reverse-domain labelling for third-party
assertions:
- **Assertion label:** `science.transaction.wai.hash.reconstruction`
- **Body:** the object in §2, serialized as a CBOR map (the C2PA assertion
serialization). Field names map 1:1 to the JSON above:
`alg` (text), `hash` (byte string), and `reconstruction` (a map of
`decoderId` (uint), `profile`/`level`/`format`/`colorSpace` (text),
`width`/`height` (uint)).
- **Placement:** this assertion is **additional**. A manifest SHOULD still carry a
standard hard binding (`c2pa.hash.data` over the codestream) referenced by the
claim's `created_assertions`, so it is valid in any C2PA validator; the
reconstruction binding sits **beside** it. The byte-binding proves *these
compressed bytes*; the reconstruction binding upgrades that to *and anyone who
decodes them per this descriptor obtains this exact image*.
- **Signature:** the assertion is covered by the C2PA claim signature like any
other, so it is tamper-evident.
The digest is the same value WAI's own receipt carries (§4), so a validator with a
WAI decoder reaches a matching hash whether it reads the C2PA CBOR assertion or the
WAI receipt.
## 4. WAI receipt carriage
The binding MAY be carried in a WAI receipt alongside an [asset
provenance](provenance.md) receipt: provenance binds the opaque asset bytes
(content-hash-equivalence, BLAKE3 over the envelope); this binds the reconstructed
pixels (decode-equivalence, SHA-256 over the canonical RGB). Together they cover
both halves of the thesis — the bytes that are carried and the pixels that are
reconstructed. The reference impl emits the §2 JSON via
`ReconstructionBinding::to_json`.
## 5. Machine-readable AI-content marking
A reproducible-reconstruction binding is a substrate for **machine-readable marking
of AI-processed media**. Where a regime requires content to be marked in a
machine-readable, detectable format (for example EU AI Act Article 50, whose
transparency obligations apply from 2 August 2026, with the technical marking
standards developed via the Code of Practice), a signed assertion that binds the
*reconstructed* content — carried through the neural coding step rather than
invalidated by it — is a mechanism that survives decode. WAI states this as a
capability (a signed, reproducible binding over decoded content); it does not make
a legal claim, and any deployment MUST confirm sufficiency against the applicable
regime.
## 6. Reference implementation
`wai-rs`, feature `provenance`, module `reconstruction_binding`:
`ReconstructionDescriptor`, `ReconstructionBinding::{bind, verify, hash_hex,
to_json, C2PA_LABEL}`. `bind` hashes the canonical `RGB8` buffer under a
descriptor; `verify` re-hashes a freshly decoded buffer and compares. The digest is
a plain SHA-256, reproducible by any SHA-256 implementation — which is what makes it
portable into a C2PA validator.

View file

@ -130,7 +130,7 @@ score = ["world"]
# Asset provenance (extensions/provenance): a signed, energy-accounted
# lineage receipt for opaque captured content WAI does not reconstruct.
# Reuses the JWP Merkle + Ed25519; no codec/ML deps.
provenance = ["world"]
provenance = ["world", "dep:sha2"]
# wai.world.model — generative-world transport: a signed, replay-equivalent action
# trace (reuses the world CausalOp log) + per-frame provenance. The neural sibling
# of wai.world.replay; the renderer is a declared world model (sink-supplied, or the

View file

@ -22,7 +22,7 @@ pub const CAP_IMAGE_PNG: &str = "wai.image.png"; // universal floor
pub const CAP_IMAGE_JPEG: &str = "wai.image.jpeg"; // most-supported lossy
pub const CAP_IMAGE_AVIF: &str = "wai.image.avif"; // modern lossy (AV1-based)
pub const CAP_IMAGE_JXL: &str = "wai.image.jxl"; // JPEG-XL (lossless + lossy)
pub const CAP_IMAGE_JPEGAI: &str = "wai.image.jpegai"; // JPEG AI (ITU-T T.840.1 / ISO/IEC 6048-1:2025, royalty-free baseline, ~30% over conventional SOTA) — standardized neural image codec; decoder library not yet in this crate (sink/library-supplied)
pub const CAP_IMAGE_JPEGAI: &str = "wai.image.jpegai"; // JPEG AI (ITU-T T.840.1 / ISO/IEC 6048-1:2025) standardized neural image codec. Byte-exact decoder in-crate: codestream→RGB via jpegai_decode::decode (integer entropy/dequant, deterministic synthesis, ±1 LSB of the reference). Model weights sink-supplied.
// Audio
pub const CAP_AUDIO_OPUS: &str = "wai.audio.opus"; // modern lossy
pub const CAP_AUDIO_FLAC: &str = "wai.audio.flac"; // lossless

View file

@ -296,6 +296,44 @@ impl IcciNet {
let v_o = head(&self.conv_last_v, &v16, self.scale_v);
(y_o, u_o, v_o)
}
/// Parse the whole eICCI net from the `icci` weight export
/// (`{luma, chroma, conv_last_v, scale_v}`, produced by
/// `tools/wai_jpegai_export_weights.py icci`).
pub fn from_json(v: &serde_json::Value) -> Self {
IcciNet {
luma: IcciPath::from_json(&v["luma"]),
chroma: IcciPath::from_json(&v["chroma"]),
conv_last_v: Conv::from_json(&v["conv_last_v"]),
scale_v: v["scale_v"].as_f64().unwrap() as f32,
}
}
}
impl Conv {
/// Parse a BN-folded conv from an export node `{w, b, cin, cout, kh, kw, ph, pw}`.
pub fn from_json(v: &serde_json::Value) -> Self {
let g = |k: &str| v[k].as_u64().unwrap() as usize;
let f = |k: &str| v[k].as_array().unwrap().iter().map(|x| x.as_f64().unwrap() as f32).collect();
Conv { w: f("w"), b: f("b"), cin: g("cin"), cout: g("cout"),
kh: g("kh"), kw: g("kw"), ph: g("ph"), pw: g("pw") }
}
}
impl IcciPath {
/// Parse one plane-path (`{conv_first, blocks:[{c1,c2,scale}], conv_last, scale}`).
pub fn from_json(v: &serde_json::Value) -> Self {
IcciPath {
conv_first: Conv::from_json(&v["conv_first"]),
blocks: v["blocks"].as_array().unwrap().iter().map(|b| ResBlock {
conv1: Conv::from_json(&b["c1"]),
conv2: Conv::from_json(&b["c2"]),
scale: b["scale"].as_f64().unwrap() as f32,
}).collect(),
conv_last: Conv::from_json(&v["conv_last"]),
scale: v["scale"].as_f64().unwrap() as f32,
}
}
}
#[cfg(test)]

View file

@ -192,6 +192,11 @@ pub mod score;
#[cfg(feature = "provenance")]
pub mod provenance;
// Reproducible-reconstruction binding: a C2PA/JPEG-Trust hard binding over the
// deterministically decoded pixels (the reconstruct-side analogue of `provenance`).
#[cfg(feature = "provenance")]
pub mod reconstruction_binding;
// wai.world.model — generative-world transport: a signed, replay-equivalent action
// trace (the world CausalOp log) + per-frame provenance. The wire carries the
// action trace + a declared model_id, not the pixels; the sink renders. The neural

View file

@ -3,6 +3,14 @@
//! (`jpeg-ai-reference-software/src/codec/entropy_coding/cpp_exts/mans/`,
//! BSD-3, ITU/ISO/IEC), the entropy engine of the JPEG AI verification model.
//!
//! LEGACY / ORT PATH ONLY. This decoder backs the float ORT decode
//! ([`super::jpegai`]), the fallback route for `wai.image.jpegai`. The canonical
//! byte-exact decoder — the one the deterministic dispatch prefers — is
//! [`crate::jpegai_mans`] at the crate root (a later port with a flat-table /
//! sliced-offset API, in the cross-arch byte-equality matrix). New code MUST use the
//! crate-root decoder; this copy stays only while the ORT fallback does, and is slated
//! for removal once the deterministic path fully supersedes it.
//!
//! Structure (mirrors the C++ exactly — the interleave/flush ORDER is the wire
//! format):
//! - Bitstream is read **backward** from the end of each thread segment; a

View file

@ -203,16 +203,74 @@ pub fn decode_envelope(env: &Wai, registry: &ModelRegistry) -> Result<Decoded, D
// `path` is the model directory, not a single ONNX. The learned transforms
// (hyper-decoder, MCM value nets, synthesis, eICCI) are sink ONNX via ort.
"wai.image.jpegai" => {
let model = jpegai::JpegAiModel::load(path)
.map_err(|e| DecodeError::InvalidPayload(e.to_string()))?;
jpegai::decode(&env.payload, &model)
.map(Decoded::Image)
.map_err(|e| DecodeError::InvalidPayload(e.to_string()))
// Prefer the byte-exact deterministic decoder when the sink ships the
// integer weight bundle (`<dir>/weights/*.json` + `<dir>/norm.json`) — that
// path reconstructs the codestream to RGB with a portable decoded-pixel hash
// (`wai.image.jpegai` decode-equivalence). Otherwise fall back to the ORT
// float decode, so existing ONNX bundles keep working unchanged.
if let Some(img) = try_jpegai_exact(&env.payload, path) {
Ok(Decoded::Image(img))
} else {
let model = jpegai::JpegAiModel::load(path)
.map_err(|e| DecodeError::InvalidPayload(e.to_string()))?;
jpegai::decode(&env.payload, &model)
.map(Decoded::Image)
.map_err(|e| DecodeError::InvalidPayload(e.to_string()))
}
}
other => Err(DecodeError::UnknownCapability(other.to_string())),
}
}
/// Load the byte-exact JPEG-AI integer weight bundle from a model directory and run the
/// deterministic [`crate::jpegai_decode::decode`]. Returns `None` — so the caller falls
/// back to the ORT decode — if the integer bundle is absent, malformed, or the geometry
/// is unsupported. The bundle is what `tools/wai_jpegai_build_e2e_inputs.py` produces:
/// a `weights/` dir of per-plane / MCM / synthesis / eICCI JSON plus a `norm.json` of
/// the normative tables. Keeps `jpegai_decode` itself I/O-free; the file reads live here.
fn try_jpegai_exact(payload: &[u8], dir: &Path) -> Option<image::DecodedImage> {
use crate::jpegai_decode::{decode, DecoderWeights, NormTables, PlaneWeights};
use crate::jpegai_icci::IcciNet;
use crate::jpegai_synthesis::{IntNet, Weights};
let wdir = dir.join("weights");
let norm_path = dir.join("norm.json");
if !wdir.is_dir() || !norm_path.exists() {
return None;
}
let jload = |p: PathBuf| -> Option<serde_json::Value> {
serde_json::from_slice(&std::fs::read(p).ok()?).ok()
};
let w = |name: &str| -> Option<Weights> { Some(Weights::from_json(&jload(wdir.join(name))?)) };
let net = |name: &str| -> Option<IntNet> { Some(IntNet::from_json(&jload(wdir.join(name))?)) };
let weights = DecoderWeights {
luma: PlaneWeights { hyper_scale: net("hs_y.json")?, hyper_decoder: w("hyp_y.json")?, synthesis: w("synth_y.json")? },
chroma: PlaneWeights { hyper_scale: net("hs_uv.json")?, hyper_decoder: w("hyp_uv.json")?, synthesis: w("synth_uv.json")? },
mcm_y: [w("mcm0_y.json")?, w("mcm1_y.json")?, w("mcm2_y.json")?, w("mcm3_y.json")?],
icci: IcciNet::from_json(&jload(wdir.join("icci.json"))?),
};
let nv = jload(norm_path)?;
let i64v = |k: &str| nv[k].as_array().unwrap().iter().map(|x| x.as_i64().unwrap()).collect::<Vec<i64>>();
let u8v = |k: &str| nv[k].as_array().unwrap().iter().map(|x| x.as_u64().unwrap() as u8).collect::<Vec<u8>>();
let i32v = |k: &str| nv[k].as_array().unwrap().iter().map(|x| x.as_i64().unwrap() as i32).collect::<Vec<i32>>();
let i3 = |k: &str| { let a = nv[k].as_array().unwrap(); [a[0].as_i64().unwrap(), a[1].as_i64().unwrap(), a[2].as_i64().unwrap()] };
let f3 = |k: &str| { let a = nv[k].as_array().unwrap(); [a[0].as_f64().unwrap() as f32, a[1].as_f64().unwrap() as f32, a[2].as_f64().unwrap() as f32] };
let norm = NormTables {
pmf: nv["pmf"].as_array()?.iter().map(|r| r.as_array().unwrap().iter().map(|x| x.as_i64().unwrap()).collect()).collect(),
bounds: u8v("bounds"),
per_channel: nv["per_channel"].as_u64()? as usize,
cdfs_y: u8v("cdfs_y"), cdfs_uv: u8v("cdfs_uv"),
gain_y: i64v("gain_y"), gain_uv: i64v("gain_uv"),
lsbs_t0: i32v("lsbs_t0"), lsbs_t1: i32v("lsbs_t1"),
lef_thr: i3("lef_thr0"), lef_mag: f3("lef_mag0"),
};
let rgb = decode(payload, &weights, &norm)?;
Some(image::DecodedImage { rgb: rgb.data, width: rgb.w as u32, height: rgb.h as u32 })
}
/// Decode like [`decode_envelope`], but first enforce WAI's replicate-soundness
/// rule: an envelope whose `intent` is `replicate` MUST pin an **integer-exact**
/// prior, because single-threaded float is not byte-identical across hardware
@ -232,3 +290,33 @@ pub fn decode_envelope_strict(env: &Wai, registry: &ModelRegistry) -> Result<Dec
}
decode_envelope(env, registry)
}
#[cfg(test)]
mod jpegai_exact_dispatch_tests {
use std::path::Path;
/// The deterministic dispatch path: given the integer weight bundle
/// (`<dir>/weights/*.json` + `<dir>/norm.json`) and a codestream, the loader wires
/// up `jpegai_decode::decode`. Gated on assets; skips cleanly without them.
#[test]
fn jpegai_exact_bundle_decodes() {
let (Ok(dir), Ok(cs)) = (
std::env::var("WAI_JPEGAI_MODEL_DIR"),
std::env::var("WAI_JPEGAI_CODESTREAM"),
) else {
eprintln!("skip: set WAI_JPEGAI_MODEL_DIR + WAI_JPEGAI_CODESTREAM");
return;
};
let Ok(payload) = std::fs::read(&cs) else { eprintln!("skip: no codestream"); return };
let img = super::try_jpegai_exact(&payload, Path::new(&dir))
.expect("integer bundle present → deterministic decode");
assert_eq!((img.width, img.height), (256, 256));
assert_eq!(img.rgb.len(), 256 * 256 * 3);
}
/// Absent bundle → `None`, so dispatch falls back to the ORT path unchanged.
#[test]
fn missing_bundle_falls_back() {
assert!(super::try_jpegai_exact(&[], Path::new("/nonexistent/wai/model/dir")).is_none());
}
}

View file

@ -0,0 +1,201 @@
//! **Reproducible-reconstruction binding** — a cryptographic *hard* binding over the
//! deterministically decoded pixels of a neural-coded asset.
//!
//! Provenance systems (C2PA, JPEG Trust) bind a manifest to an asset with a **hard
//! binding**: a hash over the asset bytes. For neural-coded media that hash covers the
//! *compressed* codestream — and it is the only hard option available, because a neural
//! decode is normally not reproducible: two conformant decoders can produce different
//! pixels, so a hash of the *decoded image* is not portable. JPEG AI makes this explicit
//! — its normative conformance point is the output of the entropy decoder, and its
//! multi-branch design lets one codestream reconstruct through up to three synthesis
//! transforms. So provenance over the *rendered* result has had to fall back to fuzzy
//! **soft** bindings (perceptual hashes, watermarks).
//!
//! WAI decodes to RGB deterministically (one pinned reconstruction, byte-identical on
//! every machine), which restores the precondition a hard binding needs: a stable,
//! portable hash of the decoded pixels. This module produces that binding — a SHA-256
//! (the C2PA default hash) over the canonical decoded RGB, plus the descriptor that
//! makes the decode reproducible (which synthesis branch, profile/level, pixel layout,
//! colour space). A verifier with a WAI decoder re-decodes per the descriptor, re-hashes,
//! and compares.
//!
//! It is the reconstruct-side analogue of [`crate::provenance`]: that binds the opaque
//! asset bytes (content-hash-equivalence); this binds the reconstructed pixels. The
//! digest is expressed as a custom C2PA / JPEG-Trust assertion — see
//! `extensions/reconstruction-binding.md` for the on-the-wire mapping. WAI is
//! self-referential about it: it states the capability (a reproducible-reconstruction
//! binding), not a comparison.
use sha2::{Digest, Sha256};
/// What must be pinned for the decode to be reproducible. Different values here decode
/// the same codestream to different pixels, so the descriptor is part of the binding —
/// a verifier decodes under exactly these parameters before hashing.
#[derive(Clone, Debug, PartialEq, Eq)]
pub struct ReconstructionDescriptor {
/// The synthesis branch the reconstruction is pinned to (e.g. the JPEG-AI
/// `decoderID`, 0/1/2). Multi-branch codecs decode one codestream several ways;
/// the binding fixes one.
pub decoder_id: u8,
/// Codec profile identifier (capability-specific, e.g. a JPEG-AI profile id).
pub profile: String,
/// Codec level identifier.
pub level: String,
pub width: u32,
pub height: u32,
/// Canonical pixel layout of the hashed buffer. This module hashes `RGB8`:
/// row-major, 3 bytes/pixel, no row padding, no alpha.
pub format: String,
/// Colour space / transfer of the decoded pixels (e.g. `"bt709"`).
pub color_space: String,
}
impl ReconstructionDescriptor {
/// The byte length the canonical `RGB8` buffer must have.
pub fn canonical_len(&self) -> usize {
self.width as usize * self.height as usize * 3
}
}
/// A reproducible-reconstruction binding: the descriptor plus a hash over the canonical
/// decoded pixels, ready to be carried as a C2PA / JPEG-Trust assertion or in a WAI
/// receipt.
#[derive(Clone, Debug)]
pub struct ReconstructionBinding {
/// Hash algorithm id, in the C2PA registry's spelling. Always `"sha256"` here — the
/// C2PA default every validator is required to support.
pub alg: String,
/// SHA-256 over the canonical `RGB8` buffer.
pub hash: [u8; 32],
pub descriptor: ReconstructionDescriptor,
}
impl ReconstructionBinding {
/// The custom C2PA / JPEG-Trust assertion label (reverse-domain, as the C2PA spec
/// requires for third-party assertions). The full CBOR mapping is specified in
/// `extensions/reconstruction-binding.md`.
pub const C2PA_LABEL: &'static str = "science.transaction.wai.hash.reconstruction";
/// Bind a decoded image: hash the canonical `RGB8` buffer under `descriptor`.
/// `rgb` MUST be exactly `descriptor.canonical_len()` bytes (row-major RGB8).
pub fn bind(rgb: &[u8], descriptor: ReconstructionDescriptor) -> Self {
assert_eq!(
rgb.len(),
descriptor.canonical_len(),
"rgb buffer must be canonical RGB8 (width*height*3, no padding)"
);
let hash: [u8; 32] = Sha256::digest(rgb).into();
ReconstructionBinding { alg: "sha256".to_string(), hash, descriptor }
}
/// Re-verify: recompute the hash over a freshly decoded `rgb` and compare. The
/// caller MUST have decoded under this binding's `descriptor`.
pub fn verify(&self, rgb: &[u8]) -> bool {
self.alg == "sha256"
&& rgb.len() == self.descriptor.canonical_len()
&& <[u8; 32]>::from(Sha256::digest(rgb)) == self.hash
}
/// Lowercase-hex of the digest.
pub fn hash_hex(&self) -> String {
let mut s = String::with_capacity(64);
for b in self.hash {
use std::fmt::Write;
let _ = write!(s, "{b:02x}");
}
s
}
/// The binding as JSON, the shape a WAI receipt carries. The C2PA CBOR assertion
/// (label [`Self::C2PA_LABEL`]) is the same fields under the mapping in
/// `extensions/reconstruction-binding.md`; the digest is identical, so a validator
/// that re-decodes with a WAI decoder gets a matching hash either way.
pub fn to_json(&self) -> String {
let d = &self.descriptor;
format!(
"{{\"alg\":\"{}\",\"hash\":\"{}\",\"reconstruction\":{{\"decoderId\":{},\
\"profile\":{},\"level\":{},\"width\":{},\"height\":{},\"format\":{},\
\"colorSpace\":{}}}}}",
self.alg,
self.hash_hex(),
d.decoder_id,
json_str(&d.profile),
json_str(&d.level),
d.width,
d.height,
json_str(&d.format),
json_str(&d.color_space),
)
}
}
/// Minimal JSON string escaping for the small, controlled descriptor fields.
fn json_str(s: &str) -> String {
let mut out = String::with_capacity(s.len() + 2);
out.push('"');
for c in s.chars() {
match c {
'"' => out.push_str("\\\""),
'\\' => out.push_str("\\\\"),
c => out.push(c),
}
}
out.push('"');
out
}
#[cfg(test)]
mod tests {
use super::*;
fn desc(w: u32, h: u32) -> ReconstructionDescriptor {
ReconstructionDescriptor {
decoder_id: 0,
profile: "main".into(),
level: "L1".into(),
width: w,
height: h,
format: "RGB8".into(),
color_space: "bt709".into(),
}
}
#[test]
fn bind_then_verify_roundtrips() {
let rgb: Vec<u8> = (0..4 * 4 * 3).map(|i| (i * 7 % 251) as u8).collect();
let b = ReconstructionBinding::bind(&rgb, desc(4, 4));
assert!(b.verify(&rgb), "the same pixels must verify");
assert_eq!(b.alg, "sha256");
assert_eq!(b.hash_hex().len(), 64);
}
#[test]
fn one_flipped_pixel_fails_verification() {
let rgb: Vec<u8> = (0..4 * 4 * 3).map(|i| (i * 7 % 251) as u8).collect();
let b = ReconstructionBinding::bind(&rgb, desc(4, 4));
let mut tampered = rgb.clone();
tampered[10] ^= 1;
assert!(!b.verify(&tampered), "a single flipped byte must break the hard binding");
}
#[test]
fn hash_is_the_c2pa_default_over_canonical_bytes() {
// The digest is a plain SHA-256 over the RGB8 buffer — reproducible by any
// SHA-256, which is exactly what makes it portable into a C2PA validator.
let rgb = vec![0u8; 2 * 2 * 3];
let b = ReconstructionBinding::bind(&rgb, desc(2, 2));
let expect: [u8; 32] = Sha256::digest(&rgb).into();
assert_eq!(b.hash, expect);
}
#[test]
fn json_carries_descriptor_and_hash() {
let rgb = vec![1u8; 2 * 2 * 3];
let b = ReconstructionBinding::bind(&rgb, desc(2, 2));
let j = b.to_json();
assert!(j.contains("\"alg\":\"sha256\""));
assert!(j.contains("\"decoderId\":0"));
assert!(j.contains(&format!("\"hash\":\"{}\"", b.hash_hex())));
assert!(j.contains("\"colorSpace\":\"bt709\""));
}
}