From c4475c4ecbca6f1ffce584bee861bb354387db41 Mon Sep 17 00:00:00 2001 From: Transaction Science Date: Mon, 27 Jul 2026 19:59:01 -0400 Subject: [PATCH 1/6] wai: register the byte-exact JPEG-AI decoder + restore the byte-equality matrix MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Integration follow-through on the merged jpegai_decode: make the decoder visible where the standard describes itself, and get it into the cross-arch matrix. SPEC/docs: - Register `wai.image.jpegai` in SPEC.md §5 (neural table) — a live, dispatchable capability was absent from the normative menu. Framed as decode-equivalence: one pinned reconstruction of an ISO/IEC 6048 codestream to RGB, reproducible on every machine (a portable decoded-pixel hash), entropy/dequant integer-exact, synthesis within 1 LSB of the reference. - CHANGELOG entry for the byte-exact codestream→RGB decoder, distinguishing its decode-equivalence profile from the all-i64 hash-portable in-house codecs. - Fix the stale CAP_IMAGE_JPEGAI comment ("decoder library not yet in this crate" — untrue post-merge) and drop its "~30% over conventional SOTA" comparison. byte-exact-conformance: - The crate did not compile on main: `rans.rs` was moved out of `neural/` to the wai-rs crate root, but the shim still #[path]-included the old `neural/rans.rs`; the moved file also references `crate::int_entropy`. Repoint rans to the crate root and shim `int_entropy` (self-contained). This restores the cross-arch byte-equality build. - Add a golden that decodes the same ISO reference mANS stream through the MERGED decoder's entropy stage (crate-root `jpegai_mans`, flat-table/sliced-offset API) and asserts the identical symbols. The decoder `wai.image.jpegai` actually reconstructs with is now in the byte-equality matrix, not only the older `neural/` port. 15 conformance tests pass. Co-Authored-By: Claude Opus 4.8 --- wai/CHANGELOG.md | 21 ++++++++++++ wai/SPEC.md | 1 + .../src/jpegai_golden.rs | 34 +++++++++++++++++++ wai/byte-exact-conformance/src/lib.rs | 6 ++++ wai/wai-rs/src/codecs/mod.rs | 2 +- 5 files changed, 63 insertions(+), 1 deletion(-) diff --git a/wai/CHANGELOG.md b/wai/CHANGELOG.md index e958d2506..f70ea7733 100644 --- a/wai/CHANGELOG.md +++ b/wai/CHANGELOG.md @@ -6,6 +6,27 @@ wire format may change and such changes are recorded here. ## v0.4.0 — current +### JPEG AI — a byte-exact codestream→RGB decoder (`jpegai_decode::decode`) + +- `wai::jpegai_decode::decode(codestream, &DecoderWeights, &NormTables) -> Rgb` + reconstructs an ISO/IEC 6048 JPEG-AI codestream to RGB through a fully + integer entropy front end and a deterministic value/synthesis/post-filter + path — one pinned reconstruction, reproducible byte-for-byte on every machine, + so the decoded pixels have a portable hash. No per-image fixtures: every + per-image quantity is derived from the bitstream; only checkpoint constants + come from a sink-supplied model bundle. +- Determinism profile: the entropy decode (mANS), the dequant, and the derived + entropy inputs (skip mask, σ index, RVS band, dequant gain) are integer-exact + against the reference; the PTQ value/synthesis stages are deterministic and + land within 1 LSB of the ISO reference decoder. Distinct from the in-house + `wai.neural.int_*` codecs (all-i64, hash-portable): this decodes a published + external standard's codestream deterministically. +- Modules: `jpegai_decode` (the composed entry point) over `jpegai_bitstream`, + `jpegai_mans`, `jpegai_pipeline`, `jpegai_synthesis`, `jpegai_icci` + (feature `neural_int`). Inputs are regenerable from the shipped assets via + `tools/wai_jpegai_build_e2e_inputs.py`. The whole chain — codestream → RGB — + is exercised end to end against the reference PNG. + ### JPEG AI — a live, self-sufficient `wai.image.jpegai` decode (2026-06-22) - `neural::jpegai::decode(codestream, &JpegAiModel)` decodes a real ISO/IEC 6048 diff --git a/wai/SPEC.md b/wai/SPEC.md index 60798779a..ef63a8e36 100644 --- a/wai/SPEC.md +++ b/wai/SPEC.md @@ -194,6 +194,7 @@ decode route for each is implemented in the reference sink (see the | `wai.neural.mbt2018_mean` | mean-scale hyperprior (Gaussian-conditional entropy) | conditional-entropy family, stronger than factorized | `neural/hyperprior.rs` | | `wai.neural.int_synth` | integer fixed-point synthesis (all-i64 transposed-conv, no float/ONNX) | bit-identical on every machine — soundly backs `intent = replicate` | `neural/int_synth.rs` (QAT weights sink-supplied) | | `wai.neural.int_hyper` | integer fixed-point **conditional** codec — whole bitstream→image path integer (integer hyper-decoder σ → integer-σ entropy → integer synthesis) | the integer-entropy contract fully assembled; SOTA quality (PTQ of mbt2018_mean) bit-identical on every machine | `int_hyper_synth.rs` (tables sink-supplied) | +| `wai.image.jpegai` | ISO/IEC 6048 JPEG-AI codestream → RGB — integer entropy front end (mANS) + integer dequant/value path + fixed-order synthesis and post-filters; every per-image quantity is derived from the bitstream, only checkpoint constants are sink-supplied | **decode-equivalence**: one pinned reconstruction of a published-standard codestream to RGB, reproducible on every machine (a portable decoded-pixel hash); entropy and dequant are integer-exact, synthesis reconstructs to within 1 LSB of the reference | `jpegai_decode.rs` (+ `jpegai_{bitstream,mans,pipeline,synthesis,icci}.rs`) | | `wai.generate.spectral` | deterministic integer GENERATIVE synthesis (`intent=create`) — prompt+seed → integer-LUT spectral composition, no float/ONNX | byte-identical on every machine, pinned to (model, prompt, seed, steps); the create-side of "conformance is a hash" (generation-equivalence) | `generate.rs` (small pinned model bundle) | | `wai.neural.elic` | checkerboard context model (hyperprior + 2-pass spatial context) | the JPEG-AI / MLIC++ entropy class | `neural/context.rs` | | `wai.neural.mlicpp` | MLIC++ multi-reference entropy (channel-AR slices + checkerboard + global attention) | −13.39% BD-rate vs VVC/VTM-17 | research code, `neural/mlicpp.rs` | diff --git a/wai/byte-exact-conformance/src/jpegai_golden.rs b/wai/byte-exact-conformance/src/jpegai_golden.rs index 30d7949ca..b9b0bdab9 100644 --- a/wai/byte-exact-conformance/src/jpegai_golden.rs +++ b/wai/byte-exact-conformance/src/jpegai_golden.rs @@ -100,3 +100,37 @@ fn jpegai_mans_decodes_reference_stream_identically() { } } } + +/// The SAME reference stream through the **merged byte-exact decoder's** mANS +/// (crate-root `jpegai_mans`, the one `wai.image.jpegai` reconstructs with) — a +/// different, later port with a flat-table / sliced-offset API. It must decode the +/// ISO stream to the identical symbols, so the decoder that actually ships is in the +/// cross-arch byte-equality matrix, not only the older `neural/` port. +#[test] +fn jpegai_exact_mans_decodes_reference_stream_identically() { + use crate::jpegai_mans_exact::{build_decode_transitions, cdf_from_pmf, MansDecoder}; + + let mut pmf: Vec> = Vec::new(); + let mut off = 0usize; + for &n in &PMF_PREFIX_LENS { + let mut row = vec![0i64; PMF_WIDTH]; + row[..n].copy_from_slice(&PMF_PREFIXES_FLAT[off..off + n]); + off += n; + pmf.push(row); + } + // The flat-table API: build [u32;256] tables from (pmf, cdf) and flatten them. + let tables = build_decode_transitions(&pmf, &cdf_from_pmf(&pmf)); + let flat: Vec = tables.iter().flatten().copied().collect(); + + let mut dec = MansDecoder::new(&Y_MEMORY, &[Y_OFFSET]); + dec.set_sgm_transitions(&flat, &BOUNDS); + let masks: Vec = MASKS.iter().map(|&m| m != 0).collect(); + let mut r = vec![0i16; EXPECTED_R.len()]; + dec.decode_sgm(&SIGMA, &mut r, &masks); + for i in 0..r.len() { + if masks[i] { + assert_eq!(r[i], EXPECTED_R[i], + "byte-exact JPEG-AI mANS symbol {i} diverged on this platform"); + } + } +} diff --git a/wai/byte-exact-conformance/src/lib.rs b/wai/byte-exact-conformance/src/lib.rs index 9e2614ce2..7baa9886e 100644 --- a/wai/byte-exact-conformance/src/lib.rs +++ b/wai/byte-exact-conformance/src/lib.rs @@ -17,5 +17,11 @@ pub mod neural; #[path = "../../wai-rs/src/int_entropy.rs"] pub mod int_entropy; +// The merged byte-exact JPEG-AI decoder's entropy stage (crate-root, self-contained — +// no shims needed). Included so the cross-arch byte-equality matrix covers the decoder +// that `wai.image.jpegai` actually reconstructs with, not only the older `neural/` port. +#[path = "../../wai-rs/src/jpegai_mans.rs"] +pub mod jpegai_mans_exact; + mod golden; mod jpegai_golden; diff --git a/wai/wai-rs/src/codecs/mod.rs b/wai/wai-rs/src/codecs/mod.rs index f50de8690..438a32595 100644 --- a/wai/wai-rs/src/codecs/mod.rs +++ b/wai/wai-rs/src/codecs/mod.rs @@ -22,7 +22,7 @@ pub const CAP_IMAGE_PNG: &str = "wai.image.png"; // universal floor pub const CAP_IMAGE_JPEG: &str = "wai.image.jpeg"; // most-supported lossy pub const CAP_IMAGE_AVIF: &str = "wai.image.avif"; // modern lossy (AV1-based) pub const CAP_IMAGE_JXL: &str = "wai.image.jxl"; // JPEG-XL (lossless + lossy) -pub const CAP_IMAGE_JPEGAI: &str = "wai.image.jpegai"; // JPEG AI (ITU-T T.840.1 / ISO/IEC 6048-1:2025, royalty-free baseline, ~30% over conventional SOTA) — standardized neural image codec; decoder library not yet in this crate (sink/library-supplied) +pub const CAP_IMAGE_JPEGAI: &str = "wai.image.jpegai"; // JPEG AI (ITU-T T.840.1 / ISO/IEC 6048-1:2025) standardized neural image codec. Byte-exact decoder in-crate: codestream→RGB via jpegai_decode::decode (integer entropy/dequant, deterministic synthesis, ±1 LSB of the reference). Model weights sink-supplied. // Audio pub const CAP_AUDIO_OPUS: &str = "wai.audio.opus"; // modern lossy pub const CAP_AUDIO_FLAC: &str = "wai.audio.flac"; // lossless -- 2.54.0 (Apple Git-157) From ea31d2285487a979e7984d1ed81c77d6225034f9 Mon Sep 17 00:00:00 2001 From: Transaction Science Date: Thu, 30 Jul 2026 23:36:10 -0400 Subject: [PATCH 2/6] =?UTF-8?q?wai:=20house-style=20pass=20on=20public=20c?= =?UTF-8?q?opy=20=E2=80=94=20drop=20comparisons,=20vendor=20names,=20vanit?= =?UTF-8?q?y=20counts?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Bring the git-visible surfaces in line with the framing rules (self-referential; no competitive comparisons, no vanity metrics, describe by capability): - SPEC.md neural table: drop company prefixes (Meta/Descript-company/Kyutai → the capability's own model identifier), drop BD-rate "vs VTM/VVC" comparisons and the "current SOTA" superlative (describe each capability's regime instead), and reword the "vs the float X family that desyncs" lines as self-referential statements of why the integer contract exists ("a float decode desyncs on…"). - README status line: "passes 11 … tests" → "round-trips every registered capability through the full envelope" (a count that describes coverage, not a number that drifts). - glossary: "canonical 342-entry dataset" → "canonical survey dataset". Unchanged on purpose: "dispatches to SOTA standard codecs" and similar — that is the ingest thesis (we route to the best existing codecs), self-referential, not a comparison of our own work; and the source-column model IDs, which are factual dependency pointers. Co-Authored-By: Claude Opus 4.8 --- README.md | 2 +- wai/README.md | 2 +- wai/SPEC.md | 20 ++++++++++---------- 3 files changed, 12 insertions(+), 12 deletions(-) diff --git a/README.md b/README.md index 0d42d80ff..9f9d1eb41 100644 --- a/README.md +++ b/README.md @@ -36,7 +36,7 @@ A container + capability-dispatch standard for media. WAI does not re-implement - **Site:** [wai.transaction.science](https://wai.transaction.science) - **Licence:** Apache-2.0 (see [`wai/LICENSE`](wai/LICENSE)) -- **Status:** v1.0 draft standard. `wai-rs/` reference implementation (Rust lib + cdylib + staticlib, C FFI) passes 11 capability + envelope round-trip tests. Known consumer: CommunicationOS. +- **Status:** v1.0 draft standard. `wai-rs/` reference implementation (Rust lib + cdylib + staticlib, C FFI) round-trips every registered capability through the full envelope. Known consumer: CommunicationOS. ### [jouleclaw/](jouleclaw/) — Energy-optimised AI runtime diff --git a/wai/README.md b/wai/README.md index 235cdeff0..d939cb1f0 100644 --- a/wai/README.md +++ b/wai/README.md @@ -23,7 +23,7 @@ The capability a file requires is *named*, not supplied. |------|----------| | [`SPEC.md`](SPEC.md) | WAI v1.0 spec — envelope, manifest, capability menu | | [`wai-rs/`](wai-rs/) | Rust reference implementation (lib + cdylib + staticlib + C FFI) | -| [`glossary/`](glossary/) | Reference catalog of media codecs surveyed for capability registration. [`codecs.json`](glossary/codecs.json) is the canonical 342-entry dataset; [`SCHEMA.md`](glossary/SCHEMA.md) defines the shape and contribution rules. Rendered at [wai.transaction.science/glossary](https://wai.transaction.science/glossary). | +| [`glossary/`](glossary/) | Reference catalog of media codecs surveyed for capability registration. [`codecs.json`](glossary/codecs.json) is the canonical survey dataset; [`SCHEMA.md`](glossary/SCHEMA.md) defines the shape and contribution rules. Rendered at [wai.transaction.science/glossary](https://wai.transaction.science/glossary). | | [`corpus/`](corpus/) | Standard test material (Kodak/CLIC/Tecnick/SIPI images, Xiph Derf + UVG video, SQAM-class audio, enwik8 + Silesia text). Not committed; `CORPUS.md` documents reacquisition. | ## Quickstart diff --git a/wai/SPEC.md b/wai/SPEC.md index ef63a8e36..15ea80041 100644 --- a/wai/SPEC.md +++ b/wai/SPEC.md @@ -180,9 +180,9 @@ decode route for each is implemented in the reference sink (see the | capability | model | win vs zeroth | source | |---|---|---|---| -| `wai.neural.encodec32` | Meta EnCodec at 32 kHz, 4 codebooks | ~3 kbps acceptable music | `facebook/encodec_32khz` | -| `wai.neural.dac` | Descript Audio Codec at 44.1 kHz | 6 kbps near-transparent music | `descript/dac_44khz` | -| `wai.neural.mimi` | Kyutai Mimi, 12.5 Hz frame rate | ~1.1 kbps real-time speech | `kyutai/mimi` | +| `wai.neural.encodec32` | EnCodec at 32 kHz, 4 codebooks | ~3 kbps acceptable music | `facebook/encodec_32khz` | +| `wai.neural.dac` | DAC at 44.1 kHz | 6 kbps near-transparent music | `descript/dac_44khz` | +| `wai.neural.mimi` | Mimi, 12.5 Hz frame rate | ~1.1 kbps real-time speech | `kyutai/mimi` | | `wai.neural.wavtokenizer` | WavTokenizer single-codebook VQ | 0.9 kbps beats DAC @ 9 kbps on UTMOS | `novateur/WavTokenizer-large-*` | | `wai.neural.snac` | SNAC multi-resolution RVQ (hierarchical code levels, own sample rate) | low-bitrate TTS standard | `neural/snac.rs` | @@ -193,12 +193,12 @@ decode route for each is implemented in the reference sink (see the | `wai.neural.bmshj2018` | bmshj2018-factorized + rANS (byte-exact CompressAI port) | ~80× vs raw RGB at q=3 (~0.26 bpp, 29 dB PSNR) | CompressAI | | `wai.neural.mbt2018_mean` | mean-scale hyperprior (Gaussian-conditional entropy) | conditional-entropy family, stronger than factorized | `neural/hyperprior.rs` | | `wai.neural.int_synth` | integer fixed-point synthesis (all-i64 transposed-conv, no float/ONNX) | bit-identical on every machine — soundly backs `intent = replicate` | `neural/int_synth.rs` (QAT weights sink-supplied) | -| `wai.neural.int_hyper` | integer fixed-point **conditional** codec — whole bitstream→image path integer (integer hyper-decoder σ → integer-σ entropy → integer synthesis) | the integer-entropy contract fully assembled; SOTA quality (PTQ of mbt2018_mean) bit-identical on every machine | `int_hyper_synth.rs` (tables sink-supplied) | +| `wai.neural.int_hyper` | integer fixed-point **conditional** codec — whole bitstream→image path integer (integer hyper-decoder σ → integer-σ entropy → integer synthesis) | the integer-entropy contract fully assembled; high quality (PTQ of mbt2018_mean) bit-identical on every machine | `int_hyper_synth.rs` (tables sink-supplied) | | `wai.image.jpegai` | ISO/IEC 6048 JPEG-AI codestream → RGB — integer entropy front end (mANS) + integer dequant/value path + fixed-order synthesis and post-filters; every per-image quantity is derived from the bitstream, only checkpoint constants are sink-supplied | **decode-equivalence**: one pinned reconstruction of a published-standard codestream to RGB, reproducible on every machine (a portable decoded-pixel hash); entropy and dequant are integer-exact, synthesis reconstructs to within 1 LSB of the reference | `jpegai_decode.rs` (+ `jpegai_{bitstream,mans,pipeline,synthesis,icci}.rs`) | | `wai.generate.spectral` | deterministic integer GENERATIVE synthesis (`intent=create`) — prompt+seed → integer-LUT spectral composition, no float/ONNX | byte-identical on every machine, pinned to (model, prompt, seed, steps); the create-side of "conformance is a hash" (generation-equivalence) | `generate.rs` (small pinned model bundle) | | `wai.neural.elic` | checkerboard context model (hyperprior + 2-pass spatial context) | the JPEG-AI / MLIC++ entropy class | `neural/context.rs` | -| `wai.neural.mlicpp` | MLIC++ multi-reference entropy (channel-AR slices + checkerboard + global attention) | −13.39% BD-rate vs VVC/VTM-17 | research code, `neural/mlicpp.rs` | -| `wai.neural.mlicv2` | MLICv2 / MLICv2+ (2025), MetaFormer transform | −20.46% BD-rate vs VTM-17 — current SOTA learned image; shares the mlicpp route | research code, `neural/mlicpp.rs` | +| `wai.neural.mlicpp` | MLIC++ multi-reference entropy (channel-AR slices + checkerboard + global attention) | learned image, low-bitrate high-fidelity | research code, `neural/mlicpp.rs` | +| `wai.neural.mlicv2` | MLICv2 / MLICv2+ (2025), MetaFormer transform | learned-image fidelity frontier; shares the mlicpp route | research code, `neural/mlicpp.rs` | | `wai.neural.cmic` | Content-Adaptive Mamba (ICLR 2026), SSM transform + SCCTX entropy | learned-image frontier; shares the mlicpp route | arXiv 2508.02192, `neural/mlicpp.rs` | | `wai.neural.glc` | Generative Latent Coding — categorical / VQ entropy | <0.05 bpp where JPEG-XL collapses | research code, `neural/glc.rs` | @@ -207,9 +207,9 @@ decode route for each is implemented in the reference sink (see the | capability | model | win vs zeroth | source | |---|---|---|---| | `wai.neural.video_bmshj2018` | per-frame bmshj2018 | 1000×+ vs raw RGB, browser-decodable on WebGPU | CompressAI | -| `wai.neural.dcvc_rt` | DCVC-RT (2025), inter-frame with reference-feature propagation | −21% vs VTM; real-time at 1080p/4K | research code, `neural/video_dcvc.rs` | +| `wai.neural.dcvc_rt` | DCVC-RT (2025), inter-frame with reference-feature propagation | real-time neural video at 1080p/4K | research code, `neural/video_dcvc.rs` | | `wai.neural.dcvc_fm` | DCVC-FM (feature modulation), wide single-model quality range (RGB + YUV) | SOTA-RD neural video; shares the inter-frame route | research code, `neural/video_dcvc.rs` | -| `wai.video.int_motion` | DETERMINISTIC INTEGER inter-frame — I-frame + integer block motion-compensated P-frames + integer-quantized residual over rANS, no float/ONNX; motion is whole-pel or **half-pel** (`mv_precision`), the latter predicting at the half-grid by integer bilinear interpolation (`(a+b+1)>>1`, `(a+b+c+d+2)>>2`) so real footage — which pans a non-integer number of pixels per frame — costs fewer residual bits while staying byte-exact | byte-identical frame-by-frame on every machine (*frame-state-equivalence*, a per-frame portable hash); the temporal extension of the integer contract, vs the float DCVC family that desyncs down the GOP | `int_video.rs` (whole-pel `tools/wai_int_video_encode.py`; half-pel from real footage `tools/wai_int_video_real_encode.py`) | +| `wai.video.int_motion` | DETERMINISTIC INTEGER inter-frame — I-frame + integer block motion-compensated P-frames + integer-quantized residual over rANS, no float/ONNX; motion is whole-pel or **half-pel** (`mv_precision`), the latter predicting at the half-grid by integer bilinear interpolation (`(a+b+1)>>1`, `(a+b+c+d+2)>>2`) so real footage — which pans a non-integer number of pixels per frame — costs fewer residual bits while staying byte-exact | byte-identical frame-by-frame on every machine (*frame-state-equivalence*, a per-frame portable hash); the temporal extension of the integer contract, where a float inter-frame decode desyncs down the GOP | `int_video.rs` (whole-pel `tools/wai_int_video_encode.py`; half-pel from real footage `tools/wai_int_video_real_encode.py`) | | `wai.video.int_hyper` | NEURAL-KEYFRAME inter-frame — the I-frame is the LEARNED integer codec `int_hyper` decoded byte-exact, the P-frames are integer block-MC residuals on top | a learned keyframe you can put in a receipt + integer inter-frames; whole decode integer, byte-identical everywhere | `int_video::decode_pframes` ∘ `int_hyper_synth::decode_conditional` | **Sink-architecture note:** the inter-frame neural video codecs (DCVC-RT, @@ -261,12 +261,12 @@ standard. See the linked extension for each. | `wai.splat.spz` | SPZ (Niantic) | asset, content-addressed; presentation only | | `wai.audio.scene` | WSCN scene (objects + trajectories) | mixdown-equivalence · [spatial-audio](extensions/spatial-audio.md) | | `wai.audio.score` | WSCR container (note events) | music as instructions; sink synthesizes, mixdown-equivalence · [score](extensions/score.md) | -| `wai.audio.int_codec` | LEARNED 1-D transform codec — integer transposed-conv synthesis + integer IGDN over a factorized-rANS latent (model + bitstream from `tools/wai_int_audio_export.py`) | **sample-equivalence**: int16 PCM byte-identical on every machine (a portable PCM hash); the audio analog of `wai.neural.int_hyper`, vs the float SoundStream/EnCodec family (`wai.neural.{encodec32,dac,mimi,…}`) that desyncs on ~1e-5 round-off. Pure i64, no float/ONNX (`int_audio.rs`) | +| `wai.audio.int_codec` | LEARNED 1-D transform codec — integer transposed-conv synthesis + integer IGDN over a factorized-rANS latent (model + bitstream from `tools/wai_int_audio_export.py`) | **sample-equivalence**: int16 PCM byte-identical on every machine (a portable PCM hash); the audio analog of `wai.neural.int_hyper`, where a float learned-audio decode desyncs on ~1e-5 round-off. Pure i64, no float/ONNX (`int_audio.rs`) | | `wai.haptic.signal` | WHAP envelope (intensity/sharpness keyframes) | sample-equivalence · [haptics](extensions/haptics.md) | | `wai.feed.state` | WWLD, sim `wai.world.sim.feed0` | replay-equivalence · [state-feeds](extensions/state-feeds.md) | | `wai.avatar.pose` | WWLD, sim `wai.world.sim.skeletal0` | pose-equivalence · [avatars](extensions/avatars.md) | | `wai.splat.4d` | W4DS container (per-Gaussian keyframed tracks) | frame-state-equivalence · [volumetric](extensions/volumetric.md) | -| `wai.splat.int_codec` | LEARNED Gaussian-splat ATTRIBUTE codec — a learned transform over the `[14,S,S]` attribute image (pos/scale/rot/color/opacity), integer synthesis (int_hyper machinery, 14 channels) over a factorized-rANS latent (`int_splat.rs`; model + bitstream from `tools/wai_int_splat_export.py`) | **splat-equivalence**: the decoded attribute tensor is byte-identical on every machine (a portable cloud hash) — the int_hyper of 3D, vs the float SOG / self-organizing-Gaussian codecs whose decode desyncs. The float rasterizer stays sink-supplied presentation; the *attributes* are the contract. Pure i64, no float/ONNX | +| `wai.splat.int_codec` | LEARNED Gaussian-splat ATTRIBUTE codec — a learned transform over the `[14,S,S]` attribute image (pos/scale/rot/color/opacity), integer synthesis (int_hyper machinery, 14 channels) over a factorized-rANS latent (`int_splat.rs`; model + bitstream from `tools/wai_int_splat_export.py`) | **splat-equivalence**: the decoded attribute tensor is byte-identical on every machine (a portable cloud hash) — the int_hyper of 3D, where a float splat-attribute decode desyncs. The float rasterizer stays sink-supplied presentation; the *attributes* are the contract. Pure i64, no float/ONNX | | `wai.film.linear` | WFLM container (sealed world + camera track + frame schedule) | film-equivalence · [linear-film](extensions/linear-film.md) | | `wai.film.reel` | WREL container (sealed world + a sequence of shots = camera track + schedule each) | reel-equivalence (film-equivalence with cuts) · [linear-film](extensions/linear-film.md) | | `wai.ternary.tt` | WQTT container (balanced-ternary MPS cores — packed trits + integer per-column scales) | a **quantum-inspired** (tensor-network) transform ported from the ternary/MPS fabric with ONE change: the per-column scale is an **integer**, so the whole decode is `i64`/`i128` and the reconstruction is byte-identical on every machine — **reconstruction-equivalence** (a portable recon hash), receiptable. Honest scope: a **provenance** transform, NOT a rate-competitive codec (measured RD is mediocre — `Lossy` tier; see the extension's §5 verdict). Encoder-side float SVD only · [ternary-tt](extensions/ternary-tt.md) | -- 2.54.0 (Apple Git-157) From 64fd36d98c276c4c867c737ccc088d01763ff7fd Mon Sep 17 00:00:00 2001 From: Transaction Science Date: Fri, 31 Jul 2026 01:08:50 -0400 Subject: [PATCH 3/6] wai-rs: dispatch wai.image.jpegai to the deterministic decoder when its bundle ships MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The envelope dispatched `wai.image.jpegai` to the old ORT float decode even after the byte-exact decoder merged. Wire the deterministic path in, additively: when the model directory carries the integer weight bundle (`weights/*.json` + `norm.json`, as `tools/wai_jpegai_build_e2e_inputs.py` produces), decode through `jpegai_decode::decode` — the reconstruction with a portable decoded-pixel hash. When that bundle is absent, fall back to the ORT decode unchanged, so existing ONNX bundles keep working. `neural` already implies `neural_int`, so the deterministic module is reachable from the dispatch. The file I/O lives at the dispatch boundary (`try_jpegai_exact`), keeping `jpegai_decode` itself I/O-free; the weight/norm loaders reuse `IntNet::from_json`, `Weights::from_json`, and a new `IcciNet::from_json` (+ `Conv`/`IcciPath::from_json`). Tested (feature `neural`): with the bundle present the loader decodes the real test256 codestream to a 256×256 RGB image; with it absent `try_jpegai_exact` returns None so dispatch falls back. Builds clean under the ORT feature. Co-Authored-By: Claude Opus 4.8 --- wai/wai-rs/src/jpegai_icci.rs | 38 ++++++++++++++ wai/wai-rs/src/neural/mod.rs | 98 +++++++++++++++++++++++++++++++++-- 2 files changed, 131 insertions(+), 5 deletions(-) diff --git a/wai/wai-rs/src/jpegai_icci.rs b/wai/wai-rs/src/jpegai_icci.rs index 642d2fa7f..a5ae8ebac 100644 --- a/wai/wai-rs/src/jpegai_icci.rs +++ b/wai/wai-rs/src/jpegai_icci.rs @@ -296,6 +296,44 @@ impl IcciNet { let v_o = head(&self.conv_last_v, &v16, self.scale_v); (y_o, u_o, v_o) } + + /// Parse the whole eICCI net from the `icci` weight export + /// (`{luma, chroma, conv_last_v, scale_v}`, produced by + /// `tools/wai_jpegai_export_weights.py icci`). + pub fn from_json(v: &serde_json::Value) -> Self { + IcciNet { + luma: IcciPath::from_json(&v["luma"]), + chroma: IcciPath::from_json(&v["chroma"]), + conv_last_v: Conv::from_json(&v["conv_last_v"]), + scale_v: v["scale_v"].as_f64().unwrap() as f32, + } + } +} + +impl Conv { + /// Parse a BN-folded conv from an export node `{w, b, cin, cout, kh, kw, ph, pw}`. + pub fn from_json(v: &serde_json::Value) -> Self { + let g = |k: &str| v[k].as_u64().unwrap() as usize; + let f = |k: &str| v[k].as_array().unwrap().iter().map(|x| x.as_f64().unwrap() as f32).collect(); + Conv { w: f("w"), b: f("b"), cin: g("cin"), cout: g("cout"), + kh: g("kh"), kw: g("kw"), ph: g("ph"), pw: g("pw") } + } +} + +impl IcciPath { + /// Parse one plane-path (`{conv_first, blocks:[{c1,c2,scale}], conv_last, scale}`). + pub fn from_json(v: &serde_json::Value) -> Self { + IcciPath { + conv_first: Conv::from_json(&v["conv_first"]), + blocks: v["blocks"].as_array().unwrap().iter().map(|b| ResBlock { + conv1: Conv::from_json(&b["c1"]), + conv2: Conv::from_json(&b["c2"]), + scale: b["scale"].as_f64().unwrap() as f32, + }).collect(), + conv_last: Conv::from_json(&v["conv_last"]), + scale: v["scale"].as_f64().unwrap() as f32, + } + } } #[cfg(test)] diff --git a/wai/wai-rs/src/neural/mod.rs b/wai/wai-rs/src/neural/mod.rs index 86d230ec5..a6254d7b1 100644 --- a/wai/wai-rs/src/neural/mod.rs +++ b/wai/wai-rs/src/neural/mod.rs @@ -203,16 +203,74 @@ pub fn decode_envelope(env: &Wai, registry: &ModelRegistry) -> Result { - let model = jpegai::JpegAiModel::load(path) - .map_err(|e| DecodeError::InvalidPayload(e.to_string()))?; - jpegai::decode(&env.payload, &model) - .map(Decoded::Image) - .map_err(|e| DecodeError::InvalidPayload(e.to_string())) + // Prefer the byte-exact deterministic decoder when the sink ships the + // integer weight bundle (`/weights/*.json` + `/norm.json`) — that + // path reconstructs the codestream to RGB with a portable decoded-pixel hash + // (`wai.image.jpegai` decode-equivalence). Otherwise fall back to the ORT + // float decode, so existing ONNX bundles keep working unchanged. + if let Some(img) = try_jpegai_exact(&env.payload, path) { + Ok(Decoded::Image(img)) + } else { + let model = jpegai::JpegAiModel::load(path) + .map_err(|e| DecodeError::InvalidPayload(e.to_string()))?; + jpegai::decode(&env.payload, &model) + .map(Decoded::Image) + .map_err(|e| DecodeError::InvalidPayload(e.to_string())) + } } other => Err(DecodeError::UnknownCapability(other.to_string())), } } +/// Load the byte-exact JPEG-AI integer weight bundle from a model directory and run the +/// deterministic [`crate::jpegai_decode::decode`]. Returns `None` — so the caller falls +/// back to the ORT decode — if the integer bundle is absent, malformed, or the geometry +/// is unsupported. The bundle is what `tools/wai_jpegai_build_e2e_inputs.py` produces: +/// a `weights/` dir of per-plane / MCM / synthesis / eICCI JSON plus a `norm.json` of +/// the normative tables. Keeps `jpegai_decode` itself I/O-free; the file reads live here. +fn try_jpegai_exact(payload: &[u8], dir: &Path) -> Option { + use crate::jpegai_decode::{decode, DecoderWeights, NormTables, PlaneWeights}; + use crate::jpegai_icci::IcciNet; + use crate::jpegai_synthesis::{IntNet, Weights}; + + let wdir = dir.join("weights"); + let norm_path = dir.join("norm.json"); + if !wdir.is_dir() || !norm_path.exists() { + return None; + } + let jload = |p: PathBuf| -> Option { + serde_json::from_slice(&std::fs::read(p).ok()?).ok() + }; + let w = |name: &str| -> Option { Some(Weights::from_json(&jload(wdir.join(name))?)) }; + let net = |name: &str| -> Option { Some(IntNet::from_json(&jload(wdir.join(name))?)) }; + + let weights = DecoderWeights { + luma: PlaneWeights { hyper_scale: net("hs_y.json")?, hyper_decoder: w("hyp_y.json")?, synthesis: w("synth_y.json")? }, + chroma: PlaneWeights { hyper_scale: net("hs_uv.json")?, hyper_decoder: w("hyp_uv.json")?, synthesis: w("synth_uv.json")? }, + mcm_y: [w("mcm0_y.json")?, w("mcm1_y.json")?, w("mcm2_y.json")?, w("mcm3_y.json")?], + icci: IcciNet::from_json(&jload(wdir.join("icci.json"))?), + }; + + let nv = jload(norm_path)?; + let i64v = |k: &str| nv[k].as_array().unwrap().iter().map(|x| x.as_i64().unwrap()).collect::>(); + let u8v = |k: &str| nv[k].as_array().unwrap().iter().map(|x| x.as_u64().unwrap() as u8).collect::>(); + let i32v = |k: &str| nv[k].as_array().unwrap().iter().map(|x| x.as_i64().unwrap() as i32).collect::>(); + let i3 = |k: &str| { let a = nv[k].as_array().unwrap(); [a[0].as_i64().unwrap(), a[1].as_i64().unwrap(), a[2].as_i64().unwrap()] }; + let f3 = |k: &str| { let a = nv[k].as_array().unwrap(); [a[0].as_f64().unwrap() as f32, a[1].as_f64().unwrap() as f32, a[2].as_f64().unwrap() as f32] }; + let norm = NormTables { + pmf: nv["pmf"].as_array()?.iter().map(|r| r.as_array().unwrap().iter().map(|x| x.as_i64().unwrap()).collect()).collect(), + bounds: u8v("bounds"), + per_channel: nv["per_channel"].as_u64()? as usize, + cdfs_y: u8v("cdfs_y"), cdfs_uv: u8v("cdfs_uv"), + gain_y: i64v("gain_y"), gain_uv: i64v("gain_uv"), + lsbs_t0: i32v("lsbs_t0"), lsbs_t1: i32v("lsbs_t1"), + lef_thr: i3("lef_thr0"), lef_mag: f3("lef_mag0"), + }; + + let rgb = decode(payload, &weights, &norm)?; + Some(image::DecodedImage { rgb: rgb.data, width: rgb.w as u32, height: rgb.h as u32 }) +} + /// Decode like [`decode_envelope`], but first enforce WAI's replicate-soundness /// rule: an envelope whose `intent` is `replicate` MUST pin an **integer-exact** /// prior, because single-threaded float is not byte-identical across hardware @@ -232,3 +290,33 @@ pub fn decode_envelope_strict(env: &Wai, registry: &ModelRegistry) -> Result/weights/*.json` + `/norm.json`) and a codestream, the loader wires + /// up `jpegai_decode::decode`. Gated on assets; skips cleanly without them. + #[test] + fn jpegai_exact_bundle_decodes() { + let (Ok(dir), Ok(cs)) = ( + std::env::var("WAI_JPEGAI_MODEL_DIR"), + std::env::var("WAI_JPEGAI_CODESTREAM"), + ) else { + eprintln!("skip: set WAI_JPEGAI_MODEL_DIR + WAI_JPEGAI_CODESTREAM"); + return; + }; + let Ok(payload) = std::fs::read(&cs) else { eprintln!("skip: no codestream"); return }; + let img = super::try_jpegai_exact(&payload, Path::new(&dir)) + .expect("integer bundle present → deterministic decode"); + assert_eq!((img.width, img.height), (256, 256)); + assert_eq!(img.rgb.len(), 256 * 256 * 3); + } + + /// Absent bundle → `None`, so dispatch falls back to the ORT path unchanged. + #[test] + fn missing_bundle_falls_back() { + assert!(super::try_jpegai_exact(&[], Path::new("/nonexistent/wai/model/dir")).is_none()); + } +} -- 2.54.0 (Apple Git-157) From 47ed2c6b8b7e86643038437ec71e3db9bb87abe0 Mon Sep 17 00:00:00 2001 From: Transaction Science Date: Fri, 31 Jul 2026 09:25:20 -0400 Subject: [PATCH 4/6] =?UTF-8?q?wai:=20reproducible-reconstruction=20bindin?= =?UTF-8?q?g=20=E2=80=94=20a=20C2PA/JPEG-Trust=20hard=20binding=20over=20p?= =?UTF-8?q?ixels?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The byte-exact decoder's payoff: a provenance binding neural-coded media normally cannot carry. A hard binding hashes asset bytes; over the decoded image it was impossible, because a neural decode isn't reproducible — JPEG AI's own normative conformance point is the entropy-decoder output (not pixels), and its multi-branch design lets one codestream reconstruct three ways, so a decoded-pixel hash isn't portable. Provenance over the rendered result has had to fall back to fuzzy soft bindings. WAI decodes to RGB deterministically, which restores a stable, portable decoded-pixel hash — pinning one reconstruction is a determinism guarantee beyond what the base standard requires. This adds: - reconstruction_binding (feature provenance): ReconstructionBinding::bind hashes the canonical RGB8 buffer (row-major, no padding) under a descriptor that pins the reproducible decode (synthesis branch / profile / level / layout / color space); verify re-decodes-and-compares. SHA-256 — the C2PA default — so the digest is reproducible by any validator. bind/verify/tamper/JSON tests pass. - extensions/reconstruction-binding.md: the standard. Decode-equivalence conformance; carriage as a custom C2PA/JPEG-Trust assertion (science.transaction.wai.hash. reconstruction, CBOR, beside the standard byte-binding); and a note that a signed binding over reconstructed content is a substrate for machine-readable AI-content marking (e.g. EU AI Act Article 50, from 2 Aug 2026) — stated as a capability, not a legal claim. It is the reconstruct-side analogue of the provenance extension: that binds the opaque carried bytes; this binds the reconstructed pixels. Co-Authored-By: Claude Opus 4.8 --- wai/extensions/reconstruction-binding.md | 134 +++++++++++++++ wai/wai-rs/Cargo.toml | 2 +- wai/wai-rs/src/lib.rs | 5 + wai/wai-rs/src/reconstruction_binding.rs | 201 +++++++++++++++++++++++ 4 files changed, 341 insertions(+), 1 deletion(-) create mode 100644 wai/extensions/reconstruction-binding.md create mode 100644 wai/wai-rs/src/reconstruction_binding.rs diff --git a/wai/extensions/reconstruction-binding.md b/wai/extensions/reconstruction-binding.md new file mode 100644 index 000000000..282ecee99 --- /dev/null +++ b/wai/extensions/reconstruction-binding.md @@ -0,0 +1,134 @@ +# WAI Extension: Reproducible-Reconstruction Binding + +> Status: Draft. A cryptographic **hard binding over the deterministically +> decoded pixels** of a neural-coded asset — the reconstruct-side analogue of +> the [Asset Provenance](provenance.md) extension, and a bridge from WAI's +> byte-exact decode into C2PA / JPEG Trust provenance. Reference impl: the +> `reconstruction_binding` module of `wai-rs` (feature `provenance`). +> Keywords **MUST**, **MUST NOT**, **SHOULD**, **MAY** are RFC 2119/8174. + +## 1. Scope and model + +A provenance manifest binds itself to an asset with a **hard binding**: a +cryptographic hash over the asset's bytes. For neural-coded media that hash +covers the *compressed* codestream, and it has been the only hard option +available — because a neural decode is normally not reproducible. Two conformant +decoders may produce different pixels, so a hash of the *decoded image* is not +portable across implementations, and provenance over the rendered result has had +to fall back to fuzzy **soft** bindings (perceptual hashes, watermarks), which +are approximate and spoofable. + +This is not incidental to the codec. The JPEG AI standard (ISO/IEC 6048 / +ITU-T T.840) defines its normative bit-exact conformance point at the **output +of the entropy decoder**, not at reconstruction to pixels; and its multi-branch +design specifies **up to three synthesis transforms** (`decoderID` 0/1/2), any of +which produces a conformant reconstruction of the same codestream. Reconstruction +to pixels is therefore, by design, **not** required to be bit-exact, and one +codestream may legally decode to different images. A hash of the decoded pixels is +not portable under the base standard. + +WAI decodes to RGB **deterministically** — one pinned reconstruction, byte-identical +on every machine (integer entropy and dequant; fixed-order, reproducible synthesis). +Pinning one reconstruction is a determinism guarantee **beyond what the base standard +requires**, and it restores exactly the precondition a hard binding needs: a stable, +portable hash of the decoded pixels. This extension expresses that hash as a +provenance assertion. + +A `reconstruction-binding` is **not a content capability**. The asset declares an +ordinary capability (`wai.image.jpegai`, `wai.neural.int_hyper`, …) and is carried +and decoded normally; the binding is metadata *about* the decoded result, the way +[`provenance`](provenance.md) is metadata about the carried bytes. + +## 2. The binding + +A binding is a hash plus the descriptor that makes the decode reproducible: + +```json +{ + "alg": "sha256", + "hash": "", + "reconstruction": { + "decoderId": 0, // the pinned synthesis branch (multi-branch codecs) + "profile": "", // codec profile + "level": "", // codec level + "width": 256, + "height": 256, + "format": "RGB8", // canonical: row-major, 3 bytes/pixel, no padding, no alpha + "colorSpace": "bt709" + } +} +``` + +- **Hash algorithm** — `sha256` (the C2PA default; every C2PA validator supports + it). MUST be an algorithm in the C2PA hash-algorithm registry. +- **Canonical buffer** — the hash is computed over the decoded image as a `RGB8` + buffer: row-major, three bytes per pixel, no row padding, no alpha, exactly + `width * height * 3` bytes. No other pre-processing. +- **The descriptor is part of the binding.** A multi-branch codec decodes one + codestream several ways; a verifier MUST decode under exactly the descriptor's + `decoderId`, `profile`, and `level` before hashing, or the comparison is + meaningless. The descriptor pins the one reconstruction the hash is over. + +Conformance criterion: **decode-equivalence** — a verifier that decodes the +codestream with a WAI decoder under the binding's descriptor, hashes the canonical +`RGB8` buffer, and compares MUST obtain the recorded `hash`. + +## 3. C2PA / JPEG Trust carriage + +C2PA (and JPEG Trust, ISO/IEC 21617-1, which is built on the C2PA architecture) +has **no standard assertion that hashes decoded/rendered pixels** — every standard +hard binding (`c2pa.hash.data`, `c2pa.hash.boxes`, `c2pa.hash.bmff`) hashes the +compressed bytes. A reproducible-reconstruction binding is therefore expressed as a +**custom assertion**, using C2PA's reverse-domain labelling for third-party +assertions: + +- **Assertion label:** `science.transaction.wai.hash.reconstruction` +- **Body:** the object in §2, serialized as a CBOR map (the C2PA assertion + serialization). Field names map 1:1 to the JSON above: + `alg` (text), `hash` (byte string), and `reconstruction` (a map of + `decoderId` (uint), `profile`/`level`/`format`/`colorSpace` (text), + `width`/`height` (uint)). +- **Placement:** this assertion is **additional**. A manifest SHOULD still carry a + standard hard binding (`c2pa.hash.data` over the codestream) referenced by the + claim's `created_assertions`, so it is valid in any C2PA validator; the + reconstruction binding sits **beside** it. The byte-binding proves *these + compressed bytes*; the reconstruction binding upgrades that to *and anyone who + decodes them per this descriptor obtains this exact image*. +- **Signature:** the assertion is covered by the C2PA claim signature like any + other, so it is tamper-evident. + +The digest is the same value WAI's own receipt carries (§4), so a validator with a +WAI decoder reaches a matching hash whether it reads the C2PA CBOR assertion or the +WAI receipt. + +## 4. WAI receipt carriage + +The binding MAY be carried in a WAI receipt alongside an [asset +provenance](provenance.md) receipt: provenance binds the opaque asset bytes +(content-hash-equivalence, BLAKE3 over the envelope); this binds the reconstructed +pixels (decode-equivalence, SHA-256 over the canonical RGB). Together they cover +both halves of the thesis — the bytes that are carried and the pixels that are +reconstructed. The reference impl emits the §2 JSON via +`ReconstructionBinding::to_json`. + +## 5. Machine-readable AI-content marking + +A reproducible-reconstruction binding is a substrate for **machine-readable marking +of AI-processed media**. Where a regime requires content to be marked in a +machine-readable, detectable format (for example EU AI Act Article 50, whose +transparency obligations apply from 2 August 2026, with the technical marking +standards developed via the Code of Practice), a signed assertion that binds the +*reconstructed* content — carried through the neural coding step rather than +invalidated by it — is a mechanism that survives decode. WAI states this as a +capability (a signed, reproducible binding over decoded content); it does not make +a legal claim, and any deployment MUST confirm sufficiency against the applicable +regime. + +## 6. Reference implementation + +`wai-rs`, feature `provenance`, module `reconstruction_binding`: +`ReconstructionDescriptor`, `ReconstructionBinding::{bind, verify, hash_hex, +to_json, C2PA_LABEL}`. `bind` hashes the canonical `RGB8` buffer under a +descriptor; `verify` re-hashes a freshly decoded buffer and compares. The digest is +a plain SHA-256, reproducible by any SHA-256 implementation — which is what makes it +portable into a C2PA validator. diff --git a/wai/wai-rs/Cargo.toml b/wai/wai-rs/Cargo.toml index afcc71f0c..f45c871a6 100644 --- a/wai/wai-rs/Cargo.toml +++ b/wai/wai-rs/Cargo.toml @@ -130,7 +130,7 @@ score = ["world"] # Asset provenance (extensions/provenance): a signed, energy-accounted # lineage receipt for opaque captured content WAI does not reconstruct. # Reuses the JWP Merkle + Ed25519; no codec/ML deps. -provenance = ["world"] +provenance = ["world", "dep:sha2"] # wai.world.model — generative-world transport: a signed, replay-equivalent action # trace (reuses the world CausalOp log) + per-frame provenance. The neural sibling # of wai.world.replay; the renderer is a declared world model (sink-supplied, or the diff --git a/wai/wai-rs/src/lib.rs b/wai/wai-rs/src/lib.rs index d21c90e8a..9849cd0de 100644 --- a/wai/wai-rs/src/lib.rs +++ b/wai/wai-rs/src/lib.rs @@ -192,6 +192,11 @@ pub mod score; #[cfg(feature = "provenance")] pub mod provenance; +// Reproducible-reconstruction binding: a C2PA/JPEG-Trust hard binding over the +// deterministically decoded pixels (the reconstruct-side analogue of `provenance`). +#[cfg(feature = "provenance")] +pub mod reconstruction_binding; + // wai.world.model — generative-world transport: a signed, replay-equivalent action // trace (the world CausalOp log) + per-frame provenance. The wire carries the // action trace + a declared model_id, not the pixels; the sink renders. The neural diff --git a/wai/wai-rs/src/reconstruction_binding.rs b/wai/wai-rs/src/reconstruction_binding.rs new file mode 100644 index 000000000..51e3fda49 --- /dev/null +++ b/wai/wai-rs/src/reconstruction_binding.rs @@ -0,0 +1,201 @@ +//! **Reproducible-reconstruction binding** — a cryptographic *hard* binding over the +//! deterministically decoded pixels of a neural-coded asset. +//! +//! Provenance systems (C2PA, JPEG Trust) bind a manifest to an asset with a **hard +//! binding**: a hash over the asset bytes. For neural-coded media that hash covers the +//! *compressed* codestream — and it is the only hard option available, because a neural +//! decode is normally not reproducible: two conformant decoders can produce different +//! pixels, so a hash of the *decoded image* is not portable. JPEG AI makes this explicit +//! — its normative conformance point is the output of the entropy decoder, and its +//! multi-branch design lets one codestream reconstruct through up to three synthesis +//! transforms. So provenance over the *rendered* result has had to fall back to fuzzy +//! **soft** bindings (perceptual hashes, watermarks). +//! +//! WAI decodes to RGB deterministically (one pinned reconstruction, byte-identical on +//! every machine), which restores the precondition a hard binding needs: a stable, +//! portable hash of the decoded pixels. This module produces that binding — a SHA-256 +//! (the C2PA default hash) over the canonical decoded RGB, plus the descriptor that +//! makes the decode reproducible (which synthesis branch, profile/level, pixel layout, +//! colour space). A verifier with a WAI decoder re-decodes per the descriptor, re-hashes, +//! and compares. +//! +//! It is the reconstruct-side analogue of [`crate::provenance`]: that binds the opaque +//! asset bytes (content-hash-equivalence); this binds the reconstructed pixels. The +//! digest is expressed as a custom C2PA / JPEG-Trust assertion — see +//! `extensions/reconstruction-binding.md` for the on-the-wire mapping. WAI is +//! self-referential about it: it states the capability (a reproducible-reconstruction +//! binding), not a comparison. + +use sha2::{Digest, Sha256}; + +/// What must be pinned for the decode to be reproducible. Different values here decode +/// the same codestream to different pixels, so the descriptor is part of the binding — +/// a verifier decodes under exactly these parameters before hashing. +#[derive(Clone, Debug, PartialEq, Eq)] +pub struct ReconstructionDescriptor { + /// The synthesis branch the reconstruction is pinned to (e.g. the JPEG-AI + /// `decoderID`, 0/1/2). Multi-branch codecs decode one codestream several ways; + /// the binding fixes one. + pub decoder_id: u8, + /// Codec profile identifier (capability-specific, e.g. a JPEG-AI profile id). + pub profile: String, + /// Codec level identifier. + pub level: String, + pub width: u32, + pub height: u32, + /// Canonical pixel layout of the hashed buffer. This module hashes `RGB8`: + /// row-major, 3 bytes/pixel, no row padding, no alpha. + pub format: String, + /// Colour space / transfer of the decoded pixels (e.g. `"bt709"`). + pub color_space: String, +} + +impl ReconstructionDescriptor { + /// The byte length the canonical `RGB8` buffer must have. + pub fn canonical_len(&self) -> usize { + self.width as usize * self.height as usize * 3 + } +} + +/// A reproducible-reconstruction binding: the descriptor plus a hash over the canonical +/// decoded pixels, ready to be carried as a C2PA / JPEG-Trust assertion or in a WAI +/// receipt. +#[derive(Clone, Debug)] +pub struct ReconstructionBinding { + /// Hash algorithm id, in the C2PA registry's spelling. Always `"sha256"` here — the + /// C2PA default every validator is required to support. + pub alg: String, + /// SHA-256 over the canonical `RGB8` buffer. + pub hash: [u8; 32], + pub descriptor: ReconstructionDescriptor, +} + +impl ReconstructionBinding { + /// The custom C2PA / JPEG-Trust assertion label (reverse-domain, as the C2PA spec + /// requires for third-party assertions). The full CBOR mapping is specified in + /// `extensions/reconstruction-binding.md`. + pub const C2PA_LABEL: &'static str = "science.transaction.wai.hash.reconstruction"; + + /// Bind a decoded image: hash the canonical `RGB8` buffer under `descriptor`. + /// `rgb` MUST be exactly `descriptor.canonical_len()` bytes (row-major RGB8). + pub fn bind(rgb: &[u8], descriptor: ReconstructionDescriptor) -> Self { + assert_eq!( + rgb.len(), + descriptor.canonical_len(), + "rgb buffer must be canonical RGB8 (width*height*3, no padding)" + ); + let hash: [u8; 32] = Sha256::digest(rgb).into(); + ReconstructionBinding { alg: "sha256".to_string(), hash, descriptor } + } + + /// Re-verify: recompute the hash over a freshly decoded `rgb` and compare. The + /// caller MUST have decoded under this binding's `descriptor`. + pub fn verify(&self, rgb: &[u8]) -> bool { + self.alg == "sha256" + && rgb.len() == self.descriptor.canonical_len() + && <[u8; 32]>::from(Sha256::digest(rgb)) == self.hash + } + + /// Lowercase-hex of the digest. + pub fn hash_hex(&self) -> String { + let mut s = String::with_capacity(64); + for b in self.hash { + use std::fmt::Write; + let _ = write!(s, "{b:02x}"); + } + s + } + + /// The binding as JSON, the shape a WAI receipt carries. The C2PA CBOR assertion + /// (label [`Self::C2PA_LABEL`]) is the same fields under the mapping in + /// `extensions/reconstruction-binding.md`; the digest is identical, so a validator + /// that re-decodes with a WAI decoder gets a matching hash either way. + pub fn to_json(&self) -> String { + let d = &self.descriptor; + format!( + "{{\"alg\":\"{}\",\"hash\":\"{}\",\"reconstruction\":{{\"decoderId\":{},\ + \"profile\":{},\"level\":{},\"width\":{},\"height\":{},\"format\":{},\ + \"colorSpace\":{}}}}}", + self.alg, + self.hash_hex(), + d.decoder_id, + json_str(&d.profile), + json_str(&d.level), + d.width, + d.height, + json_str(&d.format), + json_str(&d.color_space), + ) + } +} + +/// Minimal JSON string escaping for the small, controlled descriptor fields. +fn json_str(s: &str) -> String { + let mut out = String::with_capacity(s.len() + 2); + out.push('"'); + for c in s.chars() { + match c { + '"' => out.push_str("\\\""), + '\\' => out.push_str("\\\\"), + c => out.push(c), + } + } + out.push('"'); + out +} + +#[cfg(test)] +mod tests { + use super::*; + + fn desc(w: u32, h: u32) -> ReconstructionDescriptor { + ReconstructionDescriptor { + decoder_id: 0, + profile: "main".into(), + level: "L1".into(), + width: w, + height: h, + format: "RGB8".into(), + color_space: "bt709".into(), + } + } + + #[test] + fn bind_then_verify_roundtrips() { + let rgb: Vec = (0..4 * 4 * 3).map(|i| (i * 7 % 251) as u8).collect(); + let b = ReconstructionBinding::bind(&rgb, desc(4, 4)); + assert!(b.verify(&rgb), "the same pixels must verify"); + assert_eq!(b.alg, "sha256"); + assert_eq!(b.hash_hex().len(), 64); + } + + #[test] + fn one_flipped_pixel_fails_verification() { + let rgb: Vec = (0..4 * 4 * 3).map(|i| (i * 7 % 251) as u8).collect(); + let b = ReconstructionBinding::bind(&rgb, desc(4, 4)); + let mut tampered = rgb.clone(); + tampered[10] ^= 1; + assert!(!b.verify(&tampered), "a single flipped byte must break the hard binding"); + } + + #[test] + fn hash_is_the_c2pa_default_over_canonical_bytes() { + // The digest is a plain SHA-256 over the RGB8 buffer — reproducible by any + // SHA-256, which is exactly what makes it portable into a C2PA validator. + let rgb = vec![0u8; 2 * 2 * 3]; + let b = ReconstructionBinding::bind(&rgb, desc(2, 2)); + let expect: [u8; 32] = Sha256::digest(&rgb).into(); + assert_eq!(b.hash, expect); + } + + #[test] + fn json_carries_descriptor_and_hash() { + let rgb = vec![1u8; 2 * 2 * 3]; + let b = ReconstructionBinding::bind(&rgb, desc(2, 2)); + let j = b.to_json(); + assert!(j.contains("\"alg\":\"sha256\"")); + assert!(j.contains("\"decoderId\":0")); + assert!(j.contains(&format!("\"hash\":\"{}\"", b.hash_hex()))); + assert!(j.contains("\"colorSpace\":\"bt709\"")); + } +} -- 2.54.0 (Apple Git-157) From 5ac0d5c86c78f81061253007ff657543630b7de7 Mon Sep 17 00:00:00 2001 From: Transaction Science Date: Fri, 31 Jul 2026 09:32:53 -0400 Subject: [PATCH 5/6] wai-rs: mark the legacy ORT mANS decoder, point new code at the canonical one MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Two mANS decoders exist: the crate-root `jpegai_mans` (byte-exact, in the cross-arch byte-equality matrix, used by the deterministic decode) and this `neural/jpegai_mans` (the older port backing the float ORT fallback). They are cleanly separated by path — only `neural/jpegai.rs` uses this one — so rather than force a risky merge of the fallback's entropy stage, mark this copy legacy/ORT-only with a pointer to the canonical crate-root decoder and the deprecation direction: it stays only while the ORT fallback does, and goes when the deterministic path fully supersedes it. Also verified this pass (not a code change): the byte-exact decode compiles to wasm32-unknown-unknown under `neural_int`, so the deterministic reconstruction runs on every host including the browser — the portability half of the byte-exact contract, alongside the entropy stage's presence in the cross-arch byte-equality CI matrix. Co-Authored-By: Claude Opus 4.8 --- wai/wai-rs/src/neural/jpegai_mans.rs | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/wai/wai-rs/src/neural/jpegai_mans.rs b/wai/wai-rs/src/neural/jpegai_mans.rs index c88d1553f..bdd45b9fd 100644 --- a/wai/wai-rs/src/neural/jpegai_mans.rs +++ b/wai/wai-rs/src/neural/jpegai_mans.rs @@ -3,6 +3,14 @@ //! (`jpeg-ai-reference-software/src/codec/entropy_coding/cpp_exts/mans/`, //! BSD-3, ITU/ISO/IEC), the entropy engine of the JPEG AI verification model. //! +//! LEGACY / ORT PATH ONLY. This decoder backs the float ORT decode +//! ([`super::jpegai`]), the fallback route for `wai.image.jpegai`. The canonical +//! byte-exact decoder — the one the deterministic dispatch prefers — is +//! [`crate::jpegai_mans`] at the crate root (a later port with a flat-table / +//! sliced-offset API, in the cross-arch byte-equality matrix). New code MUST use the +//! crate-root decoder; this copy stays only while the ORT fallback does, and is slated +//! for removal once the deterministic path fully supersedes it. +//! //! Structure (mirrors the C++ exactly — the interleave/flush ORDER is the wire //! format): //! - Bitstream is read **backward** from the end of each thread segment; a -- 2.54.0 (Apple Git-157) From afb4e603cc08e6d39d1be915cbafd0d8e6618252 Mon Sep 17 00:00:00 2001 From: Transaction Science Date: Sat, 1 Aug 2026 05:25:41 -0400 Subject: [PATCH 6/6] wai: document the reconstruction binding + dispatch/conformance changes MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit - SPEC §5: reference extensions/reconstruction-binding.md alongside the other provenance sub-specs, so the new capability is discoverable from the spec. - CHANGELOG: entries for the reproducible-reconstruction binding (the C2PA/ JPEG-Trust hard binding over decoded pixels), the wai.image.jpegai dispatch now preferring the byte-exact decoder, and the restored byte-equality matrix. Docs only; no code change. Co-Authored-By: Claude Opus 4.8 --- wai/CHANGELOG.md | 25 +++++++++++++++++++++++++ wai/SPEC.md | 6 ++++++ 2 files changed, 31 insertions(+) diff --git a/wai/CHANGELOG.md b/wai/CHANGELOG.md index f70ea7733..9e68c8a31 100644 --- a/wai/CHANGELOG.md +++ b/wai/CHANGELOG.md @@ -6,6 +6,31 @@ wire format may change and such changes are recorded here. ## v0.4.0 — current +### Reproducible-reconstruction binding — a C2PA/JPEG-Trust hard binding over pixels + +- New `wai.*` provenance capability: `reconstruction_binding` (feature + `provenance`) + [`extensions/reconstruction-binding.md`]. A cryptographic hard + binding over the *deterministically decoded* pixels of a neural-coded asset — + which such media normally can't carry, because a neural decode isn't + reproducible (JPEG AI's normative conformance point is the entropy-decoder + output, not pixels; its multi-branch design reconstructs one codestream several + ways). WAI's byte-exact decode restores a portable decoded-pixel hash. SHA-256 + (the C2PA default) over the canonical RGB8 buffer plus a descriptor pinning the + reproducible decode; carried as a custom C2PA/JPEG-Trust assertion + (`science.transaction.wai.hash.reconstruction`) beside the standard byte-binding. + A substrate for machine-readable AI-content marking (e.g. EU AI Act Art. 50). + +### `wai.image.jpegai` now dispatches to the byte-exact decoder + +- `decode_envelope` prefers the deterministic `jpegai_decode::decode` when the + model directory ships the integer weight bundle (`weights/*.json` + `norm.json`), + falling back to the ONNX-runtime decode otherwise — existing bundles keep + working. `wai.image.jpegai` is now registered in SPEC §5 (it was a live + capability missing from the normative menu). +- The cross-arch byte-equality matrix now covers the merged decoder's entropy + stage (`byte-exact-conformance`), and that crate's build — broken on a moved + `rans.rs` include — is restored. + ### JPEG AI — a byte-exact codestream→RGB decoder (`jpegai_decode::decode`) - `wai::jpegai_decode::decode(codestream, &DecoderWeights, &NormTables) -> Rgb` diff --git a/wai/SPEC.md b/wai/SPEC.md index 15ea80041..bebdd09d1 100644 --- a/wai/SPEC.md +++ b/wai/SPEC.md @@ -244,6 +244,12 @@ Merkle roots, Ed25519 group signatures, and cross-hop preservation. See [`extensions/confidentiality.md`](extensions/confidentiality.md) for the private-channel sub-spec: MLS (RFC 9420) group keys + SFrame (RFC 9605) object encryption, the byte-level framing behind §6. +See [`extensions/reconstruction-binding.md`](extensions/reconstruction-binding.md) +for the reproducible-reconstruction binding: a C2PA / JPEG-Trust hard binding +over the *deterministically decoded pixels* of a neural-coded asset (SHA-256 over +the canonical RGB plus the descriptor that pins the reconstruction) — the +reconstruct-side companion to `provenance`, applicable to `wai.image.jpegai` and +the byte-exact `wai.neural.int_*` codecs. ### Interactive, temporal & computed content (extensions) -- 2.54.0 (Apple Git-157)