From e492370b8fdaa45a1c353d70f42cf07d8cc422cf Mon Sep 17 00:00:00 2001 From: Transaction Science Date: Mon, 24 Aug 2026 11:09:40 -0400 Subject: [PATCH] =?UTF-8?q?wai:=20research=20note=20=E2=80=94=20the=20dete?= =?UTF-8?q?rminism=20contract=20generalizes=20to=203D=20neural=20coding?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Landscape record (not normative). Learned coding is moving into 3D: JPEG Pleno point-cloud coding is finalized and reuses JPEG AI for its colour attributes, and Gaussian Splat Coding is under standardization applying Neural Network Coding to splat parameters. Both inherit the cross-platform non-determinism of learned 2D codecs. WAI already decodes one 3D representation byte-exactly — wai.splat.int_codec, a pure-i64 Gaussian-splat attribute codec at the decode-equivalence tier (splat-equivalence: the attribute tensor is byte-identical everywhere). The note records that the determinism contract + reconstruction binding generalize to decoded 3D representations exactly as to pixels, and is honest about scope: WAI's splat codec is WAI-native, NOT the ISO point-cloud bitstream — the shared thing is the problem and the approach. Because JPEG-Pleno PCC reuses JPEG AI for colour, the byte-exact JPEG-AI decode WAI has is a component a future wai.pointcloud.* path could build on. Recorded as the natural next capability, not a separate track. Framed self-referentially; standards named as standards, no comparisons. Co-Authored-By: Claude Opus 4.8 --- wai/research/3d-neural-coding-landscape.md | 75 ++++++++++++++++++++++ 1 file changed, 75 insertions(+) create mode 100644 wai/research/3d-neural-coding-landscape.md diff --git a/wai/research/3d-neural-coding-landscape.md b/wai/research/3d-neural-coding-landscape.md new file mode 100644 index 000000000..072205516 --- /dev/null +++ b/wai/research/3d-neural-coding-landscape.md @@ -0,0 +1,75 @@ +# 3D Neural Coding — where the determinism contract goes next + +> Landscape note, not a normative extension. Learned coding is moving from flat +> media into 3D — point clouds and Gaussian splats — and it carries the same +> cross-platform non-determinism that flat learned codecs have. WAI already +> decodes one such 3D representation byte-exactly (`wai.splat.int_codec`); this +> records how the determinism contract and the reconstruction binding generalize +> to the emerging 3D coding standards, and where WAI does and does not connect to +> them. Framed self-referentially — the standards are named as standards, no +> comparisons. + +## The landscape + +The learned image-codec machinery is being reused one dimension up: + +- **JPEG Pleno Learning-based Point Cloud Coding** is finalized, and it **reuses + JPEG AI** to code the point cloud's colour attributes — the same learned-2D + transform, applied to attribute images. +- **Gaussian Splat Coding** is under standardization (MPEG WG working groups, + JPEG), applying **Neural Network Coding (NNC)** to the splat parameters — + spherical-harmonic coefficients, positions, scales. + +Both inherit the property that made the flat learned codecs hard to deploy: a +learned decode is not reproducible across hardware. A float attribute/coefficient +decode drifts by round-off, the entropy stage desyncs, and — exactly as for 2D — +the reconstruction is reproducible only on the same build. The determinism +problem, and its fix, are the same shape in 3D. + +## What WAI already has + +`wai.splat.int_codec` is a byte-exact learned Gaussian-splat **attribute** codec: +a learned transform over the `[14, S, S]` attribute image (position, scale, +rotation, colour, opacity), integer synthesis over a factorized-rANS latent, pure +`i64` — no float, no ONNX. Its conformance criterion is **splat-equivalence**: the +decoded attribute tensor is byte-identical on every machine, a portable cloud +hash. The float rasterizer stays sink-supplied presentation; the *attributes* are +the contract. + +That is the 3D instance of the same thesis WAI applies to `wai.image.jpegai`: the +learned decode to the medium's canonical representation is pinned and integer, so +the reconstruction has a portable hash. `wai.splat.int_codec` already sits at the +**`decode-equivalence`** tier (see [`extensions/determinism-tiers.md`]) — the +attribute tensor is the "final medium" whose bytes reproduce. + +## Where it connects — and where it doesn't + +- **The contract generalizes.** `decode-equivalence` + a + [reproducible-reconstruction binding](../extensions/reconstruction-binding.md) + over a decoded 3D representation is exactly the same construction as over decoded + pixels: hash the canonical decoded buffer (here the attribute tensor, or a point + cloud's decoded attributes), pin the descriptor, and the rendered-content + provenance binding neural-coded media could not carry becomes available for 3D + too. As the 3D standards adopt learned coding, the determinism-for-provenance gap + is the same one WAI closes for 2D. +- **Honest scope — WAI does not decode the ISO point-cloud bitstream.** + `wai.splat.int_codec` is a WAI-native integer codec, not the JPEG-Pleno PCC or + the MPEG splat-coding stream. The shared thing is the *problem and the approach* + (a learned 3D decode that needs to be reproducible), not the wire format. A + future `wai.pointcloud.*` that byte-exactly decodes a published 3D standard's + codestream would be the 3D analogue of what `wai.image.jpegai` is for JPEG AI — + and, because JPEG-Pleno PCC reuses JPEG AI for its colour attributes, the + byte-exact JPEG-AI decode WAI already has is a component that path could build + on. Not built; recorded as the natural next capability if the 3D neural-coding + standards land in scope. + +## Takeaway + +The determinism contract is not image-specific. WAI's byte-exact family already +spans image (`jpegai`, `int_hyper`), audio (`int_codec`), and 3D +(`splat.int_codec`); the reconstruction binding and the tier vocabulary apply to +all of them. The emerging 3D neural-coding standards face the same +non-determinism WAI addresses for 2D, and the JPEG-AI decode WAI has is a +component of the point-cloud path — so the ingest is: keep the determinism +contract media-general, and treat a byte-exact decode of a published 3D standard +as the natural next capability, not a separate track. -- 2.54.0 (Apple Git-157)