wai/research/3d-neural-coding-landscape.md

@@ -0,0 +1,75 @@
+# 3D Neural Coding — where the determinism contract goes next
+
+> Landscape note, not a normative extension. Learned coding is moving from flat
+> media into 3D — point clouds and Gaussian splats — and it carries the same
+> cross-platform non-determinism that flat learned codecs have. WAI already
+> decodes one such 3D representation byte-exactly (`wai.splat.int_codec`); this
+> records how the determinism contract and the reconstruction binding generalize
+> to the emerging 3D coding standards, and where WAI does and does not connect to
+> them. Framed self-referentially — the standards are named as standards, no
+> comparisons.
+
+## The landscape
+
+The learned image-codec machinery is being reused one dimension up:
+
+- **JPEG Pleno Learning-based Point Cloud Coding** is finalized, and it **reuses
+  JPEG AI** to code the point cloud's colour attributes — the same learned-2D
+  transform, applied to attribute images.
+- **Gaussian Splat Coding** is under standardization (MPEG WG working groups,
+  JPEG), applying **Neural Network Coding (NNC)** to the splat parameters —
+  spherical-harmonic coefficients, positions, scales.
+
+Both inherit the property that made the flat learned codecs hard to deploy: a
+learned decode is not reproducible across hardware. A float attribute/coefficient
+decode drifts by round-off, the entropy stage desyncs, and — exactly as for 2D —
+the reconstruction is reproducible only on the same build. The determinism
+problem, and its fix, are the same shape in 3D.
+
+## What WAI already has
+
+`wai.splat.int_codec` is a byte-exact learned Gaussian-splat **attribute** codec:
+a learned transform over the `[14, S, S]` attribute image (position, scale,
+rotation, colour, opacity), integer synthesis over a factorized-rANS latent, pure
+`i64` — no float, no ONNX. Its conformance criterion is **splat-equivalence**: the
+decoded attribute tensor is byte-identical on every machine, a portable cloud
+hash. The float rasterizer stays sink-supplied presentation; the *attributes* are
+the contract.
+
+That is the 3D instance of the same thesis WAI applies to `wai.image.jpegai`: the
+learned decode to the medium's canonical representation is pinned and integer, so
+the reconstruction has a portable hash. `wai.splat.int_codec` already sits at the
+**`decode-equivalence`** tier (see [`extensions/determinism-tiers.md`]) — the
+attribute tensor is the "final medium" whose bytes reproduce.
+
+## Where it connects — and where it doesn't
+
+- **The contract generalizes.** `decode-equivalence` + a
+  [reproducible-reconstruction binding](../extensions/reconstruction-binding.md)
+  over a decoded 3D representation is exactly the same construction as over decoded
+  pixels: hash the canonical decoded buffer (here the attribute tensor, or a point
+  cloud's decoded attributes), pin the descriptor, and the rendered-content
+  provenance binding neural-coded media could not carry becomes available for 3D
+  too. As the 3D standards adopt learned coding, the determinism-for-provenance gap
+  is the same one WAI closes for 2D.
+- **Honest scope — WAI does not decode the ISO point-cloud bitstream.**
+  `wai.splat.int_codec` is a WAI-native integer codec, not the JPEG-Pleno PCC or
+  the MPEG splat-coding stream. The shared thing is the *problem and the approach*
+  (a learned 3D decode that needs to be reproducible), not the wire format. A
+  future `wai.pointcloud.*` that byte-exactly decodes a published 3D standard's
+  codestream would be the 3D analogue of what `wai.image.jpegai` is for JPEG AI —
+  and, because JPEG-Pleno PCC reuses JPEG AI for its colour attributes, the
+  byte-exact JPEG-AI decode WAI already has is a component that path could build
+  on. Not built; recorded as the natural next capability if the 3D neural-coding
+  standards land in scope.
+
+## Takeaway
+
+The determinism contract is not image-specific. WAI's byte-exact family already
+spans image (`jpegai`, `int_hyper`), audio (`int_codec`), and 3D
+(`splat.int_codec`); the reconstruction binding and the tier vocabulary apply to
+all of them. The emerging 3D neural-coding standards face the same
+non-determinism WAI addresses for 2D, and the JPEG-AI decode WAI has is a
+component of the point-cloud path — so the ingest is: keep the determinism
+contract media-general, and treat a byte-exact decode of a published 3D standard
+as the natural next capability, not a separate track.