open-standards/wai/research/3d-neural-coding-landscape.md

3D Neural Coding — where the determinism contract goes next

Landscape note, not a normative extension. Learned coding is moving from flat media into 3D — point clouds and Gaussian splats — and it carries the same cross-platform non-determinism that flat learned codecs have. WAI already decodes one such 3D representation byte-exactly (wai.splat.int_codec); this records how the determinism contract and the reconstruction binding generalize to the emerging 3D coding standards, and where WAI does and does not connect to them. Framed self-referentially — the standards are named as standards, no comparisons.

The landscape

The learned image-codec machinery is being reused one dimension up:

• JPEG Pleno Learning-based Point Cloud Coding is finalized, and it reuses JPEG AI to code the point cloud's colour attributes — the same learned-2D transform, applied to attribute images.
• Gaussian Splat Coding is under standardization (MPEG WG working groups, JPEG), applying Neural Network Coding (NNC) to the splat parameters — spherical-harmonic coefficients, positions, scales.

Both inherit the property that made the flat learned codecs hard to deploy: a learned decode is not reproducible across hardware. A float attribute/coefficient decode drifts by round-off, the entropy stage desyncs, and — exactly as for 2D — the reconstruction is reproducible only on the same build. The determinism problem, and its fix, are the same shape in 3D.

What WAI already has

wai.splat.int_codec is a byte-exact learned Gaussian-splat attribute codec: a learned transform over the [14, S, S] attribute image (position, scale, rotation, colour, opacity), integer synthesis over a factorized-rANS latent, pure i64 — no float, no ONNX. Its conformance criterion is splat-equivalence: the decoded attribute tensor is byte-identical on every machine, a portable cloud hash. The float rasterizer stays sink-supplied presentation; the attributes are the contract.

That is the 3D instance of the same thesis WAI applies to wai.image.jpegai: the learned decode to the medium's canonical representation is pinned and integer, so the reconstruction has a portable hash. wai.splat.int_codec already sits at the decode-equivalence tier (see [extensions/determinism-tiers.md]) — the attribute tensor is the "final medium" whose bytes reproduce.

Where it connects — and where it doesn't

• The contract generalizes. decode-equivalence + a reproducible-reconstruction binding over a decoded 3D representation is exactly the same construction as over decoded pixels: hash the canonical decoded buffer (here the attribute tensor, or a point cloud's decoded attributes), pin the descriptor, and the rendered-content provenance binding neural-coded media could not carry becomes available for 3D too. As the 3D standards adopt learned coding, the determinism-for-provenance gap is the same one WAI closes for 2D.
• Honest scope — WAI does not decode the ISO point-cloud bitstream. wai.splat.int_codec is a WAI-native integer codec, not the JPEG-Pleno PCC or the MPEG splat-coding stream. The shared thing is the problem and the approach (a learned 3D decode that needs to be reproducible), not the wire format. A future wai.pointcloud.* that byte-exactly decodes a published 3D standard's codestream would be the 3D analogue of what wai.image.jpegai is for JPEG AI — and, because JPEG-Pleno PCC reuses JPEG AI for its colour attributes, the byte-exact JPEG-AI decode WAI already has is a component that path could build on. Not built; recorded as the natural next capability if the 3D neural-coding standards land in scope.

Takeaway

The determinism contract is not image-specific. WAI's byte-exact family already spans image (jpegai, int_hyper), audio (int_codec), and 3D (splat.int_codec); the reconstruction binding and the tier vocabulary apply to all of them. The emerging 3D neural-coding standards face the same non-determinism WAI addresses for 2D, and the JPEG-AI decode WAI has is a component of the point-cloud path — so the ingest is: keep the determinism contract media-general, and treat a byte-exact decode of a published 3D standard as the natural next capability, not a separate track.