e492370b8f
All checks were successful
no-vanity-metrics / Scan committed Markdown for vanity metrics (pull_request) Successful in 4s
Landscape record (not normative). Learned coding is moving into 3D: JPEG Pleno point-cloud coding is finalized and reuses JPEG AI for its colour attributes, and Gaussian Splat Coding is under standardization applying Neural Network Coding to splat parameters. Both inherit the cross-platform non-determinism of learned 2D codecs. WAI already decodes one 3D representation byte-exactly — wai.splat.int_codec, a pure-i64 Gaussian-splat attribute codec at the decode-equivalence tier (splat-equivalence: the attribute tensor is byte-identical everywhere). The note records that the determinism contract + reconstruction binding generalize to decoded 3D representations exactly as to pixels, and is honest about scope: WAI's splat codec is WAI-native, NOT the ISO point-cloud bitstream — the shared thing is the problem and the approach. Because JPEG-Pleno PCC reuses JPEG AI for colour, the byte-exact JPEG-AI decode WAI has is a component a future wai.pointcloud.* path could build on. Recorded as the natural next capability, not a separate track. Framed self-referentially; standards named as standards, no comparisons. Co-Authored-By: Claude Opus 4.8 <[email protected]>
75 lines
4.3 KiB
Markdown
75 lines
4.3 KiB
Markdown
# 3D Neural Coding — where the determinism contract goes next
|
|
|
|
> Landscape note, not a normative extension. Learned coding is moving from flat
|
|
> media into 3D — point clouds and Gaussian splats — and it carries the same
|
|
> cross-platform non-determinism that flat learned codecs have. WAI already
|
|
> decodes one such 3D representation byte-exactly (`wai.splat.int_codec`); this
|
|
> records how the determinism contract and the reconstruction binding generalize
|
|
> to the emerging 3D coding standards, and where WAI does and does not connect to
|
|
> them. Framed self-referentially — the standards are named as standards, no
|
|
> comparisons.
|
|
|
|
## The landscape
|
|
|
|
The learned image-codec machinery is being reused one dimension up:
|
|
|
|
- **JPEG Pleno Learning-based Point Cloud Coding** is finalized, and it **reuses
|
|
JPEG AI** to code the point cloud's colour attributes — the same learned-2D
|
|
transform, applied to attribute images.
|
|
- **Gaussian Splat Coding** is under standardization (MPEG WG working groups,
|
|
JPEG), applying **Neural Network Coding (NNC)** to the splat parameters —
|
|
spherical-harmonic coefficients, positions, scales.
|
|
|
|
Both inherit the property that made the flat learned codecs hard to deploy: a
|
|
learned decode is not reproducible across hardware. A float attribute/coefficient
|
|
decode drifts by round-off, the entropy stage desyncs, and — exactly as for 2D —
|
|
the reconstruction is reproducible only on the same build. The determinism
|
|
problem, and its fix, are the same shape in 3D.
|
|
|
|
## What WAI already has
|
|
|
|
`wai.splat.int_codec` is a byte-exact learned Gaussian-splat **attribute** codec:
|
|
a learned transform over the `[14, S, S]` attribute image (position, scale,
|
|
rotation, colour, opacity), integer synthesis over a factorized-rANS latent, pure
|
|
`i64` — no float, no ONNX. Its conformance criterion is **splat-equivalence**: the
|
|
decoded attribute tensor is byte-identical on every machine, a portable cloud
|
|
hash. The float rasterizer stays sink-supplied presentation; the *attributes* are
|
|
the contract.
|
|
|
|
That is the 3D instance of the same thesis WAI applies to `wai.image.jpegai`: the
|
|
learned decode to the medium's canonical representation is pinned and integer, so
|
|
the reconstruction has a portable hash. `wai.splat.int_codec` already sits at the
|
|
**`decode-equivalence`** tier (see [`extensions/determinism-tiers.md`]) — the
|
|
attribute tensor is the "final medium" whose bytes reproduce.
|
|
|
|
## Where it connects — and where it doesn't
|
|
|
|
- **The contract generalizes.** `decode-equivalence` + a
|
|
[reproducible-reconstruction binding](../extensions/reconstruction-binding.md)
|
|
over a decoded 3D representation is exactly the same construction as over decoded
|
|
pixels: hash the canonical decoded buffer (here the attribute tensor, or a point
|
|
cloud's decoded attributes), pin the descriptor, and the rendered-content
|
|
provenance binding neural-coded media could not carry becomes available for 3D
|
|
too. As the 3D standards adopt learned coding, the determinism-for-provenance gap
|
|
is the same one WAI closes for 2D.
|
|
- **Honest scope — WAI does not decode the ISO point-cloud bitstream.**
|
|
`wai.splat.int_codec` is a WAI-native integer codec, not the JPEG-Pleno PCC or
|
|
the MPEG splat-coding stream. The shared thing is the *problem and the approach*
|
|
(a learned 3D decode that needs to be reproducible), not the wire format. A
|
|
future `wai.pointcloud.*` that byte-exactly decodes a published 3D standard's
|
|
codestream would be the 3D analogue of what `wai.image.jpegai` is for JPEG AI —
|
|
and, because JPEG-Pleno PCC reuses JPEG AI for its colour attributes, the
|
|
byte-exact JPEG-AI decode WAI already has is a component that path could build
|
|
on. Not built; recorded as the natural next capability if the 3D neural-coding
|
|
standards land in scope.
|
|
|
|
## Takeaway
|
|
|
|
The determinism contract is not image-specific. WAI's byte-exact family already
|
|
spans image (`jpegai`, `int_hyper`), audio (`int_codec`), and 3D
|
|
(`splat.int_codec`); the reconstruction binding and the tier vocabulary apply to
|
|
all of them. The emerging 3D neural-coding standards face the same
|
|
non-determinism WAI addresses for 2D, and the JPEG-AI decode WAI has is a
|
|
component of the point-cloud path — so the ingest is: keep the determinism
|
|
contract media-general, and treat a byte-exact decode of a published 3D standard
|
|
as the natural next capability, not a separate track.
|