fb103ac5ae
Every registered string now lives in one table (codecs/registry.rs) with its kind, status, media class, registering SPEC section, edition and registration date. `wai registry export` writes registry/capabilities.json, its JSON Schema, the generated table in REGISTRY.md and vocab v2; --check fails on drift. vocab v1 stays stable. A candidate capability (a registered string whose payload format no revision has pinned) must not be emitted. The C packers, the browser writers and `wai wrap` refuse one, and `codecs::is_dispatchable` is the single dispatch and selection predicate (SPEC §4): candidates, companion-only capabilities, records and derivations are not dispatchable, and an unregistered string is decoded only where the sink advertises it. The codec glossary is linked to the registry through a reviewed table and held to its schema by a library test; `org` becomes `body`, from a closed allowlist. The SPEC gains §5 sections on kinds and candidates, a §8 edition rule, corrected citations and rewritten appendices. The quantum-ops vectors are regenerated with neutral device ids. CI declares the feature-gated test targets with required-features, checks the registry export, builds wai-web for wasm32, and runs two public-text checks (comparative prose, and private name patterns) with a self-test. The specification, extensions, research notes, READMEs, glossary and source comments are rewritten to name no company or product and to make no comparisons. Co-Authored-By: Claude Opus 5.5 <[email protected]>
101 lines
6.4 KiB
Markdown
101 lines
6.4 KiB
Markdown
# 3D Neural Coding — where the determinism contract goes next
|
|
|
|
> Landscape note, not a normative extension. Learned coding is moving from flat
|
|
> media into 3D — point clouds and Gaussian splats — and it carries the same
|
|
> cross-platform non-determinism that flat learned codecs have. WAI already
|
|
> decodes one such 3D representation byte-exactly (`wai.splat.int_codec`); this
|
|
> records how the determinism contract and the reconstruction binding generalize
|
|
> to the emerging 3D coding standards, and where WAI does and does not connect to
|
|
> them. Framed self-referentially — the standards are named as standards, no
|
|
> comparisons.
|
|
|
|
## The landscape
|
|
|
|
The learned image-codec machinery is being reused one dimension up:
|
|
|
|
- **JPEG Pleno Learning-based Point Cloud Coding** is reported to reuse JPEG AI to
|
|
code the point cloud's colour attributes — the same learned 2-D transform,
|
|
applied to attribute images. *(Not verified here: the tree records no source for
|
|
this, and none was read for this note's 2026-09-30 revision.)*
|
|
- **Gaussian splat coding** is being standardized in MPEG along two paths, per
|
|
MPEG's pages and public output documents read on 2026-09-30 (listed in
|
|
[`media-codec-registry.md`](media-codec-registry.md), *MPEG's Gaussian-splat and
|
|
point-cloud work*): amendments that add Gaussian splats to the two point-cloud
|
|
standards, V-PCC (ISO/IEC 23090-5, at Draft Amendment) and G-PCC (ISO/IEC
|
|
23090-9, at Committee Draft Amendment); and a joint exploration (Exploration 45),
|
|
coordinated across WG 4, WG 5/JVET and WG 7 of ISO/IEC JTC 1/SC 29, that is
|
|
preparing a call for proposals for content in the I-3DGS representation.
|
|
Separately, a Committee Draft amendment of **Neural Network Coding (NNC)**
|
|
extends NNC to "parameters of 3D and 4D scene representations such as Gaussian
|
|
splats" ([2030 landscape](2030-transport-landscape.md) §6.4). No source read here
|
|
establishes a JPEG activity on splat coding.
|
|
- **AI-based point cloud coding** is MPEG-AI Part 5 (ISO/IEC 23888-5); its part
|
|
page lists common test conditions at MPEG 155
|
|
([2026-09 sweep](2026-09-global-sweep.md) §2.1).
|
|
|
|
Where these paths use a learned decode, they inherit the property that made the
|
|
flat learned codecs hard to deploy: a float decode drifts by round-off, the
|
|
entropy stage desyncs, and the reconstruction is reproducible only on the same
|
|
build. Whether a given path's decode is learned is not established here: the
|
|
amendment texts are not public, and the call for proposals has selected no
|
|
technology. The V-PCC amendment specifies a reconstruction profile because
|
|
"without a clearly specified reconstruction profile, different implementations
|
|
could decode the same bitstream into different Gaussian Splat parameters"
|
|
(ISO/IEC JTC 1/SC 29/WG 4 N 837).
|
|
|
|
## What WAI already has
|
|
|
|
`wai.splat.int_codec` is a byte-exact learned Gaussian-splat **attribute** codec:
|
|
a learned transform over the `[14, S, S]` attribute image (position, scale,
|
|
rotation, colour, opacity), integer synthesis over a factorized-rANS latent, pure
|
|
`i64` — no float, no ONNX. Its conformance criterion is **splat-equivalence**: the
|
|
decoded attribute tensor is byte-identical on every machine, a portable cloud
|
|
hash. The float rasterizer stays sink-supplied presentation; the *attributes* are
|
|
the contract.
|
|
|
|
That is the 3D instance of the same thesis WAI applies to its integer image
|
|
codecs (`wai.neural.int_hyper`): the learned decode to the medium's canonical
|
|
representation is pinned and integer, so the reconstruction has a portable hash. `wai.splat.int_codec` already sits at the
|
|
**`decode-equivalence`** tier (see [`extensions/determinism-tiers.md`](../extensions/determinism-tiers.md)) — the
|
|
attribute tensor is the "final medium" whose bytes reproduce.
|
|
|
|
## Where it connects — and where it doesn't
|
|
|
|
- **The contract generalizes.** `decode-equivalence` + a
|
|
[reproducible-reconstruction binding](../extensions/reconstruction-binding.md)
|
|
over a decoded 3D representation is exactly the same construction as over decoded
|
|
pixels: hash the canonical decoded buffer (here the attribute tensor, or a point
|
|
cloud's decoded attributes), pin the descriptor, and the rendered-content
|
|
provenance binding neural-coded media could not carry becomes available for 3D
|
|
too. As the 3D standards adopt learned coding, the determinism-for-provenance gap
|
|
is the same one WAI closes for 2D.
|
|
- **Scope — WAI does not decode the ISO/IEC point-cloud bitstreams.**
|
|
`wai.splat.int_codec` is a WAI-native integer codec, not a JPEG Pleno or MPEG
|
|
bitstream; the shared thing is the *problem and the approach*, not the wire
|
|
format. WAI already registers MPEG's point-cloud standards as sink-supplied
|
|
capabilities — `wai.pc.gpcc` (G-PCC, ISO/IEC 23090-9) and `wai.pc.vpcc` (V-PCC,
|
|
ISO/IEC 23090-5), replicate class `StandardUnclassified`, with no decoder in
|
|
`wai-rs` — and the research registry records a sink-side G-PCC decode and a V-PCC
|
|
encode with a structural V3C parse through the MPEG reference tools. A decoder in
|
|
WAI that reproduces a registered standard's reconstruction byte for byte would
|
|
decode that capability's existing payload, as WAI's exact JPEG AI decoder decodes
|
|
`wai.image.jpegai`; it adds no capability string. A 3D standard WAI does not
|
|
register yet would need a string of its own (SPEC §5, §8); which editions and
|
|
amendments the two `wai.pc.*` strings carry follows SPEC §8's edition rule, and
|
|
what remains open is recorded in [`media-codec-registry.md`](media-codec-registry.md),
|
|
open item (i). If JPEG Pleno's learning-based coding reuses JPEG AI for colour
|
|
attributes (unverified, above), WAI's JPEG AI decoder — integer entropy decode
|
|
and synthesis networks, with `wai.image.jpegai` registered at
|
|
`entropy-consistency` because its later stages are float — is a component that
|
|
path could build on. Not built.
|
|
|
|
## Takeaway
|
|
|
|
The determinism contract is not image-specific. WAI's byte-exact family — the
|
|
registry's `NeuralIntegerExact` class (`codecs::capability_replicate_class`) —
|
|
already spans image, video, audio and 3D, with `jpegai` at `entropy-consistency`
|
|
beside it; the reconstruction binding and the tier vocabulary apply to all of
|
|
them. The emerging 3D coding standards that adopt a learned decode face the same
|
|
non-determinism WAI addresses for 2D, so the ingest is: keep the determinism
|
|
contract media-general, and treat a byte-exact decode of a published 3D standard
|
|
as the natural next capability, not a separate track.
|