Watch

Transaction Science fb103ac5ae wai: one registry table, candidate capabilities, public-text checks

Every registered string now lives in one table (codecs/registry.rs) with
its kind, status, media class, registering SPEC section, edition and
registration date. `wai registry export` writes registry/capabilities.json,
its JSON Schema, the generated table in REGISTRY.md and vocab v2; --check
fails on drift. vocab v1 stays stable.

A candidate capability (a registered string whose payload format no
revision has pinned) must not be emitted. The C packers, the browser
writers and `wai wrap` refuse one, and `codecs::is_dispatchable` is the
single dispatch and selection predicate (SPEC §4): candidates,
companion-only capabilities, records and derivations are not
dispatchable, and an unregistered string is decoded only where the sink
advertises it.

The codec glossary is linked to the registry through a reviewed table and
held to its schema by a library test; `org` becomes `body`, from a closed
allowlist. The SPEC gains §5 sections on kinds and candidates, a §8
edition rule, corrected citations and rewritten appendices. The
quantum-ops vectors are regenerated with neutral device ids.

CI declares the feature-gated test targets with required-features, checks
the registry export, builds wai-web for wasm32, and runs two public-text
checks (comparative prose, and private name patterns) with a self-test.
The specification, extensions, research notes, READMEs, glossary and
source comments are rewritten to name no company or product and to make
no comparisons.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

2026-09-30 22:38:09 -04:00

6.4 KiB

Raw Permalink Blame History

3D Neural Coding — where the determinism contract goes next

Landscape note, not a normative extension. Learned coding is moving from flat media into 3D — point clouds and Gaussian splats — and it carries the same cross-platform non-determinism that flat learned codecs have. WAI already decodes one such 3D representation byte-exactly (wai.splat.int_codec); this records how the determinism contract and the reconstruction binding generalize to the emerging 3D coding standards, and where WAI does and does not connect to them. Framed self-referentially — the standards are named as standards, no comparisons.

The landscape

The learned image-codec machinery is being reused one dimension up:

JPEG Pleno Learning-based Point Cloud Coding is reported to reuse JPEG AI to code the point cloud's colour attributes — the same learned 2-D transform, applied to attribute images. (Not verified here: the tree records no source for this, and none was read for this note's 2026-09-30 revision.)
Gaussian splat coding is being standardized in MPEG along two paths, per MPEG's pages and public output documents read on 2026-09-30 (listed in media-codec-registry.md, MPEG's Gaussian-splat and point-cloud work): amendments that add Gaussian splats to the two point-cloud standards, V-PCC (ISO/IEC 23090-5, at Draft Amendment) and G-PCC (ISO/IEC 23090-9, at Committee Draft Amendment); and a joint exploration (Exploration 45), coordinated across WG 4, WG 5/JVET and WG 7 of ISO/IEC JTC 1/SC 29, that is preparing a call for proposals for content in the I-3DGS representation. Separately, a Committee Draft amendment of Neural Network Coding (NNC) extends NNC to "parameters of 3D and 4D scene representations such as Gaussian splats" (2030 landscape §6.4). No source read here establishes a JPEG activity on splat coding.
AI-based point cloud coding is MPEG-AI Part 5 (ISO/IEC 23888-5); its part page lists common test conditions at MPEG 155 (2026-09 sweep §2.1).

Where these paths use a learned decode, they inherit the property that made the flat learned codecs hard to deploy: a float decode drifts by round-off, the entropy stage desyncs, and the reconstruction is reproducible only on the same build. Whether a given path's decode is learned is not established here: the amendment texts are not public, and the call for proposals has selected no technology. The V-PCC amendment specifies a reconstruction profile because "without a clearly specified reconstruction profile, different implementations could decode the same bitstream into different Gaussian Splat parameters" (ISO/IEC JTC 1/SC 29/WG 4 N 837).

What WAI already has

wai.splat.int_codec is a byte-exact learned Gaussian-splat attribute codec: a learned transform over the [14, S, S] attribute image (position, scale, rotation, colour, opacity), integer synthesis over a factorized-rANS latent, pure i64 — no float, no ONNX. Its conformance criterion is splat-equivalence: the decoded attribute tensor is byte-identical on every machine, a portable cloud hash. The float rasterizer stays sink-supplied presentation; the attributes are the contract.

That is the 3D instance of the same thesis WAI applies to its integer image codecs (wai.neural.int_hyper): the learned decode to the medium's canonical representation is pinned and integer, so the reconstruction has a portable hash. wai.splat.int_codec already sits at the decode-equivalence tier (see extensions/determinism-tiers.md) — the attribute tensor is the "final medium" whose bytes reproduce.

Where it connects — and where it doesn't

The contract generalizes. decode-equivalence + a reproducible-reconstruction binding over a decoded 3D representation is exactly the same construction as over decoded pixels: hash the canonical decoded buffer (here the attribute tensor, or a point cloud's decoded attributes), pin the descriptor, and the rendered-content provenance binding neural-coded media could not carry becomes available for 3D too. As the 3D standards adopt learned coding, the determinism-for-provenance gap is the same one WAI closes for 2D.
Scope — WAI does not decode the ISO/IEC point-cloud bitstreams. wai.splat.int_codec is a WAI-native integer codec, not a JPEG Pleno or MPEG bitstream; the shared thing is the problem and the approach, not the wire format. WAI already registers MPEG's point-cloud standards as sink-supplied capabilities — wai.pc.gpcc (G-PCC, ISO/IEC 23090-9) and wai.pc.vpcc (V-PCC, ISO/IEC 23090-5), replicate class StandardUnclassified, with no decoder in wai-rs — and the research registry records a sink-side G-PCC decode and a V-PCC encode with a structural V3C parse through the MPEG reference tools. A decoder in WAI that reproduces a registered standard's reconstruction byte for byte would decode that capability's existing payload, as WAI's exact JPEG AI decoder decodes wai.image.jpegai; it adds no capability string. A 3D standard WAI does not register yet would need a string of its own (SPEC §5, §8); which editions and amendments the two wai.pc.* strings carry follows SPEC §8's edition rule, and what remains open is recorded in media-codec-registry.md, open item (i). If JPEG Pleno's learning-based coding reuses JPEG AI for colour attributes (unverified, above), WAI's JPEG AI decoder — integer entropy decode and synthesis networks, with wai.image.jpegai registered at entropy-consistency because its later stages are float — is a component that path could build on. Not built.

Takeaway

The determinism contract is not image-specific. WAI's byte-exact family — the registry's NeuralIntegerExact class (codecs::capability_replicate_class) — already spans image, video, audio and 3D, with jpegai at entropy-consistency beside it; the reconstruction binding and the tier vocabulary apply to all of them. The emerging 3D coding standards that adopt a learned decode face the same non-determinism WAI addresses for 2D, so the ingest is: keep the determinism contract media-general, and treat a byte-exact decode of a published 3D standard as the natural next capability, not a separate track.

6.4 KiB Raw Permalink Blame History