df20c392d8
Adds the "carries whatever it can reconstruct, verifies whatever it
carries" surface on top of the existing neural/codec WAI. All builds
green against this base (wai-rs all-features tests + clippy, wai-web
wasm, wai-spz-gen, wai-relay).
Reconstruct half — each a spec + conformance corpus + in-tab browser
sink that re-derives the corpus hash, over one wai.det.fixed64 floor:
- interactive worlds replay-equivalence (3 sim registrants,
relay multiplayer, session receipts, glTF + real-SPZ assets)
- spatial audio mixdown-equivalence
- haptics sample-equivalence
- state-feeds replay-equivalence (non-physics sim)
- avatars / digital humans pose-equivalence (+ telepresence)
- volumetric / 4D splats frame-state-equivalence (WebGPU EWA render)
Verify half (lever 4):
- asset provenance — signed, energy-accounted lineage for opaque
captured content WAI does not reconstruct; tamper-evident in-tab.
Cross-cutting: per-class JWP receipt profiles; a native energy meter
(IOReport via macmon, no sudo) emitting receipts with real measured
joules. Codecs gated behind a `codecs` feature (default on) so the
world/wasm sinks build without the C codec chain.
Excludes the separately-evolving neural subsystem (semantic path, GLC
rANS, ONNX tooling) to avoid reverting newer origin work.
Co-Authored-By: Claude Opus 4.8 <[email protected]>
112 lines
4.8 KiB
Markdown
112 lines
4.8 KiB
Markdown
# WAI Extension: Volumetric / 4D Splats
|
||
|
||
> Status: Draft. Phase 5 of the cargo roadmap (the frontier). A scene
|
||
> that is volumetric *and* moves: ship a Gaussian cloud as per-Gaussian
|
||
> keyframed tracks, the sink interpolates the cloud at any frame and
|
||
> renders it with the EWA pipeline. Conformance is
|
||
> **frame-state-equivalence**. Reference impl: the `splat4d` feature of
|
||
> `wai-rs`; corpus: `volumetric-conformance/`; live sink:
|
||
> wai.transaction.science/volumetric.
|
||
Keywords **MUST**, **MUST NOT**, **SHOULD**, **MAY** are RFC 2119/8174.
|
||
|
||
## 1. Scope and model
|
||
|
||
A `wai.splat.4d` object is a **temporal Gaussian splat**: the static
|
||
splat (`wai.splat.spz`) is a single anisotropic-Gaussian cloud; this
|
||
extension gives each Gaussian a keyframed track — position, scale,
|
||
rotation, color, opacity over time — and the sink reconstructs the cloud
|
||
at any frame by interpolating the tracks in the `wai.det.fixed64` floor.
|
||
It is lever 3 (instructions-at-the-sink) for volumetric capture: the wire
|
||
carries the trajectories, the sink reconstructs the frame.
|
||
|
||
In scope: the container, the per-Gaussian keyframe model, the
|
||
deterministic reconstruct contract, the **frame-state-equivalence**
|
||
conformance criterion, and the receipt profile.
|
||
|
||
Out of scope: a transport; the render (EWA splatting, depth sort, alpha
|
||
blend — the sink's, exactly as the static splat's rasterization is); a
|
||
codec for the cloud bytes (a real capture is authored by an upstream
|
||
3DGS/4DGS pipeline; this extension carries and verifies its trajectories,
|
||
it does not fit Gaussians to video).
|
||
|
||
Relationship to the core spec: mirrors haptics / spatial audio. The
|
||
reconstruct runs in the `wai.det.fixed64` floor; `media` takes the
|
||
informational value `"volumetric"`; `wai.splat.4d` is authoritative.
|
||
|
||
## 2. Container (`W4DS`)
|
||
|
||
```
|
||
"W4DS" | u16 n_sections | section table (u8 kind, u32 off, u32 len) | sections
|
||
```
|
||
|
||
| kind | section | required |
|
||
|---|---|---|
|
||
| `0x01` | contract (canonical JSON) | REQUIRED |
|
||
| `0x02` | Gaussian tracks | REQUIRED |
|
||
|
||
### 2.1 Contract
|
||
|
||
```json
|
||
{ "frame_rate": 30, "n_frames": 13, "n_gaussians": 2,
|
||
"numeric": "wai.det.fixed64" }
|
||
```
|
||
|
||
### 2.2 Gaussian tracks
|
||
|
||
`n_gaussians u32`, then per Gaussian: `n_keys u16`, then
|
||
`n × ( frame u64 | pos[3] | scale[3] | rot[4] | color[3] | opacity )`,
|
||
the 14 attributes as `i64` Fx in that field order. Each track MUST be
|
||
ascending by `frame`. `color` and `opacity` are in `[0, 1]`; `rot` is a
|
||
quaternion `(x, y, z, w)`.
|
||
|
||
## 3. Reconstruct contract
|
||
|
||
For each Gaussian, at frame `f`: each of the 14 attributes is the `Fx`
|
||
linear interpolation of the track's keyframes at `f`; before the first /
|
||
after the last key the endpoint holds. The canonical reconstruction of a
|
||
frame is `n_gaussians u32 | per-Gaussian 14×i64` in the field order
|
||
above. The floor stays trig-free — rotations interpolate component-wise
|
||
(nlerp without renormalization, deterministic); the sink normalizes the
|
||
quaternion for rendering, which is presentation.
|
||
|
||
## 4. Conformance — frame-state-equivalence
|
||
|
||
> Given the same `wai.splat.4d`, a conforming sink MUST reconstruct the
|
||
> identical Gaussian set at every frame, hence the identical BLAKE3 — on
|
||
> every machine, no tolerance parameter.
|
||
|
||
```
|
||
frame_hash = BLAKE3("wai:splat4d-frame\x01" || frame_le || canonical frame bytes)
|
||
clip_hash = BLAKE3("wai:splat4d-clip\x01" || contract_bytes || tracks)
|
||
```
|
||
|
||
`frame_hash` is the temporal counterpart of the static splat's content
|
||
hash; `clip_hash` is the frame-independent identity.
|
||
|
||
## 5. Fidelity and size (honest)
|
||
|
||
A splat **approximates** a captured scene, so `wai.splat.4d` is tiered
|
||
`Lossy` like `wai.splat.spz` — even though the per-frame reconstruction
|
||
from the tracks is *frame-state-exact*. Volumetric is **heavy**: a real
|
||
cloud is megabytes per frame. The win here is the same as the static
|
||
splat — content-addressing, dedup, one signed envelope, and a verifiable
|
||
per-frame state — **not** payload reduction. We state that plainly rather
|
||
than imply a compression claim the format does not make.
|
||
|
||
## 6. Receipt (JWP profile)
|
||
|
||
`clip_hash` is the identity; the group's objects are per-frame
|
||
reconstruction blocks (`frame_hash` per frame), Merkle-bound; one
|
||
Ed25519 signature over clip_hash + root + `work` (= n_frames ×
|
||
n_gaussians) + carried `joules_micro`; `parent_receipt_hash` chains — the
|
||
worlds / audio / haptics receipt shape reused.
|
||
|
||
## Appendix
|
||
|
||
Frame-state-equivalence is sample-equivalence with the haptic sample
|
||
replaced by a whole Gaussian cloud, and the static splat extended to
|
||
time. The same fixed-point floor that makes a world replay, a mix sum, an
|
||
actuator buzz, a feed converge, and a body pose now makes a volumetric
|
||
frame reconstruct — one floor, six cargo classes, one verification. The
|
||
representation is heavy and approximate; the *reconstruction* of it is
|
||
exact and auditable, and that is the honest thing the standard adds.
|