open-standards/wai/extensions/volumetric.md

# WAI Extension: Volumetric / 4D Splats

> Status: Draft. Phase 5 of the cargo roadmap (the frontier). A scene
> that is volumetric *and* moves: ship a Gaussian cloud as per-Gaussian
> keyframed tracks, the sink interpolates the cloud at any frame and
> renders it with the EWA pipeline. Conformance is
> **frame-state-equivalence**. Reference impl: the `splat4d` feature of
> `wai-rs`; corpus: `volumetric-conformance/`; live sink:
> wai.transaction.science/volumetric.
Keywords **MUST**, **MUST NOT**, **SHOULD**, **MAY** are RFC 2119/8174.

## 1. Scope and model

A `wai.splat.4d` object is a **temporal Gaussian splat**: the static
splat (`wai.splat.spz`) is a single anisotropic-Gaussian cloud; this
extension gives each Gaussian a keyframed track — position, scale,
rotation, color, opacity over time — and the sink reconstructs the cloud
at any frame by interpolating the tracks in the `wai.det.fixed64` floor.
It is lever 3 (instructions-at-the-sink) for volumetric capture: the wire
carries the trajectories, the sink reconstructs the frame.

In scope: the container, the per-Gaussian keyframe model, the
deterministic reconstruct contract, the **frame-state-equivalence**
conformance criterion, and the receipt profile.

Out of scope: a transport; the render (EWA splatting, depth sort, alpha
blend — the sink's, exactly as the static splat's rasterization is); a
codec for the cloud bytes (a real capture is authored by an upstream
3DGS/4DGS pipeline; this extension carries and verifies its trajectories,
it does not fit Gaussians to video).

Relationship to the core spec: mirrors haptics / spatial audio. The
reconstruct runs in the `wai.det.fixed64` floor; `media` takes the
informational value `"volumetric"`; `wai.splat.4d` is authoritative.

## 2. Container (`W4DS`)

```
"W4DS" | u16 n_sections | section table (u8 kind, u32 off, u32 len) | sections
```

| kind | section | required |
|---|---|---|
| `0x01` | contract (canonical JSON) | REQUIRED |
| `0x02` | Gaussian tracks | REQUIRED |

### 2.1 Contract

```json
{ "frame_rate": 30, "n_frames": 13, "n_gaussians": 2,
  "numeric": "wai.det.fixed64" }
```

**Reader limit.** A reader MUST NOT refuse a clip for its length while `n_frames` is at
most 1 048 576 (2²⁰, the video limit of SPEC §2), and MAY refuse one past it
(`duration_too_long`), when it reads the contract, before it reconstructs anything. A
conforming encoder writes no clip past it.

### 2.2 Gaussian tracks

`n_gaussians u32`, then per Gaussian: `n_keys u16`, then
`n × ( frame u64 | pos[3] | scale[3] | rot[4] | color[3] | opacity )`,
the 14 attributes as `i64` Fx in that field order. Each track MUST be
ascending by `frame`. `color` and `opacity` are in `[0, 1]`; `rot` is a
quaternion `(x, y, z, w)`.

## 3. Reconstruct contract

For each Gaussian, at frame `f`: each of the 14 attributes is the `Fx`
linear interpolation of the track's keyframes at `f`; before the first /
after the last key the endpoint holds. The canonical reconstruction of a
frame is `n_gaussians u32 | per-Gaussian 14×i64` in the field order
above. The floor stays trig-free — rotations interpolate component-wise
(nlerp without renormalization, deterministic); the sink normalizes the
quaternion for rendering, which is presentation.

## 4. Conformance — frame-state-equivalence

> Given the same `wai.splat.4d`, a conforming sink MUST reconstruct the
> identical Gaussian set at every frame, hence the identical BLAKE3 — on
> every machine, no tolerance parameter.

```
frame_hash = BLAKE3("wai:splat4d-frame\x01" || frame_le || canonical frame bytes)
clip_hash  = BLAKE3("wai:splat4d-clip\x01"  || contract_bytes || tracks)
```

`frame_hash` is the temporal counterpart of the static splat's content
hash; `clip_hash` is the frame-independent identity.

## 5. Fidelity and size (honest)

A splat **approximates** a captured scene, so `wai.splat.4d` is tiered
`Lossy` like `wai.splat.spz` — even though the per-frame reconstruction
from the tracks is *frame-state-exact*. Volumetric is **heavy**: a real
cloud is megabytes per frame. The win here is the same as the static
splat — content-addressing, dedup, one signed envelope, and a verifiable
per-frame state — **not** payload reduction. We state that plainly rather
than imply a compression claim the format does not make.

## 6. Receipt (JWP profile)

`clip_hash` is the identity; the group's objects are per-frame
reconstruction blocks (`frame_hash` per frame), Merkle-bound; one
Ed25519 signature (verified as SPEC §7 *Signatures* requires) over clip_hash + root + `work` (= n_frames ×
n_gaussians) + carried `joules_micro`; `parent_receipt_hash` chains — the
worlds / audio / haptics receipt shape reused. The figure's acquisition class rides beside it as an optional signed label
([energy-measurement](energy-measurement.md) §2.8): absent, the receipt keeps
its legacy bytes and the figure is unlabelled; present, it is inside the
signature. The reference meter labels every figure it measures
`OnChipCounter`, with its declared uncertainty.

## Appendix

Frame-state-equivalence is sample-equivalence with the haptic sample
replaced by a whole Gaussian cloud, and the static splat extended to
time. The same fixed-point floor that makes a world replay, a mix sum, an
actuator buzz, a feed converge, and a body pose now makes a volumetric
frame reconstruct — one floor, six cargo classes, one verification. The
representation is heavy and approximate; the *reconstruction* of it is
exact and auditable, and that is the honest thing the standard adds.