614 lines
36 KiB
Markdown
614 lines
36 KiB
Markdown
# WAI Extension: Exact Splats
|
||
|
||
> Status: Draft. Registers the record `wai.splat.exact`: a Gaussian splat set that travels in a
|
||
> glTF 2.0 asset as an integer-coded WAI object, and a renderer whose frames are the same bytes
|
||
> on every machine, so a frame can be signed. A `wai.splat.int_codec` object rides in a mesh
|
||
> primitive of the asset through the draft compression extension
|
||
> `EXT_gaussian_splatting_compression_wai`, nested in the primitive's `KHR_gaussian_splatting`
|
||
> object: a buffer view holds the object's `WAI1` envelope, and the extension states its content
|
||
> hash and its parameter set (§3). The decode is integer throughout, so the attribute planes are
|
||
> bit-exact on every machine, and §4 maps them onto the base extension's attributes. The renderer
|
||
> `wai-splat-render:i1` (§5) is integer too: a fixed-point camera and projection, an order by
|
||
> squared camera distance with the Gaussian's index breaking every tie, alpha compositing with
|
||
> each rounding stated, and 16 × 16 tiles in raster order. The origin signs a **commitment**
|
||
> (§6): the object, its parameter set, the digest of its decoded attributes, and the SHA-256 of
|
||
> the frame at each of a set of **canonical poses**, so a viewer recomputes what the scene looks
|
||
> like from each of them, bit for bit. A sink signs a **view statement** for a frame it
|
||
> presented at a pose (§7). A view is a conversion of the splat set under the class lattice of
|
||
> [synthesis](synthesis.md) §3.2, so it is never labelled lower than the splat set it shows (§8).
|
||
> No new transport. The reference implementation is the `splat_exact` module of `wai-rs`
|
||
> (feature `splat_exact`); the corpus is [`live-conformance/splat/`](../live-conformance/splat/),
|
||
> and its second implementation is `live-conformance/splat_exact_verify.py`.
|
||
> Keywords **MUST**, **MUST NOT**, **SHOULD**, **MAY** are RFC 2119/8174.
|
||
>
|
||
> Texts read for this revision, each pinned to the version named: glTF 2.0 as ISO/IEC
|
||
> 12113:2022, and the glTF extension registry's repository as it stood on 22 October 2026: the
|
||
> mesh primitive extension `KHR_gaussian_splatting`, whose text states its status as complete and
|
||
> ratified, added by pull request #2490 (merged 27 January 2026) and last changed by #2677 (merged
|
||
> 13 October 2026), with its schema `mesh.primitive.KHR_gaussian_splatting.schema.json`; the
|
||
> compression extensions proposed for it and still under review on that date,
|
||
> `KHR_gaussian_splatting_compression_spz_2` (pull request #2531), `KHR_gaussian_splatting_lgsc`
|
||
> (#2551) and `KHR_gaussian_splatting_compression_EGSC` (#2605); the earlier single proposal
|
||
> those replaced, `KHR_spz_gaussian_splats_compression` (draft #2486, closed); and the general
|
||
> compression extension `KHR_meshopt_compression` (#2517, merged). A section number of one of
|
||
> those texts in this extension is that version's.
|
||
|
||
## 1. Scope
|
||
|
||
In scope: the records and their encodings (§2); how a glTF asset carries a `wai.splat.int_codec`
|
||
object and the draft extension's schema (§3); the attribute layout, its dequantization and its
|
||
mapping onto `KHR_gaussian_splatting` (§4); the renderer `wai-splat-render:i1` (§5); canonical
|
||
poses and the commitment (§6); the view statement (§7); provenance (§8); verification (§9).
|
||
|
||
Out of scope: a transport; how a splat set is trained or encoded; the glTF node hierarchy (a
|
||
pose is in the primitive's own frame, §2); view-dependent colour (spherical harmonics of degree
|
||
above 0); lighting. WAI does not submit this extension to the glTF extension registry; its name
|
||
is WAI's draft, and §3.5 says what would change if the registry took up a WAI form.
|
||
|
||
**Why a compression extension.** `KHR_gaussian_splatting` defines a splat primitive and admits
|
||
compression extensions nested in its own `extensions` object: such an extension says how its data
|
||
decodes back to the base attributes. The base text fixes the order of a render (back to front by
|
||
camera distance) and its 3σ kernel, and leaves the arithmetic to the renderer. This extension
|
||
carries an integer-coded object whose decode is fixed to the bit, and pins a renderer, so the step
|
||
from bytes to pixels is the same on every machine.
|
||
|
||
**Why a pinned renderer.** A splat renderer that sorts, projects and blends in floating point
|
||
does so in orders and widths that differ between devices and drivers: two viewers of one asset
|
||
see different pixels, and a frame cannot be signed. §12 measures it: the same floating-point
|
||
renderer in 32-bit and in 64-bit arithmetic gives different bytes for views of the corpus, and
|
||
for more of them the larger the frame. `wai-splat-render:i1` is integer from the attribute bytes
|
||
to the frame, so its frame is a function of the asset and the pose, and its digest can be
|
||
committed to and signed.
|
||
|
||
Relationship to the core spec: `wai.splat.exact` is a **record** ([SPEC](../SPEC.md) §5): it names
|
||
no payload format and is not dispatchable (SPEC §4). The payload it carries is
|
||
`wai.splat.int_codec`'s (`WIS1`, [integer-payloads](integer-payloads.md) §6).
|
||
|
||
## 2. Records
|
||
|
||
Every record of this extension is a JSON object (RFC 8259) under the encodings of
|
||
[jwp-receipts](jwp-receipts.md) §2.4.3: byte strings in lowercase hexadecimal; integers unsigned,
|
||
at most 2^53 − 1, in plain decimal; no other number; no boolean; a **term** and a **date-time** as
|
||
defined there. A record is read as jwp-receipts §3.1 step 1 reads a claim: RFC 8259's grammar, no
|
||
member named twice in one object, at most 32 levels of nesting. Each record has exactly the
|
||
members its section lists, no member missing and none added, at any level, and a `kind` naming it;
|
||
one that does not is `record_malformed`.
|
||
|
||
Every record carries `signer_pubkey` (32 bytes, the Ed25519 key of RFC 8032 that signs it),
|
||
`issued_at` (a date-time, when its signer signed it, self-asserted) and `sig` (64 bytes):
|
||
|
||
```
|
||
sig = Ed25519(signer_key, "wai:splat-exact\x01" || JCS(record without "sig"))
|
||
link_hash = BLAKE3("wai:splat-exact-link\x01" || JCS(record))
|
||
```
|
||
|
||
JCS is RFC 8785. The signature is verified strictly ([SPEC](../SPEC.md) §7 *Signatures*): a key
|
||
that fails step 1 or 2 is `bad_key`, and a signature that fails a later step `bad_signature`. A
|
||
record is named by its link hash wherever another names it. An object's **content hash** is
|
||
jwp-receipts §2.1's: BLAKE3 over `moq-jwp:object\x01` and the envelope's bytes.
|
||
|
||
A **signed integer** is a string: `0`, or an optional `-` and a digit 1–9 followed by digits (so
|
||
`-0`, `+1` and `01` are not one), of magnitude at most 2^28.
|
||
|
||
A **pose** is an object of exactly these members:
|
||
|
||
| member | type | meaning |
|
||
|---|---|---|
|
||
| `eye` | 3 signed integers | the camera's position, at 2^16, in the primitive's own frame (the node transform is not applied) |
|
||
| `target` | 3 signed integers | the point it looks at, at 2^16 |
|
||
| `up` | 3 signed integers | its up direction, at 2^16, of any length |
|
||
| `focal` | integer | the focal length in pixels, 1 to 4096 |
|
||
| `width`, `height` | integer | the frame's size in pixels, each 1 to 1024, their product at most 2^20 |
|
||
|
||
A pose that breaks a rule of this table is `record_malformed`. Two poses are the same pose when
|
||
every member is equal.
|
||
|
||
## 3. Carriage in glTF
|
||
|
||
### 3.1 The primitive
|
||
|
||
A splat set travels as one mesh primitive of a glTF 2.0 asset (ISO/IEC 12113:2022; its
|
||
`asset.version` is `"2.0"`) that carries
|
||
the extension object `KHR_gaussian_splatting`, and in it, under that object's own `extensions`,
|
||
the object `EXT_gaussian_splatting_compression_wai` (§3.2). The primitive:
|
||
|
||
- has `mode` 0 (`POINTS`), written;
|
||
- holds `KHR_gaussian_splatting` with `kernel` `"ellipse"` and `colorSpace`
|
||
`"srgb_rec709_display"`, and `projection` and `sortingMethod` absent or `"perspective"` and
|
||
`"cameraDistance"`;
|
||
- names exactly five attributes, each an accessor of component type `FLOAT` (5126) with no
|
||
`bufferView` and a `count`: `POSITION` (`VEC3`), `KHR_gaussian_splatting:ROTATION` (`VEC4`),
|
||
`KHR_gaussian_splatting:SCALE` (`VEC3`), `KHR_gaussian_splatting:OPACITY` (`SCALAR`) and
|
||
`KHR_gaussian_splatting:SH_DEGREE_0_COEF_0` (`VEC3`). glTF 2.0 initialises an accessor with no buffer view
|
||
with zeros and lets an extension override them; a sink that implements this extension fills
|
||
them from the decode (§4.3).
|
||
|
||
An asset carries one such primitive in this revision. The asset's `extensionsUsed` and
|
||
`extensionsRequired` both list `KHR_gaussian_splatting` and
|
||
`EXT_gaussian_splatting_compression_wai`: the accessors carry no data, so a reader without this
|
||
extension has nothing to fall back to, and the base extension asks that such an extension be
|
||
required.
|
||
|
||
### 3.2 The extension object
|
||
|
||
`EXT_gaussian_splatting_compression_wai` has these members:
|
||
|
||
| member | type | presence | meaning |
|
||
|---|---|---|---|
|
||
| `bufferView` | integer | required | the buffer view that holds the object's `WAI1` envelope ([SPEC](../SPEC.md) §2), and nothing else |
|
||
| `capability` | string | required | the envelope's capability: `"wai.splat.int_codec"` in this revision |
|
||
| `object` | 32 B hex | required | the envelope's content hash (§2) |
|
||
| `set` | object | required | the parameter set: `{"sha256", "bufferView"}`, its SHA-256 (the envelope's pin, SPEC §3.1) and, where the asset carries the set, the buffer view that holds its one file (`g_s.whm`, [integer-payloads](integer-payloads.md) §9); `bufferView` absent where it does not |
|
||
| `commitment` | integer | optional | the buffer view that holds the commitment's text (§6.1) |
|
||
| `extensions`, `extras` | object | optional | glTF's own |
|
||
|
||
An object that lacks a required member, whose buffer views do not read (a view that names no
|
||
buffer, a buffer whose `byteLength` is not its bytes' length, or a range past its buffer), or
|
||
whose `object` or `set.sha256` is not 32 bytes of hex, is `asset_malformed`. A `capability` that is
|
||
a string other than `"wai.splat.int_codec"` is `capability_mismatch`. The members are read in
|
||
this order: the buffer view of `bufferView`, `capability`, `object`, `set`, `commitment`.
|
||
|
||
The commitment is carried as text in a buffer, not as JSON inside the asset, so it is read as §2
|
||
reads a record, byte for byte, whatever the reader of the asset's JSON does with its numbers and
|
||
members.
|
||
|
||
### 3.3 The envelope, its set and the decode
|
||
|
||
The buffer view holds exactly one `WAI1` envelope: its magic, its manifest's length and manifest,
|
||
its payload's length and payload, and no byte after them (otherwise `asset_malformed`). Then:
|
||
|
||
1. Its capability is `wai.splat.int_codec` (`capability_mismatch`).
|
||
2. Its content hash is the extension's `object` (`object_mismatch`).
|
||
3. Its manifest's pin (`conditioning.prior`, SPEC §3.1) names the extension's `set.sha256`
|
||
(`set_mismatch`). The set is the buffer view `set.bufferView` names, or else the file a
|
||
verifier holds whose SHA-256 is `set.sha256` (`set_not_held` where it holds none). Its SHA-256
|
||
is `set.sha256`, and the pin is verified as SPEC §3.1 has it (`set_mismatch`).
|
||
4. The payload is decoded as [integer-payloads](integer-payloads.md) §6 has it, against that set;
|
||
a payload the decode refuses, with any code of integer-payloads §8, is `decode_refused`.
|
||
|
||
The decode gives `attr` planes of `s × s` bytes and the payload's `n_gaussians` and ranges. Every
|
||
step of it is integer, so these bytes are the same on every machine: the buffer of the
|
||
capability's splat-equivalence digest. Their SHA-256 is the splat set's **attribute digest**.
|
||
|
||
### 3.4 The accessors
|
||
|
||
After the decode, every accessor's `count` is `n_gaussians`, the decode has at least three planes,
|
||
and `POSITION` has `min` and `max`, three numbers each, equal to the least and the greatest
|
||
dequantized coordinate on each axis (§4.2) divided by 2^16. Every such value is exact in binary64,
|
||
so a reader of the asset's JSON reads the same number. Otherwise `asset_malformed`. (glTF 2.0
|
||
requires `min` and `max` on `POSITION`; this makes them the decode's own.)
|
||
|
||
### 3.5 The draft schema, and the registry
|
||
|
||
The extension object's JSON Schema, draft 2020-12, is
|
||
[`gltf/EXT_gaussian_splatting_compression_wai.schema.json`](gltf/EXT_gaussian_splatting_compression_wai.schema.json).
|
||
It states what §3.2 states of each member's type and presence; the rules that need the envelope,
|
||
the set or the decode are this section's, not the schema's.
|
||
|
||
The name has the prefix glTF's registry gives an extension that more than one implementation is
|
||
meant to read. WAI does not propose it to the registry. Were the registry to take up a WAI form,
|
||
the extension object would keep its members, and the commitment and the renderer would stay WAI's:
|
||
they are records and a profile, not part of a decode back to the base attributes.
|
||
|
||
## 4. The attributes
|
||
|
||
### 4.1 The layout
|
||
|
||
This revision renders splat sets of 14 planes (`attr` = 14), the layout `wai.splat.int_codec`
|
||
codes (SPEC §5): Gaussian `k`, for `k` < `n_gaussians`, is cell `k` of every plane, in raster
|
||
order, and its planes are:
|
||
|
||
| planes | attribute |
|
||
|---|---|
|
||
| 0–2 | position *x*, *y*, *z* |
|
||
| 3–5 | scale: the standard deviation along the Gaussian's own *x*, *y*, *z* axes |
|
||
| 6–9 | rotation, a quaternion *w*, *x*, *y*, *z* (of any length) |
|
||
| 10–12 | colour *r*, *g*, *b*, display-referred, the base extension's `srgb_rec709_display` |
|
||
| 13 | opacity |
|
||
|
||
A splat set of another plane count is `layout_mismatch` at the renderer (§5.2).
|
||
|
||
### 4.2 Dequantization
|
||
|
||
Each plane `p` has the range `(lo_p, hi_p)` the payload states as two binary32 values. Each is
|
||
taken at 2^16, rounded half up and computed exactly from the binary32 value:
|
||
`LO_p = ⌊lo_p · 2^16 + 1/2⌋`, `HI_p = ⌊hi_p · 2^16 + 1/2⌋`. A byte `u` of plane `p` is then
|
||
|
||
```
|
||
q = LO_p + ⌊((HI_p − LO_p) · u + 127) / 255⌋
|
||
```
|
||
|
||
the value at 2^16 (the nearest to `lo + (hi − lo)·u/255`, a tie upward). The payload makes
|
||
`lo_p ≤ hi_p`, so `HI_p − LO_p ≥ 0`.
|
||
|
||
### 4.3 The base extension's attributes
|
||
|
||
A sink that fills the accessors (§3.1) writes, for Gaussian `k`, each value rounded to the nearest
|
||
binary32: `POSITION` = (*x*, *y*, *z*); `KHR_gaussian_splatting:ROTATION` = (*x*, *y*, *z*, *w*) of
|
||
the quaternion divided by its length, or (0, 0, 0, 1) where it is zero; `KHR_gaussian_splatting:SCALE`
|
||
= the three scales; `KHR_gaussian_splatting:OPACITY` = the opacity clamped to [0, 1]; and
|
||
`KHR_gaussian_splatting:SH_DEGREE_0_COEF_0` = (*c* − 1/2) / *C*₀ per channel, with *c* the colour
|
||
clamped to [0, 1] and *C*₀ = 1/(2√π), the base extension's degree-0 term; each value being *q* /
|
||
2^16. These float attributes are presentation for a renderer of its own: the attribute digest and
|
||
the renderer of §5 read the bytes and §4.2's integers, never them.
|
||
|
||
## 5. The renderer `wai-splat-render:i1`
|
||
|
||
The renderer maps a splat set's attribute planes and a pose to a frame: `width × height` pixels of
|
||
`RGB8`, rows top to bottom, pixels left to right, three bytes a pixel, no padding — the canonical
|
||
buffer of [reconstruction-binding](reconstruction-binding.md) §2. Every value is an integer and no
|
||
formula wraps or saturates unless it says so.
|
||
|
||
### 5.1 Integer functions
|
||
|
||
`⌊a / b⌋`, `R_half_up(p, s)`, `isqrt`, `isqrt_ceil`, `clamp` and `exp_neg_q16` are
|
||
[integer-operators](integer-operators.md) §2's. In addition, for `b > 0`:
|
||
|
||
```
|
||
rdiv(a, b) = ⌊(2a + b) / 2b⌋ a / b to the nearest integer, a tie toward +∞
|
||
```
|
||
|
||
`a >> s` is `⌊a / 2^s⌋`; `×` and `·` are exact products.
|
||
|
||
### 5.2 A renderable splat set
|
||
|
||
A splat set is renderable when it has 14 planes (`layout_mismatch` otherwise) and, for every plane,
|
||
`|LO_p|` and `|HI_p|` are at most 2^28, and `LO_p ≥ 0` for the scale planes 3–5
|
||
(`range_unrenderable` otherwise). Each Gaussian `k` then has, at 2^16: its position **P**, its
|
||
scale (*s*ₓ, *s*ᵧ, *s*_z) ≥ 0, its quaternion (*w*, *x*, *y*, *z*), its colour and its opacity
|
||
(§4.2).
|
||
|
||
### 5.3 The camera
|
||
|
||
From the pose's `eye` **E**, `target` and `up` **U**:
|
||
|
||
```
|
||
F = target − E n_f = isqrt(F·F); n_f = 0 is pose_degenerate
|
||
f = rdiv(F_i · 2^14, n_f) the forward row at 2^14
|
||
R = F × U n_r = isqrt(R·R); n_r = 0 is pose_degenerate
|
||
r = rdiv(R_i · 2^14, n_r) the right row at 2^14
|
||
u = R_half_up((r × f)_i, 14) the up row at 2^14
|
||
```
|
||
|
||
`·` and `×` are the dot and cross products of the integer vectors. With `F` along −*z* and `U`
|
||
along +*y*, `r` is +*x* and `u` is +*y*: the frame's columns go right and its rows go down.
|
||
|
||
### 5.4 Projecting a Gaussian
|
||
|
||
With `W`, `H` and `f_l` the pose's width, height and focal length, for Gaussian `k`:
|
||
|
||
```
|
||
D = P − E
|
||
x_c = R_half_up(r·D, 14) y_c = R_half_up(u·D, 14) z_c = R_half_up(f·D, 14) at 2^16
|
||
key = D·D at 2^32
|
||
m_x = 128·W + rdiv(256·f_l·x_c, z_c) m_y = 128·H − rdiv(256·f_l·y_c, z_c) at 2^8 pixels
|
||
```
|
||
|
||
The pixel in column *i* and row *j* has its centre at (256·*i* + 128, 256·*j* + 128) at 2^8.
|
||
|
||
The screen covariance follows the elliptical weighted average splatting model, with the camera
|
||
coordinates held within 1.3 times the frame's half-extent as the Jacobian is formed:
|
||
|
||
```
|
||
L_x = rdiv(13·W·z_c, 20·f_l) x_l = clamp(x_c, −L_x, L_x)
|
||
L_y = rdiv(13·H·z_c, 20·f_l) y_l = clamp(y_c, −L_y, L_y)
|
||
j00 = rdiv(f_l · 2^32, z_c) j02 = rdiv((−f_l · x_l) · 2^32, z_c²)
|
||
j11 = −j00 j12 = rdiv((f_l · y_l) · 2^32, z_c²)
|
||
T0_c = R_half_up(j00·r_c + j02·f_c, 14) T1_c = R_half_up(j11·u_c + j12·f_c, 14) c = 0, 1, 2
|
||
```
|
||
|
||
The rotation, at 2^16, from the quaternion with `n = w² + x² + y² + z²`: the identity
|
||
(2^16 on the diagonal) where `n = 0`; otherwise, each entry `rdiv(e · 2^16, n)` of
|
||
|
||
```
|
||
| w²+x²−y²−z² 2(xy − wz) 2(xz + wy) |
|
||
| 2(xy + wz) w²−x²+y²−z² 2(yz − wx) |
|
||
| 2(xz − wy) 2(yz + wx) w²−x²−y²+z² |
|
||
```
|
||
|
||
Then `M_ab = R_half_up(Rot_ab · s_b, 16)`, `S_ab = Σ_c M_ac · M_bc` (at 2^32), and
|
||
|
||
```
|
||
V_ab = Σ_c Σ_d T_a,c · S_cd · T_b,d a, b ∈ {0, 1}
|
||
A = R_half_up(V_00, 48) + 19661 B = R_half_up(V_01, 48) C = R_half_up(V_11, 48) + 19661
|
||
```
|
||
|
||
the screen covariance at 2^16 pixels², with 19661 (0.3 at 2^16) added to each variance as the low
|
||
pass every splat renderer of this model adds.
|
||
|
||
### 5.5 Culling, the rectangle and the order
|
||
|
||
A Gaussian is **culled**, and adds to no pixel, when any of these holds, in this order:
|
||
|
||
1. `z_c < 13107` (0.2 at 2^16: behind the camera or inside its near plane);
|
||
2. `A ≤ 0` or `C ≤ 0`;
|
||
3. with `r_x = ⌈isqrt_ceil(9·A) / 256⌉` and `r_y = ⌈isqrt_ceil(9·C) / 256⌉` (three standard
|
||
deviations, in whole pixels), `r_x > 1023` or `r_y > 1023`;
|
||
4. its **rectangle** is empty: columns `x0 = max(0, ⌈(m_x − 256·r_x − 128) / 256⌉)` to
|
||
`x1 = min(W − 1, ⌊(m_x + 256·r_x − 128) / 256⌋)`, rows `y0`, `y1` the same with `m_y`, `r_y`
|
||
and `H`; empty when `x0 > x1` or `y0 > y1`;
|
||
5. `Δ = A·C − B² ≤ 0`;
|
||
6. with the **whitening coefficients** at 2^16
|
||
|
||
```
|
||
k1 = rdiv(2^40, isqrt(A · 2^32))
|
||
σ = isqrt(A · Δ · 2^16)
|
||
k2 = rdiv(A · 2^32, σ) k3 = rdiv(−B · 2^32, σ)
|
||
```
|
||
|
||
one of `|k1|`, `|k2|`, `|k3|` exceeds 2^31 − 1.
|
||
|
||
For each Gaussian kept, compositing reads its `m_x`, `m_y`, its rectangle, `k1`, `k2`, `k3`, its
|
||
opacity `o = clamp(opacity, 0, 2^16)` and its colour `c_ch = clamp(colour_ch, 0, 2^16)`.
|
||
|
||
The kept Gaussians are put in **order**: by `key`, the squared distance from the eye, nearest
|
||
first; a tie of `key` by the Gaussian's index `k`, lower first. This is the base extension's
|
||
`cameraDistance` order, made total.
|
||
|
||
### 5.6 Compositing a pixel
|
||
|
||
Each pixel starts with transmittance `T = 2^16` and accumulators `acc_r = acc_g = acc_b = 0`, and
|
||
takes the kept Gaussians in order. It skips a Gaussian whose rectangle does not hold it. For one
|
||
that does, at pixel (*i*, *j*):
|
||
|
||
```
|
||
dx = 256·i + 128 − m_x dy = 256·j + 128 − m_y at 2^8
|
||
e1 = k1·dx e2 = k3·dx + k2·dy at 2^24
|
||
```
|
||
|
||
- If `|e1| > 3·2^24` or `|e2| > 3·2^24`, the Gaussian adds nothing here.
|
||
- `a1 = R_half_up(e1, 12)`, `a2 = R_half_up(e2, 12)`, `d2 = R_half_up(a1² + a2², 8)`: the squared
|
||
distance in standard deviations, at 2^16. If `d2 > 9·2^16`, it adds nothing (the base
|
||
extension's 3σ cut-off).
|
||
- `w = exp_neg_q16(−⌊d2 / 2⌋)` and `α = min(64880, ⌊o · w / 2^16⌋)` (64880 is 0.99 at 2^16). If
|
||
`α < 257` (1/255 at 2^16), it adds nothing.
|
||
- `c = R_half_up(α · T, 16)` and `T' = T − c`. If `T' < 7` (0.0001 at 2^16), the pixel is
|
||
**done**: this Gaussian adds nothing and no later one is taken.
|
||
- Otherwise each `acc_ch += R_half_up(c_ch · c, 16)`, and `T = T'`.
|
||
|
||
The pixel's bytes are `min(255, R_half_up(acc_ch · 255, 16))`. The background is black.
|
||
|
||
Taking Gaussians nearest first and keeping the transmittance is the same blend as the base
|
||
extension's back-to-front alpha blending, in exact arithmetic; here every rounding is stated, the
|
||
transmittance falls by exactly what each Gaussian adds, and a pixel can stop once it is opaque.
|
||
|
||
### 5.7 Tiles
|
||
|
||
The frame is cut into tiles of 16 × 16 pixels, in raster order, the last column and row of tiles
|
||
cut short by the frame's edge. A tile's **list** is the kept Gaussians whose rectangle meets the
|
||
tile, in order. A pixel composes its tile's list (§5.6). A Gaussian whose rectangle holds a pixel
|
||
is in that pixel's tile's list, so the frame is the one §5.6 gives over the whole order, whatever
|
||
order the tiles are computed in and however the work is divided among them: tiles and pixels are
|
||
independent, and only the order within one pixel's list matters.
|
||
|
||
### 5.8 The frame's digest
|
||
|
||
A frame's **digest** is the SHA-256 (FIPS 180-4) of its bytes.
|
||
|
||
### 5.9 Widths
|
||
|
||
Within the caps of §2 and §5.2, every value §5.3–§5.5 forms fits in 127 bits plus a sign, and
|
||
§5.5's rule 3 bounds `A` and `C` before `Δ` and `σ` are formed. Every value compositing reads
|
||
fits in 32 bits, and §5.6 forms every value in 32 bits except the three products and the sum that
|
||
make `e1` and `e2`, which fit in 64: `|k| ≤ 2^31 − 1` and `|dx|`, `|dy| ≤ 256·1024 + 256`, and
|
||
each `e` is narrowed only after it is checked against `3·2^24`. `α · T` and `c_ch · c` are below
|
||
2^32 because `α ≤ 64880` and `T ≤ 2^16`.
|
||
|
||
The renderer is not a graph of [integer-operators](integer-operators.md): its order and its
|
||
rectangles are not graph operators. It uses that extension's integer functions (§2 there), so
|
||
`exp_neg_q16` is the one the operator set's `softmax`, `sigmoid` and `elu` use.
|
||
|
||
## 6. Canonical poses and the commitment
|
||
|
||
### 6.1 The commitment
|
||
|
||
The origin of a splat set signs a commitment, `kind` `"wai-splat-commitment"`:
|
||
|
||
| member | type | meaning |
|
||
|---|---|---|
|
||
| `kind` | string | `"wai-splat-commitment"` |
|
||
| `object` | 32 B hex | the content hash of the object the asset carries (§3.2) |
|
||
| `set` | 32 B hex | the SHA-256 of its parameter set |
|
||
| `attributes` | 32 B hex | its attribute digest (§3.3) |
|
||
| `renderer` | term | the renderer: `wai-splat-render:i1` in this revision |
|
||
| `class` | string | the splat set's class under [synthesis](synthesis.md) §3.2: `captured`, `reconstructed`, `assistive` or `synthesized` |
|
||
| `origin` | 32 B hex or `null` | the link hash of the splat set's origin claim (synthesis §3.1), where it has one |
|
||
| `views` | array | 1 to 64 entries, each `{"pose", "frame"}`: a **canonical pose** (§2) and the digest of the frame §5 gives at it; no pose twice |
|
||
| `issued_at`, `signer_pubkey`, `sig` | | §2 |
|
||
|
||
A commitment that breaks a rule of this table is `record_malformed`.
|
||
|
||
### 6.2 What it binds
|
||
|
||
A commitment names one object by content hash, so its decoded attributes, by digest, and through
|
||
them every frame of every pose: the canonical poses are the ones whose frames the origin states.
|
||
A viewer that recomputes a canonical frame and finds its digest knows its own decode and renderer
|
||
reproduce what the origin saw from there, bit for bit; a viewer at any other pose renders a frame
|
||
that every other conforming viewer at that pose renders too. The origin chooses the poses: views
|
||
it wants recognised (a product shot, a catalogue's angles, the poses a scene was captured from).
|
||
|
||
## 7. The view statement
|
||
|
||
A sink signs a view statement, `kind` `"wai-splat-view"`, for a frame it presented, under a key
|
||
of its viewing session (no identity of the viewer or the device):
|
||
|
||
| member | type | meaning |
|
||
|---|---|---|
|
||
| `kind` | string | `"wai-splat-view"` |
|
||
| `session` | 32 B hex | fresh random bytes of the session |
|
||
| `commitment` | 32 B hex | the link hash of the asset's commitment |
|
||
| `object` | 32 B hex | the content hash of the object it rendered |
|
||
| `pose` | object | the pose (§2) |
|
||
| `canonical` | integer or `null` | the index in the commitment's `views` of this pose, where the sink presents it as canonical; `null` otherwise |
|
||
| `frame` | 32 B hex | the digest of the frame it presented |
|
||
| `renderer` | term | the renderer: `wai-splat-render:i1` |
|
||
| `class` | string | the frame's class (§8) |
|
||
| `at` | integer | when it was presented, in microseconds of the session's clock |
|
||
| `issued_at`, `signer_pubkey`, `sig` | | §2 |
|
||
|
||
A statement that breaks a rule of this table is `record_malformed`. A statement whose `canonical`
|
||
is an index states that its pose is that canonical pose, and its frame the one the commitment
|
||
binds there; with `null` it states only that its frame is §5's at its pose, which anyone holding
|
||
the asset recomputes.
|
||
|
||
## 8. Provenance
|
||
|
||
A rendered view is a **conversion** of the splat set under [synthesis](synthesis.md) §3.2: it
|
||
rearranges and composes what the splat set holds, generates nothing and enhances nothing, and
|
||
the renderer names no model. Its class is therefore the greatest of `reconstructed` and the splat
|
||
set's class: the view of a captured or reconstructed scene is `reconstructed`, of an assistive one
|
||
`assistive`, of a synthesized one `synthesized`. A view statement's `class` is that class
|
||
(`class_mismatch` otherwise).
|
||
|
||
A splat set's own origin claim (synthesis §3.1) names `media` `volumetric`, and its subject is the
|
||
object the asset carries, by content hash. A sink that publishes a rendered view as an object
|
||
signs its origin claim as synthesis §3.4 has a conversion's: `step` `convert`, `media` `image`,
|
||
`operation` `wai-splat-render:i1`, `record` the link hash of its view statement, and one input,
|
||
the splat set's object with its claim's link hash, class and source.
|
||
|
||
A view statement is a presentation statement for one frame: what a sink presented, from which
|
||
pose, of which committed object, with what class. Its frame is recomputable from the asset, so a
|
||
verifier holding the asset proves it and does not take it on trust.
|
||
|
||
## 9. Verification
|
||
|
||
### 9.1 Inputs
|
||
|
||
A verifier is given an asset (its JSON and the bytes of the buffers it names), the parameter sets
|
||
it holds, the splat set's origin claim where it holds one, and view statements.
|
||
|
||
### 9.2 Steps
|
||
|
||
In this order, a case stopping at its first refusal; the place of each refusal is named after it:
|
||
|
||
1. The asset (§3.1, §3.2): `asset_malformed`, `capability_mismatch` — *asset*.
|
||
2. The envelope, its set and the decode (§3.3): `asset_malformed`, `capability_mismatch`,
|
||
`object_mismatch`, `set_mismatch`, `set_not_held`, `decode_refused` — *asset*.
|
||
3. The accessors (§3.4): `asset_malformed` — *asset*. An asset with no commitment is accepted
|
||
here.
|
||
4. The commitment (§6.1), read and its signature verified; its `object` is the extension's
|
||
(`object_mismatch`), its `set` the extension's `set.sha256` (`set_mismatch`), its `attributes`
|
||
the attribute digest (`attributes_mismatch`), its `renderer` `wai-splat-render:i1`
|
||
(`renderer_unknown`); the splat set is renderable (§5.2: `layout_mismatch`,
|
||
`range_unrenderable`) — *commitment*.
|
||
5. Each view in order: the frame at its pose (`pose_degenerate`), whose digest is its `frame`
|
||
(`frame_mismatch`) — *view n*, from 0.
|
||
6. Where the commitment's `origin` is not `null` and the verifier holds the claim: the claim reads
|
||
and verifies as synthesis §2 has it, its link hash is `origin`, its subject is the object, its
|
||
class is the commitment's `class` and the greatest of its step's class and its inputs' classes
|
||
(synthesis §3.2); otherwise `origin_mismatch` — *origin*.
|
||
7. Each view statement in order, read and its signature verified; its `commitment` is the
|
||
commitment's link hash (`commitment_mismatch`), its `object` the object (`object_mismatch`), its
|
||
`renderer` `wai-splat-render:i1` (`renderer_unknown`); with an index in `canonical`, that index
|
||
names a view whose pose is the statement's (`pose_not_canonical`) and whose frame is the
|
||
statement's (`frame_mismatch`); with `null`, the frame at its pose (`pose_degenerate`) has the
|
||
statement's digest (`frame_mismatch`); its `class` is the view class of §8 (`class_mismatch`) —
|
||
*statement n*, from 0.
|
||
|
||
### 9.3 What an accepted case states
|
||
|
||
The object's content hash, the attribute digest and the Gaussian count; the digest of each
|
||
canonical frame, in the commitment's order; and, for each statement, its `canonical` and its
|
||
`class`.
|
||
|
||
## 10. Reasons
|
||
|
||
| code | meaning |
|
||
|---|---|
|
||
| `record_malformed` | a record or a pose breaks a rule of its table (§2, §6.1, §7) |
|
||
| `bad_key` | a record's key fails step 1 or 2 of SPEC §7's strict verification |
|
||
| `bad_signature` | a record's signature does not verify |
|
||
| `asset_malformed` | the asset breaks a rule of §3.1, §3.2 or §3.4, or its buffer view does not hold one envelope |
|
||
| `capability_mismatch` | the extension or the envelope names a capability other than `wai.splat.int_codec` |
|
||
| `object_mismatch` | a content hash the extension, the commitment or a statement states is not the envelope's |
|
||
| `set_mismatch` | the pin, the extension and the set disagree, or the pin does not verify |
|
||
| `set_not_held` | the set is named by digest only, and the verifier holds no file of that digest |
|
||
| `decode_refused` | the payload's decode refuses it |
|
||
| `attributes_mismatch` | the commitment's attribute digest is not the decode's: a commitment over other splats |
|
||
| `renderer_unknown` | a record names a renderer this revision does not pin |
|
||
| `layout_mismatch` | the splat set does not have the 14 planes of §4.1 |
|
||
| `range_unrenderable` | a range endpoint is past 2^28 at 2^16, or a scale range goes below zero |
|
||
| `pose_degenerate` | a pose whose eye is its target, or whose up direction is along its line of sight |
|
||
| `frame_mismatch` | a frame's digest is not the digest of the frame §5 gives at its pose |
|
||
| `origin_mismatch` | the origin claim the commitment names does not match it, or is not that claim |
|
||
| `commitment_mismatch` | a statement names another commitment than the asset's |
|
||
| `pose_not_canonical` | a statement claims a pose canonical that is not the commitment's pose at that index |
|
||
| `class_mismatch` | a statement's class is not the class of a view of the splat set (§8) |
|
||
|
||
## 11. Conformance
|
||
|
||
The corpus [`live-conformance/splat/`](../live-conformance/splat/) holds glTF assets, their buffers
|
||
and commitments, origin claims and view statements, each case with the outcome a verifier
|
||
reaches. Its scenes: the learned cloud of `wai-rs/tests/vectors/int_splat/` (its payload and its
|
||
`WHM1` synthesis, held by digest), and two scenes authored as attribute bytes and carried through
|
||
the **identity set**, a one-layer `WHM1` network that decodes each latent value `v` at 2^8 to the
|
||
byte `(255·v + 128) >> 8`, so any planes can be coded. The identity set is a conformance device: it
|
||
codes each plane at the entropy of its bytes and learns nothing.
|
||
|
||
A verifier conforms when it reaches every case's stated outcome, place and statements. The
|
||
corpus refuses a case with every code of §10, and holds these negative controls among them: a
|
||
reordered sort (frames committed with a tie of distance broken by the greater index), a frame that
|
||
does not recompute (rendered without §5.4's low pass), a commitment over the wrong splats (another
|
||
scene's attribute digest), and a pose outside the pinned set claimed canonical.
|
||
|
||
Two implementations reach the same outcomes, sharing no code: the `splat_exact` module of `wai-rs`,
|
||
which also writes the corpus (`wai_splat_exact_vectors gen`, and `verify` to regenerate it byte for
|
||
byte), and `live-conformance/splat_exact_verify.py`, standard library only, written from this
|
||
text with its own integer renderer. The second also drops each of its own checks in turn and
|
||
changes each of its renderer's rules in turn (the tie order, sorting by depth instead of distance,
|
||
the low pass, the rounding of `rdiv` and of a contribution, `exp_neg_q16`, the stop, the alpha
|
||
ceiling, the cut-off, the rectangle's rounding, the near plane), and requires the corpus to catch
|
||
every one.
|
||
|
||
**On a GPU.** Compositing reads 32-bit words and forms 32-bit values but for `e1` and `e2`
|
||
(§5.9), so it runs as a compute kernel in which every pixel is an invocation and every tile a
|
||
workgroup, with `e1` and `e2` formed by the 64-bit products of the integer operator set's GPU
|
||
executor (the `wai-ops-gpu` crate), whose arithmetic every device passes a probe on before it runs
|
||
anything. The host projects, culls and orders (§5.3–§5.5). `splat_exact::gpu` in `wai-rs` (with the
|
||
features `splat_exact` and `ops_gpu`) composes that way, and its test holds every canonical and
|
||
own-pose view of every scene of the corpus to the CPU's frame, byte for byte, on every adapter it
|
||
finds, and requires the corpus's views to catch each of three edited kernels (a contribution
|
||
rounded down, no alpha ceiling, the cut-off moved). Measured on 22 October 2026 with Metal, on the
|
||
integrated GPU of an arm64 desktop (macOS 27.0.1): every view equal, every edited kernel caught.
|
||
The `macos` job of `wai-ops-gpu.yml` runs it, and an absent adapter fails there.
|
||
|
||
## 12. The quality cost
|
||
|
||
`wai_splat_exact_measure` renders each scene of the corpus at its canonical poses and at the pose
|
||
of its own-pose view, with `wai-splat-render:i1` and with a floating-point renderer of the same
|
||
model (the same camera, low pass, cut-off, alpha limits, stop and order, the platform's `exp` and
|
||
`sqrt`) in `f64` and in `f32`. Measured on one machine (the corpus's frames, and the same poses at
|
||
four times the side and focal length):
|
||
|
||
| scene | frames | PSNR of the integer frame against `f64` | largest byte difference | bytes in which `f32` and `f64` differ |
|
||
|---|---|---|---|---|
|
||
| learned cloud | 96 × 96, 80 × 64 | 62.3–64.5 dB | 2 | 0–2 per frame |
|
||
| learned cloud | 384 × 384, 320 × 256 | 62.3–64.8 dB | 4 | 7–24 per frame |
|
||
| authored cluster | 80 × 60 | 59.4–61.5 dB | 2 | 0–2 per frame |
|
||
| authored cluster | 320 × 240 | 59.4–61.7 dB | 3 | 3–17 per frame |
|
||
| authored three Gaussians | 40 × 30 | 63.2–67.2 dB | 1 | 0 |
|
||
| authored three Gaussians | 160 × 120 | 64.5–67.0 dB | 3 | 0–1 per frame |
|
||
|
||
So the cost of the integer arithmetic, against the same model in `f64`, is a byte that differs by
|
||
at most 4 levels, at about 60 dB and above; it includes §4.2's fixed-point dequantization, which a
|
||
float renderer does in float. A float renderer is not one renderer: in `f32` it gives other bytes
|
||
than in `f64` in some of the corpus's frames and in nearly all of them at four times the side, and so do two devices that order or fuse its operations differently,
|
||
so no digest of a float frame is stable. These figures are this corpus's scenes; a scene of many
|
||
more Gaussians composes more terms per pixel and adds more rounding.
|
||
|
||
## 13. Security and privacy considerations
|
||
|
||
A commitment says what the origin's splat set looks like from each canonical pose, under this
|
||
renderer; it says nothing about the scene in the world it was made from. Its class and origin
|
||
claim say where the splat set came from (§8).
|
||
|
||
A view statement reveals a pose and a time: where a viewer looked, and when. It is signed under a
|
||
session key and names no viewer or device. A sink **SHOULD** use a fresh session key and fresh
|
||
`session` bytes for each viewing session, and **SHOULD NOT** sign statements for poses a viewer
|
||
did not choose to share.
|
||
|
||
The decode's caps are integer-payloads §7's; the renderer's work is bounded by the frame's size
|
||
(at most 2^20 pixels) times the Gaussians whose rectangles hold a pixel, and §5.5 culls a
|
||
Gaussian whose rectangle is wider than 2047 pixels before its pixels are visited. A verifier
|
||
**MAY** refuse to render a commitment whose frames, added up, exceed its own budget, and says so;
|
||
the corpus's are small.
|