46c5688b86
Commit the WAI standard source: a content + capability-dispatch standard
whose thesis is "carries whatever it can reconstruct, verifies whatever
it carries." Source only — large vendored/model assets and build output
are gitignored.
Reconstruct classes (each: spec + conformance corpus + in-tab browser
sink that re-derives the corpus hash, one wai.det.fixed64 fixed-point
floor, one *-equivalence apiece):
- interactive worlds (replay-equivalence) + 3 sim registrants,
multiplayer over a relay, session receipts, glTF + real SPZ assets
- spatial audio (mixdown-equivalence)
- haptics (sample-equivalence)
- state-feeds (replay-equivalence, non-physics sim)
- avatars / digital humans (pose-equivalence) + telepresence over the relay
- volumetric / 4D splats (frame-state-equivalence) + WebGPU EWA render,
real-SPZ-derived clip
Verify half (lever 4):
- asset provenance: a signed, energy-accounted lineage for opaque
captured content WAI does not reconstruct; tamper-evident, in-tab.
Cross-cutting: JWP receipt profiles per class; a native energy meter
(IOReport via macmon, no sudo) emitting receipts with REAL measured
joules; the learned-image/neural codec family and JPEG AI entropy port
from earlier work.
Co-Authored-By: Claude Opus 4.8 <[email protected]>
146 lines
5.8 KiB
Markdown
146 lines
5.8 KiB
Markdown
# WAI Extension: Object-Based Spatial Audio (DRAFT)
|
||
|
||
> Status: **PROMOTED 2026-06-13** — canonical copy at
|
||
> `extensions/spatial-audio.md` (site mirror `src/pages/extensions/`).
|
||
> This is the working draft it came from; edit the canonical copy.
|
||
> Reference impl: `wai-rs` `audio_scene`; corpus: `audio-conformance/`;
|
||
> live sink: wai.transaction.science/audio.
|
||
|
||
Keywords **MUST**, **MUST NOT**, **SHOULD**, **MAY** are RFC 2119/8174.
|
||
|
||
## 1. Scope and model
|
||
|
||
A `wai.audio.scene` object describes a set of **audio objects** — each a
|
||
content-addressed source plus a position trajectory and gain — that a
|
||
sink renders to *its own* speaker layout. Today's audio ships a fixed
|
||
channel mix (stereo, 5.1); this ships the scene and lets the sink make
|
||
the mix it can actually play. It is lever 3 (instructions-at-the-sink)
|
||
for hearing.
|
||
|
||
In scope: the scene container, the deterministic mixdown render
|
||
contract, the **mixdown-equivalence** conformance criterion, the
|
||
fallback floor, and the receipt profile.
|
||
|
||
Out of scope: a transport; the source codecs (sources are ordinary
|
||
`wai.audio.*` WAI objects, decoded by the existing path); HRTF design
|
||
(an HRTF is named as a requirement-of-existence, never shipped).
|
||
|
||
Relationship to the core spec: this mirrors interactive-worlds exactly.
|
||
A scene is a content-addressed artifact; sources are content-addressed
|
||
WAI assets (§4.1 discipline, domain `wai:asset\x01`); the render is
|
||
deterministic under a declared numeric profile; receipts are a JWP
|
||
group profile. `media` takes the informational value `"audio-scene"`;
|
||
the capability `wai.audio.scene` is authoritative.
|
||
|
||
## 2. Container (`WSCN`)
|
||
|
||
```
|
||
"WSCN" | u16 n_sections | section table | sections
|
||
| n × (u8 kind, u32 off, u32 len), 8-byte-aligned bodies
|
||
```
|
||
|
||
| kind | section | required |
|
||
|---|---|---|
|
||
| `0x01` | scene contract (canonical JSON) | REQUIRED |
|
||
| `0x02` | object table | REQUIRED |
|
||
| `0x03` | asset table (§4.1, the source `wai.audio.*` objects) | REQUIRED |
|
||
| `0x04` | seal: duration + final mixdown hash + Ed25519 sig | OPTIONAL (a sealed scene pins its reference mix) |
|
||
|
||
Unknown section kinds MUST be ignored (forward compat within MINOR).
|
||
|
||
### 2.1 Scene contract (canonical JSON)
|
||
|
||
```json
|
||
{
|
||
"sample_rate": 48000,
|
||
"numeric": "wai.det.fixed64",
|
||
"pan_law": "wai.audio.pan.constpower",
|
||
"distance": "wai.audio.dist.inverse_clamped",
|
||
"duration": 240000,
|
||
"floor_layout":"stereo"
|
||
}
|
||
```
|
||
|
||
`numeric` is the same Q32.32 fixed-point floor as worlds — the mixdown
|
||
is bit-exact on every machine. `floor_layout` (`stereo`) is the
|
||
mandatory render target; richer layouts (`5.1`, `binaural@<hrtf-id>`)
|
||
are OPTIONAL and, for binaural, perceptual (the HRTF is a
|
||
requirement-of-existence, not bit-exact across HRTFs).
|
||
|
||
### 2.2 Object table
|
||
|
||
`n_objects u32`, then per object:
|
||
|
||
```
|
||
source_hash (32 B) // content hash of a wai.audio.* WAI envelope (asset table)
|
||
gain (i64 Fx)
|
||
n_keys (u16)
|
||
keys: n × ( pos_sample u64 | x i64 | y i64 | z i64 ) // listener-relative position, Fx metres
|
||
```
|
||
|
||
Position between keyframes is linear-interpolated in `Fx` (so it is
|
||
reproducible). One keyframe ⇒ a static source.
|
||
|
||
## 3. Render contract (mixdown)
|
||
|
||
For output channel `c` and sample `t`, in `Fx`:
|
||
|
||
```
|
||
out[c][t] = Σ_objects src_o[t] · gain_o · dist_atten(|p_o(t)|) · pan_gain(p_o(t), c)
|
||
```
|
||
|
||
- `src_o[t]`: object `o`'s source decoded to mono at `sample_rate`
|
||
(resample deterministically if needed — the resampler is part of the
|
||
registration's normative pseudocode).
|
||
- `dist_atten(d) = 1 / (1 + d)` clamped to `[0,1]` (`inverse_clamped`).
|
||
- `pan_gain` for `constpower` stereo: from the azimuth θ of `p_o(t)`,
|
||
left = `cos(π/4·(1+sin θ))`, right = `sin(...)` — a documented
|
||
constant-power law, evaluated through the `Fx` transcendental tables
|
||
(the same tables worlds' numeric profile publishes).
|
||
- Accumulate in `Fx`, then quantize to the canonical output: interleaved
|
||
**i16 LE** PCM.
|
||
|
||
## 4. Conformance — mixdown-equivalence
|
||
|
||
> Given the same scene and the same **declared output layout**, a
|
||
> conforming sink MUST produce the identical canonical PCM, hence the
|
||
> identical `BLAKE3` over it — on every machine, with no tolerance
|
||
> parameter.
|
||
|
||
```
|
||
mixdown_hash = BLAKE3("wai:audio-mixdown\x01" || layout_id || pcm_i16le)
|
||
```
|
||
|
||
The `stereo` floor is mandatory and bit-exact. `binaural@<hrtf>` is
|
||
perceptual (conformance is similarity, not bytes — a sink advertising
|
||
it states the metric, like the `wai.semantic.*` path). The conformance
|
||
corpus ships scenes + the expected `stereo` mixdown hash; pass = hash
|
||
equality.
|
||
|
||
## 5. Fallback floor
|
||
|
||
`model_requirement.fallback` SHOULD name `wai.audio.opus` (or `flac`),
|
||
and the encoder SHOULD ship the `WAI2` multi-rendition form: the scene
|
||
plus a pre-rendered stereo recording. A sink without the scene runtime
|
||
plays the recording. Scene-capable sinks render to their real layout;
|
||
below-floor sinks hear a fixed mix — both opened one envelope.
|
||
|
||
## 6. Session receipts (JWP profile)
|
||
|
||
A render session receipts as a JWP group (jwp-receipts.md):
|
||
- `world_hash` analog = `scene_hash` = `BLAKE3("wai:audio-scene\x01" ||
|
||
contract_bytes || object_table_bytes)` — scene identity.
|
||
- objects in the group = output PCM blocks (one per second), content-
|
||
hashed; the group's checkpoint list is the per-block `mixdown_hash`.
|
||
- `work_steps` = output samples × objects mixed (exact, signed);
|
||
`joules_micro` = the native meter's measured CPU energy for the mix
|
||
(the `wai_world_meter` pattern, applied to the mixer).
|
||
|
||
## Appendix — why this is the cleanest first cargo class
|
||
|
||
Mixdown-equivalence is replay-equivalence with the tick replaced by the
|
||
audio sample; the `Fx` floor, the content-addressed assets, the seal,
|
||
the receipt profile, and the fallback floor are all reused verbatim. A
|
||
spatial-audio scene is, structurally, a world whose "sim" is a mixer —
|
||
which is exactly why it proves the interactive-worlds thesis
|
||
generalizes to a second sense.
|