open-standards/wai/research/spatial-audio-draft.md

# WAI Extension: Object-Based Spatial Audio (DRAFT)

> Status: **PROMOTED 2026-06-13** — canonical copy at
> `extensions/spatial-audio.md` (site mirror `src/pages/extensions/`).
> This is the working draft it came from; edit the canonical copy.
> Reference impl: `wai-rs` `audio_scene`; corpus: `audio-conformance/`;
> live sink: wai.transaction.science/audio.

Keywords **MUST**, **MUST NOT**, **SHOULD**, **MAY** are RFC 2119/8174.

## 1. Scope and model

A `wai.audio.scene` object describes a set of **audio objects** — each a
content-addressed source plus a position trajectory and gain — that a
sink renders to *its own* speaker layout. Today's audio ships a fixed
channel mix (stereo, 5.1); this ships the scene and lets the sink make
the mix it can actually play. It is lever 3 (instructions-at-the-sink)
for hearing.

In scope: the scene container, the deterministic mixdown render
contract, the **mixdown-equivalence** conformance criterion, the
fallback floor, and the receipt profile.

Out of scope: a transport; the source codecs (sources are ordinary
`wai.audio.*` WAI objects, decoded by the existing path); HRTF design
(an HRTF is named as a requirement-of-existence, never shipped).

Relationship to the core spec: this mirrors interactive-worlds exactly.
A scene is a content-addressed artifact; sources are content-addressed
WAI assets (§4.1 discipline, domain `wai:asset\x01`); the render is
deterministic under a declared numeric profile; receipts are a JWP
group profile. `media` takes the informational value `"audio-scene"`;
the capability `wai.audio.scene` is authoritative.

## 2. Container (`WSCN`)

```
"WSCN" | u16 n_sections | section table | sections
        | n × (u8 kind, u32 off, u32 len), 8-byte-aligned bodies
```

| kind | section | required |
|---|---|---|
| `0x01` | scene contract (canonical JSON) | REQUIRED |
| `0x02` | object table | REQUIRED |
| `0x03` | asset table (§4.1, the source `wai.audio.*` objects) | REQUIRED |
| `0x04` | seal: duration + final mixdown hash + Ed25519 sig | OPTIONAL (a sealed scene pins its reference mix) |

Unknown section kinds MUST be ignored (forward compat within MINOR).

### 2.1 Scene contract (canonical JSON)

```json
{
  "sample_rate": 48000,
  "numeric":     "wai.det.fixed64",
  "pan_law":     "wai.audio.pan.constpower",
  "distance":    "wai.audio.dist.inverse_clamped",
  "duration":    240000,
  "floor_layout":"stereo"
}
```

`numeric` is the same Q32.32 fixed-point floor as worlds — the mixdown
is bit-exact on every machine. `floor_layout` (`stereo`) is the
mandatory render target; richer layouts (`5.1`, `binaural@<hrtf-id>`)
are OPTIONAL and, for binaural, perceptual (the HRTF is a
requirement-of-existence, not bit-exact across HRTFs).

### 2.2 Object table

`n_objects u32`, then per object:

```
source_hash (32 B)        // content hash of a wai.audio.* WAI envelope (asset table)
gain        (i64 Fx)
n_keys      (u16)
keys: n × ( pos_sample u64 | x i64 | y i64 | z i64 )   // listener-relative position, Fx metres
```

Position between keyframes is linear-interpolated in `Fx` (so it is
reproducible). One keyframe ⇒ a static source.

## 3. Render contract (mixdown)

For output channel `c` and sample `t`, in `Fx`:

```
out[c][t] = Σ_objects  src_o[t] · gain_o · dist_atten(|p_o(t)|) · pan_gain(p_o(t), c)
```

- `src_o[t]`: object `o`'s source decoded to mono at `sample_rate`
  (resample deterministically if needed — the resampler is part of the
  registration's normative pseudocode).
- `dist_atten(d) = 1 / (1 + d)` clamped to `[0,1]` (`inverse_clamped`).
- `pan_gain` for `constpower` stereo: from the azimuth θ of `p_o(t)`,
  left = `cos(π/4·(1+sin θ))`, right = `sin(...)` — a documented
  constant-power law, evaluated through the `Fx` transcendental tables
  (the same tables worlds' numeric profile publishes).
- Accumulate in `Fx`, then quantize to the canonical output: interleaved
  **i16 LE** PCM.

## 4. Conformance — mixdown-equivalence

> Given the same scene and the same **declared output layout**, a
> conforming sink MUST produce the identical canonical PCM, hence the
> identical `BLAKE3` over it — on every machine, with no tolerance
> parameter.

```
mixdown_hash = BLAKE3("wai:audio-mixdown\x01" || layout_id || pcm_i16le)
```

The `stereo` floor is mandatory and bit-exact. `binaural@<hrtf>` is
perceptual (conformance is similarity, not bytes — a sink advertising
it states the metric, like the `wai.semantic.*` path). The conformance
corpus ships scenes + the expected `stereo` mixdown hash; pass = hash
equality.

## 5. Fallback floor

`model_requirement.fallback` SHOULD name `wai.audio.opus` (or `flac`),
and the encoder SHOULD ship the `WAI2` multi-rendition form: the scene
plus a pre-rendered stereo recording. A sink without the scene runtime
plays the recording. Scene-capable sinks render to their real layout;
below-floor sinks hear a fixed mix — both opened one envelope.

## 6. Session receipts (JWP profile)

A render session receipts as a JWP group (jwp-receipts.md):
- `world_hash` analog = `scene_hash` = `BLAKE3("wai:audio-scene\x01" ||
  contract_bytes || object_table_bytes)` — scene identity.
- objects in the group = output PCM blocks (one per second), content-
  hashed; the group's checkpoint list is the per-block `mixdown_hash`.
- `work_steps` = output samples × objects mixed (exact, signed);
  `joules_micro` = the native meter's measured CPU energy for the mix
  (the `wai_world_meter` pattern, applied to the mixer).

## Appendix — why this is the first cargo class

Mixdown-equivalence is replay-equivalence with the tick replaced by the
audio sample; the `Fx` floor, the content-addressed assets, the seal,
the receipt profile, and the fallback floor are all reused verbatim. A
spatial-audio scene is, structurally, a world whose "sim" is a mixer,
so the interactive-worlds rules carry over to a second sense.