open-standards/wai/research/spatial-audio-draft.md

WAI Extension: Object-Based Spatial Audio (DRAFT)

Status: PROMOTED 2026-06-13 — canonical copy at extensions/spatial-audio.md (site mirror src/pages/extensions/). This is the working draft it came from; edit the canonical copy. Reference impl: wai-rs audio_scene; corpus: audio-conformance/; live sink: wai.transaction.science/audio.

Keywords MUST, MUST NOT, SHOULD, MAY are RFC 2119/8174.

1. Scope and model

A wai.audio.scene object describes a set of audio objects — each a content-addressed source plus a position trajectory and gain — that a sink renders to its own speaker layout. Today's audio ships a fixed channel mix (stereo, 5.1); this ships the scene and lets the sink make the mix it can actually play. It is lever 3 (instructions-at-the-sink) for hearing.

In scope: the scene container, the deterministic mixdown render contract, the mixdown-equivalence conformance criterion, the fallback floor, and the receipt profile.

Out of scope: a transport; the source codecs (sources are ordinary wai.audio.* WAI objects, decoded by the existing path); HRTF design (an HRTF is named as a requirement-of-existence, never shipped).

Relationship to the core spec: this mirrors interactive-worlds exactly. A scene is a content-addressed artifact; sources are content-addressed WAI assets (§4.1 discipline, domain wai:asset\x01); the render is deterministic under a declared numeric profile; receipts are a JWP group profile. media takes the informational value "audio-scene"; the capability wai.audio.scene is authoritative.

2. Container (WSCN)

"WSCN" | u16 n_sections | section table | sections
        | n × (u8 kind, u32 off, u32 len), 8-byte-aligned bodies

kind	section	required
0x01	scene contract (canonical JSON)	REQUIRED
0x02	object table	REQUIRED
0x03	asset table (§4.1, the source wai.audio.* objects)	REQUIRED
0x04	seal: duration + final mixdown hash + Ed25519 sig	OPTIONAL (a sealed scene pins its reference mix)

Unknown section kinds MUST be ignored (forward compat within MINOR).

2.1 Scene contract (canonical JSON)

{
  "sample_rate": 48000,
  "numeric":     "wai.det.fixed64",
  "pan_law":     "wai.audio.pan.constpower",
  "distance":    "wai.audio.dist.inverse_clamped",
  "duration":    240000,
  "floor_layout":"stereo"
}

numeric is the same Q32.32 fixed-point floor as worlds — the mixdown is bit-exact on every machine. floor_layout (stereo) is the mandatory render target; richer layouts (5.1, binaural@<hrtf-id>) are OPTIONAL and, for binaural, perceptual (the HRTF is a requirement-of-existence, not bit-exact across HRTFs).

2.2 Object table

n_objects u32, then per object:

source_hash (32 B)        // content hash of a wai.audio.* WAI envelope (asset table)
gain        (i64 Fx)
n_keys      (u16)
keys: n × ( pos_sample u64 | x i64 | y i64 | z i64 )   // listener-relative position, Fx metres

Position between keyframes is linear-interpolated in Fx (so it is reproducible). One keyframe ⇒ a static source.

3. Render contract (mixdown)

For output channel c and sample t, in Fx:

out[c][t] = Σ_objects  src_o[t] · gain_o · dist_atten(|p_o(t)|) · pan_gain(p_o(t), c)

• src_o[t]: object o's source decoded to mono at sample_rate (resample deterministically if needed — the resampler is part of the registration's normative pseudocode).
• dist_atten(d) = 1 / (1 + d) clamped to [0,1] (inverse_clamped).
• pan_gain for constpower stereo: from the azimuth θ of p_o(t), left = cos(π/4·(1+sin θ)), right = sin(...) — a documented constant-power law, evaluated through the Fx transcendental tables (the same tables worlds' numeric profile publishes).
• Accumulate in Fx, then quantize to the canonical output: interleaved i16 LE PCM.

4. Conformance — mixdown-equivalence

Given the same scene and the same declared output layout, a conforming sink MUST produce the identical canonical PCM, hence the identical BLAKE3 over it — on every machine, with no tolerance parameter.

mixdown_hash = BLAKE3("wai:audio-mixdown\x01" || layout_id || pcm_i16le)

The stereo floor is mandatory and bit-exact. binaural@<hrtf> is perceptual (conformance is similarity, not bytes — a sink advertising it states the metric, like the wai.semantic.* path). The conformance corpus ships scenes + the expected stereo mixdown hash; pass = hash equality.

5. Fallback floor

model_requirement.fallback SHOULD name wai.audio.opus (or flac), and the encoder SHOULD ship the WAI2 multi-rendition form: the scene plus a pre-rendered stereo recording. A sink without the scene runtime plays the recording. Scene-capable sinks render to their real layout; below-floor sinks hear a fixed mix — both opened one envelope.

6. Session receipts (JWP profile)

A render session receipts as a JWP group (jwp-receipts.md):

• world_hash analog = scene_hash = BLAKE3("wai:audio-scene\x01" || contract_bytes || object_table_bytes) — scene identity.
• objects in the group = output PCM blocks (one per second), content- hashed; the group's checkpoint list is the per-block mixdown_hash.
• work_steps = output samples × objects mixed (exact, signed); joules_micro = the native meter's measured CPU energy for the mix (the wai_world_meter pattern, applied to the mixer).

Appendix — why this is the cleanest first cargo class

Mixdown-equivalence is replay-equivalence with the tick replaced by the audio sample; the Fx floor, the content-addressed assets, the seal, the receipt profile, and the fallback floor are all reused verbatim. A spatial-audio scene is, structurally, a world whose "sim" is a mixer — which is exactly why it proves the interactive-worlds thesis generalizes to a second sense.