open-standards/wai/wai-web/src/audio.rs
Transaction Science df20c392d8 wai: interactive + temporal cargo classes, provenance, telepresence
Adds the "carries whatever it can reconstruct, verifies whatever it
carries" surface on top of the existing neural/codec WAI. All builds
green against this base (wai-rs all-features tests + clippy, wai-web
wasm, wai-spz-gen, wai-relay).

Reconstruct half — each a spec + conformance corpus + in-tab browser
sink that re-derives the corpus hash, over one wai.det.fixed64 floor:
  - interactive worlds      replay-equivalence   (3 sim registrants,
    relay multiplayer, session receipts, glTF + real-SPZ assets)
  - spatial audio           mixdown-equivalence
  - haptics                 sample-equivalence
  - state-feeds             replay-equivalence  (non-physics sim)
  - avatars / digital humans  pose-equivalence  (+ telepresence)
  - volumetric / 4D splats  frame-state-equivalence (WebGPU EWA render)

Verify half (lever 4):
  - asset provenance — signed, energy-accounted lineage for opaque
    captured content WAI does not reconstruct; tamper-evident in-tab.

Cross-cutting: per-class JWP receipt profiles; a native energy meter
(IOReport via macmon, no sudo) emitting receipts with real measured
joules. Codecs gated behind a `codecs` feature (default on) so the
world/wasm sinks build without the C codec chain.

Excludes the separately-evolving neural subsystem (semantic path, GLC
rANS, ONNX tooling) to avoid reverting newer origin work.

Co-Authored-By: Claude Opus 4.8 <[email protected]>
2026-06-13 23:16:42 -04:00

107 lines
3.7 KiB
Rust
Raw Permalink Blame History

This file contains ambiguous Unicode characters

This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.

//! Browser sink for `wai.audio.scene` — the deterministic spatial-audio
//! mixer bound to the page. Same honest split as the worlds sink: the
//! WASM produces the **exact** mixdown (the bytes mixdown-equivalence is
//! defined over — the page plays these and checks the hash against the
//! corpus), while richer presentation (Web Audio HRTF panning) is the
//! JS layer's option.
//!
//! Sources arrive as the corpus `.sources.bin` (decoded mono PCM keyed
//! by content hash) — in a real sink they'd be decoded `wai.audio.*`
//! assets; decoding is the sink's job, exactly as glb/spz is.
use wai::audio_scene::{SpatialScene, Sources};
use wasm_bindgen::prelude::*;
#[wasm_bindgen]
pub struct AudioSink {
scene: SpatialScene,
sources: Sources,
}
fn js_err(e: impl std::fmt::Display) -> JsError {
JsError::new(&e.to_string())
}
fn hex(b: &[u8]) -> String {
b.iter().map(|x| format!("{x:02x}")).collect()
}
#[wasm_bindgen]
impl AudioSink {
/// Open a `WSCN` scene container.
#[wasm_bindgen(constructor)]
pub fn new(wscn: &[u8]) -> Result<AudioSink, JsError> {
Ok(AudioSink {
scene: SpatialScene::from_bytes(wscn).map_err(js_err)?,
sources: Sources::new(),
})
}
/// Load the corpus `.sources.bin`:
/// `n u32 | n × (hash[32], len u32, i16le[len])`.
pub fn load_sources_bin(&mut self, bin: &[u8]) -> Result<(), JsError> {
if bin.len() < 4 {
return Err(JsError::new("sources bin truncated"));
}
let n = u32::from_le_bytes(bin[..4].try_into().unwrap()) as usize;
let mut p = 4usize;
for _ in 0..n {
if p + 36 > bin.len() {
return Err(JsError::new("sources bin truncated"));
}
let hash: [u8; 32] = bin[p..p + 32].try_into().unwrap();
p += 32;
let len = u32::from_le_bytes(bin[p..p + 4].try_into().unwrap()) as usize;
p += 4;
if p + len * 2 > bin.len() {
return Err(JsError::new("sources bin truncated"));
}
let pcm: Vec<i16> = bin[p..p + len * 2]
.chunks_exact(2)
.map(|c| i16::from_le_bytes([c[0], c[1]]))
.collect();
p += len * 2;
self.sources.insert(hash, pcm);
}
Ok(())
}
pub fn sample_rate(&self) -> u32 {
self.scene.contract.sample_rate
}
pub fn duration(&self) -> u64 {
self.scene.contract.duration
}
pub fn floor_layout(&self) -> String {
self.scene.contract.floor_layout.clone()
}
pub fn num_objects(&self) -> u32 {
self.scene.objects.len() as u32
}
pub fn scene_hash_hex(&self) -> String {
hex(&self.scene.scene_hash())
}
/// The exact deterministic mixdown — interleaved i16 PCM for the
/// declared layout. These are the bytes the page plays.
pub fn mix(&self, layout: &str) -> Result<Box<[i16]>, JsError> {
Ok(self.scene.mixdown(&self.sources, layout).map_err(js_err)?.into_boxed_slice())
}
/// BLAKE3 over the canonical PCM — compare to the corpus's expected
/// `mixdown_hash` (in-browser mixdown-equivalence, like the worlds
/// state-hash check).
pub fn mixdown_hash_hex(&self, layout: &str) -> Result<String, JsError> {
Ok(hex(&self.scene.mixdown_hash(&self.sources, layout).map_err(js_err)?))
}
/// Flat `[x, z]` per object at playback sample `t` — for a top-down
/// radar (y/elevation omitted).
pub fn object_xz(&self, t: u64) -> Box<[f32]> {
let mut out = Vec::with_capacity(self.scene.objects.len() * 2);
for p in self.scene.object_positions(t) {
out.push(p[0]);
out.push(p[2]);
}
out.into_boxed_slice()
}
}