bd1e94481e
The audio analog of int_hyper — a LEARNED audio codec whose decode is byte-exact,
extending "conformance is a hash" to audio. A neural audio codec (SoundStream /
EnCodec family) decodes through a float network, so ~1e-5 round-off makes the
waveform — and its hash — diverge between machines: never receiptable. Here the
synthesis is a stack of integer 1-D transposed convolutions + integer IGDN over a
per-channel factorized-rANS latent, all i64, so the int16 PCM is byte-identical on
every machine (NeuralIntegerExact; sample-equivalence, a portable PCM hash).
- decoder: src/int_audio.rs — WAM1 model parse (int12 transposed-conv weights as
i16, IGDN gamma as i32), 1-D conv_t + integer IGDN (reuses int_transform::igdn)
over the factorized-rANS latent (y_q = s·2^afrac + median_q) → int16 PCM. Pure
i64, no float/ONNX; compiles to wasm32. Feature audio_int = ["neural_int"].
- encoder: tools/wai_int_audio_export.py — trains a small 1-D transform codec
(Conv1d+GDN / ConvTranspose1d+IGDN) on real captured audio (float; only the
DECODE must be integer), PTQs g_s, factorized-codes the latent (data-fit CDF +
CompressAI rANS, the byte-exact match of the Rust RansDecoder), and runs an
INTEGER reference decode mirroring the Rust arithmetic → PCM FNV-1a.
- capability: wai.audio.int_codec registered (Lossy tier, NeuralIntegerExact
replicate class; nie set now {int_synth, int_hyper, video.int_hyper, audio}).
- conformance: tests/vectors/int_audio/ (a tiny 8192-sample C=16 clip) +
int_audio.rs never-skips. The Rust 1-D conv / IGDN / rANS / PCM-quantization
reproduce the Python integer reference sample-for-sample.
- web: wai-web::decodeIntAudio (audio_int feature) — the same pure-i64 decode in
the browser; reuses fnv1aHex over the LE PCM bytes for the in-page receipt check.
Co-Authored-By: Claude Opus 4.8 <[email protected]>
43 lines
1.8 KiB
Rust
43 lines
1.8 KiB
Rust
//! In-browser **byte-exact learned-audio** decode — zero JS ML runtime.
|
|
//!
|
|
//! The same pure-i64 Rust that decodes the `wai.audio.int_codec` codec natively
|
|
//! compiles to wasm32 and produces the IDENTICAL int16 PCM here: a learned 1-D
|
|
//! transform codec whose synthesis is integer transposed convolutions + integer
|
|
//! IGDN over a factorized-rANS latent — no float anywhere. So a publisher can pin
|
|
//! the PCM hash and the page re-checks it client-side — *sample-equivalence*, a
|
|
//! waveform you can put in a receipt. Reuses `fnv1aHex` (the `generate` binding)
|
|
//! over the little-endian PCM bytes for the in-page check.
|
|
|
|
use wasm_bindgen::prelude::*;
|
|
|
|
/// Decoded audio: int16 PCM, mono — to be fed to a Web Audio buffer and hashed
|
|
/// (its little-endian bytes) for the in-page receipt check.
|
|
#[wasm_bindgen]
|
|
pub struct DecodedAudio {
|
|
pcm: Vec<i16>,
|
|
}
|
|
|
|
#[wasm_bindgen]
|
|
impl DecodedAudio {
|
|
/// The decoded int16 PCM samples (mono).
|
|
#[wasm_bindgen(getter)]
|
|
pub fn pcm(&self) -> Box<[i16]> {
|
|
self.pcm.clone().into_boxed_slice()
|
|
}
|
|
#[wasm_bindgen(getter, js_name = nSamples)]
|
|
pub fn n_samples(&self) -> u32 {
|
|
self.pcm.len() as u32
|
|
}
|
|
}
|
|
|
|
/// Decode a learned-audio model (`audio_model.bin`, WAM1) + bitstream
|
|
/// (`audio_bitstream.json`) to int16 PCM in the browser, byte-identical to the
|
|
/// native reference. No float, no ML runtime.
|
|
#[wasm_bindgen(js_name = decodeIntAudio)]
|
|
pub fn decode_int_audio(model_bin: &[u8], bitstream_json: &[u8]) -> Result<DecodedAudio, JsError> {
|
|
let model = wai::int_audio::AudioModel::from_bytes(model_bin).map_err(JsError::new)?;
|
|
let bs: serde_json::Value =
|
|
serde_json::from_slice(bitstream_json).map_err(|e| JsError::new(&e.to_string()))?;
|
|
let pcm = wai::int_audio::decode(&model, &bs).map_err(JsError::new)?;
|
|
Ok(DecodedAudio { pcm })
|
|
}
|