bd1e94481e
The audio analog of int_hyper — a LEARNED audio codec whose decode is byte-exact,
extending "conformance is a hash" to audio. A neural audio codec (SoundStream /
EnCodec family) decodes through a float network, so ~1e-5 round-off makes the
waveform — and its hash — diverge between machines: never receiptable. Here the
synthesis is a stack of integer 1-D transposed convolutions + integer IGDN over a
per-channel factorized-rANS latent, all i64, so the int16 PCM is byte-identical on
every machine (NeuralIntegerExact; sample-equivalence, a portable PCM hash).
- decoder: src/int_audio.rs — WAM1 model parse (int12 transposed-conv weights as
i16, IGDN gamma as i32), 1-D conv_t + integer IGDN (reuses int_transform::igdn)
over the factorized-rANS latent (y_q = s·2^afrac + median_q) → int16 PCM. Pure
i64, no float/ONNX; compiles to wasm32. Feature audio_int = ["neural_int"].
- encoder: tools/wai_int_audio_export.py — trains a small 1-D transform codec
(Conv1d+GDN / ConvTranspose1d+IGDN) on real captured audio (float; only the
DECODE must be integer), PTQs g_s, factorized-codes the latent (data-fit CDF +
CompressAI rANS, the byte-exact match of the Rust RansDecoder), and runs an
INTEGER reference decode mirroring the Rust arithmetic → PCM FNV-1a.
- capability: wai.audio.int_codec registered (Lossy tier, NeuralIntegerExact
replicate class; nie set now {int_synth, int_hyper, video.int_hyper, audio}).
- conformance: tests/vectors/int_audio/ (a tiny 8192-sample C=16 clip) +
int_audio.rs never-skips. The Rust 1-D conv / IGDN / rANS / PCM-quantization
reproduce the Python integer reference sample-for-sample.
- web: wai-web::decodeIntAudio (audio_int feature) — the same pure-i64 decode in
the browser; reuses fnv1aHex over the LE PCM bytes for the in-page receipt check.
Co-Authored-By: Claude Opus 4.8 <[email protected]>
39 lines
1.8 KiB
Rust
39 lines
1.8 KiB
Rust
//! `wai.audio.int_codec` — **sample-equivalence**: a LEARNED 1-D audio transform
|
|
//! codec whose integer synthesis (transposed convs + IGDN) decodes a factorized
|
|
//! rANS latent to int16 PCM byte-identically on every machine, so the PCM hash is
|
|
//! portable — the audio analog of int_hyper. Committed tiny vector under
|
|
//! `tests/vectors/int_audio/`; never skips.
|
|
#![cfg(feature = "audio_int")]
|
|
|
|
use std::path::PathBuf;
|
|
|
|
use wai::int_audio::{decode, pcm_fnv1a, AudioModel};
|
|
|
|
fn dir() -> PathBuf {
|
|
PathBuf::from(env!("CARGO_MANIFEST_DIR")).join("tests/vectors/int_audio")
|
|
}
|
|
|
|
#[test]
|
|
fn decodes_the_learned_audio_clip_sample_for_sample() {
|
|
// The committed integer model (WAM1) + factorized bitstream decode — no float,
|
|
// no ML runtime — to the SAME int16 PCM the Python integer reference produced.
|
|
// If the Rust 1-D conv / IGDN / rANS / PCM-quantization disagreed with the
|
|
// exporter by a single sample, this FNV-1a would diverge.
|
|
let model = AudioModel::from_bytes(&std::fs::read(dir().join("audio_model.bin")).unwrap())
|
|
.expect("parse WAM1 model");
|
|
let bs: serde_json::Value =
|
|
serde_json::from_slice(&std::fs::read(dir().join("audio_bitstream.json")).unwrap()).unwrap();
|
|
let conf: serde_json::Value =
|
|
serde_json::from_slice(&std::fs::read(dir().join("audio.json")).unwrap()).unwrap();
|
|
|
|
let pcm = decode(&model, &bs).expect("decode");
|
|
assert_eq!(pcm.len(), conf["n_samples"].as_u64().unwrap() as usize, "sample count");
|
|
|
|
let got = format!("{:016x}", pcm_fnv1a(&pcm));
|
|
assert_eq!(got, conf["pcm_fnv1a"].as_str().unwrap(), "PCM must reproduce its committed hash");
|
|
println!(
|
|
"int_audio: {} samples @ {} Hz -> sample-exact, pcm_hash {got}",
|
|
pcm.len(),
|
|
conf["sample_rate"].as_u64().unwrap()
|
|
);
|
|
}
|