Transaction Science 32b148cd75 wai: EnCodec 24/48 kHz and DAC 16/24 kHz registered with a pinned payload; SoundStream a candidate ...

The neural audio models WAI did not yet name, each at the operating points of its own
publication or release, royalty-free (EnCodec's repository and DAC's weights are MIT),
neural-float, one ONNX file each:

- wai.neural.encodec24: mono, 2/4/8/16/32 codebooks of 1024 at 75 Hz (1.5 to 24 kbps)
- wai.neural.encodec48: stereo, one-second chunks overlapping by 1% with a scale each,
  2/4/8/16 codebooks at 150 Hz (3 to 24 kbps)
- wai.neural.dac16: 1 to 12 codebooks at 50 Hz; wai.neural.dac24: 1 to 32 at 75 Hz
- wai.neural.soundstream: a candidate. The publication's models were not released, and
  the open speech codec built on its architecture ships its own non-ONNX models and
  packet, so no payload format or parameter set is pinned; never dispatched.

SPEC §5 gains *Payloads of the neural audio codecs*: the payload (the layout the route
already read, and a chunked form for the 48 kHz model), the checks made before the
decoder runs, and the decoder contract: the model's full codebook count always, a row
of 1024 marking a codebook not used, so one file decodes every bitrate. DAC's 16/24 kHz
decoders give t*320 - 8 samples, and the bound says so.

neural/audio.rs checks the payload, fills the unused rows, decodes chunk by chunk and
joins the 48 kHz chunks by the model's linear overlap-add; DecodedAudio gains
`channels`, two interleaved. The C ABI has no two-channel form, so encodec48 is not
advertised through it and is refused before decoding.

tools/wai_neural_audio_export_onnx.py exports each decoder from the released weights
and checks it against the model's own decode with the first q codebooks: on Thor all
four agreed within 1.5e-5 at every operating point tested. Vectors in
wai-rs/tests/vectors/neural_audio/ (stdlib generator, its own Python decode as the
second implementation); tests/neural_audio_vectors.rs decodes each through the dispatch
within 1e-5 of it and refuses a payload outside its model, an unpinned decoder, the
candidate and two channels through the C ABI; the neural gates workflow runs it.
Registry exports, wai.h, the glossary links and pin_verify.py's one-file set follow.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

2026-11-09 09:40:50 -05:00

2 KiB

Raw Permalink History

View raw