The neural audio models WAI did not yet name, each at the operating points of its own publication or release, royalty-free (EnCodec's repository and DAC's weights are MIT), neural-float, one ONNX file each: - wai.neural.encodec24: mono, 2/4/8/16/32 codebooks of 1024 at 75 Hz (1.5 to 24 kbps) - wai.neural.encodec48: stereo, one-second chunks overlapping by 1% with a scale each, 2/4/8/16 codebooks at 150 Hz (3 to 24 kbps) - wai.neural.dac16: 1 to 12 codebooks at 50 Hz; wai.neural.dac24: 1 to 32 at 75 Hz - wai.neural.soundstream: a candidate. The publication's models were not released, and the open speech codec built on its architecture ships its own non-ONNX models and packet, so no payload format or parameter set is pinned; never dispatched. SPEC §5 gains *Payloads of the neural audio codecs*: the payload (the layout the route already read, and a chunked form for the 48 kHz model), the checks made before the decoder runs, and the decoder contract: the model's full codebook count always, a row of 1024 marking a codebook not used, so one file decodes every bitrate. DAC's 16/24 kHz decoders give t*320 - 8 samples, and the bound says so. neural/audio.rs checks the payload, fills the unused rows, decodes chunk by chunk and joins the 48 kHz chunks by the model's linear overlap-add; DecodedAudio gains `channels`, two interleaved. The C ABI has no two-channel form, so encodec48 is not advertised through it and is refused before decoding. tools/wai_neural_audio_export_onnx.py exports each decoder from the released weights and checks it against the model's own decode with the first q codebooks: on Thor all four agreed within 1.5e-5 at every operating point tested. Vectors in wai-rs/tests/vectors/neural_audio/ (stdlib generator, its own Python decode as the second implementation); tests/neural_audio_vectors.rs decodes each through the dispatch within 1e-5 of it and refuses a payload outside its model, an unpinned decoder, the candidate and two channels through the C ABI; the neural gates workflow runs it. Registry exports, wai.h, the glossary links and pin_verify.py's one-file set follow. Co-Authored-By: Claude Opus 5.5 <[email protected]>
2.2 KiB
Neural audio route vectors
One vector for each capability whose payload SPEC §5 pins (Payloads of the neural audio
codecs): wai.neural.encodec24, wai.neural.encodec48, wai.neural.dac16 and
wai.neural.dac24. They hold the route of the reference sink (neural/audio.rs) to the
pinned layout and decode; tests/neural_audio_vectors.rs decodes them.
Provenance. Generated by tools/wai_neural_audio_vectors.py from fixed dyadic
parameters and SplitMix64 codes (seeds 1 to 4). No audio and no model weights: each decoder
has its model's geometry and contract, not its network. Apache-2.0, as the repository.
Each directory holds:
decoder.onnx— a small decoder with the model's codebook countQ, codebooks of 1024, a hop of 320 and its channels: inputsaudio_codes(int64[1, 1, Q, T]) andaudio_scales(float[1]), outputaudio_values(float[1, C, T·320 − d],d= 8 for the two DAC models, else 0). A codecof codebookiaddsc·A[i] + B[i]to its frame's latentz, a row holding 1024 adds nothing, and samplekof the frame on channelchistanh(z·W[ch][k])times the scale.A,BandWare dyadic, so every value before thetanhis exact in binary32.vector.wai— aWAI1envelope of the capability, pinned todecoder.onnx, whose payload carries fewer codebooks than the model holds (all 32 fordac24), so the route must mark the others unused;encodec48carries two chunks with scales 0.75 and 0.5, its last chunk six frames.expected.f32— the decode at the samplesmeta.jsonselects (every one, or forencodec48every sixteenth and every one in the chunks' overlap), each sample's channels in order, as little-endian binary32. The generator computes it itself, in Python, from the payload and the decoder's parameters: the second implementation of the route.meta.json— the capability, its geometry, the payload's shape, the selection rule and the decoder's SHA-256.
The reference's decode is within 10⁻⁵ of expected.f32 (the decoder's tanh is the
runtime's). python3 -I tools/wai_neural_audio_vectors.py --check rebuilds every file and
compares: the decoders and envelopes byte for byte, expected.f32 within 10⁻⁶.