| Filename | Latest commit message | Latest commit date |
|---|---|---|
|
Some checks failed
wai-quantum energy-landscape jobs / does this change touch it (push) Successful in 14s
WAI breaking changes and the conformance runner / WAI breaking · does this change touch WAI (push) Successful in 18s
WAI breaking changes and the conformance runner / WAI breaking · wire, record and API against main · linux (push) Successful in 1m43s
wai-audit collector and player integration / does this change touch it (push) Has been cancelled
wai-audit collector and player integration / collector · linux (push) Has been cancelled
wai-audit collector and player integration / collector, player integration and the browser end to end · macos (push) Has been cancelled
WAI operation catalogue / does this change touch it (push) Has been cancelled
WAI operation catalogue / the catalogue and every door · linux (push) Has been cancelled
wai-quantum for JavaScript / does this change touch it (push) Has been cancelled
wai-quantum for JavaScript / javascript · linux (push) Has been cancelled
repository checks / lockfiles, advisories, tests in CI, workflow hygiene, spec and section references, public text (push) Has been cancelled
wai-quantum studio and desktop / does this change touch it (push) Has been cancelled
wai-quantum studio and desktop / studio · browser (push) Has been cancelled
wai-quantum studio and desktop / desktop app · macos (push) Has been cancelled
wai-quantum studio and desktop / desktop app · linux (push) Has been cancelled
wai-quantum studio and desktop / desktop app · windows (push) Has been cancelled
wai-sink-android native sink library / does this change touch it (push) Has been cancelled
wai-sink-android native sink library / the JNI layer and the platform-free half on the host JVM · linux-arm64 (push) Has been cancelled
wai-sink-android native sink library / every ABI, the AAR, and the instrumented tests in a headless emulator · macos (push) Has been cancelled
wai-sink-swift native sink package / does this change touch it (push) Has been cancelled
wai-sink-swift native sink package / libwai slices, desktop tests, simulator tests · macos (push) Has been cancelled
WAI Studio / does this change touch it (push) Has been cancelled
WAI Studio / the catalogue's actions, the unit tests and the core · linux (push) Has been cancelled
WAI Studio / Studio in Chromium and Firefox · macos (push) Has been cancelled
WAI Studio / Studio in WebKit · linux (push) Has been cancelled
wai-quantum energy-landscape jobs / the module under wasm32-wasip2 (wasmtime) (push) Successful in 2m33s
joule-moq-remote host agent / joule-moq-remote · does this change touch it (push) Has been cancelled
joule-moq-remote host agent / joule-moq-remote, every target (linux-arm64) (push) Has been cancelled
joule-moq-remote host agent / joule-moq-remote, every target (macos) (push) Has been cancelled
joule-moq-room room service / joule-moq-room · does this change touch it (push) Has been cancelled
joule-moq-room room service / joule-moq-room, every target (push) Has been cancelled
joule-moq-web browser client / joule-moq-web · does this change touch it (push) Has been cancelled
joule-moq-web browser client / joule-moq-web · browser end-to-end (macos) (push) Has been cancelled
wai-app and Create / wai-app · does this change touch it (push) Has been cancelled
wai-app and Create / wai-app · tests, clippy, the line language's verifier (linux) (push) Has been cancelled
wai-app and Create / wai-app · Create, browser end-to-end (macos) (push) Has been cancelled
wai-fleet service / does this change touch it (push) Has been cancelled
wai-fleet service / service · linux (push) Has been cancelled
wai-fleet service / service · macos (push) Has been cancelled
wai-ledger service / does this change touch it (push) Has been cancelled
wai-ledger service / service · linux (push) Has been cancelled
wai-ledger service / service · macos (push) Has been cancelled
wai-quantum energy-landscape jobs / the same bits · macos (push) Successful in 4m6s
wai-quantum energy-landscape jobs / native against WebAssembly, over many seeds (push) Successful in 4m38s
WAI breaking changes and the conformance runner / WAI breaking and conform · their tests, and the independent verifiers through wai conform · linux (push) Successful in 5m17s
wai-key-proxy / does this change touch it (push) Has been cancelled
wai-key-proxy / proxy · linux (push) Has been cancelled
wai-quantum energy-landscape jobs / the same bits · linux-arm64 (push) Has been cancelled
wai-quantum energy-landscape jobs / reader, kernels, answers and the stated result · linux, release (push) Has been cancelled
The neural audio models WAI did not yet name, each at the operating points of its own publication or release, royalty-free (EnCodec's repository and DAC's weights are MIT), neural-float, one ONNX file each: - wai.neural.encodec24: mono, 2/4/8/16/32 codebooks of 1024 at 75 Hz (1.5 to 24 kbps) - wai.neural.encodec48: stereo, one-second chunks overlapping by 1% with a scale each, 2/4/8/16 codebooks at 150 Hz (3 to 24 kbps) - wai.neural.dac16: 1 to 12 codebooks at 50 Hz; wai.neural.dac24: 1 to 32 at 75 Hz - wai.neural.soundstream: a candidate. The publication's models were not released, and the open speech codec built on its architecture ships its own non-ONNX models and packet, so no payload format or parameter set is pinned; never dispatched. SPEC §5 gains *Payloads of the neural audio codecs*: the payload (the layout the route already read, and a chunked form for the 48 kHz model), the checks made before the decoder runs, and the decoder contract: the model's full codebook count always, a row of 1024 marking a codebook not used, so one file decodes every bitrate. DAC's 16/24 kHz decoders give t*320 - 8 samples, and the bound says so. neural/audio.rs checks the payload, fills the unused rows, decodes chunk by chunk and joins the 48 kHz chunks by the model's linear overlap-add; DecodedAudio gains `channels`, two interleaved. The C ABI has no two-channel form, so encodec48 is not advertised through it and is refused before decoding. tools/wai_neural_audio_export_onnx.py exports each decoder from the released weights and checks it against the model's own decode with the first q codebooks: on Thor all four agreed within 1.5e-5 at every operating point tested. Vectors in wai-rs/tests/vectors/neural_audio/ (stdlib generator, its own Python decode as the second implementation); tests/neural_audio_vectors.rs decodes each through the dispatch within 1e-5 of it and refuses a payload outside its model, an unpinned decoder, the candidate and two channels through the C ABI; the neural gates workflow runs it. Registry exports, wai.h, the glossary links and pin_verify.py's one-file set follow. Co-Authored-By: Claude Opus 5.5 <[email protected]> |
||
| .. | ||
| dac16 | ||
| dac24 | ||
| encodec24 | ||
| encodec48 | ||
| .gitattributes | ||
| README.md | ||
Neural audio route vectors
One vector for each capability whose payload SPEC §5 pins (Payloads of the neural audio
codecs): wai.neural.encodec24, wai.neural.encodec48, wai.neural.dac16 and
wai.neural.dac24. They hold the route of the reference sink (neural/audio.rs) to the
pinned layout and decode; tests/neural_audio_vectors.rs decodes them.
Provenance. Generated by tools/wai_neural_audio_vectors.py from fixed dyadic
parameters and SplitMix64 codes (seeds 1 to 4). No audio and no model weights: each decoder
has its model's geometry and contract, not its network. Apache-2.0, as the repository.
Each directory holds:
decoder.onnx— a small decoder with the model's codebook countQ, codebooks of 1024, a hop of 320 and its channels: inputsaudio_codes(int64[1, 1, Q, T]) andaudio_scales(float[1]), outputaudio_values(float[1, C, T·320 − d],d= 8 for the two DAC models, else 0). A codecof codebookiaddsc·A[i] + B[i]to its frame's latentz, a row holding 1024 adds nothing, and samplekof the frame on channelchistanh(z·W[ch][k])times the scale.A,BandWare dyadic, so every value before thetanhis exact in binary32.vector.wai— aWAI1envelope of the capability, pinned todecoder.onnx, whose payload carries fewer codebooks than the model holds (all 32 fordac24), so the route must mark the others unused;encodec48carries two chunks with scales 0.75 and 0.5, its last chunk six frames.expected.f32— the decode at the samplesmeta.jsonselects (every one, or forencodec48every sixteenth and every one in the chunks' overlap), each sample's channels in order, as little-endian binary32. The generator computes it itself, in Python, from the payload and the decoder's parameters: the second implementation of the route.meta.json— the capability, its geometry, the payload's shape, the selection rule and the decoder's SHA-256.
The reference's decode is within 10⁻⁵ of expected.f32 (the decoder's tanh is the
runtime's). python3 -I tools/wai_neural_audio_vectors.py --check rebuilds every file and
compares: the decoders and envelopes byte for byte, expected.f32 within 10⁻⁶.