| Filename | Latest commit message | Latest commit date |
|---|---|---|
0252dc8a82
parse_tools_header decodes the post-filter chain of the TON substream in filters-factory order (EFE-linear → ICCI → eICCI → EFE-nonlinear → LEF), each tool gated by its own enable flag per the recursive header walk. Adds BitReader::read_max / bits_for_max: the header's universal width rule, bits = ceil(log2(max_value+1)) (the reference HeaderCoder::calc_bits_count). No header field has a fixed width, and some are runtime-dependent — a max_symbol decoded earlier in a segment sets the width of the weights after it (65535 ⇒ 16 bits, then 99 ⇒ 7). Validated on real test256.bin, every field matching the reference and end_bit=973 — all TON fields consumed exactly, up to the 3-bit pad closing the 63-byte segment. The decoded parameters ARE the post-filters' fixture inputs: B1/100 = 125.61/130.91 = efl_chroma's plane means; luma_min/max = 497/25320 and the 8 A1 weights = efn_filter's luma_min_q/luma_max_q/weights_q15; icci_model_signalled_idx[Y]=0,[UV]=1 = the per-tile eICCI model selection. Header parser and post-filters agree from opposite directions. Also adds an end-to-end chain test: test256.bin → parse_container → SOZ → mANS fac decode → −31 → z (2560 vals, 75 non-zero) → decode_hyper_scale → i_s, byte-exact 40960/40960 against the reference. Two independently-built subsystems, chained off raw codestream bytes with no fixture input beyond the normative CDF/weight assets. parse_tools_header takes the first filter's bit offset rather than the TON payload start: _stream_header_part defaults to "tool_header", so other tools also emit into TON ahead of the filter chain (two such bits precede EFE-linear here); locating that prefix in general needs the full tool-tree walk. Co-Authored-By: Claude Opus 4.8 <[email protected]> |
||
| .. | ||
| .github/workflows | ||
| audio-conformance | ||
| avatar-conformance | ||
| byte-exact-conformance | ||
| extensions | ||
| feed-conformance | ||
| film-conformance | ||
| glossary | ||
| haptic-conformance | ||
| neural-conformance | ||
| research | ||
| score-conformance | ||
| tools | ||
| vocab | ||
| volumetric-conformance | ||
| wai-relay | ||
| wai-rs | ||
| wai-spz-gen | ||
| wai-web | ||
| world-conformance | ||
| .gitignore | ||
| CHANGELOG.md | ||
| CONTRIBUTING.md | ||
| LICENSE | ||
| README.md | ||
| SPEC.md | ||
WAI — Web AI Media Transport & Execution
A container + capability-dispatch standard for media. WAI does not re-implement codecs; it dispatches to SOTA standard libraries (AVIF, JPEG-XL, PNG, JPEG, Opus, FLAC, AV1, zstd, XZ) and adds an envelope that lets the neural-shared-prior path coexist with the model-free floor.
Two paths open every conforming file:
- Neural condition — the sink regenerates the media against a
shared ambient prior (a model). The model is a requirement of
existence, never shipped (an
.mp3doesn't ship its decoder either). - Zeroth condition — a menu of registered SOTA codecs. Always satisfiable on any device that ships ffmpeg or equivalent.
The capability a file requires is named, not supplied.
What's here
| Path | Contents |
|---|---|
SPEC.md |
WAI v1.0 spec — envelope, manifest, capability menu |
wai-rs/ |
Rust reference implementation (lib + cdylib + staticlib + C FFI) |
glossary/ |
Reference catalog of media codecs surveyed for capability registration. codecs.json is the canonical 342-entry dataset; SCHEMA.md defines the shape and contribution rules. Rendered at wai.transaction.science/glossary. |
corpus/ |
Standard test material (Kodak/CLIC/Tecnick/SIPI images, Xiph Derf + UVG video, SQAM-class audio, enwik8 + Silesia text). Not committed; CORPUS.md documents reacquisition. |
Quickstart
cd wai-rs
cargo test --lib # every registered capability + envelope round-trip
cargo build --release --bin wai
cargo build --release --lib # produces libwai.dylib/.so/.dll + libwai.a
On macOS, libflac lives under the Homebrew "flac" formula prefix that
Cargo doesn't probe automatically. The committed .cargo/config.toml
adds the path; if you build outside the repo set
RUSTFLAGS="-L /opt/homebrew/opt/flac/lib".
SDK consumption
wai-rs exposes a stable C ABI in src/ffi.rs —
the cdylib drops in as libwai.dylib/.so/.dll. Bindings for any
language (Python ctypes, Node N-API, Swift, JNI, Go cgo, etc.) call
into:
wai_image_{png,jpeg,avif,jxl}_{encode,decode}wai_audio_{opus,flac}_{encode,decode}wai_text_{zstd,xz}_{encode,decode}wai_envelope_{pack,unpack}— wrap/unwrap the WAI containerwai_buffer_free,wai_last_error— memory + error handling
Registered capabilities (the zeroth menu)
| Capability | Codec | Lossless? | Mandatory? |
|---|---|---|---|
wai.image.png |
PNG | yes | mandatory floor |
wai.image.jxl |
JPEG-XL | both | recommended |
wai.image.avif |
AVIF (AV1-based) | lossy | recommended |
wai.image.jpeg |
JPEG | lossy | recommended (legacy compat) |
wai.audio.flac |
FLAC | yes | mandatory floor |
wai.audio.opus |
Opus | lossy | recommended |
wai.video.av1 |
AV1 | lossy | recommended |
wai.video.av1.lossless |
AV1 (quantizer=0) |
YUV-lossless | recommended |
wai.text.zstd |
zstd | yes | mandatory floor |
wai.text.xz |
XZ/LZMA2 | yes | recommended |
Neural capabilities (wai.neural.*) are OPTIONAL and sink-advertised. Most
run float inference, so their decode is exact only on the build that produced
it. A distinguished subset is integer-exact by construction — an all-i64
decode with no float and no ONNX — so the reconstructed artifact is
byte-identical on every machine and its hash is portable. Conformance is a
hash: a deployer can require intent=replicate to dispatch only to these.
| Capability | Integer-exact decode of | Conformance |
|---|---|---|
wai.neural.int_hyper |
image — integer hyperprior σ source + integer entropy + integer synthesis | byte-exact (hash-portable) |
wai.neural.int_synth |
image — integer synthesis only | byte-exact (hash-portable) |
wai.video.int_hyper |
video — int_hyper I-frame + integer half-pel motion-compensated P-frames |
byte-exact (hash-portable) |
wai.audio.int_codec |
audio — learned 1-D transform, integer synthesis → int16 PCM | byte-exact (hash-portable) |
wai.splat.int_codec |
3D — learned Gaussian-splat-attribute transform, integer synthesis | byte-exact (hash-portable) |
Interactive & temporal content (extensions)
Beyond the codec menu, WAI carries content that consumes input or moves
over time — reconstructed from instructions at the sink, over one
wai.det.fixed64 fixed-point floor, each with its own deterministic
*-equivalence conformance criterion:
| Capability | What it carries | Conformance |
|---|---|---|
wai.world.{scene,replay,live} |
interactive worlds (signed op log) | replay-equivalence |
wai.world.model |
action-conditioned world model (signed action ops → next frame, integer transition + synthesis) | replay-equivalence |
wai.mesh.glb / wai.splat.spz |
3D mesh / Gaussian-splat assets | content-addressed |
wai.audio.scene |
object-based spatial audio | mixdown-equivalence |
wai.audio.score |
music as note events (synthesized at the sink) | mixdown-equivalence |
wai.haptic.signal |
keyframed touch envelopes | sample-equivalence |
wai.feed.state |
scoreboards / telemetry / market feeds | replay-equivalence |
wai.avatar.pose |
digital humans (skeletal pose stream) | pose-equivalence |
wai.splat.4d |
volumetric / 4D temporal Gaussian splats | frame-state-equivalence |
wai.film.linear |
animation / CGI film (sealed world + camera track) | film-equivalence |
wai.film.reel |
a film with cuts (one world, a sequence of shots) | reel-equivalence |
And the verify half — a signed, energy-accounted lineage for opaque
captured content WAI does not reconstruct (wai.asset.provenance).
See SPEC.md §5 and extensions/ for each.
License
Apache-2.0 — see LICENSE.