| Filename | Latest commit message | Latest commit date |
|---|---|---|
954307816d
Ports the last two inputs the mANS sgm decoder needed but that were still being read
from an e2e fixture. With these, every input of the decode path is computable from the
codestream plus normative assets.
skip_mask scale_hat = i_s + gain_per_ch ; mask = scale_hat > 382
NOTE the skip decision uses the scale WITHOUT the RVS additive, while
sigma_index uses it WITH — the two differ by exactly the rvs values and
are easy to conflate.
rvs_additive likely = clamp((sum_8x8 |i_s| + 32) >> 6, 0, 4351)
band = #{ e in [1003,1352,1937] : likely >= e }
rvs = A[grfs_flag ? 0 : 1][band]
Band edges are precomputed as idx2log(thr * 2^4) against the reference
Log2LinConvertion table, so the 4352-entry table is not needed at decode time; the
per-band additives come from the normative scale list scaled by the GRFS branch factor.
Band membership is INCLUSIVE at the lower edge — only the luma plane exercises this
(one block lands exactly on 1352 and the reference places it in the band above), which
is why both planes are tested.
Validated against the ISO reference on real test256.bin, both planes, including the
composition (derived rvs fed through sigma_index must reproduce the indices the
reference entropy coder consumed):
luma rvs 40960/40960, mask 40960/40960 (703 coded), sigma 40960/40960
chroma rvs 24576/24576, mask 24576/24576 (537 coded), sigma 24576/24576
Co-Authored-By: Claude Opus 4.8 <[email protected]>
|
||
| .. | ||
| .github/workflows | ||
| audio-conformance | ||
| avatar-conformance | ||
| byte-exact-conformance | ||
| extensions | ||
| feed-conformance | ||
| film-conformance | ||
| glossary | ||
| haptic-conformance | ||
| neural-conformance | ||
| research | ||
| score-conformance | ||
| tools | ||
| vocab | ||
| volumetric-conformance | ||
| wai-relay | ||
| wai-rs | ||
| wai-spz-gen | ||
| wai-web | ||
| world-conformance | ||
| .gitignore | ||
| CHANGELOG.md | ||
| CONTRIBUTING.md | ||
| LICENSE | ||
| README.md | ||
| SPEC.md | ||
WAI — Web AI Media Transport & Execution
A container + capability-dispatch standard for media. WAI does not re-implement codecs; it dispatches to SOTA standard libraries (AVIF, JPEG-XL, PNG, JPEG, Opus, FLAC, AV1, zstd, XZ) and adds an envelope that lets the neural-shared-prior path coexist with the model-free floor.
Two paths open every conforming file:
- Neural condition — the sink regenerates the media against a
shared ambient prior (a model). The model is a requirement of
existence, never shipped (an
.mp3doesn't ship its decoder either). - Zeroth condition — a menu of registered SOTA codecs. Always satisfiable on any device that ships ffmpeg or equivalent.
The capability a file requires is named, not supplied.
What's here
| Path | Contents |
|---|---|
SPEC.md |
WAI v1.0 spec — envelope, manifest, capability menu |
wai-rs/ |
Rust reference implementation (lib + cdylib + staticlib + C FFI) |
glossary/ |
Reference catalog of media codecs surveyed for capability registration. codecs.json is the canonical 342-entry dataset; SCHEMA.md defines the shape and contribution rules. Rendered at wai.transaction.science/glossary. |
corpus/ |
Standard test material (Kodak/CLIC/Tecnick/SIPI images, Xiph Derf + UVG video, SQAM-class audio, enwik8 + Silesia text). Not committed; CORPUS.md documents reacquisition. |
Quickstart
cd wai-rs
cargo test --lib # every registered capability + envelope round-trip
cargo build --release --bin wai
cargo build --release --lib # produces libwai.dylib/.so/.dll + libwai.a
On macOS, libflac lives under the Homebrew "flac" formula prefix that
Cargo doesn't probe automatically. The committed .cargo/config.toml
adds the path; if you build outside the repo set
RUSTFLAGS="-L /opt/homebrew/opt/flac/lib".
SDK consumption
wai-rs exposes a stable C ABI in src/ffi.rs —
the cdylib drops in as libwai.dylib/.so/.dll. Bindings for any
language (Python ctypes, Node N-API, Swift, JNI, Go cgo, etc.) call
into:
wai_image_{png,jpeg,avif,jxl}_{encode,decode}wai_audio_{opus,flac}_{encode,decode}wai_text_{zstd,xz}_{encode,decode}wai_envelope_{pack,unpack}— wrap/unwrap the WAI containerwai_buffer_free,wai_last_error— memory + error handling
Registered capabilities (the zeroth menu)
| Capability | Codec | Lossless? | Mandatory? |
|---|---|---|---|
wai.image.png |
PNG | yes | mandatory floor |
wai.image.jxl |
JPEG-XL | both | recommended |
wai.image.avif |
AVIF (AV1-based) | lossy | recommended |
wai.image.jpeg |
JPEG | lossy | recommended (legacy compat) |
wai.audio.flac |
FLAC | yes | mandatory floor |
wai.audio.opus |
Opus | lossy | recommended |
wai.video.av1 |
AV1 | lossy | recommended |
wai.video.av1.lossless |
AV1 (quantizer=0) |
YUV-lossless | recommended |
wai.text.zstd |
zstd | yes | mandatory floor |
wai.text.xz |
XZ/LZMA2 | yes | recommended |
Neural capabilities (wai.neural.*) are OPTIONAL and sink-advertised. Most
run float inference, so their decode is exact only on the build that produced
it. A distinguished subset is integer-exact by construction — an all-i64
decode with no float and no ONNX — so the reconstructed artifact is
byte-identical on every machine and its hash is portable. Conformance is a
hash: a deployer can require intent=replicate to dispatch only to these.
| Capability | Integer-exact decode of | Conformance |
|---|---|---|
wai.neural.int_hyper |
image — integer hyperprior σ source + integer entropy + integer synthesis | byte-exact (hash-portable) |
wai.neural.int_synth |
image — integer synthesis only | byte-exact (hash-portable) |
wai.video.int_hyper |
video — int_hyper I-frame + integer half-pel motion-compensated P-frames |
byte-exact (hash-portable) |
wai.audio.int_codec |
audio — learned 1-D transform, integer synthesis → int16 PCM | byte-exact (hash-portable) |
wai.splat.int_codec |
3D — learned Gaussian-splat-attribute transform, integer synthesis | byte-exact (hash-portable) |
Interactive & temporal content (extensions)
Beyond the codec menu, WAI carries content that consumes input or moves
over time — reconstructed from instructions at the sink, over one
wai.det.fixed64 fixed-point floor, each with its own deterministic
*-equivalence conformance criterion:
| Capability | What it carries | Conformance |
|---|---|---|
wai.world.{scene,replay,live} |
interactive worlds (signed op log) | replay-equivalence |
wai.world.model |
action-conditioned world model (signed action ops → next frame, integer transition + synthesis) | replay-equivalence |
wai.mesh.glb / wai.splat.spz |
3D mesh / Gaussian-splat assets | content-addressed |
wai.audio.scene |
object-based spatial audio | mixdown-equivalence |
wai.audio.score |
music as note events (synthesized at the sink) | mixdown-equivalence |
wai.haptic.signal |
keyframed touch envelopes | sample-equivalence |
wai.feed.state |
scoreboards / telemetry / market feeds | replay-equivalence |
wai.avatar.pose |
digital humans (skeletal pose stream) | pose-equivalence |
wai.splat.4d |
volumetric / 4D temporal Gaussian splats | frame-state-equivalence |
wai.film.linear |
animation / CGI film (sealed world + camera track) | film-equivalence |
wai.film.reel |
a film with cuts (one world, a sequence of shots) | reel-equivalence |
And the verify half — a signed, energy-accounted lineage for opaque
captured content WAI does not reconstruct (wai.asset.provenance).
See SPEC.md §5 and extensions/ for each.
License
Apache-2.0 — see LICENSE.