open-standards/wai

WAI — Web AI Media Transport & Execution

A container + capability-dispatch standard for media. WAI does not re-implement codecs; it dispatches to SOTA standard libraries (AVIF, JPEG-XL, PNG, JPEG, Opus, FLAC, AV1, zstd, XZ) and adds an envelope that lets the neural-shared-prior path coexist with the model-free floor.

Two paths open every conforming file:

• Neural condition — the sink regenerates the media against a shared ambient prior (a model). The model is a requirement of existence, never shipped (an .mp3 doesn't ship its decoder either).
• Zeroth condition — a menu of registered SOTA codecs. Always satisfiable on any device that ships ffmpeg or equivalent.

The capability a file requires is named, not supplied.

Building

wai-rs builds the codec chain by default (the codecs feature), and those crates link system libraries rather than vendoring them — consistent with "WAI does not re-implement codecs". Without them the build stops in a codec crate's build script, not in WAI's own code:

# macOS
brew install jpeg-xl libavif opus dav1d xz

# Debian / Ubuntu
apt-get install libjxl-dev libavif-dev libopus-dev libdav1d-dev liblzma-dev

To build without the C codec chain — the envelope, manifest and world engine do not need it, which is also how the wasm32 target is built:

cargo check -p wai --no-default-features

What's here

Path	Contents
SPEC.md	WAI v1.0 spec — envelope, manifest, capability menu
wai-rs/	Rust reference implementation (lib + cdylib + staticlib + C FFI)
glossary/	Reference catalog of media codecs surveyed for capability registration. codecs.json is the canonical survey dataset; SCHEMA.md defines the shape and contribution rules. Rendered at wai.transaction.science/glossary.
corpus/	Standard test material (Kodak/CLIC/Tecnick/SIPI images, Xiph Derf + UVG video, SQAM-class audio, enwik8 + Silesia text). Not committed; CORPUS.md documents reacquisition.

Quickstart

cd wai-rs
cargo test --lib    # every registered capability + envelope round-trip
cargo build --release --bin wai
cargo build --release --lib    # produces libwai.dylib/.so/.dll + libwai.a

On macOS, libflac lives under the Homebrew "flac" formula prefix that Cargo doesn't probe automatically. The committed .cargo/config.toml adds the path; if you build outside the repo set RUSTFLAGS="-L /opt/homebrew/opt/flac/lib".

SDK consumption

wai-rs exposes a stable C ABI in src/ffi.rs — the cdylib drops in as libwai.dylib/.so/.dll. Bindings for any language (Python ctypes, Node N-API, Swift, JNI, Go cgo, etc.) call into:

• wai_image_{png,jpeg,avif,jxl}_{encode,decode}
• wai_audio_{opus,flac}_{encode,decode}
• wai_text_{zstd,xz}_{encode,decode}
• wai_envelope_{pack,unpack} — wrap/unwrap the WAI container
• wai_buffer_free, wai_last_error — memory + error handling

Registered capabilities (the zeroth menu)

Capability	Codec	Lossless?	Mandatory?
wai.image.png	PNG	yes	mandatory floor
wai.image.jxl	JPEG-XL	both	recommended
wai.image.avif	AVIF (AV1-based)	lossy	recommended
wai.image.jpeg	JPEG	lossy	recommended (legacy compat)
wai.audio.flac	FLAC	yes	mandatory floor
wai.audio.opus	Opus	lossy	recommended
wai.video.av1	AV1	lossy	recommended
wai.video.av1.lossless	AV1 (quantizer=0)	YUV-lossless	recommended
wai.text.zstd	zstd	yes	mandatory floor
wai.text.xz	XZ/LZMA2	yes	recommended

Neural capabilities (wai.neural.*) are OPTIONAL and sink-advertised. Most run float inference, so their decode is exact only on the build that produced it. A distinguished subset is integer-exact by construction — an all-i64 decode with no float and no ONNX — so the reconstructed artifact is byte-identical on every machine and its hash is portable. Conformance is a hash: a deployer can require intent=replicate to dispatch only to these.

Capability	Integer-exact decode of	Conformance
wai.neural.int_hyper	image — integer hyperprior σ source + integer entropy + integer synthesis	byte-exact (hash-portable)
wai.neural.int_synth	image — integer synthesis only	byte-exact (hash-portable)
wai.video.int_hyper	video — int_hyper I-frame + integer half-pel motion-compensated P-frames	byte-exact (hash-portable)
wai.audio.int_codec	audio — learned 1-D transform, integer synthesis → int16 PCM	byte-exact (hash-portable)
wai.splat.int_codec	3D — learned Gaussian-splat-attribute transform, integer synthesis	byte-exact (hash-portable)

Interactive & temporal content (extensions)

Beyond the codec menu, WAI carries content that consumes input or moves over time — reconstructed from instructions at the sink, over one wai.det.fixed64 fixed-point floor, each with its own deterministic *-equivalence conformance criterion:

Capability	What it carries	Conformance
wai.world.{scene,replay,live}	interactive worlds (signed op log)	replay-equivalence
wai.world.model	action-conditioned world model (signed action ops → next frame, integer transition + synthesis)	replay-equivalence
wai.mesh.glb / wai.splat.spz	3D mesh / Gaussian-splat assets	content-addressed
wai.audio.scene	object-based spatial audio	mixdown-equivalence
wai.audio.score	music as note events (synthesized at the sink)	mixdown-equivalence
wai.haptic.signal	keyframed touch envelopes	sample-equivalence
wai.feed.state	scoreboards / telemetry / market feeds	replay-equivalence
wai.avatar.pose	digital humans (skeletal pose stream)	pose-equivalence
wai.splat.4d	volumetric / 4D temporal Gaussian splats	frame-state-equivalence
wai.film.linear	animation / CGI film (sealed world + camera track)	film-equivalence
wai.film.reel	a film with cuts (one world, a sequence of shots)	reel-equivalence

And the verify half — a signed, energy-accounted lineage for opaque captured content WAI does not reconstruct (wai.asset.provenance). See SPEC.md §5 and extensions/ for each.

License

Apache-2.0 — see LICENSE.