open-standards/wai/README.md

WAI — Web AI Media Transport & Execution

A container + capability-dispatch standard for media. A standard codec (AVIF, JPEG XL, PNG, JPEG, Opus, FLAC, AV1, zstd, XZ and the others the registry names) is dispatched to the library a sink already has; WAI also defines its own capability classes — integer codecs, worlds, scores, films — each with a conformance criterion, and an envelope that lets the neural-shared-prior path coexist with the model-free floor.

Two paths open every conforming file:

• Neural condition — the sink regenerates the media against a shared ambient prior (a model). The model is a requirement of existence, never shipped (an .mp3 doesn't ship its decoder either).
• Zeroth condition — a menu of registered standard codecs, decoded with the libraries a device already ships.

The capability a file requires is named, not supplied.

Building

wai-rs builds the codec chain by default (the codecs feature), and those crates link system libraries rather than vendoring them — consistent with "WAI does not re-implement codecs". Without them the build stops in a codec crate's build script, not in WAI's own code:

# macOS
brew install jpeg-xl libavif opus dav1d xz

# Debian / Ubuntu
apt-get install libjxl-dev libavif-dev libopus-dev libdav1d-dev liblzma-dev

To build without the C codec chain — the envelope, manifest and world engine do not need it, which is also how the wasm32 target is built:

cargo check -p wai --no-default-features

What's here

Path	Contents
SPEC.md	WAI v1.0 spec — envelope, manifest, capability menu
wai-rs/	Rust reference implementation (lib + cdylib + staticlib + C FFI)
REGISTRY.md, registry/capabilities.json	The registry: every registered string with its kind, status, classes and edition, generated from the reference implementation (wai registry export), with its JSON Schema and the policy for registering a string
vocab/	RDF vocabulary: v2 generated from the registry, v1 frozen
extensions/	Sub-specifications (confidentiality, receipts, worlds, film, quantum operations, integer operators, …)
research/	Research notes and dated records: graded, not normative
glossary/	Reference catalog of media codecs surveyed for capability registration. codecs.json is the dataset, linked to the registry; SCHEMA.md defines the shape and contribution rules. Rendered at wai.transaction.science/glossary.
CONTRIBUTING.md	How to change the specification, register a capability and add tests CI runs

Quickstart

cd wai-rs
cargo test --no-default-features --lib --test carriage   # registry, envelope, every capability round-trips
cargo run --no-default-features --bin wai -- capabilities  # the registry, one row per capability
cargo build --release --lib    # with the codecs feature: libwai.dylib/.so/.dll + libwai.a

On macOS, libflac lives under the Homebrew "flac" formula prefix that Cargo doesn't probe automatically. The committed .cargo/config.toml adds the path; if you build outside the repo set RUSTFLAGS="-L /opt/homebrew/opt/flac/lib".

SDK consumption

wai-rs exposes a stable C ABI in src/ffi.rs — the cdylib drops in as libwai.dylib/.so/.dll. Bindings for any language (Python ctypes, Node N-API, Swift, JNI, Go cgo, etc.) call into:

• wai_image_{png,jpeg,avif,jxl}_{encode,decode}
• wai_audio_{opus,flac}_{encode,decode}
• wai_text_{zstd,xz}_{encode,decode}
• wai_envelope_{pack,unpack} — wrap/unwrap the WAI container
• wai_buffer_free, wai_last_error — memory + error handling

Registered capabilities (the zeroth menu)

Capability	Codec	Lossless?	Mandatory?
wai.image.png	PNG	yes	mandatory floor
wai.image.jxl	JPEG-XL	both	recommended
wai.image.avif	AVIF (AV1-based)	lossy	recommended
wai.image.jpeg	JPEG	lossy	recommended (legacy compat)
wai.audio.flac	FLAC	yes	mandatory floor
wai.audio.opus	Opus	lossy	recommended
wai.video.av1	AV1	lossy	recommended
wai.video.av1.lossless	AV1 (quantizer=0)	YUV-lossless	recommended
wai.text.zstd	zstd	yes	mandatory floor
wai.text.xz	XZ/LZMA2	yes	recommended

Neural capabilities (wai.neural.*) are OPTIONAL and sink-advertised. Most run float inference, so their decode is exact only on the build that produced it. A distinguished subset is integer-exact by construction — an all-i64 decode with no float and no ONNX — so the reconstructed artifact is byte-identical on every machine and its hash is portable. Conformance is a hash: a deployer can require intent=replicate to dispatch only to these.

Capability	Integer-exact decode of	Conformance
wai.neural.int_hyper	image — integer hyperprior σ source + integer entropy + integer synthesis	byte-exact (hash-portable)
wai.neural.int_mlicpp	image — MLIC++ entropy topology (channel slices × checkerboard passes, an entropy network re-run before each pass) on integer networks	byte-exact (hash-portable)
wai.neural.int_synth	image — integer synthesis only	byte-exact (hash-portable)
wai.video.int_hyper	video — int_hyper I-frame + integer half-pel motion-compensated P-frames	byte-exact (hash-portable)
wai.audio.int_codec	audio — learned 1-D transform, integer synthesis → int16 PCM	byte-exact (hash-portable)
wai.splat.int_codec	3D — learned Gaussian-splat-attribute transform, integer synthesis	byte-exact (hash-portable)

Their payloads are binary formats SPEC §5 pins — WIV1 for the integer video codecs, WIH1, WIA1 and WIS1 (extensions/integer-payloads.md). The JSON the encode and export tools write is converted to them, never carried. The same extension writes down the wai.neural.int_synth payload and the WIM1 payload of wai.neural.int_mlicpp. Their networks' integer arithmetic is written down as the operator set wai-ops/1 (extensions/integer-operators.md, Draft).

Interactive & temporal content (extensions)

Beyond the codec menu, WAI carries content that consumes input or moves over time — reconstructed from instructions at the sink, over one wai.det.fixed64 fixed-point floor, each with its own deterministic *-equivalence conformance criterion:

Capability	What it carries	Conformance
wai.world.{scene,replay,live}	interactive worlds (signed op log)	replay-equivalence
wai.world.model	action-conditioned world model (signed action ops → next frame, integer transition + synthesis)	replay-equivalence
wai.mesh.glb / wai.splat.spz	3D mesh / Gaussian-splat assets	content-addressed
wai.audio.scene	object-based spatial audio	mixdown-equivalence
wai.audio.score	music as note events (synthesized at the sink)	mixdown-equivalence
wai.haptic.signal	keyframed touch envelopes	sample-equivalence
wai.feed.state	scoreboards / telemetry / market feeds	replay-equivalence
wai.avatar.pose	digital humans (skeletal pose stream)	pose-equivalence
wai.splat.4d	volumetric / 4D temporal Gaussian splats	frame-state-equivalence
wai.film.linear	animation / CGI film (sealed world + camera track)	film-equivalence
wai.film.reel	a film with cuts (one world, a sequence of shots)	reel-equivalence

And the verify half — a signed, energy-accounted lineage for opaque captured content WAI does not reconstruct (wai.asset.provenance). See SPEC.md §5 and extensions/ for each.

Quantum — deterministic, native and in cloud WASM

WAI carries a full quantum stack in pure Rust: circuit simulation and the whole operational layer around it, every stage byte-exact and sealable into a signed, energy-accounted receipt. No QPU and no cloud service. Because it is pure Rust with no host bindings, one source builds for native and wasm32-wasip2 and produces byte-identical results — the same circuit reconstructs to the same statevector hash on a laptop or at the edge.

Only wai.quantum.circuit is a registered payload capability, and wai.quantum.calibration and wai.quantum.job are registered records; the other names below are module labels (REGISTRY.md lists the receipt labels no revision registers).

Module	What it does	Conformance
wai.quantum.circuit	byte-exact statevector simulation (dyadic Clifford+T+P(k))	statevector-equivalence
wai.quantum.toolchain	four backends (statevector / stabilizer / tensor-network MPS / sparse-Pauli), algorithm library, OpenQASM 3.0 in-and-out, QIR export + ingest, Bloch + entanglement analysis	statevector-equivalence
wai.quantum.compile	Clifford routing + stabilizer-tableau equivalence proof	signed equivalence certificate
wai.quantum.cal.*	calibration, filter-function robust control, DD noise spectroscopy	reproducible + receipt
wai.quantum.cal.noise_learn	Cycle Benchmarking / CER sparse Pauli–Lindblad learning	signed noise model
wai.quantum.mitigate	ZNE + readout M3 + classical shadows	byte-exact post-processing
wai.quantum.decode	CSS codes (rotated surface, toric, bivariate-bicycle) + union-find and Relay-BP decoders	signed decode + joules
wai.quantum.rearrange	neutral-atom register prep (Hungarian / LSAP)	signed defect-free plan
wai.quantum.phasor	interference substrate + energy-accounted phasor receipts	reproducible f64
wai.quantum.kernel	the quantum kernel deployed as phasor features	reproducible f64
wai.quantum.qdata	learning-from-quantum-data sample-complexity separation	reproducible f64
wai.quantum.qbom	Quantum Bill of Materials — stage receipts into one provenance manifest	signed manifest

Quickstart

# native
cargo build --release --no-default-features \
  --features "quantum_toolchain,quantum_receipt" --bin wai-quantum

./target/release/wai-quantum list                  # algorithm catalog
./target/release/wai-quantum run ghz -n 5          # simulate + hash + outcomes
./target/release/wai-quantum run grover -n 4 -p 9  # amplitude amplification
./target/release/wai-quantum qasm bell             # emit OpenQASM 3.0
./target/release/wai-quantum qir  bell             # emit QIR (LLVM IR)
./target/release/wai-quantum qir  qft -n 4 | \\
  ./target/release/wai-quantum run --qir           # and read it back
./target/release/wai-quantum receipt ghz -n 3 | ./target/release/wai-quantum verify

Same source, at the edge

cargo build --release --target wasm32-wasip2 --no-default-features \
  --features "quantum_toolchain,quantum_receipt" --bin wai-quantum

wasmtime run target/wasm32-wasip2/release/wai-quantum.wasm run ghz -n 5

Both print the identical state-hash. A receipt sealed in cloud WASM verifies natively and vice-versa, and the receipt bytes themselves are identical — the signature travels because the computation is reproducible, not because the two runtimes agreed to trust each other.

As a serverless function

wai-quantum-serve/ exports the same stack as a WASI-HTTP component (wasi:http/proxy), so quantum computation runs as an ordinary HTTP handler on a component-model host:

cd wai-quantum-serve
cargo build --release --target wasm32-wasip2
wasmtime serve -S cli=y --addr 127.0.0.1:8231 \
  target/wasm32-wasip2/release/wai_quantum_serve.wasm

GET  /health                      GET  /algorithms
GET  /run?algo=ghz&n=5&top=8      POST /run       (body = OpenQASM 3.0)
GET  /qasm?algo=bell              GET  /receipt?algo=ghz&n=3[&shots=&seed=]
POST /verify                      (body = receipt JSON; 200 valid / 409 invalid / 400 unparseable)

The endpoint returns the same statevector_hash as the native CLI for the same circuit, and a receipt sealed by the HTTP handler verifies with the native wai-quantum verify — the JSON content is byte-identical (the CLI adds a trailing newline; nothing else differs). Both verify paths accept a receipt in either form: as sealed, or labelled with its figure's acquisition class, as wai_quantum_meter emits a measured figure (quantum_energy::MaybeLabelled, energy-measurement §2.8). Both restate the class, or say the figure has none.

That wasm32-wasip2 build needs -S cli=y, because Rust's std imports wasi:cli/environment at startup and the bare wasi:http/proxy world does not provide it. For a component that runs on any wasi:http host with no host flags at all, build the pure proxy-world artifact instead:

./build-proxy.sh
wasmtime serve --addr 127.0.0.1:8080 target/wai_quantum_serve.proxy.wasm

That path compiles a wasm32-wasip1 core module and adapts it with the Bytecode Alliance proxy adapter, leaving only imports the proxy world defines (wasi:http/types, wasi:io/*, wasi:cli/std{in,out,err}, wasi:random). The script fails the build if wasi:cli/environment ever reappears, and the served results are byte-identical either way.

Explore the whole stack in the browser (it is the same Rust, compiled to wasm): the toolchain IDE, the ML layer, and a working course.

License

Apache-2.0 — see LICENSE. Implementing a registered capability carries that capability's own licence class (royalty-free, patent-pool, proprietary or non-commercial), recorded in the registry. Companion implementations outside this tree keep their own licences.