open-standards/wai/README.md

WAI — Web AI Media Transport & Execution

A container + capability-dispatch standard for media. A standard codec (AVIF, JPEG XL, PNG, JPEG, Opus, FLAC, AV1, zstd, XZ and the others the registry names) is dispatched to the library a sink already has; WAI also defines its own capability classes — integer codecs, worlds, scores, films — each with a conformance criterion, and an envelope that lets the neural-shared-prior path coexist with the model-free floor.

Two paths open every conforming file:

• Neural condition — the sink regenerates the media against a shared ambient prior (a model). The model is a requirement of existence, never shipped (an .mp3 doesn't ship its decoder either).
• Zeroth condition — a menu of registered standard codecs, decoded with the libraries a device already ships.

The capability a file requires is named, not supplied.

Building

wai-rs builds the codec chain by default (the codecs feature), and those crates link system libraries rather than vendoring them — consistent with "WAI does not re-implement codecs". Without them the build stops in a codec crate's build script, not in WAI's own code:

# macOS
brew install jpeg-xl libavif opus dav1d xz

# Debian / Ubuntu
apt-get install libjxl-dev libavif-dev libopus-dev libdav1d-dev liblzma-dev

To build without the C codec chain — the envelope, manifest and world engine do not need it, which is also how the wasm32 target is built:

cargo check -p wai --no-default-features

What's here

Path	Contents
SPEC.md	WAI v1.0 spec — envelope, manifest, capability menu
wai-rs/	Rust reference implementation (lib + cdylib + staticlib + C FFI)
REGISTRY.md, registry/capabilities.json	The registry: every registered string with its kind, status, classes and edition, generated from the reference implementation (wai registry export), with its JSON Schema and the policy for registering a string
vocab/	RDF vocabulary: v2 generated from the registry, v1 frozen
extensions/	Sub-specifications (confidentiality, receipts, worlds, film, quantum operations, integer operators, …)
research/	Research notes and dated records: graded, not normative
glossary/	Reference catalog of media codecs surveyed for capability registration. codecs.json is the dataset, linked to the registry; SCHEMA.md defines the shape and contribution rules. Rendered at wai.transaction.science/glossary.
CONTRIBUTING.md	How to change the specification, register a capability and add tests CI runs

Quickstart

cd wai-rs
cargo test --no-default-features --lib --test carriage   # registry, envelope, every capability round-trips
cargo run --no-default-features --bin wai -- capabilities  # the registry, one row per capability
cargo build --release --lib    # with the codecs feature: libwai.dylib/.so/.dll + libwai.a

On macOS, libflac lives under the Homebrew "flac" formula prefix that Cargo doesn't probe automatically. The committed .cargo/config.toml adds the path; if you build outside the repo set RUSTFLAGS="-L /opt/homebrew/opt/flac/lib".

SDK consumption

wai-rs exposes a stable C ABI in src/ffi.rs — the cdylib drops in as libwai.dylib/.so/.dll. Bindings for any language (Python ctypes, Node N-API, Swift, JNI, Go cgo, etc.) call into:

• wai_image_{png,jpeg,avif,jxl}_{encode,decode}
• wai_audio_{opus,flac}_{encode,decode}
• wai_text_{zstd,xz}_{encode,decode}
• wai_envelope_{pack,unpack} — wrap/unwrap the WAI container
• wai_buffer_free, wai_last_error — memory + error handling

Registered capabilities (the zeroth menu)

Capability	Codec	Lossless?	Mandatory?
wai.image.png	PNG	yes	mandatory floor
wai.image.jxl	JPEG-XL	both	recommended
wai.image.avif	AVIF (AV1-based)	lossy	recommended
wai.image.jpeg	JPEG	lossy	recommended (legacy compat)
wai.audio.flac	FLAC	yes	mandatory floor
wai.audio.opus	Opus	lossy	recommended
wai.video.av1	AV1	lossy	recommended
wai.video.av1.lossless	AV1 (quantizer=0)	YUV-lossless	recommended
wai.text.zstd	zstd	yes	mandatory floor
wai.text.xz	XZ/LZMA2	yes	recommended

Neural capabilities (wai.neural.*) are OPTIONAL and sink-advertised. Most run float inference, so their decode is exact only on the build that produced it. A distinguished subset is integer-exact by construction — an all-i64 decode with no float and no ONNX — so the reconstructed artifact is byte-identical on every machine and its hash is portable. Conformance is a hash: a deployer can require intent=replicate to dispatch only to these.

Capability	Integer-exact decode of	Conformance
wai.neural.int_hyper	image — integer hyperprior σ source + integer entropy + integer synthesis	byte-exact (hash-portable)
wai.neural.int_mlicpp	image — MLIC++ entropy topology (channel slices × checkerboard passes, an entropy network re-run before each pass) on integer networks	byte-exact (hash-portable)
wai.neural.int_synth	image — integer synthesis only	byte-exact (hash-portable)
wai.video.int_hyper	video — int_hyper I-frame + integer half-pel motion-compensated P-frames	byte-exact (hash-portable)
wai.audio.int_codec	audio — learned 1-D transform, integer synthesis → int16 PCM	byte-exact (hash-portable)
wai.splat.int_codec	3D — learned Gaussian-splat-attribute transform, integer synthesis	byte-exact (hash-portable)

Their payloads are binary formats SPEC §5 pins — WIV1 for the integer video codecs, WIH1, WIA1 and WIS1 (extensions/integer-payloads.md). The JSON the encode and export tools write is converted to them, never carried. The same extension writes down the wai.neural.int_synth payload and the WIM1 payload of wai.neural.int_mlicpp. Their networks' integer arithmetic is written down as the operator set wai-ops/1 (extensions/integer-operators.md, Draft).

Interactive & temporal content (extensions)

Beyond the codec menu, WAI carries content that consumes input or moves over time — reconstructed from instructions at the sink, over one wai.det.fixed64 fixed-point floor, each with its own deterministic *-equivalence conformance criterion:

Capability	What it carries	Conformance
wai.world.{scene,replay,live}	interactive worlds (signed op log)	replay-equivalence
wai.world.model	action-conditioned world model (signed action ops → next frame, integer transition + synthesis)	replay-equivalence
wai.mesh.glb / wai.splat.spz	3D mesh / Gaussian-splat assets	content-addressed
wai.audio.scene	object-based spatial audio	mixdown-equivalence
wai.audio.score	music as note events (synthesized at the sink)	mixdown-equivalence
wai.haptic.signal	keyframed touch envelopes	sample-equivalence
wai.feed.state	scoreboards / telemetry / market feeds	replay-equivalence
wai.avatar.pose	digital humans (skeletal pose stream)	pose-equivalence
wai.splat.4d	volumetric / 4D temporal Gaussian splats	frame-state-equivalence
wai.film.linear	animation / CGI film (sealed world + camera track)	film-equivalence
wai.film.reel	a film with cuts (one world, a sequence of shots)	reel-equivalence

And the verify half — a signed, energy-accounted lineage for opaque captured content WAI does not reconstruct (wai.asset.provenance). See SPEC.md §5 and extensions/ for each.

Quantum — deterministic, native and in cloud WASM

WAI carries a full quantum stack in pure Rust: circuit simulation and the whole operational layer around it, every stage byte-exact and sealable into a signed, energy-accounted receipt. No QPU and no cloud service. Because it is pure Rust with no host bindings, one source builds for native and wasm32-wasip2 and produces byte-identical results — the same circuit reconstructs to the same statevector hash on a laptop or at the edge.

wai.quantum.circuit2 and wai.quantum.circuit are the registered payload capabilities, and wai.quantum.calibration and wai.quantum.job are registered records; the other names below are module labels (REGISTRY.md lists the receipt labels no revision registers). The two circuit capabilities share the container and the gates and differ in the phase table: circuit2's holds the integers nearest to each phase and is the default for new content, and wai.quantum.circuit is frozen so that every file, hash and receipt made with it reproduces (quantum-sim §8).

Module	What it does	Conformance
wai.quantum.circuit2	byte-exact statevector simulation (dyadic Clifford+T+P(k)), with the correctly rounded phase table	statevector-equivalence
wai.quantum.circuit	the same, with the frozen phase table existing content was made with	statevector-equivalence
wai.quantum.toolchain	four backends (statevector / stabilizer / tensor-network MPS / sparse-Pauli), algorithm library, OpenQASM 3.0 in-and-out, QIR export + ingest, Bloch + entanglement analysis	statevector-equivalence
wai.quantum.compile	Clifford routing + stabilizer-tableau equivalence proof	signed equivalence certificate
wai.quantum.cal.*	calibration, filter-function robust control, DD noise spectroscopy	reproducible + receipt
wai.quantum.cal.noise_learn	Cycle Benchmarking / CER sparse Pauli–Lindblad learning	signed noise model
wai.quantum.mitigate	ZNE + readout M3 + classical shadows	byte-exact post-processing
wai.quantum.decode	CSS codes (rotated surface, toric, bivariate-bicycle) + union-find and Relay-BP decoders	signed decode + joules
wai.quantum.rearrange	neutral-atom register prep (Hungarian / LSAP)	signed defect-free plan
wai.quantum.phasor	interference substrate + energy-accounted phasor receipts	reproducible f64
wai.quantum.kernel	the quantum kernel deployed as phasor features	reproducible f64
wai.quantum.qdata	learning-from-quantum-data sample-complexity separation	reproducible f64
wai.quantum.qbom	Quantum Bill of Materials — stage receipts into one provenance manifest	signed manifest

Quickstart

# native
cargo build --release --no-default-features \
  --features "quantum_toolchain,quantum_receipt" --bin wai-quantum

./target/release/wai-quantum list                  # algorithm catalog
./target/release/wai-quantum run ghz -n 5          # simulate + hash + outcomes
./target/release/wai-quantum run grover -n 4 -p 9  # amplitude amplification
./target/release/wai-quantum qasm bell             # emit OpenQASM 3.0
./target/release/wai-quantum qir  bell             # emit QIR (LLVM IR)
./target/release/wai-quantum qir  qft -n 4 | \\
  ./target/release/wai-quantum run --qir           # and read it back
./target/release/wai-quantum receipt ghz -n 3 | ./target/release/wai-quantum verify
./target/release/wai-quantum run qft -n 5 --gateset v1   # the frozen wai.quantum.circuit

Circuits are wai.quantum.circuit2 unless --gateset v1 asks for wai.quantum.circuit, which reproduces the circuit and statevector hashes earlier releases gave.

Same source, at the edge

cargo build --release --target wasm32-wasip2 --no-default-features \
  --features "quantum_toolchain,quantum_receipt" --bin wai-quantum

wasmtime run target/wasm32-wasip2/release/wai-quantum.wasm run ghz -n 5

Both print the identical state-hash. A receipt sealed in cloud WASM verifies natively and vice-versa, and the receipt bytes themselves are identical — the signature travels because the computation is reproducible, not because the two runtimes agreed to trust each other.

As a serverless function

wai-quantum-serve/ exports the same stack as a WASI-HTTP component (wasi:http/proxy), so quantum computation runs as an ordinary HTTP handler on a component-model host:

cd wai-quantum-serve
cargo build --release --target wasm32-wasip2
wasmtime serve -S cli=y --addr 127.0.0.1:8231 \
  target/wasm32-wasip2/release/wai_quantum_serve.wasm

GET  /health                      GET  /algorithms
GET  /run?algo=ghz&n=5&top=8      POST /run       (body = OpenQASM 3.0)
GET  /qasm?algo=bell              GET  /receipt?algo=ghz&n=3[&shots=&seed=]
                                  (/run, /qasm and /receipt also take &gateset=v1|v2)
POST /verify                      (body = receipt JSON; 200 valid / 409 invalid / 400 unparseable)

The endpoint returns the same statevector_hash as the native CLI for the same circuit, and a receipt sealed by the HTTP handler verifies with the native wai-quantum verify — the JSON content is byte-identical (the CLI adds a trailing newline; nothing else differs). Both verify paths accept a receipt in either form: as sealed, or labelled with its figure's acquisition class, as wai_quantum_meter emits a measured figure (quantum_energy::MaybeLabelled, energy-measurement §2.8). Both restate the class, or say the figure has none.

That wasm32-wasip2 build needs -S cli=y, because Rust's std imports wasi:cli/environment at startup and the bare wasi:http/proxy world does not provide it. For a component that runs on any wasi:http host with no host flags at all, build the pure proxy-world artifact instead:

./build-proxy.sh
wasmtime serve --addr 127.0.0.1:8080 target/wai_quantum_serve.proxy.wasm

That path compiles a wasm32-wasip1 core module and adapts it with the Bytecode Alliance proxy adapter, leaving only imports the proxy world defines (wasi:http/types, wasi:io/*, wasi:cli/std{in,out,err}, wasi:random). The script fails the build if wasi:cli/environment ever reappears, and the served results are byte-identical either way.

Explore the whole stack in the browser (it is the same Rust, compiled to wasm): the toolchain IDE, the ML layer, and a working course.

License

Apache-2.0 — see LICENSE. Implementing a registered capability carries that capability's own licence class (royalty-free, patent-pool, proprietary or non-commercial), recorded in the registry. Companion implementations outside this tree keep their own licences.