open-standards/wai

.WAI™ — Web AI Media Transport & Execution

A container + capability-dispatch standard for media. A standard codec (AVIF, JPEG XL, PNG, JPEG, Opus, FLAC, AV1, zstd, XZ and the others the registry names) is dispatched to the library a sink already has; WAI also defines its own capability classes — integer codecs, worlds, scores, films — each with a conformance criterion, and an envelope that lets the neural-shared-prior path coexist with the model-free floor.

Two paths open every conforming file:

• Neural condition — the sink regenerates the media against a shared ambient prior (a model). The model is a requirement of existence, never shipped (an .mp3 doesn't ship its decoder either).
• Zeroth condition — a menu of registered standard codecs, decoded with the libraries a device already ships.

The capability a file requires is named, not supplied.

Building

wai-rs builds the codec chain by default (the codecs feature), and those crates link system libraries rather than vendoring them — consistent with "WAI does not re-implement codecs". Without them the build stops in a codec crate's build script, not in WAI's own code:

# macOS
brew install jpeg-xl libavif opus dav1d xz

# Debian / Ubuntu
apt-get install libjxl-dev libavif-dev libopus-dev libdav1d-dev liblzma-dev

To build without the C codec chain — the envelope, manifest and world engine do not need it, which is also how the wasm32 target is built:

cargo check -p wai --no-default-features

With the feature floor alone, wai-rs decodes the mandatory floor (PNG, FLAC, zstd) with no system library: PNG and FLAC in pure Rust, zstd from its reference source compiled in. The quickstart feature adds wai hello and wai verify; it is how the prebuilt wai is built:

cargo install --locked --path wai-rs --no-default-features --features quickstart --bin wai

What's here

Path	Contents
SPEC.md	WAI spec — envelope, manifest, capability menu
wai-rs/	Rust reference implementation (lib + cdylib + staticlib + C FFI)
examples/	Small runnable projects, one per template of wai create: a video room, watching together, messaging and a call, an encoder into a room, packaging, receipts in Rust and Python, native sinks on Apple and Android platforms, remote desktop
quickstart/	The prebuilt command line's archive: what it holds, how it is built and released, and its installer
schema/	The JSON Schemas of wai.json, wai.lock and the build record, and what each member pins
project-conformance/	wai.lock's bytes, and a second reader that recomputes every digest with the standard library alone
catalogue/	The operation catalogue as JSON (wai ops export): every operation of the toolchain, noun.verb, and the name each door gives it
REGISTRY.md, registry/capabilities.json	The registry: every registered string with its kind, status, classes and edition, generated from the reference implementation (wai registry export), with its JSON Schema and the policy for registering a string
vocab/	RDF vocabulary: v2 generated from the registry, v1 frozen
extensions/	Sub-specifications (confidentiality, receipts, worlds, film, quantum operations, integer operators, vector graphics, …)
research/	Research notes and dated records: graded, not normative
glossary/	Reference catalog of media codecs surveyed for capability registration. codecs.json is the dataset, linked to the registry; SCHEMA.md defines the shape and contribution rules. Rendered at wai.transaction.science/glossary.
CONTRIBUTING.md	How to change the specification, register a capability and add tests CI runs
SECURITY.md	How to report a vulnerability, and where the reference implementation reads untrusted input: the manifest limits, the native codec libraries, module authorisation, the fingerprinting risk of advertised parameter sets

Quickstart

The command line, with the relay and the room service it runs locally: the prebuilt archive (quickstart/, built by quickstart/build.sh), or from source.

wai hello                                   # name, dispatch, rebuild and receipt, on a sample
wai create my-room --template video-room    # a project from one of the examples (examples/)
cd my-room && wai dev                       # the relay, the room service and your page, on 127.0.0.1:8080
wai verify ../wai-hello                     # the receipts hello wrote, checked offline

examples/ holds each template, with a test that runs it.

Every command is an operation of one catalogue, named noun.verb (envelope.inspect, project.create): wai ops list lists them, wai ops search "<task>" finds one offline, and wai ops export writes the catalogue as JSON (catalogue/ops.json). Every command takes --json (one JSON envelope on stdout), --dry-run where it would write, and --yes where a change waits for confirmation (--confirm, deprecated, does the same), and exits with one of eight codes (wai --help lists them). wai sends nothing off your machine unless you name a remote service.

The library's own checks, from a clone:

cd wai-rs
cargo test --no-default-features --lib --test carriage   # registry, envelope, every capability round-trips
cargo run --no-default-features --bin wai -- capabilities  # the registry, one row per capability
cargo build --release --lib    # with the codecs feature: libwai.dylib/.so/.dll + libwai.a

Two commands hold what is published to what was published before, and other implementations to WAI's corpora:

wai breaking --against origin/main     # what this tree breaks: wire, record and API (CI runs it on every pull request)
wai conform verifier --suite claim --impl "my-verifier verdicts {corpus}"   # a corpus run against your implementation
wai verify claim.wai-conform.json      # the signed result, checked

A project is pinned, checked and built by three more, each in its directory:

wai lock     # wai.lock: the toolchain, the template, the registry, parameter sets, the policy and corpora, by digest
wai check    # the project validated without building it: every finding, and what could not run says not checked
wai build    # dist/, the same bytes from the same project and lock, with a record naming what it came from

A break passes only when the changelog names it: an entry this change adds, as a file of changes/ or to CHANGELOG.md, that says Breaking and gives the finding's id in backticks. wai conform --list names every kind (verifier, decoder, packager, origin, sink) and suite, with its adapters: a command line, or an http:// URL. wai verify --conformant admits a result only for a run that passed, against the corpus of the checkout it runs in.

With the codecs feature, wai-rs/build.rs finds libFLAC through pkg-config and, on macOS without pkg-config, in the Homebrew "flac" formula prefix (/opt/homebrew/opt/flac/lib or /usr/local/opt/flac/lib). Where neither finds it, set RUSTFLAGS="-L <the directory holding libFLAC>".

For coding agents

wai setup-agent makes a project ready for a coding agent to use WAI. In the project directory it writes AGENTS.md, the instructions agents read, adds an import of it to a CLAUDE.md the project already has, and registers wai mcp in the project's MCP configuration: .mcp.json, and the project-level MCP file of each agent whose directory the project already holds (--all writes every one it knows, --dry-run lists them, --toolset core,dev registers the server with those toolsets). A file that does not exist is created; one that exists is changed only after its diff is shown and the change confirmed (asked on a terminal, or --yes); without either, nothing that exists changes and the command exits 4 with each proposed change and the command that confirms. It writes nothing outside the project directory. With --json its output is the toolchain's JSON envelope.

wai mcp is a Model Context Protocol server on standard input and output. Each tool is an operation of the toolchain, named by it with _ for the dot, and the tools come in toolsets:

Toolset	Tools
core (the default)	envelope_inspect, media_inspect, registry_list, registry_show, receipt_verify, envelope_wrap, conformance_list, conformance_run, docs_search
dev	dev_start, dev_stop, room_grant, room_join, room_leave, session_show
app (a build with the feature app)	app_guide, app_check, app_stream, app_export: Create's, below

receipt_verify verifies a delivery receipt, a claim, a live-room session or a stream audit bundle, and answers with the verifier's verdict and reason code. The dev tools, marked development only, act on the target local alone: they start a relay and a room service on loopback (wai-room dev), mint grants for test principals, join and publish as them, and read the session for verification; the grant tool refuses any service that is not on a loopback address. --read-only offers only the tools that read or run a verifier; the packaging tool writes only inside the project directory. No tool sends anything off the machine.

cd wai-rs
cargo install --path . --no-default-features --features mcp   # `wai` with `wai mcp`, no codec libraries
cd /path/to/your/project && wai setup-agent --toolset core,dev

The quickstart's wai holds wai mcp too.

Create (wai-app) builds a WAI app from a description: a model writes it in joule-ui's line language and the page renders it as it streams. Its program, app.oui, becomes a project with wai create my-app --from app.oui (a build with the feature app): an ordinary WAI project that wai app check checks and wai dev runs.

The room service's HTTP API is described, OpenAPI 3.1, in joule-moq-room/openapi.json, each operation's operationId its name in the toolchain's catalogue.

SDK consumption

wai-rs exposes a stable C ABI in src/ffi.rs — the cdylib drops in as libwai.dylib/.so/.dll. Bindings for any language (Python ctypes, Node N-API, Swift, JNI, Go cgo, etc.) call into:

• wai_image_{png,jpeg,avif,jxl}_{encode,decode}
• wai_audio_{opus,flac}_{encode,decode}
• wai_text_{zstd,xz}_{encode,decode}
• wai_envelope_{pack,unpack} — wrap/unwrap the WAI container
• wai_buffer_free, wai_last_error — memory + error handling

In the browser, on older engines: the reach builds

wai-web is the same Rust compiled to wasm32 for a page. Its shipped package needs WebAssembly with sign-extension, non-trapping float-to-int and bulk memory, and i64 values crossing the JS boundary as BigInt. The reach builds (wai-web/reach/) run that code where those are missing, and give the same outputs byte for byte:

rung	build	for engines with
shipped	the package	every feature above
L1	the shipped binary, legalised at the JS interface	the same WebAssembly, no BigInt at the boundary
L2	MVP-lowered and legalised	MVP WebAssembly
J	the wasm2js translation, ES5	ES5 and typed arrays, no WebAssembly

L2 and J also come in reduced profiles: receipt, verify and verify_decode. wai-load.js, an ES5 loader, picks the rung from feature probes alone and says why. A rung can be forced for testing. reach/build.sh makes every build with pinned tools and records each output's SHA-256 in reach/manifest.json.

CI runs each rung on a Node image of the engine line it serves, and every output must equal the shipped build's; the reach builds made from the package of record (kept in reach/record/) must equal reach/manifest.json. The images stand in for device classes; no real old device has been run.

Registered capabilities (the zeroth menu)

Capability	Codec	Lossless?	Mandatory?
wai.image.png	PNG	yes	mandatory floor
wai.image.jxl	JPEG-XL	both	recommended
wai.image.avif	AVIF (AV1-based)	lossy	recommended
wai.image.jpeg	JPEG	lossy	recommended (legacy compat)
wai.audio.flac	FLAC	yes	mandatory floor
wai.audio.opus	Opus	lossy	recommended
wai.video.av1	AV1	lossy	recommended
wai.video.av1.lossless	AV1 (quantizer=0)	YUV-lossless	recommended
wai.text.zstd	zstd	yes	mandatory floor
wai.text.xz	XZ/LZMA2	yes	recommended

Neural capabilities (wai.neural.*) are OPTIONAL and sink-advertised. Most run float inference, so their decode is exact only on the build that produced it. A distinguished subset is integer-exact by construction — an all-i64 decode with no float and no ONNX — so the reconstructed artifact is byte-identical on every machine and its hash is portable. Conformance is a hash: a deployer can require intent=replicate to dispatch only to these.

Capability	Integer-exact decode of	Conformance
wai.neural.int_hyper	image — integer hyperprior σ source + integer entropy + integer synthesis	byte-exact (hash-portable)
wai.neural.int_mlicpp	image — MLIC++ entropy topology (channel slices × checkerboard passes, an entropy network re-run before each pass) on integer networks	byte-exact (hash-portable)
wai.neural.int_synth	image — integer synthesis only	byte-exact (hash-portable)
wai.video.int_hyper	video — int_hyper I-frame + integer half-pel motion-compensated P-frames	byte-exact (hash-portable)
wai.audio.int_codec	audio — learned 1-D transform, integer synthesis → int16 PCM	byte-exact (hash-portable)
wai.splat.int_codec	3D — learned Gaussian-splat-attribute transform, integer synthesis	byte-exact (hash-portable)

Their payloads are binary formats SPEC §5 pins — WIV1 for the integer video codecs, WIH1, WIA1 and WIS1 (extensions/integer-payloads.md). The JSON the encode and export tools write is converted to them, never carried. The same extension writes down the wai.neural.int_synth payload and the WIM1 payload of wai.neural.int_mlicpp. Their networks' integer arithmetic is written down as the operator set wai-ops/1 (extensions/integer-operators.md, Draft).

Interactive & temporal content (extensions)

Beyond the codec menu, WAI carries content that consumes input or moves over time — reconstructed from instructions at the sink, over one wai.det.fixed64 fixed-point floor, each with its own deterministic *-equivalence conformance criterion:

Capability	What it carries	Conformance
wai.world.{scene,replay,live}	interactive worlds (signed op log)	replay-equivalence
wai.world.model	action-conditioned world model (signed action ops → next frame, integer transition + synthesis)	replay-equivalence
wai.mesh.glb / wai.splat.spz	3D mesh / Gaussian-splat assets	content-addressed
wai.audio.scene	object-based spatial audio	mixdown-equivalence
wai.audio.score	music as note events (synthesized at the sink)	mixdown-equivalence
wai.haptic.signal	keyframed touch envelopes	sample-equivalence
wai.feed.state	scoreboards / telemetry / market feeds	replay-equivalence
wai.avatar.pose	digital humans (skeletal pose stream)	pose-equivalence
wai.splat.4d	volumetric / 4D temporal Gaussian splats	frame-state-equivalence
wai.film.linear	animation / CGI film (sealed world + camera track)	film-equivalence
wai.film.reel	a film with cuts (one world, a sequence of shots)	reel-equivalence

And the verify half — a signed, energy-accounted lineage for opaque captured content WAI does not reconstruct (wai.asset.provenance). See SPEC.md §5 and extensions/ for each.

Quantum — deterministic, native and in cloud WASM

WAI carries a full quantum stack in pure Rust: circuit simulation and the whole operational layer around it, every stage byte-exact and sealable into a signed, energy-accounted receipt. No QPU and no cloud service. Because it is pure Rust with no host bindings, one source builds for native and wasm32-wasip2 and produces byte-identical results — the same circuit reconstructs to the same statevector hash on a laptop or at the edge.

wai.quantum.circuit2 and wai.quantum.circuit are the registered payload capabilities, and wai.quantum.calibration and wai.quantum.job are registered records; the other names below are module labels, which no revision registers (REGISTRY.md lists them with every other wai.quantum.* label, and says what each names). The two circuit capabilities share the container and the gates and differ in the phase table: circuit2's holds the integers nearest to each phase and is the default for new content, and wai.quantum.circuit is frozen so that every file, hash and receipt made with it reproduces (quantum-sim §8).

Module	What it does	Conformance
wai.quantum.circuit2	byte-exact statevector simulation (dyadic Clifford+T+P(k)), with the correctly rounded phase table	statevector-equivalence
wai.quantum.circuit	the same, with the frozen phase table existing content was made with	statevector-equivalence
wai.quantum.toolchain	four backends (statevector / stabilizer / tensor-network MPS / sparse-Pauli), algorithm library, OpenQASM 3.0 in-and-out, QIR export + ingest, Bloch + entanglement analysis	statevector-equivalence
wai.quantum.compile	Clifford routing + stabilizer-tableau equivalence proof	signed equivalence certificate
wai.quantum.cal.*	calibration, filter-function robust control, DD noise spectroscopy	reproducible + receipt
wai.quantum.cal.noise_learn	Cycle Benchmarking / CER sparse Pauli–Lindblad learning	signed noise model
wai.quantum.mitigate	ZNE + readout M3 + classical shadows	byte-exact post-processing
wai.quantum.decode	CSS codes (rotated surface, toric, bivariate-bicycle) + union-find and Relay-BP decoders	signed decode + joules
wai.quantum.rearrange	neutral-atom register prep (Hungarian / LSAP)	signed defect-free plan
wai.quantum.phasor	interference substrate + energy-accounted phasor receipts	reproducible f64
wai.quantum.kernel	the quantum kernel deployed as phasor features	reproducible f64
wai.quantum.qdata	learning-from-quantum-data sample-complexity separation	reproducible f64
wai.quantum.qbom	Quantum Bill of Materials — stage receipts into one provenance manifest	signed manifest

Quickstart

The command line is wai-quantum-cli: OpenQASM programs run here as the quantum job service runs them, each result naming its method and accuracy and signed with this user's key.

cd wai-quantum-cli && cargo build --release      # or: cargo install wai-quantum-cli
W=./target/release/wai-quantum

$W run bell.qasm --shots 1000 -o "Z0 Z1"          # planned as the service plans it
$W run bell.qasm --json > job.json && $W verify job.json
$W plan prog.qasm; $W lint prog.qasm; $W draw prog.qasm
$W submit prog.qasm                               # on the job service
$W lab list                                       # the algorithm catalogue
$W lab run ghz -n 5                               # simulate + hash + outcomes
$W lab qasm bell                                  # emit OpenQASM 3.0
$W lab qir qft -n 4 | $W lab run --qir            # QIR out, and back in
$W lab receipt ghz -n 3 | $W verify
$W lab run qft -n 5 --gateset v1                  # the frozen wai.quantum.circuit

Circuits built by lab are wai.quantum.circuit2 unless --gateset v1 asks for wai.quantum.circuit, which reproduces the circuit and statevector hashes earlier releases gave.

Same source, at the edge

cd wai-quantum-cli && cargo build --release --target wasm32-wasip2 --no-default-features

wasmtime run -W nan-canonicalization=y -W relaxed-simd-deterministic=y target/wasm32-wasip2/release/wai-quantum.wasm lab run ghz -n 5
wasmtime run -W nan-canonicalization=y -W relaxed-simd-deterministic=y target/wasm32-wasip2/release/wai-quantum.wasm lab receipt ghz -n 3 --demo-key

Both print the identical state-hash. A receipt sealed in cloud WASM verifies natively and vice-versa, and with the same key (here the published demo key) the receipt bytes themselves are identical: the signature travels because the computation is reproducible, not because the two runtimes agreed to trust each other.

As a serverless function

wai-quantum-serve/ exports the same stack as a WASI-HTTP component (wasi:http/proxy), so quantum computation runs as an ordinary HTTP handler on a component-model host:

cd wai-quantum-serve
cargo build --release --target wasm32-wasip2
wasmtime serve -S cli=y --addr 127.0.0.1:8231 \
  target/wasm32-wasip2/release/wai_quantum_serve.wasm

GET  /health                      GET  /algorithms
GET  /run?algo=ghz&n=5&top=8      POST /run       (body = OpenQASM 3.0)
GET  /qasm?algo=bell              GET  /receipt?algo=ghz&n=3[&shots=&seed=]
                                  (/run, /qasm and /receipt also take &gateset=v1|v2)
POST /verify                      (body = receipt JSON; 200 valid / 409 invalid / 400 unparseable)

The endpoint returns the same statevector_hash as the native CLI for the same circuit, and a receipt sealed by the HTTP handler verifies with the native wai-quantum verify — the JSON content is byte-identical (the CLI adds a trailing newline; nothing else differs). Both verify paths accept a receipt in either form: as sealed, or labelled with its figure's acquisition class, as wai_quantum_meter emits a measured figure (quantum_energy::MaybeLabelled, energy-measurement §2.8). Both restate the class, or say the figure has none.

That wasm32-wasip2 build needs -S cli=y, because Rust's std imports wasi:cli/environment at startup and the bare wasi:http/proxy world does not provide it. For a component that runs on any wasi:http host with no host flags at all, build the pure proxy-world artifact instead:

./build-proxy.sh
wasmtime serve --addr 127.0.0.1:8080 target/wai_quantum_serve.proxy.wasm

That path compiles a wasm32-wasip1 core module and adapts it with the Bytecode Alliance proxy adapter, leaving only imports the proxy world defines (wasi:http/types, wasi:io/*, wasi:cli/std{in,out,err}, wasi:random). The script fails the build if wasi:cli/environment ever reappears, and the served results are byte-identical either way.

Explore the whole stack in the browser (it is the same Rust, compiled to wasm): the toolchain IDE, the ML layer, and a working course.

License

Apache-2.0 — see LICENSE. Implementing a registered capability carries that capability's own licence class (royalty-free, patent-pool, proprietary or non-commercial), recorded in the registry. Companion implementations outside this tree keep their own licences.