open-standards/wai/README.md

WAI — Web AI Media Transport & Execution

A container + capability-dispatch standard for media. A standard codec (AVIF, JPEG XL, PNG, JPEG, Opus, FLAC, AV1, zstd, XZ and the others the registry names) is dispatched to the library a sink already has; WAI also defines its own capability classes — integer codecs, worlds, scores, films — each with a conformance criterion, and an envelope that lets the neural-shared-prior path coexist with the model-free floor.

Two paths open every conforming file:

• Neural condition — the sink regenerates the media against a shared ambient prior (a model). The model is a requirement of existence, never shipped (an .mp3 doesn't ship its decoder either).
• Zeroth condition — a menu of registered standard codecs, decoded with the libraries a device already ships.

The capability a file requires is named, not supplied.

Building

wai-rs builds the codec chain by default (the codecs feature), and those crates link system libraries rather than vendoring them — consistent with "WAI does not re-implement codecs". Without them the build stops in a codec crate's build script, not in WAI's own code:

# macOS
brew install jpeg-xl libavif opus dav1d xz

# Debian / Ubuntu
apt-get install libjxl-dev libavif-dev libopus-dev libdav1d-dev liblzma-dev

To build without the C codec chain — the envelope, manifest and world engine do not need it, which is also how the wasm32 target is built:

cargo check -p wai --no-default-features

With the feature floor alone, wai-rs decodes the mandatory floor (PNG, FLAC, zstd) with no system library: PNG and FLAC in pure Rust, zstd from its reference source compiled in. The quickstart feature adds wai hello and wai verify; it is how the prebuilt wai is built:

cargo install --locked --path wai-rs --no-default-features --features quickstart --bin wai

What's here

Path	Contents
SPEC.md	WAI v1.0 spec — envelope, manifest, capability menu
wai-rs/	Rust reference implementation (lib + cdylib + staticlib + C FFI)
examples/	Small runnable projects, one per template of wai create: a video room, watching together, messaging and a call, an encoder into a room, packaging, receipts in Rust and Python, native sinks on Apple and Android platforms, remote desktop
quickstart/	The prebuilt command line's archive: what it holds, how it is built and released, and its installer
catalogue/	The operation catalogue as JSON (wai ops export): every operation of the toolchain, noun.verb, and the name each door gives it
REGISTRY.md, registry/capabilities.json	The registry: every registered string with its kind, status, classes and edition, generated from the reference implementation (wai registry export), with its JSON Schema and the policy for registering a string
vocab/	RDF vocabulary: v2 generated from the registry, v1 frozen
extensions/	Sub-specifications (confidentiality, receipts, worlds, film, quantum operations, integer operators, vector graphics, …)
research/	Research notes and dated records: graded, not normative
glossary/	Reference catalog of media codecs surveyed for capability registration. codecs.json is the dataset, linked to the registry; SCHEMA.md defines the shape and contribution rules. Rendered at wai.transaction.science/glossary.
CONTRIBUTING.md	How to change the specification, register a capability and add tests CI runs
SECURITY.md	How to report a vulnerability, and where the reference implementation reads untrusted input: the manifest limits, the native codec libraries, module authorisation, the fingerprinting risk of advertised parameter sets

Quickstart

The command line, with the relay and the room service it runs locally: the prebuilt archive (quickstart/, built by quickstart/build.sh), or from source.

wai hello                                   # name, dispatch, rebuild and receipt, on a sample
wai create my-room --template video-room    # a project from one of the examples (examples/)
cd my-room && wai dev                       # the relay, the room service and your page, on 127.0.0.1:8080
wai verify ../wai-hello                     # the receipts hello wrote, checked offline

examples/ holds each template, with a test that runs it.

Every command is an operation of one catalogue, named noun.verb (envelope.inspect, project.create): wai ops list lists them, wai ops search "<task>" finds one offline, and wai ops export writes the catalogue as JSON (catalogue/ops.json). Every command takes --json (one JSON envelope on stdout), --dry-run where it would write, and --yes where a change waits for confirmation (--confirm, deprecated, does the same), and exits with one of eight codes (wai --help lists them). wai sends nothing off your machine unless you name a remote service.

The library's own checks, from a clone:

cd wai-rs
cargo test --no-default-features --lib --test carriage   # registry, envelope, every capability round-trips
cargo run --no-default-features --bin wai -- capabilities  # the registry, one row per capability
cargo build --release --lib    # with the codecs feature: libwai.dylib/.so/.dll + libwai.a

Two commands hold what is published to what was published before, and other implementations to WAI's corpora:

wai breaking --against origin/main     # what this tree breaks: wire, record and API (CI runs it on every pull request)
wai conform verifier --suite claim --impl "my-verifier verdicts {corpus}"   # a corpus run against your implementation
wai verify claim.wai-conform.json      # the signed result, checked

A break passes only when CHANGELOG.md names it: an entry this change adds that says Breaking and gives the finding's id in backticks. wai conform --list names every kind (verifier, decoder, packager, origin) and suite, with its adapters: a command line, or an http:// URL.

With the codecs feature, wai-rs/build.rs finds libFLAC through pkg-config and, on macOS without pkg-config, in the Homebrew "flac" formula prefix (/opt/homebrew/opt/flac/lib or /usr/local/opt/flac/lib). Where neither finds it, set RUSTFLAGS="-L <the directory holding libFLAC>".

For coding agents

wai setup-agent makes a project ready for a coding agent to use WAI. In the project directory it writes AGENTS.md, the instructions agents read, adds an import of it to a CLAUDE.md the project already has, and registers wai mcp in the project's MCP configuration: .mcp.json, and the project-level MCP file of each agent whose directory the project already holds (--all writes every one it knows, --dry-run lists them, --toolset core,dev registers the server with those toolsets). A file that does not exist is created; one that exists is changed only after its diff is shown and the change confirmed (asked on a terminal, or --yes); without either, nothing that exists changes and the command exits 4 with each proposed change and the command that confirms. It writes nothing outside the project directory. With --json its output is the toolchain's JSON envelope.

wai mcp is a Model Context Protocol server on standard input and output. Each tool is an operation of the toolchain, named by it with _ for the dot, and the tools come in toolsets:

Toolset	Tools
core (the default)	envelope_inspect, registry_list, registry_show, receipt_verify, envelope_wrap, conformance_list, conformance_run, docs_search
dev	dev_start, dev_stop, room_grant, room_join, room_leave, session_show
app (a build with the feature app)	app_guide, app_check, app_stream, app_export: Create's, below

receipt_verify verifies a delivery receipt, a claim, a live-room session or a stream audit bundle, and answers with the verifier's verdict and reason code. The dev tools, marked development only, act on the target local alone: they start a relay and a room service on loopback (wai-room dev), mint grants for test principals, join and publish as them, and read the session for verification; the grant tool refuses any service that is not on a loopback address. --read-only offers only the tools that read or run a verifier; the packaging tool writes only inside the project directory. No tool sends anything off the machine.

cd wai-rs
cargo install --path . --no-default-features --features mcp   # `wai` with `wai mcp`, no codec libraries
cd /path/to/your/project && wai setup-agent --toolset core,dev

The quickstart's wai holds wai mcp too.

Create (wai-app) builds a WAI app from a description: a model writes it in joule-ui's line language and the page renders it as it streams. Its program, app.oui, becomes a project with wai create my-app --from app.oui (a build with the feature app): an ordinary WAI project that wai app check checks and wai dev runs.

The room service's HTTP API is described, OpenAPI 3.1, in joule-moq-room/openapi.json, each operation's operationId its name in the toolchain's catalogue.

SDK consumption

wai-rs exposes a stable C ABI in src/ffi.rs — the cdylib drops in as libwai.dylib/.so/.dll. Bindings for any language (Python ctypes, Node N-API, Swift, JNI, Go cgo, etc.) call into:

• wai_image_{png,jpeg,avif,jxl}_{encode,decode}
• wai_audio_{opus,flac}_{encode,decode}
• wai_text_{zstd,xz}_{encode,decode}
• wai_envelope_{pack,unpack} — wrap/unwrap the WAI container
• wai_buffer_free, wai_last_error — memory + error handling

In the browser, on older engines: the reach builds

wai-web is the same Rust compiled to wasm32 for a page. Its shipped package needs WebAssembly with sign-extension, non-trapping float-to-int and bulk memory, and i64 values crossing the JS boundary as BigInt. The reach builds (wai-web/reach/) run that code where those are missing, and give the same outputs byte for byte:

rung	build	for engines with
shipped	the package	every feature above
L1	the shipped binary, legalised at the JS interface	the same WebAssembly, no BigInt at the boundary
L2	MVP-lowered and legalised	MVP WebAssembly
J	the wasm2js translation, ES5	ES5 and typed arrays, no WebAssembly

L2 and J also come in reduced profiles: receipt, verify and verify_decode. wai-load.js, an ES5 loader, picks the rung from feature probes alone and says why. A rung can be forced for testing. reach/build.sh makes every build with pinned tools and records each output's SHA-256 in reach/manifest.json.

CI runs each rung on a Node image of the engine line it serves, and every output must equal the shipped build's; the reach builds made from the package of record (kept in reach/record/) must equal reach/manifest.json. The images stand in for device classes; no real old device has been run.

Registered capabilities (the zeroth menu)

Capability	Codec	Lossless?	Mandatory?
wai.image.png	PNG	yes	mandatory floor
wai.image.jxl	JPEG-XL	both	recommended
wai.image.avif	AVIF (AV1-based)	lossy	recommended
wai.image.jpeg	JPEG	lossy	recommended (legacy compat)
wai.audio.flac	FLAC	yes	mandatory floor
wai.audio.opus	Opus	lossy	recommended
wai.video.av1	AV1	lossy	recommended
wai.video.av1.lossless	AV1 (quantizer=0)	YUV-lossless	recommended
wai.text.zstd	zstd	yes	mandatory floor
wai.text.xz	XZ/LZMA2	yes	recommended

Neural capabilities (wai.neural.*) are OPTIONAL and sink-advertised. Most run float inference, so their decode is exact only on the build that produced it. A distinguished subset is integer-exact by construction — an all-i64 decode with no float and no ONNX — so the reconstructed artifact is byte-identical on every machine and its hash is portable. Conformance is a hash: a deployer can require intent=replicate to dispatch only to these.

Capability	Integer-exact decode of	Conformance
wai.neural.int_hyper	image — integer hyperprior σ source + integer entropy + integer synthesis	byte-exact (hash-portable)
wai.neural.int_mlicpp	image — MLIC++ entropy topology (channel slices × checkerboard passes, an entropy network re-run before each pass) on integer networks	byte-exact (hash-portable)
wai.neural.int_synth	image — integer synthesis only	byte-exact (hash-portable)
wai.video.int_hyper	video — int_hyper I-frame + integer half-pel motion-compensated P-frames	byte-exact (hash-portable)
wai.audio.int_codec	audio — learned 1-D transform, integer synthesis → int16 PCM	byte-exact (hash-portable)
wai.splat.int_codec	3D — learned Gaussian-splat-attribute transform, integer synthesis	byte-exact (hash-portable)

Their payloads are binary formats SPEC §5 pins — WIV1 for the integer video codecs, WIH1, WIA1 and WIS1 (extensions/integer-payloads.md). The JSON the encode and export tools write is converted to them, never carried. The same extension writes down the wai.neural.int_synth payload and the WIM1 payload of wai.neural.int_mlicpp. Their networks' integer arithmetic is written down as the operator set wai-ops/1 (extensions/integer-operators.md, Draft).

Interactive & temporal content (extensions)

Beyond the codec menu, WAI carries content that consumes input or moves over time — reconstructed from instructions at the sink, over one wai.det.fixed64 fixed-point floor, each with its own deterministic *-equivalence conformance criterion:

Capability	What it carries	Conformance
wai.world.{scene,replay,live}	interactive worlds (signed op log)	replay-equivalence
wai.world.model	action-conditioned world model (signed action ops → next frame, integer transition + synthesis)	replay-equivalence
wai.mesh.glb / wai.splat.spz	3D mesh / Gaussian-splat assets	content-addressed
wai.audio.scene	object-based spatial audio	mixdown-equivalence
wai.audio.score	music as note events (synthesized at the sink)	mixdown-equivalence
wai.haptic.signal	keyframed touch envelopes	sample-equivalence
wai.feed.state	scoreboards / telemetry / market feeds	replay-equivalence
wai.avatar.pose	digital humans (skeletal pose stream)	pose-equivalence
wai.splat.4d	volumetric / 4D temporal Gaussian splats	frame-state-equivalence
wai.film.linear	animation / CGI film (sealed world + camera track)	film-equivalence
wai.film.reel	a film with cuts (one world, a sequence of shots)	reel-equivalence

And the verify half — a signed, energy-accounted lineage for opaque captured content WAI does not reconstruct (wai.asset.provenance). See SPEC.md §5 and extensions/ for each.

Quantum — deterministic, native and in cloud WASM

WAI carries a full quantum stack in pure Rust: circuit simulation and the whole operational layer around it, every stage byte-exact and sealable into a signed, energy-accounted receipt. No QPU and no cloud service. Because it is pure Rust with no host bindings, one source builds for native and wasm32-wasip2 and produces byte-identical results — the same circuit reconstructs to the same statevector hash on a laptop or at the edge.

wai.quantum.circuit2 and wai.quantum.circuit are the registered payload capabilities, and wai.quantum.calibration and wai.quantum.job are registered records; the other names below are module labels (REGISTRY.md lists the receipt labels no revision registers). The two circuit capabilities share the container and the gates and differ in the phase table: circuit2's holds the integers nearest to each phase and is the default for new content, and wai.quantum.circuit is frozen so that every file, hash and receipt made with it reproduces (quantum-sim §8).

Module	What it does	Conformance
wai.quantum.circuit2	byte-exact statevector simulation (dyadic Clifford+T+P(k)), with the correctly rounded phase table	statevector-equivalence
wai.quantum.circuit	the same, with the frozen phase table existing content was made with	statevector-equivalence
wai.quantum.toolchain	four backends (statevector / stabilizer / tensor-network MPS / sparse-Pauli), algorithm library, OpenQASM 3.0 in-and-out, QIR export + ingest, Bloch + entanglement analysis	statevector-equivalence
wai.quantum.compile	Clifford routing + stabilizer-tableau equivalence proof	signed equivalence certificate
wai.quantum.cal.*	calibration, filter-function robust control, DD noise spectroscopy	reproducible + receipt
wai.quantum.cal.noise_learn	Cycle Benchmarking / CER sparse Pauli–Lindblad learning	signed noise model
wai.quantum.mitigate	ZNE + readout M3 + classical shadows	byte-exact post-processing
wai.quantum.decode	CSS codes (rotated surface, toric, bivariate-bicycle) + union-find and Relay-BP decoders	signed decode + joules
wai.quantum.rearrange	neutral-atom register prep (Hungarian / LSAP)	signed defect-free plan
wai.quantum.phasor	interference substrate + energy-accounted phasor receipts	reproducible f64
wai.quantum.kernel	the quantum kernel deployed as phasor features	reproducible f64
wai.quantum.qdata	learning-from-quantum-data sample-complexity separation	reproducible f64
wai.quantum.qbom	Quantum Bill of Materials — stage receipts into one provenance manifest	signed manifest

Quickstart

The command line is wai-quantum-cli: OpenQASM programs run here as the quantum job service runs them, each result naming its method and accuracy and signed with this user's key.

cd wai-quantum-cli && cargo build --release      # or: cargo install wai-quantum-cli
W=./target/release/wai-quantum

$W run bell.qasm --shots 1000 -o "Z0 Z1"          # planned as the service plans it
$W run bell.qasm --json > job.json && $W verify job.json
$W plan prog.qasm; $W lint prog.qasm; $W draw prog.qasm
$W submit prog.qasm                               # on the job service
$W lab list                                       # the algorithm catalogue
$W lab run ghz -n 5                               # simulate + hash + outcomes
$W lab qasm bell                                  # emit OpenQASM 3.0
$W lab qir qft -n 4 | $W lab run --qir            # QIR out, and back in
$W lab receipt ghz -n 3 | $W verify
$W lab run qft -n 5 --gateset v1                  # the frozen wai.quantum.circuit

Circuits built by lab are wai.quantum.circuit2 unless --gateset v1 asks for wai.quantum.circuit, which reproduces the circuit and statevector hashes earlier releases gave.

Same source, at the edge

cd wai-quantum-cli && cargo build --release --target wasm32-wasip2 --no-default-features

wasmtime run target/wasm32-wasip2/release/wai-quantum.wasm lab run ghz -n 5
wasmtime run target/wasm32-wasip2/release/wai-quantum.wasm lab receipt ghz -n 3 --demo-key

Both print the identical state-hash. A receipt sealed in cloud WASM verifies natively and vice-versa, and with the same key (here the published demo key) the receipt bytes themselves are identical: the signature travels because the computation is reproducible, not because the two runtimes agreed to trust each other.

As a serverless function

wai-quantum-serve/ exports the same stack as a WASI-HTTP component (wasi:http/proxy), so quantum computation runs as an ordinary HTTP handler on a component-model host:

cd wai-quantum-serve
cargo build --release --target wasm32-wasip2
wasmtime serve -S cli=y --addr 127.0.0.1:8231 \
  target/wasm32-wasip2/release/wai_quantum_serve.wasm

GET  /health                      GET  /algorithms
GET  /run?algo=ghz&n=5&top=8      POST /run       (body = OpenQASM 3.0)
GET  /qasm?algo=bell              GET  /receipt?algo=ghz&n=3[&shots=&seed=]
                                  (/run, /qasm and /receipt also take &gateset=v1|v2)
POST /verify                      (body = receipt JSON; 200 valid / 409 invalid / 400 unparseable)

The endpoint returns the same statevector_hash as the native CLI for the same circuit, and a receipt sealed by the HTTP handler verifies with the native wai-quantum verify — the JSON content is byte-identical (the CLI adds a trailing newline; nothing else differs). Both verify paths accept a receipt in either form: as sealed, or labelled with its figure's acquisition class, as wai_quantum_meter emits a measured figure (quantum_energy::MaybeLabelled, energy-measurement §2.8). Both restate the class, or say the figure has none.

That wasm32-wasip2 build needs -S cli=y, because Rust's std imports wasi:cli/environment at startup and the bare wasi:http/proxy world does not provide it. For a component that runs on any wasi:http host with no host flags at all, build the pure proxy-world artifact instead:

./build-proxy.sh
wasmtime serve --addr 127.0.0.1:8080 target/wai_quantum_serve.proxy.wasm

That path compiles a wasm32-wasip1 core module and adapts it with the Bytecode Alliance proxy adapter, leaving only imports the proxy world defines (wasi:http/types, wasi:io/*, wasi:cli/std{in,out,err}, wasi:random). The script fails the build if wasi:cli/environment ever reappears, and the served results are byte-identical either way.

Explore the whole stack in the browser (it is the same Rust, compiled to wasm): the toolchain IDE, the ML layer, and a working course.

License

Apache-2.0 — see LICENSE. Implementing a registered capability carries that capability's own licence class (royalty-free, patent-pool, proprietary or non-commercial), recorded in the registry. Companion implementations outside this tree keep their own licences.