open-standards/wai

.WAI™ — Web AI Media Transport & Execution

A container + capability-dispatch standard for media. A standard codec (AVIF, JPEG XL, PNG, JPEG, Opus, FLAC, AV1, zstd, XZ and the others the registry names) is dispatched to the library a sink already has; WAI also defines its own capability classes — integer codecs, worlds, scores, films — each with a conformance criterion, and an envelope that lets the neural-shared-prior path coexist with the model-free floor.

Two paths open every conforming file:

• Neural condition — the sink regenerates the media against a shared ambient prior (a model). The model is a requirement of existence, never shipped (an .mp3 doesn't ship its decoder either).
• Zeroth condition — a menu of registered standard codecs, decoded with the libraries a device already ships.

The capability a file requires is named, not supplied.

Building

wai-rs builds the codec chain by default (the codecs feature), and those crates link system libraries rather than vendoring them — consistent with "WAI does not re-implement codecs". Without them the build stops in a codec crate's build script, not in WAI's own code:

# macOS
brew install jpeg-xl libavif opus dav1d xz

# Debian / Ubuntu
apt-get install libjxl-dev libavif-dev libopus-dev libdav1d-dev liblzma-dev

To build without the C codec chain — the envelope, manifest and world engine do not need it, which is also how the wasm32 target is built:

cargo check -p wai --no-default-features

With the feature floor alone, wai-rs decodes the mandatory floor (PNG, FLAC, zstd) with no system library: PNG and FLAC in pure Rust, zstd from its reference source compiled in. The quickstart feature adds wai hello and wai verify; it is how the prebuilt wai is built:

cargo install --locked --path wai-rs --no-default-features --features quickstart --bin wai

What's here

Path	Contents
SPEC.md	WAI spec — envelope, manifest, capability menu
wai-rs/	Rust reference implementation (lib + cdylib + staticlib + C FFI)
examples/	Small runnable projects, one per template of wai create: a video room, watching together, messaging and a call, an encoder into a room, packaging, receipts in Rust and Python, native sinks on Apple and Android platforms, remote desktop
quickstart/	The prebuilt command line's archive: what it holds, how it is built and released, and its installer
schema/	The JSON Schemas of wai.json, wai.lock and the build record, and what each member pins
project-conformance/	wai.lock's bytes, and a second reader that recomputes every digest with the standard library alone
catalogue/	The operation catalogue as JSON (wai ops export): every operation of the toolchain, noun.verb, and the name each door gives it
REGISTRY.md, registry/capabilities.json	The registry: every registered string with its kind, status, classes and edition, generated from the reference implementation (wai registry export), with its JSON Schema and the policy for registering a string
vocab/	RDF vocabulary: v2 generated from the registry, v1 frozen
extensions/	Sub-specifications (confidentiality, receipts, worlds, film, quantum operations, integer operators, vector graphics, …)
research/	Research notes and dated records: graded, not normative
glossary/	Reference catalog of media codecs surveyed for capability registration. codecs.json is the dataset, linked to the registry; SCHEMA.md defines the shape and contribution rules. Rendered at wai.transaction.science/glossary.
CONTRIBUTING.md	How to change the specification, register a capability and add tests CI runs
SECURITY.md	How to report a vulnerability, and where the reference implementation reads untrusted input: the manifest limits, the native codec libraries, module authorisation, the fingerprinting risk of advertised parameter sets

Quickstart

The command line, with the relay and the room service it runs locally: the prebuilt archive (quickstart/, built by quickstart/build.sh), or from source.

wai hello                                   # name, dispatch, rebuild and receipt, on a sample
wai create my-room --template video-room    # a project from one of the examples (examples/)
cd my-room && wai dev                       # the relay, the room service and your page, on 127.0.0.1:8080
wai verify ../wai-hello                     # the receipts hello wrote, checked offline
wai doctor                                  # whether this machine can run the toolchain, one check to a line

examples/ holds each template, with a test that runs it.

wai dev is the one local stack: the relay and the room service run as their own processes, from configuration it writes in .wai/dev/, as a deployment runs them. The relay holds each publisher's media to its receipts, and the room service holds the room's keys. A relay or room service of another toolchain than wai's is refused before anything starts. With --control it takes JSON control lines on standard input, answered by synthetic participants (wai-room participant): join (a test principal enters with its grant and publishes with its receipts), leave, session, test, load, describe and stop. With --impair loss=2%,rtt=80,jitter=10 the relay also listens for QUIC on loopback, and the synthetic participants enter over it through a path that drops, delays and spreads their datagrams; a page's WebSocket session rides TCP and is not impaired. Against a stack that runs:

wai room test                    # a publisher and a viewer: objects received of those sent, how late, the path
wai room load --viewers 32       # one publisher, 32 viewers: deliveries made of those owed

Each finds the stack by its record, .wai/dev/stack.json, and refuses a stale one: a record whose wai dev is no longer running, or one started by another toolchain. Over QUIC, the answer gives the packets QUIC declared lost and its round trip beside the datagrams the impaired path dropped.

Every command is an operation of one catalogue, named noun.verb (envelope.inspect, project.create): wai ops list lists them, wai ops search "<task>" finds one offline, and wai ops export writes the catalogue as JSON (catalogue/ops.json). Every command takes --json (one JSON envelope on stdout), --dry-run where it would write, and --yes where a change waits for confirmation (--confirm, deprecated, does the same), and exits with one of eight codes (wai --help lists them). wai sends nothing off your machine unless you name a remote service.

A remote service is named with --target, the same flag on every operation that acts on one: a target's name, or the service's address. wai target add staging --url https://… keeps a name for an address in the project's wai.json (or in your own list), and wai target use staging makes it the one taken when none is named. wai auth login --url https://… < key-file keeps the key a service takes, outside the project: in the keychain on macOS, else in a file only you read; it is never on a command line, and WAI_TOKEN, when set, is sent instead. Every operation of a service's API runs as wai <noun> <verb> --target <name> (wai room show --target staging), and wai mcp --target staging serves them to a coding agent, reading the target unless it may write (--allow-write).

The library's own checks, from a clone:

cd wai-rs
cargo test --no-default-features --lib --test carriage   # registry, envelope, every capability round-trips
cargo run --no-default-features --bin wai -- capabilities  # the registry, one row per capability
cargo build --release --lib    # with the codecs feature: libwai.dylib/.so/.dll + libwai.a

Two commands hold what is published to what was published before, and other implementations to WAI's corpora:

wai breaking --against origin/main     # what this tree breaks: wire, record and API (CI runs it on every pull request)
wai conform verifier --suite claim --impl "my-verifier verdicts {corpus}"   # a corpus run against your implementation
wai verify claim.wai-conform.json      # the signed result, checked
wai conform --independent --out records --format junit   # every corpus's independent verifier, a record each

A project is pinned, checked and built by three more, each in its directory:

wai lock     # wai.lock: the toolchain, the template, the registry, parameter sets, the policy and corpora, by digest
wai check    # the project validated without building it: every finding, and what could not run says not checked
wai build    # dist/, the same bytes from the same project and lock, with a record naming what it came from
wai breaking --baseline published/   # what the project breaks: its package's sink classes, and the API it names in wai.json

A break passes only when the changelog names it: an entry this change adds, as a file of changes/ or to CHANGELOG.md, that says Breaking and gives the finding's id in backticks. In a project, a break passes when breaking.allow in its wai.json names it with the reason it is accepted (VERSIONING.md). wai conform --list names every kind (verifier, decoder, packager, origin, sink) and suite, with its adapters: a command line, or an http:// URL; and every corpus's independent verifier, its second implementation, with the command line it runs. --independent beside a kind runs the corpus's independent verifier on the same cases, and the run passes only when both conform and agree on every case. --format junit or --format html prints a report made from the signed records. wai verify --conformant admits a result only for a run that passed, against the corpus of the checkout it runs in.

With the codecs feature, wai-rs/build.rs finds libFLAC through pkg-config and, on macOS without pkg-config, in the Homebrew "flac" formula prefix (/opt/homebrew/opt/flac/lib or /usr/local/opt/flac/lib). Where neither finds it, set RUSTFLAGS="-L <the directory holding libFLAC>".

For coding agents

wai setup-agent makes a project ready for a coding agent to use WAI. In the project directory it writes AGENTS.md, the instructions agents read, adds an import of it to a CLAUDE.md the project already has, writes the skill agents load for a WAI task (.agents/skills/wai/SKILL.md, and the same file in the skills directory of an agent that keeps its own, where the project holds that agent's directory), and registers wai mcp in the project's MCP configuration: .mcp.json, and the project-level MCP file of each agent whose directory the project already holds (--all writes every one it knows, --dry-run lists them, --toolset core,dev registers the server with those toolsets). A file that does not exist is created; one that exists is changed only after its diff is shown and the change confirmed (asked on a terminal, or --yes); without either, nothing that exists changes and the command exits 4 with each proposed change and the command that confirms. In a Markdown file it owns one block between two marker lines, and a rerun replaces that block and nothing outside it. Its text is in the binary, so it works offline and matches the version installed. It writes nothing outside the project directory. With --json its output is the toolchain's JSON envelope.

wai mcp is a Model Context Protocol server on standard input and output. Each tool is an operation of the toolchain, named by it with _ for the dot, and the tools come in toolsets:

Toolset	Tools
core (the default)	envelope_inspect, media_inspect, registry_list, registry_show, receipt_verify, envelope_wrap, conformance_list, conformance_run, project_check, project_lock, project_build, docs_search, ops_search, ops_show, toolchain_doctor
dev	dev_start, dev_stop, room_grant, room_join, room_leave, session_show, room_test, room_load
app (a build with the feature app)	app_guide, app_check, app_stream, app_export: Create's, below
quantum	quantum_run, quantum_inspect, quantum_check, quantum_verify, quantum_list: the quantum command line's, below

receipt_verify verifies a delivery receipt, a claim, a live-room session or a stream audit bundle, and answers with the verifier's verdict and reason code. The dev tools, marked development only, act on the target local alone: they start wai dev --control on loopback, in a directory of its own, mint grants for test principals, join and publish as them with their receipts (or without, to see the relay drop them), read the session for verification, and test and load the stack with synthetic participants; the grant tool refuses any service that is not on a loopback address. The same operations have doors on the command line, on the stack wai dev runs in the directory a command runs in: wai room join --principal NAME, wai room leave, wai session show, wai room test|load and wai room invite, with --target local its room service, and wai log tail and wai session list read its logs and its relay's session summaries. --read-only offers only the tools that read or run a verifier; the packaging tool writes only inside the project directory. No tool sends anything off the machine.

The quantum tools are the quantum command line's run, plan, lint, verify and methods: wai mcp starts that command line's own server, wai-quantum mcp (--quantum-bin, WAI_QUANTUM_BIN, beside wai, or on PATH), lists each tool with the schemas it states, and returns its answer to each call unchanged. wai-quantum mcp serves the same tools under its own names. On the command line the same verbs are wai quantum run|plan|lint|verify|methods, each wai-quantum itself, and no other verb of it is reached through wai.

One command for the rest of the toolchain. A program of the toolchain that has its own command line answers through wai too, run as itself, so what it prints and its exit status are its own: wai audit report|check|verify is the stream audit's collector (wai-audit); wai serve <role> starts a service from its configuration (wai serve relay --config relay.toml, and room, ingest, key-release, audit, fleet, ledger, quantum-service, each wai-<role>), and with --check has it read the file and report, starting nothing; wai cmcd encode|decode|verify is the delivery telemetry corpus's CMCD tool. wai receipt create <file>... --from <input>... signs, for files made outside the toolchain, the receipt its own writes leave, and wai verify reads every record by what it is: receipt indexes, delivery receipts, claims, room sessions, audit bundles (--owner, --channel), sealed playlists, operation receipts, conformance results, fleet bundles and envelopes. A program that is not installed is exit 5, naming where wai looked for it.

cd wai-rs
cargo install --path . --no-default-features --features mcp   # `wai` with `wai mcp`, no codec libraries
cd /path/to/your/project && wai setup-agent --toolset core,dev

The quickstart's wai holds wai mcp too.

Create (wai-app) builds a WAI app from a description: a model writes it in joule-ui's line language and the page renders it as it streams. Its program, app.oui, becomes a project with wai create my-app --from app.oui (a build with the feature app): an ordinary WAI project that wai app check checks and wai dev runs.

The room service's HTTP API is described, OpenAPI 3.1, in joule-moq-room/openapi.json, each operation's operationId its name in the toolchain's catalogue.

SDK consumption

wai-rs exposes a stable C ABI in src/ffi.rs — the cdylib drops in as libwai.dylib/.so/.dll. Bindings for any language (Python ctypes, Node N-API, Swift, JNI, Go cgo, etc.) call into:

• wai_image_{png,jpeg,avif,jxl}_{encode,decode}
• wai_audio_{opus,flac}_{encode,decode}
• wai_text_{zstd,xz}_{encode,decode}
• wai_envelope_{pack,unpack} — wrap/unwrap the WAI container
• wai_buffer_free, wai_last_error — memory + error handling

In the browser, on older engines: the reach builds

wai-web is the same Rust compiled to wasm32 for a page. Its shipped package needs WebAssembly with sign-extension, non-trapping float-to-int and bulk memory, and i64 values crossing the JS boundary as BigInt. The reach builds (wai-web/reach/) run that code where those are missing, and give the same outputs byte for byte:

rung	build	for engines with
shipped	the package	every feature above
L1	the shipped binary, legalised at the JS interface	the same WebAssembly, no BigInt at the boundary
L2	MVP-lowered and legalised	MVP WebAssembly
J	the wasm2js translation, ES5	ES5 and typed arrays, no WebAssembly

L2 and J also come in reduced profiles: receipt, verify and verify_decode. wai-load.js, an ES5 loader, picks the rung from feature probes alone and says why. A rung can be forced for testing. reach/build.sh makes every build with pinned tools and records each output's SHA-256 in reach/manifest.json.

CI runs each rung on a Node image of the engine line it serves, and every output must equal the shipped build's; the reach builds made from the package of record (kept in reach/record/) must equal reach/manifest.json. The images stand in for device classes; no real old device has been run.

Registered capabilities (the zeroth menu)

Capability	Codec	Lossless?	Mandatory?
wai.image.png	PNG	yes	mandatory floor
wai.image.jxl	JPEG-XL	both	recommended
wai.image.avif	AVIF (AV1-based)	lossy	recommended
wai.image.jpeg	JPEG	lossy	recommended (legacy compat)
wai.audio.flac	FLAC	yes	mandatory floor
wai.audio.opus	Opus	lossy	recommended
wai.video.av1	AV1	lossy	recommended
wai.video.av1.lossless	AV1 (quantizer=0)	YUV-lossless	recommended
wai.text.zstd	zstd	yes	mandatory floor
wai.text.xz	XZ/LZMA2	yes	recommended

Neural capabilities (wai.neural.*) are OPTIONAL and sink-advertised. Most run float inference, so their decode is exact only on the build that produced it. A distinguished subset is integer-exact by construction — an all-i64 decode with no float and no ONNX — so the reconstructed artifact is byte-identical on every machine and its hash is portable. Conformance is a hash: a deployer can require intent=replicate to dispatch only to these.

Capability	Integer-exact decode of	Conformance
wai.neural.int_hyper	image — integer hyperprior σ source + integer entropy + integer synthesis	byte-exact (hash-portable)
wai.neural.int_mlicpp	image — MLIC++ entropy topology (channel slices × checkerboard passes, an entropy network re-run before each pass) on integer networks	byte-exact (hash-portable)
wai.neural.int_synth	image — integer synthesis only	byte-exact (hash-portable)
wai.video.int_hyper	video — int_hyper I-frame + integer half-pel motion-compensated P-frames	byte-exact (hash-portable)
wai.audio.int_codec	audio — learned 1-D transform, integer synthesis → int16 PCM	byte-exact (hash-portable)
wai.splat.int_codec	3D — learned Gaussian-splat-attribute transform, integer synthesis	byte-exact (hash-portable)

Their payloads are binary formats SPEC §5 pins — WIV1 for the integer video codecs, WIH1, WIA1 and WIS1 (extensions/integer-payloads.md). The JSON the encode and export tools write is converted to them, never carried. The same extension writes down the wai.neural.int_synth payload and the WIM1 payload of wai.neural.int_mlicpp. Their networks' integer arithmetic is written down as the operator set wai-ops/1 (extensions/integer-operators.md, Draft).

Interactive & temporal content (extensions)

Beyond the codec menu, WAI carries content that consumes input or moves over time — reconstructed from instructions at the sink, over one wai.det.fixed64 fixed-point floor, each with its own deterministic *-equivalence conformance criterion:

Capability	What it carries	Conformance
wai.world.{scene,replay,live}	interactive worlds (signed op log)	replay-equivalence
wai.world.model	action-conditioned world model (signed action ops → next frame, integer transition + synthesis)	replay-equivalence
wai.mesh.glb / wai.splat.spz	3D mesh / Gaussian-splat assets	content-addressed
wai.audio.scene	object-based spatial audio	mixdown-equivalence
wai.audio.score	music as note events (synthesized at the sink)	mixdown-equivalence
wai.haptic.signal	keyframed touch envelopes	sample-equivalence
wai.feed.state	scoreboards / telemetry / market feeds	replay-equivalence
wai.avatar.pose	digital humans (skeletal pose stream)	pose-equivalence
wai.splat.4d	volumetric / 4D temporal Gaussian splats	frame-state-equivalence
wai.film.linear	animation / CGI film (sealed world + camera track)	film-equivalence
wai.film.reel	a film with cuts (one world, a sequence of shots)	reel-equivalence

And the verify half — a signed, energy-accounted lineage for opaque captured content WAI does not reconstruct (wai.asset.provenance). See SPEC.md §5 and extensions/ for each.

Quantum — deterministic, native and in cloud WASM

WAI carries a full quantum stack in pure Rust: circuit simulation and the whole operational layer around it, every stage byte-exact and sealable into a signed, energy-accounted receipt. No QPU and no cloud service. Because it is pure Rust with no host bindings, one source builds for native and wasm32-wasip2 and produces byte-identical results — the same circuit reconstructs to the same statevector hash on a laptop or at the edge.

wai.quantum.circuit2 and wai.quantum.circuit are the registered payload capabilities, and wai.quantum.calibration and wai.quantum.job are registered records; the other names below are module labels, which no revision registers (REGISTRY.md lists them with every other wai.quantum.* label, and says what each names). The two circuit capabilities share the container and the gates and differ in the phase table: circuit2's holds the integers nearest to each phase and is the default for new content, and wai.quantum.circuit is frozen so that every file, hash and receipt made with it reproduces (quantum-sim §8).

Module	What it does	Conformance
wai.quantum.circuit2	byte-exact statevector simulation (dyadic Clifford+T+P(k)), with the correctly rounded phase table	statevector-equivalence
wai.quantum.circuit	the same, with the frozen phase table existing content was made with	statevector-equivalence
wai.quantum.toolchain	four backends (statevector / stabilizer / tensor-network MPS / sparse-Pauli), algorithm library, OpenQASM 3.0 in-and-out, QIR export + ingest, Bloch + entanglement analysis	statevector-equivalence
wai.quantum.compile	Clifford routing + stabilizer-tableau equivalence proof	signed equivalence certificate
wai.quantum.cal.*	calibration, filter-function robust control, DD noise spectroscopy	reproducible + receipt
wai.quantum.cal.noise_learn	Cycle Benchmarking / CER sparse Pauli–Lindblad learning	signed noise model
wai.quantum.mitigate	ZNE + readout M3 + classical shadows	byte-exact post-processing
wai.quantum.decode	CSS codes (rotated surface, toric, bivariate-bicycle) + union-find and Relay-BP decoders	signed decode + joules
wai.quantum.rearrange	neutral-atom register prep (Hungarian / LSAP)	signed defect-free plan
wai.quantum.phasor	interference substrate + energy-accounted phasor receipts	reproducible f64
wai.quantum.kernel	the quantum kernel deployed as phasor features	reproducible f64
wai.quantum.qdata	learning-from-quantum-data sample-complexity separation	reproducible f64
wai.quantum.qbom	Quantum Bill of Materials — stage receipts into one provenance manifest	signed manifest

Quickstart

The command line is wai-quantum-cli: OpenQASM programs run here as the quantum job service runs them, each result naming its method and accuracy and signed with this user's key.

cd wai-quantum-cli && cargo build --release      # or: cargo install wai-quantum-cli
W=./target/release/wai-quantum

$W run bell.qasm --shots 1000 -o "Z0 Z1"          # planned as the service plans it
$W run bell.qasm --json > job.json && $W verify job.json
$W plan prog.qasm; $W lint prog.qasm; $W draw prog.qasm
$W submit prog.qasm                               # on the job service
$W lab list                                       # the algorithm catalogue
$W lab run ghz -n 5                               # simulate + hash + outcomes
$W lab qasm bell                                  # emit OpenQASM 3.0
$W lab qir qft -n 4 | $W lab run --qir            # QIR out, and back in
$W lab receipt ghz -n 3 | $W verify
$W lab run qft -n 5 --gateset v1                  # the frozen wai.quantum.circuit

Circuits built by lab are wai.quantum.circuit2 unless --gateset v1 asks for wai.quantum.circuit, which reproduces the circuit and statevector hashes earlier releases gave.

Same source, at the edge

cd wai-quantum-cli && cargo build --release --target wasm32-wasip2 --no-default-features

wasmtime run -W nan-canonicalization=y -W relaxed-simd-deterministic=y target/wasm32-wasip2/release/wai-quantum.wasm lab run ghz -n 5
wasmtime run -W nan-canonicalization=y -W relaxed-simd-deterministic=y target/wasm32-wasip2/release/wai-quantum.wasm lab receipt ghz -n 3 --demo-key

Both print the identical state-hash. A receipt sealed in cloud WASM verifies natively and vice-versa, and with the same key (here the published demo key) the receipt bytes themselves are identical: the signature travels because the computation is reproducible, not because the two runtimes agreed to trust each other.

As a serverless function

wai-quantum-serve/ exports the same stack as a WASI-HTTP component (wasi:http/proxy), so quantum computation runs as an ordinary HTTP handler on a component-model host:

cd wai-quantum-serve
cargo build --release --target wasm32-wasip2
wasmtime serve -S cli=y --addr 127.0.0.1:8231 \
  target/wasm32-wasip2/release/wai_quantum_serve.wasm

GET  /health                      GET  /algorithms
GET  /run?algo=ghz&n=5&top=8      POST /run       (body = OpenQASM 3.0)
GET  /qasm?algo=bell              GET  /receipt?algo=ghz&n=3[&shots=&seed=]
                                  (/run, /qasm and /receipt also take &gateset=v1|v2)
POST /verify                      (body = receipt JSON; 200 valid / 409 invalid / 400 unparseable)

The endpoint returns the same statevector_hash as the native CLI for the same circuit, and a receipt sealed by the HTTP handler verifies with the native wai-quantum verify — the JSON content is byte-identical (the CLI adds a trailing newline; nothing else differs). Both verify paths accept a receipt in either form: as sealed, or labelled with its figure's acquisition class, as wai_quantum_meter emits a measured figure (quantum_energy::MaybeLabelled, energy-measurement §2.8). Both restate the class, or say the figure has none.

That wasm32-wasip2 build needs -S cli=y, because Rust's std imports wasi:cli/environment at startup and the bare wasi:http/proxy world does not provide it. For a component that runs on any wasi:http host with no host flags at all, build the pure proxy-world artifact instead:

./build-proxy.sh
wasmtime serve --addr 127.0.0.1:8080 target/wai_quantum_serve.proxy.wasm

That path compiles a wasm32-wasip1 core module and adapts it with the Bytecode Alliance proxy adapter, leaving only imports the proxy world defines (wasi:http/types, wasi:io/*, wasi:cli/std{in,out,err}, wasi:random). The script fails the build if wasi:cli/environment ever reappears, and the served results are byte-identical either way.

Explore the whole stack in the browser (it is the same Rust, compiled to wasm): the toolchain IDE, the ML layer, and a working course.

License

Apache-2.0 — see LICENSE. Implementing a registered capability carries that capability's own licence class (royalty-free, patent-pool, proprietary or non-commercial), recorded in the registry. Companion implementations outside this tree keep their own licences.