open-standards/wai/README.md

# WAI — Web AI Media Transport & Execution

A container + capability-dispatch standard for media. A standard codec
(AVIF, JPEG XL, PNG, JPEG, Opus, FLAC, AV1, zstd, XZ and the others the
registry names) is dispatched to the library a sink already has; WAI also
defines its own capability classes — integer codecs, worlds, scores, films —
each with a conformance criterion, and an envelope that lets the
*neural-shared-prior* path coexist with the model-free floor.

Two paths open every conforming file:

- **Neural condition** — the sink regenerates the media against a
  shared ambient prior (a model). The model is a *requirement of
  existence*, never shipped (an `.mp3` doesn't ship its decoder
  either).
- **Zeroth condition** — a *menu of registered standard codecs*, decoded with
  the libraries a device already ships.

The capability a file requires is *named*, not supplied.

## Building

`wai-rs` builds the codec chain by default (the `codecs` feature), and those
crates link **system** libraries rather than vendoring them — consistent with
"WAI does not re-implement codecs". Without them the build stops in a codec
crate's build script, not in WAI's own code:

```sh
# macOS
brew install jpeg-xl libavif opus dav1d xz

# Debian / Ubuntu
apt-get install libjxl-dev libavif-dev libopus-dev libdav1d-dev liblzma-dev
```

To build without the C codec chain — the envelope, manifest and `world`
engine do not need it, which is also how the wasm32 target is built:

```sh
cargo check -p wai --no-default-features
```

## What's here

| Path | Contents |
|------|----------|
| [`SPEC.md`](SPEC.md) | WAI v1.0 spec — envelope, manifest, capability menu |
| [`wai-rs/`](wai-rs/) | Rust reference implementation (lib + cdylib + staticlib + C FFI) |
| [`REGISTRY.md`](REGISTRY.md), [`registry/capabilities.json`](registry/capabilities.json) | The registry: every registered string with its kind, status, classes and edition, generated from the reference implementation (`wai registry export`), with its JSON Schema and the policy for registering a string |
| [`vocab/`](vocab/) | RDF vocabulary: v2 generated from the registry, v1 frozen |
| [`extensions/`](extensions/) | Sub-specifications (confidentiality, receipts, worlds, film, quantum operations, integer operators, vector graphics, …) |
| [`research/`](research/) | Research notes and dated records: graded, not normative |
| [`glossary/`](glossary/) | Reference catalog of media codecs surveyed for capability registration. [`codecs.json`](glossary/codecs.json) is the dataset, linked to the registry; [`SCHEMA.md`](glossary/SCHEMA.md) defines the shape and contribution rules. Rendered at [wai.transaction.science/glossary](https://wai.transaction.science/glossary). |
| [`CONTRIBUTING.md`](CONTRIBUTING.md) | How to change the specification, register a capability and add tests CI runs |
| [`SECURITY.md`](SECURITY.md) | How to report a vulnerability, and where the reference implementation reads untrusted input: the manifest limits, the native codec libraries, module authorisation, the fingerprinting risk of advertised parameter sets |

## Quickstart

```bash
cd wai-rs
cargo test --no-default-features --lib --test carriage   # registry, envelope, every capability round-trips
cargo run --no-default-features --bin wai -- capabilities  # the registry, one row per capability
cargo build --release --lib    # with the codecs feature: libwai.dylib/.so/.dll + libwai.a
```

With the `codecs` feature, `wai-rs/build.rs` finds libFLAC through pkg-config
and, on macOS without pkg-config, in the Homebrew "flac" formula prefix
(`/opt/homebrew/opt/flac/lib` or `/usr/local/opt/flac/lib`). Where neither finds
it, set `RUSTFLAGS="-L <the directory holding libFLAC>"`.

## SDK consumption

`wai-rs` exposes a stable C ABI in [`src/ffi.rs`](wai-rs/src/ffi.rs) —
the cdylib drops in as `libwai.dylib`/`.so`/`.dll`. Bindings for any
language (Python ctypes, Node N-API, Swift, JNI, Go cgo, etc.) call
into:

- `wai_image_{png,jpeg,avif,jxl}_{encode,decode}`
- `wai_audio_{opus,flac}_{encode,decode}`
- `wai_text_{zstd,xz}_{encode,decode}`
- `wai_envelope_{pack,unpack}` — wrap/unwrap the WAI container
- `wai_buffer_free`, `wai_last_error` — memory + error handling

## In the browser, on older engines: the reach builds

`wai-web` is the same Rust compiled to wasm32 for a page. Its shipped package needs
WebAssembly with sign-extension, non-trapping float-to-int and bulk memory, and i64
values crossing the JS boundary as `BigInt`. The reach builds
([`wai-web/reach/`](wai-web/reach/README.md)) run that code where those are missing, and
give the same outputs byte for byte:

| rung | build | for engines with |
|---|---|---|
| shipped | the package | every feature above |
| L1 | the shipped binary, legalised at the JS interface | the same WebAssembly, no `BigInt` at the boundary |
| L2 | MVP-lowered and legalised | MVP WebAssembly |
| J | the wasm2js translation, ES5 | ES5 and typed arrays, no WebAssembly |

L2 and J also come in reduced profiles: `receipt`, `verify` and `verify_decode`.
`wai-load.js`, an ES5 loader, picks the rung from feature probes alone and says why. A
rung can be forced for testing. `reach/build.sh` makes every build with pinned tools
and records each output's SHA-256 in `reach/manifest.json`.

CI runs each rung on a Node image of the engine line it serves, and every output must
equal the shipped build's; the reach builds made from the package of record (kept in
`reach/record/`) must equal `reach/manifest.json`. The images stand in for device classes;
no real old device has been run.

## Registered capabilities (the zeroth menu)

| Capability | Codec | Lossless? | Mandatory? |
|---|---|---|---|
| `wai.image.png` | PNG | yes | **mandatory floor** |
| `wai.image.jxl` | JPEG-XL | both | recommended |
| `wai.image.avif` | AVIF (AV1-based) | lossy | recommended |
| `wai.image.jpeg` | JPEG | lossy | recommended (legacy compat) |
| `wai.audio.flac` | FLAC | yes | **mandatory floor** |
| `wai.audio.opus` | Opus | lossy | recommended |
| `wai.video.av1` | AV1 | lossy | recommended |
| `wai.video.av1.lossless` | AV1 (`quantizer=0`) | YUV-lossless | recommended |
| `wai.text.zstd` | zstd | yes | **mandatory floor** |
| `wai.text.xz` | XZ/LZMA2 | yes | recommended |

Neural capabilities (`wai.neural.*`) are OPTIONAL and sink-advertised. Most
run float inference, so their decode is exact only on the build that produced
it. A distinguished subset is **integer-exact by construction** — an all-`i64`
decode with no float and no ONNX — so the reconstructed artifact is
byte-identical on every machine and its hash is portable. Conformance is a
hash: a deployer can require `intent=replicate` to dispatch only to these.

| Capability | Integer-exact decode of | Conformance |
|---|---|---|
| `wai.neural.int_hyper` | image — integer hyperprior σ source + integer entropy + integer synthesis | byte-exact (hash-portable) |
| `wai.neural.int_mlicpp` | image — MLIC++ entropy topology (channel slices × checkerboard passes, an entropy network re-run before each pass) on integer networks | byte-exact (hash-portable) |
| `wai.neural.int_synth` | image — integer synthesis only | byte-exact (hash-portable) |
| `wai.video.int_hyper` | video — `int_hyper` I-frame + integer half-pel motion-compensated P-frames | byte-exact (hash-portable) |
| `wai.audio.int_codec` | audio — learned 1-D transform, integer synthesis → int16 PCM | byte-exact (hash-portable) |
| `wai.splat.int_codec` | 3D — learned Gaussian-splat-attribute transform, integer synthesis | byte-exact (hash-portable) |

Their payloads are binary formats SPEC §5 pins — `WIV1` for the integer video
codecs, `WIH1`, `WIA1` and `WIS1`
([`extensions/integer-payloads.md`](extensions/integer-payloads.md)). The JSON the
encode and export tools write is converted to them, never carried. The same extension
writes down the `wai.neural.int_synth` payload and the `WIM1` payload of
`wai.neural.int_mlicpp`. Their networks' integer arithmetic is written down as the
operator set `wai-ops/1`
([`extensions/integer-operators.md`](extensions/integer-operators.md), Draft).

## Interactive & temporal content (extensions)

Beyond the codec menu, WAI carries content that consumes input or moves
over time — reconstructed from instructions at the sink, over one
`wai.det.fixed64` fixed-point floor, each with its own deterministic
`*-equivalence` conformance criterion:

| Capability | What it carries | Conformance |
|---|---|---|
| `wai.world.{scene,replay,live}` | interactive worlds (signed op log) | replay-equivalence |
| `wai.world.model` | action-conditioned world model (signed action ops → next frame, integer transition + synthesis) | replay-equivalence |
| `wai.mesh.glb` / `wai.splat.spz` | 3D mesh / Gaussian-splat assets | content-addressed |
| `wai.audio.scene` | object-based spatial audio | mixdown-equivalence |
| `wai.audio.score` | music as note events (synthesized at the sink) | mixdown-equivalence |
| `wai.haptic.signal` | keyframed touch envelopes | sample-equivalence |
| `wai.feed.state` | scoreboards / telemetry / market feeds | replay-equivalence |
| `wai.avatar.pose` | digital humans (skeletal pose stream) | pose-equivalence |
| `wai.splat.4d` | volumetric / 4D temporal Gaussian splats | frame-state-equivalence |
| `wai.film.linear` | animation / CGI film (sealed world + camera track) | film-equivalence |
| `wai.film.reel` | a film with cuts (one world, a sequence of shots) | reel-equivalence |

And the verify half — a signed, energy-accounted lineage for opaque
captured content WAI does **not** reconstruct (`wai.asset.provenance`).
See [`SPEC.md`](SPEC.md) §5 and [`extensions/`](extensions/) for each.

## Quantum — deterministic, native and in cloud WASM

WAI carries a full quantum stack in pure Rust: circuit simulation and the whole
operational layer around it, every stage byte-exact and sealable into a signed,
energy-accounted receipt. No QPU and no cloud service. Because it is
pure Rust with no host bindings, **one source builds for native and
`wasm32-wasip2` and produces byte-identical results** — the same circuit
reconstructs to the same statevector hash on a laptop or at the edge.

`wai.quantum.circuit2` and `wai.quantum.circuit` are the registered payload
capabilities, and `wai.quantum.calibration` and `wai.quantum.job` are registered
records; the other names below are module labels ([`REGISTRY.md`](REGISTRY.md)
lists the receipt labels no revision registers). The two circuit capabilities
share the container and the gates and differ in the phase table: circuit2's holds
the integers nearest to each phase and is the default for new content, and
`wai.quantum.circuit` is frozen so that every file, hash and receipt made with it
reproduces ([quantum-sim](extensions/quantum-sim.md) §8).

| Module | What it does | Conformance |
|---|---|---|
| `wai.quantum.circuit2` | byte-exact statevector simulation (dyadic Clifford+T+P(k)), with the correctly rounded phase table | statevector-equivalence |
| `wai.quantum.circuit` | the same, with the frozen phase table existing content was made with | statevector-equivalence |
| `wai.quantum.toolchain` | four backends (statevector / stabilizer / tensor-network MPS / sparse-Pauli), algorithm library, OpenQASM 3.0 in-and-out, **QIR export + ingest**, Bloch + entanglement analysis | statevector-equivalence |
| `wai.quantum.compile` | Clifford routing + stabilizer-tableau equivalence proof | signed equivalence certificate |
| `wai.quantum.cal.*` | calibration, filter-function robust control, DD noise spectroscopy | reproducible + receipt |
| `wai.quantum.cal.noise_learn` | Cycle Benchmarking / CER sparse Pauli–Lindblad learning | signed noise model |
| `wai.quantum.mitigate` | ZNE + readout M3 + classical shadows | byte-exact post-processing |
| `wai.quantum.decode` | CSS codes (rotated surface, toric, bivariate-bicycle) + union-find and Relay-BP decoders | signed decode + joules |
| `wai.quantum.rearrange` | neutral-atom register prep (Hungarian / LSAP) | signed defect-free plan |
| `wai.quantum.phasor` | interference substrate + energy-accounted phasor receipts | reproducible f64 |
| `wai.quantum.kernel` | the quantum kernel deployed as phasor features | reproducible f64 |
| `wai.quantum.qdata` | learning-from-quantum-data sample-complexity separation | reproducible f64 |
| `wai.quantum.qbom` | Quantum Bill of Materials — stage receipts into one provenance manifest | signed manifest |

### Quickstart

```bash
# native
cargo build --release --no-default-features \
  --features "quantum_toolchain,quantum_receipt" --bin wai-quantum

./target/release/wai-quantum list                  # algorithm catalog
./target/release/wai-quantum run ghz -n 5          # simulate + hash + outcomes
./target/release/wai-quantum run grover -n 4 -p 9  # amplitude amplification
./target/release/wai-quantum qasm bell             # emit OpenQASM 3.0
./target/release/wai-quantum qir  bell             # emit QIR (LLVM IR)
./target/release/wai-quantum qir  qft -n 4 | \\
  ./target/release/wai-quantum run --qir           # and read it back
./target/release/wai-quantum receipt ghz -n 3 | ./target/release/wai-quantum verify
./target/release/wai-quantum run qft -n 5 --gateset v1   # the frozen wai.quantum.circuit
```

Circuits are `wai.quantum.circuit2` unless `--gateset v1` asks for
`wai.quantum.circuit`, which reproduces the circuit and statevector hashes
earlier releases gave.

### Same source, at the edge

```bash
cargo build --release --target wasm32-wasip2 --no-default-features \
  --features "quantum_toolchain,quantum_receipt" --bin wai-quantum

wasmtime run target/wasm32-wasip2/release/wai-quantum.wasm run ghz -n 5
```

Both print the identical `state-hash`. A receipt sealed in cloud WASM verifies
natively and vice-versa, and the receipt bytes themselves are identical — the
signature travels because the computation is reproducible, not because the two
runtimes agreed to trust each other.

### As a serverless function

[`wai-quantum-serve/`](wai-quantum-serve/) exports the same stack as a
**WASI-HTTP component** (`wasi:http/proxy`), so quantum computation runs as an
ordinary HTTP handler on a component-model host:

```bash
cd wai-quantum-serve
cargo build --release --target wasm32-wasip2
wasmtime serve -S cli=y --addr 127.0.0.1:8231 \
  target/wasm32-wasip2/release/wai_quantum_serve.wasm
```

```text
GET  /health                      GET  /algorithms
GET  /run?algo=ghz&n=5&top=8      POST /run       (body = OpenQASM 3.0)
GET  /qasm?algo=bell              GET  /receipt?algo=ghz&n=3[&shots=&seed=]
                                  (/run, /qasm and /receipt also take &gateset=v1|v2)
POST /verify                      (body = receipt JSON; 200 valid / 409 invalid / 400 unparseable)
```

The endpoint returns the **same** `statevector_hash` as the native CLI for the
same circuit, and a receipt sealed by the HTTP handler verifies with the native
`wai-quantum verify` — the JSON content is byte-identical (the CLI adds a
trailing newline; nothing else differs). Both verify paths accept a receipt in
either form: as sealed, or labelled with its figure's acquisition class, as
`wai_quantum_meter` emits a measured figure (`quantum_energy::MaybeLabelled`,
[energy-measurement](extensions/energy-measurement.md) §2.8). Both restate the
class, or say the figure has none.

That `wasm32-wasip2` build needs `-S cli=y`, because Rust's std imports
`wasi:cli/environment` at startup and the bare `wasi:http/proxy` world does not
provide it. For a component that runs on **any** `wasi:http` host with no host
flags at all, build the pure proxy-world artifact instead:

```bash
./build-proxy.sh
wasmtime serve --addr 127.0.0.1:8080 target/wai_quantum_serve.proxy.wasm
```

That path compiles a `wasm32-wasip1` core module and adapts it with the Bytecode
Alliance proxy adapter, leaving only imports the proxy world defines
(`wasi:http/types`, `wasi:io/*`, `wasi:cli/std{in,out,err}`, `wasi:random`). The
script fails the build if `wasi:cli/environment` ever reappears, and the served
results are byte-identical either way.

Explore the whole stack in the browser (it is the same Rust, compiled to wasm):
the [toolchain IDE](https://wai.transaction.science/quantum-ide), the
[ML layer](https://wai.transaction.science/quantum-ml), and a
[working course](https://wai.transaction.science/quantum-learn).

## License

Apache-2.0 — see [`LICENSE`](LICENSE). Implementing a registered capability
carries that capability's own licence class (royalty-free, patent-pool,
proprietary or non-commercial), recorded in the registry. Companion
implementations outside this tree keep their own licences.