Transaction-Science/open-standards
Watch
1
0
Fork
You've already forked open-standards
 0
Code Issues Pull requests Projects Releases 15 Packages Wiki Activity Actions 163
open-standards/wai/README.md
Transaction Science 84b630956a The fleet's operations in the catalogue, and one confirmation flag, --yes 
enrolment.create, enrolment.list, report.create and report.export join the catalogue as the
other operations are: placed on the client against a fleet service, with their effect, toolset,
input and output schemas, receipt and the features they need (fleet for the two that sign or
verify). wai enrolment|report run through the door, and the fleet's command module keeps the
agent rules of dx/envelope.rs instead of its own copy of them: the envelope with --json, every
file through out.rs and every request to the service named in a dry run's plan and not sent
(out::would_send), and --yes before the service is changed or a file replaced, exit 4 off a
terminal with the proposal and the confirming command.

A catalogue entry now says when it waits for confirmation (Op::confirms: never, a file the
command line did not name, any file, the target or a file, always), and the rules refuse a
change to a target that does not wait and a read that does. The export carries it and the
agent flags (AGENT_FLAGS), from which wai --help is written: --yes is the one confirmation
flag, and --confirm, which wai conform, wai setup-agent and the fleet took, is its deprecated
spelling, still accepted, so named in help, in every confirming operation's help and in the
export. wai conform confirms through the same module. wai verify's entry names the
conformance results and fleet bundles it reads.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-28 05:32:21 -04:00

398 lines
22 KiB
Markdown
Raw Permalink Blame History

This file contains ambiguous Unicode characters

This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.

# WAI — Web AI Media Transport & Execution
A container + capability-dispatch standard for media. A standard codec
(AVIF, JPEG XL, PNG, JPEG, Opus, FLAC, AV1, zstd, XZ and the others the
registry names) is dispatched to the library a sink already has; WAI also
defines its own capability classes — integer codecs, worlds, scores, films —
each with a conformance criterion, and an envelope that lets the
*neural-shared-prior* path coexist with the model-free floor.
Two paths open every conforming file:
- **Neural condition** — the sink regenerates the media against a
shared ambient prior (a model). The model is a *requirement of
existence*, never shipped (an `.mp3` doesn't ship its decoder
either).
- **Zeroth condition** — a *menu of registered standard codecs*, decoded with
the libraries a device already ships.
The capability a file requires is *named*, not supplied.
## Building
`wai-rs` builds the codec chain by default (the `codecs` feature), and those
crates link **system** libraries rather than vendoring them — consistent with
"WAI does not re-implement codecs". Without them the build stops in a codec
crate's build script, not in WAI's own code:
```sh
# macOS
brew install jpeg-xl libavif opus dav1d xz
# Debian / Ubuntu
apt-get install libjxl-dev libavif-dev libopus-dev libdav1d-dev liblzma-dev
```
To build without the C codec chain — the envelope, manifest and `world`
engine do not need it, which is also how the wasm32 target is built:
```sh
cargo check -p wai --no-default-features
```
With the feature `floor` alone, `wai-rs` decodes the mandatory floor (PNG, FLAC, zstd) with
no system library: PNG and FLAC in pure Rust, zstd from its reference source compiled in. The
`quickstart` feature adds `wai hello` and `wai verify`; it is how the prebuilt `wai` is built:
```sh
cargo install --locked --path wai-rs --no-default-features --features quickstart --bin wai
```
## What's here
| Path | Contents |
|------|----------|
| [`SPEC.md`](SPEC.md) | WAI v1.0 spec — envelope, manifest, capability menu |
| [`wai-rs/`](wai-rs/) | Rust reference implementation (lib + cdylib + staticlib + C FFI) |
| [`examples/`](examples/README.md) | Small runnable projects, one per template of `wai create`: a video room, watching together, messaging and a call, an encoder into a room, packaging, receipts in Rust and Python, native sinks on Apple and Android platforms, remote desktop |
| [`quickstart/`](quickstart/README.md) | The prebuilt command line's archive: what it holds, how it is built and released, and its installer |
| [`catalogue/`](catalogue/README.md) | The operation catalogue as JSON (`wai ops export`): every operation of the toolchain, `noun.verb`, and the name each door gives it |
| [`REGISTRY.md`](REGISTRY.md), [`registry/capabilities.json`](registry/capabilities.json) | The registry: every registered string with its kind, status, classes and edition, generated from the reference implementation (`wai registry export`), with its JSON Schema and the policy for registering a string |
| [`vocab/`](vocab/) | RDF vocabulary: v2 generated from the registry, v1 frozen |
| [`extensions/`](extensions/) | Sub-specifications (confidentiality, receipts, worlds, film, quantum operations, integer operators, vector graphics, …) |
| [`research/`](research/) | Research notes and dated records: graded, not normative |
| [`glossary/`](glossary/) | Reference catalog of media codecs surveyed for capability registration. [`codecs.json`](glossary/codecs.json) is the dataset, linked to the registry; [`SCHEMA.md`](glossary/SCHEMA.md) defines the shape and contribution rules. Rendered at [wai.transaction.science/glossary](https://wai.transaction.science/glossary). |
| [`CONTRIBUTING.md`](CONTRIBUTING.md) | How to change the specification, register a capability and add tests CI runs |
| [`SECURITY.md`](SECURITY.md) | How to report a vulnerability, and where the reference implementation reads untrusted input: the manifest limits, the native codec libraries, module authorisation, the fingerprinting risk of advertised parameter sets |
## Quickstart
The command line, with the relay and the room service it runs locally: the prebuilt archive
([`quickstart/`](quickstart/README.md), built by `quickstart/build.sh`), or from source.
```bash
wai hello # name, dispatch, rebuild and receipt, on a sample
wai create my-room --template video-room # a project from one of the examples (examples/)
cd my-room && wai dev # the relay, the room service and your page, on 127.0.0.1:8080
wai verify ../wai-hello # the receipts hello wrote, checked offline
```
[`examples/`](examples/README.md) holds each template, with a test that runs it.
Every command is an operation of one catalogue, named `noun.verb` (`envelope.inspect`,
`project.create`): `wai ops list` lists them, `wai ops search "<task>"` finds one offline, and
`wai ops export` writes the catalogue as JSON ([`catalogue/ops.json`](catalogue/README.md)).
Every command takes `--json` (one JSON envelope on stdout), `--dry-run` where it would write,
and `--yes` where a change waits for confirmation (`--confirm`, deprecated, does the same), and
exits with one of eight codes (`wai --help` lists them). `wai` sends nothing off your machine
unless you name a remote service.
The library's own checks, from a clone:
```bash
cd wai-rs
cargo test --no-default-features --lib --test carriage # registry, envelope, every capability round-trips
cargo run --no-default-features --bin wai -- capabilities # the registry, one row per capability
cargo build --release --lib # with the codecs feature: libwai.dylib/.so/.dll + libwai.a
```
Two commands hold what is published to what was published before, and other implementations
to WAI's corpora:
```bash
wai breaking --against origin/main # what this tree breaks: wire, record and API (CI runs it on every pull request)
wai conform verifier --suite claim --impl "my-verifier verdicts {corpus}" # a corpus run against your implementation
wai verify claim.wai-conform.json # the signed result, checked
```
A break passes only when [`CHANGELOG.md`](CHANGELOG.md) names it: an entry this change adds that
says **Breaking** and gives the finding's id in backticks. `wai conform --list` names every kind
(verifier, decoder, packager, origin) and suite, with its adapters: a command line, or an
`http://` URL.
With the `codecs` feature, `wai-rs/build.rs` finds libFLAC through pkg-config
and, on macOS without pkg-config, in the Homebrew "flac" formula prefix
(`/opt/homebrew/opt/flac/lib` or `/usr/local/opt/flac/lib`). Where neither finds
it, set `RUSTFLAGS="-L <the directory holding libFLAC>"`.
## For coding agents
`wai setup-agent` makes a project ready for a coding agent to use WAI. In the project directory
it writes `AGENTS.md`, the instructions agents read, adds an import of it to a `CLAUDE.md` the
project already has, and registers `wai mcp` in the project's MCP configuration: `.mcp.json`,
and the project-level MCP file of each agent whose directory the project already holds (`--all`
writes every one it knows, `--dry-run` lists them, `--toolset core,dev` registers the server
with those toolsets). A file that does not exist is created; one that exists is changed only
after its diff is shown and the change confirmed (asked on a terminal, or `--yes`); without
either, nothing that exists changes and the command exits 4 with each proposed change and the
command that confirms. It writes nothing outside the project directory. With `--json` its
output is the toolchain's JSON envelope.
`wai mcp` is a Model Context Protocol server on standard input and output. Each tool is an
operation of the toolchain, named by it with `_` for the dot, and the tools come in toolsets:
| Toolset | Tools |
|---|---|
| `core` (the default) | `envelope_inspect`, `registry_list`, `registry_show`, `receipt_verify`, `envelope_wrap`, `conformance_list`, `conformance_run`, `docs_search` |
| `dev` | `dev_start`, `dev_stop`, `room_grant`, `room_join`, `room_leave`, `session_show` |
| `app` (a build with the feature `app`) | `app_guide`, `app_check`, `app_stream`, `app_export`: Create's, below |
`receipt_verify` verifies a delivery receipt, a claim, a live-room session or a stream audit
bundle, and answers with the verifier's verdict and reason code. The `dev` tools, marked
development only, act on the target `local` alone: they start a relay and a room service on
loopback (`wai-room dev`), mint grants for test principals, join and publish as them, and
read the session for verification; the grant tool refuses any service that is not on a loopback
address. `--read-only` offers only the tools that read or run a verifier; the packaging tool
writes only inside the project directory. No tool sends anything off the machine.
```bash
cd wai-rs
cargo install --path . --no-default-features --features mcp # `wai` with `wai mcp`, no codec libraries
cd /path/to/your/project && wai setup-agent --toolset core,dev
```
The quickstart's `wai` holds `wai mcp` too.
**Create** ([`wai-app`](wai-app/)) builds a WAI app from a description: a model writes it in
joule-ui's line language and the page renders it as it streams. Its program, `app.oui`, becomes
a project with `wai create my-app --from app.oui` (a build with the feature `app`): an ordinary
WAI project that `wai app check` checks and `wai dev` runs.
The room service's HTTP API is described, OpenAPI 3.1, in [`joule-moq-room/openapi.json`](../joule-code/joule-code-rs/crates/joule-moq-room/openapi.json),
each operation's `operationId` its name in the toolchain's catalogue.
## SDK consumption
`wai-rs` exposes a stable C ABI in [`src/ffi.rs`](wai-rs/src/ffi.rs) —
the cdylib drops in as `libwai.dylib`/`.so`/`.dll`. Bindings for any
language (Python ctypes, Node N-API, Swift, JNI, Go cgo, etc.) call
into:
- `wai_image_{png,jpeg,avif,jxl}_{encode,decode}`
- `wai_audio_{opus,flac}_{encode,decode}`
- `wai_text_{zstd,xz}_{encode,decode}`
- `wai_envelope_{pack,unpack}` — wrap/unwrap the WAI container
- `wai_buffer_free`, `wai_last_error` — memory + error handling
## In the browser, on older engines: the reach builds
`wai-web` is the same Rust compiled to wasm32 for a page. Its shipped package needs
WebAssembly with sign-extension, non-trapping float-to-int and bulk memory, and i64
values crossing the JS boundary as `BigInt`. The reach builds
([`wai-web/reach/`](wai-web/reach/README.md)) run that code where those are missing, and
give the same outputs byte for byte:
| rung | build | for engines with |
|---|---|---|
| shipped | the package | every feature above |
| L1 | the shipped binary, legalised at the JS interface | the same WebAssembly, no `BigInt` at the boundary |
| L2 | MVP-lowered and legalised | MVP WebAssembly |
| J | the wasm2js translation, ES5 | ES5 and typed arrays, no WebAssembly |
L2 and J also come in reduced profiles: `receipt`, `verify` and `verify_decode`.
`wai-load.js`, an ES5 loader, picks the rung from feature probes alone and says why. A
rung can be forced for testing. `reach/build.sh` makes every build with pinned tools
and records each output's SHA-256 in `reach/manifest.json`.
CI runs each rung on a Node image of the engine line it serves, and every output must
equal the shipped build's; the reach builds made from the package of record (kept in
`reach/record/`) must equal `reach/manifest.json`. The images stand in for device classes;
no real old device has been run.
## Registered capabilities (the zeroth menu)
| Capability | Codec | Lossless? | Mandatory? |
|---|---|---|---|
| `wai.image.png` | PNG | yes | **mandatory floor** |
| `wai.image.jxl` | JPEG-XL | both | recommended |
| `wai.image.avif` | AVIF (AV1-based) | lossy | recommended |
| `wai.image.jpeg` | JPEG | lossy | recommended (legacy compat) |
| `wai.audio.flac` | FLAC | yes | **mandatory floor** |
| `wai.audio.opus` | Opus | lossy | recommended |
| `wai.video.av1` | AV1 | lossy | recommended |
| `wai.video.av1.lossless` | AV1 (`quantizer=0`) | YUV-lossless | recommended |
| `wai.text.zstd` | zstd | yes | **mandatory floor** |
| `wai.text.xz` | XZ/LZMA2 | yes | recommended |
Neural capabilities (`wai.neural.*`) are OPTIONAL and sink-advertised. Most
run float inference, so their decode is exact only on the build that produced
it. A distinguished subset is **integer-exact by construction** — an all-`i64`
decode with no float and no ONNX — so the reconstructed artifact is
byte-identical on every machine and its hash is portable. Conformance is a
hash: a deployer can require `intent=replicate` to dispatch only to these.
| Capability | Integer-exact decode of | Conformance |
|---|---|---|
| `wai.neural.int_hyper` | image — integer hyperprior σ source + integer entropy + integer synthesis | byte-exact (hash-portable) |
| `wai.neural.int_mlicpp` | image — MLIC++ entropy topology (channel slices × checkerboard passes, an entropy network re-run before each pass) on integer networks | byte-exact (hash-portable) |
| `wai.neural.int_synth` | image — integer synthesis only | byte-exact (hash-portable) |
| `wai.video.int_hyper` | video — `int_hyper` I-frame + integer half-pel motion-compensated P-frames | byte-exact (hash-portable) |
| `wai.audio.int_codec` | audio — learned 1-D transform, integer synthesis → int16 PCM | byte-exact (hash-portable) |
| `wai.splat.int_codec` | 3D — learned Gaussian-splat-attribute transform, integer synthesis | byte-exact (hash-portable) |
Their payloads are binary formats SPEC §5 pins — `WIV1` for the integer video
codecs, `WIH1`, `WIA1` and `WIS1`
([`extensions/integer-payloads.md`](extensions/integer-payloads.md)). The JSON the
encode and export tools write is converted to them, never carried. The same extension
writes down the `wai.neural.int_synth` payload and the `WIM1` payload of
`wai.neural.int_mlicpp`. Their networks' integer arithmetic is written down as the
operator set `wai-ops/1`
([`extensions/integer-operators.md`](extensions/integer-operators.md), Draft).
## Interactive & temporal content (extensions)
Beyond the codec menu, WAI carries content that consumes input or moves
over time — reconstructed from instructions at the sink, over one
`wai.det.fixed64` fixed-point floor, each with its own deterministic
`*-equivalence` conformance criterion:
| Capability | What it carries | Conformance |
|---|---|---|
| `wai.world.{scene,replay,live}` | interactive worlds (signed op log) | replay-equivalence |
| `wai.world.model` | action-conditioned world model (signed action ops → next frame, integer transition + synthesis) | replay-equivalence |
| `wai.mesh.glb` / `wai.splat.spz` | 3D mesh / Gaussian-splat assets | content-addressed |
| `wai.audio.scene` | object-based spatial audio | mixdown-equivalence |
| `wai.audio.score` | music as note events (synthesized at the sink) | mixdown-equivalence |
| `wai.haptic.signal` | keyframed touch envelopes | sample-equivalence |
| `wai.feed.state` | scoreboards / telemetry / market feeds | replay-equivalence |
| `wai.avatar.pose` | digital humans (skeletal pose stream) | pose-equivalence |
| `wai.splat.4d` | volumetric / 4D temporal Gaussian splats | frame-state-equivalence |
| `wai.film.linear` | animation / CGI film (sealed world + camera track) | film-equivalence |
| `wai.film.reel` | a film with cuts (one world, a sequence of shots) | reel-equivalence |
And the verify half — a signed, energy-accounted lineage for opaque
captured content WAI does **not** reconstruct (`wai.asset.provenance`).
See [`SPEC.md`](SPEC.md) §5 and [`extensions/`](extensions/) for each.
## Quantum — deterministic, native and in cloud WASM
WAI carries a full quantum stack in pure Rust: circuit simulation and the whole
operational layer around it, every stage byte-exact and sealable into a signed,
energy-accounted receipt. No QPU and no cloud service. Because it is
pure Rust with no host bindings, **one source builds for native and
`wasm32-wasip2` and produces byte-identical results** — the same circuit
reconstructs to the same statevector hash on a laptop or at the edge.
`wai.quantum.circuit2` and `wai.quantum.circuit` are the registered payload
capabilities, and `wai.quantum.calibration` and `wai.quantum.job` are registered
records; the other names below are module labels ([`REGISTRY.md`](REGISTRY.md)
lists the receipt labels no revision registers). The two circuit capabilities
share the container and the gates and differ in the phase table: circuit2's holds
the integers nearest to each phase and is the default for new content, and
`wai.quantum.circuit` is frozen so that every file, hash and receipt made with it
reproduces ([quantum-sim](extensions/quantum-sim.md) §8).
| Module | What it does | Conformance |
|---|---|---|
| `wai.quantum.circuit2` | byte-exact statevector simulation (dyadic Clifford+T+P(k)), with the correctly rounded phase table | statevector-equivalence |
| `wai.quantum.circuit` | the same, with the frozen phase table existing content was made with | statevector-equivalence |
| `wai.quantum.toolchain` | four backends (statevector / stabilizer / tensor-network MPS / sparse-Pauli), algorithm library, OpenQASM 3.0 in-and-out, **QIR export + ingest**, Bloch + entanglement analysis | statevector-equivalence |
| `wai.quantum.compile` | Clifford routing + stabilizer-tableau equivalence proof | signed equivalence certificate |
| `wai.quantum.cal.*` | calibration, filter-function robust control, DD noise spectroscopy | reproducible + receipt |
| `wai.quantum.cal.noise_learn` | Cycle Benchmarking / CER sparse Pauli–Lindblad learning | signed noise model |
| `wai.quantum.mitigate` | ZNE + readout M3 + classical shadows | byte-exact post-processing |
| `wai.quantum.decode` | CSS codes (rotated surface, toric, bivariate-bicycle) + union-find and Relay-BP decoders | signed decode + joules |
| `wai.quantum.rearrange` | neutral-atom register prep (Hungarian / LSAP) | signed defect-free plan |
| `wai.quantum.phasor` | interference substrate + energy-accounted phasor receipts | reproducible f64 |
| `wai.quantum.kernel` | the quantum kernel deployed as phasor features | reproducible f64 |
| `wai.quantum.qdata` | learning-from-quantum-data sample-complexity separation | reproducible f64 |
| `wai.quantum.qbom` | Quantum Bill of Materials — stage receipts into one provenance manifest | signed manifest |
### Quickstart
The command line is [`wai-quantum-cli`](wai-quantum-cli/README.md): OpenQASM
programs run here as the quantum job service runs them, each result naming its
method and accuracy and signed with this user's key.
```bash
cd wai-quantum-cli && cargo build --release # or: cargo install wai-quantum-cli
W=./target/release/wai-quantum
$W run bell.qasm --shots 1000 -o "Z0 Z1" # planned as the service plans it
$W run bell.qasm --json > job.json && $W verify job.json
$W plan prog.qasm; $W lint prog.qasm; $W draw prog.qasm
$W submit prog.qasm # on the job service
$W lab list # the algorithm catalogue
$W lab run ghz -n 5 # simulate + hash + outcomes
$W lab qasm bell # emit OpenQASM 3.0
$W lab qir qft -n 4 | $W lab run --qir # QIR out, and back in
$W lab receipt ghz -n 3 | $W verify
$W lab run qft -n 5 --gateset v1 # the frozen wai.quantum.circuit
```
Circuits built by `lab` are `wai.quantum.circuit2` unless `--gateset v1` asks for
`wai.quantum.circuit`, which reproduces the circuit and statevector hashes
earlier releases gave.
### Same source, at the edge
```bash
cd wai-quantum-cli && cargo build --release --target wasm32-wasip2 --no-default-features
wasmtime run target/wasm32-wasip2/release/wai-quantum.wasm lab run ghz -n 5
wasmtime run target/wasm32-wasip2/release/wai-quantum.wasm lab receipt ghz -n 3 --demo-key
```
Both print the identical `state-hash`. A receipt sealed in cloud WASM verifies
natively and vice-versa, and with the same key (here the published demo key)
the receipt bytes themselves are identical: the signature travels because the
computation is reproducible, not because the two runtimes agreed to trust each
other.
### As a serverless function
[`wai-quantum-serve/`](wai-quantum-serve/) exports the same stack as a
**WASI-HTTP component** (`wasi:http/proxy`), so quantum computation runs as an
ordinary HTTP handler on a component-model host:
```bash
cd wai-quantum-serve
cargo build --release --target wasm32-wasip2
wasmtime serve -S cli=y --addr 127.0.0.1:8231 \
target/wasm32-wasip2/release/wai_quantum_serve.wasm
```
```text
GET /health GET /algorithms
GET /run?algo=ghz&n=5&top=8 POST /run (body = OpenQASM 3.0)
GET /qasm?algo=bell GET /receipt?algo=ghz&n=3[&shots=&seed=]
(/run, /qasm and /receipt also take &gateset=v1|v2)
POST /verify (body = receipt JSON; 200 valid / 409 invalid / 400 unparseable)
```
The endpoint returns the **same** `statevector_hash` as the native CLI for the
same circuit, and a receipt sealed by the HTTP handler verifies with the native
`wai-quantum verify` — the JSON content is byte-identical (the CLI adds a
trailing newline; nothing else differs). Both verify paths accept a receipt in
either form: as sealed, or labelled with its figure's acquisition class, as
`wai_quantum_meter` emits a measured figure (`quantum_energy::MaybeLabelled`,
[energy-measurement](extensions/energy-measurement.md) §2.8). Both restate the
class, or say the figure has none.
That `wasm32-wasip2` build needs `-S cli=y`, because Rust's std imports
`wasi:cli/environment` at startup and the bare `wasi:http/proxy` world does not
provide it. For a component that runs on **any** `wasi:http` host with no host
flags at all, build the pure proxy-world artifact instead:
```bash
./build-proxy.sh
wasmtime serve --addr 127.0.0.1:8080 target/wai_quantum_serve.proxy.wasm
```
That path compiles a `wasm32-wasip1` core module and adapts it with the Bytecode
Alliance proxy adapter, leaving only imports the proxy world defines
(`wasi:http/types`, `wasi:io/*`, `wasi:cli/std{in,out,err}`, `wasi:random`). The
script fails the build if `wasi:cli/environment` ever reappears, and the served
results are byte-identical either way.
Explore the whole stack in the browser (it is the same Rust, compiled to wasm):
the [toolchain IDE](https://wai.transaction.science/quantum-ide), the
[ML layer](https://wai.transaction.science/quantum-ml), and a
[working course](https://wai.transaction.science/quantum-learn).
## License
Apache-2.0 — see [`LICENSE`](LICENSE). Implementing a registered capability
carries that capability's own licence class (royalty-free, patent-pool,
proprietary or non-commercial), recorded in the registry. Companion
implementations outside this tree keep their own licences.
Reference in a new issue View git blame Copy permalink