open-standards/wai/CHANGELOG.md
Transaction Science b5b7bd3686
Some checks failed
standards workspaces build + test / jouleclaw-rs (pull_request) Successful in 3m28s
standards workspaces build + test / aac-rs (pull_request) Successful in 44s
standards workspaces build + test / joule-code-rs (pull_request) Successful in 49s
standards workspaces build + test / joulecontract-rs (pull_request) Successful in 41s
standards workspaces build + test / map-rs (pull_request) Successful in 35s
standards workspaces build + test / mesh-rs (pull_request) Successful in 30s
joule-code conformance gate / Build + round-trip C1-C4 vectors (pull_request) Successful in 1m29s
standards workspaces build + test / openpay (pull_request) Successful in 1m37s
wai cross-architecture byte-equality / linux-aarch64 (pull_request) Successful in 18s
jouleclaw public surface must stay documented / Public-surface audit (jouleclaw-rs) (pull_request) Successful in 58s
no-vanity-metrics / Scan committed Markdown for vanity metrics (pull_request) Successful in 3s
WAI conformance / cargo-class corpora · linux-arm64 (pull_request) Successful in 56s
WAI conformance / cross-arch byte-equality · linux-arm64 (pull_request) Successful in 17s
standards workspaces build + test / eoc-rs (pull_request) Successful in 1m48s
standards workspaces build + test / jcp-rs (pull_request) Successful in 1m19s
standards workspaces build + test / proof-rs (pull_request) Successful in 1m11s
standards workspaces build + test / sandbox-rs (pull_request) Successful in 2m7s
standards workspaces build + test / smart-byte-rs (pull_request) Successful in 2m33s
wai cross-architecture byte-equality / linux-x86_64 (pull_request) Successful in 14s
WAI conformance / cargo-class corpora · linux (pull_request) Successful in 34s
WAI conformance / cross-arch byte-equality · linux (pull_request) Successful in 17s
WAI conformance / C2PA manifest emission · signed, read back, validated (pull_request) Successful in 42s
WAI neural decoders end-to-end / synthetic fixtures → Rust decode → reference (±4) (pull_request) Failing after 1m5s
standards workspaces build + test / arl-rs (pull_request) Successful in 2m0s
standards workspaces build + test / joule-ui-rs (pull_request) Successful in 2m15s
standards workspaces build + test / jouledesk-rs (pull_request) Successful in 2m20s
standards workspaces build + test / joulehook-rs (pull_request) Successful in 2m14s
standards workspaces build + test / sense-rs (pull_request) Successful in 2m16s
wai cross-architecture byte-equality / macos-arm64 (reference) (pull_request) Successful in 14s
WAI conformance / cargo-class corpora · macos (pull_request) Successful in 47s
WAI conformance / cross-arch byte-equality · macos (pull_request) Successful in 14s
wai cross-architecture byte-equality / windows-x86_64 (pull_request) Has been cancelled
WAI conformance / cargo-class corpora · windows (pull_request) Has been cancelled
WAI conformance / cross-arch byte-equality · windows (pull_request) Has been cancelled
wai: reconcile the WebCodecs mapping with capability constraints
This mapping was written on 2026-08-03 and sat unmerged while
`extensions/capability-constraints.md` landed, which makes dispatch a
precondition check: a sink satisfies a capability when it advertises `cap` AND
the object's declared parameters satisfy a registered constraint set. Rebased
onto current main, the two extensions read as overlapping claims about §4 with
nothing saying how they relate.

They are not in tension, and the newer one makes this one sharper.
`isConfigSupported()` is not a codec-string test — it takes a whole
`VideoDecoderConfig`, codec string plus `codedWidth`, `codedHeight` and the rest
— so it is a constraint evaluator already, and a browser satisfies the
constraint extension's three requirements structurally rather than through extra
machinery: the query is pre-flight, so evaluation happens before dispatch rather
than during it; it answers for a configuration, so the answer is per object and
not per sink; and `supported: false` is a negative answer returned before any
payload is committed to, which is the required treatment of an object that
satisfies no set. The call also returns the normalised config the platform would
accept, which is the accepted parameter set, so a sink SHOULD use it rather than
re-derive one.

The limit is stated rather than glossed. WebCodecs' parameter vocabulary is the
conventional-media one — codec, resolution, frame rate, bit depth, profile —
which is the same shape as the ratified registers `capability-constraints.md` §3
draws on, and it has the same gap: no WebCodecs configuration describes the
neural, quantum, world or haptic families. Those have no native tier to
constrain at all, resolve through the WASM tier, and stay unconstrained until
WAI authors their register entries. That is the state §3 already describes, not
a shortfall introduced here.

Framing renumbered §5 → §6. Docs only; `check-no-vanity-metrics.sh` passes.

Co-Authored-By: Claude Opus 5 <[email protected]>
2026-09-21 13:49:10 -04:00

53 KiB
Raw Permalink Blame History

WAI — changelog

Notable changes to the WAI standard and the reference implementation. Newest at the top. WAI is a draft standard; until v1.0 freezes, the wire format may change and such changes are recorded here.

v0.4.0 — current

Determinism tiers — naming the reproducibility spectrum for neural decode

  • New [extensions/determinism-tiers.md]: names the two guarantees a neural capability can make, so a manifest can declare which it satisfies and a verifier knows what it may check. entropy-consistency — the entropy decode reproduces across devices (integer entropy arithmetic or explicit parameter transmission), the guarantee the base learned-codec standards make (JPEG AI's normative conformance point is the entropy-decoder output) and the mainstream cross-platform fix; the floor for a sound neural replicate. decode-equivalence — the whole decode to the final medium reproduces, so the rendered result has a portable hash, the tier WAI adds on top and the precondition for a hard reconstruction binding. Referenced from SPEC §7 (Replicate-soundness); framed self-referentially.

Browser capability negotiation — WebCodecs mapping

  • New [extensions/webcodecs-negotiation.md]: how a browser sink implements §4's "advertises cap" test — the platform's isConfigSupported() for the native tier, a WebAssembly decoder as the always-satisfiable fallback. Grounds WAI's dispatch in the browser's own capability-negotiation pattern (ask if a config is supported; fall back to wasm) and documents what the live wasm sinks already do: native codec when the platform reports it, WAI's pure-integer wasm decoders otherwise, the classical floor always reachable. Referenced from SPEC §4. Its §5 reconciles the mapping with [extensions/capability-constraints.md], which landed later: isConfigSupported() takes a whole VideoDecoderConfig, so the native tier is already a per-object constraint check evaluated before dispatch — and it carries the same register gap, since no WebCodecs configuration describes the neural, quantum, world or haptic families.

Reproducible-reconstruction binding — a C2PA/JPEG-Trust hard binding over pixels

  • New wai.* provenance capability: reconstruction_binding (feature provenance) + [extensions/reconstruction-binding.md]. A cryptographic hard binding over the deterministically decoded pixels of a neural-coded asset — which such media normally can't carry, because a neural decode isn't reproducible (JPEG AI's normative conformance point is the entropy-decoder output, not pixels; its multi-branch design reconstructs one codestream several ways). WAI's byte-exact decode restores a portable decoded-pixel hash. SHA-256 (the C2PA default) over the canonical RGB8 buffer plus a descriptor pinning the reproducible decode; carried as a custom C2PA/JPEG-Trust assertion (science.transaction.wai.hash.reconstruction) beside the standard byte-binding. A substrate for machine-readable AI-content marking (e.g. EU AI Act Art. 50).

wai.image.jpegai now dispatches to the byte-exact decoder

  • decode_envelope prefers the deterministic jpegai_decode::decode when the model directory ships the integer weight bundle (weights/*.json + norm.json), falling back to the ONNX-runtime decode otherwise — existing bundles keep working. wai.image.jpegai is now registered in SPEC §5 (it was a live capability missing from the normative menu).
  • The cross-arch byte-equality matrix now covers the merged decoder's entropy stage (byte-exact-conformance), and that crate's build — broken on a moved rans.rs include — is restored.

JPEG AI — a byte-exact codestream→RGB decoder (jpegai_decode::decode)

  • wai::jpegai_decode::decode(codestream, &DecoderWeights, &NormTables) -> Rgb reconstructs an ISO/IEC 6048 JPEG-AI codestream to RGB through a fully integer entropy front end and a deterministic value/synthesis/post-filter path — one pinned reconstruction, reproducible byte-for-byte on every machine, so the decoded pixels have a portable hash. No per-image fixtures: every per-image quantity is derived from the bitstream; only checkpoint constants come from a sink-supplied model bundle.
  • Determinism profile: the entropy decode (mANS), the dequant, and the derived entropy inputs (skip mask, σ index, RVS band, dequant gain) are integer-exact against the reference; the PTQ value/synthesis stages are deterministic and land within 1 LSB of the ISO reference decoder. Distinct from the in-house wai.neural.int_* codecs (all-i64, hash-portable): this decodes a published external standard's codestream deterministically.
  • Modules: jpegai_decode (the composed entry point) over jpegai_bitstream, jpegai_mans, jpegai_pipeline, jpegai_synthesis, jpegai_icci (feature neural_int). Inputs are regenerable from the shipped assets via tools/wai_jpegai_build_e2e_inputs.py. The whole chain — codestream → RGB — is exercised end to end against the reference PNG.

JPEG AI — a live, self-sufficient wai.image.jpegai decode (2026-06-22)

  • neural::jpegai::decode(codestream, &JpegAiModel) decodes a real ISO/IEC 6048 codestream to RGB with no per-image schedule fixtures: every per-image quantity is derived from the bitstream, and only checkpoint constants come from a sink-supplied model bundle. Wired into decode_envelope — JPEG AI is now a dispatchable capability, not just reference/conformance support.
  • The two derivations that closed it, traced bit-exact from the reference: the residual coding mask is sigma_skip > 382 (the σ surface before the rvs addition — on the bucketed index it looks ambiguous, on sigma_skip it is exact, 0 violations / 65536 positions); the dequant is resi × rvs_dq[grfs[ch]][lk] / 2^16 / scaler_vec[ch], with grfs determined the same way the σ rvs does so the per-channel scaler_vec is consistent. Matches the reference decoder's PNG to within rounding (max diff 1).

Film reel — wai.film.reel, cuts over one world (2026-06-17)

  • The editor's layer over wai.film.linear: a WREL embeds one sealed world
    • a sequence of shots (each its own camera track + frame schedule) played back to back. Cutting wide→close, or one beat from two angles, is film grammar; a reel carries it as a few kilobytes of camera keys, the world shared across every shot. Conformance is reel-equivalence — film-equivalence extended over cuts.
  • Shots MAY revisit earlier world ticks (a cut back in time from a new angle), so the world is sampled at the sorted/de-duplicated union of all frame ticks — convergence still holds because replay-equivalence is order-independent.
  • Reuses the film machinery (shared camera interpolation + pose bytes) + the FilmReceipt profile (identity = reel_hash); wai_meter grows a .wrel branch. Registered + tiered Lossless. Corpus vector coverage (the character beat cut three ways, incl. a cut back in time) joins film-conformance/, verified by wai_film_vectors on the four-platform matrix. SPEC §5 row + extensions/linear-film.md §7.

Linear film — wai.film.linear (2026-06-16)

  • "A film is a sealed, camera-on-rails world." A generated film ships not as pixels but as the world that produces it (an embedded sealed wai.world.replay) + a camera track (keyframed pose over the world's ticks) + a frame schedule (which tick each frame samples). The sink renders the frames — lever 3 (instructions-at-the-sink) for the synthesizable slice of "video" (animation / CGI / motion graphics). The conformance corpus carries whole films in ~1 KB.
  • wai-rs film feature (reuses the world engine + Fx floor + blake3): the WFLM container (contract + embedded WWLD + camera track) and the deterministic reconstruct contract. Conformance is film-equivalence — per-frame BLAKE3(world_state_hash(frame_tick) || interpolated camera pose) is bit-identical on every machine. It composes the embedded world's replay-equivalence (inherited unchanged) with the camera track in the wai.det.fixed64 floor; the render (pixels) stays the sink's.
  • The world engine gained one general primitive: collect_checkpoints_ sampled / run_sampled_dispatch — read the state hash at an arbitrary frame cadence, independent of the contract's checkpoint cadence (a renderer samples the timeline at its own frame rate).
  • Corpus film-conformance/ + wai_film_vectors gen|verify (dolly, two-shot, crane-up) — all pass, on the four-platform CI matrix. Registered
    • tiered Lossless (film-equivalence is exact). FilmReceipt (per-frame Merkle leaves) + a wai_meter .wflm branch emitting measured joules.
  • Spec PROMOTED to extensions/linear-film.md, SPEC §5 temporal-content row, content-taxonomy updated (animation-film → ✅). The biggest commercial wedge in the taxonomy, now shipped.

Music as instructions — wai.audio.score, LIVE at /score (2026-06-14)

  • The deepest audio lever: a score carries no recording, only note events — ship the composition, not the recording. wai-rs score feature (reuses the world Fx floor + blake3): the WSCR container (contract + note events) and a trig-free deterministic reference synth — a phase accumulator drives a piecewise-linear oscillator (saw/square/triangle/pulse) shaped by a linear ADSR, all + − × ÷. Pitch is a per-sample phase increment authored off-line (no exponential in the floor). mixdown_hash is the mixdown-equivalence quantity, the music counterpart of spatial audio; a soundfont/neural instrument is the sink's presentation.
  • Corpus score-conformance/ + wai_score_vectors gen|verify (scale, chord/polyphony, timbre-changing arpeggio) — all pass. Registered + tiered Lossless (exact reference mix).
  • Browser sink ScoreSink (wai-web) + /score page, LIVE: synthesizes the score in-tab, verifies the mixdown hash vs the corpus, draws a piano roll, and plays the reconstructed PCM via Web Audio. wasm pkg uniquely named wai_score.
  • Spec PROMOTED to extensions/score.md (site mirror), SPEC §5 row, content-taxonomy updated (music → ✅). Completes the audio surface: recorded objects (audio.scene) and synthesized notes (audio.score).

Neural subsystem reconciled onto main (2026-06-14)

  • Wires the full learned-image/audio decoder family into neural/mod.rs and brings the categorical rANS (decode_categorical) it needs: hyperprior (mbt2018_mean), checkerboard context (elic), channel-autoregressive (mlicpp/mlicv2/cmic), integer-exact (int_synth), glc (VQ), inter-frame video (dcvc_rt/dcvc_fm), snac audio — plus the ISO/IEC 6048-1 JPEG-AI entropy + codestream
    • σ/value/filter port (jpegai_*).
  • Restores the semantic / generative path (semantic.rs, CLIP-similarity conformance, ModelRequirement.min_sim/sim_metric) and the shared deterministic ONNX loader (ort_session.rs), behind the semantic feature.
  • Re-adds byte-exact-conformance (cross-arch rANS + int_synth + JPEG-AI mANS goldens) and the model export/encode tooling under tools/.
  • All features build + test green together (neural + semantic + the interactive/temporal cargo classes); clippy clean.

Asset provenance — wai.asset.provenance, LIVE at /provenance (2026-06-13)

  • Lever 4 — the "verifies whatever it carries" half. For content that cannot be reduced to instructions (a photo, a recording), WAI does not reconstruct — it carries the opaque bytes content-addressed and attaches a signed, energy-accounted lineage. AssetReceipt (provenance.rs, provenance feature): a JWP group whose objects are processing steps (op + input/output content hash + work + measured joules), Merkle-bound, Ed25519-signed, with chain-linkage verification (each step's input = the previous output, final output = the asset hash) and parent_receipt_hash chaining. from_json so a verifying sink can re-parse it.
  • wai_prov tool: synthesizes a real image, builds a 3-step lineage (origin → PNG master → published JPEG) with the real measured CPU energy of each encode (IOReport via macmon, no sudo), wraps the JPEG as a content-addressed wai.image.jpeg envelope, and emits the signed receipt (origin 0 + PNG 0.43 J + JPEG 0.14 J = 0.57 J measured).
  • Browser sink — AssetProvenance (wai-web) + /provenance demo, LIVE: recomputes the asset's content hash in-tab and compares to the receipt (tamper-evidence — a "corrupt one byte" button flips it red), verifies the Ed25519 signature + Merkle chain, shows the energy- accounted lineage, and hands the carried JPEG to a native <img> (the browser decodes it; WAI does not). wasm pkg uniquely named wai_prov.
  • Spec PROMOTED to extensions/provenance.md (site mirror). It is a receipt profile about a carried asset (like jwp-receipts), not a §5 payload codec — the asset keeps its own capability (wai.image.jpeg). Completes the thesis: carries whatever it can reconstruct, verifies whatever it carries.

Cargo-class polish wave — real cloud, EWA render, receipts, telepresence (2026-06-13)

Four follow-on builds across the new cargo classes:

  • Real-SPZ-derived 4D clip (wai-spz-gen 4d): the bloom clip animates 1200 genuine Niantic-SPZ Gaussians (the same cloud the static wai.splat.spz asset ships, round-tripped through the SPZ library) — real cloud, authored motion. LIVE on /volumetric. Honest framing: the win is a verifiable per-frame state, not capture fidelity.
  • WebGPU EWA render on /volumetric: the page now renders the reconstructed cloud with a real anisotropic EWA splat pipeline (the same one the worlds demo uses) — per-Gaussian covariance projection, conic, 3σ bounding quad, premultiplied blend, back-to-front sort — with a 2D alpha-disc fallback where WebGPU is unavailable. The hash-verify badge is render-independent. Confirmed on WebGPU (Chrome) + 2D fallback.
  • Per-class receipts with measured joules: HapticReceipt (haptic/receipt.rs) + Splat4dReceipt (splat4d/receipt.rs) — the worlds/audio JWP receipt profile transposed to touch (per-channel blocks) and volumetric (per-frame blocks), reusing world::receipt:: merkle_root. New wai_meter bin measures real CPU energy (IOReport via macmon, no sudo) for haptic/splat4d reconstruction and emits a signed, self-verified receipt carrying the measured joules_micro; wai_world_meter now also dispatches feed0/skeletal0, so feed and avatar worlds get the same measured SessionReceipt. Every cargo class now has a measured-energy receipt path (haptic 57 µJ, splat4d 0.21 J for 1200 Gaussians × 17 frames, feed/avatar mJ-scale).
  • Avatar telepresence — LIVE at /telepresence: a live skeletal world of N avatars over the existing worlds relay. New SkeletalActor (wasm) signs CREATE + SET_ROTATION CausalOps; the worlds transport (relay-first + BroadcastChannel fallback, hello/sync/op/hash, boundary-hash convergence) is reused verbatim. Each tab creates and animates its own arm rig; sinks converge on the pose hash ("poses agree @ tick N"). Verified on prod through the real relay with two isolated contexts — identical pose hashes, zero errors.

Volumetric / 4D splats — wai.splat.4d, LIVE at /volumetric (2026-06-13)

  • Phase 5 complete — the roadmap frontier. A wai.splat.4d is a temporal Gaussian splat: the static splat extended to time. Each Gaussian carries a keyframed track (position / scale / rotation / color / opacity); the sink interpolates the cloud at frame N in the wai.det.fixed64 floor. Conformance is frame-state-equivalence: the reconstructed Gaussian set at frame N hashes identically on every sink, no tolerance — the temporal counterpart of the static splat.
  • wai-rs splat4d feature (reuses the world Fx floor + blake3, no codec/ML deps): the W4DS container (contract + per-Gaussian keyframe tracks, 14 Fx per key) + a deterministic reconstruct (Fx linear interpolation, hold outside the track). Trig-free (quaternions interpolate component-wise; the sink normalizes for rendering). frame_hash = the frame-state-equivalence quantity; clip_hash = frame-independent identity. Registered + tiered Lossy (consistent with wai.splat.spz — the splat representation approximates the captured scene, even though the reconstruction is frame-state-exact).
  • volumetric-conformance/ corpus + wai_splat4d_vectors gen|verify: orbit (two Gaussians swap places), pulse (scale + fade), morph (color shift) — every frame's hash checked, all pass.
  • Browser sink — Splat4dSink (wai-web) + /volumetric demo page, LIVE: reconstructs the cloud in-tab and verifies every frame's hash against the corpus; a frame scrubber re-derives the cloud, drawn as painter-sorted alpha Gaussians (a sink with WebGPU renders the same set with the full EWA pipeline — presentation). wasm pkg uniquely named wai_splat4d.
  • Spec PROMOTED to extensions/volumetric.md (site mirror), registered in SPEC.md §5 (wai.splat.4d). Honest about size: the win is content-addressing + dedup + a verifiable per-frame state, NOT payload reduction (a real cloud is MB-scale) — stated plainly.
  • The cargo set is complete across the medium: sight (worlds/replay), hearing (audio/mixdown), touch (haptics/sample), live state (feed/replay), a moving body (avatar/pose), and now a moving volume (splat.4d/frame-state) — one wai.det.fixed64 floor, one verification.

Avatars / digital humans — wai.avatar.pose, LIVE at /avatar (2026-06-13)

  • Phase 4 complete — digital humans: motion as instructions. An avatar is a skeletal world — the worlds container/op-log/ordering/ sealing/state-hash/receipt-chain, with the step reduced to the identity. Conformance is pose-equivalence (the per-bone local transforms hash identically), a special case of replay-equivalence.
  • wai.world.sim.skeletal0 (in wai-rs world feature): a bone is a parent[16] + translation[3] + rotation-quaternion[4] + scale[3] (all wai.det.fixed64). Mutations: CREATE / DELETE / SET_ROTATION (the animation driver) / SET_TRANSLATION / SET_SCALE, all LWW. Trig-free floor: rotations are quaternions and the FK composition is + − × only; axis-angle authoring happens off-line. The forward-kinematics → world space and the skinning of a wai.mesh.glb are presentation (a pure function of the hashed pose), never in the hash. Registered + tiered Lossless; dispatched in run_sealed_dispatch.
  • avatar-conformance/ corpus + wai_avatar_vectors gen|verify: a sealed 3-bone arm wave, a sealed 5-bone humanoid pose, and three seeded adversarial-arrival shuffles of the arm wave — all converge, all pass.
  • Browser sink — SkeletalSink (wai-web) + /avatar demo page, LIVE: replays a sealed animation in-tab, re-derives the sealed pose hash (checked against the native corpus and the file's own seal), and exposes FK world-space joints + bone segments to draw the skeleton on a rotating orthographic canvas with a tick scrubber. wasm pkg uniquely named wai_avatar.
  • Spec PROMOTED to extensions/avatars.md (site mirror), registered in SPEC.md §5 (wai.avatar.pose). A moving body joins sight (worlds), sound (audio), touch (haptics), and live state (feeds) under one floor.

State-feeds — wai.feed.state, LIVE at /feed (2026-06-13)

  • Phase 3 complete — live state-feeds: state as instructions. A feed is a non-physics world — the same WWLD container, op log, canonical ordering, sealing, state hash, and receipt chain as interactive-worlds, with the simulation step reduced to the identity. Conformance is therefore replay-equivalence, reused wholesale: no new criterion.
  • wai.world.sim.feed0 (in wai-rs world feature): nodes are records of self-describing typed cells (Int i64, Fixed Q32.32, Text UTF-8 — hashed without a schema). Mutations: CREATE / DELETE / SET_CELL (LWW) and ADD_INT / ADD_FIXED (SUM — commutative accumulation). Wrong-typed / out-of-range / absent-node updates are deterministic no-ops. Registered + tiered Lossless in the central codecs registry; dispatched in run_sealed_dispatch.
  • feed-conformance/ corpus + wai_feed_vectors gen|verify: sealed scoreboard (SUM scoring + an LWW correction), telemetry (LWW gauge + SUM packet count, incl. a negative ADD_FIXED), market tape (LWW last + SUM volume), and three seeded adversarial-arrival shuffles of the scoreboard — all converge, all pass (hash equality).
  • Browser sink — FeedSink (wai-web) + /feed demo page, LIVE: loads a sealed feed, replays it in-tab with the same engine that made the corpus, and re-derives the publisher's sealed BLAKE3 state hash (the file's seal also equals the native corpus hash — both checked). A tick scrubber re-derives the feed at any tick; the record table renders the typed cells generically. wasm pkg uniquely named wai_feed (avoids the deploy fingerprinter's basename collision).
  • Spec PROMOTED to extensions/state-feeds.md (site mirror), registered in SPEC.md §5 (wai.feed.state). The broadcast profile (one authoritative publisher, many read-only sinks) is documented as a deployment shape of wai.world.live, not a new format. State joins sight (worlds/replay), hearing (audio/mixdown), and touch (haptics/sample) under one floor — verified the same way.

Interactive worlds — world feature + conformance corpus (2026-06-11)

  • New cargo class implemented: the interactive-worlds extension (spec draft staged in wai-transaction-science-web/research/, destined for extensions/interactive-worlds.md). Media that consumes input: a world is a content-addressed artifact, every event is a signed CausalOp, and conformance is replay-equivalence — identical BLAKE3 state hash at every checkpoint, no tolerance parameter. One container (WWLD) serves wai.world.scene / wai.world.replay / wai.world.live, differing only in op-log seal state; an input log and a multiplayer op log are the same object.
  • wai-rs world feature (pure Rust, no ML dep chain; deps: blake3 + ed25519-dalek): Fx Q32.32 fixed-point (wai.det.fixed64, the floor profile — bit-exact unconditionally), xoshiro256++ contract rng, signed/dep-gated ops with canonical (effect_tick, lamport, actor) ordering, coordination-free BLAKE3 node minting, WWLD section container with content-binding seal, Sim trait (the extension standardizes the contract, not a physics engine) + kinematic0 reference registrant, sealed Replayer + LiveWorld engines (buffering, past-tick re-derivation, equivocation rejection, receipted-checkpoint history sealing).
  • world-conformance/ corpus + wai_world_vectors gen|verify: sealed replays, LWW/SUM conflict batteries, dep-gap / past-tick / shuffled-arrival live streams, expected hashes. All vectors pass; clippy clean on the new code.
  • Spec PROMOTED to extensions/interactive-worlds.md (site mirror src/pages/extensions/), registered in SPEC.md §5: wai.world.scene / wai.world.replay / wai.world.live + asset media wai.mesh.glb (canonical glTF 2.0 Binary, Khronos — decoder sink/library-supplied); media: "world" noted in §3. Capability strings live in the central codecs registry (known_capabilities()), tiered Lossless — replay-equivalence is exact state reproduction, the simulation analog of lossless.
  • Second sim registrant wai.world.sim.ballistic0 (gravity + ground-plane bounce, per-node radius/restitution, G = −10, all Q32.32 — bit-exact through every bounce) proving the registration story: entered registry, dispatch, and corpus (ballistic_bounce vector) without one line of the extension changing. New run_sealed_dispatch routes on contract.sim; a missing sim reports the world inert (UnknownSimCapability), mirroring SPEC.md §4 step-6 semantics so callers can fall back per the envelope.
  • Native energy meter — measured joules, no sudo (2026-06-13): corrected a wrong assumption that real energy needed RAPL/sudo (unavailable on the arm64 prod box). On Apple Silicon the SoC energy counters are readable through IOReport with no privileges (the source macmon/asitop use; verified euid≠0 reads CPU/GPU/ANE power on the M3 Ultra), and proc_pid_rusage gives exact per-process retired cycles/instructions. New wai_world_meter bin (world feature): brackets a tight replay burst of a sealed .wwld against an idle baseline, samples CPU power via macmon, and reports the marginal measured CPU energy — e.g. the truss replay metered at ~512 µJ / replay (~4.3 µJ/tick) at 3.2 W marginal, a real number that fills a receipt's joules_micro where a native sink runs. The burst runs on N threads so the marginal draw clears a shared box's idle fluctuation (single-thread 512 µJ vs 10-thread 524 µJ/replay cross-validates the method — per-replay energy is invariant to parallelism). With a second arg the meter emits a full §8 SessionReceipt carrying the MEASURED joules + exact work_steps, signed and self-verified — real measured energy end to end. Spec §8 + the /worlds page corrected: the sink-metering path needs no sudo (IOReport on Apple Silicon, RAPL on x86); the browser estimate is the fallback, not the ceiling. clippy clean, 94 tests green.
  • Phase 2 begun — haptics (wai.haptic.signal) (2026-06-13): touch as instructions. Spec drafted (research/haptics-draft.md), capability registered + tiered. wai-rs haptic feature (reuses the world Fx floor, no codec/ML deps): the WHAP container + a deterministic reconstruct — per channel, a keyframed intensity/sharpness envelope (the Core-Haptics pair every actuator API maps to) linear-interpolated in Fx and quantized to u8. Trig-free by construction (amplitude + sharpness, not a modulated carrier), so the floor reconstructs it exactly with +,−,×,÷ alone. signal_hash is the sample-equivalence quantity. wai_haptic_vectors gen|verify + haptic-conformance/ (ramp, thud, dual-rumble pulse) — all pass, hash equality. 104 tests, clippy clean. Next: Gamepad/Vibration browser sink + /haptics demo, then promote. This completes the sight → hearing → touch trio's engines.
  • Spatial-audio: receipt profile + spec PROMOTED (2026-06-13) — Phase 1 complete. audio_scene::SceneReceipt (the worlds receipt transposed to audio): identity scene_hash, the JWP group's objects are the mix's PCM blocks (BLAKE3("wai:audio-block")), Merkle-bound with JWP's exact domains, one Ed25519 sig over scene_hash + root + blocks + work_mac (samples × objects, exact signed) + carried joules_micro; parent_receipt_hash chains renders. Reuses world::receipt::merkle_root — minimal new code. 2 receipt tests (101 total). Draft promoted to extensions/spatial-audio.md (+ site mirror, SPEC §5 row + extension pointer) — and the canonical spec corrected to the as-built sqrt_constpower pan (sqrt-only, no transcendental tables) and the real receipt shape, since registration froze the format. clippy clean; spec page live. Phase 1 (spatial audio) DONE: spec · register · mixer · corpus · live /audio sink · receipt · promoted. Next: Phase 2 — haptics.
  • Spatial-audio browser sink — LIVE at /audio (2026-06-13): wai-web AudioSink (wasm-bindgen): parse WSCN, load the corpus sources, run the exact deterministic mixdown, expose mixdown_hash_hex + scene_hash_hex + mix (i16 PCM) + object_xz(t). The /audio page (site-styled, in nav) loads a scene, verifies the mixdown hash in-browser against the corpus (mixdown-equivalence, the same proof worlds does for state), plays the bit-exact stereo mix via an AudioBuffer, and shows a top-down radar of the objects panning. Proven on prod: all three scenes verify, zero errors — wasm-in-Chrome mixes byte-identically to native. Honest split kept: plays the exact stereo floor; binaural/HRTF noted as the richer optional presentation.
  • Spatial-audio conformance corpus (2026-06-13): wai_audio_vectors gen|verify + audio-conformance/ — the mixdown-equivalence analog of the worlds corpus. Three vectors (static stereo pair, distance near/far, keyframed left→right sweep) ship a .wscn scene + a .sources.bin (decoded mono tones) + expected scene_hash and mixdown_hash; verify mixes each and checks hash equality, no tolerance. All pass; clippy clean.
  • Spatial-audio reference mixer — mixdown-equivalence (2026-06-13): wai-rs audio_scene feature (reuses the world Fx floor + blake3, no codec/ML deps): the WSCN container (scene contract + object table
    • asset table + optional seal, mirrors WWLD) and a fully deterministic mixdown — Fx position interpolation, inverse_clamped distance attenuation, sqrt_constpower pan (sqrt only, no transcendental tables), Fx accumulate, round-half-away i16 quantization in integer space. mixdown_hash (BLAKE3 over canonical i16LE PCM for a declared layout) is the mixdown-equivalence quantity; scene_hash is the identity. 5 tests prove it: centered = equal L/R, hard-left = silent right, deterministic + layout-bound hash, distance attenuates, a moving source sweeps L→R + container round-trips. Sources arrive as decoded mono PCM keyed by content hash — decoding the wai.audio.* assets is the sink's job. clippy clean. Next: conformance corpus bin + Web Audio browser sink + demo.
  • Content taxonomy + cargo roadmap; spatial-audio Phase 1 begun (2026-06-13): mapped the full space of web-transported content WAI's approach covers (research/content-taxonomy.md) — a capture→synthesizable→executable→ symbolic spectrum × four levers (codec dispatch, neural prior, instructions-at-sink, determinism+receipts), with a per-row mechanism/extension/status table and the white space called out. A phased build plan (research/cargo-roadmap.md) takes each white-space class through the proven interactive-worlds recipe. Phase 1 kicked off — object-based spatial audio (wai.audio.scene): the audio analog of interactive-worlds (ship sound objects + a scene, render to the sink's layout; mixdown-equivalence = replay-equivalence with the tick replaced by the audio sample). Spec drafted (research/spatial-audio-draft.md, WSCN container, Fx mixdown, content-addressed wai.audio.* sources, stereo bit-exact floor + binaural perceptual tier, WAI2 recorded-mix fallback, JWP receipt profile); wai.audio.scene registered + tiered Lossless. 94 tests green. Remaining phases: haptics, live state-feeds, digital humans, volumetric/4D splats.
  • Real SPZ decode — wai.splat.spz end to end (2026-06-13): the splat gallery now renders a genuine Niantic-SPZ file, not a mesh-derived stand-in. New wai-spz-gen crate depends on the field's spz reference library (the sink-supplied codec WAI dispatches to), encodes a 2,400-Gaussian cube cloud to a real .spz (validated by re-loading through the library), and wraps it as a content-addressed wai.splat.spz WAI object (nebula.spz.wai). gen-gallery adds it to the scene's asset table (role 1) beside the glb. Browser: WorldSink.asset_payload(name, env, cap) generalizes the §4.1 verify to any capability; the page gunzips (DecompressionStream) and decodes the SPZ bitstream in-tab — 16-byte header, non-interleaved arrays, 24-bit fixed-point positions, log scales, smallest-three quaternions, byte/255 opacity+color — into per-Gaussian world covariance for the EWA pipeline. Decoded count matches the encoded 2,400 (validates the layout); the cube silhouette + surface-aligned discs validate the quaternion math. Page copy corrected: the splat view decodes a real SPZ asset (no longer "derived from mesh"). Verified locally (2,400 Gaussians, zero errors); prod serves the identical artifacts.
  • Anisotropic EWA splatting (2026-06-13): upgraded the splat renderer from isotropic billboards to real 3D Gaussian splatting. Each splat carries an anisotropic 3×3 covariance (surface-aligned flat discs: wide in the triangle tangent plane, thin along the normal); the vertex shader projects it to a screen-space conic (Σ' = J·W·Σ·Wᵀ·Jᵀ with the perspective Jacobian), emits a tight 3σ bounding quad, and the fragment evaluates the conic — the textbook 3DGS pipeline, still CPU-sorted back-to-front and alpha-blended. Cam uniform carries view + proj separately (+ viewport/focal) for the Jacobian. Visibly sharper, surface-coherent splats; proven on prod, zero errors.
  • 3D render layer — PBR mesh + Gaussian splats (2026-06-13): the /worlds gallery now renders the content-addressed, hash-verified wai.mesh.glb two ways in a real 3D path (perspective camera + depth buffer, all in wai-web/demo/world.js, no wasm change): a solid PBR-shaded cube (flat normals from screen-space derivatives, Lambert
    • spec + sky-ambient, gamma) and a Gaussian-splat cloud — instanced camera-facing quads with a Gaussian falloff, back-to-front depth-sorted each frame (painter's order) and alpha-blended. The splat cloud is derived from the same verified mesh, making the thesis visible: one content-addressed asset, two sink-side renderings, presentation never touches the state hash. SPZ binary decode stays sink-supplied (stated honestly on the page); the renderer is the real deliverable. Two gallery vectors (gallery / gallery_splat); the 2D sim path is untouched. Proven on prod: both render, 768 splats from the cube, zero errors.
  • Receipt chaining + work-meter + wai.splat.spz (2026-06-12): three follow-ons. (1) SessionReceipt gains parent_receipt_hash (BLAKE3 of the prior receipt's canonical bytes — the JWP linkage) + receipt_hash() + verify_chain: successive sealed sessions of one world (shared world_hash, group_id+1) form an append-only signed timeline. (2) An honest energy meter: receipts carry exact, signed, deterministic work_steps (Σ node-steps, from collect_checkpoints_and_work) AND a separately carried joules_micro — never derived in the receipt; the browser supplies a wall-clock estimate at a declared coefficient (labeled "est@4W"), native sinks supply real RAPL, an unmetered sink signs 0. (3) Registered wai.splat.spz — Gaussian-splat scenes as an asset capability parallel to wai.mesh.glb (presentation only, never in the state hash; the win is content-addressing + dedup, not payload size; renderer sink-supplied). Browser seal_session(seed, id, joules, group, parent) chains via localStorage; the /worlds HUD shows chain #N ↳linked · N steps · ≈X mJ est@4W · ✓ re-verified. Proven on prod: two seals chain (g0 parent null → g1 parent = g0.receipt_hash), exact work 248 steps, est joules carried, zero errors. Spec §8 updated (chaining + the two-quantity energy model). 94 native tests, 10/10 vectors, clippy clean.
  • Asset path SHIPPED — wai.mesh.glb, content-addressed (2026-06-12, extension §4.1): worlds are made of WAI objects. New world::asset_content_hash (BLAKE3 wai:asset\x01 over a complete WAI envelope) + DOMAIN_ASSET; native tests PROVE the load-bearing claim — a scene with an asset table has the byte-identical state hash AND world hash as the same scene without it (assets are presentation-only, never in the state hash), and the asset hash binds the exact envelope. Generator gen-gallery writes a gallery.wwld scene + assets/cube.glb.wai: a real glTF 2.0 Binary cube (hand-written GLB writer) wrapped as a wai.mesh.glb WAI envelope and content-addressed in the table. Browser: WorldSink.assets() + asset_glb(name, envelope) (verifies content hash in WASM, rejects a mismatch, unwraps the WAI envelope to the .glb payload); the page decodes the GLB (minimal JS decoder) and renders the verified mesh as a spinning wireframe. Proven on prod: 1/1 verified ✓ · cube.glb, and the state hash stays fixed while the mesh spins — §4.1 made visible. 92 native tests, 10/10 vectors, clippy clean.
  • Session receipts + seal-to-replay SHIPPED (2026-06-12, extension §8 implemented): world/receipt.rs builds a SessionReceipt — a JWP profile, not a fork: op batches (one per checkpoint interval, content-hashed wai:world-ops\x01) are Merkle-bound with JWP's exact leaf/node domain separators, and one Ed25519 signature covers the world-profile payload (world hash + checkpoint state_hash list + carried microjoules). Joules are carried, never fabricated — the builder takes the figure; an unmetered session signs 0 and says so. New world_hash (BLAKE3 over scene = contract + tick-0 graph, the §7.4 shareable identity), collect_checkpoints (unsealed replay → checkpoint list). Browser: WorldSink.seal_session(seed, signer_id, joules) → SealedSession{replay_bytes, receipt_json, root_hash, …}; the /worlds page's "⏷ seal & download replay" turns a live, shared, edited world into a portable .wwld + .receipt.json, re-opens the fresh replay in a throwaway sink to prove it re-verifies, and shows the Merkle root + batch count in the HUD. Proven on prod: fracture a tower across the relay → seal → 3-batch receipt, valid sig, replay re-verified. 90 native tests, 10/10 vectors, clippy clean.
  • Cross-device multiplayer SHIPPED (2026-06-12, wai-relay/ + wai-web Actor binding): every tab/device is a signed actor. New Actor wasm-bindgen class (browser-held Ed25519 key; clicks author real CausalOps; Lamport discipline via observe_lamport); wai-world-relay reference transport (axum WS, per-room fan-out, retains the signed op log for late joiners, verifies every op's Ed25519 signature at the door with the same wai crate — it adjudicates nothing; sinks converge because the math does). Page transport is relay-first (room in the URL hash) with BroadcastChannel fallback. Deployed at wai.transaction.science/worlds-ws/* (launchd com.wai.world-relay, localhost:9670). Proven on prod with two ISOLATED browser contexts: point-blank axial shoves snap two members (23→21 nodes) and both contexts converge — "✓ hashes agree". Also fixed build.rs emitting libFLAC link flags regardless of features (pkg-config probe side effect): now gated on codecs.
  • Browser sink SHIPPED (2026-06-12, wai-web/src/world.rs + wai-web/demo/world.{html,js}): a wai.world.replay sink in a tab. NOT a port — the wai crate itself now compiles to wasm32 via the new default-on codecs feature (C codec chain gated; envelope + world engine build with --no-default-features --features world), and wai-web binds it (WorldSink: advance/particles/members/ state-hash/verify-seal; sim state stays exact Q32.32, f32 only for display). WebGPU renderer (instanced SDF circles + member lines). Proven in real Chrome via Playwright: the 1,282-byte truss_fracture payload replays 120 ticks, the member snaps, and the browser's BLAKE3 state hash equals the native corpus hash byte-for-byte — replay-equivalence across arm64-native and wasm32-in-Chrome, demonstrated, with the Ed25519 seal verified in-page.
  • Solver-backed registrant wai.world.sim.truss0 (2026-06-12, world/truss.rs): structural simulation — particles + distance members solved per tick by position-based dynamics (fixed 8-pass Gauss–Seidel projection in canonical member order), entirely in wai.det.fixed64 including a new floor-rounded Fx::sqrt (128-bit integer Newton). Members carry a yield strain measured on predicted positions before the solve; an overloaded member fractures as an emergent solver consequence — two actors pulling the same member in opposite directions (the SUM rule feeding the solver) snap it bit-identically on every machine (truss_fracture corpus vector). Registration constants: G = −10, ITERATIONS = 8, ground at y = 0, velocity damping 63/64/tick (static equilibrium). LiveWorld gained state()/current_tick() accessors (what a renderer reads). Where the float-based CAD solvers can't transplant (floats aren't cross-platform bit-exact), solver form does: iterative constraint projection in the deterministic profile.

Inter-frame video, categorical/VQ images, and real-model validation

  • wai.neural.dcvc_rt / wai.neural.dcvc_fm — inter-frame video decoder implemented (neural/video_dcvc.rs). "Requires CUDA" was a speed claim: inter-frame decode is a frame loop that propagates a reference feature, and ONNX Runtime runs it on CPU. The per-frame entropy decode reuses the CompressAI-validated hyperprior path; the temporal net is sink-supplied. Validated end-to-end (3 inter-frames, feature propagation confirmed).
  • wai.neural.glc — categorical / VQ decoder implemented (neural/glc.rs). Not the Gaussian-conditional family: codebook indices decoded via a per-position softmax → pmf_to_quantized_cdf (bit-exact with CompressAI's C++, unit-tested) → a bypass-free categorical rANS over a checkerboard → VQ synthesis. Surfaced that categorical entropy is the most determinism-sensitive path — a per-position CDF flips on ~1e-5 logit drift where Gaussian scale buckets don't.
  • Real-model validation beyond synthetic stand-ins. SNAC's genuine snac_24khz now exports (tools/wai_snac_export.py: zero-noise so the decode is deterministic at all + a reshape-free snake) and wai-rs decodes it to ~9e-7 of the torch reference. A real PTQ-trained, activation-calibrated int_synth model decodes bit-identically (max|Δ|=0) to its integer-forward reference (tools/wai_int_synth_train.py; also fixed a zero-gradient torch.floor bug in the QAT trainer).

Learned-image codec family — decoders implemented and validated

  • The reference crate now decodes the full conditional-entropy family natively, not just the factorized baseline: wai.neural.mbt2018_mean (mean-scale hyperprior, Gaussian-conditional), wai.neural.elic (checkerboard spatial context), wai.neural.mlicpp (channel-autoregressive multi-reference), and the integer-exact wai.neural.int_synth (all-integer transposed-conv synthesis — a decode that is bit-identical on every machine by construction, the first capability that can soundly back intent=replicate cross-hardware). The rANS + entropy decode live in wai-rs; the analysis/synthesis nets are sink-supplied ONNX.
  • Current-SOTA entries added as live routes: wai.neural.mlicv2 (MLICv2/v2+, the current SOTA learned image) and wai.neural.cmic (Content-Aware Mamba) both decode through the multi-reference core — their entropy model is the same channel-slice + checkerboard structure; the gains live in their sink-supplied transforms.
  • Validation against the reference codec. bmshj2018 and mbt2018_mean are checked byte-for-byte against pretrained CompressAI models (reconstruction matches to within final-layer float rounding); elic/mlicpp are checked end-to-end against minimal CompressAI- compatible models; int_synth against a constructed integer model. Surfacing this also caught (and the spec now documents) a real determinism hazard: a learned codec's entropy decode desyncs unless the encoder and decoder compute the scale buckets with the same model — float drift between two implementations of h_s flips bucket assignments and corrupts the stream.

Audio: SNAC hierarchical codec

  • wai.neural.snac — SNAC's multi-resolution RVQ, distinct from the flat single-grid RVQ runtime: N variable-length code streams at hierarchical rates, decoded via dynamic-arity ONNX inputs. Ubiquitous in low-bitrate TTS.

Removed: off-charter Python runtime (wai-neural/)

  • Moved wai-neural/ OUT of this repo to ../../wai-neural-prototype/. It was a parallel Python runtime (WAI4 generative-semantic container + CLIP-embedding payloads + SD-XL-Turbo regeneration + CLIP-similarity conformance) — a disconnected second implementation in the wrong language. The reference implementation is Rust (wai-rs); nothing referenced the Python package. The idea it proved — a behavioral/perceptual neural path that conforms by semantic similarity rather than a weights hash — is sound and belongs in wai-rs as a future Rust path alongside the bit-exact, prior-pinned path. The Python is kept only as a prototype reference (and it hardcodes a machine-local model path, so it never ran portably). This does NOT touch the Python model-export tools in tools/, which are legitimate dev tooling.

Neural condition: working end-to-end

  • tools/wai_encodec_encode.py / wai_encodec_decode.py — first working neural capability (wai.neural.encodec32): source encodes audio via locally-installed EnCodec, ships only the quantized tokens in a standard WAI envelope (236 MB of model weights stay on the source's disk, never enter the envelope); sink with EnCodec installed decodes. Measured: 5 s of audio → 2.3 KB envelope (141× compressed), model on the wire = 0 B.
  • Rust SDK gains wai inspect CLI: parses an envelope, resolves the capability + fallback against sink_capabilities(), and prints verdict (DECODABLE via primary / via fallback / INERT here).

Browser runtime (wai-web/)

  • New crate: wasm-bindgen wrapper exposing parse(bytes) → WaiEnvelope, pack(manifest_json, payload), and known_capabilities(). 107 KB wasm + 16 KB JS glue.
  • wai-web/demo/wai.js — thin JS shim with WaiRuntime.register(cap, handler) dispatch. Backends are pluggable (browser-native APIs, transformers.js, WebGPU compute, WebNN, ONNX runtime web) — WAI doesn't bundle any. The sink-side compute fabric is heterogeneous; the deployer picks per capability.
  • wai-web/demo/index.html — drop-and-decode demo with handlers wired to: createImageBitmap for wai.image.{png,jpeg,avif,jxl}, AudioContext for wai.audio.{opus,flac}, placeholders for wai.neural.* showing where transformers.js / WebNN / WebGPU plug in. Run with python3 wai-web/demo/serve.py.
  • 4 sample .wai files in wai-web/demo/samples/ covering PNG, JXL, Opus, and EnCodec for testing.

Spec: v1.1 multi-rendition envelope (SPEC.md §6.1)

  • New magic WAI2 for envelopes that carry multiple renditions of the same content (e.g. EnCodec + Opus, or AVIF + JPEG-XL). Reframed from "graceful fallback for missing capability" to policy selection — in a self-contained ecosystem every sink has every required capability; renditions exist so the sink picks by compute budget, bandwidth, or runtime preference (browser-native vs WebGPU vs WebNN vs native ML). v1.0 (WAI1) remains valid and unchanged.

v0.3.0

Pivot: WAI is the envelope + capability-dispatch + neural-shared- prior standard. The zeroth condition is a menu of registered SOTA codec capabilities (AVIF, JPEG-XL, PNG, JPEG, Opus, FLAC, AV1, zstd, XZ), not a custom codec. The Python prototype and all hand-rolled codec implementations from v0.2.x were removed; Rust is the single reference implementation and the SDK.

Standard

  • SPEC.md rewritten as a capability-menu spec. Mandatory floor: wai.image.png, wai.audio.flac, wai.text.zstd (every conforming sink MUST decode these). Recommended modern set: wai.image.jxl, wai.image.avif, wai.image.jpeg, wai.audio.opus, wai.video.av1, wai.video.av1.lossless, wai.text.xz. Neural capabilities (wai.neural.*) remain OPTIONAL and sink-advertised.
  • Container envelope (WAI1 magic + length-prefixed manifest + length- prefixed payload) and manifest schema are unchanged from v0.2.x.
  • Conformance: bit-exact decode-equivalence with each capability's reference library (libjxl / libavif / libopus / libFLAC / rav1e / etc.).

Rust reference (wai-rs/, crate wai 0.3.0)

  • cargo build --release produces libwai.dylib/.so/.dll (cdylib), libwai.a (staticlib), and libwai.rlib (Rust-native). The crate type list is ["lib", "cdylib", "staticlib"] — drop-in for any language's C-FFI binding.
  • Codec wrappers: image (PNG, JPEG via jpeg-encoder, AVIF via ravif + libavif-image for decode, JPEG-XL via jpegxl-rs), opus (RFC 7845 Ogg-Opus container so output drops into ffmpeg/VLC/browsers), claxon+flac-bound for FLAC, rav1e+dav1d for AV1 (lossy + lossless), zstd+xz2 for text.
  • Generated C header: include/wai.h is regenerated by build.rs via cbindgen on every build. SDK consumers get a typed header rather than hand-writing extern "C" decls.
  • Cross-platform linkage: build.rs probes pkg-config for libflac; the macOS-hardcoded .cargo/config.toml from v0.2.x is gone. Builds clean on macOS (Homebrew) and Linux (apt) without RUSTFLAGS.

C FFI / SDK surface

All wai_* functions return a wai_status_t:

code meaning
WAI_OK (0) success
WAI_ERR_INVALID_ARG (1) null pointer / mismatched length / out-of-range dimension
WAI_ERR_CODEC (2) codec-level failure (details in wai_last_error())
WAI_ERR_CONTAINER (3) bad magic / truncated envelope / unparseable manifest JSON
WAI_ERR_PANIC (4) internal Rust panic caught at the boundary (a wrapper bug — never UB into the C caller)

Every FFI function is wrapped in std::panic::catch_unwind so Rust panics never unwind into C (UB). av1_decode validates header bounds to prevent libdav1d's C-level abort(). wai_last_error() is per- thread, populated on any non-OK return.

Testing

  • 13 lib tests: per-codec round-trip + envelope round-trip
  • 21 FFI robustness tests: null pointers, length mismatches, dimension overflow, malformed magic, truncated payloads, bad JSON, invalid Opus sample rates, zero fps, garbage decoder input
  • 10 fuzz smoke targets: 1024 adversarial random byte sequences per decoder asserting no panic. Stable Rust, no cargo-fuzz toolchain needed. Found and fixed a real envelope-truncation panic.
  • wai_bench corpus walker: quick mode (4 images + 4 audio + 3 text + 2 video) for spot-checks; --full walks the entire corpus (169 images + 5 audio + 13 text + 12 video sequences = 588 codec runs in ~9.4 min). Per-codec aggregate stats: runs, ok / fail counts, WAI vs reference byte sizes, PSNR delta vs ffmpeg, encode time.
  • --report file.json writes a structured bench report; wai_bench_diff baseline.json head.json flags regressions on bytes (±5% default tolerance), encode time (±25%), or new failures. Non-zero exit on regression — ready for CI gating.
  • GitHub Actions CI (.github/workflows/ci.yml): ubuntu + macOS matrix, apt/brew codec deps, cargo test --release, cdylib/staticlib artifact verification, quick bench, JSON report uploaded as artifact.

Honest codec measurements (vs ffmpeg reference, full Kodak/Tecnick/

CLIC/synthetic corpus = 169 images, plus SQAM audio + Silesia text + Derf CIF video)

codec runs size vs ref PSNR Δ vs ref notes
PNG 169 — lossless ffmpeg-interop ok
JPEG q=50 169 +12.7% +0.27 dB q-knob calibrated to libjpeg
JPEG q=80 169 +19.7% −0.31 dB residual gap is pure-Rust vs libjpeg-turbo asm
AVIF q=70 169 — — ffmpeg-interop ok
JPEG-XL lossless 169 — lossless bit-exact via libjxl
FLAC 5 −10.1% lossless WAI compression_level 8 vs ffmpeg default 5
Opus 64k 5 — — standard Ogg-Opus container, ffmpeg-interop ok
zstd-19 13 ±0.0% — byte-identical to system zstd
xz-6 13 ±0.0% — byte-identical to system xz
AV1 lossy 12 +28.3% — rav1e vs ffmpeg libsvtav1 (different encoders)

All 588 corpus runs and all 44 tests pass.

Removed

  • The Python prototype (wai/) and Python BD-rate harness (bench/). v0.2.x's hand-rolled wire format is dead; Rust is the single source of truth.
  • The webm_lottie_new/wai-rs/ pre-pivot Rust crate (Wasm-decoder-in- bundle model). Replaced by the clean wai-rs/ SOTA-wrapper crate.

v0.2.0 — superseded (historical record)

The v0.2.x direction was hand-rolling custom codec wire formats (DCT + quantization + rANS for image, MDCT for audio, I/P inter-frame for video). Two byte-interoperable reference implementations (Python + Rust) were built and cross-validated.

Why superseded: re-implementing JPEG/MPEG/Opus poorly is strictly worse than wrapping the field's SOTA libraries. WAI's actual value is the envelope + capability dispatch + the neural-shared-prior model, which v0.3 keeps. Custom codecs are gone; standard libraries are in.

What v0.2 had

  • SPEC.md defined byte-precise wire formats for image (8×8 DCT-II + YCbCr + JPEG-Q + zigzag + DC-Δ + rANS), audio (orthonormal MDCT + sine window + TDAC), video (I/P inter-frame with integer-pel motion).
  • Python (wai/) + Rust (wai-rs/) implementations, byte-equivalent on the lossless paths (rANS, container) and reconstruction-equivalent on the lossy paths.
  • Honest measured standing: image at JPEG-1992 parity (~5% behind libjpeg), audio uniform-MDCT (not perceptually competitive with MP3), video I/P at ~41% smaller than MJPEG floor but far behind H.264.

Licensing (unchanged)

  • Apache-2.0 for the standard and the reference implementation. Royalty-free implementation is a precondition for adoption (VP9/AV1 open-codec precedent; the H.264/H.265 royalty trap is avoided).