open-standards/wai/CHANGELOG.md

WAI — changelog

Notable changes to the WAI standard and the reference implementation. Newest at the top. WAI is a draft standard; until v1.0 freezes, the wire format may change and such changes are recorded here.

v0.4.0 — current

JPEG AI — a live, self-sufficient wai.image.jpegai decode (2026-06-22)

• neural::jpegai::decode(codestream, &JpegAiModel) decodes a real ISO/IEC 6048 codestream to RGB with no per-image schedule fixtures: every per-image quantity is derived from the bitstream, and only checkpoint constants come from a sink-supplied model bundle. Wired into decode_envelope — JPEG AI is now a dispatchable capability, not just reference/conformance support.
• The two derivations that closed it, traced bit-exact from the reference: the residual coding mask is sigma_skip > 382 (the σ surface before the rvs addition — on the bucketed index it looks ambiguous, on sigma_skip it is exact, 0 violations / 65536 positions); the dequant is resi × rvs_dq[grfs[ch]][lk] / 2^16 / scaler_vec[ch], with grfs determined the same way the σ rvs does so the per-channel scaler_vec is consistent. Matches the reference decoder's PNG to within rounding (max diff 1).

Film reel — wai.film.reel, cuts over one world (2026-06-17)

• The editor's layer over wai.film.linear: a WREL embeds one sealed world
  • a sequence of shots (each its own camera track + frame schedule) played back to back. Cutting wide→close, or one beat from two angles, is film grammar; a reel carries it as a few kilobytes of camera keys, the world shared across every shot. Conformance is reel-equivalence — film-equivalence extended over cuts.
• Shots MAY revisit earlier world ticks (a cut back in time from a new angle), so the world is sampled at the sorted/de-duplicated union of all frame ticks — convergence still holds because replay-equivalence is order-independent.
• Reuses the film machinery (shared camera interpolation + pose bytes) + the FilmReceipt profile (identity = reel_hash); wai_meter grows a .wrel branch. Registered + tiered Lossless. Corpus vector coverage (the character beat cut three ways, incl. a cut back in time) joins film-conformance/, verified by wai_film_vectors on the four-platform matrix. SPEC §5 row + extensions/linear-film.md §7.

Linear film — wai.film.linear (2026-06-16)

• "A film is a sealed, camera-on-rails world." A generated film ships not as pixels but as the world that produces it (an embedded sealed wai.world.replay) + a camera track (keyframed pose over the world's ticks) + a frame schedule (which tick each frame samples). The sink renders the frames — lever 3 (instructions-at-the-sink) for the synthesizable slice of "video" (animation / CGI / motion graphics). The conformance corpus carries whole films in ~1 KB.
• wai-rs film feature (reuses the world engine + Fx floor + blake3): the WFLM container (contract + embedded WWLD + camera track) and the deterministic reconstruct contract. Conformance is film-equivalence — per-frame BLAKE3(world_state_hash(frame_tick) || interpolated camera pose) is bit-identical on every machine. It composes the embedded world's replay-equivalence (inherited unchanged) with the camera track in the wai.det.fixed64 floor; the render (pixels) stays the sink's.
• The world engine gained one general primitive: collect_checkpoints_ sampled / run_sampled_dispatch — read the state hash at an arbitrary frame cadence, independent of the contract's checkpoint cadence (a renderer samples the timeline at its own frame rate).
• Corpus film-conformance/ + wai_film_vectors gen|verify (dolly, two-shot, crane-up) — all pass, on the four-platform CI matrix. Registered
  • tiered Lossless (film-equivalence is exact). FilmReceipt (per-frame Merkle leaves) + a wai_meter .wflm branch emitting measured joules.
• Spec PROMOTED to extensions/linear-film.md, SPEC §5 temporal-content row, content-taxonomy updated (animation-film → ✅). The biggest commercial wedge in the taxonomy, now shipped.

Music as instructions — wai.audio.score, LIVE at /score (2026-06-14)

• The deepest audio lever: a score carries no recording, only note events — ship the composition, not the recording. wai-rs score feature (reuses the world Fx floor + blake3): the WSCR container (contract + note events) and a trig-free deterministic reference synth — a phase accumulator drives a piecewise-linear oscillator (saw/square/triangle/pulse) shaped by a linear ADSR, all + − × ÷. Pitch is a per-sample phase increment authored off-line (no exponential in the floor). mixdown_hash is the mixdown-equivalence quantity, the music counterpart of spatial audio; a soundfont/neural instrument is the sink's presentation.
• Corpus score-conformance/ + wai_score_vectors gen|verify (scale, chord/polyphony, timbre-changing arpeggio) — all pass. Registered + tiered Lossless (exact reference mix).
• Browser sink ScoreSink (wai-web) + /score page, LIVE: synthesizes the score in-tab, verifies the mixdown hash vs the corpus, draws a piano roll, and plays the reconstructed PCM via Web Audio. wasm pkg uniquely named wai_score.
• Spec PROMOTED to extensions/score.md (site mirror), SPEC §5 row, content-taxonomy updated (music → ✅). Completes the audio surface: recorded objects (audio.scene) and synthesized notes (audio.score).

Neural subsystem reconciled onto main (2026-06-14)

• Wires the full learned-image/audio decoder family into neural/mod.rs and brings the categorical rANS (decode_categorical) it needs: hyperprior (mbt2018_mean), checkerboard context (elic), channel-autoregressive (mlicpp/mlicv2/cmic), integer-exact (int_synth), glc (VQ), inter-frame video (dcvc_rt/dcvc_fm), snac audio — plus the ISO/IEC 6048-1 JPEG-AI entropy + codestream
  • σ/value/filter port (jpegai_*).
• Restores the semantic / generative path (semantic.rs, CLIP-similarity conformance, ModelRequirement.min_sim/sim_metric) and the shared deterministic ONNX loader (ort_session.rs), behind the semantic feature.
• Re-adds byte-exact-conformance (cross-arch rANS + int_synth + JPEG-AI mANS goldens) and the model export/encode tooling under tools/.
• All features build + test green together (neural + semantic + the interactive/temporal cargo classes); clippy clean.

Asset provenance — wai.asset.provenance, LIVE at /provenance (2026-06-13)

• Lever 4 — the "verifies whatever it carries" half. For content that cannot be reduced to instructions (a photo, a recording), WAI does not reconstruct — it carries the opaque bytes content-addressed and attaches a signed, energy-accounted lineage. AssetReceipt (provenance.rs, provenance feature): a JWP group whose objects are processing steps (op + input/output content hash + work + measured joules), Merkle-bound, Ed25519-signed, with chain-linkage verification (each step's input = the previous output, final output = the asset hash) and parent_receipt_hash chaining. from_json so a verifying sink can re-parse it.
• wai_prov tool: synthesizes a real image, builds a 3-step lineage (origin → PNG master → published JPEG) with the real measured CPU energy of each encode (IOReport via macmon, no sudo), wraps the JPEG as a content-addressed wai.image.jpeg envelope, and emits the signed receipt (origin 0 + PNG 0.43 J + JPEG 0.14 J = 0.57 J measured).
• Browser sink — AssetProvenance (wai-web) + /provenance demo, LIVE: recomputes the asset's content hash in-tab and compares to the receipt (tamper-evidence — a "corrupt one byte" button flips it red), verifies the Ed25519 signature + Merkle chain, shows the energy- accounted lineage, and hands the carried JPEG to a native <img> (the browser decodes it; WAI does not). wasm pkg uniquely named wai_prov.
• Spec PROMOTED to extensions/provenance.md (site mirror). It is a receipt profile about a carried asset (like jwp-receipts), not a §5 payload codec — the asset keeps its own capability (wai.image.jpeg). Completes the thesis: carries whatever it can reconstruct, verifies whatever it carries.

Cargo-class polish wave — real cloud, EWA render, receipts, telepresence (2026-06-13)

Four follow-on builds across the new cargo classes:

• Real-SPZ-derived 4D clip (wai-spz-gen 4d): the bloom clip animates 1200 genuine Niantic-SPZ Gaussians (the same cloud the static wai.splat.spz asset ships, round-tripped through the SPZ library) — real cloud, authored motion. LIVE on /volumetric. Honest framing: the win is a verifiable per-frame state, not capture fidelity.
• WebGPU EWA render on /volumetric: the page now renders the reconstructed cloud with a real anisotropic EWA splat pipeline (the same one the worlds demo uses) — per-Gaussian covariance projection, conic, 3σ bounding quad, premultiplied blend, back-to-front sort — with a 2D alpha-disc fallback where WebGPU is unavailable. The hash-verify badge is render-independent. Confirmed on WebGPU (Chrome) + 2D fallback.
• Per-class receipts with measured joules: HapticReceipt (haptic/receipt.rs) + Splat4dReceipt (splat4d/receipt.rs) — the worlds/audio JWP receipt profile transposed to touch (per-channel blocks) and volumetric (per-frame blocks), reusing world::receipt:: merkle_root. New wai_meter bin measures real CPU energy (IOReport via macmon, no sudo) for haptic/splat4d reconstruction and emits a signed, self-verified receipt carrying the measured joules_micro; wai_world_meter now also dispatches feed0/skeletal0, so feed and avatar worlds get the same measured SessionReceipt. Every cargo class now has a measured-energy receipt path (haptic 57 µJ, splat4d 0.21 J for 1200 Gaussians × 17 frames, feed/avatar mJ-scale).
• Avatar telepresence — LIVE at /telepresence: a live skeletal world of N avatars over the existing worlds relay. New SkeletalActor (wasm) signs CREATE + SET_ROTATION CausalOps; the worlds transport (relay-first + BroadcastChannel fallback, hello/sync/op/hash, boundary-hash convergence) is reused verbatim. Each tab creates and animates its own arm rig; sinks converge on the pose hash ("poses agree @ tick N"). Verified on prod through the real relay with two isolated contexts — identical pose hashes, zero errors.

Volumetric / 4D splats — wai.splat.4d, LIVE at /volumetric (2026-06-13)

• Phase 5 complete — the roadmap frontier. A wai.splat.4d is a temporal Gaussian splat: the static splat extended to time. Each Gaussian carries a keyframed track (position / scale / rotation / color / opacity); the sink interpolates the cloud at frame N in the wai.det.fixed64 floor. Conformance is frame-state-equivalence: the reconstructed Gaussian set at frame N hashes identically on every sink, no tolerance — the temporal counterpart of the static splat.
• wai-rs splat4d feature (reuses the world Fx floor + blake3, no codec/ML deps): the W4DS container (contract + per-Gaussian keyframe tracks, 14 Fx per key) + a deterministic reconstruct (Fx linear interpolation, hold outside the track). Trig-free (quaternions interpolate component-wise; the sink normalizes for rendering). frame_hash = the frame-state-equivalence quantity; clip_hash = frame-independent identity. Registered + tiered Lossy (consistent with wai.splat.spz — the splat representation approximates the captured scene, even though the reconstruction is frame-state-exact).
• volumetric-conformance/ corpus + wai_splat4d_vectors gen|verify: orbit (two Gaussians swap places), pulse (scale + fade), morph (color shift) — every frame's hash checked, all pass.
• Browser sink — Splat4dSink (wai-web) + /volumetric demo page, LIVE: reconstructs the cloud in-tab and verifies every frame's hash against the corpus; a frame scrubber re-derives the cloud, drawn as painter-sorted alpha Gaussians (a sink with WebGPU renders the same set with the full EWA pipeline — presentation). wasm pkg uniquely named wai_splat4d.
• Spec PROMOTED to extensions/volumetric.md (site mirror), registered in SPEC.md §5 (wai.splat.4d). Honest about size: the win is content-addressing + dedup + a verifiable per-frame state, NOT payload reduction (a real cloud is MB-scale) — stated plainly.
• The cargo set is complete across the medium: sight (worlds/replay), hearing (audio/mixdown), touch (haptics/sample), live state (feed/replay), a moving body (avatar/pose), and now a moving volume (splat.4d/frame-state) — one wai.det.fixed64 floor, one verification.

Avatars / digital humans — wai.avatar.pose, LIVE at /avatar (2026-06-13)

• Phase 4 complete — digital humans: motion as instructions. An avatar is a skeletal world — the worlds container/op-log/ordering/ sealing/state-hash/receipt-chain, with the step reduced to the identity. Conformance is pose-equivalence (the per-bone local transforms hash identically), a special case of replay-equivalence.
• wai.world.sim.skeletal0 (in wai-rs world feature): a bone is a parent[16] + translation[3] + rotation-quaternion[4] + scale[3] (all wai.det.fixed64). Mutations: CREATE / DELETE / SET_ROTATION (the animation driver) / SET_TRANSLATION / SET_SCALE, all LWW. Trig-free floor: rotations are quaternions and the FK composition is + − × only; axis-angle authoring happens off-line. The forward-kinematics → world space and the skinning of a wai.mesh.glb are presentation (a pure function of the hashed pose), never in the hash. Registered + tiered Lossless; dispatched in run_sealed_dispatch.
• avatar-conformance/ corpus + wai_avatar_vectors gen|verify: a sealed 3-bone arm wave, a sealed 5-bone humanoid pose, and three seeded adversarial-arrival shuffles of the arm wave — all converge, all pass.
• Browser sink — SkeletalSink (wai-web) + /avatar demo page, LIVE: replays a sealed animation in-tab, re-derives the sealed pose hash (checked against the native corpus and the file's own seal), and exposes FK world-space joints + bone segments to draw the skeleton on a rotating orthographic canvas with a tick scrubber. wasm pkg uniquely named wai_avatar.
• Spec PROMOTED to extensions/avatars.md (site mirror), registered in SPEC.md §5 (wai.avatar.pose). A moving body joins sight (worlds), sound (audio), touch (haptics), and live state (feeds) under one floor.

State-feeds — wai.feed.state, LIVE at /feed (2026-06-13)

• Phase 3 complete — live state-feeds: state as instructions. A feed is a non-physics world — the same WWLD container, op log, canonical ordering, sealing, state hash, and receipt chain as interactive-worlds, with the simulation step reduced to the identity. Conformance is therefore replay-equivalence, reused wholesale: no new criterion.
• wai.world.sim.feed0 (in wai-rs world feature): nodes are records of self-describing typed cells (Int i64, Fixed Q32.32, Text UTF-8 — hashed without a schema). Mutations: CREATE / DELETE / SET_CELL (LWW) and ADD_INT / ADD_FIXED (SUM — commutative accumulation). Wrong-typed / out-of-range / absent-node updates are deterministic no-ops. Registered + tiered Lossless in the central codecs registry; dispatched in run_sealed_dispatch.
• feed-conformance/ corpus + wai_feed_vectors gen|verify: sealed scoreboard (SUM scoring + an LWW correction), telemetry (LWW gauge + SUM packet count, incl. a negative ADD_FIXED), market tape (LWW last + SUM volume), and three seeded adversarial-arrival shuffles of the scoreboard — all converge, all pass (hash equality).
• Browser sink — FeedSink (wai-web) + /feed demo page, LIVE: loads a sealed feed, replays it in-tab with the same engine that made the corpus, and re-derives the publisher's sealed BLAKE3 state hash (the file's seal also equals the native corpus hash — both checked). A tick scrubber re-derives the feed at any tick; the record table renders the typed cells generically. wasm pkg uniquely named wai_feed (avoids the deploy fingerprinter's basename collision).
• Spec PROMOTED to extensions/state-feeds.md (site mirror), registered in SPEC.md §5 (wai.feed.state). The broadcast profile (one authoritative publisher, many read-only sinks) is documented as a deployment shape of wai.world.live, not a new format. State joins sight (worlds/replay), hearing (audio/mixdown), and touch (haptics/sample) under one floor — verified the same way.

Interactive worlds — world feature + conformance corpus (2026-06-11)

• New cargo class implemented: the interactive-worlds extension (spec draft staged in wai-transaction-science-web/research/, destined for extensions/interactive-worlds.md). Media that consumes input: a world is a content-addressed artifact, every event is a signed CausalOp, and conformance is replay-equivalence — identical BLAKE3 state hash at every checkpoint, no tolerance parameter. One container (WWLD) serves wai.world.scene / wai.world.replay / wai.world.live, differing only in op-log seal state; an input log and a multiplayer op log are the same object.
• wai-rs world feature (pure Rust, no ML dep chain; deps: blake3 + ed25519-dalek): Fx Q32.32 fixed-point (wai.det.fixed64, the floor profile — bit-exact unconditionally), xoshiro256++ contract rng, signed/dep-gated ops with canonical (effect_tick, lamport, actor) ordering, coordination-free BLAKE3 node minting, WWLD section container with content-binding seal, Sim trait (the extension standardizes the contract, not a physics engine) + kinematic0 reference registrant, sealed Replayer + LiveWorld engines (buffering, past-tick re-derivation, equivocation rejection, receipted-checkpoint history sealing).
• world-conformance/ corpus + wai_world_vectors gen|verify: sealed replays, LWW/SUM conflict batteries, dep-gap / past-tick / shuffled-arrival live streams, expected hashes. All vectors pass; clippy clean on the new code.
• Spec PROMOTED to extensions/interactive-worlds.md (site mirror src/pages/extensions/), registered in SPEC.md §5: wai.world.scene / wai.world.replay / wai.world.live + asset media wai.mesh.glb (canonical glTF 2.0 Binary, Khronos — decoder sink/library-supplied); media: "world" noted in §3. Capability strings live in the central codecs registry (known_capabilities()), tiered Lossless — replay-equivalence is exact state reproduction, the simulation analog of lossless.
• Second sim registrant wai.world.sim.ballistic0 (gravity + ground-plane bounce, per-node radius/restitution, G = −10, all Q32.32 — bit-exact through every bounce) proving the registration story: entered registry, dispatch, and corpus (ballistic_bounce vector) without one line of the extension changing. New run_sealed_dispatch routes on contract.sim; a missing sim reports the world inert (UnknownSimCapability), mirroring SPEC.md §4 step-6 semantics so callers can fall back per the envelope.
• Native energy meter — measured joules, no sudo (2026-06-13): corrected a wrong assumption that real energy needed RAPL/sudo (unavailable on the arm64 prod box). On Apple Silicon the SoC energy counters are readable through IOReport with no privileges (the source macmon/asitop use; verified euid≠0 reads CPU/GPU/ANE power on the M3 Ultra), and proc_pid_rusage gives exact per-process retired cycles/instructions. New wai_world_meter bin (world feature): brackets a tight replay burst of a sealed .wwld against an idle baseline, samples CPU power via macmon, and reports the marginal measured CPU energy — e.g. the truss replay metered at ~512 µJ / replay (~4.3 µJ/tick) at 3.2 W marginal, a real number that fills a receipt's joules_micro where a native sink runs. The burst runs on N threads so the marginal draw clears a shared box's idle fluctuation (single-thread 512 µJ vs 10-thread 524 µJ/replay cross-validates the method — per-replay energy is invariant to parallelism). With a second arg the meter emits a full §8 SessionReceipt carrying the MEASURED joules + exact work_steps, signed and self-verified — real measured energy end to end. Spec §8 + the /worlds page corrected: the sink-metering path needs no sudo (IOReport on Apple Silicon, RAPL on x86); the browser estimate is the fallback, not the ceiling. clippy clean, 94 tests green.
• Phase 2 begun — haptics (wai.haptic.signal) (2026-06-13): touch as instructions. Spec drafted (research/haptics-draft.md), capability registered + tiered. wai-rs haptic feature (reuses the world Fx floor, no codec/ML deps): the WHAP container + a deterministic reconstruct — per channel, a keyframed intensity/sharpness envelope (the Core-Haptics pair every actuator API maps to) linear-interpolated in Fx and quantized to u8. Trig-free by construction (amplitude + sharpness, not a modulated carrier), so the floor reconstructs it exactly with +,−,×,÷ alone. signal_hash is the sample-equivalence quantity. wai_haptic_vectors gen|verify + haptic-conformance/ (ramp, thud, dual-rumble pulse) — all pass, hash equality. 104 tests, clippy clean. Next: Gamepad/Vibration browser sink + /haptics demo, then promote. This completes the sight → hearing → touch trio's engines.
• Spatial-audio: receipt profile + spec PROMOTED (2026-06-13) — Phase 1 complete. audio_scene::SceneReceipt (the worlds receipt transposed to audio): identity scene_hash, the JWP group's objects are the mix's PCM blocks (BLAKE3("wai:audio-block")), Merkle-bound with JWP's exact domains, one Ed25519 sig over scene_hash + root + blocks + work_mac (samples × objects, exact signed) + carried joules_micro; parent_receipt_hash chains renders. Reuses world::receipt::merkle_root — minimal new code. 2 receipt tests (101 total). Draft promoted to extensions/spatial-audio.md (+ site mirror, SPEC §5 row + extension pointer) — and the canonical spec corrected to the as-built sqrt_constpower pan (sqrt-only, no transcendental tables) and the real receipt shape, since registration froze the format. clippy clean; spec page live. Phase 1 (spatial audio) DONE: spec · register · mixer · corpus · live /audio sink · receipt · promoted. Next: Phase 2 — haptics.
• Spatial-audio browser sink — LIVE at /audio (2026-06-13): wai-web AudioSink (wasm-bindgen): parse WSCN, load the corpus sources, run the exact deterministic mixdown, expose mixdown_hash_hex + scene_hash_hex + mix (i16 PCM) + object_xz(t). The /audio page (site-styled, in nav) loads a scene, verifies the mixdown hash in-browser against the corpus (mixdown-equivalence, the same proof worlds does for state), plays the bit-exact stereo mix via an AudioBuffer, and shows a top-down radar of the objects panning. Proven on prod: all three scenes verify, zero errors — wasm-in-Chrome mixes byte-identically to native. Honest split kept: plays the exact stereo floor; binaural/HRTF noted as the richer optional presentation.
• Spatial-audio conformance corpus (2026-06-13): wai_audio_vectors gen|verify + audio-conformance/ — the mixdown-equivalence analog of the worlds corpus. Three vectors (static stereo pair, distance near/far, keyframed left→right sweep) ship a .wscn scene + a .sources.bin (decoded mono tones) + expected scene_hash and mixdown_hash; verify mixes each and checks hash equality, no tolerance. All pass; clippy clean.
• Spatial-audio reference mixer — mixdown-equivalence (2026-06-13): wai-rs audio_scene feature (reuses the world Fx floor + blake3, no codec/ML deps): the WSCN container (scene contract + object table
  • asset table + optional seal, mirrors WWLD) and a fully deterministic mixdown — Fx position interpolation, inverse_clamped distance attenuation, sqrt_constpower pan (sqrt only, no transcendental tables), Fx accumulate, round-half-away i16 quantization in integer space. mixdown_hash (BLAKE3 over canonical i16LE PCM for a declared layout) is the mixdown-equivalence quantity; scene_hash is the identity. 5 tests prove it: centered = equal L/R, hard-left = silent right, deterministic + layout-bound hash, distance attenuates, a moving source sweeps L→R + container round-trips. Sources arrive as decoded mono PCM keyed by content hash — decoding the wai.audio.* assets is the sink's job. clippy clean. Next: conformance corpus bin + Web Audio browser sink + demo.
• Content taxonomy + cargo roadmap; spatial-audio Phase 1 begun (2026-06-13): mapped the full space of web-transported content WAI's approach covers (research/content-taxonomy.md) — a capture→synthesizable→executable→ symbolic spectrum × four levers (codec dispatch, neural prior, instructions-at-sink, determinism+receipts), with a per-row mechanism/extension/status table and the white space called out. A phased build plan (research/cargo-roadmap.md) takes each white-space class through the proven interactive-worlds recipe. Phase 1 kicked off — object-based spatial audio (wai.audio.scene): the audio analog of interactive-worlds (ship sound objects + a scene, render to the sink's layout; mixdown-equivalence = replay-equivalence with the tick replaced by the audio sample). Spec drafted (research/spatial-audio-draft.md, WSCN container, Fx mixdown, content-addressed wai.audio.* sources, stereo bit-exact floor + binaural perceptual tier, WAI2 recorded-mix fallback, JWP receipt profile); wai.audio.scene registered + tiered Lossless. 94 tests green. Remaining phases: haptics, live state-feeds, digital humans, volumetric/4D splats.
• Real SPZ decode — wai.splat.spz end to end (2026-06-13): the splat gallery now renders a genuine Niantic-SPZ file, not a mesh-derived stand-in. New wai-spz-gen crate depends on the field's spz reference library (the sink-supplied codec WAI dispatches to), encodes a 2,400-Gaussian cube cloud to a real .spz (validated by re-loading through the library), and wraps it as a content-addressed wai.splat.spz WAI object (nebula.spz.wai). gen-gallery adds it to the scene's asset table (role 1) beside the glb. Browser: WorldSink.asset_payload(name, env, cap) generalizes the §4.1 verify to any capability; the page gunzips (DecompressionStream) and decodes the SPZ bitstream in-tab — 16-byte header, non-interleaved arrays, 24-bit fixed-point positions, log scales, smallest-three quaternions, byte/255 opacity+color — into per-Gaussian world covariance for the EWA pipeline. Decoded count matches the encoded 2,400 (validates the layout); the cube silhouette + surface-aligned discs validate the quaternion math. Page copy corrected: the splat view decodes a real SPZ asset (no longer "derived from mesh"). Verified locally (2,400 Gaussians, zero errors); prod serves the identical artifacts.
• Anisotropic EWA splatting (2026-06-13): upgraded the splat renderer from isotropic billboards to real 3D Gaussian splatting. Each splat carries an anisotropic 3×3 covariance (surface-aligned flat discs: wide in the triangle tangent plane, thin along the normal); the vertex shader projects it to a screen-space conic (Σ' = J·W·Σ·Wᵀ·Jᵀ with the perspective Jacobian), emits a tight 3σ bounding quad, and the fragment evaluates the conic — the textbook 3DGS pipeline, still CPU-sorted back-to-front and alpha-blended. Cam uniform carries view + proj separately (+ viewport/focal) for the Jacobian. Visibly sharper, surface-coherent splats; proven on prod, zero errors.
• 3D render layer — PBR mesh + Gaussian splats (2026-06-13): the /worlds gallery now renders the content-addressed, hash-verified wai.mesh.glb two ways in a real 3D path (perspective camera + depth buffer, all in wai-web/demo/world.js, no wasm change): a solid PBR-shaded cube (flat normals from screen-space derivatives, Lambert
  • spec + sky-ambient, gamma) and a Gaussian-splat cloud — instanced camera-facing quads with a Gaussian falloff, back-to-front depth-sorted each frame (painter's order) and alpha-blended. The splat cloud is derived from the same verified mesh, making the thesis visible: one content-addressed asset, two sink-side renderings, presentation never touches the state hash. SPZ binary decode stays sink-supplied (stated honestly on the page); the renderer is the real deliverable. Two gallery vectors (gallery / gallery_splat); the 2D sim path is untouched. Proven on prod: both render, 768 splats from the cube, zero errors.
• Receipt chaining + work-meter + wai.splat.spz (2026-06-12): three follow-ons. (1) SessionReceipt gains parent_receipt_hash (BLAKE3 of the prior receipt's canonical bytes — the JWP linkage) + receipt_hash() + verify_chain: successive sealed sessions of one world (shared world_hash, group_id+1) form an append-only signed timeline. (2) An honest energy meter: receipts carry exact, signed, deterministic work_steps (Σ node-steps, from collect_checkpoints_and_work) AND a separately carried joules_micro — never derived in the receipt; the browser supplies a wall-clock estimate at a declared coefficient (labeled "est@4W"), native sinks supply real RAPL, an unmetered sink signs 0. (3) Registered wai.splat.spz — Gaussian-splat scenes as an asset capability parallel to wai.mesh.glb (presentation only, never in the state hash; the win is content-addressing + dedup, not payload size; renderer sink-supplied). Browser seal_session(seed, id, joules, group, parent) chains via localStorage; the /worlds HUD shows chain #N ↳linked · N steps · ≈X mJ est@4W · ✓ re-verified. Proven on prod: two seals chain (g0 parent null → g1 parent = g0.receipt_hash), exact work 248 steps, est joules carried, zero errors. Spec §8 updated (chaining + the two-quantity energy model). 94 native tests, 10/10 vectors, clippy clean.
• Asset path SHIPPED — wai.mesh.glb, content-addressed (2026-06-12, extension §4.1): worlds are made of WAI objects. New world::asset_content_hash (BLAKE3 wai:asset\x01 over a complete WAI envelope) + DOMAIN_ASSET; native tests PROVE the load-bearing claim — a scene with an asset table has the byte-identical state hash AND world hash as the same scene without it (assets are presentation-only, never in the state hash), and the asset hash binds the exact envelope. Generator gen-gallery writes a gallery.wwld scene + assets/cube.glb.wai: a real glTF 2.0 Binary cube (hand-written GLB writer) wrapped as a wai.mesh.glb WAI envelope and content-addressed in the table. Browser: WorldSink.assets() + asset_glb(name, envelope) (verifies content hash in WASM, rejects a mismatch, unwraps the WAI envelope to the .glb payload); the page decodes the GLB (minimal JS decoder) and renders the verified mesh as a spinning wireframe. Proven on prod: 1/1 verified ✓ · cube.glb, and the state hash stays fixed while the mesh spins — §4.1 made visible. 92 native tests, 10/10 vectors, clippy clean.
• Session receipts + seal-to-replay SHIPPED (2026-06-12, extension §8 implemented): world/receipt.rs builds a SessionReceipt — a JWP profile, not a fork: op batches (one per checkpoint interval, content-hashed wai:world-ops\x01) are Merkle-bound with JWP's exact leaf/node domain separators, and one Ed25519 signature covers the world-profile payload (world hash + checkpoint state_hash list + carried microjoules). Joules are carried, never fabricated — the builder takes the figure; an unmetered session signs 0 and says so. New world_hash (BLAKE3 over scene = contract + tick-0 graph, the §7.4 shareable identity), collect_checkpoints (unsealed replay → checkpoint list). Browser: WorldSink.seal_session(seed, signer_id, joules) → SealedSession{replay_bytes, receipt_json, root_hash, …}; the /worlds page's "⏷ seal & download replay" turns a live, shared, edited world into a portable .wwld + .receipt.json, re-opens the fresh replay in a throwaway sink to prove it re-verifies, and shows the Merkle root + batch count in the HUD. Proven on prod: fracture a tower across the relay → seal → 3-batch receipt, valid sig, replay re-verified. 90 native tests, 10/10 vectors, clippy clean.
• Cross-device multiplayer SHIPPED (2026-06-12, wai-relay/ + wai-web Actor binding): every tab/device is a signed actor. New Actor wasm-bindgen class (browser-held Ed25519 key; clicks author real CausalOps; Lamport discipline via observe_lamport); wai-world-relay reference transport (axum WS, per-room fan-out, retains the signed op log for late joiners, verifies every op's Ed25519 signature at the door with the same wai crate — it adjudicates nothing; sinks converge because the math does). Page transport is relay-first (room in the URL hash) with BroadcastChannel fallback. Deployed at wai.transaction.science/worlds-ws/* (launchd com.wai.world-relay, localhost:9670). Proven on prod with two ISOLATED browser contexts: point-blank axial shoves snap two members (23→21 nodes) and both contexts converge — "✓ hashes agree". Also fixed build.rs emitting libFLAC link flags regardless of features (pkg-config probe side effect): now gated on codecs.
• Browser sink SHIPPED (2026-06-12, wai-web/src/world.rs + wai-web/demo/world.{html,js}): a wai.world.replay sink in a tab. NOT a port — the wai crate itself now compiles to wasm32 via the new default-on codecs feature (C codec chain gated; envelope + world engine build with --no-default-features --features world), and wai-web binds it (WorldSink: advance/particles/members/ state-hash/verify-seal; sim state stays exact Q32.32, f32 only for display). WebGPU renderer (instanced SDF circles + member lines). Proven in real Chrome via Playwright: the 1,282-byte truss_fracture payload replays 120 ticks, the member snaps, and the browser's BLAKE3 state hash equals the native corpus hash byte-for-byte — replay-equivalence across arm64-native and wasm32-in-Chrome, demonstrated, with the Ed25519 seal verified in-page.
• Solver-backed registrant wai.world.sim.truss0 (2026-06-12, world/truss.rs): structural simulation — particles + distance members solved per tick by position-based dynamics (fixed 8-pass Gauss–Seidel projection in canonical member order), entirely in wai.det.fixed64 including a new floor-rounded Fx::sqrt (128-bit integer Newton). Members carry a yield strain measured on predicted positions before the solve; an overloaded member fractures as an emergent solver consequence — two actors pulling the same member in opposite directions (the SUM rule feeding the solver) snap it bit-identically on every machine (truss_fracture corpus vector). Registration constants: G = −10, ITERATIONS = 8, ground at y = 0, velocity damping 63/64/tick (static equilibrium). LiveWorld gained state()/current_tick() accessors (what a renderer reads). Where the float-based CAD solvers can't transplant (floats aren't cross-platform bit-exact), solver form does: iterative constraint projection in the deterministic profile.

Inter-frame video, categorical/VQ images, and real-model validation

• wai.neural.dcvc_rt / wai.neural.dcvc_fm — inter-frame video decoder implemented (neural/video_dcvc.rs). "Requires CUDA" was a speed claim: inter-frame decode is a frame loop that propagates a reference feature, and ONNX Runtime runs it on CPU. The per-frame entropy decode reuses the CompressAI-validated hyperprior path; the temporal net is sink-supplied. Validated end-to-end (3 inter-frames, feature propagation confirmed).
• wai.neural.glc — categorical / VQ decoder implemented (neural/glc.rs). Not the Gaussian-conditional family: codebook indices decoded via a per-position softmax → pmf_to_quantized_cdf (bit-exact with CompressAI's C++, unit-tested) → a bypass-free categorical rANS over a checkerboard → VQ synthesis. Surfaced that categorical entropy is the most determinism-sensitive path — a per-position CDF flips on ~1e-5 logit drift where Gaussian scale buckets don't.
• Real-model validation beyond synthetic stand-ins. SNAC's genuine snac_24khz now exports (tools/wai_snac_export.py: zero-noise so the decode is deterministic at all + a reshape-free snake) and wai-rs decodes it to ~9e-7 of the torch reference. A real PTQ-trained, activation-calibrated int_synth model decodes bit-identically (max|Δ|=0) to its integer-forward reference (tools/wai_int_synth_train.py; also fixed a zero-gradient torch.floor bug in the QAT trainer).

Learned-image codec family — decoders implemented and validated

• The reference crate now decodes the full conditional-entropy family natively, not just the factorized baseline: wai.neural.mbt2018_mean (mean-scale hyperprior, Gaussian-conditional), wai.neural.elic (checkerboard spatial context), wai.neural.mlicpp (channel-autoregressive multi-reference), and the integer-exact wai.neural.int_synth (all-integer transposed-conv synthesis — a decode that is bit-identical on every machine by construction, the first capability that can soundly back intent=replicate cross-hardware). The rANS + entropy decode live in wai-rs; the analysis/synthesis nets are sink-supplied ONNX.
• Current-SOTA entries added as live routes: wai.neural.mlicv2 (MLICv2/v2+, the current SOTA learned image) and wai.neural.cmic (Content-Aware Mamba) both decode through the multi-reference core — their entropy model is the same channel-slice + checkerboard structure; the gains live in their sink-supplied transforms.
• Validation against the reference codec. bmshj2018 and mbt2018_mean are checked byte-for-byte against pretrained CompressAI models (reconstruction matches to within final-layer float rounding); elic/mlicpp are checked end-to-end against minimal CompressAI- compatible models; int_synth against a constructed integer model. Surfacing this also caught (and the spec now documents) a real determinism hazard: a learned codec's entropy decode desyncs unless the encoder and decoder compute the scale buckets with the same model — float drift between two implementations of h_s flips bucket assignments and corrupts the stream.

Audio: SNAC hierarchical codec

• wai.neural.snac — SNAC's multi-resolution RVQ, distinct from the flat single-grid RVQ runtime: N variable-length code streams at hierarchical rates, decoded via dynamic-arity ONNX inputs. Ubiquitous in low-bitrate TTS.

Removed: off-charter Python runtime (wai-neural/)

• Moved wai-neural/ OUT of this repo to ../../wai-neural-prototype/. It was a parallel Python runtime (WAI4 generative-semantic container + CLIP-embedding payloads + SD-XL-Turbo regeneration + CLIP-similarity conformance) — a disconnected second implementation in the wrong language. The reference implementation is Rust (wai-rs); nothing referenced the Python package. The idea it proved — a behavioral/perceptual neural path that conforms by semantic similarity rather than a weights hash — is sound and belongs in wai-rs as a future Rust path alongside the bit-exact, prior-pinned path. The Python is kept only as a prototype reference (and it hardcodes a machine-local model path, so it never ran portably). This does NOT touch the Python model-export tools in tools/, which are legitimate dev tooling.

Neural condition: working end-to-end

• tools/wai_encodec_encode.py / wai_encodec_decode.py — first working neural capability (wai.neural.encodec32): source encodes audio via locally-installed EnCodec, ships only the quantized tokens in a standard WAI envelope (236 MB of model weights stay on the source's disk, never enter the envelope); sink with EnCodec installed decodes. Measured: 5 s of audio → 2.3 KB envelope (141× compressed), model on the wire = 0 B.
• Rust SDK gains wai inspect CLI: parses an envelope, resolves the capability + fallback against sink_capabilities(), and prints verdict (DECODABLE via primary / via fallback / INERT here).

Browser runtime (wai-web/)

• New crate: wasm-bindgen wrapper exposing parse(bytes) → WaiEnvelope, pack(manifest_json, payload), and known_capabilities(). 107 KB wasm + 16 KB JS glue.
• wai-web/demo/wai.js — thin JS shim with WaiRuntime.register(cap, handler) dispatch. Backends are pluggable (browser-native APIs, transformers.js, WebGPU compute, WebNN, ONNX runtime web) — WAI doesn't bundle any. The sink-side compute fabric is heterogeneous; the deployer picks per capability.
• wai-web/demo/index.html — drop-and-decode demo with handlers wired to: createImageBitmap for wai.image.{png,jpeg,avif,jxl}, AudioContext for wai.audio.{opus,flac}, placeholders for wai.neural.* showing where transformers.js / WebNN / WebGPU plug in. Run with python3 wai-web/demo/serve.py.
• 4 sample .wai files in wai-web/demo/samples/ covering PNG, JXL, Opus, and EnCodec for testing.

Spec: v1.1 multi-rendition envelope (SPEC.md §6.1)

• New magic WAI2 for envelopes that carry multiple renditions of the same content (e.g. EnCodec + Opus, or AVIF + JPEG-XL). Reframed from "graceful fallback for missing capability" to policy selection — in a self-contained ecosystem every sink has every required capability; renditions exist so the sink picks by compute budget, bandwidth, or runtime preference (browser-native vs WebGPU vs WebNN vs native ML). v1.0 (WAI1) remains valid and unchanged.

v0.3.0

Pivot: WAI is the envelope + capability-dispatch + neural-shared- prior standard. The zeroth condition is a menu of registered SOTA codec capabilities (AVIF, JPEG-XL, PNG, JPEG, Opus, FLAC, AV1, zstd, XZ), not a custom codec. The Python prototype and all hand-rolled codec implementations from v0.2.x were removed; Rust is the single reference implementation and the SDK.

Standard

• SPEC.md rewritten as a capability-menu spec. Mandatory floor: wai.image.png, wai.audio.flac, wai.text.zstd (every conforming sink MUST decode these). Recommended modern set: wai.image.jxl, wai.image.avif, wai.image.jpeg, wai.audio.opus, wai.video.av1, wai.video.av1.lossless, wai.text.xz. Neural capabilities (wai.neural.*) remain OPTIONAL and sink-advertised.
• Container envelope (WAI1 magic + length-prefixed manifest + length- prefixed payload) and manifest schema are unchanged from v0.2.x.
• Conformance: bit-exact decode-equivalence with each capability's reference library (libjxl / libavif / libopus / libFLAC / rav1e / etc.).

Rust reference (wai-rs/, crate wai 0.3.0)

• cargo build --release produces libwai.dylib/.so/.dll (cdylib), libwai.a (staticlib), and libwai.rlib (Rust-native). The crate type list is ["lib", "cdylib", "staticlib"] — drop-in for any language's C-FFI binding.
• Codec wrappers: image (PNG, JPEG via jpeg-encoder, AVIF via ravif + libavif-image for decode, JPEG-XL via jpegxl-rs), opus (RFC 7845 Ogg-Opus container so output drops into ffmpeg/VLC/browsers), claxon+flac-bound for FLAC, rav1e+dav1d for AV1 (lossy + lossless), zstd+xz2 for text.
• Generated C header: include/wai.h is regenerated by build.rs via cbindgen on every build. SDK consumers get a typed header rather than hand-writing extern "C" decls.
• Cross-platform linkage: build.rs probes pkg-config for libflac; the macOS-hardcoded .cargo/config.toml from v0.2.x is gone. Builds clean on macOS (Homebrew) and Linux (apt) without RUSTFLAGS.

C FFI / SDK surface

All wai_* functions return a wai_status_t:

code	meaning
WAI_OK (0)	success
WAI_ERR_INVALID_ARG (1)	null pointer / mismatched length / out-of-range dimension
WAI_ERR_CODEC (2)	codec-level failure (details in wai_last_error())
WAI_ERR_CONTAINER (3)	bad magic / truncated envelope / unparseable manifest JSON
WAI_ERR_PANIC (4)	internal Rust panic caught at the boundary (a wrapper bug — never UB into the C caller)

Every FFI function is wrapped in std::panic::catch_unwind so Rust panics never unwind into C (UB). av1_decode validates header bounds to prevent libdav1d's C-level abort(). wai_last_error() is per- thread, populated on any non-OK return.

Testing

• 13 lib tests: per-codec round-trip + envelope round-trip
• 21 FFI robustness tests: null pointers, length mismatches, dimension overflow, malformed magic, truncated payloads, bad JSON, invalid Opus sample rates, zero fps, garbage decoder input
• 10 fuzz smoke targets: 1024 adversarial random byte sequences per decoder asserting no panic. Stable Rust, no cargo-fuzz toolchain needed. Found and fixed a real envelope-truncation panic.
• wai_bench corpus walker: quick mode (4 images + 4 audio + 3 text + 2 video) for spot-checks; --full walks the entire corpus (169 images + 5 audio + 13 text + 12 video sequences = 588 codec runs in ~9.4 min). Per-codec aggregate stats: runs, ok / fail counts, WAI vs reference byte sizes, PSNR delta vs ffmpeg, encode time.
• --report file.json writes a structured bench report; wai_bench_diff baseline.json head.json flags regressions on bytes (±5% default tolerance), encode time (±25%), or new failures. Non-zero exit on regression — ready for CI gating.
• GitHub Actions CI (.github/workflows/ci.yml): ubuntu + macOS matrix, apt/brew codec deps, cargo test --release, cdylib/staticlib artifact verification, quick bench, JSON report uploaded as artifact.

Honest codec measurements (vs ffmpeg reference, full Kodak/Tecnick/

CLIC/synthetic corpus = 169 images, plus SQAM audio + Silesia text + Derf CIF video)

codec	runs	size vs ref	PSNR Δ vs ref	notes
PNG	169	—	lossless	ffmpeg-interop ok
JPEG q=50	169	+12.7%	+0.27 dB	q-knob calibrated to libjpeg
JPEG q=80	169	+19.7%	−0.31 dB	residual gap is pure-Rust vs libjpeg-turbo asm
AVIF q=70	169	—	—	ffmpeg-interop ok
JPEG-XL lossless	169	—	lossless	bit-exact via libjxl
FLAC	5	−10.1%	lossless	WAI compression_level 8 vs ffmpeg default 5
Opus 64k	5	—	—	standard Ogg-Opus container, ffmpeg-interop ok
zstd-19	13	±0.0%	—	byte-identical to system zstd
xz-6	13	±0.0%	—	byte-identical to system xz
AV1 lossy	12	+28.3%	—	rav1e vs ffmpeg libsvtav1 (different encoders)

All 588 corpus runs and all 44 tests pass.

Removed

• The Python prototype (wai/) and Python BD-rate harness (bench/). v0.2.x's hand-rolled wire format is dead; Rust is the single source of truth.
• The webm_lottie_new/wai-rs/ pre-pivot Rust crate (Wasm-decoder-in- bundle model). Replaced by the clean wai-rs/ SOTA-wrapper crate.

---

v0.2.0 — superseded (historical record)

The v0.2.x direction was hand-rolling custom codec wire formats (DCT + quantization + rANS for image, MDCT for audio, I/P inter-frame for video). Two byte-interoperable reference implementations (Python + Rust) were built and cross-validated.

Why superseded: re-implementing JPEG/MPEG/Opus poorly is strictly worse than wrapping the field's SOTA libraries. WAI's actual value is the envelope + capability dispatch + the neural-shared-prior model, which v0.3 keeps. Custom codecs are gone; standard libraries are in.

What v0.2 had

• SPEC.md defined byte-precise wire formats for image (8×8 DCT-II + YCbCr + JPEG-Q + zigzag + DC-Δ + rANS), audio (orthonormal MDCT + sine window + TDAC), video (I/P inter-frame with integer-pel motion).
• Python (wai/) + Rust (wai-rs/) implementations, byte-equivalent on the lossless paths (rANS, container) and reconstruction-equivalent on the lossy paths.
• Honest measured standing: image at JPEG-1992 parity (~5% behind libjpeg), audio uniform-MDCT (not perceptually competitive with MP3), video I/P at ~41% smaller than MJPEG floor but far behind H.264.

Licensing (unchanged)

• Apache-2.0 for the standard and the reference implementation. Royalty-free implementation is a precondition for adoption (VP9/AV1 open-codec precedent; the H.264/H.265 royalty trap is avoided).