This mapping was written on 2026-08-03 and sat unmerged while `extensions/capability-constraints.md` landed, which makes dispatch a precondition check: a sink satisfies a capability when it advertises `cap` AND the object's declared parameters satisfy a registered constraint set. Rebased onto current main, the two extensions read as overlapping claims about §4 with nothing saying how they relate. They are not in tension, and the newer one makes this one sharper. `isConfigSupported()` is not a codec-string test — it takes a whole `VideoDecoderConfig`, codec string plus `codedWidth`, `codedHeight` and the rest — so it is a constraint evaluator already, and a browser satisfies the constraint extension's three requirements structurally rather than through extra machinery: the query is pre-flight, so evaluation happens before dispatch rather than during it; it answers for a configuration, so the answer is per object and not per sink; and `supported: false` is a negative answer returned before any payload is committed to, which is the required treatment of an object that satisfies no set. The call also returns the normalised config the platform would accept, which is the accepted parameter set, so a sink SHOULD use it rather than re-derive one. The limit is stated rather than glossed. WebCodecs' parameter vocabulary is the conventional-media one — codec, resolution, frame rate, bit depth, profile — which is the same shape as the ratified registers `capability-constraints.md` §3 draws on, and it has the same gap: no WebCodecs configuration describes the neural, quantum, world or haptic families. Those have no native tier to constrain at all, resolve through the WASM tier, and stay unconstrained until WAI authors their register entries. That is the state §3 already describes, not a shortfall introduced here. Framing renumbered §5 → §6. Docs only; `check-no-vanity-metrics.sh` passes. Co-Authored-By: Claude Opus 5 <[email protected]>
53 KiB
WAI — changelog
Notable changes to the WAI standard and the reference implementation. Newest at the top. WAI is a draft standard; until v1.0 freezes, the wire format may change and such changes are recorded here.
v0.4.0 — current
Determinism tiers — naming the reproducibility spectrum for neural decode
- New [
extensions/determinism-tiers.md]: names the two guarantees a neural capability can make, so a manifest can declare which it satisfies and a verifier knows what it may check.entropy-consistency— the entropy decode reproduces across devices (integer entropy arithmetic or explicit parameter transmission), the guarantee the base learned-codec standards make (JPEG AI's normative conformance point is the entropy-decoder output) and the mainstream cross-platform fix; the floor for a sound neuralreplicate.decode-equivalence— the whole decode to the final medium reproduces, so the rendered result has a portable hash, the tier WAI adds on top and the precondition for a hard reconstruction binding. Referenced from SPEC §7 (Replicate-soundness); framed self-referentially.
Browser capability negotiation — WebCodecs mapping
- New [
extensions/webcodecs-negotiation.md]: how a browser sink implements §4's "advertisescap" test — the platform'sisConfigSupported()for the native tier, a WebAssembly decoder as the always-satisfiable fallback. Grounds WAI's dispatch in the browser's own capability-negotiation pattern (ask if a config is supported; fall back to wasm) and documents what the live wasm sinks already do: native codec when the platform reports it, WAI's pure-integer wasm decoders otherwise, the classical floor always reachable. Referenced from SPEC §4. Its §5 reconciles the mapping with [extensions/capability-constraints.md], which landed later:isConfigSupported()takes a wholeVideoDecoderConfig, so the native tier is already a per-object constraint check evaluated before dispatch — and it carries the same register gap, since no WebCodecs configuration describes the neural, quantum, world or haptic families.
Reproducible-reconstruction binding — a C2PA/JPEG-Trust hard binding over pixels
- New
wai.*provenance capability:reconstruction_binding(featureprovenance) + [extensions/reconstruction-binding.md]. A cryptographic hard binding over the deterministically decoded pixels of a neural-coded asset — which such media normally can't carry, because a neural decode isn't reproducible (JPEG AI's normative conformance point is the entropy-decoder output, not pixels; its multi-branch design reconstructs one codestream several ways). WAI's byte-exact decode restores a portable decoded-pixel hash. SHA-256 (the C2PA default) over the canonical RGB8 buffer plus a descriptor pinning the reproducible decode; carried as a custom C2PA/JPEG-Trust assertion (science.transaction.wai.hash.reconstruction) beside the standard byte-binding. A substrate for machine-readable AI-content marking (e.g. EU AI Act Art. 50).
wai.image.jpegai now dispatches to the byte-exact decoder
decode_envelopeprefers the deterministicjpegai_decode::decodewhen the model directory ships the integer weight bundle (weights/*.json+norm.json), falling back to the ONNX-runtime decode otherwise — existing bundles keep working.wai.image.jpegaiis now registered in SPEC §5 (it was a live capability missing from the normative menu).- The cross-arch byte-equality matrix now covers the merged decoder's entropy
stage (
byte-exact-conformance), and that crate's build — broken on a movedrans.rsinclude — is restored.
JPEG AI — a byte-exact codestream→RGB decoder (jpegai_decode::decode)
wai::jpegai_decode::decode(codestream, &DecoderWeights, &NormTables) -> Rgbreconstructs an ISO/IEC 6048 JPEG-AI codestream to RGB through a fully integer entropy front end and a deterministic value/synthesis/post-filter path — one pinned reconstruction, reproducible byte-for-byte on every machine, so the decoded pixels have a portable hash. No per-image fixtures: every per-image quantity is derived from the bitstream; only checkpoint constants come from a sink-supplied model bundle.- Determinism profile: the entropy decode (mANS), the dequant, and the derived
entropy inputs (skip mask, σ index, RVS band, dequant gain) are integer-exact
against the reference; the PTQ value/synthesis stages are deterministic and
land within 1 LSB of the ISO reference decoder. Distinct from the in-house
wai.neural.int_*codecs (all-i64, hash-portable): this decodes a published external standard's codestream deterministically. - Modules:
jpegai_decode(the composed entry point) overjpegai_bitstream,jpegai_mans,jpegai_pipeline,jpegai_synthesis,jpegai_icci(featureneural_int). Inputs are regenerable from the shipped assets viatools/wai_jpegai_build_e2e_inputs.py. The whole chain — codestream → RGB — is exercised end to end against the reference PNG.
JPEG AI — a live, self-sufficient wai.image.jpegai decode (2026-06-22)
neural::jpegai::decode(codestream, &JpegAiModel)decodes a real ISO/IEC 6048 codestream to RGB with no per-image schedule fixtures: every per-image quantity is derived from the bitstream, and only checkpoint constants come from a sink-supplied model bundle. Wired intodecode_envelope— JPEG AI is now a dispatchable capability, not just reference/conformance support.- The two derivations that closed it, traced bit-exact from the reference: the
residual coding mask is
sigma_skip > 382(the σ surface before the rvs addition — on the bucketed index it looks ambiguous, onsigma_skipit is exact, 0 violations / 65536 positions); the dequant isresi × rvs_dq[grfs[ch]][lk] / 2^16 / scaler_vec[ch], withgrfsdetermined the same way the σ rvs does so the per-channelscaler_vecis consistent. Matches the reference decoder's PNG to within rounding (max diff 1).
Film reel — wai.film.reel, cuts over one world (2026-06-17)
- The editor's layer over
wai.film.linear: aWRELembeds one sealed world- a sequence of shots (each its own camera track + frame schedule) played back to back. Cutting wide→close, or one beat from two angles, is film grammar; a reel carries it as a few kilobytes of camera keys, the world shared across every shot. Conformance is reel-equivalence — film-equivalence extended over cuts.
- Shots MAY revisit earlier world ticks (a cut back in time from a new angle), so the world is sampled at the sorted/de-duplicated union of all frame ticks — convergence still holds because replay-equivalence is order-independent.
- Reuses the film machinery (shared camera interpolation + pose bytes) + the
FilmReceiptprofile (identity =reel_hash);wai_metergrows a.wrelbranch. Registered + tiered Lossless. Corpus vectorcoverage(the character beat cut three ways, incl. a cut back in time) joinsfilm-conformance/, verified bywai_film_vectorson the four-platform matrix. SPEC §5 row +extensions/linear-film.md §7.
Linear film — wai.film.linear (2026-06-16)
- "A film is a sealed, camera-on-rails world." A generated film ships not
as pixels but as the world that produces it (an embedded sealed
wai.world.replay) + a camera track (keyframed pose over the world's ticks) + a frame schedule (which tick each frame samples). The sink renders the frames — lever 3 (instructions-at-the-sink) for the synthesizable slice of "video" (animation / CGI / motion graphics). The conformance corpus carries whole films in ~1 KB. wai-rsfilmfeature (reuses the world engine +Fxfloor + blake3): theWFLMcontainer (contract + embeddedWWLD+ camera track) and the deterministic reconstruct contract. Conformance is film-equivalence — per-frameBLAKE3(world_state_hash(frame_tick) || interpolated camera pose)is bit-identical on every machine. It composes the embedded world's replay-equivalence (inherited unchanged) with the camera track in thewai.det.fixed64floor; the render (pixels) stays the sink's.- The world engine gained one general primitive:
collect_checkpoints_ sampled/run_sampled_dispatch— read the state hash at an arbitrary frame cadence, independent of the contract's checkpoint cadence (a renderer samples the timeline at its own frame rate). - Corpus
film-conformance/+wai_film_vectors gen|verify(dolly, two-shot, crane-up) — all pass, on the four-platform CI matrix. Registered- tiered Lossless (film-equivalence is exact).
FilmReceipt(per-frame Merkle leaves) + awai_meter.wflmbranch emitting measured joules.
- tiered Lossless (film-equivalence is exact).
- Spec PROMOTED to
extensions/linear-film.md, SPEC §5 temporal-content row, content-taxonomy updated (animation-film → ✅). The biggest commercial wedge in the taxonomy, now shipped.
Music as instructions — wai.audio.score, LIVE at /score (2026-06-14)
- The deepest audio lever: a score carries no recording, only note
events — ship the composition, not the recording.
wai-rsscorefeature (reuses the worldFxfloor + blake3): theWSCRcontainer (contract + note events) and a trig-free deterministic reference synth — a phase accumulator drives a piecewise-linear oscillator (saw/square/triangle/pulse) shaped by a linear ADSR, all+ − × ÷. Pitch is a per-sample phase increment authored off-line (no exponential in the floor).mixdown_hashis the mixdown-equivalence quantity, the music counterpart of spatial audio; a soundfont/neural instrument is the sink's presentation. - Corpus
score-conformance/+wai_score_vectors gen|verify(scale, chord/polyphony, timbre-changing arpeggio) — all pass. Registered + tiered Lossless (exact reference mix). - Browser sink
ScoreSink(wai-web) +/scorepage, LIVE: synthesizes the score in-tab, verifies the mixdown hash vs the corpus, draws a piano roll, and plays the reconstructed PCM via Web Audio. wasm pkg uniquely namedwai_score. - Spec PROMOTED to
extensions/score.md(site mirror), SPEC §5 row, content-taxonomy updated (music → ✅). Completes the audio surface: recorded objects (audio.scene) and synthesized notes (audio.score).
Neural subsystem reconciled onto main (2026-06-14)
- Wires the full learned-image/audio decoder family into
neural/mod.rsand brings the categoricalrANS(decode_categorical) it needs: hyperprior (mbt2018_mean), checkerboard context (elic), channel-autoregressive (mlicpp/mlicv2/cmic), integer-exact (int_synth),glc(VQ), inter-frame video (dcvc_rt/dcvc_fm),snacaudio — plus the ISO/IEC 6048-1 JPEG-AI entropy + codestream- σ/value/filter port (
jpegai_*).
- σ/value/filter port (
- Restores the semantic / generative path (
semantic.rs, CLIP-similarity conformance,ModelRequirement.min_sim/sim_metric) and the shared deterministic ONNX loader (ort_session.rs), behind thesemanticfeature. - Re-adds
byte-exact-conformance(cross-arch rANS + int_synth + JPEG-AI mANS goldens) and the model export/encode tooling undertools/. - All features build + test green together (
neural+semantic+ the interactive/temporal cargo classes); clippy clean.
Asset provenance — wai.asset.provenance, LIVE at /provenance (2026-06-13)
- Lever 4 — the "verifies whatever it carries" half. For content that
cannot be reduced to instructions (a photo, a recording), WAI does not
reconstruct — it carries the opaque bytes content-addressed and attaches
a signed, energy-accounted lineage.
AssetReceipt(provenance.rs,provenancefeature): a JWP group whose objects are processing steps (op + input/output content hash + work + measured joules), Merkle-bound, Ed25519-signed, with chain-linkage verification (each step's input = the previous output, final output = the asset hash) andparent_receipt_hashchaining.from_jsonso a verifying sink can re-parse it. wai_provtool: synthesizes a real image, builds a 3-step lineage (origin → PNG master → published JPEG) with the real measured CPU energy of each encode (IOReport via macmon, no sudo), wraps the JPEG as a content-addressedwai.image.jpegenvelope, and emits the signed receipt (origin 0 + PNG 0.43 J + JPEG 0.14 J = 0.57 J measured).- Browser sink —
AssetProvenance(wai-web) +/provenancedemo, LIVE: recomputes the asset's content hash in-tab and compares to the receipt (tamper-evidence — a "corrupt one byte" button flips it red), verifies the Ed25519 signature + Merkle chain, shows the energy- accounted lineage, and hands the carried JPEG to a native<img>(the browser decodes it; WAI does not). wasm pkg uniquely namedwai_prov. - Spec PROMOTED to
extensions/provenance.md(site mirror). It is a receipt profile about a carried asset (likejwp-receipts), not a §5 payload codec — the asset keeps its own capability (wai.image.jpeg). Completes the thesis: carries whatever it can reconstruct, verifies whatever it carries.
Cargo-class polish wave — real cloud, EWA render, receipts, telepresence (2026-06-13)
Four follow-on builds across the new cargo classes:
- Real-SPZ-derived 4D clip (
wai-spz-gen 4d): thebloomclip animates 1200 genuine Niantic-SPZ Gaussians (the same cloud the staticwai.splat.spzasset ships, round-tripped through the SPZ library) — real cloud, authored motion. LIVE on/volumetric. Honest framing: the win is a verifiable per-frame state, not capture fidelity. - WebGPU EWA render on
/volumetric: the page now renders the reconstructed cloud with a real anisotropic EWA splat pipeline (the same one the worlds demo uses) — per-Gaussian covariance projection, conic, 3σ bounding quad, premultiplied blend, back-to-front sort — with a 2D alpha-disc fallback where WebGPU is unavailable. The hash-verify badge is render-independent. Confirmed on WebGPU (Chrome) + 2D fallback. - Per-class receipts with measured joules:
HapticReceipt(haptic/receipt.rs) +Splat4dReceipt(splat4d/receipt.rs) — the worlds/audio JWP receipt profile transposed to touch (per-channel blocks) and volumetric (per-frame blocks), reusingworld::receipt:: merkle_root. Newwai_meterbin measures real CPU energy (IOReport via macmon, no sudo) for haptic/splat4d reconstruction and emits a signed, self-verified receipt carrying the measuredjoules_micro;wai_world_meternow also dispatchesfeed0/skeletal0, so feed and avatar worlds get the same measuredSessionReceipt. Every cargo class now has a measured-energy receipt path (haptic 57 µJ, splat4d 0.21 J for 1200 Gaussians × 17 frames, feed/avatar mJ-scale). - Avatar telepresence — LIVE at
/telepresence: a live skeletal world of N avatars over the existing worlds relay. NewSkeletalActor(wasm) signsCREATE+SET_ROTATIONCausalOps; the worlds transport (relay-first + BroadcastChannel fallback, hello/sync/op/hash, boundary-hash convergence) is reused verbatim. Each tab creates and animates its own arm rig; sinks converge on the pose hash ("poses agree @ tick N"). Verified on prod through the real relay with two isolated contexts — identical pose hashes, zero errors.
Volumetric / 4D splats — wai.splat.4d, LIVE at /volumetric (2026-06-13)
- Phase 5 complete — the roadmap frontier. A
wai.splat.4dis a temporal Gaussian splat: the static splat extended to time. Each Gaussian carries a keyframed track (position / scale / rotation / color / opacity); the sink interpolates the cloud at frame N in thewai.det.fixed64floor. Conformance is frame-state-equivalence: the reconstructed Gaussian set at frame N hashes identically on every sink, no tolerance — the temporal counterpart of the static splat. wai-rssplat4dfeature (reuses the worldFxfloor + blake3, no codec/ML deps): theW4DScontainer (contract + per-Gaussian keyframe tracks, 14Fxper key) + a deterministic reconstruct (Fx linear interpolation, hold outside the track). Trig-free (quaternions interpolate component-wise; the sink normalizes for rendering).frame_hash= the frame-state-equivalence quantity;clip_hash= frame-independent identity. Registered + tiered Lossy (consistent withwai.splat.spz— the splat representation approximates the captured scene, even though the reconstruction is frame-state-exact).volumetric-conformance/corpus +wai_splat4d_vectors gen|verify: orbit (two Gaussians swap places), pulse (scale + fade), morph (color shift) — every frame's hash checked, all pass.- Browser sink —
Splat4dSink(wai-web) +/volumetricdemo page, LIVE: reconstructs the cloud in-tab and verifies every frame's hash against the corpus; a frame scrubber re-derives the cloud, drawn as painter-sorted alpha Gaussians (a sink with WebGPU renders the same set with the full EWA pipeline — presentation). wasm pkg uniquely namedwai_splat4d. - Spec PROMOTED to
extensions/volumetric.md(site mirror), registered in SPEC.md §5 (wai.splat.4d). Honest about size: the win is content-addressing + dedup + a verifiable per-frame state, NOT payload reduction (a real cloud is MB-scale) — stated plainly. - The cargo set is complete across the medium: sight (worlds/replay),
hearing (audio/mixdown), touch (haptics/sample), live state
(feed/replay), a moving body (avatar/pose), and now a moving volume
(splat.4d/frame-state) — one
wai.det.fixed64floor, one verification.
Avatars / digital humans — wai.avatar.pose, LIVE at /avatar (2026-06-13)
- Phase 4 complete — digital humans: motion as instructions. An
avatar is a skeletal world — the worlds container/op-log/ordering/
sealing/state-hash/receipt-chain, with the
stepreduced to the identity. Conformance is pose-equivalence (the per-bone local transforms hash identically), a special case of replay-equivalence. wai.world.sim.skeletal0(inwai-rsworldfeature): a bone is aparent[16] + translation[3] + rotation-quaternion[4] + scale[3](allwai.det.fixed64). Mutations:CREATE/DELETE/SET_ROTATION(the animation driver) /SET_TRANSLATION/SET_SCALE, all LWW. Trig-free floor: rotations are quaternions and the FK composition is+ − ×only; axis-angle authoring happens off-line. The forward-kinematics → world space and the skinning of awai.mesh.glbare presentation (a pure function of the hashed pose), never in the hash. Registered + tieredLossless; dispatched inrun_sealed_dispatch.avatar-conformance/corpus +wai_avatar_vectors gen|verify: a sealed 3-bone arm wave, a sealed 5-bone humanoid pose, and three seeded adversarial-arrival shuffles of the arm wave — all converge, all pass.- Browser sink —
SkeletalSink(wai-web) +/avatardemo page, LIVE: replays a sealed animation in-tab, re-derives the sealed pose hash (checked against the native corpus and the file's own seal), and exposes FK world-space joints + bone segments to draw the skeleton on a rotating orthographic canvas with a tick scrubber. wasm pkg uniquely namedwai_avatar. - Spec PROMOTED to
extensions/avatars.md(site mirror), registered in SPEC.md §5 (wai.avatar.pose). A moving body joins sight (worlds), sound (audio), touch (haptics), and live state (feeds) under one floor.
State-feeds — wai.feed.state, LIVE at /feed (2026-06-13)
- Phase 3 complete — live state-feeds: state as instructions. A
feed is a non-physics world — the same
WWLDcontainer, op log, canonical ordering, sealing, state hash, and receipt chain as interactive-worlds, with the simulationstepreduced to the identity. Conformance is therefore replay-equivalence, reused wholesale: no new criterion. wai.world.sim.feed0(inwai-rsworldfeature): nodes are records of self-describing typed cells (Inti64,FixedQ32.32,TextUTF-8 — hashed without a schema). Mutations:CREATE/DELETE/SET_CELL(LWW) andADD_INT/ADD_FIXED(SUM — commutative accumulation). Wrong-typed / out-of-range / absent-node updates are deterministic no-ops. Registered + tieredLosslessin the centralcodecsregistry; dispatched inrun_sealed_dispatch.feed-conformance/corpus +wai_feed_vectors gen|verify: sealed scoreboard (SUM scoring + an LWW correction), telemetry (LWW gauge + SUM packet count, incl. a negativeADD_FIXED), market tape (LWW last + SUM volume), and three seeded adversarial-arrival shuffles of the scoreboard — all converge, all pass (hash equality).- Browser sink —
FeedSink(wai-web) +/feeddemo page, LIVE: loads a sealed feed, replays it in-tab with the same engine that made the corpus, and re-derives the publisher's sealed BLAKE3 state hash (the file's seal also equals the native corpus hash — both checked). A tick scrubber re-derives the feed at any tick; the record table renders the typed cells generically. wasm pkg uniquely namedwai_feed(avoids the deploy fingerprinter's basename collision). - Spec PROMOTED to
extensions/state-feeds.md(site mirror), registered in SPEC.md §5 (wai.feed.state). The broadcast profile (one authoritative publisher, many read-only sinks) is documented as a deployment shape ofwai.world.live, not a new format. State joins sight (worlds/replay), hearing (audio/mixdown), and touch (haptics/sample) under one floor — verified the same way.
Interactive worlds — world feature + conformance corpus (2026-06-11)
- New cargo class implemented: the interactive-worlds extension
(spec draft staged in
wai-transaction-science-web/research/, destined forextensions/interactive-worlds.md). Media that consumes input: a world is a content-addressed artifact, every event is a signedCausalOp, and conformance is replay-equivalence — identical BLAKE3 state hash at every checkpoint, no tolerance parameter. One container (WWLD) serveswai.world.scene/wai.world.replay/wai.world.live, differing only in op-log seal state; an input log and a multiplayer op log are the same object. wai-rsworldfeature (pure Rust, no ML dep chain; deps: blake3 + ed25519-dalek):FxQ32.32 fixed-point (wai.det.fixed64, the floor profile — bit-exact unconditionally), xoshiro256++ contract rng, signed/dep-gated ops with canonical(effect_tick, lamport, actor)ordering, coordination-free BLAKE3 node minting, WWLD section container with content-binding seal,Simtrait (the extension standardizes the contract, not a physics engine) +kinematic0reference registrant, sealedReplayer+LiveWorldengines (buffering, past-tick re-derivation, equivocation rejection, receipted-checkpoint history sealing).world-conformance/corpus +wai_world_vectors gen|verify: sealed replays, LWW/SUM conflict batteries, dep-gap / past-tick / shuffled-arrival live streams, expected hashes. All vectors pass; clippy clean on the new code.- Spec PROMOTED to
extensions/interactive-worlds.md(site mirrorsrc/pages/extensions/), registered in SPEC.md §5:wai.world.scene/wai.world.replay/wai.world.live+ asset mediawai.mesh.glb(canonical glTF 2.0 Binary, Khronos — decoder sink/library-supplied);media: "world"noted in §3. Capability strings live in the centralcodecsregistry (known_capabilities()), tieredLossless— replay-equivalence is exact state reproduction, the simulation analog of lossless. - Second sim registrant
wai.world.sim.ballistic0(gravity + ground-plane bounce, per-node radius/restitution,G = −10, all Q32.32 — bit-exact through every bounce) proving the registration story: entered registry, dispatch, and corpus (ballistic_bouncevector) without one line of the extension changing. Newrun_sealed_dispatchroutes oncontract.sim; a missing sim reports the world inert (UnknownSimCapability), mirroring SPEC.md §4 step-6 semantics so callers can fall back per the envelope. - Native energy meter — measured joules, no sudo (2026-06-13):
corrected a wrong assumption that real energy needed RAPL/sudo
(unavailable on the arm64 prod box). On Apple Silicon the SoC energy
counters are readable through IOReport with no privileges (the
source
macmon/asitopuse; verified euid≠0 reads CPU/GPU/ANE power on the M3 Ultra), andproc_pid_rusagegives exact per-process retired cycles/instructions. Newwai_world_meterbin (worldfeature): brackets a tight replay burst of a sealed.wwldagainst an idle baseline, samples CPU power via macmon, and reports the marginal measured CPU energy — e.g. the truss replay metered at ~512 µJ / replay (~4.3 µJ/tick) at 3.2 W marginal, a real number that fills a receipt'sjoules_microwhere a native sink runs. The burst runs on N threads so the marginal draw clears a shared box's idle fluctuation (single-thread 512 µJ vs 10-thread 524 µJ/replay cross-validates the method — per-replay energy is invariant to parallelism). With a second arg the meter emits a full §8SessionReceiptcarrying the MEASURED joules + exactwork_steps, signed and self-verified — real measured energy end to end. Spec §8 + the/worldspage corrected: the sink-metering path needs no sudo (IOReport on Apple Silicon, RAPL on x86); the browser estimate is the fallback, not the ceiling. clippy clean, 94 tests green. - Phase 2 begun — haptics (
wai.haptic.signal) (2026-06-13): touch as instructions. Spec drafted (research/haptics-draft.md), capability registered + tiered.wai-rshapticfeature (reuses the worldFxfloor, no codec/ML deps): theWHAPcontainer + a deterministic reconstruct — per channel, a keyframed intensity/sharpness envelope (the Core-Haptics pair every actuator API maps to) linear-interpolated inFxand quantized tou8. Trig-free by construction (amplitude + sharpness, not a modulated carrier), so the floor reconstructs it exactly with+,−,×,÷alone.signal_hashis the sample-equivalence quantity.wai_haptic_vectors gen|verify+haptic-conformance/(ramp, thud, dual-rumble pulse) — all pass, hash equality. 104 tests, clippy clean. Next: Gamepad/Vibration browser sink +/hapticsdemo, then promote. This completes the sight → hearing → touch trio's engines. - Spatial-audio: receipt profile + spec PROMOTED (2026-06-13) —
Phase 1 complete.
audio_scene::SceneReceipt(the worlds receipt transposed to audio): identityscene_hash, the JWP group's objects are the mix's PCM blocks (BLAKE3("wai:audio-block")), Merkle-bound with JWP's exact domains, one Ed25519 sig over scene_hash + root + blocks +work_mac(samples × objects, exact signed) + carriedjoules_micro;parent_receipt_hashchains renders. Reusesworld::receipt::merkle_root— minimal new code. 2 receipt tests (101 total). Draft promoted toextensions/spatial-audio.md(+ site mirror, SPEC §5 row + extension pointer) — and the canonical spec corrected to the as-builtsqrt_constpowerpan (sqrt-only, no transcendental tables) and the real receipt shape, since registration froze the format. clippy clean; spec page live. Phase 1 (spatial audio) DONE: spec · register · mixer · corpus · live /audio sink · receipt · promoted. Next: Phase 2 — haptics. - Spatial-audio browser sink — LIVE at /audio (2026-06-13):
wai-webAudioSink(wasm-bindgen): parseWSCN, load the corpus sources, run the exact deterministic mixdown, exposemixdown_hash_hex+scene_hash_hex+mix(i16 PCM) +object_xz(t). The/audiopage (site-styled, in nav) loads a scene, verifies the mixdown hash in-browser against the corpus (mixdown-equivalence, the same proof worlds does for state), plays the bit-exact stereo mix via an AudioBuffer, and shows a top-down radar of the objects panning. Proven on prod: all three scenes verify, zero errors — wasm-in-Chrome mixes byte-identically to native. Honest split kept: plays the exact stereo floor; binaural/HRTF noted as the richer optional presentation. - Spatial-audio conformance corpus (2026-06-13):
wai_audio_vectors gen|verify+audio-conformance/— the mixdown-equivalence analog of the worlds corpus. Three vectors (static stereo pair, distance near/far, keyframed left→right sweep) ship a.wscnscene + a.sources.bin(decoded mono tones) + expectedscene_hashandmixdown_hash;verifymixes each and checks hash equality, no tolerance. All pass; clippy clean. - Spatial-audio reference mixer — mixdown-equivalence (2026-06-13):
wai-rsaudio_scenefeature (reuses the worldFxfloor + blake3, no codec/ML deps): theWSCNcontainer (scene contract + object table- asset table + optional seal, mirrors WWLD) and a fully deterministic
mixdown —
Fxposition interpolation,inverse_clampeddistance attenuation,sqrt_constpowerpan (sqrt only, no transcendental tables),Fxaccumulate, round-half-away i16 quantization in integer space.mixdown_hash(BLAKE3 over canonical i16LE PCM for a declared layout) is the mixdown-equivalence quantity;scene_hashis the identity. 5 tests prove it: centered = equal L/R, hard-left = silent right, deterministic + layout-bound hash, distance attenuates, a moving source sweeps L→R + container round-trips. Sources arrive as decoded mono PCM keyed by content hash — decoding thewai.audio.*assets is the sink's job. clippy clean. Next: conformance corpus bin + Web Audio browser sink + demo.
- asset table + optional seal, mirrors WWLD) and a fully deterministic
mixdown —
- Content taxonomy + cargo roadmap; spatial-audio Phase 1 begun (2026-06-13):
mapped the full space of web-transported content WAI's approach covers
(
research/content-taxonomy.md) — a capture→synthesizable→executable→ symbolic spectrum × four levers (codec dispatch, neural prior, instructions-at-sink, determinism+receipts), with a per-row mechanism/extension/status table and the white space called out. A phased build plan (research/cargo-roadmap.md) takes each white-space class through the proven interactive-worlds recipe. Phase 1 kicked off — object-based spatial audio (wai.audio.scene): the audio analog of interactive-worlds (ship sound objects + a scene, render to the sink's layout; mixdown-equivalence = replay-equivalence with the tick replaced by the audio sample). Spec drafted (research/spatial-audio-draft.md,WSCNcontainer,Fxmixdown, content-addressedwai.audio.*sources, stereo bit-exact floor + binaural perceptual tier, WAI2 recorded-mix fallback, JWP receipt profile);wai.audio.sceneregistered + tiered Lossless. 94 tests green. Remaining phases: haptics, live state-feeds, digital humans, volumetric/4D splats. - Real SPZ decode —
wai.splat.spzend to end (2026-06-13): the splat gallery now renders a genuine Niantic-SPZ file, not a mesh-derived stand-in. Newwai-spz-gencrate depends on the field'sspzreference library (the sink-supplied codec WAI dispatches to), encodes a 2,400-Gaussian cube cloud to a real.spz(validated by re-loading through the library), and wraps it as a content-addressedwai.splat.spzWAI object (nebula.spz.wai).gen-galleryadds it to the scene's asset table (role 1) beside the glb. Browser:WorldSink.asset_payload(name, env, cap)generalizes the §4.1 verify to any capability; the page gunzips (DecompressionStream) and decodes the SPZ bitstream in-tab — 16-byte header, non-interleaved arrays, 24-bit fixed-point positions, log scales, smallest-three quaternions, byte/255 opacity+color — into per-Gaussian world covariance for the EWA pipeline. Decoded count matches the encoded 2,400 (validates the layout); the cube silhouette + surface-aligned discs validate the quaternion math. Page copy corrected: the splat view decodes a real SPZ asset (no longer "derived from mesh"). Verified locally (2,400 Gaussians, zero errors); prod serves the identical artifacts. - Anisotropic EWA splatting (2026-06-13): upgraded the splat renderer from isotropic billboards to real 3D Gaussian splatting. Each splat carries an anisotropic 3×3 covariance (surface-aligned flat discs: wide in the triangle tangent plane, thin along the normal); the vertex shader projects it to a screen-space conic (Σ' = J·W·Σ·Wᵀ·Jᵀ with the perspective Jacobian), emits a tight 3σ bounding quad, and the fragment evaluates the conic — the textbook 3DGS pipeline, still CPU-sorted back-to-front and alpha-blended. Cam uniform carries view + proj separately (+ viewport/focal) for the Jacobian. Visibly sharper, surface-coherent splats; proven on prod, zero errors.
- 3D render layer — PBR mesh + Gaussian splats (2026-06-13): the
/worldsgallery now renders the content-addressed, hash-verifiedwai.mesh.glbtwo ways in a real 3D path (perspective camera + depth buffer, all inwai-web/demo/world.js, no wasm change): a solid PBR-shaded cube (flat normals from screen-space derivatives, Lambert- spec + sky-ambient, gamma) and a Gaussian-splat cloud —
instanced camera-facing quads with a Gaussian falloff, back-to-front
depth-sorted each frame (painter's order) and alpha-blended. The splat
cloud is derived from the same verified mesh, making the thesis
visible: one content-addressed asset, two sink-side renderings,
presentation never touches the state hash. SPZ binary decode stays
sink-supplied (stated honestly on the page); the renderer is the real
deliverable. Two gallery vectors (
gallery/gallery_splat); the 2D sim path is untouched. Proven on prod: both render, 768 splats from the cube, zero errors.
- spec + sky-ambient, gamma) and a Gaussian-splat cloud —
instanced camera-facing quads with a Gaussian falloff, back-to-front
depth-sorted each frame (painter's order) and alpha-blended. The splat
cloud is derived from the same verified mesh, making the thesis
visible: one content-addressed asset, two sink-side renderings,
presentation never touches the state hash. SPZ binary decode stays
sink-supplied (stated honestly on the page); the renderer is the real
deliverable. Two gallery vectors (
- Receipt chaining + work-meter +
wai.splat.spz(2026-06-12): three follow-ons. (1)SessionReceiptgainsparent_receipt_hash(BLAKE3 of the prior receipt's canonical bytes — the JWP linkage) +receipt_hash()+verify_chain: successive sealed sessions of one world (sharedworld_hash,group_id+1) form an append-only signed timeline. (2) An honest energy meter: receipts carry exact, signed, deterministicwork_steps(Σ node-steps, fromcollect_checkpoints_and_work) AND a separately carriedjoules_micro— never derived in the receipt; the browser supplies a wall-clock estimate at a declared coefficient (labeled "est@4W"), native sinks supply real RAPL, an unmetered sink signs0. (3) Registeredwai.splat.spz— Gaussian-splat scenes as an asset capability parallel towai.mesh.glb(presentation only, never in the state hash; the win is content-addressing + dedup, not payload size; renderer sink-supplied). Browserseal_session(seed, id, joules, group, parent)chains via localStorage; the/worldsHUD showschain #N ↳linked · N steps · ≈X mJ est@4W · ✓ re-verified. Proven on prod: two seals chain (g0 parent null → g1 parent = g0.receipt_hash), exact work 248 steps, est joules carried, zero errors. Spec §8 updated (chaining + the two-quantity energy model). 94 native tests, 10/10 vectors, clippy clean. - Asset path SHIPPED —
wai.mesh.glb, content-addressed (2026-06-12, extension §4.1): worlds are made of WAI objects. Newworld::asset_content_hash(BLAKE3wai:asset\x01over a complete WAI envelope) +DOMAIN_ASSET; native tests PROVE the load-bearing claim — a scene with an asset table has the byte-identical state hash AND world hash as the same scene without it (assets are presentation-only, never in the state hash), and the asset hash binds the exact envelope. Generatorgen-gallerywrites agallery.wwldscene +assets/cube.glb.wai: a real glTF 2.0 Binary cube (hand-written GLB writer) wrapped as awai.mesh.glbWAI envelope and content-addressed in the table. Browser:WorldSink.assets()+asset_glb(name, envelope)(verifies content hash in WASM, rejects a mismatch, unwraps the WAI envelope to the.glbpayload); the page decodes the GLB (minimal JS decoder) and renders the verified mesh as a spinning wireframe. Proven on prod:1/1 verified ✓ · cube.glb, and the state hash stays fixed while the mesh spins — §4.1 made visible. 92 native tests, 10/10 vectors, clippy clean. - Session receipts + seal-to-replay SHIPPED (2026-06-12, extension
§8 implemented):
world/receipt.rsbuilds aSessionReceipt— a JWP profile, not a fork: op batches (one per checkpoint interval, content-hashedwai:world-ops\x01) are Merkle-bound with JWP's exact leaf/node domain separators, and one Ed25519 signature covers the world-profile payload (world hash + checkpointstate_hashlist + carried microjoules). Joules are carried, never fabricated — the builder takes the figure; an unmetered session signs0and says so. Newworld_hash(BLAKE3 over scene = contract + tick-0 graph, the §7.4 shareable identity),collect_checkpoints(unsealed replay → checkpoint list). Browser:WorldSink.seal_session(seed, signer_id, joules)→SealedSession{replay_bytes, receipt_json, root_hash, …}; the/worldspage's "⏷ seal & download replay" turns a live, shared, edited world into a portable.wwld+.receipt.json, re-opens the fresh replay in a throwaway sink to prove it re-verifies, and shows the Merkle root + batch count in the HUD. Proven on prod: fracture a tower across the relay → seal → 3-batch receipt, valid sig, replay re-verified. 90 native tests, 10/10 vectors, clippy clean. - Cross-device multiplayer SHIPPED (2026-06-12,
wai-relay/+wai-webActor binding): every tab/device is a signed actor. NewActorwasm-bindgen class (browser-held Ed25519 key; clicks author realCausalOps; Lamport discipline viaobserve_lamport);wai-world-relayreference transport (axum WS, per-room fan-out, retains the signed op log for late joiners, verifies every op's Ed25519 signature at the door with the samewaicrate — it adjudicates nothing; sinks converge because the math does). Page transport is relay-first (room in the URL hash) with BroadcastChannel fallback. Deployed atwai.transaction.science/worlds-ws/*(launchdcom.wai.world-relay, localhost:9670). Proven on prod with two ISOLATED browser contexts: point-blank axial shoves snap two members (23→21 nodes) and both contexts converge — "✓ hashes agree". Also fixedbuild.rsemitting libFLAC link flags regardless of features (pkg-config probe side effect): now gated oncodecs. - Browser sink SHIPPED (2026-06-12,
wai-web/src/world.rs+wai-web/demo/world.{html,js}): awai.world.replaysink in a tab. NOT a port — thewaicrate itself now compiles to wasm32 via the new default-oncodecsfeature (C codec chain gated; envelope +worldengine build with--no-default-features --features world), and wai-web binds it (WorldSink: advance/particles/members/ state-hash/verify-seal; sim state stays exact Q32.32, f32 only for display). WebGPU renderer (instanced SDF circles + member lines). Proven in real Chrome via Playwright: the 1,282-byte truss_fracture payload replays 120 ticks, the member snaps, and the browser's BLAKE3 state hash equals the native corpus hash byte-for-byte — replay-equivalence across arm64-native and wasm32-in-Chrome, demonstrated, with the Ed25519 seal verified in-page. - Solver-backed registrant
wai.world.sim.truss0(2026-06-12,world/truss.rs): structural simulation — particles + distance members solved per tick by position-based dynamics (fixed 8-pass Gauss–Seidel projection in canonical member order), entirely inwai.det.fixed64including a new floor-roundedFx::sqrt(128-bit integer Newton). Members carry a yield strain measured on predicted positions before the solve; an overloaded member fractures as an emergent solver consequence — two actors pulling the same member in opposite directions (the SUM rule feeding the solver) snap it bit-identically on every machine (truss_fracturecorpus vector). Registration constants:G = −10,ITERATIONS = 8, ground aty = 0, velocity damping63/64/tick (static equilibrium).LiveWorldgainedstate()/current_tick()accessors (what a renderer reads). Where the float-based CAD solvers can't transplant (floats aren't cross-platform bit-exact), solver form does: iterative constraint projection in the deterministic profile.
Inter-frame video, categorical/VQ images, and real-model validation
wai.neural.dcvc_rt/wai.neural.dcvc_fm— inter-frame video decoder implemented (neural/video_dcvc.rs). "Requires CUDA" was a speed claim: inter-frame decode is a frame loop that propagates a reference feature, and ONNX Runtime runs it on CPU. The per-frame entropy decode reuses the CompressAI-validated hyperprior path; the temporal net is sink-supplied. Validated end-to-end (3 inter-frames, feature propagation confirmed).wai.neural.glc— categorical / VQ decoder implemented (neural/glc.rs). Not the Gaussian-conditional family: codebook indices decoded via a per-position softmax →pmf_to_quantized_cdf(bit-exact with CompressAI's C++, unit-tested) → a bypass-free categorical rANS over a checkerboard → VQ synthesis. Surfaced that categorical entropy is the most determinism-sensitive path — a per-position CDF flips on ~1e-5 logit drift where Gaussian scale buckets don't.- Real-model validation beyond synthetic stand-ins. SNAC's genuine
snac_24khznow exports (tools/wai_snac_export.py: zero-noise so the decode is deterministic at all + a reshape-free snake) andwai-rsdecodes it to ~9e-7 of the torch reference. A real PTQ-trained, activation-calibratedint_synthmodel decodes bit-identically (max|Δ|=0) to its integer-forward reference (tools/wai_int_synth_train.py; also fixed a zero-gradienttorch.floorbug in the QAT trainer).
Learned-image codec family — decoders implemented and validated
- The reference crate now decodes the full conditional-entropy family
natively, not just the factorized baseline:
wai.neural.mbt2018_mean(mean-scale hyperprior, Gaussian-conditional),wai.neural.elic(checkerboard spatial context),wai.neural.mlicpp(channel-autoregressive multi-reference), and the integer-exactwai.neural.int_synth(all-integer transposed-conv synthesis — a decode that is bit-identical on every machine by construction, the first capability that can soundly backintent=replicatecross-hardware). The rANS + entropy decode live inwai-rs; the analysis/synthesis nets are sink-supplied ONNX. - Current-SOTA entries added as live routes:
wai.neural.mlicv2(MLICv2/v2+, the current SOTA learned image) andwai.neural.cmic(Content-Aware Mamba) both decode through the multi-reference core — their entropy model is the same channel-slice + checkerboard structure; the gains live in their sink-supplied transforms. - Validation against the reference codec.
bmshj2018andmbt2018_meanare checked byte-for-byte against pretrained CompressAI models (reconstruction matches to within final-layer float rounding);elic/mlicppare checked end-to-end against minimal CompressAI- compatible models;int_synthagainst a constructed integer model. Surfacing this also caught (and the spec now documents) a real determinism hazard: a learned codec's entropy decode desyncs unless the encoder and decoder compute the scale buckets with the same model — float drift between two implementations ofh_sflips bucket assignments and corrupts the stream.
Audio: SNAC hierarchical codec
wai.neural.snac— SNAC's multi-resolution RVQ, distinct from the flat single-grid RVQ runtime: N variable-length code streams at hierarchical rates, decoded via dynamic-arity ONNX inputs. Ubiquitous in low-bitrate TTS.
Removed: off-charter Python runtime (wai-neural/)
- Moved
wai-neural/OUT of this repo to../../wai-neural-prototype/. It was a parallel Python runtime (WAI4generative-semantic container + CLIP-embedding payloads + SD-XL-Turbo regeneration + CLIP-similarity conformance) — a disconnected second implementation in the wrong language. The reference implementation is Rust (wai-rs); nothing referenced the Python package. The idea it proved — a behavioral/perceptual neural path that conforms by semantic similarity rather than a weights hash — is sound and belongs inwai-rsas a future Rust path alongside the bit-exact, prior-pinned path. The Python is kept only as a prototype reference (and it hardcodes a machine-local model path, so it never ran portably). This does NOT touch the Python model-export tools intools/, which are legitimate dev tooling.
Neural condition: working end-to-end
tools/wai_encodec_encode.py/wai_encodec_decode.py— first working neural capability (wai.neural.encodec32): source encodes audio via locally-installed EnCodec, ships only the quantized tokens in a standard WAI envelope (236 MB of model weights stay on the source's disk, never enter the envelope); sink with EnCodec installed decodes. Measured: 5 s of audio → 2.3 KB envelope (141× compressed), model on the wire = 0 B.- Rust SDK gains
wai inspectCLI: parses an envelope, resolves the capability + fallback againstsink_capabilities(), and prints verdict (DECODABLE via primary/via fallback/INERT here).
Browser runtime (wai-web/)
- New crate: wasm-bindgen wrapper exposing
parse(bytes) → WaiEnvelope,pack(manifest_json, payload), andknown_capabilities(). 107 KB wasm + 16 KB JS glue. wai-web/demo/wai.js— thin JS shim withWaiRuntime.register(cap, handler)dispatch. Backends are pluggable (browser-native APIs, transformers.js, WebGPU compute, WebNN, ONNX runtime web) — WAI doesn't bundle any. The sink-side compute fabric is heterogeneous; the deployer picks per capability.wai-web/demo/index.html— drop-and-decode demo with handlers wired to:createImageBitmapforwai.image.{png,jpeg,avif,jxl},AudioContextforwai.audio.{opus,flac}, placeholders forwai.neural.*showing where transformers.js / WebNN / WebGPU plug in. Run withpython3 wai-web/demo/serve.py.- 4 sample
.waifiles inwai-web/demo/samples/covering PNG, JXL, Opus, and EnCodec for testing.
Spec: v1.1 multi-rendition envelope (SPEC.md §6.1)
- New magic
WAI2for envelopes that carry multiple renditions of the same content (e.g. EnCodec + Opus, or AVIF + JPEG-XL). Reframed from "graceful fallback for missing capability" to policy selection — in a self-contained ecosystem every sink has every required capability; renditions exist so the sink picks by compute budget, bandwidth, or runtime preference (browser-native vs WebGPU vs WebNN vs native ML). v1.0 (WAI1) remains valid and unchanged.
v0.3.0
Pivot: WAI is the envelope + capability-dispatch + neural-shared- prior standard. The zeroth condition is a menu of registered SOTA codec capabilities (AVIF, JPEG-XL, PNG, JPEG, Opus, FLAC, AV1, zstd, XZ), not a custom codec. The Python prototype and all hand-rolled codec implementations from v0.2.x were removed; Rust is the single reference implementation and the SDK.
Standard
SPEC.mdrewritten as a capability-menu spec. Mandatory floor:wai.image.png,wai.audio.flac,wai.text.zstd(every conforming sink MUST decode these). Recommended modern set:wai.image.jxl,wai.image.avif,wai.image.jpeg,wai.audio.opus,wai.video.av1,wai.video.av1.lossless,wai.text.xz. Neural capabilities (wai.neural.*) remain OPTIONAL and sink-advertised.- Container envelope (
WAI1magic + length-prefixed manifest + length- prefixed payload) and manifest schema are unchanged from v0.2.x. - Conformance: bit-exact decode-equivalence with each capability's reference library (libjxl / libavif / libopus / libFLAC / rav1e / etc.).
Rust reference (wai-rs/, crate wai 0.3.0)
cargo build --releaseproduceslibwai.dylib/.so/.dll(cdylib),libwai.a(staticlib), andlibwai.rlib(Rust-native). The crate type list is["lib", "cdylib", "staticlib"]— drop-in for any language's C-FFI binding.- Codec wrappers:
image(PNG, JPEG viajpeg-encoder, AVIF viaravif+libavif-imagefor decode, JPEG-XL viajpegxl-rs),opus(RFC 7845 Ogg-Opus container so output drops into ffmpeg/VLC/browsers),claxon+flac-boundfor FLAC,rav1e+dav1dfor AV1 (lossy + lossless),zstd+xz2for text. - Generated C header:
include/wai.his regenerated bybuild.rsviacbindgenon every build. SDK consumers get a typed header rather than hand-writingextern "C"decls. - Cross-platform linkage:
build.rsprobespkg-configfor libflac; the macOS-hardcoded.cargo/config.tomlfrom v0.2.x is gone. Builds clean on macOS (Homebrew) and Linux (apt) withoutRUSTFLAGS.
C FFI / SDK surface
All wai_* functions return a wai_status_t:
| code | meaning |
|---|---|
WAI_OK (0) |
success |
WAI_ERR_INVALID_ARG (1) |
null pointer / mismatched length / out-of-range dimension |
WAI_ERR_CODEC (2) |
codec-level failure (details in wai_last_error()) |
WAI_ERR_CONTAINER (3) |
bad magic / truncated envelope / unparseable manifest JSON |
WAI_ERR_PANIC (4) |
internal Rust panic caught at the boundary (a wrapper bug — never UB into the C caller) |
Every FFI function is wrapped in std::panic::catch_unwind so Rust
panics never unwind into C (UB). av1_decode validates header bounds
to prevent libdav1d's C-level abort(). wai_last_error() is per-
thread, populated on any non-OK return.
Testing
- 13 lib tests: per-codec round-trip + envelope round-trip
- 21 FFI robustness tests: null pointers, length mismatches, dimension overflow, malformed magic, truncated payloads, bad JSON, invalid Opus sample rates, zero fps, garbage decoder input
- 10 fuzz smoke targets: 1024 adversarial random byte sequences per
decoder asserting no panic. Stable Rust, no
cargo-fuzztoolchain needed. Found and fixed a real envelope-truncation panic. wai_benchcorpus walker: quick mode (4 images + 4 audio + 3 text + 2 video) for spot-checks;--fullwalks the entire corpus (169 images + 5 audio + 13 text + 12 video sequences = 588 codec runs in ~9.4 min). Per-codec aggregate stats: runs, ok / fail counts, WAI vs reference byte sizes, PSNR delta vs ffmpeg, encode time.--report file.jsonwrites a structured bench report;wai_bench_diff baseline.json head.jsonflags regressions on bytes (±5% default tolerance), encode time (±25%), or new failures. Non-zero exit on regression — ready for CI gating.- GitHub Actions CI (
.github/workflows/ci.yml): ubuntu + macOS matrix, apt/brew codec deps,cargo test --release, cdylib/staticlib artifact verification, quick bench, JSON report uploaded as artifact.
Honest codec measurements (vs ffmpeg reference, full Kodak/Tecnick/
CLIC/synthetic corpus = 169 images, plus SQAM audio + Silesia text + Derf CIF video)
| codec | runs | size vs ref | PSNR Δ vs ref | notes |
|---|---|---|---|---|
| PNG | 169 | — | lossless | ffmpeg-interop ok |
| JPEG q=50 | 169 | +12.7% | +0.27 dB | q-knob calibrated to libjpeg |
| JPEG q=80 | 169 | +19.7% | −0.31 dB | residual gap is pure-Rust vs libjpeg-turbo asm |
| AVIF q=70 | 169 | — | — | ffmpeg-interop ok |
| JPEG-XL lossless | 169 | — | lossless | bit-exact via libjxl |
| FLAC | 5 | −10.1% | lossless | WAI compression_level 8 vs ffmpeg default 5 |
| Opus 64k | 5 | — | — | standard Ogg-Opus container, ffmpeg-interop ok |
| zstd-19 | 13 | ±0.0% | — | byte-identical to system zstd |
| xz-6 | 13 | ±0.0% | — | byte-identical to system xz |
| AV1 lossy | 12 | +28.3% | — | rav1e vs ffmpeg libsvtav1 (different encoders) |
All 588 corpus runs and all 44 tests pass.
Removed
- The Python prototype (
wai/) and Python BD-rate harness (bench/). v0.2.x's hand-rolled wire format is dead; Rust is the single source of truth. - The
webm_lottie_new/wai-rs/pre-pivot Rust crate (Wasm-decoder-in- bundle model). Replaced by the cleanwai-rs/SOTA-wrapper crate.
v0.2.0 — superseded (historical record)
The v0.2.x direction was hand-rolling custom codec wire formats (DCT + quantization + rANS for image, MDCT for audio, I/P inter-frame for video). Two byte-interoperable reference implementations (Python + Rust) were built and cross-validated.
Why superseded: re-implementing JPEG/MPEG/Opus poorly is strictly worse than wrapping the field's SOTA libraries. WAI's actual value is the envelope + capability dispatch + the neural-shared-prior model, which v0.3 keeps. Custom codecs are gone; standard libraries are in.
What v0.2 had
SPEC.mddefined byte-precise wire formats for image (8×8 DCT-II + YCbCr + JPEG-Q + zigzag + DC-Δ + rANS), audio (orthonormal MDCT + sine window + TDAC), video (I/P inter-frame with integer-pel motion).- Python (
wai/) + Rust (wai-rs/) implementations, byte-equivalent on the lossless paths (rANS, container) and reconstruction-equivalent on the lossy paths. - Honest measured standing: image at JPEG-1992 parity (~5% behind libjpeg), audio uniform-MDCT (not perceptually competitive with MP3), video I/P at ~41% smaller than MJPEG floor but far behind H.264.
Licensing (unchanged)
- Apache-2.0 for the standard and the reference implementation. Royalty-free implementation is a precondition for adoption (VP9/AV1 open-codec precedent; the H.264/H.265 royalty trap is avoided).