46c5688b86
Commit the WAI standard source: a content + capability-dispatch standard
whose thesis is "carries whatever it can reconstruct, verifies whatever
it carries." Source only — large vendored/model assets and build output
are gitignored.
Reconstruct classes (each: spec + conformance corpus + in-tab browser
sink that re-derives the corpus hash, one wai.det.fixed64 fixed-point
floor, one *-equivalence apiece):
- interactive worlds (replay-equivalence) + 3 sim registrants,
multiplayer over a relay, session receipts, glTF + real SPZ assets
- spatial audio (mixdown-equivalence)
- haptics (sample-equivalence)
- state-feeds (replay-equivalence, non-physics sim)
- avatars / digital humans (pose-equivalence) + telepresence over the relay
- volumetric / 4D splats (frame-state-equivalence) + WebGPU EWA render,
real-SPZ-derived clip
Verify half (lever 4):
- asset provenance: a signed, energy-accounted lineage for opaque
captured content WAI does not reconstruct; tamper-evident, in-tab.
Cross-cutting: JWP receipt profiles per class; a native energy meter
(IOReport via macmon, no sudo) emitting receipts with REAL measured
joules; the learned-image/neural codec family and JPEG AI entropy port
from earlier work.
Co-Authored-By: Claude Opus 4.8 <[email protected]>
187 lines
8 KiB
Markdown
187 lines
8 KiB
Markdown
# WAI Cargo-Class Roadmap
|
|
|
|
> The build plan for the white-space content classes in
|
|
> `content-taxonomy.md`. Every phase follows the **same proven recipe**
|
|
> the interactive-worlds extension established — so this is execution,
|
|
> not invention.
|
|
|
|
## The recipe (every cargo class follows it)
|
|
|
|
1. **Draft the extension spec** in `research/<class>-draft.md` — scope,
|
|
payload format, conformance criterion, fallback floor, receipt profile.
|
|
2. **Register the capability** in `wai-rs/src/codecs/mod.rs`
|
|
(`known_capabilities` + fidelity tier). Registration freezes the
|
|
payload format forever (SPEC §7) — so it lands *with* the drafted format.
|
|
3. **Reference implementation** in `wai-rs` behind a feature: the
|
|
container, the deterministic core, and the conformance criterion.
|
|
4. **Conformance corpus** + a `gen|verify` bin: vectors + expected
|
|
hashes; pass = hash equality, no tolerance parameter.
|
|
5. **Browser sink** in `wai-web` (wasm-bindgen) + a `/worlds`-style demo
|
|
page; prove it on real hardware via Playwright.
|
|
6. **JWP receipt profile** — what the §8 receipt carries for this class.
|
|
7. **Promote** the draft to `extensions/`, mirror to the site, update
|
|
SPEC §5/§8, CHANGELOG, deploy.
|
|
|
|
The invariants every class inherits for free: content-addressing,
|
|
fallback floor, Ed25519 seals, the JWP receipt chain, exact signed
|
|
`work_steps` + measured `joules_micro` (the native `wai_world_meter`
|
|
pattern), and a conformance criterion that is the class's own form of
|
|
*-equivalence*.
|
|
|
|
## Conformance criterion per class (the load-bearing design choice)
|
|
|
|
Each class needs a deterministic, hash-checkable equivalence — the
|
|
generalization of decode-equivalence:
|
|
|
|
| class | equivalence | the determinism floor |
|
|
|---|---|---|
|
|
| interactive worlds | **replay**-equivalence | `wai.det.fixed64` (shipped) |
|
|
| spatial audio | **mixdown**-equivalence | fixed-point PCM mix to a declared layout |
|
|
| haptics | **sample**-equivalence | fixed-point actuator samples at a declared rate |
|
|
| state-feeds | **replay**-equivalence | reuse the worlds op-log + state hash |
|
|
| digital humans | **pose**-equivalence | skeletal transforms in `wai.det.fixed64` |
|
|
| volumetric / 4D splats | **frame-state**-equivalence | per-frame Gaussian-cloud hash |
|
|
|
|
## Phase 0 — shipped (baseline)
|
|
|
|
Core media codecs (image/audio/video/text + neural family) · JWP
|
|
receipts · interactive-worlds (`wai.world.{scene,replay,live}`, three sim
|
|
registrants, corpus, browser sink, tab + cross-device relay multiplayer,
|
|
seal-to-replay, session receipts) · `wai.mesh.glb` + `wai.splat.spz`
|
|
assets · PBR + EWA-splat renderers · native energy meter. Siblings:
|
|
`map`, `joule-ui`, `Access`/`Lumina`.
|
|
|
|
---
|
|
|
|
## Phase 1 — Object-based spatial audio · `wai.audio.scene`
|
|
|
|
**Why first:** the cleanest technical analog to interactive-worlds.
|
|
Ship sound *objects* + a scene; the sink renders to its own speaker
|
|
layout. Mixdown-equivalence is the audio counterpart of
|
|
replay-equivalence, and it composes the asset path (each source is a
|
|
content-addressed `wai.audio.*` object) + receipts + determinism.
|
|
|
|
**Payload (`WSCN` container, sketch):**
|
|
- scene contract: sample rate, channel layout target(s), pan law id,
|
|
numeric profile (`wai.det.fixed64` for the exact mix).
|
|
- object table: N objects, each = `{ source_asset_hash (a wai.audio.*
|
|
WAI object), gain, trajectory (keyframed position over time) }`.
|
|
- a deterministic **render contract**: given the objects + a declared
|
|
output layout (`stereo`, `5.1`, `binaural@<hrtf-id>`), the mixed PCM
|
|
is bit-reproducible.
|
|
|
|
**Conformance:** mixdown-equivalence — same scene + same declared layout
|
|
→ identical BLAKE3 over the canonical PCM. Reference mixer: distance
|
|
attenuation + constant-power panning, fixed-point accumulate.
|
|
|
|
**Reference impl:** `wai-rs` `audio_scene` feature — scene container,
|
|
deterministic mixer to stereo (+ a binaural profile naming an HRTF as a
|
|
requirement-of-existence, like a neural prior). Sources decoded via the
|
|
existing audio path.
|
|
|
|
**Browser sink:** Web Audio API for the *nice* render (PannerNode/HRTF),
|
|
plus the deterministic reference mixer for the hash check — same
|
|
honest split as worlds (exact state vs. presentation).
|
|
|
|
**Effort:** medium (DSP: resample, pan, fixed-point mix). **Risk:** the
|
|
fixed-point mix must be exact across machines — same discipline as
|
|
`Fx`.
|
|
|
|
---
|
|
|
|
## Phase 2 — Haptics · `wai.haptic.signal`
|
|
|
|
**Why second:** lowest effort, pure instructions-at-sink, trivially
|
|
deterministic. A tiny event/waveform stream rendered on actuators.
|
|
|
|
**Payload:** a contract (sample rate, channel→actuator map) + either a
|
|
keyframed envelope or a fixed-point waveform per channel. Often a few
|
|
hundred bytes for an effect.
|
|
|
|
**Conformance:** sample-equivalence — the reconstructed actuator sample
|
|
buffer hashes identically.
|
|
|
|
**Reference impl:** `wai-rs` `haptic` feature — container + reconstruct
|
|
to a PCM-like buffer. **Browser sink:** Gamepad `vibrationActuator` /
|
|
the Vibration API; visualize the waveform on canvas as the fallback.
|
|
|
|
**Effort:** low. **Risk:** low. Good momentum builder after Phase 1.
|
|
|
|
---
|
|
|
|
## Phase 3 — Live state-feeds · broadcast profile + `wai.feed.state`
|
|
|
|
**Why:** the multiplayer-games precedent applied to broadcast — "ship
|
|
state, not pixels" for sports/events/telemetry. Most of it already
|
|
exists.
|
|
|
|
**Two pieces:**
|
|
1. **Broadcast profile of `wai.world.live`** — one authoritative
|
|
publisher, many read-only sinks, over the existing relay. A spectator
|
|
floor (an AV1 stream of a capable participant's render) per §9.
|
|
2. **`wai.feed.state`** — a *non-physics* state feed (scoreboard,
|
|
telemetry, market ticks): a signed op log over a declared schema with
|
|
the same canonical ordering + state hash, no sim. Effectively
|
|
interactive-worlds with a trivial "sim = apply the typed field
|
|
updates."
|
|
|
|
**Conformance:** replay-equivalence (reuse the worlds machinery).
|
|
|
|
**Effort:** medium (mostly profiling + relay broadcast mode). Reuses
|
|
relay, receipts, op log, state hash.
|
|
|
|
---
|
|
|
|
## Phase 4 — Digital humans / avatars · skinned mesh + skeletal animation
|
|
|
|
**Why:** highest-bandwidth capture, most reducible to rig + stream.
|
|
Composes shipped pieces.
|
|
|
|
**Pieces:**
|
|
- **Skinned-mesh asset**: `wai.mesh.glb` already carries skeletal
|
|
armatures + skin weights (glTF) — decode is sink-supplied.
|
|
- **Skeletal sim registrant** (`wai.world.sim.skeletal`): nodes are
|
|
bones; ops set joint transforms in `wai.det.fixed64`; the sink skins
|
|
the mesh by the deterministic pose.
|
|
- **Animation / expression op stream**: the existing CausalOp log drives
|
|
the rig — a telepresence stream is a live world of one avatar.
|
|
|
|
**Conformance:** pose-equivalence (bone transforms hash identically) —
|
|
a special case of replay-equivalence.
|
|
|
|
**Effort:** medium-high (skinning on the GPU; pose determinism).
|
|
|
|
---
|
|
|
|
## Phase 5 — Volumetric / light-field · `wai.splat.4d` (temporal Gaussians)
|
|
|
|
**Why last:** the frontier of "video"; no clean envelope exists. Extends
|
|
the splat path to time.
|
|
|
|
**Payload:** a base Gaussian cloud + per-frame deltas, OR per-Gaussian
|
|
trajectories (position/rotation/scale/opacity keyframes). Sink
|
|
interpolates and renders with the existing EWA pipeline.
|
|
|
|
**Conformance:** frame-state-equivalence — the Gaussian set at frame N
|
|
hashes identically (presentation render stays the renderer's business).
|
|
|
|
**Effort:** high. **Risk:** size (volumetric is heavy — the win is
|
|
content-addressing + dedup + receipts, *not* payload reduction; state
|
|
that honestly, as with static splats).
|
|
|
|
---
|
|
|
|
## Sequencing rationale
|
|
|
|
1. **Spatial audio** — cleanest analog, highest "proves the thesis
|
|
generalizes" value, composes the asset path.
|
|
2. **Haptics** — fast win, completes the "every sense" story (sight via
|
|
worlds/splats, hearing via audio-scene, touch via haptics).
|
|
3. **State-feeds** — biggest commercial wedge, mostly already built.
|
|
4. **Digital humans** — high value, composes worlds + mesh.
|
|
5. **Volumetric** — frontier, heaviest, do last.
|
|
|
|
Cross-cutting, anytime: surface **content-addressing + JWP receipts +
|
|
measured joules** on the captured-reality classes (photos, recordings,
|
|
telemetry) — the lever-4 win that needs no reconstruction and applies to
|
|
the content that can't be reduced to instructions.
|