# WAI Cargo-Class Roadmap

> The build plan for the white-space content classes in
> `content-taxonomy.md`. Every phase follows the **same proven recipe**
> the interactive-worlds extension established — so this is execution,
> not invention.

## The recipe (every cargo class follows it)

1. **Draft the extension spec** in `research/<class>-draft.md` — scope,
   payload format, conformance criterion, fallback floor, receipt profile.
2. **Register the capability** in `wai-rs/src/codecs/mod.rs`
   (`known_capabilities` + fidelity tier). Registration freezes the
   payload format forever (SPEC §7) — so it lands *with* the drafted format.
3. **Reference implementation** in `wai-rs` behind a feature: the
   container, the deterministic core, and the conformance criterion.
4. **Conformance corpus** + a `gen|verify` bin: vectors + expected
   hashes; pass = hash equality, no tolerance parameter.
5. **Browser sink** in `wai-web` (wasm-bindgen) + a `/worlds`-style demo
   page; prove it on real hardware via Playwright.
6. **JWP receipt profile** — what the §8 receipt carries for this class.
7. **Promote** the draft to `extensions/`, mirror to the site, update
   SPEC §5/§8, CHANGELOG, deploy.

The invariants every class inherits for free: content-addressing,
fallback floor, Ed25519 seals, the JWP receipt chain, exact signed
`work_steps` + measured `joules_micro` (the native `wai_world_meter`
pattern), and a conformance criterion that is the class's own form of
*-equivalence*.

## Conformance criterion per class (the load-bearing design choice)

Each class needs a deterministic, hash-checkable equivalence — the
generalization of decode-equivalence:

| class | equivalence | the determinism floor |
|---|---|---|
| interactive worlds | **replay**-equivalence | `wai.det.fixed64` (shipped) |
| spatial audio | **mixdown**-equivalence | fixed-point PCM mix to a declared layout |
| haptics | **sample**-equivalence | fixed-point actuator samples at a declared rate |
| state-feeds | **replay**-equivalence | reuse the worlds op-log + state hash |
| digital humans | **pose**-equivalence | skeletal transforms in `wai.det.fixed64` |
| volumetric / 4D splats | **frame-state**-equivalence | per-frame Gaussian-cloud hash |

## Phase 0 — shipped (baseline)

Core media codecs (image/audio/video/text + neural family) · JWP
receipts · interactive-worlds (`wai.world.{scene,replay,live}`, three sim
registrants, corpus, browser sink, tab + cross-device relay multiplayer,
seal-to-replay, session receipts) · `wai.mesh.glb` + `wai.splat.spz`
assets · PBR + EWA-splat renderers · native energy meter. Siblings:
`map`, `joule-ui`, `Access`/`Lumina`.

---

## Phase 1 — Object-based spatial audio · `wai.audio.scene`

**Why first:** the cleanest technical analog to interactive-worlds.
Ship sound *objects* + a scene; the sink renders to its own speaker
layout. Mixdown-equivalence is the audio counterpart of
replay-equivalence, and it composes the asset path (each source is a
content-addressed `wai.audio.*` object) + receipts + determinism.

**Payload (`WSCN` container, sketch):**
- scene contract: sample rate, channel layout target(s), pan law id,
  numeric profile (`wai.det.fixed64` for the exact mix).
- object table: N objects, each = `{ source_asset_hash (a wai.audio.*
  WAI object), gain, trajectory (keyframed position over time) }`.
- a deterministic **render contract**: given the objects + a declared
  output layout (`stereo`, `5.1`, `binaural@<hrtf-id>`), the mixed PCM
  is bit-reproducible.

**Conformance:** mixdown-equivalence — same scene + same declared layout
→ identical BLAKE3 over the canonical PCM. Reference mixer: distance
attenuation + constant-power panning, fixed-point accumulate.

**Reference impl:** `wai-rs` `audio_scene` feature — scene container,
deterministic mixer to stereo (+ a binaural profile naming an HRTF as a
requirement-of-existence, like a neural prior). Sources decoded via the
existing audio path.

**Browser sink:** Web Audio API for the *nice* render (PannerNode/HRTF),
plus the deterministic reference mixer for the hash check — same
honest split as worlds (exact state vs. presentation).

**Effort:** medium (DSP: resample, pan, fixed-point mix). **Risk:** the
fixed-point mix must be exact across machines — same discipline as
`Fx`.

---

## Phase 2 — Haptics · `wai.haptic.signal`

**Why second:** lowest effort, pure instructions-at-sink, trivially
deterministic. A tiny event/waveform stream rendered on actuators.

**Payload:** a contract (sample rate, channel→actuator map) + either a
keyframed envelope or a fixed-point waveform per channel. Often a few
hundred bytes for an effect.

**Conformance:** sample-equivalence — the reconstructed actuator sample
buffer hashes identically.

**Reference impl:** `wai-rs` `haptic` feature — container + reconstruct
to a PCM-like buffer. **Browser sink:** Gamepad `vibrationActuator` /
the Vibration API; visualize the waveform on canvas as the fallback.

**Effort:** low. **Risk:** low. Good momentum builder after Phase 1.

---

## Phase 3 — Live state-feeds · broadcast profile + `wai.feed.state`

**Why:** the multiplayer-games precedent applied to broadcast — "ship
state, not pixels" for sports/events/telemetry. Most of it already
exists.

**Two pieces:**
1. **Broadcast profile of `wai.world.live`** — one authoritative
   publisher, many read-only sinks, over the existing relay. A spectator
   floor (an AV1 stream of a capable participant's render) per §9.
2. **`wai.feed.state`** — a *non-physics* state feed (scoreboard,
   telemetry, market ticks): a signed op log over a declared schema with
   the same canonical ordering + state hash, no sim. Effectively
   interactive-worlds with a trivial "sim = apply the typed field
   updates."

**Conformance:** replay-equivalence (reuse the worlds machinery).

**Effort:** medium (mostly profiling + relay broadcast mode). Reuses
relay, receipts, op log, state hash.

---

## Phase 4 — Digital humans / avatars · skinned mesh + skeletal animation

**Why:** highest-bandwidth capture, most reducible to rig + stream.
Composes shipped pieces.

**Pieces:**
- **Skinned-mesh asset**: `wai.mesh.glb` already carries skeletal
  armatures + skin weights (glTF) — decode is sink-supplied.
- **Skeletal sim registrant** (`wai.world.sim.skeletal`): nodes are
  bones; ops set joint transforms in `wai.det.fixed64`; the sink skins
  the mesh by the deterministic pose.
- **Animation / expression op stream**: the existing CausalOp log drives
  the rig — a telepresence stream is a live world of one avatar.

**Conformance:** pose-equivalence (bone transforms hash identically) —
a special case of replay-equivalence.

**Effort:** medium-high (skinning on the GPU; pose determinism).

---

## Phase 5 — Volumetric / light-field · `wai.splat.4d` (temporal Gaussians)

**Why last:** the frontier of "video"; no clean envelope exists. Extends
the splat path to time.

**Payload:** a base Gaussian cloud + per-frame deltas, OR per-Gaussian
trajectories (position/rotation/scale/opacity keyframes). Sink
interpolates and renders with the existing EWA pipeline.

**Conformance:** frame-state-equivalence — the Gaussian set at frame N
hashes identically (presentation render stays the renderer's business).

**Effort:** high. **Risk:** size (volumetric is heavy — the win is
content-addressing + dedup + receipts, *not* payload reduction; state
that honestly, as with static splats).

---

## Sequencing rationale

1. **Spatial audio** — cleanest analog, highest "proves the thesis
   generalizes" value, composes the asset path.
2. **Haptics** — fast win, completes the "every sense" story (sight via
   worlds/splats, hearing via audio-scene, touch via haptics).
3. **State-feeds** — biggest commercial wedge, mostly already built.
4. **Digital humans** — high value, composes worlds + mesh.
5. **Volumetric** — frontier, heaviest, do last.

Cross-cutting, anytime: surface **content-addressing + JWP receipts +
measured joules** on the captured-reality classes (photos, recordings,
telemetry) — the lever-4 win that needs no reconstruction and applies to
the content that can't be reduced to instructions.