348a2d9259
The retrofit's commit says the research notes, extensions and source comments make no comparisons. These lines still ranked or promoted: - research/cargo-roadmap.md: the sequencing rationale and the phase reasons now say what each phase reuses or carries, and the status note points to content-taxonomy.md for each class's status. - research/content-taxonomy.md: the "white space" table is now the cargo-roadmap phase table, with each phase marked shipped and its capability named, so it no longer contradicts the taxonomy rows above it; the closing clause of the spatial-audio row and "That's the whole map" are removed. - extensions/spatial-audio.md and research/spatial-audio-draft.md: the appendix says why spatial audio was first without ranking it. - SPEC.md §5, extensions/ternary-tt.md and the CAP_TERNARY_TT comment: wai.ternary.tt is stated by its own measured operating point (13.3 dB SNR at 3.8x with six residual layers, the extension's table) and its Lossy tier. - research/2030-transport-landscape.md: three lines restated as what WAI specifies; the CHANGELOG entry now says comparative framing in dated records is restated in place, as names are redacted. - wai-rs/src/lib.rs: the quantum_qec comment ends at the metric it carries. scripts/check-comparative-prose.sh gains five phrases (commercial wedge, unclaimed, fast win, differentiator, rate-competitive), each with a case in scripts/test-public-text-checks.sh. REGISTRY.md and vocab/README.md say that the v2 SHACL shape checks the capability and fallback against this revision's registry and only the form of media, and that a violation for an unregistered string means it is not registered in this revision, not that the envelope fails to conform or cannot be dispatched (SPEC §4, §8). The CHANGELOG's candidate-capability bullet states the MINOR rule as a full sentence. Co-Authored-By: Claude Opus 5.5 <[email protected]>
194 lines
8.4 KiB
Markdown
194 lines
8.4 KiB
Markdown
# WAI Cargo-Class Roadmap
|
||
|
||
> **Status: Phases 1–5 complete (v0.4.0).** Spatial audio, haptics,
|
||
> state-feeds, avatars, and volumetric/4D splats all shipped, promoted to
|
||
> `extensions/`, and live. This document is now the archived plan of
|
||
> record — see `CHANGELOG.md` for current state and `content-taxonomy.md`
|
||
> for each content class's status.
|
||
>
|
||
> The build plan for the content classes phased in `content-taxonomy.md`.
|
||
> Every phase follows the **same recipe** the interactive-worlds extension
|
||
> established.
|
||
|
||
## The recipe (every cargo class follows it)
|
||
|
||
1. **Draft the extension spec** in `research/<class>-draft.md` — scope,
|
||
payload format, conformance criterion, fallback floor, receipt profile.
|
||
2. **Register the capability** in `wai-rs/src/codecs/mod.rs`
|
||
(`known_capabilities` + fidelity tier). Registration freezes the
|
||
payload format forever (SPEC §7) — so it lands *with* the drafted format.
|
||
3. **Reference implementation** in `wai-rs` behind a feature: the
|
||
container, the deterministic core, and the conformance criterion.
|
||
4. **Conformance corpus** + a `gen|verify` bin: vectors + expected
|
||
hashes; pass = hash equality, no tolerance parameter.
|
||
5. **Browser sink** in `wai-web` (wasm-bindgen) + a `/worlds`-style demo
|
||
page; prove it on real hardware via Playwright.
|
||
6. **JWP receipt profile** — what the §8 receipt carries for this class.
|
||
7. **Promote** the draft to `extensions/`, mirror to the site, update
|
||
SPEC §5/§8, CHANGELOG, deploy.
|
||
|
||
The invariants every class inherits for free: content-addressing,
|
||
fallback floor, Ed25519 seals, the JWP receipt chain, exact signed
|
||
`work_steps` + measured `joules_micro` (the native `wai_world_meter`
|
||
pattern), and a conformance criterion that is the class's own form of
|
||
*-equivalence*.
|
||
|
||
## Conformance criterion per class (the load-bearing design choice)
|
||
|
||
Each class needs a deterministic, hash-checkable equivalence — the
|
||
generalization of decode-equivalence:
|
||
|
||
| class | equivalence | the determinism floor |
|
||
|---|---|---|
|
||
| interactive worlds | **replay**-equivalence | `wai.det.fixed64` (shipped) |
|
||
| spatial audio | **mixdown**-equivalence | fixed-point PCM mix to a declared layout |
|
||
| haptics | **sample**-equivalence | fixed-point actuator samples at a declared rate |
|
||
| state-feeds | **replay**-equivalence | reuse the worlds op-log + state hash |
|
||
| digital humans | **pose**-equivalence | skeletal transforms in `wai.det.fixed64` |
|
||
| volumetric / 4D splats | **frame-state**-equivalence | per-frame Gaussian-cloud hash |
|
||
|
||
## Phase 0 — shipped (baseline)
|
||
|
||
Core media codecs (image/audio/video/text + neural family) · JWP
|
||
receipts · interactive-worlds (`wai.world.{scene,replay,live}`, three sim
|
||
registrants, corpus, browser sink, tab + cross-device relay multiplayer,
|
||
seal-to-replay, session receipts) · `wai.mesh.glb` + `wai.splat.spz`
|
||
assets · PBR + EWA-splat renderers · native energy meter. Siblings:
|
||
`map`, `joule-ui`, `Access`/`Lumina`.
|
||
|
||
---
|
||
|
||
## Phase 1 — Object-based spatial audio · `wai.audio.scene`
|
||
|
||
**Why first:** it is interactive-worlds with the tick replaced by the
|
||
audio sample. Ship sound *objects* + a scene; the sink renders to its own speaker
|
||
layout. Mixdown-equivalence is the audio counterpart of
|
||
replay-equivalence, and it composes the asset path (each source is a
|
||
content-addressed `wai.audio.*` object) + receipts + determinism.
|
||
|
||
**Payload (`WSCN` container, sketch):**
|
||
- scene contract: sample rate, channel layout target(s), pan law id,
|
||
numeric profile (`wai.det.fixed64` for the exact mix).
|
||
- object table: N objects, each = `{ source_asset_hash (a wai.audio.*
|
||
WAI object), gain, trajectory (keyframed position over time) }`.
|
||
- a deterministic **render contract**: given the objects + a declared
|
||
output layout (`stereo`, `5.1`, `binaural@<hrtf-id>`), the mixed PCM
|
||
is bit-reproducible.
|
||
|
||
**Conformance:** mixdown-equivalence — same scene + same declared layout
|
||
→ identical BLAKE3 over the canonical PCM. Reference mixer: distance
|
||
attenuation + constant-power panning, fixed-point accumulate.
|
||
|
||
**Reference impl:** `wai-rs` `audio_scene` feature — scene container,
|
||
deterministic mixer to stereo (+ a binaural profile naming an HRTF as a
|
||
requirement-of-existence, like a neural prior). Sources decoded via the
|
||
existing audio path.
|
||
|
||
**Browser sink:** Web Audio API for the *nice* render (PannerNode/HRTF),
|
||
plus the deterministic reference mixer for the hash check — same
|
||
honest split as worlds (exact state vs. presentation).
|
||
|
||
**Effort:** medium (DSP: resample, pan, fixed-point mix). **Risk:** the
|
||
fixed-point mix must be exact across machines — same discipline as
|
||
`Fx`.
|
||
|
||
---
|
||
|
||
## Phase 2 — Haptics · `wai.haptic.signal`
|
||
|
||
**Why second:** lowest effort, pure instructions-at-sink, trivially
|
||
deterministic. A tiny event/waveform stream rendered on actuators.
|
||
|
||
**Payload:** a contract (sample rate, channel→actuator map) + either a
|
||
keyframed envelope or a fixed-point waveform per channel. Often a few
|
||
hundred bytes for an effect.
|
||
|
||
**Conformance:** sample-equivalence — the reconstructed actuator sample
|
||
buffer hashes identically.
|
||
|
||
**Reference impl:** `wai-rs` `haptic` feature — container + reconstruct
|
||
to a PCM-like buffer. **Browser sink:** Gamepad `vibrationActuator` /
|
||
the Vibration API; visualize the waveform on canvas as the fallback.
|
||
|
||
**Effort:** low. **Risk:** low. Good momentum builder after Phase 1.
|
||
|
||
---
|
||
|
||
## Phase 3 — Live state-feeds · broadcast profile + `wai.feed.state`
|
||
|
||
**Why:** the multiplayer-games precedent applied to broadcast — "ship
|
||
state, not pixels" for sports/events/telemetry. Most of it already
|
||
exists.
|
||
|
||
**Two pieces:**
|
||
1. **Broadcast profile of `wai.world.live`** — one authoritative
|
||
publisher, many read-only sinks, over the existing relay. A spectator
|
||
floor (an AV1 stream of a capable participant's render) per §9.
|
||
2. **`wai.feed.state`** — a *non-physics* state feed (scoreboard,
|
||
telemetry, market ticks): a signed op log over a declared schema with
|
||
the same canonical ordering + state hash, no sim. Effectively
|
||
interactive-worlds with a trivial "sim = apply the typed field
|
||
updates."
|
||
|
||
**Conformance:** replay-equivalence (reuse the worlds machinery).
|
||
|
||
**Effort:** medium (mostly profiling + relay broadcast mode). Reuses
|
||
relay, receipts, op log, state hash.
|
||
|
||
---
|
||
|
||
## Phase 4 — Digital humans / avatars · skinned mesh + skeletal animation
|
||
|
||
**Why:** highest-bandwidth capture, most reducible to rig + stream.
|
||
Composes shipped pieces.
|
||
|
||
**Pieces:**
|
||
- **Skinned-mesh asset**: `wai.mesh.glb` already carries skeletal
|
||
armatures + skin weights (glTF) — decode is sink-supplied.
|
||
- **Skeletal sim registrant** (`wai.world.sim.skeletal`): nodes are
|
||
bones; ops set joint transforms in `wai.det.fixed64`; the sink skins
|
||
the mesh by the deterministic pose.
|
||
- **Animation / expression op stream**: the existing CausalOp log drives
|
||
the rig — a telepresence stream is a live world of one avatar.
|
||
|
||
**Conformance:** pose-equivalence (bone transforms hash identically) —
|
||
a special case of replay-equivalence.
|
||
|
||
**Effort:** medium-high (skinning on the GPU; pose determinism).
|
||
|
||
---
|
||
|
||
## Phase 5 — Volumetric / light-field · `wai.splat.4d` (temporal Gaussians)
|
||
|
||
**Why last:** it is the heaviest payload of the five. Extends the splat
|
||
path to time.
|
||
|
||
**Payload:** a base Gaussian cloud + per-frame deltas, OR per-Gaussian
|
||
trajectories (position/rotation/scale/opacity keyframes). Sink
|
||
interpolates and renders with the existing EWA pipeline.
|
||
|
||
**Conformance:** frame-state-equivalence — the Gaussian set at frame N
|
||
hashes identically (presentation render stays the renderer's business).
|
||
|
||
**Effort:** high. **Risk:** size (volumetric is heavy — what the class
|
||
carries is content-addressing + dedup + receipts, *not* payload
|
||
reduction; state that plainly, as with static splats).
|
||
|
||
---
|
||
|
||
## Sequencing rationale
|
||
|
||
1. **Spatial audio** — reuses the worlds floor, seal and receipt profile
|
||
verbatim with a mixer as the sim, and composes the asset path.
|
||
2. **Haptics** — a small payload and a pure recipe; with it, sight
|
||
(worlds/splats), hearing (audio-scene) and touch (haptics) each have a
|
||
class.
|
||
3. **State-feeds** — the broadcast profile of `wai.world.live`; most of it
|
||
was already built.
|
||
4. **Digital humans** — composes worlds + mesh.
|
||
5. **Volumetric** — the heaviest payload; built last.
|
||
|
||
Cross-cutting, anytime: surface **content-addressing + JWP receipts +
|
||
measured joules** on the captured-reality classes (photos, recordings,
|
||
telemetry) — lever 4, which needs no reconstruction and applies to
|
||
the content that can't be reduced to instructions.
|