348a2d9259
The retrofit's commit says the research notes, extensions and source comments make no comparisons. These lines still ranked or promoted: - research/cargo-roadmap.md: the sequencing rationale and the phase reasons now say what each phase reuses or carries, and the status note points to content-taxonomy.md for each class's status. - research/content-taxonomy.md: the "white space" table is now the cargo-roadmap phase table, with each phase marked shipped and its capability named, so it no longer contradicts the taxonomy rows above it; the closing clause of the spatial-audio row and "That's the whole map" are removed. - extensions/spatial-audio.md and research/spatial-audio-draft.md: the appendix says why spatial audio was first without ranking it. - SPEC.md §5, extensions/ternary-tt.md and the CAP_TERNARY_TT comment: wai.ternary.tt is stated by its own measured operating point (13.3 dB SNR at 3.8x with six residual layers, the extension's table) and its Lossy tier. - research/2030-transport-landscape.md: three lines restated as what WAI specifies; the CHANGELOG entry now says comparative framing in dated records is restated in place, as names are redacted. - wai-rs/src/lib.rs: the quantum_qec comment ends at the metric it carries. scripts/check-comparative-prose.sh gains five phrases (commercial wedge, unclaimed, fast win, differentiator, rate-competitive), each with a case in scripts/test-public-text-checks.sh. REGISTRY.md and vocab/README.md say that the v2 SHACL shape checks the capability and fallback against this revision's registry and only the form of media, and that a violation for an unregistered string means it is not registered in this revision, not that the envelope fails to conform or cannot be dispatched (SPEC §4, §8). The CHANGELOG's candidate-capability bullet states the MINOR rule as a full sentence. Co-Authored-By: Claude Opus 5.5 <[email protected]>
121 lines
7.8 KiB
Markdown
121 lines
7.8 KiB
Markdown
# WAI Content Taxonomy — the bearings doc
|
||
|
||
> Living reference for *what web-transported content benefits from WAI,
|
||
> how, and which extension covers it.* Not normative; it's the map we
|
||
> plan cargo classes against. Edit as classes ship.
|
||
|
||
## 1. The reframe: a spectrum, not a list of media types
|
||
|
||
Everything transported over the web sits on one axis:
|
||
|
||
```
|
||
captured reality ──► synthesizable ──► executable ──► symbolic
|
||
(samples) (rendered from (it IS a (already
|
||
a description) program) instructions)
|
||
```
|
||
|
||
**WAI's leverage is proportional to how much of the content the sink can
|
||
reconstruct instead of receiving pre-rendered.**
|
||
The discipline it enforces: do *not* claim "everything becomes
|
||
instructions" — a real photograph is irreducibly samples. Different
|
||
position on the axis → a different lever, and a different gain.
|
||
|
||
## 2. The four levers
|
||
|
||
| # | Lever | What it does | Where it applies |
|
||
|---|---|---|---|
|
||
| 1 | **Codec dispatch** | name a registered codec; the sink decodes it or falls back | all media |
|
||
| 2 | **Neural shared-prior** | ship conditioning, regenerate against a requirement-of-existence model | perceptual media a model can reconstruct |
|
||
| 3 | **Instructions-at-the-sink** | ship the recipe / state / intent; the sink renders | anything whose payload is secretly a rendered result |
|
||
| 4 | **Determinism + receipts** | replay/decode/mixdown-equivalence, content-addressing, JWP signatures, joules | **cross-cutting** — applies even to content that can't be reconstructed |
|
||
|
||
Levers 1–2 are the **compression** story. Lever 3 is the **medium-shift**
|
||
story: the unit of value moves from bytes delivered to energy per experience.
|
||
Lever 4 is the **trust-and-energy** story, and it rides on top of all of
|
||
them.
|
||
|
||
## 3. The taxonomy (with mechanism + coverage)
|
||
|
||
Status: ✅ shipped · 🟡 partial/composable from shipped pieces · ⬜ white space.
|
||
|
||
### A. Captured reality — irreducible samples (levers 1, 2, 4)
|
||
|
||
| Content | Primary lever | Capability / extension | Status |
|
||
|---|---|---|---|
|
||
| Photographs / images | codec dispatch + neural | `wai.image.*` (PNG/JXL/AVIF/JPEG, neural family) | ✅ |
|
||
| Recorded audio | codec dispatch + neural | `wai.audio.{flac,opus}`, `wai.neural.{dac,encodec32,…}` | ✅ |
|
||
| Recorded / cinematographic video | codec dispatch + neural | `wai.video.av1*`, `wai.neural.{dcvc_rt,dcvc_fm}` | ✅ |
|
||
| Live capture: conferencing, telephony, **radio**, broadcast | codec dispatch + receipts | core audio/video caps + JWP per object | 🟡 (transport out of scope; caps + receipts exist) |
|
||
| Sensor / telemetry / IoT | content-addressing + receipts | `wai.text.zstd` + JWP (or a `wai.feed.*` class) | 🟡 |
|
||
| Scientific captures: microscopy, medical imaging, LiDAR/point clouds, genomic | codec dispatch + content-addressing | core caps; point clouds → splat/volumetric path | 🟡 |
|
||
| Biosignals (EEG/ECG) | content-addressing + receipts | `wai.text.zstd` + JWP | 🟡 |
|
||
|
||
### B. Synthesizable media — shipped today as the render (lever 3)
|
||
|
||
| Content | Primary lever | Capability / extension | Status |
|
||
|---|---|---|---|
|
||
| **Animation / CGI film** | instructions-at-sink | `wai.film.linear` (a sealed world + camera track + frame schedule, film-equivalence) | ✅ |
|
||
| Motion graphics / titles / ads | instructions-at-sink | interactive-worlds / joule-ui | 🟡 |
|
||
| Music as score / synthesis | instructions-at-sink | `wai.audio.score` (note events, deterministic synth, mixdown-equivalence) | ✅ |
|
||
| Procedural / game audio, soundscapes | instructions-at-sink | spatial-audio (`wai.audio.scene`) | 🟡 Phase 1 |
|
||
| Text-to-speech | neural prior | ship text + a voice prior (`wai.neural.*` speech) | 🟡 |
|
||
| Data visualizations / charts | instructions-at-sink | ship data + spec (joule-ui) | 🟡 |
|
||
| Diagrams, vector art, **fonts** | instructions-at-sink | formats the web already carries as instructions; envelope only | ✅ (precedent) |
|
||
| **Maps / geospatial** | instructions-at-sink | `map` standard | ✅ (sibling) |
|
||
|
||
### C. Interactive / executable — it IS a program (levers 3, 4)
|
||
|
||
| Content | Primary lever | Capability / extension | Status |
|
||
|---|---|---|---|
|
||
| **Videogames / interactive worlds** | instructions + determinism + receipts | interactive-worlds (`wai.world.*`) | ✅ |
|
||
| 3D scenes / models | content-addressed assets | `wai.mesh.glb` | ✅ |
|
||
| Gaussian splats | content-addressed assets | `wai.splat.spz` | ✅ |
|
||
| **AR / VR / XR** immersive scenes | instructions-at-sink | interactive-worlds + a sink pose/camera mode | 🟡 (substrate exists; XR sink mode unbuilt) |
|
||
| **Volumetric / light-field / holographic / 6DoF video** | instructions-at-sink | `wai.splat.4d` (temporal Gaussian tracks, frame-state-equivalence) | ✅ |
|
||
| **Avatars / digital humans** | instructions-at-sink | `wai.avatar.pose` (skeletal sim + animation ops, skins a `wai.mesh.glb`) | ✅ |
|
||
| UIs / applications | instructions-at-sink + receipts | `joule-ui` standard | ✅ (sibling) |
|
||
| Documents-as-programs (interactive/reactive) | instructions-at-sink | joule-ui / interactive-worlds | 🟡 |
|
||
| Simulations / digital twins | instructions + determinism | interactive-worlds sim registrants; CAD/mathground lineage | 🟡 |
|
||
|
||
### D. Structured / symbolic — already instructions (levers 1, 4)
|
||
|
||
| Content | Primary lever | Capability / extension | Status |
|
||
|---|---|---|---|
|
||
| Text / markup / documents | codec + content-addressing | `wai.text.{zstd,xz}`; reading surface = `Access`/`Lumina` | ✅ |
|
||
| Code / software / packages | content-addressing + receipts | `wai.text.zstd` + JWP | 🟡 |
|
||
| Datasets / tabular / scientific data / knowledge graphs | content-addressing + receipts | `wai.text.*` + JWP; `JouleDB` lineage | 🟡 |
|
||
| Notifications / messages / event-&-state feeds | instructions + receipts | `wai.feed.state` (broadcast profile, replay-equivalence) | ✅ |
|
||
|
||
### The cargo-roadmap phases
|
||
|
||
Five classes were built as phases of `cargo-roadmap.md`; each has shipped.
|
||
|
||
| Content | What it shares with the thesis | Phase and status |
|
||
|---|---|---|
|
||
| **Object-based / spatial audio** | the audio form of the video thesis: ship sound objects + a scene, render to the listener's speaker config; mixdown-equivalence is the audio counterpart of replay-equivalence | **Phase 1**, shipped (`wai.audio.scene`) |
|
||
| **Haptics** | instruction streams to actuators; tiny payloads, pure recipe, deterministic | **Phase 2**, shipped (`wai.haptic.signal`) |
|
||
| **Live state-feeds** | sports/events as *state*, not pixels — the multiplayer-games precedent applied to broadcast | **Phase 3**, shipped (`wai.feed.state`, the broadcast profile of `wai.world.live`) |
|
||
| **Digital humans / avatars** | a high-bandwidth capture that reduces to a rig plus a pose stream | **Phase 4**, shipped (`wai.avatar.pose`) |
|
||
| **Volumetric / light fields** | the splat path extended in time; frame-state-equivalence | **Phase 5**, shipped (`wai.splat.4d`) |
|
||
| **Full-sensory / olfactory** | the far end of the axis | on the axis; not scheduled |
|
||
|
||
## 4. The one sentence to navigate by
|
||
|
||
> **WAI carries whatever it can reconstruct, and verifies whatever it
|
||
> carries.** Reconstruction (instructions-at-sink, neural priors) is the
|
||
> compression story; verification (determinism, content-addressing,
|
||
> receipts, joules) is the trust-and-energy story — and the second
|
||
> applies even to the content that can't be reconstructed.
|
||
|
||
## 5. Reading the map
|
||
|
||
- Lever 3 applies most directly to animation film (a sealed
|
||
camera-on-rails world) and to live state-feeds.
|
||
- Object-based spatial audio carries the same equivalence rule to
|
||
audio: mixdown-equivalence is the audio counterpart of
|
||
replay-equivalence.
|
||
- Lever 4 needs no reconstruction at all: content-addressing, JWP
|
||
receipts and measured joules apply to *every* class, including raw
|
||
captured reality.
|
||
|
||
See `cargo-roadmap.md` for the phased build plan.
|