open-standards/wai/extensions/interactive-worlds.md

# WAI Extension: Interactive Worlds

> Status: Draft. Sub-spec of WAI v1.0/v1.1. Reference implementation:
> the `world` feature of `wai-rs` (container, ops, fixed-point, replay
> + live engines, `kinematic0` + `ballistic0` sim registrants).
> Conformance corpus: `world-conformance/` at the WAI repo root.

Keywords **MUST**, **MUST NOT**, **SHOULD**, **MAY**, **REQUIRED**,
**OPTIONAL** are RFC 2119 / RFC 8174.

---

## 1. Scope and model

This extension registers a cargo class for **interactive worlds**:
media that consumes input. A world is a deterministic, content-
addressed artifact; everything that ever happens in it is a signed
**operation**; and any sequence of operations replays to a bit-exact
state on every conformant sink.

The single-player input log and the multiplayer operation log are the
**same object**. An input event is an operation emitted by one actor.
One actor's log is a single-player session; N interleaved actor logs
are a shared world; a sealed log is a replay. This extension defines
the operation once and derives all three forms from it.

In scope:

- Three registered capabilities: `wai.world.scene`,
  `wai.world.replay`, `wai.world.live` (§3).
- The world-container payload format the three capabilities share (§4).
- The signed operation (`CausalOp`) wire format and its ordering,
  causality, and convergence rules (§5).
- The simulation contract: tick semantics, deterministic numeric
  profiles, seeded randomness, conflict resolution (§6).
- State hashing, checkpoints, and the **replay-equivalence**
  conformance criterion (§7).
- Session receipts, as a profile of the JWP receipt chain (§8).
- The fallback floor: how a world degrades to passive media on sinks
  below the capability floor (§9).

Out of scope:

- **A transport.** Live sessions need ops moved between actors; this
  extension defines the op bytes and their semantics, not the wire
  that carries them (WebRTC, WebSocket, MoQT, sneakernet — all
  conformant if the bytes arrive intact).
- **An engine.** Rendering, spatial indexing, rollback strategy,
  level-of-detail, asset pipelines, and solver implementations are
  sink-side choices. The extension constrains *state*, never *how a
  sink derives it efficiently*.
- **Sessions, identity, matchmaking.** Actor keys are Ed25519 public
  keys; how actors find each other and authorize entry is the
  deployer's business.
- **Visual conformance.** Conformance is state-level (§7). Pixel
  output is the renderer's business; deployments that need pinned
  visual output compose this extension with the existing
  hash-pinned-prior mechanisms in the core spec.

Relationship to WAI SPEC.md:

- §2 (container) — unchanged. World payloads ride the `WAI1`/`WAI2`
  envelopes as opaque capability-owned bytes, per SPEC.md §5: "each
  capability owns its payload format."
- §3 (manifest) — unchanged. `media` takes the new informational value
  `"world"`; the capability string remains authoritative. The
  simulation contract lives in the payload (§4), not the manifest.
- §4 (dispatch) — unchanged. A sink without `wai.world.*` falls back
  per the declared `fallback`, or reports the file inert.
- §6.1 (multi-rendition) — the `WAI2` envelope is the natural carrier
  for a world plus its passive-media floor rendition (§9).
- §7 (versioning) — these capabilities are added by registration in a
  MINOR revision; the envelope is unchanged.
- `extensions/jwp-receipts.md` — session receipts (§8) are a profile
  of the JWP group receipt; nothing there is modified.

Precedent inside the core spec: `wai.neural.int_synth` already
registers a capability whose defining property is **bit-identical
output on every machine**. This extension generalizes that property
from decode to simulation.

---

## 2. Terminology

- **World** — a content-addressed graph of nodes plus a simulation
  contract. Identified by the BLAKE3-256 of its canonical
  serialization (§7.1).
- **Node** — the unit of identity in a world. 128-bit id (§5.4).
- **Actor** — the holder of an Ed25519 keypair who emits operations.
- **Operation (op)** — a signed structural mutation targeting a node,
  bound to an effect tick (§5).
- **Tick** — the discrete simulation step. All state is defined
  per-tick; nothing is defined between ticks.
- **Checkpoint** — a tick at which the state hash is computed and
  (in receipts and sealed replays) recorded.
- **Sealed log** — an op log with a terminal marker and final state
  hash; the replay form.

---

## 3. Registered capabilities

| capability | payload | role |
|---|---|---|
| `wai.world.scene` | world container, op log empty | a world at tick 0 — a shareable, explorable artifact |
| `wai.world.replay` | world container, op log sealed | a finished session — re-derives every state from kilobytes |
| `wai.world.live` | world container, op log open | a joinable world — ops continue to arrive by out-of-scope transport |

All three share the container format (§4) and differ only in the op
log's seal state. A sink that advertises any `wai.world.*` capability
MUST implement all of §§4–7; `wai.world.live` additionally requires
the live-ingestion rules in §5.6.

`intent` for all three is `replicate`: the sink reconstructs exactly
the state the ops define. Generative intents over worlds are not
defined in this revision.

---

## 4. World container (payload format)

The payload bytes for every `wai.world.*` capability:

```
+--------+----------------+------------------+----------...----+
| "WWLD" | u16 n_sections | section table    | sections        |
| 4 B    | little-endian  | n × (u8 kind,    | packed,         |
|        |                |   u32 off,len)   | 8-byte aligned  |
+--------+----------------+------------------+-----------------+
```

Section offsets are relative to the start of the sections block;
each section is zero-padded to an 8-byte boundary; `len` is the
unpadded body length.

Section kinds:

| kind | section | required |
|---|---|---|
| `0x01` | simulation contract (§6), canonical JSON | REQUIRED |
| `0x02` | world graph: canonical state bytes at tick 0 (§7.1 format) | REQUIRED |
| `0x03` | asset table (§4.1) | OPTIONAL |
| `0x04` | op log (§5), concatenated `CausalOp` records | REQUIRED (may be empty) |
| `0x05` | seal: final tick (u64) + final state hash (32 B) + publisher Ed25519 signature (64 B) | REQUIRED for `wai.world.replay`, MUST be absent otherwise |

Unknown section kinds MUST be ignored (forward compatibility within
this extension's MINOR revisions).

The seal signature is computed over:

```
seal_preimage = "wai:world-seal\x01"
             || BLAKE3(contract section bytes)
             || BLAKE3(graph section bytes)
             || BLAKE3(op-log section bytes)
             || final_tick (u64 LE) || final_hash (32 B)
```

Hashing the three sections — rather than raw container bytes — makes
the commitment independent of section-table order and padding: the
seal binds content, not framing. The seal's signature, every op's
(§5.1) and the session receipt's (§8) are verified as SPEC §7
*Signatures* requires.

### 4.1. Asset table — worlds are made of WAI objects

Each asset entry is:

```
asset_hash (32 B BLAKE3-256) | u8 role | u32 name_len | name (UTF-8)
```

`asset_hash` is the content hash of a **complete WAI envelope** —
computed exactly as in jwp-receipts §2.1, same domain separation
discipline, domain `"wai:asset\x01"`. An asset is itself a WAI object
(a JPEG-XL texture, a FLAC ambience, a zstd-compressed mesh) and is
fetched, verified, and decoded by ordinary WAI capability dispatch.
The world container never embeds asset bytes; it names them.

Consequences, all deliberate:

- Assets dedupe globally across worlds (content addressing).
- Asset delivery inherits the JWP receipt chain unchanged.
- New asset media (mesh, material, armature codecs) enter by ordinary
  capability registration without touching this extension.

Assets MUST NOT influence simulation state. They bind to nodes for
*presentation*; a sink missing an asset MUST still simulate
identically (and SHOULD present a declared placeholder). This is what
keeps conformance state-level while assets stream in.

For geometry, materials, textures, and skeletal armatures the
registered asset capability is **`wai.mesh.glb`**: the payload is a
canonical glTF 2.0 Binary (`.glb`, Khronos, royalty-free) — the same
zeroth-condition approach as the rest of the core registry: dispatch to
a published standard. Decoders are sink- or
library-supplied (every engine and DCC tool ships one). Further asset
media register in the core capability table as usual.

---

## 5. Operations

### 5.1. CausalOp wire format

```
+-----------------------------------------------------------------+
| actor_id (32 B Ed25519 pubkey) | lamport (u64) | effect_tick(u64)|
+-----------------------------------------------------------------+
| n_deps (u16) | deps: n × (32 B actor_id + u64 lamport)           |
+-----------------------------------------------------------------+
| target_node (16 B) | mutation_kind (u16) | mut_len (u32)        |
+-----------------------------------------------------------------+
| mutation bytes (mut_len, owned by the sim contract §6.4)         |
+-----------------------------------------------------------------+
| signature (64 B Ed25519, over all preceding bytes,              |
|            domain "wai:world-op\x01")                            |
+-----------------------------------------------------------------+
```

An op's identity is `op_id = (actor_id, lamport)`. An actor MUST
increment `lamport` per op emitted and MUST set it greater than every
`lamport` it has observed (standard Lamport discipline).

### 5.2. Canonical order

The canonical total order over any op set is:

```
(effect_tick, lamport, actor_id)   — ascending, bytewise on actor_id
```

This order is a pure function of op content. Two sinks holding the
same op set hold the same canonical order, always.

### 5.3. Causality

`deps` lists op_ids this op observed. Deps gate **admission**, not
ordering: a sink MUST NOT admit an op whose deps are not all already
admitted — it buffers the op until they are. Once admitted, ops apply
strictly in canonical order (§5.2); deps express observation, the
canonical key is the ordering authority. In the sealed-replay form
the log MUST be listed in strictly ascending canonical order (which
also forbids duplicate op ids) and every dep MUST name an op present
in the log; a sink MUST reject a sealed log that is dep-incomplete or
mis-ordered.

Redelivery of a byte-identical, already-admitted op is an idempotent
no-op (transports redeliver; redelivery must not be an error). A
second op with the same `(actor, lamport)` but different content is
**equivocation** and MUST be rejected.

### 5.4. Node identity and minting

Node ids are 128 bits. A node created by op `o` has
`node_id = BLAKE3(domain || actor_id || lamport)[0..16]`, domain
`"wai:world-node\x01"`. Minting is therefore deterministic,
collision-resistant, and requires **no coordination between actors**
— two actors creating nodes concurrently can never collide.
Tick-0 nodes in the world graph carry explicit ids assigned by the
authoring encoder under the same scheme with the publisher as actor.

### 5.5. Convergence requirement (normative core)

> Any conformant sink holding the same world container and the same
> op set MUST compute the identical state hash at every tick,
> regardless of arrival order.

This holds by construction: canonical order (§5.2) is content-derived,
conflict resolution (§6.5) is deterministic, and the numeric profile
(§6.2) is bit-exact. Sinks are free to apply ops optimistically and
re-derive when late ops arrive (speculation, rollback, forward-replay)
— *how* is out of scope; converging is not.

### 5.6. Live ingestion

A `wai.world.live` sink MUST: verify each arriving op's signature,
buffer dep-incomplete ops, and insert applicable ops into canonical
order — re-deriving state from the most recent checkpoint at or
before the op's `effect_tick` when an op arrives for a past tick.
A sink MUST reject ops whose `effect_tick` precedes the latest
**sealed** checkpoint it has receipted (§8): receipted history is
immutable.

---

## 6. Simulation contract

Section `0x01`, canonical JSON (no insignificant whitespace, sorted
keys — this section is hashed):

```json
{
  "tick_rate":   60,
  "numeric":     "wai.det.fixed64",
  "rng":         { "alg": "xoshiro256++", "seed": "<hex 32 B>" },
  "sim":         "<sim-capability-string>",
  "resolve":     { "default": "wai.world.resolve.lww" },
  "checkpoint_every": 600
}
```

Canonical emission: keys sorted (`checkpoint_every, numeric, resolve,
rng, sim, tick_rate`; nested objects likewise), lowercase hex, no
insignificant whitespace. Readers MUST NOT depend on key order for
semantics (same rule as the core manifest); the canonical bytes are
what the container carries and the seal commits to. The rng seed is
the xoshiro256++ state as four little-endian u64 words; the all-zero
seed (the family's one degenerate point) maps to the first four
outputs of `splitmix64(0)`, normatively.

### 6.1. Ticks

State is a pure function `state(t+1) = step(state(t), ops_at(t+1))`.
`tick_rate` is the *semantic* rate binding ticks to wall-clock for
presentation; simulation correctness never depends on wall-clock.

### 6.2. Numeric profiles

Registered in this revision:

| profile | definition | status |
|---|---|---|
| `wai.det.fixed64` | Q32.32 signed fixed-point; integer add/sub/mul/div with defined rounding (toward negative infinity) and saturation; integer-only transcendental tables published in the conformance corpus | REQUIRED for any `wai.world.*` sink |
| `wai.det.f32strict` | IEEE 754 binary32, round-to-nearest-even, FMA forbidden, fast-math forbidden, evaluation order as written in the sim capability's normative pseudocode | OPTIONAL |

`wai.det.fixed64` is the floor for the same reason PNG/FLAC/zstd are
the core floor: it is satisfiable on **every** machine, including ones
whose float units disagree. Bit-exactness is the requirement of
existence here; profiles that cannot guarantee it cannot be registered.

### 6.3. Randomness

All randomness flows from the seeded generator named in `rng`,
advanced only by simulation steps in canonical order. Sink-local
randomness MUST NOT touch state.

### 6.4. The sim capability

`sim` names the capability that owns `mutation_kind` semantics and the
`step` function — the actual physics. This extension deliberately does
not standardize a physics engine; it standardizes the **contract any
physics must honor**: deterministic under §6.2, pure per §6.1,
convergent per §5.5. Sim capabilities register like any WAI
capability, each with normative pseudocode and conformance vectors.

The reference registrant is `wai.world.sim.kinematic0` (point-mass
kinematics + one scalar field per node, all `wai.det.fixed64`): five
mutation kinds (CREATE/DELETE/SET_SCALAR/SET_POS as `lww`, IMPULSE as
`sum`; ops targeting absent nodes are deterministic no-ops; CREATE
targets MUST equal the §5.4 minted id), normative Rust in
`wai-rs/src/world/sim.rs`, exercised by the `world-conformance/`
corpus. Deliberately small — rich enough to exercise every normative
mechanism, simple enough that an independent implementation is an
afternoon. A second registrant, `wai.world.sim.ballistic0` (gravity +
ground-plane bounce with per-node radius and restitution, registration
constant `G = −10`), exists to prove the registration story in
practice: it entered the registry, the dispatch, and the conformance
corpus without one line of this extension changing.

The solver-backed registrant is `wai.world.sim.truss0`: structural
simulation — point-mass particles joined by distance members, solved
per tick by **position-based dynamics** (a fixed-count Gauss–Seidel
constraint projection, `ITERATIONS = 8`, canonical member order,
`G = −10`, ground plane at `y = 0`, velocity damping `63/64` per tick
— exactly `1 − 2⁻⁶` in Q32.32, so structures reach static equilibrium
— all `wai.det.fixed64` including the floor-rounded square root). Members carry a yield strain measured
on predicted positions before the solve; an overloaded member
**fractures** as an emergent solver consequence — including under the
load two actors apply by pulling the same member in opposite
directions (the `sum` rule feeding the solver). Stress is computed,
not animated, and failure is bit-exact on every machine. Richer sims
(rigid-body contact, fields, FEM-class) register the same way.

Sinks dispatch on the contract's `sim` string. A sink that lacks the
named sim capability MUST report the world inert — the same
missing-capability semantics as SPEC.md §4 step 6 — so the caller can
fall back per the envelope's declared `fallback`.

### 6.5. Conflict resolution

Two ops **conflict** when they share an `effect_tick` and target the
same node with mutations the sim capability declares non-commuting.
Resolution functions registered in this revision:

| function | rule |
|---|---|
| `wai.world.resolve.lww` | discrete/scalar conflicts: last writer in canonical order wins (canonical order is total, so "last" is deterministic) |
| `wai.world.resolve.sum` | continuous conflicts: concurrent mutations combine under a commutative monoid declared by the sim capability (e.g. force vectors sum; both inputs act, neither is discarded) |

A sim capability MAY map mutation kinds to either function but MUST
declare the mapping statically in its registration. Because `lww`
rides the canonical total order and `sum` is commutative-associative,
both are order-independent — convergence (§5.5) survives any arrival
order. Downstream physical consequences of combined mutations
(stress, fracture, overflow) are ordinary sim behavior, not
resolution.

---

## 7. State hashing and conformance

### 7.1. State hash

```
state_hash(t) = BLAKE3(domain || canonical_state_bytes(t))
domain        = "wai:world-state\x01"
```

`canonical_state_bytes` is:

```
n_nodes (u32 LE) | n × ( node_id (16 B) | state_len (u32 LE) | state bytes )
```

with nodes in strictly ascending node-id order and each node's state
fields in the order the sim capability's registration declares. The
WWLD graph section (§4) is this same format at tick 0 — the authored
graph IS canonical state bytes. The asset table and presentation data
are excluded (§4.1).

### 7.2. Checkpoints

A sink MUST compute `state_hash` every `checkpoint_every` ticks and at
the final tick of a sealed log. Sealed replays carry the final hash in
the seal section; a mismatch on replay is a conformance failure of
either the log or the sink — the file MUST be reported failed, never
presented as approximately right.

### 7.3. Conformance criterion

The core spec's criterion is bit-exact decode-equivalence. This
extension's criterion is its temporal generalization:

> **Replay-equivalence.** Given the same world container and op set, a
> conforming sink MUST produce the identical `state_hash` at every
> checkpoint, on every machine, under every arrival order permitted by
> §5.

The conformance corpus lives at `world-conformance/` (alongside
`byte-exact-conformance/`): sealed world containers, live op streams
in adversarial arrival orders (dep-gaps, past-tick arrivals, seeded
shuffles, conflict batteries for both resolution functions), and
expected checkpoint hashes. A sink passes by hash equality; there is
no tolerance parameter. The reference implementation is the `world`
feature of `wai-rs` (pure Rust, no ML dep chain); the corpus is
generated and checked by `wai_world_vectors gen|verify`.

### 7.4. What replay-equivalence buys

Stated once, since it is the point of the extension:

- A **recording of play is kilobytes**: world hash + op log. Any sink
  re-derives every frame. The largest interactive-video category on
  the internet becomes instructions, not pixels.
- Two actors who opened the same hash and applied the same ops are
  **provably in the same world** — not probably, provably, by hash.
- A dropped connection recovers by **fetching missing ops and
  replaying forward** from the last checkpoint — state snapshots are a
  sink-side cache, never the artifact.

---

## 8. Session receipts (JWP profile)

A world session is receipted as a JWP group (jwp-receipts.md,
unchanged):

- **Objects** in the group are op batches: each batch is the
  concatenated `CausalOp` bytes for a checkpoint interval, content-
  hashed per jwp-receipts §2.1 with domain `"wai:world-ops\x01"`.
- The group's canonical metadata carries, in addition to the JWP base
  fields: the world hash, the checkpoint `state_hash` list for the
  interval, and the publisher's measured joules for the interval
  (source-side authoring/solver joules; sink-side joules append per
  the JWP sink-metering path).
- One Ed25519 signature per group, Merkle-bound, relay-preserved
  byte-for-byte — all inherited — verified as SPEC §7 *Signatures* requires.
- The signing domain is the profile's own, `wai:world-receipt\x01`
  (`\x02` when the figure carries a label; energy-measurement.md
  §2.8), not the base group receipt's `moq-jwp:group-sign\x01`.

A receipted checkpoint **seals history**: §5.6 forbids ops landing
before it. The receipt chain therefore gives a shared world an
append-only, signed, energy-accounted timeline — the same artifact
JWP gives a video stream, generalized to state.

**Chaining (`parent_receipt_hash`).** Each receipt carries the BLAKE3
of the previous receipt's canonical bytes (the JWP linkage), or `null`
for the first. Successive sealed sessions of one world (shared
`world_hash`, `group_id` incrementing by one) thus form one verifiable
append-only timeline; a verifier walks the chain checking each
signature and each parent link. The reference engine implements this
as `SessionReceipt::parent_receipt_hash` + `receipt_hash()` +
`verify_chain`.

**Energy, honestly.** A receipt carries two distinct quantities:
- `work_steps` — the **exact, deterministic** node-step count
  (Σ over ticks of the node count integrated that tick). It is part of
  the signed payload and reproducible on any sink, because it is a
  function of the world and its ops alone.
- `joules_micro` — a **carried, measured** figure, never derived inside
  the receipt. Native sinks supply real energy via the sink-metering
  path, which needs **no elevated privileges**: on arm64 macOS the SoC
  energy counters are readable through IOReport (the source `macmon` /
  `asitop` use — no sudo), on x86 via RAPL, and per-process retired
  cycles/instructions via `proc_pid_rusage` everywhere. The reference
  `wai_world_meter` brackets a replay burst against an idle baseline and
  reports the marginal measured CPU energy. A browser, lacking those
  counters, supplies a wall-clock estimate against a **declared** power
  coefficient and labels it an estimate; an unmetered sink signs the
  unmetered marker `0`, which a reader presents as unmetered and never as a
  figure of zero joules ([energy-measurement](energy-measurement.md) §2).
  Joules are derived *from* measurement or a stated model, never
  asserted in place of one. The figure's acquisition class rides beside
  it as an optional signed label
  ([energy-measurement](energy-measurement.md) §2.8): absent, the receipt
  keeps its legacy bytes and the figure is unlabelled; `wai_world_meter`
  labels what it measures `OnChipCounter`, with its declared uncertainty;
  by §2's definitions, a browser's wall-clock figure against a declared
  coefficient is `ModelBased`.

---

## 9. Fallback floor

`model_requirement.fallback` for `wai.world.*` SHOULD name a passive
capability, and encoders SHOULD ship the `WAI2` multi-rendition form:

| primary | floor rendition |
|---|---|
| `wai.world.replay` | `wai.video.av1` — a rendered recording of the same replay |
| `wai.world.scene` | `wai.image.jxl` or `wai.image.png` — a rendered view |
| `wai.world.live` | `wai.video.av1` — a spectator stream rendered by a capable participant (production of that stream is out of scope) |

Below-floor sinks get ordinary WAI media through ordinary dispatch —
nothing about the core fallback machinery changes. The interactive
form and its floor travel in **one envelope**: the capable sink plays
the world; the legacy sink plays the video; both opened the same file.

---

## 10. Versioning

This extension follows SPEC.md §7. The `WWLD` section-table format and
the `CausalOp` layout are frozen once registered; new section kinds,
numeric profiles, sim capabilities, and resolution functions are added
by registration in MINOR revisions; redefining any registered string
requires a MAJOR.

---

## Appendix A — Design rationale (non-normative)

**Why one op format for input and multiplayer.** Deferring multiplayer
would have shipped a v1 input log that a later revision must break.
The unification is also the cheaper design: single-player is one
actor's log, a replay is a sealed log, a shared world is N logs merged
by canonical order. Three products, one definition.

**Why fixed-point is the floor.** Replay-equivalence with no tolerance
parameter is only honest if the arithmetic is bit-exact on every
machine that exists. Strict-IEEE float is achievable but
toolchain-fragile; integer arithmetic is not. The floor must be
unconditional — the same reasoning that put PNG, not AVIF, in the core
floor.

**Why the standard has no physics.** The moment the extension
specifies a solver it stops being a container standard and starts
being an engine, which an engine would have to replace rather than
implement. The contract — deterministic, pure, convergent, receipted — is
the only part that must be common.

**Why state-level, not pixel-level, conformance.** Pixels are a
function of state plus presentation assets plus renderer. State is the
shared truth two actors must agree on; presentation is the part sinks
legitimately differ on (resolution, quality tier, accessibility
rendering). Deployments needing pinned pixels can compose the world's
state determinism with the core spec's pinned-prior machinery — the
two guarantees stack.