open-standards/wai/extensions/interactive-worlds.md

WAI Extension: Interactive Worlds

Status: Draft. Sub-spec of WAI, for both envelopes (WAI1 and WAI2). Reference implementation: the world feature of wai-rs (container, ops, fixed-point, replay

• live engines, kinematic0 + ballistic0 sim registrants). Conformance corpus: world-conformance/ at the WAI repo root.

Keywords MUST, MUST NOT, SHOULD, MAY, REQUIRED, OPTIONAL are RFC 2119 / RFC 8174.

---

1. Scope and model

This extension registers a cargo class for interactive worlds: media that consumes input. A world is a deterministic, content- addressed artifact; everything that ever happens in it is a signed operation; and any sequence of operations replays to a bit-exact state on every conformant sink.

The single-player input log and the multiplayer operation log are the same object. An input event is an operation emitted by one actor. One actor's log is a single-player session; N interleaved actor logs are a shared world; a sealed log is a replay. This extension defines the operation once and derives all three forms from it.

In scope:

• Three registered capabilities: wai.world.scene, wai.world.replay, wai.world.live (§3).
• The world-container payload format the three capabilities share (§4).
• The signed operation (CausalOp) wire format and its ordering, causality, and convergence rules (§5).
• The simulation contract: tick semantics, deterministic numeric profiles, seeded randomness, conflict resolution (§6).
• State hashing, checkpoints, and the replay-equivalence conformance criterion (§7).
• Session receipts, as a profile of the JWP receipt chain (§8).
• The fallback floor: how a world degrades to passive media on sinks below the capability floor (§9).

Out of scope:

• A transport. Live sessions need ops moved between actors; this extension defines the op bytes and their semantics, not the wire that carries them (WebRTC, WebSocket, MoQT, sneakernet — all conformant if the bytes arrive intact).
• An engine. Rendering, spatial indexing, rollback strategy, level-of-detail, asset pipelines, and solver implementations are sink-side choices. The extension constrains state, never how a sink derives it efficiently.
• Sessions, identity, matchmaking. Actor keys are Ed25519 public keys; how actors find each other and authorize entry is the deployer's business.
• Visual conformance. Conformance is state-level (§7). Pixel output is the renderer's business; deployments that need pinned visual output compose this extension with the existing hash-pinned-prior mechanisms in the core spec.

Relationship to WAI SPEC.md:

• §2 (container) — unchanged. World payloads ride the WAI1/WAI2 envelopes as opaque capability-owned bytes, per SPEC.md §5: "each capability owns its payload format."
• §3 (manifest) — unchanged. media takes the new informational value "world"; the capability string remains authoritative. The simulation contract lives in the payload (§4), not the manifest.
• §4 (dispatch) — unchanged. A sink without wai.world.* falls back per the declared fallback, or reports the file inert.
• §6.1 (multi-rendition) — the WAI2 envelope is the natural carrier for a world plus its passive-media floor rendition (§9).
• §7 (versioning) — these capabilities are added by registration in a MINOR revision; the envelope is unchanged.
• extensions/jwp-receipts.md — session receipts (§8) are a profile of the JWP group receipt; nothing there is modified.

Precedent inside the core spec: wai.neural.int_synth already registers a capability whose defining property is bit-identical output on every machine. This extension generalizes that property from decode to simulation.

---

2. Terminology

• World — a content-addressed graph of nodes plus a simulation contract. Identified by the BLAKE3-256 of its canonical serialization (§7.1).
• Node — the unit of identity in a world. 128-bit id (§5.4).
• Actor — the holder of an Ed25519 keypair who emits operations.
• Operation (op) — a signed structural mutation targeting a node, bound to an effect tick (§5).
• Tick — the discrete simulation step. All state is defined per-tick; nothing is defined between ticks.
• Checkpoint — a tick at which the state hash is computed and (in receipts and sealed replays) recorded.
• Sealed log — an op log with a terminal marker and final state hash; the replay form.

---

3. Registered capabilities

capability	payload	role
wai.world.scene	world container, op log empty	a world at tick 0 — a shareable, explorable artifact
wai.world.replay	world container, op log sealed	a finished session — re-derives every state from kilobytes
wai.world.live	world container, op log open	a joinable world — ops continue to arrive by out-of-scope transport

All three share the container format (§4) and differ only in the op log's seal state. A sink that advertises any wai.world.* capability MUST implement all of §§4–7; wai.world.live additionally requires the live-ingestion rules in §5.6.

intent for all three is replicate: the sink reconstructs exactly the state the ops define. Generative intents over worlds are not defined in this revision.

---

4. World container (payload format)

The payload bytes for every wai.world.* capability:

+--------+----------------+------------------+----------...----+
| "WWLD" | u16 n_sections | section table    | sections        |
| 4 B    | little-endian  | n × (u8 kind,    | packed,         |
|        |                |   u32 off,len)   | 8-byte aligned  |
+--------+----------------+------------------+-----------------+

Section offsets are relative to the start of the sections block; each section is zero-padded to an 8-byte boundary; len is the unpadded body length.

Section kinds:

kind	section	required
0x01	simulation contract (§6), canonical JSON	REQUIRED
0x02	world graph: canonical state bytes at tick 0 (§7.1 format)	REQUIRED
0x03	asset table (§4.1)	OPTIONAL
0x04	op log (§5), concatenated CausalOp records	REQUIRED (may be empty)
0x05	seal: final tick (u64) + final state hash (32 B) + publisher Ed25519 signature (64 B)	REQUIRED for wai.world.replay, MUST be absent otherwise

Unknown section kinds MUST be ignored (forward compatibility within this extension's MINOR revisions).

The seal signature is computed over:

seal_preimage = "wai:world-seal\x01"
             || BLAKE3(contract section bytes)
             || BLAKE3(graph section bytes)
             || BLAKE3(op-log section bytes)
             || final_tick (u64 LE) || final_hash (32 B)

Hashing the three sections — rather than raw container bytes — makes the commitment independent of section-table order and padding: the seal binds content, not framing. The seal's signature, every op's (§5.1) and the session receipt's (§8) are verified as SPEC §7 Signatures requires.

4.1. Asset table — worlds are made of WAI objects

Each asset entry is:

asset_hash (32 B BLAKE3-256) | u8 role | u32 name_len | name (UTF-8)

asset_hash is the content hash of a complete WAI envelope — computed exactly as in jwp-receipts §2.1, same domain separation discipline, domain "wai:asset\x01". An asset is itself a WAI object (a JPEG-XL texture, a FLAC ambience, a zstd-compressed mesh) and is fetched, verified, and decoded by ordinary WAI capability dispatch. The world container never embeds asset bytes; it names them.

Consequences, all deliberate:

• Assets dedupe globally across worlds (content addressing).
• Asset delivery inherits the JWP receipt chain unchanged.
• New asset media (mesh, material, armature codecs) enter by ordinary capability registration without touching this extension.

Assets MUST NOT influence simulation state. They bind to nodes for presentation; a sink missing an asset MUST still simulate identically (and SHOULD present a declared placeholder). This is what keeps conformance state-level while assets stream in.

For geometry, materials, textures, and skeletal armatures the registered asset capability is wai.mesh.glb: the payload is a canonical glTF 2.0 Binary (.glb, Khronos, royalty-free) — the same zeroth-condition approach as the rest of the core registry: dispatch to a published standard. Decoders are sink- or library-supplied (every engine and DCC tool ships one). Further asset media register in the core capability table as usual.

---

5. Operations

5.1. CausalOp wire format

+-----------------------------------------------------------------+
| actor_id (32 B Ed25519 pubkey) | lamport (u64) | effect_tick(u64)|
+-----------------------------------------------------------------+
| n_deps (u16) | deps: n × (32 B actor_id + u64 lamport)           |
+-----------------------------------------------------------------+
| target_node (16 B) | mutation_kind (u16) | mut_len (u32)        |
+-----------------------------------------------------------------+
| mutation bytes (mut_len, owned by the sim contract §6.4)         |
+-----------------------------------------------------------------+
| signature (64 B Ed25519, over all preceding bytes,              |
|            domain "wai:world-op\x01")                            |
+-----------------------------------------------------------------+

The integers are little-endian, and the signature is Ed25519 over the domain followed by every byte before the signature. An op's identity is op_id = (actor_id, lamport). An actor MUST increment lamport per op emitted and MUST set it greater than every lamport it has observed (standard Lamport discipline).

5.2. Canonical order

The canonical total order over any op set is:

(effect_tick, lamport, actor_id)   — ascending, bytewise on actor_id

This order is a pure function of op content. Two sinks holding the same op set hold the same canonical order, always.

5.3. Causality

deps lists op_ids this op observed. Deps gate admission, not ordering: a sink MUST NOT admit an op whose deps are not all already admitted — it buffers the op until they are. Once admitted, ops apply strictly in canonical order (§5.2); deps express observation, the canonical key is the ordering authority. In the sealed-replay form the log MUST be listed in strictly ascending canonical order (which also forbids duplicate op ids) and every dep MUST name an op present in the log; a sink MUST reject a sealed log that is dep-incomplete or mis-ordered.

Redelivery of a byte-identical, already-admitted op is an idempotent no-op (transports redeliver; redelivery must not be an error). A second op with the same (actor, lamport) but different content is equivocation and MUST be rejected.

5.4. Node identity and minting

Node ids are 128 bits. A node created by op o has node_id = BLAKE3(domain || actor_id || lamport)[0..16], domain "wai:world-node\x01". Minting is therefore deterministic, collision-resistant, and requires no coordination between actors — two actors creating nodes concurrently can never collide. Tick-0 nodes in the world graph carry explicit ids assigned by the authoring encoder under the same scheme with the publisher as actor.

5.5. Convergence requirement (normative core)

Any conformant sink holding the same world container and the same op set MUST compute the identical state hash at every tick, regardless of arrival order.

This holds by construction: canonical order (§5.2) is content-derived, conflict resolution (§6.5) is deterministic, and the numeric profile (§6.2) is bit-exact. Sinks are free to apply ops optimistically and re-derive when late ops arrive (speculation, rollback, forward-replay) — how is out of scope; converging is not.

5.6. Live ingestion

A wai.world.live sink MUST: verify each arriving op's signature, buffer dep-incomplete ops, and insert applicable ops into canonical order — re-deriving state from the most recent checkpoint at or before the op's effect_tick when an op arrives for a past tick. A sink MUST reject ops whose effect_tick precedes the latest sealed checkpoint it has receipted (§8): receipted history is immutable.

---

6. Simulation contract

Section 0x01, canonical JSON (no insignificant whitespace, sorted keys — this section is hashed):

{
  "tick_rate":   60,
  "numeric":     "wai.det.fixed64",
  "rng":         { "alg": "xoshiro256++", "seed": "<hex 32 B>" },
  "sim":         "<sim-capability-string>",
  "resolve":     { "default": "wai.world.resolve.lww" },
  "checkpoint_every": 600
}

Canonical emission: keys sorted (checkpoint_every, numeric, resolve, rng, sim, tick_rate; nested objects likewise), lowercase hex, no insignificant whitespace. Readers MUST NOT depend on key order for semantics (same rule as the core manifest); the canonical bytes are what the container carries and the seal commits to. The rng seed is the xoshiro256++ state as four little-endian u64 words; the all-zero seed (the family's one degenerate point) maps to the first four outputs of splitmix64(0), normatively.

6.1. Ticks

State is a pure function state(t+1) = step(state(t), ops_at(t+1)). tick_rate is the semantic rate binding ticks to wall-clock for presentation; simulation correctness never depends on wall-clock.

6.2. Numeric profiles

Registered in this revision:

profile	definition	status
wai.det.fixed64	Q32.32 signed fixed-point; integer add/sub/mul/div with defined rounding (toward negative infinity) and saturation; integer-only transcendental tables published in the conformance corpus	REQUIRED for any wai.world.* sink
wai.det.f32strict	IEEE 754 binary32, round-to-nearest-even, FMA forbidden, fast-math forbidden, evaluation order as written in the sim capability's normative pseudocode	OPTIONAL

wai.det.fixed64 is the floor for the same reason PNG/FLAC/zstd are the core floor: it is satisfiable on every machine, including ones whose float units disagree. Bit-exactness is the requirement of existence here; profiles that cannot guarantee it cannot be registered.

6.3. Randomness

All randomness flows from the seeded generator named in rng, advanced only by simulation steps in canonical order. Sink-local randomness MUST NOT touch state.

6.4. The sim capability

sim names the capability that owns mutation_kind semantics and the step function — the actual physics. This extension deliberately does not standardize a physics engine; it standardizes the contract any physics must honor: deterministic under §6.2, pure per §6.1, convergent per §5.5. Sim capabilities register like any WAI capability, each with normative pseudocode and conformance vectors.

The reference registrant is wai.world.sim.kinematic0 (point-mass kinematics + one scalar field per node, all wai.det.fixed64): five mutation kinds (CREATE/DELETE/SET_SCALAR/SET_POS as lww, IMPULSE as sum; ops targeting absent nodes are deterministic no-ops; CREATE targets MUST equal the §5.4 minted id), normative Rust in wai-rs/src/world/sim.rs, exercised by the world-conformance/ corpus. Deliberately small — rich enough to exercise every normative mechanism, simple enough that an independent implementation is an afternoon. A second registrant, wai.world.sim.ballistic0 (gravity + ground-plane bounce with per-node radius and restitution, registration constant G = −10), exists to prove the registration story in practice: it entered the registry, the dispatch, and the conformance corpus without one line of this extension changing.

The solver-backed registrant is wai.world.sim.truss0: structural simulation — point-mass particles joined by distance members, solved per tick by position-based dynamics (a fixed-count Gauss–Seidel constraint projection, ITERATIONS = 8, canonical member order, G = −10, ground plane at y = 0, velocity damping 63/64 per tick — exactly 1 − 2⁻⁶ in Q32.32, so structures reach static equilibrium — all wai.det.fixed64 including the floor-rounded square root). Members carry a yield strain measured on predicted positions before the solve; an overloaded member fractures as an emergent solver consequence — including under the load two actors apply by pulling the same member in opposite directions (the sum rule feeding the solver). Stress is computed, not animated, and failure is bit-exact on every machine. Richer sims (rigid-body contact, fields, FEM-class) register the same way.

Sinks dispatch on the contract's sim string. A sink that lacks the named sim capability MUST report the world inert — the same missing-capability semantics as SPEC.md §4 step 6 — so the caller can fall back per the envelope's declared fallback.

6.5. Conflict resolution

Two ops conflict when they share an effect_tick and target the same node with mutations the sim capability declares non-commuting. Resolution functions registered in this revision:

function	rule
wai.world.resolve.lww	discrete/scalar conflicts: last writer in canonical order wins (canonical order is total, so "last" is deterministic)
wai.world.resolve.sum	continuous conflicts: concurrent mutations combine under a commutative monoid declared by the sim capability (e.g. force vectors sum; both inputs act, neither is discarded)

A sim capability MAY map mutation kinds to either function but MUST declare the mapping statically in its registration. Because lww rides the canonical total order and sum is commutative-associative, both are order-independent — convergence (§5.5) survives any arrival order. Downstream physical consequences of combined mutations (stress, fracture, overflow) are ordinary sim behavior, not resolution.

---

7. State hashing and conformance

7.1. State hash

state_hash(t) = BLAKE3(domain || canonical_state_bytes(t))
domain        = "wai:world-state\x01"

canonical_state_bytes is:

n_nodes (u32 LE) | n × ( node_id (16 B) | state_len (u32 LE) | state bytes )

with nodes in strictly ascending node-id order and each node's state fields in the order the sim capability's registration declares. The WWLD graph section (§4) is this same format at tick 0 — the authored graph IS canonical state bytes. The asset table and presentation data are excluded (§4.1).

7.2. Checkpoints

A sink MUST compute state_hash every checkpoint_every ticks and at the final tick of a sealed log. Sealed replays carry the final hash in the seal section; a mismatch on replay is a conformance failure of either the log or the sink — the file MUST be reported failed, never presented as approximately right.

7.3. Conformance criterion

The core spec's criterion is bit-exact decode-equivalence. This extension's criterion is its temporal generalization:

Replay-equivalence. Given the same world container and op set, a conforming sink MUST produce the identical state_hash at every checkpoint, on every machine, under every arrival order permitted by §5.

The conformance corpus lives at world-conformance/ (alongside byte-exact-conformance/): sealed world containers, live op streams in adversarial arrival orders (dep-gaps, past-tick arrivals, seeded shuffles, conflict batteries for both resolution functions), and expected checkpoint hashes. A sink passes by hash equality; there is no tolerance parameter. The reference implementation is the world feature of wai-rs (pure Rust, no ML dep chain); the corpus is generated and checked by wai_world_vectors gen|verify.

7.4. What replay-equivalence buys

Stated once, since it is the point of the extension:

• A recording of play is kilobytes: world hash + op log. Any sink re-derives every frame. The largest interactive-video category on the internet becomes instructions, not pixels.
• Two actors who opened the same hash and applied the same ops are provably in the same world — not probably, provably, by hash.
• A dropped connection recovers by fetching missing ops and replaying forward from the last checkpoint — state snapshots are a sink-side cache, never the artifact.

---

8. Session receipts (JWP profile)

A world session is receipted as a JWP group (jwp-receipts.md, unchanged):

• Objects in the group are op batches: each batch is the concatenated CausalOp bytes for a checkpoint interval, content- hashed per jwp-receipts §2.1 with domain "wai:world-ops\x01".
• The group's canonical metadata carries, in addition to the JWP base fields: the world hash, the checkpoint state_hash list for the interval, and the publisher's measured joules for the interval (source-side authoring/solver joules; sink-side joules append per the JWP sink-metering path).
• One Ed25519 signature per group, Merkle-bound, relay-preserved byte-for-byte — all inherited — verified as SPEC §7 Signatures requires.
• The signing domain is the profile's own, wai:world-receipt\x01 (\x02 when the figure carries a label; energy-measurement.md §2.8), not the base group receipt's moq-jwp:group-sign\x01.

A receipted checkpoint seals history: §5.6 forbids ops landing before it. The receipt chain therefore gives a shared world an append-only, signed, energy-accounted timeline — the same artifact JWP gives a video stream, generalized to state.

Chaining (parent_receipt_hash). Each receipt carries the BLAKE3 of the previous receipt's canonical bytes (the JWP linkage), or null for the first. Successive sealed sessions of one world (shared world_hash, group_id incrementing by one) thus form one verifiable append-only timeline; a verifier walks the chain checking each signature and each parent link. The reference engine implements this as SessionReceipt::parent_receipt_hash + receipt_hash() + verify_chain.

Energy, honestly. A receipt carries two distinct quantities:

• work_steps — the exact, deterministic node-step count (Σ over ticks of the node count integrated that tick). It is part of the signed payload and reproducible on any sink, because it is a function of the world and its ops alone.
• joules_micro — a carried, measured figure, never derived inside the receipt. Native sinks supply energy via the sink-metering path, which needs no elevated privileges: on arm64 macOS a processor power a non-privileged reader reports, and the kernel's per-process energy (proc_pid_rusage); on x86 RAPL. The reference wai_world_meter brackets a replay burst against an idle baseline and reports the marginal CPU energy of the reported power. A browser, lacking those counters, supplies a wall-clock estimate against a declared power coefficient and labels it an estimate; an unmetered sink signs the unmetered marker 0, which a reader presents as unmetered and never as a figure of zero joules (energy-measurement §2). Joules are derived from measurement or a stated model, never asserted in place of one. The figure's acquisition class rides beside it as an optional signed label (energy-measurement §2.8): absent, the receipt keeps its legacy bytes and the figure is unlabelled; wai_world_meter labels its figure ModelBased, because its samples are a power another program reports, not a counter it reads and shows live (energy-measurement §3.1); by §2's definitions, a browser's wall-clock figure against a declared coefficient is ModelBased too.

---

9. Fallback floor

model_requirement.fallback for wai.world.* SHOULD name a passive capability, and encoders SHOULD ship the WAI2 multi-rendition form:

primary	floor rendition
wai.world.replay	wai.video.av1 — a rendered recording of the same replay
wai.world.scene	wai.image.jxl or wai.image.png — a rendered view
wai.world.live	wai.video.av1 — a spectator stream rendered by a capable participant (production of that stream is out of scope)

Below-floor sinks get ordinary WAI media through ordinary dispatch — nothing about the core fallback machinery changes. The interactive form and its floor travel in one envelope: the capable sink plays the world; the legacy sink plays the video; both opened the same file.

---

10. Versioning

This extension follows SPEC.md §7. The WWLD section-table format and the CausalOp layout are frozen once registered; new section kinds, numeric profiles, sim capabilities, and resolution functions are added by registration in MINOR revisions; redefining any registered string requires a MAJOR.

---

Appendix A — Design rationale (non-normative)

Why one op format for input and multiplayer. Deferring multiplayer would have shipped a v1 input log that a later revision must break. The unification is also the cheaper design: single-player is one actor's log, a replay is a sealed log, a shared world is N logs merged by canonical order. Three products, one definition.

Why fixed-point is the floor. Replay-equivalence with no tolerance parameter is only honest if the arithmetic is bit-exact on every machine that exists. Strict-IEEE float is achievable but toolchain-fragile; integer arithmetic is not. The floor must be unconditional — the same reasoning that put PNG, not AVIF, in the core floor.

Why the standard has no physics. The moment the extension specifies a solver it stops being a container standard and starts being an engine, which an engine would have to replace rather than implement. The contract — deterministic, pure, convergent, receipted — is the only part that must be common.

Why state-level, not pixel-level, conformance. Pixels are a function of state plus presentation assets plus renderer. State is the shared truth two actors must agree on; presentation is the part sinks legitimately differ on (resolution, quality tier, accessibility rendering). Deployments needing pinned pixels can compose the world's state determinism with the core spec's pinned-prior machinery — the two guarantees stack.