open-standards/wai/wai-web/compose/README.md

# The composing sink

`index.html` opens a WAI package of several components, dispatches each to a decoder this
device can run, checks everything it is about to present against the package's signed
receipt, and plays the components on one timeline ([browser-sink §14](../../extensions/browser-sink.md)).

| file | what it is |
|---|---|
| `compose.js` | the runtime: `openPackage()` and the `Composition` it returns |
| `index.html`, `page.js` | the reference page: the picture, the dispatch record and the checks |
| `policy.toml`, `sources/`, `floor/`, `key.hex` | the package's policy and inputs |
| `package/` | the package it plays, written by `wai-pack` ([packaging §5.8](../../extensions/packaging.md)) |
| `staged/` | the staged packages: fields and pictures with layers the page computes ([browser-sink §14.5, §14.7](../../extensions/browser-sink.md)) |
| `mutations/forged.receipt.json` | the package's receipt signed by another key |
| `run.mjs` | the gate: the page in browser engines, every outcome held to what it expects |

## The package

One programme of seven seconds as five components under one version-2 package receipt, each
cut at the lead's piece starts, each piece one object:

| component | capability | source | presented as |
|---|---|---|---|
| `v` | `wai.video.int_motion` bases and a `wai.video.av1` floor for every piece | the fixture `compose-01`, a test pattern of 320 × 180 at 25 frames a second | the picture, and the clock |
| `g` | `wai.graphics.vector` | `vector-conformance/vectors/motion/stroke-titles.wvg1`, a title card and a credit roll at 1280 × 720 | an overlay rasterised in WebAssembly |
| `t` | `wai.text.timed` | `sources/t.vtt`, three captions | text, in the page's text layer |
| `s` | `wai.audio.score` of two channels at 48 kHz, each note voiced, and a `wai.audio.flac` floor of each piece's mixdown | `sources/s.wscr`, `score-conformance/vectors/scale48.wscr`'s notes up and down | samples synthesised in the tab in two channels (or, without the score, the floor decoded from FLAC in WebAssembly), on Web Audio |
| `f` | `wai.feed.state` | `sources/f.wwld`, a scoreboard of signed ops | the state at the clock's tick |

Each entry with a decoded output states its digest, which the page holds what it decodes to:
each video base's clip digest, each overlay piece's clip digest, each caption segment's
SHA-256, each score piece's sample digest; each feed piece is sealed by the package's key.

`wai-pack`'s corpus check (`wai-pack conformance check ../pack-conformance`, which
`tests/conformance.rs` runs) writes the sources under `sources/`, packs the package from
them and requires every file the same, verifies it, and writes the forged receipt again
(`wai_pack::multi_fixture::compose_check`; `conformance regen` writes them). By hand:

```
# from wai/wai-pack
cargo run --release --bin wai-pack -- fixture gen compose-01 -o ../wai-web/compose/src.y4m
cargo run --release --bin wai-pack -- pack --policy ../wai-web/compose/policy.toml \
  --out <dir> --key ../wai-web/compose/key.hex
cargo run --release --bin wai-pack -- verify <dir>
```

## The staged packages

`staged/` holds four packages of one image component each, sealed with `key.hex` from objects of
the corpora, byte for byte, by `src/compose_stage.rs`'s tests, which compute every prefix digest
natively and hold it to the corpus's own statement:

| package | pieces |
|---|---|
| `fields` | the orb of `volumetric-conformance/fields/` (a base carrying two of its four levels, then its levels 2 and 3 as layers 1 and 2), `mlp` and `edge` |
| `fields-digest` | `fields`, the orb's layer 2 stating stage 1's digest |
| `cond` | `cond_side_planes`, `cond_dual_gbuffer` and `cond_render` of `staged-conformance/cond/`: a PNG base, its `wai.image.int_cond_refine` layer and its side objects as asset entries |
| `cond-not-exact` | `cond_side_not_exact`'s layer over a plane the page decodes only through its fallback |

The page renders each field at its pinned view and each picture's base in WebAssembly, presents
them, then takes each layer as it arrives: a field's next level, or a picture's conditioned
layer with the side planes it names (a splat set's frame rendered in the page at its pinned
pose), each stage held to its signed digest before it is presented; a refused layer leaves the
stage below. The tests check, and `WAI_COMPOSE_STAGED_WRITE=1` writes them again:

```
# from wai/wai-web
cargo test --lib --features compose compose_stage::
```

The AV1 floor `floor/v.obu` is supplied (packaging §7.3), made the way
`pack-conformance/supplied_floor.py` makes a golden's: `wai-pack plan` cuts the fixture
(two pieces, of 100 and 75 frames), each piece is coded alone by `aomenc` with that script's
settings, and the streams are concatenated. The policy pins the floor directory's listing.

## Run the page

```
# from wai/wai-web: the compose bundle
cargo build --release --target wasm32-unknown-unknown --lib --features compose
wasm-bindgen --target web --out-dir compose/pkg --out-name wai_compose \
  target/wasm32-unknown-unknown/release/wai_web.wasm
# serve wai/, which holds the outline set the overlay names
cd .. && python3 -m http.server 8741
# → http://localhost:8741/wai-web/compose/index.html
```

Query parameters: `without=<capability>,…` forces a device profile without those
capabilities; `receipt=<url>` reads another receipt; `pkg=<dir>/` another package;
`captions=<language>` has the text layer present the text component in that language, and
`captions=off` none (the default is the first the receipt lists).

## Titles as text

A graphics component may carry a text floor: each piece's transcript as `wai.text.zstd`
([graphics-vector §6.3](../../extensions/graphics-vector.md)). On a device with no decoder of
`wai.graphics.vector` (`without=wai.graphics.vector`) the page presents it
([browser-sink §14.9](../../extensions/browser-sink.md)): it holds each floor to its piece's
base, reads the one frame the format writes and the transcript in its one encoding, and at
each tick shows the active blocks as text, each with its language and direction. A caption or
a subtitle is never shown from the floor, because the text layer carries it; a forced
narrative is shown from the floor unless a cue is showing the same words.
`pack-conformance/multi/multi-captions-01` is such a package:

```
http://localhost:8741/wai-web/compose/index.html?pkg=../../pack-conformance/multi/multi-captions-01/package/&without=wai.video.av1,wai.graphics.vector
```

## The gate

```
cd compose && npm ci
node run.mjs --engines chromium,webkit --pkg pkg
```

In each engine: the package opens with every check holding and every component presented,
and plays to its end; each score piece's samples, synthesised in the tab in the two channels
and at the rate its target states, have the digest its index entry states; every overlay
frame hashed in the page equals `vector-conformance/`'s, and each piece's clip digest its
entry's; each caption segment and each feed piece holds; without
the AV1 decoder each integer video base decodes to its signed clip digest; without the integer
rung and the still every video piece plays its AV1 floor and the page says so; without the
score every score piece plays its FLAC floor, decoded in WebAssembly to the samples the
score's signed base states for its mixdown, and the page says so; in
`pack-conformance/multi/multi-opus-01`, without FLAC, the sound track's Opus floor plays through
the engine's `AudioDecoder` where it takes Opus, each piece trimmed to exactly its samples (an
engine whose decoder refuses that golden's 64 × 36 picture refuses it at the lead); the forged
receipt is refused `receipt-signature` and an object with one byte flipped in delivery
`object-hash`, with nothing presented. Over the staged packages: each field renders at its
pinned view to the frame digest `volumetric-conformance/fields/` states, read back from the
canvas; the orb's levels, each held back in delivery and released one at a time, present each
stage, its stage hash the corpus's, before the next level is asked for; a stage that is not its
signed digest is refused `prefix-digest` and a level with a byte flipped `object-hash`, the stage
below staying presented; each picture of `cond` presents the stage hash its case states; a side
object withheld, or the splat decoder left out of the profile, refuses its layer `side-absent`, a
plane held only through its fallback `side-not-exact`, and a side object with a byte flipped is
refused `object-hash` and never held, each with the base presented. Over
`pack-conformance/multi/multi-captions-01`: on this device each graphics piece presents its
overlay and nothing comes from a floor; without `wai.graphics.vector` each presents its floor,
whose transcript is the one the script reads from the floor object, and at every frame the text
from the floor is the blocks active at that tick less the captions, the text layer shows the
cues of the component it presents (English, French with `captions=fr`, none with
`captions=off`), and a caption's words are on the page once; a floor with a byte flipped, and
two floors exchanged in delivery, are refused `object-hash`; and the bundle's reader refuses a
floor held to another piece's base `floor_base`. `--only staged` runs the staged cases alone,
`--only package` the reference package's and `--only floor` the text floor's. It exits 0 only when every check passes in every
engine, and 1 on any failure, a page error, an engine it cannot launch, fewer checks than it
makes, or an error of its own.