open-standards/wai/wai-web/compose/README.md

The composing sink

index.html opens a WAI package of several components, dispatches each to a decoder this device can run, checks everything it is about to present against the package's signed receipt, and plays the components on one timeline (browser-sink §14).

file	what it is
compose.js	the runtime: openPackage() and the Composition it returns
index.html, page.js	the reference page: the picture, the dispatch record and the checks
policy.toml, sources/, floor/, key.hex	the package's policy and inputs
package/	the package it plays, written by wai-pack (packaging §5.8)
staged/	the staged packages: fields and pictures with layers the page computes (browser-sink §14.5, §14.7)
mutations/forged.receipt.json	the package's receipt signed by another key
run.mjs	the gate: the page in browser engines, every outcome held to what it expects

The package

One programme of seven seconds as five components under one version-2 package receipt, each cut at the lead's piece starts, each piece one object:

component	capability	source	presented as
v	wai.video.int_motion bases and a wai.video.av1 floor for every piece	the fixture compose-01, a test pattern of 320 × 180 at 25 frames a second	the picture, and the clock
g	wai.graphics.vector	vector-conformance/vectors/motion/stroke-titles.wvg1, a title card and a credit roll at 1280 × 720	an overlay rasterised in WebAssembly
t	wai.text.timed	sources/t.vtt, three captions	text, in the page's text layer
s	wai.audio.score of two channels at 48 kHz, each note voiced, and a wai.audio.flac floor of each piece's mixdown	sources/s.wscr, score-conformance/vectors/scale48.wscr's notes up and down	samples synthesised in the tab in two channels (or, without the score, the floor decoded from FLAC in WebAssembly), on Web Audio
f	wai.feed.state	sources/f.wwld, a scoreboard of signed ops	the state at the clock's tick

Each entry with a decoded output states its digest, which the page holds what it decodes to: each video base's clip digest, each overlay piece's clip digest, each caption segment's SHA-256, each score piece's sample digest; each feed piece is sealed by the package's key.

wai-pack's corpus check (wai-pack conformance check ../pack-conformance, which tests/conformance.rs runs) writes the sources under sources/, packs the package from them and requires every file the same, verifies it, and writes the forged receipt again (wai_pack::multi_fixture::compose_check; conformance regen writes them). By hand:

# from wai/wai-pack
cargo run --release --bin wai-pack -- fixture gen compose-01 -o ../wai-web/compose/src.y4m
cargo run --release --bin wai-pack -- pack --policy ../wai-web/compose/policy.toml \
  --out <dir> --key ../wai-web/compose/key.hex
cargo run --release --bin wai-pack -- verify <dir>

The staged packages

staged/ holds four packages of one image component each, sealed with key.hex from objects of the corpora, byte for byte, by src/compose_stage.rs's tests, which compute every prefix digest natively and hold it to the corpus's own statement:

package	pieces
fields	the orb of volumetric-conformance/fields/ (a base carrying two of its four levels, then its levels 2 and 3 as layers 1 and 2), mlp and edge
fields-digest	fields, the orb's layer 2 stating stage 1's digest
cond	cond_side_planes, cond_dual_gbuffer and cond_render of staged-conformance/cond/: a PNG base, its wai.image.int_cond_refine layer and its side objects as asset entries
cond-not-exact	cond_side_not_exact's layer over a plane the page decodes only through its fallback

The page renders each field at its pinned view and each picture's base in WebAssembly, presents them, then takes each layer as it arrives: a field's next level, or a picture's conditioned layer with the side planes it names (a splat set's frame rendered in the page at its pinned pose), each stage held to its signed digest before it is presented; a refused layer leaves the stage below. The tests check, and WAI_COMPOSE_STAGED_WRITE=1 writes them again:

# from wai/wai-web
cargo test --lib --features compose compose_stage::

The AV1 floor floor/v.obu is supplied (packaging §7.3), made the way pack-conformance/supplied_floor.py makes a golden's: wai-pack plan cuts the fixture (two pieces, of 100 and 75 frames), each piece is coded alone by aomenc with that script's settings, and the streams are concatenated. The policy pins the floor directory's listing.

Run the page

# from wai/wai-web: the compose bundle
cargo build --release --target wasm32-unknown-unknown --lib --features compose
wasm-bindgen --target web --out-dir compose/pkg --out-name wai_compose \
  target/wasm32-unknown-unknown/release/wai_web.wasm
# serve wai/, which holds the outline set the overlay names
cd .. && python3 -m http.server 8741
# → http://localhost:8741/wai-web/compose/index.html

Query parameters: without=<capability>,… forces a device profile without those capabilities; receipt=<url> reads another receipt; pkg=<dir>/ another package; captions=<language> has the text layer present the text component in that language, and captions=off none (the default is the first the receipt lists).

Titles as text

A graphics component may carry a text floor: each piece's transcript as wai.text.zstd (graphics-vector §6.3). On a device with no decoder of wai.graphics.vector (without=wai.graphics.vector) the page presents it (browser-sink §14.9): it holds each floor to its piece's base, reads the one frame the format writes and the transcript in its one encoding, and at each tick shows the active blocks as text, each with its language and direction. A caption or a subtitle is never shown from the floor, because the text layer carries it; a forced narrative is shown from the floor unless a cue is showing the same words. pack-conformance/multi/multi-captions-01 is such a package:

http://localhost:8741/wai-web/compose/index.html?pkg=../../pack-conformance/multi/multi-captions-01/package/&without=wai.video.av1,wai.graphics.vector

The gate

cd compose && npm ci
node run.mjs --engines chromium,webkit --pkg pkg

In each engine: the package opens with every check holding and every component presented, and plays to its end; each score piece's samples, synthesised in the tab in the two channels and at the rate its target states, have the digest its index entry states; every overlay frame hashed in the page equals vector-conformance/'s, and each piece's clip digest its entry's; each caption segment and each feed piece holds; without the AV1 decoder each integer video base decodes to its signed clip digest; without the integer rung and the still every video piece plays its AV1 floor and the page says so; without the score every score piece plays its FLAC floor, decoded in WebAssembly to the samples the score's signed base states for its mixdown, and the page says so; in pack-conformance/multi/multi-opus-01, without FLAC, the sound track's Opus floor plays through the engine's AudioDecoder where it takes Opus, each piece trimmed to exactly its samples (an engine whose decoder refuses that golden's 64 × 36 picture refuses it at the lead); the forged receipt is refused receipt-signature and an object with one byte flipped in delivery object-hash, with nothing presented. Over the staged packages: each field renders at its pinned view to the frame digest volumetric-conformance/fields/ states, read back from the canvas; the orb's levels, each held back in delivery and released one at a time, present each stage, its stage hash the corpus's, before the next level is asked for; a stage that is not its signed digest is refused prefix-digest and a level with a byte flipped object-hash, the stage below staying presented; each picture of cond presents the stage hash its case states; a side object withheld, or the splat decoder left out of the profile, refuses its layer side-absent, a plane held only through its fallback side-not-exact, and a side object with a byte flipped is refused object-hash and never held, each with the base presented. Over pack-conformance/multi/multi-captions-01: on this device each graphics piece presents its overlay and nothing comes from a floor; without wai.graphics.vector each presents its floor, whose transcript is the one the script reads from the floor object, and at every frame the text from the floor is the blocks active at that tick less the captions, the text layer shows the cues of the component it presents (English, French with captions=fr, none with captions=off), and a caption's words are on the page once; a floor with a byte flipped, and two floors exchanged in delivery, are refused object-hash; and the bundle's reader refuses a floor held to another piece's base floor_base. --only staged runs the staged cases alone, --only package the reference package's and --only floor the text floor's. It exits 0 only when every check passes in every engine, and 1 on any failure, a page error, an engine it cannot launch, fewer checks than it makes, or an error of its own.