open-standards/wai/extensions/browser-sink.md

# WAI Extension: Browser Sink

> Status: Draft. How a sink that runs in a web page answers SPEC §4's "the sink advertises
> `cap`", and what the frames it decodes can be checked against. For each object it is about
> to decode, the page **routes** the object's capability to the user agent's decoder, a native
> host's or WAI's own WebAssembly decoder (§4): the user agent is asked only about codec strings
> the WebCodecs Codec Registry lists, and its decoder is preferred where it is exact for the
> tier the object needs. Among objects that are alternatives of one content, the page decodes
> the one SPEC §4's energy order puts first, inside the routing: a decode routed to the user agent
> that it answers supported, smooth and power-efficient before a software one, and a learned one
> without a sink-metered figure of its decode last (§4.7). The page evaluates the constraint sets of
> [capability-constraints](capability-constraints.md) itself (§5), and reports the joules a
> decode was measured to cost beside the user agent's `powerEfficient` boolean, which stays the
> user agent's word (§6). Every frame the page decodes carries a **content binding** (§7): the
> content hash of the receipted object it was decoded from, the link hash of the group receipt
> that object verified under, and the tier its samples hold. The binding's life through
> `clone()`, transfer, `copyTo()` and re-encode is in §8, its carriage in a page in §9, and the
> state of the frame on screen, which joins the binding to the receipts and to
> [live-provenance](live-provenance.md)'s verdict, in §10. The member a frame carries,
> `contentBinding`, is proposed for the WebCodecs VideoFrame Metadata Registry in
> [`proposals/webcodecs-content-binding.md`](../proposals/webcodecs-content-binding.md), a
> draft that has not been filed; until a registry lists the member, a page carries it in a side
> table (§9.2). No transport, no record, no change to the wire format or to SPEC §4's algorithm.
> The reference implementation is the `browser_sink` module of `wai-rs` (feature
> `browser_sink`), which `wai/wai-live-web` builds for the page; the page's own code is
> `joule-code/joule-code-rs/crates/joule-moq-web/web/room/binding.js`; the corpus is
> [`live-conformance/browser/`](../live-conformance/browser/), and its second implementation is
> `live-conformance/browser_sink_verify.py`. Each takes a `wasm` frame's decoder to be a module
> the page holds at the integer level of the WAISM™ profile (§4.3 step 6, §12).
> Keywords **MUST**, **MUST NOT**, **SHOULD**, **MAY** are RFC 2119/8174.
>
> Texts read for this revision, each pinned to the version named: WebCodecs, W3C Working Draft
> 19 October 2026: the VideoDecoder's *Output VideoFrames* steps, *Create a VideoFrame*, the
> `VideoFrameMetadata` dictionary and `metadata()`, *Copy VideoFrame metadata*, *Clone
> VideoFrame*, the constructors' metadata steps, the serialization and transfer steps, `copyTo()`
> and `EncodedVideoChunk`; the WebCodecs Codec Registry, W3C Registry Draft 24 September 2026,
> its video and audio tables; the WebCodecs VideoFrame Metadata Registry, W3C Registry Draft
> 14 April 2026, its entries and its registration requirements; and Media Capabilities, W3C
> Working Draft 18 October 2026, `MediaCapabilitiesInfo` (`supported`, `smooth`,
> `powerEfficient`) and its privacy considerations. A section number in this text is this
> extension's unless it names another document.

## 1. Scope

In scope: the capability report a page builds (§3); routing an object to a decoder (§4), and
choosing among alternatives (§4.7);
constraint sets evaluated in the page (§5); the energy entry beside `powerEfficient` (§6); the
content binding, its form and when a page writes one (§7); what happens to it through a frame's
life (§8); how a page carries it (§9); the state of the frame on screen (§10); the reasons
(§11); and a page that composes a package of several components on one timeline (§14), the
text floor of a graphics component among them (§14.9).

Out of scope: a transport; the user agent's decoders and whether they conform to their
standards, which a page cannot see beyond the answers they give and a probe (§4.4); the
decoders of a native host; how a meter measures (energy-measurement §3); and what a party does
with a state.

Relationship to the core spec and the other extensions:

- **SPEC §4.** A page answers step 3's "the sink advertises `cap`" for an object with its route
  (§4): it advertises `cap` for that object when the route is not `none`. Among alternatives it
  applies SPEC §4's energy order (§4.7).
- **[http-delivery](http-delivery.md) §4.3 and §12.** A WAI element is never put to a user-agent
  interface (§4.3 there). The route gives the Sink Capability Descriptor's `via` (§4.6).
- **[capability-constraints](capability-constraints.md).** §5 is its evaluation in a page, over
  the register of its §3.1.
- **[energy-measurement](energy-measurement.md) §2.** The classes a figure carries (§6).
- **[platform-decode](platform-decode.md) §2, §3.** The query a page asks the platform, read
  from the payload, and how the answers enter the report.
- **[jwp-receipts](jwp-receipts.md) §2.1 and §3 step 3.** The content hash a binding names, and
  the link hash of the group receipt.
- **[determinism-tiers](determinism-tiers.md).** The tier vocabulary a need and a binding use.
- **[live-provenance](live-provenance.md) §11.** The verdict on the segment a frame's object
  belongs to, for a CMAF track (§10).

## 2. Terms

- **Page**: a sink running in a web page.
- **UA decoder**: a decoder of the user agent: WebCodecs `VideoDecoder`, `AudioDecoder` or
  `ImageDecoder`. **Wasm decoder**: a WAI decoder compiled to WebAssembly that the page loads.
  **Host decoder**: a decoder of a native host the page hands a payload to (http-delivery §12.1,
  `host`).
- **Route**: `ua`, `host`, `wasm` or `none`.
- **Receipted object**: the bytes a JWP content hash covers (jwp-receipts §2.1): a WAI envelope;
  in an end-to-end encrypted room, the sealed object, over which the receipt is signed
  ([confidentiality](confidentiality.md)); in a CMAF track, a chunk (live-provenance §8.1).
- **Tier**: `decode-equivalence`, `entropy-consistency` or `none`
  ([determinism-tiers](determinism-tiers.md) §2), ordered `none` < `entropy-consistency` <
  `decode-equivalence`. A token outside this vocabulary is read as `none`, as a pin's is
  (determinism-tiers §3).
- **Registered tier** of a capability: the tier its decoded output holds on every conforming
  sink, from its replicate class ([SPEC](../SPEC.md) §7):

  | replicate class | registered tier |
  |---|---|
  | `StandardDefined`, `NeuralIntegerExact` | `decode-equivalence` |
  | `NeuralEntropyExact` | `entropy-consistency` |
  | every other class, and a string this revision does not register | `none` |

  `StandardDefined` is read as `decode-equivalence` because SPEC §7 has its decoded output the
  same bytes on every conforming sink, which is the property that tier names; determinism-tiers
  defines the tiers for neural capabilities, and this extension uses the same tokens for that
  property and claims nothing more. `GenerativeExact` output also reproduces, but it is
  generated rather than decoded, so it is held at `none` here.
- **Need**: the tier the page holds an object's decoded output to: the object's pin `tier`
  ([SPEC](../SPEC.md) §3.1) where it has one, else its track's `wai_determinism`
  ([moq-streaming-format](moq-streaming-format.md) §4), else `none`.
- **Frame**: a WebCodecs `VideoFrame`. **Binding**: a `contentBinding` value (§7).

Hexadecimal is lowercase. JSON in this text is RFC 8259.

## 3. The capability report

A page states what it knows about its decoders, and what it is about to decode, in a **capability
report**: one JSON object.

```json
{ "browser_sink": 1,
  "ua": { "<codec string>": { "supported": true, "smooth": true, "power_efficient": false,
                              "probe": "match" | "mismatch" | "not_run" } },
  "host": { "<capability>": "<tier>" },
  "wasm": [ "<capability>" ],
  "constraints": { "<capability>": [ { "<parameter>": { "enum": [ ], "minimum": 0, "maximum": 0 } } ] },
  "energy": { "<capability> <route>": { "joules_micro": 0, "decodes": 0, "window_us": 0,
                                        "class": "<class>", "relative_ppm": 0,
                                        "filtering": "On" | "Off" | "Unknown",
                                        "calibration_ref": "<text>",
                                        "coverage": "Whole" | "Partial", "fabrics": [ "cpu" ] } },
  "objects": [ { "name": "<text>", "capability": "<capability>", "codec": "<codec string>" | null,
                 "need": "<tier>", "params": { "<parameter>": 0 }, "choice": "<text>" } ] }
```

- `browser_sink` is `1`.
- `ua` holds the user agent's answers, by codec string, video and audio alike: `supported`, from
  `isConfigSupported` (`VideoDecoder`'s or `AudioDecoder`'s) for the configuration the page built
  from the object (§4.6), read from its payload ([platform-decode](platform-decode.md) §2);
  `smooth` and `power_efficient`, from Media Capabilities' `smooth` and `powerEfficient` for the
  same configuration, asked with a `video` or an `audio` member, or `null` where the page did not
  ask; `probe`, from §4.4. Where the page asked about several configurations under one codec
  string, the answer is their merge, as platform-decode §3 has it: supported only where each was,
  and a hint `true` only where each one's was.
- `host` names the capabilities a native host decodes for the page, each with the tier its
  decode holds, as the host declares it.
- `wasm` lists the capabilities the page's wasm decoders decode. A wasm decoder of an integer
  neural capability may run its `wai-ops/1-i32` graphs ([integer-operators](integer-operators.md)
  §9.1) on the page's WebGPU, through `wai/wai-ops-gpu`'s WGSL kernels compiled by the page's own
  WGSL compiler (the `gpu` unit on the `wgsl` backend, an `integer` path in
  [neural-dispatch](neural-dispatch.md) §1), once the adapter has given the host's result for
  every word of the arithmetic probe and WGSL's result for every guarded case of the
  integer-semantics suite in the page. Its
  output is then the decode's on the page's CPU, byte for byte, so the capability's tier is
  unchanged; every graph the device does not run, and every device error, runs on the page's
  CPU, which is also the decoder where no adapter passes. A capability that plays is listed only
  as §4.6 has it.
- `constraints` holds the constraint sets the page's decoders publish, by capability (§5). A
  capability with no entry is unconstrained.
- `energy` holds measurements, by capability and route (§6).
- `objects` lists what the page is about to decode, each object (or each track, whose objects
  share these values) with a `name`, its `capability`, the codec string a UA decoder would be
  asked under (`null` where none applies), its need, and its declared parameters (§5); and,
  optionally, a `choice`, a non-empty string that objects which are alternatives of one content
  share (§4.7).

A report that does not read in this shape, or whose `browser_sink` is not `1`, is refused whole
(`report_malformed`): every member above is required but an object's `choice`, which, when
present, is a non-empty string; `objects` may be empty, and a member this text does not define is
ignored. A codec string with an answer in `ua` that §4.2's gate refuses is
never read (§4.2).

## 4. Routing

### 4.1 The user agent's decoders

A capability is put to a UA decoder only through a row of this table:

| capability | UA decoder | codec string | UA tier |
|---|---|---|---|
| `wai.video.vp9` | `VideoDecoder` | `vp09.` and more | `decode-equivalence` |
| `wai.video.avc` | `VideoDecoder` | `avc1.` or `avc3.` and more | `decode-equivalence` |
| `wai.video.hevc` | `VideoDecoder` | `hev1.` or `hvc1.` and more | `decode-equivalence` |
| `wai.video.av1`, `wai.video.av1.lossless` | `VideoDecoder` | `av01.` and more | `none` |
| `wai.video.vp8` | `VideoDecoder` | `vp8` | `none` |
| `wai.audio.opus`, `wai.audio.opus.packet` | `AudioDecoder` | `opus` | `none` |
| `wai.audio.flac` | `AudioDecoder` | `flac` | `none` |
| `wai.audio.aac` | `AudioDecoder` | `mp4a.` and more | `none` |

The **UA tier** is what a UA decoder's output holds when the decoder conforms to its standard:

- `decode-equivalence` for the video codecs whose decoder conformance requires every output
  sample to equal the decoding process's (SPEC §7, `StandardDefined`), over the samples of the
  frame the decoder outputs, in the format it outputs them in (§8);
- `none` for AV1, whose film-grain synthesis is held to a perceptual criterion, for VP8, whose
  conformance this revision has not read, and for Opus and AAC (SPEC §7); and for FLAC, whose
  decoded samples are exact but which an `AudioDecoder` hands back in a sample format the user
  agent chooses, a conversion no standard fixes (SPEC §7, *What `replicate` promises over a
  standard codec*).

AV1's row stays at `none` because a codec string does not say whether a stream signals film
grain. A stream that signals none decodes to the decoding process's frames on a conforming
decoder (AV1 §7.2), and a page that has shown its decoder conforming by a probe (§4.4) may use
the frames of a **grain-free** stream as the base of a residual refinement
([platform-decode](platform-decode.md) §6): staged delivery holds that base step at
`decode-equivalence` (staged-delivery §5). That changes no route and no row of this table.

A capability with no row has no UA decoder: `wai.video.av2`, `wai.video.vvc`, `wai.video.evc`,
`wai.audio.iamf`, `wai.audio.ac4` and `wai.audio.ivas` among the registered ones, since the
registry of §4.2 lists no codec string for them (the strings their header probes read are
[platform-decode](platform-decode.md) §2.5's), and every WAI-native capability, whose decoder is
WAI's own. `ImageDecoder` has no row in
this revision: an image decode in a user agent may apply colour conversion, and the user agent
does not say whether it did.

### 4.2 The registry gate

The WebCodecs Codec Registry (W3C Registry Draft, 24 September 2026) lists these codec strings:

| media | rows |
|---|---|
| video | `av01.*`, `avc1.*`, `avc3.*`, `hev1.*`, `hvc1.*`, `vp8`, `vp09.*` |
| audio | `flac`, `mp3`, `mp4a.*`, `opus`, `vorbis`, `ulaw`, `alaw`, `pcm-*` |

A codec string **matches** a row equal to it; a row ending `.*` or `-*` is matched by a string
that begins with the row up to and including its `.` or `-` and has at least one character after
it. A codec string that matches no row:

- **MUST NOT** be passed to `isConfigSupported`, to Media Capabilities or to any other
  user-agent interface by a page that routes by this extension;
- is never answerable: an answer for it in a report is not read, whatever it says, and the
  object's UA leg is refused `codec_unregistered`.

The registry is how WebCodecs names a codec. Its abstract says no implementer is required to
support any entry; a string outside it names nothing the registry defines, so two user agents'
answers about it need not be about the same codec, and a page has no basis to read one. Such a
capability resolves through a host decoder or a wasm decoder, or not at all. A row the registry
adds later enters this table by a revision of this extension.

### 4.3 The steps

For each object `o` of a report, in order, the page computes its route:

1. **Kind.** If `o.capability` is not dispatchable (SPEC §4: a record, a derivation, a candidate,
   a companion-only or a refinement-only capability), the route is `none`
   (`not_dispatchable`).
2. **Need.** If `o.need` is above the capability's registered tier, the route is `none`
   (`tier_exceeds_class`, determinism-tiers §3).
3. **Constraints.** §5 over the report's sets for `o.capability` and `o.params`: if they cannot
   be evaluated the route is `none` (`constraints_unevaluable`), and if no set is satisfied it is
   `none` (`constraints_unmet`) — the capability is unsupported for this object, which is not a
   decode error (capability-constraints §2).
4. **The UA leg.** It is taken when every check holds; otherwise its reason is the first that
   fails:
   1. `o.codec` is a string (`no_ua_mapping`: the object names nothing to ask a UA decoder);
   2. `o.codec` passes §4.2's gate (`codec_unregistered`), whatever the capability;
   3. `o.capability` has a row in §4.1 (`no_ua_mapping`);
   4. `o.codec` is a string of that row's form (`codec_mismatch`);
   5. the report holds an answer for `o.codec` (`ua_not_asked`), and its `supported` is true
      (`ua_unsupported`);
   6. the need is at most the row's UA tier (`ua_inexact`);
   7. where the need is `decode-equivalence`, the answer's `probe` is `match` (`ua_unprobed`
      for `not_run`, `ua_probe_mismatch` for `mismatch`).
5. **The host leg.** It is taken when `o.capability` is in `host` (`no_host`) with a tier at
   least the need (`host_inexact`).
6. **The wasm leg.** It is taken when `o.capability` is in `wasm` (`no_wasm`). The route's tier
   is the registered tier, which step 2 has already held to be at least the need. A wasm
   decoder's output holds that tier only where the decoder is a module the page holds at the
   integer level of the WAISM profile
   ([waism](waism.md) §6), authorised under
   [jwp-receipts](jwp-receipts.md) §7.9 and checked before the page instantiated it; the frames
   of any other wasm decoder are bound at `none` (§7.2).
7. **Route.** The route is the first leg taken, in the order UA, host, wasm. Where none is taken
   the route is `none` (`no_decoder`), and each leg's reason is reported.

The UA leg comes first because a decoder the device ships is the one a page is expected to use,
and checks 4.6 and 4.7 take it only where its output is exact for what the object needs; the wasm
decoder is the fallback that depends on nothing the device must provide. A page **MUST** decode
an object only through its route, and **MUST NOT** decode one whose route is `none`.

Each object's entry in the **routing table** states: its `name`, its `route`, its `reason`
(`null` unless the route is `none`), each leg's outcome (`"taken"`, `"not_reached"` for a leg
after the one taken and for every leg when steps 1 to 3 decide, or the leg's reason), `asked`
(whether the page may put `o.codec` to the user agent: true exactly when steps 1 to 3 pass and
checks 4.1 to 4.4 hold), the route's `tier`, and the energy entry of §6. The entry's `reason` is
the code of the step that decided, or `no_decoder` from step 7. The route's tier is the row's UA
tier for `ua` (lowered to `none` unless the probe matched), the host's declared tier for `host`,
the registered tier for `wasm`, and `none` for `none`; in each case no higher than the
registered tier.

### 4.4 The probe

An answer of `supported` says a UA decoder accepts the configuration; it does not say the
decoder's output is the decoding process's. Where an object needs `decode-equivalence`, the page
decodes a **probe**: an object of the same capability and configuration whose decoded frames'
SHA-256 digests are published (http-delivery §12.4's probe object), each over the frame's samples
as `copyTo()` writes them with no `format`, `rect` or `layout` option (planes in order, rows
tightly packed). A frame output in `NV12` is compared after its interleaved chroma plane is split
into its two planes, which reorders samples and changes none. The probe is `match` when every
frame's digest is the published one, `mismatch` when any is not, and `not_run` when the page
decoded no probe. This revision publishes no probe for routing, so a page reports `not_run`
and a need of `decode-equivalence` is never routed to a UA decoder. It publishes one probe of
another use: a grain-free AV1 stream whose frames' digests are in
[`platform-decode-conformance/grain-free-av1/probe.json`](../platform-decode-conformance/grain-free-av1/probe.json),
by whose outcome a page takes its decoder's frames of a grain-free stream of that
configuration as a refinement base or does not ([platform-decode](platform-decode.md) §6.2).

### 4.5 The user agent's answer is a constraint check too

`isConfigSupported` answers for a whole configuration, before any decoder is configured, and
returns the configuration it would accept. It already does, for the parameters a configuration
carries, what capability-constraints asks of a sink: an answer before dispatch, for one object,
that is "unsupported" rather than a decode error. A page **SHOULD** build the configuration from
the object's own declared parameters, and **SHOULD** configure the decoder with the
configuration the answer returns. The two checks are separate and both apply: an object routes
`ua` only when its constraint sets are satisfied (step 3) and the user agent answered
`supported` (check 4.5). Neither covers the neural, quantum, world or haptic families, which have
no UA decoder; their constraints are evaluated by §5 alone.

### 4.6 What a page asks, and what it advertises

A page builds a report in two passes: from the objects alone it computes `asked` for each, and
puts to the user agent exactly the codec strings an entry marks asked; then it adds the answers
and computes the routes. Media Capabilities is asked only about a codec string the user agent
answered `supported` (http-delivery §12.2). A capability routed `ua`, `wasm` or `host` is, for
http-delivery §12.1's descriptor, `via` `"ua"` (with the codec strings asked and their answers,
`provisional` true), `"wasm"` or `"host"`; one routed `none` is absent from it.

A capability whose objects play to a deadline, such as `wai.video.int_hyper`, is in `wasm` only
where the page's decoder keeps up with the object on this device, from the decode times the page
measured on the path it decodes with (on the page's CPU, or with its graphs on WebGPU as §3 has
it): a group of `n` frames at `frame_rate_milli` frames per 1000 s, its keyframe and the frames
after it, decodes in no more microseconds, each time rounded up to a whole microsecond, than it
plays, `n · 10⁹ / frame_rate_milli`. A time that is not finite or is negative, and a group of no
frames or no rate, never keeps up. A capability whose objects are stills, such as
`wai.neural.int_hyper` and `wai.neural.int_synth`, is listed wherever the page decodes it. A
capability left out is routed `none`, its wasm leg `no_wasm`. One listed is routed `wasm`, and
§4.7 ranks it as it ranks any learned capability: after a classical alternative unless the
report holds a sink-metered figure of its decode. So a piece whose floor this device can route
plays the floor; a piece whose only alternative is the clip plays it where the decoder keeps up,
and nothing where it does not (`no_alternative`). The reference is `wai/wai-web`'s `neural_webgpu`
bundle: `IntNeural` decodes on either path, and its `capabilities` and `route` give the leg and
the report routed and chosen with it (`src/neural_leg.rs`).

### 4.7 Choosing among alternatives

Objects of a report that carry the same `choice` are **alternatives**: renditions of one content
of which the page decodes one, such as the entries of a `WAI2` envelope, a `WAI1` envelope's
capability and fallback, or the renditions of one track. For each `choice`, in order of its first
appearance in `objects`, the page puts the alternatives in SPEC §4's energy order. The order sits
inside the routing: each alternative's route is the one §4.3 gives it, and the order changes none.

- An alternative routed `ua` ranks by the user agent's answer for its codec string
  ([platform-decode](platform-decode.md) §3): `first` when `supported`, `smooth` and
  `power_efficient` are all `true`; `smooth` when `smooth` is `true` and `power_efficient` is
  not; `classical` otherwise. A `false` or a `null` is not a `true`: no answer is not a
  power-efficient one.
- An alternative routed `host` or `wasm` ranks `classical`. The user agent's answers are about
  its own decoder, which does not run this object, so they are not read for it, whatever they
  say: AV1, whose user-agent tier is `none`, and a codec whose need is `decode-equivalence` while
  no probe ran (§4.4) route elsewhere and rank on nothing the user agent said.
- An alternative whose capability is learned (SPEC §4: its replicate class is not one of the
  three standard classes, or the string is not registered) ranks `classical` when the report
  holds a measurement under its route's key (§6, `"<capability> <route>"`) that reads under §6
  with the class `OnChipCounter` or `CalibratedInstrument` and the coverage `Whole`: a
  sink-metered figure of its decode, with its class, uncertainty and scope. It ranks
  `learned-without-budget` otherwise. The figure's value is not read.
- An alternative routed `none` comes after every other, with no rank.

Report order within a rank. `powerEfficient` is the user agent's hint, used as the user agent
gave it, as an order and never as an energy figure or as evidence of hardware. The **chosen**
alternative is the first in that order whose route is not `none`, and where every route is `none`
nothing is chosen (`no_alternative`). A page **MUST** decode the chosen alternative of a choice
and no other, so a learned alternative without a qualifying measurement is decoded only where no
classical alternative can be routed. For a `WAI1` envelope this is SPEC §4's order of steps 3 and
5: the chosen alternative is tried first, and the other after it if the first does not decode.

The **choice table** has one entry a choice: `choice`; `order`, the alternatives' names in that
order, those routed `none` included; `ranks`, each one's rank in the same order, `null` for one
routed `none`; `chosen`, a name or `null`; and `reason`, `null` or `no_alternative`. An object with
no `choice` is in no entry, and a report whose objects carry none has an empty table.

## 5. Constraint sets in the page

A page evaluates [capability-constraints](capability-constraints.md) §2 in this form:

- A capability's sets are a JSON array of one or more **constraint sets**. A set is a JSON
  object whose members are parameter names, each mapped to a **constraint**: a JSON object with
  one or more of `enum` (a non-empty array of values), `minimum` and `maximum` (integers), and no
  other member. A value is a JSON string, or an integer: a JSON number with no fraction and no
  exponent, from −(2^53 − 1) to 2^53 − 1. A page whose own numbers do not keep `1920.0` apart
  from `1920` writes both as `1920`, so the integers it writes are exactly its integral
  values; a value with a fractional part stays a fraction.
- A parameter name **MUST** be one of the register of capability-constraints §3.1.
- An object's declared parameters are a JSON object of the same names and values.

`evaluate(sets, params)` is:

1. No sets for the capability: `unconstrained`.
2. The sets break the form above — not an array, an empty array, a set that is not an object, a
   constraint that is not an object, a constraint with no member or with a member other than the
   three, an empty `enum` or one holding something other than a value, a `minimum` or `maximum`
   that is not an integer, or a `minimum` above a `maximum`: `unevaluable`
   (`constraint_malformed`).
3. A set names a parameter outside the register: `unevaluable` (`parameter_unregistered`). A
   page **MUST NOT** advertise a capability whose sets it cannot evaluate (capability-constraints
   §2).
4. A set is **satisfied** when, for each parameter it names, the object declares a value for it
   and the value meets each member of the constraint: for `enum`, it equals one of the values,
   a string only a string and an integer only an integer; for `minimum` and `maximum`, it is an
   integer at least, or at most, the bound. An undeclared parameter, or a declared value that is
   neither a string nor an integer, meets no constraint. An empty set is satisfied by every
   object.
5. `satisfied` when one set is; otherwise `unsatisfied` (`constraints_unmet`).

The grammar is the one capability-constraints adopts, enumerations, minima and maxima ORed over
sets, with integers in place of rationals, so every evaluation is exact.

## 6. Energy beside `powerEfficient`

Each entry of the routing table carries an **energy entry**:

```json
{ "power_efficient": true | false | null, "figure": null | { … }, "reason": null | "<reason>" }
```

- `power_efficient` is Media Capabilities' `powerEfficient` for the object's codec string,
  copied as the user agent gave it, for an object routed `ua`; it is `null` for any other route
  and where the page did not ask. It is the user agent's word: its own specification leaves the
  meaning of optimal power draw to the user agent. A page **MUST NOT** set it, invert it or
  compute it from a figure, **MUST NOT** state it for a route the user agent did not answer for,
  and **MUST NOT** derive a figure from it.
- `figure` is the measurement the report holds under the key `"<capability> <route>"`, where
  `<route>` is the object's route, and `null` when it holds none or the route is `none`. A
  measurement under another route's key is not this route's figure. A page **MAY** take a measurement
  from its sink's decode table ([sink-energy](sink-energy.md) §4): a row whose status is
  `figure` gives one, as that extension's §1 maps it, and a row of any other status gives
  none.

A measurement is read as energy-measurement §2 has it, and refused, with `figure` `null` and the
entry's `reason`, when:

- its `class` is not one of `CalibratedInstrument`, `OnChipCounter`, `ModelBased` and `Estimator`
  (`energy_class_invalid`; `HwShunt` is never emitted, energy-measurement §2.6);
- `joules_micro` is `0` (`unmetered_as_zero`): an unmetered decode has no measurement, and a
  class beside a zero claims a measurement that did not occur;
- `joules_micro`, `decodes` or `window_us` is not a positive integer below 2^53
  (`energy_count_invalid`);
- a hardware class lacks its evidence: `relative_ppm`, a positive integer, for
  `CalibratedInstrument` and `OnChipCounter`; `filtering`, one of `On`, `Off` and `Unknown`, for
  `OnChipCounter`; `calibration_ref`, a non-empty string, for `CalibratedInstrument`
  (`energy_evidence_missing`);
- `coverage` is not `Whole` or `Partial`, or `fabrics` is not a non-empty array of distinct
  strings (`energy_evidence_missing`).

A figure that reads is reported with every member it was given and
`per_decode_micro = ⌊joules_micro / decodes⌋`; `joules_micro` and `decodes` stay beside it, so the
floor loses nothing. An object with no measurement is **unmetered**: `figure` and `reason` are
both `null`, and a page presents it as unmetered, never as 0 J. In this revision a figure never
changes a route, and none is steered on: §4.7 reads only whether a learned alternative's figure
qualifies, never its value. How a figure is measured, bracketed and
attributed is energy-measurement §3 and §4's, and a `Partial` coverage says the meter saw only
the fabrics it names.

## 7. The content binding

### 7.1 The value

A **content binding** is a JSON object, and in a page a dictionary of the same members, the
`contentBinding` member of a frame's metadata: proposed for the WebCodecs VideoFrame Metadata
Registry, and not registered (§9):

| member | value |
|---|---|
| `alg` | the algorithm `digest` is computed with. This revision defines `jwp-object-1`: the content hash of jwp-receipts §2.1, `BLAKE3("moq-jwp:object\x01" ‖ object)`. |
| `digest` | 64 hexadecimal digits: the content hash of the receipted object the frame was decoded from. |
| `receipt` | optional; 64 hexadecimal digits: the link hash (jwp-receipts §3 step 3) of the group receipt that object verified under. |
| `tier` | the tier the frame's samples hold (§7.2). |

Its **canonical form** is the JSON text of those members in that order, `receipt` left out when
absent, with no whitespace and no other member. A reader:

- refuses a value that is not an object, that lacks `alg`, `digest` or `tier`, whose members
  are not strings, whose `digest` or `receipt` is not 64 hexadecimal digits
  (`binding_malformed`);
- refuses an `alg` it does not know (`binding_alg_unknown`): it cannot check the digest;
- reads a `tier` outside the vocabulary as `none`;
- ignores a member this text does not define, as a dictionary drops one.

### 7.2 When a page binds a frame

A page binds a frame it decoded, and only then, when:

1. the frame was output for a chunk whose coded data came from one receipted object, and the page
   computed `digest` over that object's bytes as received, or holds it as the content hash of the
   object receipt that object verified against;
2. the content hash of every object the decoder was given since its last key frame — its last
   `key` chunk, or its configuration — was established: the object verified against its JWP
   object receipt (jwp-receipts §3 steps 1 to 5), or it was fetched whole under that hash as its
   content identifier, or through a verified head proof (http-delivery §6, §8.3). A frame
   predicted from data whose hash nobody established is not bound
   (`binding_unverified_reference`). A page that verifies every object before a decoder sees it
   meets this by construction.

`receipt` is written exactly when the object verified against a group receipt (jwp-receipts §3
steps 1 to 5), and is that receipt's link hash; a page **MUST NOT** write a receipt it did not
verify the object against. For an object carried over MoQ, `digest` and `receipt` are then the
`content_hash` and `group_receipt` of its `WAI_RECEIPT` property
([moq-streaming-format](moq-streaming-format.md) §5). `tier` is the route's tier (§4.3),
lowered to `none` where:

- the route is `ua` and the frame's `format` is `null`: its samples cannot be read without a
  conversion, so nothing can check them;
- the route is `wasm` and its decoder is not a module the page holds at the integer level of the
  WAISM profile (§4.3 step 6);
- the page altered the samples after the decode (determinism-tiers §2, *What a tier covers*).

Where the frame's tier is below the object's need, the page **MUST NOT** present the frame as
meeting it.

On the wasm route the wasm decoder writes the binding itself: it holds the object's bytes, hashes
them, decodes them, and returns every frame with the binding.

## 8. Through the frame's life

| operation | the binding |
|---|---|
| a decoder's output | none from the decoder: WebCodecs' *Create a VideoFrame* takes no metadata. The page binds it (§7.2, §9). |
| `clone()` | carried: the clone references the same samples, and *Clone VideoFrame* assigns it every slot but the resource reference, `[[metadata]]` among them. |
| serialization (`postMessage` without transfer) | carried: the serialization steps copy `[[metadata]]`. |
| transfer | carried to the receiving realm; the source is closed and keeps none. |
| `close()` | gone with the frame. |
| `copyTo()` | the bytes written carry none. With no `format` option they are the frame's samples, and a page **MAY** check them against a published digest (§4.4); with one they are a conversion, which no tier covers. |
| `new VideoFrame(frame, init)` | carried only when the page puts it in `init.metadata`: the constructor copies `init`'s metadata, not the source's. The new frame references the source's samples; a page **MAY** copy the binding into it, and a narrower `visibleRect` presents part of the same samples, from the same object. |
| `new VideoFrame` from a canvas, an image or a buffer | a frame of other samples: none, except on the wasm route, where the samples are the ones the wasm decoder output from the bound object (§7.2). |
| drawing the frame (`drawImage`, a texture) | the drawn pixels are a rendering and carry none; §10 says which frame a region of the screen shows. |
| `VideoEncoder.encode()`, and a decode of its output | **does not survive.** A re-encoded frame is a new object: its chunks carry no binding, and a frame decoded from them is bound, if at all, to that new object and its own receipt. |

## 9. Carriage in a page

### 9.1 As metadata

Where the user agent's `VideoFrameMetadata` has a `contentBinding` member, a page binds a decoded
frame `f` by constructing `new VideoFrame(f, { metadata: { ...f.metadata(), contentBinding } })`
and closing `f`. `clone()`, serialization and transfer then carry it by WebCodecs' own steps (§8).

### 9.2 In a side table

`VideoFrameMetadata` is a dictionary whose members are the registry's, and a dictionary keeps only
the members it declares, so a member no registry lists is dropped from `init.metadata`. Until
`contentBinding` is listed, a page carries the binding in a side table:

1. **Pending.** Before it gives a decoder a chunk, the page enters the chunk's binding under (that
   decoder, the chunk's `timestamp`). A decoder outputs a frame with the timestamp of the chunk it
   came from (WebCodecs *Output VideoFrames*), and nothing else in a frame says which chunk that
   was. Where a second chunk is given the same decoder under a timestamp still pending, both
   entries are struck and neither frame is bound (`binding_ambiguous`). A page bounds the entries
   pending per decoder, and strikes the oldest past the bound.
2. **Output.** On a decoder's output, the page takes the entry under (that decoder, the frame's
   timestamp), removes it, and attaches it to the frame object, held weakly so that it lives as
   long as the frame does. An output with no entry is unbound.
3. **Reset.** `reset()`, `close()` and a decoder error strike that decoder's pending entries, and
   so does `flush()` once it resolves, for the entries whose frames it did not output; a frame
   output afterwards for a struck entry is unbound.
4. **Clone.** `clone()` attaches the source's binding to the clone.
5. **Transfer.** A frame posted to another realm is posted with its binding beside it, and the
   receiver attaches it on arrival. A frame that arrives without one is unbound.
6. **Close.** `close()` detaches it.

A page **MAY** test whether the user agent keeps the member (construct a frame with it in
`init.metadata` and read `metadata()`), and use §9.1 where it does.

## 10. The frame on screen

For each region of the screen a page draws frames into (a tile), the page keeps the binding of the
frame it last drew there. The **state** of that frame is computed from:

- the binding, or none;
- `object`: `verified` (the object verified against its receipt), `refused`, or `unverified` (no
  receipt has been checked);
- `chain`: the group receipt's admission to its chain (jwp-receipts §3 step 3): `linked`,
  `anchor`, `history`, `held` (not yet admitted: `parent_not_held`) or `refused`;
- `provenance`: for a track with a provenance track (live-provenance §10.2), the verdict of
  live-provenance §11.4 on the segment of the frame's group, `accepted` or `refused`, or `missing`
  while its provenance object has not arrived; `not_applicable` for a track packaged as WAI
  envelopes, which has no C2PA provenance (live-provenance §1).

The state is, in this order:

1. `unbound`, with no binding or with one the reader refuses (§7.1);
2. `refused`, when `object`, `chain` or `provenance` is `refused`;
3. `pending`, when `object` is `unverified`, `chain` is `held` or `provenance` is `missing`;
4. `verified` otherwise.

`anchor` and `history` are admissions: a chain a joiner anchored, or filled below its anchor. A
page that shows `verified` for them **MUST** also say from which group it holds the chain, as
jwp-receipts §3 step 3 requires of a verifier that re-anchors, and **MUST NOT** state the groups
before it verified.

A page that shows the state, behind a control the reader turns on, shows for each tile the state,
the tier, the first digits of the digest and of the receipt, the provenance, the route and the
energy entry. The state is about the frame the tile last drew, not about the pixels on the
display: what the display emits is a rendering of the frame, scaled and converted to the
display's colour space, which no tier covers.

## 11. Reasons

| reason | raised at |
|---|---|
| `report_malformed` | §3 |
| `not_dispatchable` | §4.3 step 1 |
| `tier_exceeds_class` | §4.3 step 2 |
| `constraints_unevaluable` | §4.3 step 3 |
| `constraints_unmet` | §4.3 step 3, §5 step 5 |
| `no_ua_mapping` | §4.3 steps 4.1, 4.3 |
| `codec_mismatch` | §4.3 step 4.4 |
| `codec_unregistered` | §4.2, §4.3 step 4.2 |
| `ua_not_asked` | §4.3 step 4.5 |
| `ua_unsupported` | §4.3 step 4.5 |
| `ua_inexact` | §4.3 step 4.6 |
| `ua_unprobed` | §4.3 step 4.7 |
| `ua_probe_mismatch` | §4.3 step 4.7 |
| `no_host` | §4.3 step 5 |
| `host_inexact` | §4.3 step 5 |
| `no_wasm` | §4.3 step 6 |
| `no_decoder` | §4.3 step 7 |
| `no_alternative` | §4.7 |
| `constraint_malformed` | §5 step 2 |
| `parameter_unregistered` | §5 step 3 |
| `energy_class_invalid` | §6 |
| `unmetered_as_zero` | §6 |
| `energy_count_invalid` | §6 |
| `energy_evidence_missing` | §6 |
| `binding_malformed` | §7.1 |
| `binding_alg_unknown` | §7.1 |
| `binding_unverified_reference` | §7.2 |
| `binding_ambiguous` | §9.2 step 1 |

An object routed `none` is inert at the sink for that object, as SPEC §4 step 6 has it; a refused
energy measurement leaves the route as it is; a refused binding leaves the frame unbound.

## 12. Conformance

[`live-conformance/browser/`](../live-conformance/browser/) holds the corpus, `vectors.json`,
written by `wai_browser_sink_vectors gen`. It has:

- **Routes**: capability reports, each with the routing table §4 and §6 give it: every route and
  every reason of §4.3, the registry gate with codec strings for AV2, VVC and IAMF answered
  `supported` and never read, a need of `decode-equivalence` refused a UA decoder whose probe did
  not run and taken by one whose probe matched, the host and wasm fallbacks, and a malformed
  report.
- **Choices**: reports whose objects are alternatives, each with its routing table and the choice
  table §4.7 gives it: `first`, `smooth` and `classical` before `learned-without-budget`; a
  power-efficient answer that is not smooth below a smooth one; a power-efficient answer for a
  codec routed to a host decoder (its need above the user agent's tier) read for nothing; a
  learned alternative with a qualifying measurement (`OnChipCounter`, `CalibratedInstrument`)
  ranked `classical`, and with a `Partial` or a `ModelBased` one ranked last; a learned
  alternative chosen where no other routes; no answer read as not power-efficient; two choices
  beside an object in none; a choice with nothing to choose (`no_alternative`); and an empty and a
  non-string `choice`, each refused whole. Every measurement in them is synthetic, marked
  `"note": "synthetic: no measurement"`.
- **Constraints**: constraint sets and declared parameters, with §5's outcome: satisfied by one
  set of several, by an empty set, by `enum`, `minimum` and `maximum`; unsatisfied by an
  undeclared parameter and by a value of the wrong type; each malformed form and an unregistered
  parameter.
- **Energy**, among the routes: measurements beside answers, each with its energy entry: every
  class, the unmetered case, a figure under another route's key, and each refusal of §6.
- **Bindings**: values with their reading: canonical forms, each refusal of §7.1, an unknown tier
  read as `none`, and a member the text does not define; the tier §7.2 gives a frame for each
  route, format and alteration, a `wasm` frame's decoder being a module the page holds at the
  integer level of the WAISM profile (§4.3 step 6); and objects bound under a signed group
  receipt, each binding's `digest` the object's content hash and its `receipt` the receipt's link
  hash, the receipt's signature and the object's inclusion verifiable from the corpus.
- **Frames**: inputs to §10 with their state, covering every row.
- **Side tables**: sequences of chunks, outputs, clones, transfers and resets with the bindings
  §9.2 leaves each frame, `binding_ambiguous` among them.

`wai_browser_sink_vectors verify` and the test `tests/browser_sink.rs` check that `vectors.json`
regenerates byte for byte, that `wai-rs` gives every case its stated outcome, that every reason of
§11 appears, and that §11 lists exactly the reference's codes. `frame_tier` in `wai-rs`, which
the page calls as `frameTier`, is given no module status: it gives a `wasm` frame the tier §7.2
gives the frame of a decoder held at the integer level, and every `wasm` frame of the corpus, the
chain's among them, is of such a decoder. The page whose code the header names loads no wasm
decoder for a room's tracks, so it binds no frame on the `wasm` route.
`live-conformance/browser_sink_verify.py` is the second implementation, written from this text:
it shares no code with `wai-rs`, reads each capability's kind, status and replicate class from the
registry export ([`registry/capabilities.json`](../registry/capabilities.json)), checks
signatures with WAI's strict rule and hashes with BLAKE3 from `claim-conformance/`, and mutates
copies of the corpus and requires every copy to fail. The page runs the route, choice, constraint,
energy, binding and frame cases through its wasm build, and its side table through the sequences, in a
browser (`joule-moq-web`'s end-to-end scenario `binding`). `wai conform sink --suite browser` runs
every call of the corpus but the chain against another implementation, its answer withheld, and
signs a record of the run (conformance-results); `wai_browser_sink_vectors answers` is the
reference's answer to one call.

## 13. Security and privacy considerations

- **A binding is a statement, not a signature.** Any script in the page can bind any frame. What
  a binding gives is where to look: the receipt it names commits to the digest under a key, and a
  party that holds the object, the receipt and its key log checks them. The state of §10 is the
  page's own verdict on what it checked.
- **What `verified` covers.** The object and its group receipt, and, for a CMAF track, the
  segment's C2PA provenance. It does not cover what the display shows, how a frame was scaled, or a
  user agent's decoder: a UA decoder that misdecodes a verified object outputs a frame bound to that
  object. §4.4's probe is how a page holds a UA decoder to the decoding process where the tier
  matters.
- **What a report reveals.** A report holds the user agent's answers and, with a meter attached,
  an energy figure; together they say a good deal about a device (Media Capabilities, privacy
  considerations). Nothing in this extension sends a report anywhere: routing, binding and the
  state of a frame are computed in the page. A service that collects reports learns that much
  about each device it collects from; where it needs to record a legal basis for doing so,
  jwp-receipts §2.4.6's `lawful-basis` claim is the place.
- **An energy figure is a side channel.** A per-process energy reading, taken often enough, leaks
  what a process does. A page receives a figure only from a meter the host chose to attach, at the
  granularity the host chose, and this extension never has a page read a counter itself.
- **Ambiguity is refused, not guessed.** A side table keyed by timestamp cannot tell two chunks of
  one decoder with one timestamp apart, so neither frame is bound (§9.2).
- **Unregistered codec strings are not asked about.** Besides having no defined answer, a question
  about a codec string outside the registry adds to what a page learns about a device for nothing
  it can use.

## 14. Composing a package

A package ([packaging](packaging.md)) may carry a programme as several components, each in the
capability that suits it: camera footage in a video codec or an integer rung, titles and credits
as [vector graphics](graphics-vector.md), captions as [timed text](timed-text.md), music as a
[score](score.md), data as [signed state](state-feeds.md). A **composing page** opens such a
package, dispatches each component by §4, checks everything it is about to present against the
package's signed receipt, decodes each component on the route it chose, and presents them all on
one timeline. A piece's base may have layers ([staged-delivery](staged-delivery.md)), and the page
computes two kinds of them itself, in WebAssembly, before it presents any stage they give: the
levels of an integer neural field ([neural-fields](neural-fields.md) §7) and a picture refined by a
layer conditioned on side planes (staged-delivery §6.3) (§14.5). The reference is `wai/wai-web`'s
`compose` bundle (`src/compose.rs` and `src/compose_stage.rs`, feature `compose`) and its page,
`wai-web/compose/` (`compose.js`, `index.html`); the package it plays is
`wai-web/compose/package/`, which `wai-pack` writes under `compose/policy.toml` (packaging §5.8),
and the staged packages beside it are `wai-web/compose/staged/` (§14.7).

### 14.1 What the page reads

The page reads a version-2 package receipt (packaging §9.1), the index of each component it
names (packaging §6), and the objects the indexes list (`objects/<content hash>.wai`). Every
component is a run of pieces on its own timebase, the lead's cut at its scene cuts and every
other component's at the lead's piece starts (packaging §4.5, §5.8), each piece one object
or more. Within a component, a piece's **alternatives** are its base entry and each of its
floor entries; a refinement rides on the base it refines and is not an alternative. An asset
entry of a piece is an object a layer of that piece may name as a side object (staged-delivery
§3.1); the page reads it only when a layer names it (§14.5). The page
reads nothing else of a package: no playlist, catalog or listing, none of which the receipt
binds. A text piece names its language in its object's `target` (packaging §5.8), which is
how the page reads its segment.

### 14.2 Layers

The layer a decoded object is presented in follows from its capability, never from its
component's media class:

| capability | layer | drawn as |
|---|---|---|
| a video capability, `wai.image.png` in a video or an image component | video | frames on the picture, at the lead's frame rate; a still held for its piece |
| `wai.field.int_grid` | video | the field's render at the view its payload pins ([neural-fields](neural-fields.md) §6), held for its piece |
| `wai.graphics.vector` | overlay | an `RGBA8P` frame (graphics-vector §2.9) composited source-over above the picture |
| `wai.text.zstd`, the floor of a piece whose base is `wai.graphics.vector` | floor | the blocks of the piece's transcript that are active at the clock's tick, as text (§14.9) |
| `wai.text.timed` | text | text, in the page's own text layer, at each cue's anchor |
| an audio capability, `wai.audio.score` | audio | samples on the page's audio output |
| `wai.feed.state` | state | the records of the state at the clock's tick |

The vector overlay is rasterised at a size its payload lists (graphics-vector §3.10): the smallest
listed size at least as large as the overlay's area on the display in device pixels, else the
largest listed. The platform scales it from there (graphics-vector §1.1).

### 14.3 Dispatch

For each piece of each component, the page builds §3's report over the piece's alternatives,
each object carrying the `choice` `"<component>/<piece>"`, and decodes the alternative §4.7
chooses. The report's `wasm`, `host` and `ua` say what the device has: a decoder the page holds
in WebAssembly is in `wasm`, a capability that plays only where the decoder keeps up (§4.6); the
user agent is asked, as §4.6 has it, about every alternative whose capability platform-decode
maps (AV1, HEVC, AVC, VP9 and VP8 video; Opus, FLAC and AAC audio) and that the page does not
decode in WebAssembly itself, from a query read from that object's payload
([platform-decode](platform-decode.md) §2), after the object has been checked (§14.4); each
configuration is asked, and answers under one codec string are merged (platform-decode §3). A
capability the page decodes in WebAssembly (FLAC in the reference page) is not put to the user
agent, so its UA leg is `ua_not_asked` and it routes `wasm`: the page's decode holds the
capability's registered tier, the user agent's `none` (§4.1). A profile without the page's
decoder of it asks the user agent. A field (`wai.field.int_grid`) and the objects a conditioned
layer reads as side planes (`wai.image.png`, and `wai.splat.int_codec` for a render) the page
computes in WebAssembly too: platform-decode maps none of them, so their UA leg is
`no_ua_mapping`, the user agent is never asked about them, and they route `wasm`. The
capabilities whose layers the page applies in WebAssembly (`wai.field.int_refine`,
`wai.image.int_cond_refine`) are listed in the report's `wasm` leg beside its decoders. A layer is
not an alternative and is not routed: it is applied as §14.5 has it where its capability is in
that leg and its piece's base was chosen, and is otherwise not applied, its piece presenting the
stage below (a floor chosen takes no layer). An alternative routed `ua` is decoded through WebCodecs from
the chunks platform-decode §2.3 reads from its payload, with the first configuration the user
agent accepted under which it decodes; a video one is presented in the video layer, an audio one
on the page's audio output. An Ogg Opus stream (`wai.audio.opus`) states a pre-skip and an end
(RFC 7845 §4) that a decoder handed its packets does not apply: the page drops the pre-skip,
ends the samples at the last page's granule position, and holds an Opus floor's piece (packaging
§7.9) to exactly its piece's samples (`compose_frames` otherwise). A **forced profile** is a report from which named capabilities are
left out of every leg and never asked about; it routes and chooses as any other.

- The chosen alternative is decoded first. Where the user agent's decoder does not decode it,
  having answered the configuration supported, the page decodes the next alternative in §4.7's
  order whose route is not `none`, after checking that object as §14.4 has it: §4.7's "the
  chosen alternative is tried first, and the other after it". The dispatch record keeps what was
  tried and why it did not decode (`tried`).
- A piece whose alternative is a record (`wai.text.timed`) is not routed: SPEC §4 dispatches no
  decoder for a record, and the page draws it in its text layer. Of a package's text
  components the text layer presents one, or none (§14.9).
- A component with a piece for which nothing is chosen (`no_alternative`) is not presented on
  this device, and the page says so. The lead component (the receipt's first) carries the clock
  (§14.6): a package whose lead has such a piece is refused whole.

The **dispatch record** is what this device dispatched, and why, as data a page can show:

```json
{ "device": { "user_agent": "<text>", "forced_without": [ "<capability>" ],
              "wasm": [ "<capability>" ], "ua": { "<codec string>": { "supported": true, … } } },
  "receipt": { "receipt_hash": "<hex>", "signer_id": "<text>", … },
  "components": [ { "id": "<id>", "layer": "<layer>" | null, "presented": true,
                    "pieces": [ { "piece": 0, "chosen": "<name>" | null, "capability": "<capability>",
                                  "role": "base" | "floor" | null, "route": "ua" | "host" | "wasm" | "page" | "none",
                                  "why": "<text>", "order": [ "<name>" ],
                                  "tried": [ { "name": "<name>", "error": "<text>" } ],
                                  "alternatives": [ { "name": "<name>", "capability": "<capability>",
                                                      "role": "base" | "floor", "route": "<route>",
                                                      "reason": "<reason>" | null, "legs": { … },
                                                      "tier": "<tier>", "rank": "<rank>" | null } ],
                                  "layers": [ { "name": "<component>/<piece>/L<layer>/<capability>",
                                                "capability": "<capability>", "layer": 1,
                                                "route": "wasm" | "none",
                                                "outcome": "pending" | "applied" | "refused" | "not-applied",
                                                "reason": "<reason>" | null, "detail": "<text>" | null,
                                                "stage": 1 | null, "sha256": "<hex>" | null,
                                                "sides": [ { "entry": "<name>" | null, "object": "<hex>",
                                                             "render": { … } | null, "held": true,
                                                             "exact": true | null, "capability": "<capability>",
                                                             "via_fallback": false, "tier": "<tier>",
                                                             "why": "<text>" | null } ] } ] } ] } ] }
```

An alternative's `route`, `reason`, `legs` and `tier` are its §4.3 routing-table entry, and its
`rank` its §4.7 rank; `why` says in words which rule decided: the base first in §4.7's order, the
floor first in that order, or the floor because the base has no route here, with the base's
reason and legs. A piece with layers lists each, in layer order: its outcome (`pending` until it
arrives), the reason it was refused or not applied (a reason of staged-delivery §4.2, an object
check's code (packaging §6.3), or `prefix-digest`), the stage it presented and that stage's SHA-256, and each side
plane it named: the object, the render's pose for a render entry, whether the page held its plane,
whether exactly, how it was decoded, and why a plane was not held.

### 14.4 Checks before presenting

The page presents nothing of a package until each of these holds, in this order; the first that
fails refuses the package whole, and the page states which check failed and with which code:

1. the receipt's JSON, its indexes, their digests and stage roots, its counts and its Ed25519
   signature (packaging §9.1, `PackageReceiptV2::verify_detailed`; in the page,
   `verifyPackageReceipt`, and `verifyPackageReceiptSignature` for the signature alone);
2. each index, read now that the receipt holds it (packaging §6);
3. every object the page is about to decode, and every floor object it reads a query from,
   against its index entry: its length and content hash, its envelope, lengths and capability,
   its `refines` and `derived_from` (packaging §6.3);
4. each decoded output whose entry states decode-equivalence, against the entry's prefix digest
   (packaging §6.5), piece by piece: a video base's clip digest (a still's frame held for its
   piece); a vector piece's clip digest over its frames rasterised in the page as `RGBA8P` at
   the first size its payload lists and its component's rate; a text piece's SHA-256 over its
   segment, once the segment reads to its cues (timed-text §5.3); a score piece's SHA-256 over
   its samples as signed 16-bit little-endian integers, a two-channel score's interleaved
   (score §3.4), and a FLAC piece's over its samples, interleaved, as the page decodes them
   in WebAssembly; a field's render at its own view, computed in the page, whose SHA-256 is
   the field's stage hash (staged-delivery §5); a picture of an image component, its RGB8 frame
   decoded in the page (packaging §6.5); and a feed piece's state, replayed from its signed ops
   to its seal, against the sealed state hash ([state-feeds](state-feeds.md)),
   each piece after a component's first starting from the state the one before it seals.

A floor decoded by the user agent is held to no digest: its tier is `none` (§4.1), and the page
says so; an audio floor among them too, whatever its codec, since the user agent chooses the
sample format it hands back. A check's outcome is listed with what it compared, so a page can
show every check it made. A piece whose base has layers is presented at its base once these
hold; each layer is checked as it arrives (§14.5), and one that fails a check is not applied,
without refusing the package.

### 14.5 Refinement layers

A piece whose chosen alternative is its base presents the base first. Its layers then arrive one
at a time, in layer order, and for each the page, in this order:

1. fetches it and checks it against its entry: its length and content hash, its envelope and
   lengths, its capability and its `refines` (packaging §6.3);
2. computes the next stage in WebAssembly, with the reference implementation of staged-delivery
   §4.1 compiled for the page, which refuses with that section's reasons (§4.2):
   - over a field, the payload's level as the field's next level (neural-fields §7.3,
     staged-delivery §6.2), the field holding no layer for later (a hold bound of zero), so a
     level whose level below is not applied is refused `layer-gap`; then the field rendered
     again at its own view;
   - over a picture, `wai.image.int_cond_refine` (staged-delivery §6.3). First each side plane
     its `refines.side` names (§3.1), held by the page or not. An object the piece's asset entries
     list is fetched once and held only where the page decodes its capability in WebAssembly
     under the device's profile, its bytes pass their entry's checks (packaging §6.3), and its
     output, decoded through its manifest's capability or its fallback as SPEC §4 dispatches,
     holds the digest its entry states. A render entry's plane is the frame
     `wai-splat-render:i1` gives at the entry's pose ([splat-exact](splat-exact.md) §5), which the
     page renders itself from a splat set held so, decoded through `wai.splat.int_codec` with the
     parameter set the page holds, its pin verified. Then §4.1 step 4 refuses the layer where
     the page does not hold a plane (`side-absent`), or holds one that is not exact
     (`side-not-exact`: decoded through a fallback, or below `decode-equivalence`);
3. holds the stage's output to the prefix digest its entry states: for a field the SHA-256 of
   its render at its own view, for a picture of its RGB8 frame, each the stage hash
   staged-delivery §5 defines (`prefix-digest`);
4. presents the stage.

A layer refused at step 1 or 2 is not applied, and one refused at step 3 is computed and never
presented; either way the page presents the stage below it (the base, for a first layer) and
says why: the dispatch record states each layer's outcome and reason, and each side plane, held
or not, exact or not, and why not (§14.3). The page then takes the next layer, and the reference
implementation decides it (a level whose level below was refused at step 1 or 2 is refused
`layer-gap`). The page never presents a stage whose digest it has not compared, nor one that
failed. An asset's prefix digest is its decoded output's stage hash (staged-delivery §5),
the attribute tensor's bytes for a splat set.

### 14.6 One timeline

The lead component's video is the master clock. Its frame `n` is presented from `n / fps`
seconds, `fps` its index's timebase; every other layer is placed by that clock:

- the overlay presents its frame `g = ⌊t · num / den⌋` at its rate `num/den`, from the piece
  whose window holds `g` (a piece's frames are the whole document's, graphics-vector §3.11), and
  nothing outside its pieces' frames;
- the floor of a graphics piece presents the blocks active at the tick `⌊t · tb⌋` of its
  transcript's timebase, from the piece the clock is in (§14.9), and nothing where that piece
  presents its overlay;
- the text layer shows the cues active at `t` of the piece the clock is in (a cue across a piece
  start is in both pieces' segments, with its whole interval, timed-text §5.2);
- the state layer shows the state at tick `⌊t · tb_num / tb_den⌋` of its component's timebase,
  from the piece that holds the tick, never past that piece's seal;
- the audio layer plays its pieces' samples end to end, in their channels and at their
  rate (every piece of a component the same, or the page refuses it,
  `compose_audio_format`), from the position the clock gives, and re-places them at the
  clock's position whenever they drift from it by more than one video frame.

The reference page plays a piece's decoded frames from a clock it starts on the page's own
timer, and the score through Web Audio, so the measure of drift is the audio context's own time.
A layered piece's picture is the latest stage it presented; the reference page moves the clock to
a piece as it presents that piece's next stage, so the stage is on screen.

### 14.7 The package's shape

[packaging](packaging.md) §5 (*Components beside the lead*, 5.8) is the normative home of a
package of several components: the
media classes `graphics` (7) and `text` (8) beside packaging's others, the rungs each medium
takes, how each component is cut at the lead's piece starts, each piece's object and `target`,
and the decoded-output digest each entry states. The reference package is written by
`wai-pack` under `compose/policy.toml`: the lead `v`, a test pattern of 320 × 180 (the fixture
`compose-01`) with an integer base and a supplied AV1 floor for every piece; `g`,
`vector-conformance`'s title card and credit roll; `t`, three captions; `s`, a voiced score
of two channels at 48 kHz with a FLAC floor of each piece's mixdown, which the page decodes
in WebAssembly where the score is not routed (a floor states no digest, so the page holds it
to none; the page asks the user agent about no audio codec); and `f`, a scoreboard feed. The
corpus check of `wai-pack` packs it again from those sources and requires every byte the same.
A composing page does not read a component's media class for a layer (§14.2); it reads it for
the digest a picture states (packaging §6.5: an image's RGB8 frame, a video's clip).

Beside it, the page plays the staged packages under `wai-web/compose/staged/`. `src/compose_stage.rs`'s
tests seal them with the package's key from objects of the corpora, byte for byte, each entry
stating the prefix digest computed natively and held there to the corpus's own statement, and
require the committed files to be those bytes. Each is one image component on the timebase 1/1,
one unit a piece:

| package | its pieces |
|---|---|
| `fields` | three fields of `volumetric-conformance/fields/`: the orb, its base carrying two of its four levels and its two further levels as layers 1 and 2; `mlp`; `edge` |
| `fields-digest` | `fields`, the orb's layer 2 stating the digest of its stage 1 |
| `cond` | three pictures of `staged-conformance/cond/`, `cond_side_planes`, `cond_dual_gbuffer` and `cond_render`, each a PNG base, its `wai.image.int_cond_refine` layer and its side objects as asset entries (two planes, three G-buffers, a splat set) |
| `cond-not-exact` | `cond_side_not_exact`'s layer over a plane carried as `wai.image.jxl` with a `wai.image.png` fallback (an asset of tier none) |

They are sealed under no policy (their policy digest is zero), and an image lead and asset
entries are not among the shapes of a package's components (packaging §5.8); a page reads them
as it reads any package (§14.1).

### 14.8 Conformance

`wai-web/compose/run.mjs` plays the reference package in browser engines and passes when, in each:
the package opens with every check of §14.4 holding and every component presented, and plays to
its end with frames, overlay frames, cues, state and audio presented; the score's samples,
synthesised in the page in the channels and at the rate each piece's `target` states, have
piece by piece the SHA-256 its index entries state, which
`wai-pack` computed natively; every overlay frame rasterised in the page has the SHA-256
[`vector-conformance/`](../vector-conformance/) states, and each piece's frames the clip
digest its entry states; each text piece holds to its segment's digest and each feed piece to
its seal, chained to the piece before; under a profile without the AV1 decoder each integer
video base decodes in WebAssembly to the clip digest its entry states; under a profile without
the integer rung and the still, every video piece plays its AV1 floor through the user agent,
the base unrouted for want of a WebAssembly decoder (`no_wasm`), and the page says the floor
plays; under a profile without the score, every score piece plays its FLAC floor, decoded in
WebAssembly to the samples whose SHA-256 the score's signed base states for its mixdown, and
the page says the floor plays; in `pack-conformance/multi/multi-opus-01`, under a profile
without FLAC, where the engine's `AudioDecoder` takes Opus every piece of the sound track plays
its Opus floor through it and comes to exactly the piece's samples, and where it does not the
sound track is not presented and the picture is; a receipt signed by another key (`compose/mutations/forged.receipt.json`, which
`wai-pack`'s corpus check writes and holds) is refused `receipt-signature`; and an object with
one byte changed in delivery is refused `object-hash`; each of the last two with nothing
presented. Over the staged packages (§14.7), held to what `volumetric-conformance/fields/` and
`staged-conformance/cond/` state as well as to their signed entries: every field of `fields`
renders in the page at its pinned view to the frame digest its scene's vectors state, read back
so from the drawn picture; with each of the orb's levels held back in delivery and released one
at a time, the coarse field is presented before its first level is asked for, and each level
released presents the next stage, whose digest is the stage hash the corpus states, before the
next level is asked for; in `fields-digest` the orb's stage 2 is computed, refused
`prefix-digest`, and stage 1 stays presented; a byte of the orb's first level changed in delivery
is refused `object-hash`, its second level `layer-gap`, and the coarse field stays presented;
every picture of `cond` presents the stage hash its case states, from two exact planes, three
G-buffers and a splat set's frame rendered in the page; a side object withheld in delivery, and
the splat decoder left out of the profile, each refuse their layer `side-absent` with the base
presented and the page saying why; `cond-not-exact`'s layer is refused `side-not-exact`, its plane
held through its fallback, with the base presented; and a side object with a byte changed in
delivery is refused `object-hash`, never held, and its layer `side-absent`. Over
`pack-conformance/multi/multi-captions-01`, a title card with captions in two languages and a
text floor (§14.9): on this device every graphics piece presents its overlay and no floor;
under a profile without `wai.graphics.vector` every graphics piece presents its floor, each
floor's transcript the one this script reads from the package's own object, the base unrouted
(`no_wasm`), no overlay frame drawn, and the page says the floor plays; at every frame of the
programme the text the page shows from the floor is the title wherever its block is active
and no caption or subtitle, while the text layer shows each caption's cue over the block's
milliseconds, so the words are on the page once; with `captions=fr` the layer shows the French
component's cues and with `captions=off` none, the floor's text unchanged; a byte of a floor
object changed in delivery is refused `object-hash`, and the floors of two pieces exchanged in
delivery `object-hash`, each with nothing presented; and the bundle's reader, given in the page
one piece's floor and another piece's base, refuses `floor_base`. It exits 0 only when
every check passes in every engine named, and fails on a page error, on an engine that cannot
be launched or runs fewer checks than it makes, and on an error of its own.

### 14.9 A graphics component's text floor

A graphics component may carry, for each piece, a floor of `wai.text.zstd`: the piece's
transcript in a frame that has one encoding ([graphics-vector](graphics-vector.md) §6.3). It is
an alternative of its piece (§14.1). A page that reads it lists `wai.text.zstd` in its report's
`wasm` leg; the base and the floor then both rank `classical` (§4.7), and the base, listed
first, is chosen wherever it has a route. A page presents a piece's floor, as text, where the
piece's base has no route on this device, so that §4.7 chooses the floor: a device whose report
holds no decoder of `wai.graphics.vector`.

A piece presents its overlay or its floor and never both, and a piece with neither is not
presented (§14.3). A base that is routed and then refused, by graphics-vector §2.7 when its
payload is loaded or for frames that do not hold their signed clip digest, refuses the package
(§14.4): in this revision its floor does not stand in for it. The reference is `wai-rs`'s `browser_sink::floor`,
which the `compose` bundle calls (`TextFloor`).

**Reading the floor.** After the object's checks against its index entry (§14.4), the page
holds, in this order, and the first that fails refuses the package with its code:

1. the manifest's `derived_from` names the piece's base: its `motif` is the content hash of the
   piece's base entry, its `motif_capability` is `wai.graphics.vector` and its `renderer` is
   `wvg1-transcript/1` (`floor_base`). The index says only that a floor carries a
   `derived_from` (packaging §6.3); this check holds it to this piece, so a floor of another
   piece or of another document, consistent in every other respect, is not presented in this
   piece's place;
2. the payload is the raw frame of graphics-vector §6.3 and no other Zstandard frame: the magic
   number, the frame header descriptor `A0`, the content's length in four bytes, raw blocks of
   which every one before the last holds 2^17 bytes and the last, with its last-block bit set,
   from 1 to 2^17, and nothing after it (`floor_frame`). The frame has one encoding for each
   content, and a page that read another would present bytes no package verifier compared;
3. the manifest's `target` states `bytes`, the content's length, and `tb`, an integer from 1 to
   2^32 − 1 (`floor_target`);
4. the content is a transcript in the JSON of graphics-vector §5.4, in that section's encoding
   and no other: one array of block objects with that section's members in its order; every
   integer at most 2^53 − 1; each `dir` 0 or 1, each `mode` 0 to 2, each `group` at most 65 535
   and each `purpose` from 1 to 255 and never 13; the indices ascending; each active set
   ascending, with no interval of no ticks and no two intervals sharing a tick or an end; and
   the blocks, written again in that encoding, are the content byte for byte
   (`floor_transcript`).

A floor's entry states no decoded-output digest (packaging §6.5). What holds the words is the
object's content hash, which the signed index states and the page has checked, and the
`derived_from` of step 1.

**Presenting it.** At clock time `t` (§14.6) the page presents the floor of the piece the clock
is in, at the media tick `τ = ⌊t · tb⌋`, as graphics-vector §6.3 has a sink do: every block
whose active set holds `τ` (`on ≤ τ < off`), in block order, as text through the page's own
text rendering, each with its language, its direction and its writing mode. Two kinds of block
are not presented from the floor, and the page records why:

- a block of purpose `caption` or `subtitle`, always (`caption_track`). Graphics-vector §6.4
  holds the package to a text component in the block's language whose cues carry the block's
  string over the same milliseconds, and the page's text layer presents those cues, or does
  not, as the viewer has set it (graphics-vector §5.7). A viewer who has turned captions off
  gets none from the floor;
- a block of purpose `forced_narrative`, while the page's text layer shows a cue of the block's
  language with the block's string (`forced_in_cue`): a cue of the text component the layer
  presents whose language, in ASCII lower case, is the block's in ASCII lower case, whose text
  is the block's string byte for byte (its lines separated by U+000A,
  [timed-text](timed-text.md) §3.2), and whose interval holds the tick's millisecond, `start ≤
  ⌊1000 · τ / tb⌋ < end`. Graphics-vector §6.4 has that component carry such a cue over the
  block's milliseconds, so the words are on the page once: from the cue while the layer
  presents that component, and from the floor while it presents none or one in another
  language, since a forced narrative is shown whether or not captions are (graphics-vector
  §5.7). The page compares the cue it is showing, not the package rule, so no block is withheld
  for a cue that is not on the page.

The strings are inserted as text and never as markup, and the blocks are in the page's
document, where the platform's accessibility interface reads them (graphics-vector §5.5). The
page announces a block, politely and once for the interval that holds `τ`, only where its
purpose is one graphics-vector §5.2 has announced by default (`title`, `lower_third`,
`forced_narrative`, `legal`, `alert`) and that interval is at least a second long (`off − on ≥
tb`); it makes no assertive announcement. Where the words appear, their size and their face are
the page's, and no digest covers them.

| code | given when |
|---|---|
| `floor_base` | reading, step 1: the package is refused |
| `floor_frame` | reading, step 2: the package is refused |
| `floor_target` | reading, step 3: the package is refused |
| `floor_transcript` | reading, step 4: the package is refused |
| `caption_track` | presenting: a caption or a subtitle is left to the text layer |
| `forced_in_cue` | presenting: a forced narrative is on the page in a cue |

**The text layer's component.** A package may hold several text components, one a language
(packaging §5.8). The text layer presents one of them or none: the one in the language the
viewer chose, else the first the receipt lists, and none where the viewer turned captions off.
Every text component's segments are checked (§14.4) whether or not the layer presents it. The
reference page takes the choice as `captions=<language>` or `captions=off`.

**Conformance.** [`live-conformance/browser-floor/`](../live-conformance/browser-floor/) holds
the corpus, `vectors.json`, written by `wai_browser_floor_vectors gen`: floor objects that read
(graphics-vector §5.4's worked example, a programme of ten purposes, a transcript of no blocks)
and, each a control of one defect, floors refused for their base, their frame, their target
and their transcript; and presentations, each a floor, a tick and the cues of the text
component presented, with the blocks presented and those withheld: a window's first tick, its
last and the tick it ends at; a caption and a subtitle with and without a text component; a
forced narrative in two languages with no text component, with its cue shown, with a cue in
another language, of another string, ended and not begun, and with a language tag in another
case; and the announcement of each purpose over intervals shorter than, equal to and longer
than a second. `wai_browser_floor_vectors verify` and the test `tests/browser_floor.rs` check
that the file regenerates byte for byte, that `wai-rs` gives every case its outcome, that every
code of the table above is given by a case, that the table lists exactly the reference's codes,
and that the floor objects of `pack-conformance/multi/multi-captions-01`, which `wai-pack`
wrote, read against the bases they name. `live-conformance/browser_floor_verify.py` is the
second implementation, written from this text: it shares no code with `wai-rs`, gives every
case the same outcome, requires every mutated copy of a floor to be refused with the reason its
defect has and every mutated copy of the corpus to fail, and drops each of its own checks in
turn and requires the corpus to catch it (one check, that a frame ends with a block flagged
last, can differ from the others only in a frame of 2^17 bytes, and is caught by frames the
verifier writes itself). `wai-web`'s native test `compose::floor_tests` holds
the same package's floors to `wai-vector`: each is the transcript `wai-vector` computes from
the piece's base, in the frame it writes. In browser engines, `compose/run.mjs` plays
`multi-captions-01` (§14.8).