open-standards/wai/extensions/platform-decode.md

# WAI Extension: Platform Decode Capabilities

> Status: Draft. Maps two web platform interfaces, Media Capabilities `decodingInfo()`
> and WebCodecs `isConfigSupported()` (`VideoDecoder`'s with its hardware-acceleration
> preference, and `AudioDecoder`'s), onto the capability report of
> [browser-sink](browser-sink.md) §3: what a sink asks the platform about an entry of every
> capability browser-sink §4.1 gives a user-agent decoder, AV1, HEVC, AVC, VP9 and VP8 video
> and Opus, FLAC and AAC audio, read from the entry's payload (§2), the chunks it gives the
> decoder, read the same way (§2.3), and how the answers enter the report (§3); §2.5 says why
> the sink-supplied AV2, EVC, AC-4, IAMF and IVAS give no query in a page. The answers are hints. They order the alternatives a page can decode, inside
> browser-sink's routing, by SPEC §4's energy order (browser-sink §4.7); they never change a
> route, are never an energy figure and are never evidence of hardware. A grain-free AV1
> stream decoded by the platform is a base `wai.video.int_refine` refines, once a probe has
> shown the platform's decoder gives the decoding process's frames (§6). No manifest member
> and no wire format.
> Keywords **MUST**, **MUST NOT**, **SHOULD**, **MAY** are RFC 2119/8174.

## 1. Scope

A sink running in a web platform can decode a standard video or audio codec through the
platform's own decoders, which may be fixed-function hardware, or through a decoder of its
own: WebAssembly, or a learned reconstruction such as WAI's integer codecs. The platform
says, for a configuration, whether it can decode it, whether it can do so without
dropping frames, and whether it judges the decode power-efficient (Media Capabilities
§2.2). A learned reconstruction runs on the sink's CPU, GPU or neural engine, and what it
draws there is known only where it has been measured. This extension says how a WAI sink
asks the platform about an entry, and how it enters what the platform says in browser-sink's
capability report, where the routing (browser-sink §4) decides what the page can decode and
the energy order (SPEC §4, *Energy order*; browser-sink §4.7) orders what it can.

It reads these documents:

- Media Capabilities, W3C Working Draft, 18 October 2026: `MediaDecodingConfiguration`,
  `VideoConfiguration`, `MediaCapabilitiesInfo` (§2.1, §2.2), the decoding-info algorithm
  (§2.3.2), and the privacy considerations (§3);
- Media Capabilities, W3C Working Draft, 9 November 2026: `AudioConfiguration` and
  `MediaDecodingConfiguration` (which members are required), and the valid audio
  configuration steps;
- WebCodecs, W3C Working Draft, 19 October 2026: `VideoDecoderConfig` (§7.6),
  `isConfigSupported()` and Check Configuration Support (§4, §7.1), and
  `HardwareAcceleration` (§7.9);
- the WebCodecs AV1 codec registration (Group Note Draft, 13 October 2026) and HEVC codec
  registration (Group Note Draft, 8 June 2026), which define the codec strings by the AV1
  ISOBMFF binding (AV1 Codec ISO Media File Format Binding v1.3.0, §5) and by ISO/IEC
  14496-15 Annex E.3;
- the WebCodecs VP9 and VP8 codec registrations (Group Note Drafts, 16 October 2026): the
  codec strings, a chunk as one frame, a key chunk, and no `description`; the VP Codec ISO
  Media File Format Binding's *Codecs Parameter String* (the eight fields, which are
  mandatory, the defaults, the subsampling and level values); and VP9 Levels and Decoder
  Testing (each level's limits);
- the WebCodecs AVC, Opus, FLAC and AAC codec registrations: the codec strings (`avc1.` or
  `avc3.` and six characters; `opus`; `flac`; the `mp4a.` strings), a chunk (an access unit;
  an Opus packet, or an audio data packet in the `ogg` format; a FLAC frame; an ADTS frame or
  a raw data block), and the `description` (none for Annex B and ADTS; the identification
  header for Ogg Opus; the `fLaC` marker and `STREAMINFO`, required, for FLAC).

ISO/IEC 14496-15 was not read. The HEVC codec string §2 writes follows the form the HEVC
registration describes (four dot-separated fields of Annex E.3 and the constraint bytes
after them), checked against an open-source parser of that form (Chromium's
`ParseHEVCCodecId`) and against a platform that decodes with it (§5). The headers §2.1 and
§2.2 read (the H.264 SPS, the VP9 uncompressed header, the VP8 frame tag, Ogg pages,
`OpusHead` and the TOC byte, FLAC's metadata and frame headers, the ADTS header) are read as
the clauses cited beside them define them; ISO/IEC 14496-3 and 13818-7 were not read for this
revision, and the ADTS fields follow the AAC registration's reference to them. Every payload
of the vectors is also read by an independent media prober, whose readings the vectors keep
(§5).

## 2. The query

For an entry whose capability §2.1 or §2.2 maps, the sink forms the query from the entry's
payload. It **MUST NOT** form it from anything the envelope declares apart from the
payload: the codec string a payload's encoder declared can differ from the stream
(§5 records one that did). A payload a reader below refuses gives no query; the sink decodes
the entry, or does not, as it would without this extension.

### 2.1 Video

| capability | codec string | size | frame rate | bit rate |
|---|---|---|---|---|
| `wai.video.av1`, `wai.video.av1.lossless` | from the first temporal unit's sequence header OBU: `av01.`, the profile, the level (two digits) and the tier (`M` or `H`), the bit depth (two digits); then the monochrome flag, the chroma subsampling (three digits), the colour primaries, transfer characteristics and matrix coefficients (two digits each, 01 each where the sequence header carries no colour description), and the full-range flag; the part after the bit depth omitted where it is exactly `.0.110.01.01.01.0` (AV1-ISOBMFF §5) | `w` × `h` of SPEC §5's AV1 framing | `fps_num` / `fps_den` of the framing | the payload's bits × the frame rate ÷ `n`, rounded up |
| `wai.video.hevc` | from the first SPS NAL unit (H.265 §7.3.2.2, §7.3.3): `hev1.`; the profile space as nothing, `A`, `B` or `C`, then `general_profile_idc` in decimal; the 32 `general_profile_compatibility_flag`s as one hexadecimal value with flag *j* at bit *j*; `L` or `H` for `general_tier_flag`, then `general_level_idc` in decimal; then the six bytes from `general_progressive_source_flag` on, each in hexadecimal, trailing zero bytes omitted. `hev1` because an Annex B stream carries its parameter sets in band | `pic_width_in_luma_samples` × `pic_height_in_luma_samples`, less the conformance window; the coded size before it | none in the payload: the rate the sink will present at, where it has one | the payload's bits × the frame rate ÷ the pictures the stream holds, rounded up |
| `wai.video.avc` | from the first SPS NAL unit (H.264 §7.3.2.1.1): `avc1.`, then `profile_idc`, the byte of `constraint_set0_flag` to `reserved_zero_2bits`, and `level_idc`, two upper-case hexadecimal digits each (RFC 6381 §3.3). `avc1`, not `avc3`: the AVC registration lets the string begin with either and fixes the bitstream format by the configuration's `description`, not by the prefix (an Annex B stream has none), and one engine measured answers every `avc3.` string unsupported while every engine measured accepts `avc1.` (§5) | `PicWidthInMbs` · 16 × `FrameHeightInMbs` · 16, less the frame cropping rectangle (H.264 §7.4.2.1.1); the coded size before it | none in the payload: the rate the sink will present at, where it has one | the payload's bits × the frame rate ÷ the pictures the stream holds, rounded up |
| `wai.video.vp9` | from the first frame of the IVF container, which is a key frame (VP9 §6.2, §7.2): `vp09.`, the profile, the level (below) and the bit depth, two decimal digits each; then the chroma subsampling (`01` for 4:2:0, `02` for 4:2:2, `03` for 4:4:4), the colour primaries and transfer characteristics (`01` each: VP9 states neither), the matrix coefficients of its `color_space` (below), and `color_range`, two digits each; the part after the bit depth omitted where it is exactly `.01.01.01.01.00` (VP Codec ISO Media File Format Binding, *Codecs Parameter String*). A 4:4:0 stream has no subsampling code and gives no query | the first frame's `frame_width_minus_1` + 1 × `frame_height_minus_1` + 1, which is also the coded size | the mean over the frames' IVF timestamps: `k` − 1 frame periods from the first frame's timestamp to the last's, `k` the frames the container holds, in the time base its header states (bytes 16 to 23, as a denominator then a numerator), reduced to lowest terms; where that is not known (one frame, a zero time base, a last timestamp not after the first), the rate the sink will present at | the payload's bits × the frame rate ÷ `k`, rounded up |
| `wai.video.vp8` | `vp8` | the first frame's, which is a key frame: the 14-bit width and height after its start code (RFC 6386 §9.1) | as VP9's | as VP9's |

An Annex B stream's pictures are its slices that begin one: an H.264 slice (`nal_unit_type` 1
or 5) whose `first_mb_in_slice` is 0, an H.265 VCL NAL unit whose
`first_slice_segment_in_pic_flag` is 1. An IVF stream is read whole: the signature `DKIF`,
version 0, a header of at least 32 bytes and the capability's FourCC (`VP90`, `VP80`), then
frames of a 12-byte header and the bytes it counts, tiling the rest of the payload; the size
and frame count the IVF header states are a container's declarations and are not read. A
payload whose first frame is not a key frame, or whose key frame header breaks its
specification (a VP9 `frame_marker` other than 2, a sync code other than `0x49 0x83 0x42`, the
reserved colour space, an RGB one in profile 0 or 2; a VP8 version past 3 or a missing start
code), gives no query.

**VP9 levels.** VP9 states no level. The query's is the first level of this table (VP9 Levels
and Decoder Testing) whose maximum luma picture size, maximum luma width and height, and,
where the query has a frame rate, maximum luma sample rate, the size × the rate, the stream
does not exceed. The levels' other limits (bit rate, buffer size, compression ratio, tiles,
alt-ref distance, reference frames) are not read, so the level is the least a decoder must
support for that size and rate. A stream past level 6.2 gives no query.

| level | max luma sample rate | max luma picture size | max luma width or height |
|---|---|---|---|
| `10` | `829440` | `36864` | `512` |
| `11` | `2764800` | `73728` | `768` |
| `20` | `4608000` | `122880` | `960` |
| `21` | `9216000` | `245760` | `1344` |
| `30` | `20736000` | `552960` | `2048` |
| `31` | `36864000` | `983040` | `2752` |
| `40` | `83558400` | `2228224` | `4160` |
| `41` | `160432128` | `2228224` | `4160` |
| `50` | `311951360` | `8912896` | `8384` |
| `51` | `588251136` | `8912896` | `8384` |
| `52` | `1176502272` | `8912896` | `8384` |
| `60` | `1176502272` | `35651584` | `16832` |
| `61` | `2353004544` | `35651584` | `16832` |
| `62` | `4706009088` | `35651584` | `16832` |

**VP9 colour.** The matrix coefficients (ISO/IEC 23091-2) a VP9 `color_space` names:

| `color_space` | VP9 | matrix coefficients |
|---|---|---|
| `0` | `CS_UNKNOWN`: none stated, so the default | `01` |
| `1` | `CS_BT_601` | `05` |
| `2` | `CS_BT_709` | `01` |
| `3` | `CS_SMPTE_170` | `06` |
| `4` | `CS_SMPTE_240` | `07` |
| `5` | `CS_BT_2020` | `09` |
| `7` | `CS_RGB`, with full range and no subsampling | `00` |

`CS_BT_601` takes the BT.601 row that is not SMPTE ST 170's (`05`), and `CS_SMPTE_170` that one
(`06`); the two rows' equations are the same.

The sink asks:

- `navigator.mediaCapabilities.decodingInfo()` with a `MediaDecodingConfiguration` of
  `type` `"media-source"` and a `video` member: `contentType`
  `video/mp4; codecs="<codec string>"`, the size, the bit rate and the frame rate. Where
  the query has no frame rate it does not ask: `framerate` and `bitrate` are required
  members, and a sink **MUST NOT** invent them.
- `VideoDecoder.isConfigSupported()` with a `VideoDecoderConfig` of the codec string, the
  coded size as `codedWidth` and `codedHeight`, and the `hardwareAcceleration` the sink
  will configure, when it decodes the entry through WebCodecs.

### 2.2 Audio

| capability | codec string | sample rate | channels | `description` | bit rate |
|---|---|---|---|---|---|
| `wai.audio.opus` | `opus` | 48000, the rate Opus decodes at (RFC 7845 §5.1: the input rate `OpusHead` states is informational) | `OpusHead`'s channel count | the `OpusHead` packet | the payload's bits × 48000 ÷ (the last granule position a page states − `OpusHead`'s pre-skip), rounded up; none where that is not positive |
| `wai.audio.opus.packet` | `opus` | 48000 | 2 where the TOC byte's stereo flag is set, else 1 (RFC 6716 §3.1) | none | the payload's bits × 48000 ÷ the samples its TOC byte states, rounded up |
| `wai.audio.flac` | `flac` | `STREAMINFO`'s (RFC 9639 §8.2) | `STREAMINFO`'s | the `fLaC` marker and every metadata block, as the payload holds them | the payload's bits × the sample rate ÷ `STREAMINFO`'s total samples, rounded up; none where it states 0 |
| `wai.audio.aac` | from the first ADTS frame header (ISO/IEC 14496-3 §1.A.3.2): for `ID` 0, `mp4a.40.` and the audio object type, `profile` + 1, in decimal; for `ID` 1, `mp4a.` and the object type indication of its profile, `66` (Main), `67` (LC) or `68` (SSR) (RFC 6381 §3.3) | its `sampling_frequency_index`'s | its `channel_configuration`, 8 for 7 | none | the payload's bits × the sample rate ÷ 1024 × the raw data blocks of every frame, rounded up |

An Ogg Opus payload is read whole as one logical Ogg bitstream (RFC 3533): pages that tile the
payload, each with the capture pattern, version 0 and its CRC, the first beginning the stream,
each of its serial number and the sequence number after the one before, and a packet
continuing onto a page exactly where that page says it does; then the identification header
(`OpusHead`, a major version of 0, one channel or more, at most two under mapping family 0,
a channel mapping table under any other), the comment header (`OpusTags`), and one audio
packet or more (RFC 7845 §3). An Opus packet's samples are its configuration's frame
duration times its frame count (RFC 6716 §3.1); an empty packet, a code 3 packet with no
frame count, and a packet of more than 120 ms are refused. A FLAC payload is the `fLaC`
marker, `STREAMINFO` first, then metadata blocks up to the one whose last-block flag is set,
then frames (§2.3); a `STREAMINFO` of a zero sample rate, a depth under 4 bits or a minimum
block size under 16 is refused. An ADTS payload is frames that tile it, each header with its
sync word, layer 0, a listed sampling frequency, a `channel_configuration` other than 0 (which
leaves the channels to a program configuration element the query does not read), the first
frame's configuration (`ID`, `profile`, sampling frequency, channels), and a length at least
its header's. An HE-AAC stream signalled implicitly, inside its raw data blocks, is asked
about as the object type and the core rate its headers state.

The sink asks:

- `navigator.mediaCapabilities.decodingInfo()` with a `MediaDecodingConfiguration` of `type`
  `"media-source"` and an `audio` member (Media Capabilities, W3C Working Draft, 9 November
  2026, `AudioConfiguration`): `contentType` `audio/mp4; codecs="<codec string>"`, `channels`
  as a string, `samplerate`, and `bitrate` where the query has one. Only `contentType` is a
  required member, so a query with no bit rate is still asked.
- `AudioDecoder.isConfigSupported()` with an `AudioDecoderConfig` of the codec string, the
  sample rate, the channels as `numberOfChannels`, and the `description`, when it decodes the
  entry through WebCodecs. `AudioDecoderConfig` has no hardware-acceleration preference.

### 2.3 What a decoder is given

The chunks a sink gives a WebCodecs decoder are read from the payload by the readers of §2.1
and §2.2, so a payload that gives no query gives no chunks, and each chunk's `type` is read
from its own bytes (`platform_decode::chunks`):

| capability | `description` | a chunk | a key chunk |
|---|---|---|---|
| `wai.video.av1`, `wai.video.av1.lossless` | none | a temporal unit of SPEC §5's framing | its first frame header is of a sequence with `reduced_still_picture_header` 1, or has `show_existing_frame` 0 and `frame_type` `KEY_FRAME` |
| `wai.video.hevc`, `wai.video.avc` | none: the registrations' Annex B format | an access unit, from the start code of its first NAL unit to the next unit's | it holds an IRAP slice (H.265, `nal_unit_type` 16 to 23) or an IDR one (H.264, 5) |
| `wai.video.vp9`, `wai.video.vp8` | none | an IVF frame's data | the first frame it holds is a key frame (VP9 `frame_type` `KEY_FRAME` with `show_existing_frame` 0; VP8 the frame tag's key-frame flag) |
| `wai.audio.opus` | the `OpusHead` packet: the registration's `ogg` format | an audio packet | every chunk |
| `wai.audio.opus.packet` | none: the registration's `opus` format | the payload | every chunk |
| `wai.audio.flac` | the `fLaC` marker and the metadata blocks | a frame | every chunk |
| `wai.audio.aac` | none: the registration's `adts` format | an ADTS frame, its header included | every chunk |

An access unit begins at the stream's first NAL unit, and then, after the last VCL NAL unit of
the one before, at the first NAL unit of a type that begins one (H.264 §7.4.1.2.3: access unit
delimiter, SPS, PPS, SEI, types 14 to 18; H.265 §7.4.2.4.4: access unit delimiter, VPS, SPS,
PPS, prefix SEI, types 41 to 44 and 48 to 55) or at a slice that begins a picture; NAL units
after the stream's last VCL unit stay in its unit. A FLAC frame begins with a frame header that
reads (RFC 9639 §9.1: its sync code, no reserved block size, sample rate, channel or depth code,
its reserved bit 0, its coded number, the uncommon block size and rate it calls for, and its
CRC-8) and ends where its CRC-16 over the frame, footer included, is 0 and either the payload
ends or the next frame's header reads. A video payload whose first chunk is not a key chunk is
refused. An audio chunk's samples are those its own headers state: an Opus packet's TOC byte,
a FLAC frame's block size, 1024 × an ADTS frame's raw data blocks.

### 2.4 The order of the questions

WebCodecs gives no smoothness or power answer, and Media Capabilities has no WebCodecs
decoding type. A sink that decodes through WebCodecs asks `isConfigSupported()` first, and
asks `decodingInfo()` of the `media-source` query of the same configuration only once
`isConfigSupported()` answers supported, and only for a codec string browser-sink §4.2's
registry gate passes: Media Capabilities is a further question about a decoder already
found (http-delivery §12.2). It reads `smooth` and `powerEfficient` from that answer as
hints and nothing more (§3).

### 2.5 Codecs the registry gate refuses

The header probes of [format-probes](format-probes.md) §14 to §17 read, from a payload of
`wai.audio.ac4`, `wai.audio.iamf` or `wai.audio.ivas`, the codecs parameter string its format's
binding defines (format-probes §2.6): `ac-4.` and three versions (format-probes §15.5), `iamf.`,
the two profiles and the codec's own elements (§16.7), and `sivs` (§17.6). No such string, and
no string for `wai.video.av2`, `wai.video.evc` or `wai.video.evc.baseline`, matches a row of
browser-sink §4.2's registry gate. A page that routes by browser-sink therefore forms no query
of §2.1 or §2.2 for these capabilities and asks no user-agent interface about them; their UA leg
is `codec_unregistered` (browser-sink §4.3 step 4.2), and each routes to a host or wasm decoder
the page holds, or nowhere. `wai.video.av2` has no string at all: its binding to ISO BMFF is a
working draft, and its probe gives `codec` `null` (format-probes §14.5). `wai.video.evc` has no
probe (format-probes §18).

A sink outside a web platform that asks its own platform whether it decodes one of these formats,
through an interface that takes an RFC 6381 codecs parameter, **MUST** read the string from the
payload with the probe, as §2 requires of every query, and **MUST NOT** ask where the probe gives
`null` or refuses the payload. A row the WebCodecs registry adds for one of these strings enters
browser-sink §4.2 by a revision of that extension, and the string's query enters §2.1 or §2.2
with it.

## 3. The answers

A page enters what the platform said in browser-sink §3's capability report, one answer per
codec string, `{supported, smooth, power_efficient, probe}`, and in no other shape
(`platform_decode::report_answer`):

- **`supported`** is `isConfigSupported()`'s answer, `VideoDecoder`'s for the preference the
  sink will configure or `AudioDecoder`'s, alone. A Media Capabilities `supported` is not read: a `media-source` answer is
  about the media-source pipeline, its container support included, not about the decoder
  WebCodecs found, so a `decodingInfo()` answer of `supported: false` overrides nothing.
  `supported: false` makes the user agent's leg unavailable for that object
  (browser-sink §4.3 step 4.5, `ua_unsupported`); the object then routes to a host or a
  wasm decoder where the page holds one, or nowhere. It is not a decode error and not a
  reason to try the decode anyway.
- **`smooth` and `power_efficient`** are `decodingInfo()`'s, entered only where
  `supported` is true, and `null` otherwise or where nothing was asked. They order the
  alternatives a page can decode, and only for an object routed `ua` (browser-sink §4.7): an
  object the page routes to a host or a wasm decoder, because the user agent's tier is below
  what the object needs (AV1's user-agent tier is `none`, browser-sink §4.1) or because no
  probe ran for a codec whose need is `decode-equivalence` (browser-sink §4.4), is not
  ranked on an answer about a decoder that will not run it. `powerEfficient` is the user
  agent's judgement that the decode's power draw is optimal, its definition left to the user
  agent, which **SHOULD NOT** mark hardware decoding power-efficient by default (Media
  Capabilities §2.2). It is a hint, not a measurement and not evidence of hardware. A sink
  **MUST NOT** write it, or anything derived from it, into a receipt, a measurement claim or
  an energy figure, and **MUST NOT** present it as an acquisition class
  ([energy-measurement](energy-measurement.md) §2).
- **`probe`** is `not_run` for the answers of this section: routing runs no probe
  (browser-sink §4.4). The probe of §6.2 decides only whether a grain-free AV1 decode is a
  refinement base, and changes no route.

**One answer a codec string.** The report keys answers by codec string, and two objects can
share one while their queries differ: two sizes of one AV1 configuration, an Opus stream of one
channel and another of two. A page that asks about several configurations under one codec
string enters one answer for it (`platform_decode::merge_answers`): `supported` only where
every configuration was answered supported; then `smooth` and `power_efficient` each `true`
where every answer's is `true`, `false` where any is `false`, and `null` otherwise. No object is
then routed to a decoder that refused its configuration, and none is ranked on a hint another
configuration earned.

**The hardware-acceleration preference.** `prefer-hardware` and `prefer-software` are
hints a user agent may ignore for any reason (WebCodecs §7.9); an audio decoder takes none. An `isConfigSupported()`
answer of `supported: true` with `prefer-hardware` says the platform accepts that
configuration with that preference; it is not evidence that the decode runs on hardware,
and a sink **MUST NOT** report it as such.

**Privacy.** The answers can be used to fingerprint the sink (Media Capabilities §3.2). A
sink **SHOULD NOT** send them, or the order they give, to any party; the order is computed
where the answers are. A platform that fakes its capabilities to resist fingerprinting
(§3.2 recommends a fixed set rather than always yes or always no) gives answers the order
uses as given: the result is a choice that platform made, and the decode still conforms.

## 4. The order the answers feed

This extension defines no order of its own, and no second meaning of "the sink advertises
`cap`". What a page advertises for an object is its route (browser-sink §4.3, SPEC §4).
Among alternatives the page can route, SPEC §4's energy order applies, as browser-sink §4.7
writes it: `first` for classical decode routed `ua` that the platform answers supported,
smooth and power-efficient, then `smooth`, then `classical` (any other routed classical
decode, and a learned one with a qualifying sink-metered figure in browser-sink §6's energy
entry), then `learned-without-budget`. A `WAI1` envelope's capability and fallback are two
such alternatives. The order reads no figure's value (browser-sink §6).

## 5. Conformance

The reference implementation is `wai::platform_decode` (`wai/wai-rs/src/platform_decode.rs`):
`query` forms §2's query, `chunks` §2.3's chunks, and `report_answer` and `merge_answers`
§3's answer. It calls no platform: the answers are inputs. The vectors are in
`wai/platform-decode-conformance/`: `queries.json` holds, for every capability §2 maps,
payloads written by real encoders and muxers, each with its query and the summary of its
chunks (the packaging corpus's floor objects, whose codec string is the one the packager wrote
into its HLS master; an HEVC SPS the platform's encoder wrote through WebCodecs, configured
with `prefer-hardware`; the elementary streams of the identify and encode corpora; and
`payloads/`, VP8, VP9 4:4:4 and 10-bit BT.2020 full range, AVC High with B-frames and a
cropped height, stereo AAC, 24-bit FLAC, stereo Ogg Opus and one Opus packet, with what an
independent media prober read of each in `payloads/prober.tsv`), and planted faults, a payload
with one edit and the outcome it gives. The order's vectors are the `choices` of
[`live-conformance/browser/`](../live-conformance/browser/).

An implementation conforms when it gives every query's members and every chunk summary, and
refuses every case the vectors refuse. The reference's tests (`platform_decode::tests`) also
hold the AV1 codec string to the AV1-ISOBMFF binding's own examples, derive every floor of the
packaging corpus to the codec string and size the packager wrote into the pack's HLS master,
read constructed HEVC and H.264 SPSs (sub-layers, scaling lists, field coding, 4:2:2 and
4:4:4, emulation-prevention bytes), VP9 key frames of every profile and colour space, VP8 key
frames, Ogg pages, Opus TOC bytes and ADTS headers, hold §2.1's VP9 level table at its edges,
split Annex B streams into access units and AV1 payloads into key and other temporal units,
and hold `report_answer` and `merge_answers` to §3. `live-conformance/browser_sink_verify.py`,
given `platform-decode-conformance/` as its second argument, is the second implementation:
written from this text, with its own readers of SPEC §5's AV1 framing, IVF, Annex B, Ogg, FLAC
and ADTS, and §2.1's tables read from the text, it recomputes every case and requires every
mutated copy of the vectors to fail.

`wai-web`'s `platform_decode` bundle binds the same code for a page (`platformQuery`,
`platformChunks`, `platformMapped`, `platformAnswer`, `platformMerge`, and `platformChoose`,
browser-sink §4.7's choice), and `wai-web/demo/platform_decode.html`
(`verify_platform_decode.mjs`) runs it against a browser's platform through the composing
sink's own calls (`compose.js` `uaAnswer` and `decodeUa`): every AV1 floor object of the
packaging corpus and every payload of the vectors that is not a planted fault, queried from
the wasm32 build, which must read each codec string and chunk count the vectors state, and,
where the platform accepts a configuration, decoded through WebCodecs from §2.3's chunks;
HEVC and AVC streams the platform's encoders write, queried from the stream and decoded back;
and the choice between a learned alternative and an AV1 floor object, from the report the
page built from those answers. Every query the page asks is read from a payload.

Measured 2026-10-22 with that page on an arm64 desktop (macOS 27.0.1), in headless
Chromium 148 and a Chromium-based browser at 155, with the same results in both, before the
page's order moved into browser-sink's routing:

- every AV1 floor object of the packaging corpus (64×36, `av01.0.00M.08`) decoded every
  temporal unit through WebCodecs. The platform answered `supported` and `smooth`, not
  `powerEfficient`, and did not accept `prefer-hardware` at that size;
- the platform's HEVC encoder, configured as `hvc1.1.6.L93.B0` for 640×360 with
  `prefer-hardware`, wrote an SPS with `general_level_idc` 63, and the codec string it
  declared for its output repeated the configured one. The query read `hev1.1.6.L63.B0` from
  the stream; the platform accepted that configuration with both preferences, answered
  `supported`, `smooth` and `powerEfficient`, and decoded every picture back with it.

Measured 2026-11-09 with that page and `verify_platform_decode.mjs` on an arm64 desktop
(macOS 27.2), in headless Chromium 148 and the WebKit build Playwright drives (26.4), over 65
payloads (the 27 of the vectors read as they are, and every AV1 floor object of the packaging
corpus):

- In both engines the wasm32 build read every codec string, chunk count and key-chunk count the
  vectors state.
- Chromium accepted every configuration and decoded every payload whole: AV1 (with
  `no-preference` and `prefer-software`; it did not accept `prefer-hardware` at 64×36), HEVC,
  AVC (Constrained Baseline, High with B-frames and a cropped height, the platform's and a
  software encoder's streams), VP9 profiles 0, 1 (4:4:4) and 2 (10-bit), VP8, Ogg Opus, one
  Opus packet, 16- and 24-bit FLAC, and AAC-LC. It answered `smooth` and not `powerEfficient`
  for AV1, VP9 and AVC at 64×36 and below, `powerEfficient` for AVC at 128×72 and 256×144, for
  HEVC and for every audio configuration, and neither for VP8.
- WebKit decoded VP9 profiles 0 and 2, VP8, every AVC and HEVC payload, both Opus forms and
  AAC-LC whole, and answered VP9 4:4:4 unsupported. It accepted AV1 at 64×36 with every
  preference, and FLAC, and its decoders refused every one of those decodes (an
  `EncodingError`), so the page met a refusal after `supported` (browser-sink §14.3: the next
  alternative is decoded); the composing page's AV1 floors, at 320×180, it decoded whole. It
  answered `smooth` and `powerEfficient` for AV1, VP9, AVC and HEVC, neither for VP8, Opus or
  AAC.
- The platforms' encoders, configured with `prefer-hardware`: Chromium's HEVC encoder wrote
  `general_level_idc` 63 where it was configured and declared `hvc1.1.6.L93.B0`, and its AVC
  encoder `level_idc` 30 where it was configured and declared `avc1.42001f`; WebKit's AVC
  encoder wrote the constraint byte `E0` (`avc1.42E01F`) under the same declaration, and its
  HEVC encoder wrote a stream with no start codes though asked for Annex B, which the query
  refuses. Every stream the query read decoded back with the configuration it gave.
- The choice: in Chromium the AV1 floor object ranked `smooth`, in WebKit `first`; the learned
  alternative without a figure ranked `learned-without-budget`, with the synthetic figure
  `classical`, and the AV1 floor was chosen in both engines both times.

`compose/run.mjs` played the composing package in both engines with the dispatch asking the
user agent about every mapped alternative the page does not decode in WebAssembly (its AV1
floors; its FLAC floors routed to WebAssembly, browser-sink §14.3), every check holding. Not
measured: a platform that fakes its answers, and any engine but these two.

## 6. Grain-free AV1: a refinement base the platform decodes

AV1's decoding process is normative: a conforming decoder's output frames are identical to
the decoding process's, unless the payload signals film grain, whose synthesis is held only
to a perceptual criterion (AV1 §7.2; SPEC §5). A stream that signals no film grain decodes to
the same frames on every conforming decoder, the platform's included, and those frames are a
base a residual refinement can be written against once and applied on every sink. This
section defines that base. The layers are `wai.video.int_refine` (SPEC §5, *Refinement
capabilities*; [staged-delivery](staged-delivery.md) §6.1); the base is the zeroth-condition
`wai.video.av1` rendition, which a sink without WAI's integer decoders presents as it is.

### 6.1 The base

A `wai.video.av1` payload is **grain-free** when every sequence header OBU it holds has
`film_grain_params_present` 0 (AV1 §5.5.1, the flag after `color_config()`), and then no
frame of it applies film grain synthesis. A sink reads the payload whole before it takes it
as a base (`staged::av1::read_base`), and refuses it, in this order, when:

1. it does not read as SPEC §5's framing: exactly `n` temporal units, each of OBUs with
   their size fields and their forbidden bit 0, the first holding a sequence header
   (`av1_malformed`);
2. any sequence header has `film_grain_params_present` 1 (`av1_film_grain`);
3. any sequence header is not 8-bit 4:2:0 with chroma: `BitDepth` 8, `mono_chrome` 0,
   `subsampling_x` and `subsampling_y` 1 (`av1_format`);
4. its frames pass the residual form's caps, `w·h·3 ≤ 2^26` for one and `n·w·h·3 ≤ 2^30`
   for all (staged-delivery Appendix A.3 step 6) (`av1_too_large`).

A refused payload is not a base. The sink still presents it as SPEC §4 dispatches it; it holds
no stage set for it, so a layer naming it is held for a base and discarded at the hold bound
(`base-absent`, staged-delivery §4.3). A base's frames are what a decoder output for it, one
a temporal unit, each `w × h` as the framing states (`av1_frames` otherwise), read as §6.2's
samples.

### 6.2 The probe

A platform's answer of `supported` says its decoder accepts the configuration; it does not
say the decoder is conforming. A page shows that it is with a **probe**: a grain-free stream
of the same configuration (the codec string of §2 and the coded size) whose decoded frames'
SHA-256 digests are published. The digest of a frame is over its samples as WebCodecs
`copyTo()` writes them with no `format`, `rect` or `layout` option: `I420`, the luma plane,
then Cb, then Cr, rows tightly packed, the chroma planes `⌈w/2⌉ × ⌈h/2⌉`; a frame output as
`NV12` is compared after its interleaved chroma plane is split into its two planes, which
reorders samples and changes none (browser-sink §4.4). The probe is `match` when the decoder
output as many frames as the probe has and every frame's digest is the published one,
`mismatch` otherwise, and `not_run` when the page decoded none.

The base step of a grain-free AV1 base ([staged-delivery](staged-delivery.md) §5) holds
`decode-equivalence` when its frames are the reference decode's, or a platform decoder's whose
probe for that configuration is `match`; otherwise it holds no tier. SPEC §5's *Base
admission* admits `wai.video.int_refine` over `wai.video.av1` only where the base step holds
`decode-equivalence`, so a layer over a platform decode with no matching probe is refused
`base-not-admitted`. The probe never changes a route (browser-sink §4.3): a page decodes the
base where its routing sends it, and the probe decides only whether the frames are a base.

### 6.3 The units

The residual form refines `RGB8` frames (staged-delivery §6.1). A grain-free AV1 base's unit
is its frame converted by the JFIF conversion (ITU-T T.871) in 16-bit fixed point, each chroma
sample covering the 2×2 luma samples it sits over. With `Cb' = Cb − 128` and `Cr' = Cr − 128`:

```text
R = clamp((Y·65536 + 91881·Cr' + 32768) >> 16)
G = clamp((Y·65536 − 22554·Cb' − 46802·Cr' + 32768) >> 16)
B = clamp((Y·65536 + 116130·Cb' + 32768) >> 16)
```

for the luma sample `(i, j)` and the chroma sample `(i >> 1, j >> 1)`, where `>>` is the
arithmetic shift (floor division), every intermediate fits in 32 signed bits, and the clamp
is to `[0, 255]`. The conversion is part of the base's definition, not colour management: a
producer writes its layers against the units this conversion gives from the reference decode,
and a sink computes the same units from the same samples, natively or in a page. A sink that
presents a refined frame converts it for display as it would any `RGB8` picture.

### 6.4 Conformance

The corpus is [`platform-decode-conformance/grain-free-av1/`](../platform-decode-conformance/grain-free-av1/):
a grain-free stream (`base.wai`, 160×96, eight frames, `av01.0.31M.08`), its reference decode
(`base.i420`) and probe (`probe.json`), two `wai.video.int_refine` layers written against that
decode toward the source the stream was encoded from (`layer1.wai`, step 6; `layer2.wai`, step
1, which reaches the source), a stream with film grain (`base-grain.wai`) and a layer naming
it, and `cases.json`: the base alone; one layer and two layers over a decode whose probe
matched; two layers over the reference decode; the second layer offered first; one layer over
a decode whose probe did not run; and the stream with film grain. Each case states every
report, every unit's stage, producer, tier and stage hash, and the clip hash.

- `wai_av1_refine_vectors verify` (feature `av1_refine_vectors`) decodes `base.wai` through
  libdav1d and requires `base.i420` byte for byte, refuses `base-grain.wai` before decoding it,
  and regenerates every other file but the two streams byte for byte. `tests/av1_refine.rs`
  replays every case through `staged::StageSink`, finds the one bit that sets the grain flag
  of `base.wai` and requires the stream so changed to be refused, and requires a frame with
  one sample changed to change that frame's stage hash; with feature `av1_base` it also
  decodes through libdav1d.
- `verify.py`, beside the corpus, is a second implementation: its own reader of the framing,
  the OBUs and the sequence header, its own codec string and conversion, and the sink of
  `staged-conformance/staged_verify.py`; it replays every case, requires the mutations above
  to fail, and with `--dav1d` decodes the stream with the dav1d tool and requires
  `base.i420`.
- `wai-web`'s `StageBase.fromAv1`, `av1Base` and `av1Probe` bind the same Rust for a page;
  `demo/verify_av1_refine_wasm.mjs` replays every case through the wasm32 build from
  `base.i420`, and `demo/av1_refine.html` (`verify_av1_refine.mjs`) decodes `base.wai`
  through the browser's WebCodecs `VideoDecoder` with each hardware-acceleration preference
  the platform accepts, puts every decode to the probe, and replays every case from the
  frames of the decode configured with `no-preference`.

Measured 2026-11-09. libdav1d's decode of `base.wai`, through the library on an arm64 desktop
(macOS 27.2) and through its command-line tool, is `base.i420` byte for byte; `verify.py`, the
wasm32 replay and `tests/av1_refine.rs` reproduce every case on that host, and `verify.py` and
the test on an x86_64 Linux host. In headless Chromium 156 on the same desktop, the platform
answered `supported` for `no-preference` and `prefer-software` and not for `prefer-hardware`
at 160×96; both decodes output eight `I420` frames of 160×96, every one with its probe
digest, and every case reproduced from the `no-preference` frames, the stage hashes and clip
hashes the native sink gives.

In the WebKit build Playwright drives (27.2), on the same desktop, the platform answered
`supported` for all three preferences in one run; each decode output eight `NV12` frames of
160×96, every one with its probe digest once its chroma plane was split, and every case
reproduced from the `no-preference` frames. In runs before it, on the same host and with the
same build, it answered `isConfigSupported()` false for every AV1 configuration asked (four
levels, three sizes, two preferences) while it answered true for VP9, AVC and HEVC, and in a
run after it false again: a page that is told so routes the object elsewhere
(`ua_unsupported`) and takes no base from it. Which decoder ran is not known; an answer to a preference is not evidence of
hardware (§3). The WPE build on an arm64 Linux host answered `supported` and failed every
decode: each AV1 decoder its GStreamer offered, two hardware and one software, refused the
caps the engine gave the stream (an OBU stream of 160×96 with `alignment` `frame`), and no
other AV1 decoder was installed.

## 7. Security and privacy considerations

The answers are fingerprinting surface (§3, *Privacy*). The order reads them where they
are produced and sends nothing. The query reads only the payload's headers; a malformed
payload gives no query, and the entry is decoded, or not, as it would be without this
extension. An encoder cannot raise an entry's rank by what it declares: the codec string
is read from the payload, the class from the registry, the route from browser-sink's
routing, and a learned entry ranks with classical decode only on a sink-metered figure the
sink holds, never on one an envelope carries.

A page cannot raise a platform decode to a refinement base by what it states: the probe's
outcome is computed from the frames the decoder gave and the published digests, the grain
flag is read from every sequence header of the payload, and the units are converted from the
frames by §6.3. A stream that signals film grain anywhere is refused as a base before it is
decoded. The probe's digests are of public decoded frames and reveal nothing of the page.