# WAI Extension: Platform Decode Capabilities

> Status: Draft. Maps two web platform interfaces, Media Capabilities `decodingInfo()`
> and WebCodecs `isConfigSupported()` with its hardware-acceleration preference, onto the
> capability report of [browser-sink](browser-sink.md) §3: what a sink asks the platform
> about an entry, read from the entry's payload (§2), and how the answers enter the report
> (§3). The answers are hints. They order the alternatives a page can decode, inside
> browser-sink's routing, by SPEC §4's energy order (browser-sink §4.7); they never change a
> route, are never an energy figure and are never evidence of hardware. No manifest member
> and no wire format.
> Keywords **MUST**, **MUST NOT**, **SHOULD**, **MAY** are RFC 2119/8174.

## 1. Scope

A sink running in a web platform can decode a standard video codec through the
platform's own decoders, which may be fixed-function hardware, or through a decoder of its
own: WebAssembly, or a learned reconstruction such as WAI's integer codecs. The platform
says, for a configuration, whether it can decode it, whether it can do so without
dropping frames, and whether it judges the decode power-efficient (Media Capabilities
§2.2). A learned reconstruction runs on the sink's CPU, GPU or neural engine, and what it
draws there is known only where it has been measured. This extension says how a WAI sink
asks the platform about an entry, and how it enters what the platform says in browser-sink's
capability report, where the routing (browser-sink §4) decides what the page can decode and
the energy order (SPEC §4, *Energy order*; browser-sink §4.7) orders what it can.

It reads these documents:

- Media Capabilities, W3C Working Draft, 18 October 2026: `MediaDecodingConfiguration`,
  `VideoConfiguration`, `MediaCapabilitiesInfo` (§2.1, §2.2), the decoding-info algorithm
  (§2.3.2), and the privacy considerations (§3);
- WebCodecs, W3C Working Draft, 19 October 2026: `VideoDecoderConfig` (§7.6),
  `isConfigSupported()` and Check Configuration Support (§4, §7.1), and
  `HardwareAcceleration` (§7.9);
- the WebCodecs AV1 codec registration (Group Note Draft, 13 October 2026) and HEVC codec
  registration (Group Note Draft, 8 June 2026), which define the codec strings by the AV1
  ISOBMFF binding (AV1 Codec ISO Media File Format Binding v1.3.0, §5) and by ISO/IEC
  14496-15 Annex E.3.

ISO/IEC 14496-15 was not read. The HEVC codec string §2 writes follows the form the HEVC
registration describes (four dot-separated fields of Annex E.3 and the constraint bytes
after them), checked against an open-source parser of that form (Chromium's
`ParseHEVCCodecId`) and against a platform that decodes with it (§5).

## 2. The query

For an entry whose capability the table maps, the sink forms the query from the entry's
payload. It **MUST NOT** form it from anything the envelope declares apart from the
payload: the codec string a payload's encoder declared can differ from the stream
(§5 records one that did).

| capability | codec string | size | frame rate | bit rate |
|---|---|---|---|---|
| `wai.video.av1`, `wai.video.av1.lossless` | from the first temporal unit's sequence header OBU: `av01.`, the profile, the level (two digits) and the tier (`M` or `H`), the bit depth (two digits); then the monochrome flag, the chroma subsampling (three digits), the colour primaries, transfer characteristics and matrix coefficients (two digits each, 01 each where the sequence header carries no colour description), and the full-range flag; the part after the bit depth omitted where it is exactly `.0.110.01.01.01.0` (AV1-ISOBMFF §5) | `w` × `h` of SPEC §5's AV1 framing | `fps_num` / `fps_den` of the framing | the payload's bits × the frame rate ÷ `n`, rounded up |
| `wai.video.hevc` | from the first SPS NAL unit (H.265 §7.3.2.2, §7.3.3): `hev1.`; the profile space as nothing, `A`, `B` or `C`, then `general_profile_idc` in decimal; the 32 `general_profile_compatibility_flag`s as one hexadecimal value with flag *j* at bit *j*; `L` or `H` for `general_tier_flag`, then `general_level_idc` in decimal; then the six bytes from `general_progressive_source_flag` on, each in hexadecimal, trailing zero bytes omitted. `hev1` because an Annex B stream carries its parameter sets in band | `pic_width_in_luma_samples` × `pic_height_in_luma_samples`, less the conformance window; the coded size before it | none in the payload: the rate the sink will present at, where it has one | the payload's bits × the frame rate ÷ the pictures the stream holds, rounded up |

A capability the table does not map is not queried: the sink decodes it, or does not, as
it otherwise would, and the order of §4 places it by its class alone.

The sink asks:

- `navigator.mediaCapabilities.decodingInfo()` with a `MediaDecodingConfiguration` of
  `type` `"media-source"` and a `video` member: `contentType`
  `video/mp4; codecs="<codec string>"`, the size, the bit rate and the frame rate. Where
  the query has no frame rate it does not ask: `framerate` and `bitrate` are required
  members, and a sink **MUST NOT** invent them.
- `VideoDecoder.isConfigSupported()` with a `VideoDecoderConfig` of the codec string, the
  coded size as `codedWidth` and `codedHeight`, and the `hardwareAcceleration` the sink
  will configure, when it decodes the entry through WebCodecs.

WebCodecs gives no smoothness or power answer, and Media Capabilities has no WebCodecs
decoding type. A sink that decodes through WebCodecs asks `isConfigSupported()` first, and
asks `decodingInfo()` of the `media-source` query of the same configuration only once
`isConfigSupported()` answers supported, and only for a codec string browser-sink §4.2's
registry gate passes: Media Capabilities is a further question about a decoder already
found (http-delivery §12.2). It reads `smooth` and `powerEfficient` from that answer as
hints and nothing more (§3).

## 3. The answers

A page enters what the platform said in browser-sink §3's capability report, one answer per
codec string, `{supported, smooth, power_efficient, probe}`, and in no other shape
(`platform_decode::report_answer`):

- **`supported`** is `isConfigSupported()`'s answer for the preference the sink will
  configure, alone. A Media Capabilities `supported` is not read: a `media-source` answer is
  about the media-source pipeline, its container support included, not about the decoder
  WebCodecs found, so a `decodingInfo()` answer of `supported: false` overrides nothing.
  `supported: false` makes the user agent's leg unavailable for that object
  (browser-sink §4.3 step 4.5, `ua_unsupported`); the object then routes to a host or a
  wasm decoder where the page holds one, or nowhere. It is not a decode error and not a
  reason to try the decode anyway.
- **`smooth` and `power_efficient`** are `decodingInfo()`'s, entered only where
  `supported` is true, and `null` otherwise or where nothing was asked. They order the
  alternatives a page can decode, and only for an object routed `ua` (browser-sink §4.7): an
  object the page routes to a host or a wasm decoder, because the user agent's tier is below
  what the object needs (AV1's user-agent tier is `none`, browser-sink §4.1) or because no
  probe ran for a codec whose need is `decode-equivalence` (browser-sink §4.4), is not
  ranked on an answer about a decoder that will not run it. `powerEfficient` is the user
  agent's judgement that the decode's power draw is optimal, its definition left to the user
  agent, which **SHOULD NOT** mark hardware decoding power-efficient by default (Media
  Capabilities §2.2). It is a hint, not a measurement and not evidence of hardware. A sink
  **MUST NOT** write it, or anything derived from it, into a receipt, a measurement claim or
  an energy figure, and **MUST NOT** present it as an acquisition class
  ([energy-measurement](energy-measurement.md) §2).
- **`probe`** is `not_run`: this extension runs no probe (browser-sink §4.4).

**The hardware-acceleration preference.** `prefer-hardware` and `prefer-software` are
hints a user agent may ignore for any reason (WebCodecs §7.9). An `isConfigSupported()`
answer of `supported: true` with `prefer-hardware` says the platform accepts that
configuration with that preference; it is not evidence that the decode runs on hardware,
and a sink **MUST NOT** report it as such.

**Privacy.** The answers can be used to fingerprint the sink (Media Capabilities §3.2). A
sink **SHOULD NOT** send them, or the order they give, to any party; the order is computed
where the answers are. A platform that fakes its capabilities to resist fingerprinting
(§3.2 recommends a fixed set rather than always yes or always no) gives answers the order
uses as given: the result is a choice that platform made, and the decode still conforms.

## 4. The order the answers feed

This extension defines no order of its own, and no second meaning of "the sink advertises
`cap`". What a page advertises for an object is its route (browser-sink §4.3, SPEC §4).
Among alternatives the page can route, SPEC §4's energy order applies, as browser-sink §4.7
writes it: `first` for classical decode routed `ua` that the platform answers supported,
smooth and power-efficient, then `smooth`, then `classical` (any other routed classical
decode, and a learned one with a qualifying sink-metered figure in browser-sink §6's energy
entry), then `learned-without-budget`. A `WAI1` envelope's capability and fallback are two
such alternatives. The order reads no figure's value (browser-sink §6).

## 5. Conformance

The reference implementation is `wai::platform_decode` (`wai/wai-rs/src/platform_decode.rs`):
`query` forms §2's query, and `report_answer` §3's answer. It calls no platform: the
answers are inputs. The vectors are in `wai/platform-decode-conformance/`: `queries.json`,
two floor objects of the packaging corpus, whose codec string is the one the packager wrote
into its HLS master, and an HEVC SPS the platform's encoder wrote through WebCodecs,
configured with `prefer-hardware`. The order's vectors are the `choices` of
[`live-conformance/browser/`](../live-conformance/browser/).

An implementation conforms when it gives every query's members. The reference's tests
(`platform_decode::tests`) also hold the AV1 codec string to the AV1-ISOBMFF binding's own
examples, derive every floor of the packaging corpus to the codec string and size the
packager wrote into the pack's HLS master, read constructed SPSs with sub-layers and with
emulation-prevention bytes, and hold `report_answer` to §3.

`wai-web`'s `platform_decode` bundle binds the same code for a page (`platformQuery`,
`platformAnswer`, and `platformChoose`, browser-sink §4.7's choice), and
`wai-web/demo/platform_decode.html` (`verify_platform_decode.mjs`) runs it against a
browser's platform: every AV1 floor object of the packaging corpus, queried and decoded
through WebCodecs with the configuration its payload's query gives; an HEVC stream the
platform's encoder writes, queried from that stream and decoded back; and the choice between
a learned alternative and an AV1 floor object, from the report the page built from those
answers. Every query the page asks is read from a payload.

Measured 2026-10-22 with that page on an arm64 desktop (macOS 27.0.1), in headless
Chromium 148 and a Chromium-based browser at 155, with the same results in both, before the
page's order moved into browser-sink's routing:

- every AV1 floor object of the packaging corpus (64×36, `av01.0.00M.08`) decoded every
  temporal unit through WebCodecs. The platform answered `supported` and `smooth`, not
  `powerEfficient`, and did not accept `prefer-hardware` at that size;
- the platform's HEVC encoder, configured as `hvc1.1.6.L93.B0` for 640×360 with
  `prefer-hardware`, wrote an SPS with `general_level_idc` 63, and the codec string it
  declared for its output repeated the configured one. The query read `hev1.1.6.L63.B0` from
  the stream; the platform accepted that configuration with both preferences, answered
  `supported`, `smooth` and `powerEfficient`, and decoded every picture back with it.

The choice step of the page was rewritten with the order and has not been run in a browser
in this revision; a 1920×1080 answer the earlier page reported was for a configuration
written by hand, not read from a payload, and is not restated. Not measured: a platform
without hardware AV1 or HEVC decode, a platform that fakes its answers, and any engine but
Chromium's.

## 6. Security and privacy considerations

The answers are fingerprinting surface (§3, *Privacy*). The order reads them where they
are produced and sends nothing. The query reads only the payload's headers; a malformed
payload gives no query, and the entry is decoded, or not, as it would be without this
extension. An encoder cannot raise an entry's rank by what it declares: the codec string
is read from the payload, the class from the registry, the route from browser-sink's
routing, and a learned entry ranks with classical decode only on a sink-metered figure the
sink holds, never on one an envelope carries.