b5b7bd3686
This mapping was written on 2026-08-03 and sat unmerged while `extensions/capability-constraints.md` landed, which makes dispatch a precondition check: a sink satisfies a capability when it advertises `cap` AND the object's declared parameters satisfy a registered constraint set. Rebased onto current main, the two extensions read as overlapping claims about §4 with nothing saying how they relate. They are not in tension, and the newer one makes this one sharper. `isConfigSupported()` is not a codec-string test — it takes a whole `VideoDecoderConfig`, codec string plus `codedWidth`, `codedHeight` and the rest — so it is a constraint evaluator already, and a browser satisfies the constraint extension's three requirements structurally rather than through extra machinery: the query is pre-flight, so evaluation happens before dispatch rather than during it; it answers for a configuration, so the answer is per object and not per sink; and `supported: false` is a negative answer returned before any payload is committed to, which is the required treatment of an object that satisfies no set. The call also returns the normalised config the platform would accept, which is the accepted parameter set, so a sink SHOULD use it rather than re-derive one. The limit is stated rather than glossed. WebCodecs' parameter vocabulary is the conventional-media one — codec, resolution, frame rate, bit depth, profile — which is the same shape as the ratified registers `capability-constraints.md` §3 draws on, and it has the same gap: no WebCodecs configuration describes the neural, quantum, world or haptic families. Those have no native tier to constrain at all, resolve through the WASM tier, and stay unconstrained until WAI authors their register entries. That is the state §3 already describes, not a shortfall introduced here. Framing renumbered §5 → §6. Docs only; `check-no-vanity-metrics.sh` passes. Co-Authored-By: Claude Opus 5 <[email protected]>
6.9 KiB
WAI Extension: Browser Capability Negotiation (WebCodecs)
Status: Draft. How a browser sink implements the §4 dispatch algorithm's "if the sink advertises
cap" test — using the platform's own capability- negotiation primitive (WebCodecsisConfigSupported()) for the native tier, and a WebAssembly decoder as the always-satisfiable fallback. Informative: it maps §4 onto the browser; it does not change the wire format or the algorithm. Keywords MUST, MUST NOT, SHOULD, MAY are RFC 2119/8174.
1. The dispatch algorithm already is the browser's pattern
§4 resolves a capability by asking whether the sink advertises it, else falling
back. That is the exact shape of the browser's media-capability primitive:
WebCodecs VideoDecoder.isConfigSupported() / AudioDecoder.isConfigSupported()
(and ImageDecoder) answer, at runtime, whether a codec configuration will decode
on this device — and the established pattern when the answer is no is to fall
back to a WebAssembly decoder. A WAI browser sink is that pattern with a named
capability instead of a bare codec string, and a guaranteed WASM floor.
So a browser sink resolves §4 in two tiers:
- Native tier — the platform decodes the capability's payload with a built-in
codec, gated by
isConfigSupported(). - WASM tier — WAI's own pure-integer decoders, compiled to wasm32, decode the payload with no platform codec and no ML runtime. This tier is always satisfiable: it depends on nothing the device must ship.
2. Native tier — isConfigSupported() as "advertises cap"
For the floor and recommended-menu capabilities (§5) that correspond to a codec
the browser may implement, a sink SHOULD treat "advertises cap" as "the platform
reports the codec supported." It builds the WebCodecs config for the capability's
payload and calls isConfigSupported(config); a supported: true result means the
sink advertises that capability and decodes via the platform decoder.
| WAI capability | browser decode path (when isConfigSupported is true) |
|---|---|
wai.image.png / wai.image.jpeg / wai.image.jxl / wai.image.avif |
ImageDecoder (or <img>/createImageBitmap) with the format's MIME type |
wai.video.av1 / wai.video.av1.lossless |
VideoDecoder with the av01.* codec string |
wai.audio.opus / wai.audio.flac |
AudioDecoder with the opus / flac codec string |
A false result (or a browser without WebCodecs) means the sink does not
advertise the capability at the native tier; resolution proceeds to the WASM tier,
then to §4's fallback, then to inert.
3. WASM tier — the always-satisfiable floor and the neural path
The neural and computed capabilities (wai.image.jpegai, wai.neural.int_*,
wai.audio.int_codec, wai.splat.int_codec, the wai.world/film/quantum/...
family) have no native browser codec by design — the decoder is named, not
shipped. For these the WASM tier is the decode path: WAI's pure-integer decoders
run in wasm32 exactly as they run natively. Because the integer decoders satisfy
decode-equivalence (see determinism-tiers), the WASM
decode is byte-identical to the native one — the reconstruction has a portable hash
in the browser too. The live /jpegai and /int_hyper sinks are this tier.
The WASM tier also backs the classical floor (§5): a sink MAY carry wasm
decoders for PNG / FLAC / zstd so that even a browser whose isConfigSupported()
declines every native codec can still open some file in each media class. This is
the browser realization of the floor's guarantee — always-decodable, depending on
nothing the platform must provide.
4. Resolution order in a browser sink
For cap = model_requirement.capability, a browser sink SHOULD resolve:
- Native, if a WebCodecs mapping exists for
capandisConfigSupported()reports it supported → decode with the platform decoder. - WASM, if the sink bundles a wasm decoder for
cap→ decode in wasm. - Fallback
model_requirement.fallback, resolved by the same two tiers. - Otherwise inert at this sink (§4 step 6).
This never contradicts §4 — it is §4's "advertises cap" made precise for the
browser: native when the platform reports it, WASM otherwise, and the classical
floor always reachable through wasm.
5. The native tier is already a constraint check
capability-constraints.md makes dispatch a
precondition check: a sink satisfies a capability when it advertises cap and
the object's declared parameters satisfy one of the capability's registered
constraint sets. That extension landed after this mapping was written, and it
sharpens it rather than competing with it.
isConfigSupported() is not a codec-string test. It takes a full configuration —
for video, the codec string plus codedWidth, codedHeight, the description, and
the other members of VideoDecoderConfig — and answers for that configuration.
It is therefore a constraint evaluator already, and the browser satisfies the
extension's requirements structurally rather than by additional machinery:
- Evaluated before dispatch.
isConfigSupported()is a pre-flight query; it does not construct or configure a decoder. This is the "move the failure earlier" requirement, met by the API's own shape. - Answered per object, not per sink. A sink that decodes one object of a capability and refuses another is expressing exactly what a constraint set expresses. The parameters come from the object.
- Unsupported, not a decode error. A
supported: falseresult is a negative answer to a question, returned before any payload is committed to — which is the required treatment where an object satisfies no set.
The result also carries a config back: the normalised configuration the platform
would accept. A browser sink SHOULD treat that returned configuration as the
accepted parameter set for the object, rather than re-deriving it.
The honest limit. WebCodecs' parameter vocabulary is the conventional-media
one — codec, resolution, frame rate, bit depth, profile. That is the same shape as
the ratified registers capability-constraints.md §3 draws on, and it has the same
gap: there is no WebCodecs configuration for the neural, quantum, world or haptic
families. Those capabilities have no native tier to constrain at all. They resolve
through the WASM tier (§3) and remain unconstrained until WAI authors their
register entries, which is the state §3 of that extension describes rather than a
shortfall of this mapping.
6. Framing
WAI's capability dispatch is not a bespoke negotiation scheme; it is the pattern the browser's own media stack already uses — ask whether a config is supported, fall back to WebAssembly when it is not. WAI adds a named capability and a guaranteed WASM floor, so the fallback is always defined rather than device-dependent. Stated as a property of the standard, not a comparison.