From ba199e7e9c5ddcb332ec4a1944fc894d10478658 Mon Sep 17 00:00:00 2001 From: Transaction Science Date: Mon, 3 Aug 2026 06:09:25 -0400 Subject: [PATCH 1/2] wai: ingest the browser capability-negotiation mapping (WebCodecs) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit SOTA ingest: WAI's §4 dispatch — declare a capability, decode it if advertised, else fall back — is the exact shape of the browser's own media-capability primitive, WebCodecs isConfigSupported(), whose established fallback when a config isn't supported is a WebAssembly decoder. That is WAI's two-tier design already. New extensions/webcodecs-negotiation.md formalizes it: a browser sink answers §4's "advertises cap" with isConfigSupported() for the native tier (mapping the floor + menu codecs to VideoDecoder/AudioDecoder/ImageDecoder), and WAI's pure-integer wasm decoders as the always-satisfiable fallback — the tier the neural/computed capabilities (wai.image.jpegai, wai.neural.int_*, …) live in, decoding byte-exact in-browser (decode-equivalence), and where the classical floor stays always reachable. It documents what the live /jpegai and /int_hyper sinks already do. Referenced from SPEC §4; changelog entry. Informative, self-referential — WAI adds a named capability and a guaranteed wasm floor to the browser's own pattern. Co-Authored-By: Claude Opus 4.8 --- wai/CHANGELOG.md | 10 +++ wai/SPEC.md | 7 +++ wai/extensions/webcodecs-negotiation.md | 83 +++++++++++++++++++++++++ 3 files changed, 100 insertions(+) create mode 100644 wai/extensions/webcodecs-negotiation.md diff --git a/wai/CHANGELOG.md b/wai/CHANGELOG.md index 864eb7ef7..fc3f9b833 100644 --- a/wai/CHANGELOG.md +++ b/wai/CHANGELOG.md @@ -19,6 +19,16 @@ wire format may change and such changes are recorded here. the tier WAI adds on top and the precondition for a hard reconstruction binding. Referenced from SPEC §7 (Replicate-soundness); framed self-referentially. +### Browser capability negotiation — WebCodecs mapping + +- New [`extensions/webcodecs-negotiation.md`]: how a browser sink implements §4's + "advertises `cap`" test — the platform's `isConfigSupported()` for the native tier, + a WebAssembly decoder as the always-satisfiable fallback. Grounds WAI's dispatch + in the browser's own capability-negotiation pattern (ask if a config is supported; + fall back to wasm) and documents what the live wasm sinks already do: native codec + when the platform reports it, WAI's pure-integer wasm decoders otherwise, the + classical floor always reachable. Referenced from SPEC §4. + ### Reproducible-reconstruction binding — a C2PA/JPEG-Trust hard binding over pixels - New `wai.*` provenance capability: `reconstruction_binding` (feature diff --git a/wai/SPEC.md b/wai/SPEC.md index e44dc8b9b..0988924c2 100644 --- a/wai/SPEC.md +++ b/wai/SPEC.md @@ -126,6 +126,13 @@ If the manifest declares `security.encryption` other than `"none"` (§6), the sink MUST recover the plaintext payload before step 3 and run this algorithm on the plaintext; see §6.4. +In a **browser** sink, "the sink advertises `cap`" (step 3) is answered by the +platform's own capability primitive — WebCodecs `isConfigSupported()` — with a +WebAssembly decoder as the always-satisfiable fallback tier (native when the +platform reports the codec supported, WASM otherwise; the classical floor is always +reachable through wasm). See +[`extensions/webcodecs-negotiation.md`](extensions/webcodecs-negotiation.md). + --- ## 5. Registered capabilities (the zeroth menu) diff --git a/wai/extensions/webcodecs-negotiation.md b/wai/extensions/webcodecs-negotiation.md new file mode 100644 index 000000000..e14e3fd14 --- /dev/null +++ b/wai/extensions/webcodecs-negotiation.md @@ -0,0 +1,83 @@ +# WAI Extension: Browser Capability Negotiation (WebCodecs) + +> Status: Draft. How a **browser** sink implements the §4 dispatch algorithm's +> "if the sink advertises `cap`" test — using the platform's own capability- +> negotiation primitive (WebCodecs `isConfigSupported()`) for the native tier, and +> a WebAssembly decoder as the always-satisfiable fallback. Informative: it maps §4 +> onto the browser; it does not change the wire format or the algorithm. +> Keywords **MUST**, **MUST NOT**, **SHOULD**, **MAY** are RFC 2119/8174. + +## 1. The dispatch algorithm already is the browser's pattern + +§4 resolves a capability by asking whether the sink *advertises* it, else falling +back. That is the exact shape of the browser's media-capability primitive: +WebCodecs `VideoDecoder.isConfigSupported()` / `AudioDecoder.isConfigSupported()` +(and `ImageDecoder`) answer, at runtime, whether a codec configuration will decode +on this device — and the established pattern when the answer is *no* is to fall +back to a WebAssembly decoder. A WAI browser sink is that pattern with a named +capability instead of a bare codec string, and a guaranteed WASM floor. + +So a browser sink resolves §4 in **two tiers**: + +- **Native tier** — the platform decodes the capability's payload with a built-in + codec, gated by `isConfigSupported()`. +- **WASM tier** — WAI's own pure-integer decoders, compiled to wasm32, decode the + payload with no platform codec and no ML runtime. This tier is *always + satisfiable*: it depends on nothing the device must ship. + +## 2. Native tier — `isConfigSupported()` as "advertises `cap`" + +For the floor and recommended-menu capabilities (§5) that correspond to a codec +the browser may implement, a sink SHOULD treat "advertises `cap`" as "the platform +reports the codec supported." It builds the WebCodecs config for the capability's +payload and calls `isConfigSupported(config)`; a `supported: true` result means the +sink advertises that capability and decodes via the platform decoder. + +| WAI capability | browser decode path (when `isConfigSupported` is true) | +|---|---| +| `wai.image.png` / `wai.image.jpeg` / `wai.image.jxl` / `wai.image.avif` | `ImageDecoder` (or ``/`createImageBitmap`) with the format's MIME type | +| `wai.video.av1` / `wai.video.av1.lossless` | `VideoDecoder` with the `av01.*` codec string | +| `wai.audio.opus` / `wai.audio.flac` | `AudioDecoder` with the `opus` / `flac` codec string | + +A `false` result (or a browser without WebCodecs) means the sink does **not** +advertise the capability at the native tier; resolution proceeds to the WASM tier, +then to §4's `fallback`, then to inert. + +## 3. WASM tier — the always-satisfiable floor and the neural path + +The neural and computed capabilities (`wai.image.jpegai`, `wai.neural.int_*`, +`wai.audio.int_codec`, `wai.splat.int_codec`, the `wai.world/film/quantum/...` +family) have **no** native browser codec by design — the decoder is *named, not +shipped*. For these the WASM tier is the decode path: WAI's pure-integer decoders +run in wasm32 exactly as they run natively. Because the integer decoders satisfy +`decode-equivalence` (see [determinism-tiers](determinism-tiers.md)), the WASM +decode is byte-identical to the native one — the reconstruction has a portable hash +in the browser too. The live `/jpegai` and `/int_hyper` sinks are this tier. + +The WASM tier also backs the **classical floor** (§5): a sink MAY carry wasm +decoders for PNG / FLAC / zstd so that even a browser whose `isConfigSupported()` +declines every native codec can still open *some* file in each media class. This is +the browser realization of the floor's guarantee — always-decodable, depending on +nothing the platform must provide. + +## 4. Resolution order in a browser sink + +For `cap = model_requirement.capability`, a browser sink SHOULD resolve: + +1. **Native**, if a WebCodecs mapping exists for `cap` and + `isConfigSupported()` reports it supported → decode with the platform decoder. +2. **WASM**, if the sink bundles a wasm decoder for `cap` → decode in wasm. +3. **Fallback** `model_requirement.fallback`, resolved by the same two tiers. +4. Otherwise **inert at this sink** (§4 step 6). + +This never contradicts §4 — it is §4's "advertises `cap`" made precise for the +browser: native when the platform reports it, WASM otherwise, and the classical +floor always reachable through wasm. + +## 5. Framing + +WAI's capability dispatch is not a bespoke negotiation scheme; it is the pattern +the browser's own media stack already uses — ask whether a config is supported, +fall back to WebAssembly when it is not. WAI adds a *named* capability and a +guaranteed WASM floor, so the fallback is always defined rather than +device-dependent. Stated as a property of the standard, not a comparison. -- 2.54.0 (Apple Git-157) From b5b7bd3686faf4dafc3f29ddda7076b6738be10a Mon Sep 17 00:00:00 2001 From: Transaction Science Date: Mon, 21 Sep 2026 13:49:10 -0400 Subject: [PATCH 2/2] wai: reconcile the WebCodecs mapping with capability constraints MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit This mapping was written on 2026-08-03 and sat unmerged while `extensions/capability-constraints.md` landed, which makes dispatch a precondition check: a sink satisfies a capability when it advertises `cap` AND the object's declared parameters satisfy a registered constraint set. Rebased onto current main, the two extensions read as overlapping claims about §4 with nothing saying how they relate. They are not in tension, and the newer one makes this one sharper. `isConfigSupported()` is not a codec-string test — it takes a whole `VideoDecoderConfig`, codec string plus `codedWidth`, `codedHeight` and the rest — so it is a constraint evaluator already, and a browser satisfies the constraint extension's three requirements structurally rather than through extra machinery: the query is pre-flight, so evaluation happens before dispatch rather than during it; it answers for a configuration, so the answer is per object and not per sink; and `supported: false` is a negative answer returned before any payload is committed to, which is the required treatment of an object that satisfies no set. The call also returns the normalised config the platform would accept, which is the accepted parameter set, so a sink SHOULD use it rather than re-derive one. The limit is stated rather than glossed. WebCodecs' parameter vocabulary is the conventional-media one — codec, resolution, frame rate, bit depth, profile — which is the same shape as the ratified registers `capability-constraints.md` §3 draws on, and it has the same gap: no WebCodecs configuration describes the neural, quantum, world or haptic families. Those have no native tier to constrain at all, resolve through the WASM tier, and stay unconstrained until WAI authors their register entries. That is the state §3 already describes, not a shortfall introduced here. Framing renumbered §5 → §6. Docs only; `check-no-vanity-metrics.sh` passes. Co-Authored-By: Claude Opus 5 --- wai/CHANGELOG.md | 7 ++++- wai/extensions/webcodecs-negotiation.md | 39 ++++++++++++++++++++++++- 2 files changed, 44 insertions(+), 2 deletions(-) diff --git a/wai/CHANGELOG.md b/wai/CHANGELOG.md index fc3f9b833..55ec7222f 100644 --- a/wai/CHANGELOG.md +++ b/wai/CHANGELOG.md @@ -27,7 +27,12 @@ wire format may change and such changes are recorded here. in the browser's own capability-negotiation pattern (ask if a config is supported; fall back to wasm) and documents what the live wasm sinks already do: native codec when the platform reports it, WAI's pure-integer wasm decoders otherwise, the - classical floor always reachable. Referenced from SPEC §4. + classical floor always reachable. Referenced from SPEC §4. Its §5 reconciles the + mapping with [`extensions/capability-constraints.md`], which landed later: + `isConfigSupported()` takes a whole `VideoDecoderConfig`, so the native tier is + already a per-object constraint check evaluated before dispatch — and it carries + the same register gap, since no WebCodecs configuration describes the neural, + quantum, world or haptic families. ### Reproducible-reconstruction binding — a C2PA/JPEG-Trust hard binding over pixels diff --git a/wai/extensions/webcodecs-negotiation.md b/wai/extensions/webcodecs-negotiation.md index e14e3fd14..2f4ed9833 100644 --- a/wai/extensions/webcodecs-negotiation.md +++ b/wai/extensions/webcodecs-negotiation.md @@ -74,7 +74,44 @@ This never contradicts §4 — it is §4's "advertises `cap`" made precise for t browser: native when the platform reports it, WASM otherwise, and the classical floor always reachable through wasm. -## 5. Framing +## 5. The native tier is already a constraint check + +[`capability-constraints.md`](capability-constraints.md) makes dispatch a +precondition check: a sink satisfies a capability when it advertises `cap` **and** +the object's declared parameters satisfy one of the capability's registered +constraint sets. That extension landed after this mapping was written, and it +sharpens it rather than competing with it. + +`isConfigSupported()` is not a codec-string test. It takes a full configuration — +for video, the codec string plus `codedWidth`, `codedHeight`, the description, and +the other members of `VideoDecoderConfig` — and answers for *that configuration*. +It is therefore a constraint evaluator already, and the browser satisfies the +extension's requirements structurally rather than by additional machinery: + +- **Evaluated before dispatch.** `isConfigSupported()` is a pre-flight query; it + does not construct or configure a decoder. This is the "move the failure earlier" + requirement, met by the API's own shape. +- **Answered per object, not per sink.** A sink that decodes one object of a + capability and refuses another is expressing exactly what a constraint set + expresses. The parameters come from the object. +- **Unsupported, not a decode error.** A `supported: false` result is a negative + answer to a question, returned before any payload is committed to — which is the + required treatment where an object satisfies no set. + +The result also carries a `config` back: the normalised configuration the platform +would accept. A browser sink **SHOULD** treat that returned configuration as the +accepted parameter set for the object, rather than re-deriving it. + +**The honest limit.** WebCodecs' parameter vocabulary is the conventional-media +one — codec, resolution, frame rate, bit depth, profile. That is the same shape as +the ratified registers `capability-constraints.md` §3 draws on, and it has the same +gap: there is no WebCodecs configuration for the neural, quantum, world or haptic +families. Those capabilities have no native tier to constrain at all. They resolve +through the WASM tier (§3) and remain unconstrained until WAI authors their +register entries, which is the state §3 of that extension describes rather than a +shortfall of this mapping. + +## 6. Framing WAI's capability dispatch is not a bespoke negotiation scheme; it is the pattern the browser's own media stack already uses — ask whether a config is supported, -- 2.54.0 (Apple Git-157)