Transaction-Science/open-standards
Watch
1
0
Fork
You've already forked open-standards
 0
Code Issues Pull requests Projects Releases 15 Packages Wiki Activity Actions

wai.media.upscale: super-resolution as an exact derivation, residuals against it, per-title priors, and an account of sink-made pictures (C3) #408

Closed
dcharlot wants to merge 0 commits from wai/exact-upscale into main
pull from: wai/exact-upscale
merge into: Transaction-Science:main
Conversation 0 Commits - Files changed -
dcharlot commented 2026-10-21 15:05:27 -04:00
Owner
Copy link

What this is

The browser sink, as one new extension and its code: how a sink in a web page answers SPEC §4's "the sink advertises cap", and what the frame it shows can be checked against.

  • wai/extensions/browser-sink.md (new). §4 routes each object to the user agent's decoder, a native host's or WAI's wasm decoder. The user agent is asked only about codec strings the WebCodecs Codec Registry (Registry Draft, 24 September 2026) lists, so AV2, VVC and IAMF are never put to isConfigSupported and an answer about them is never read. The UA decoder is taken only where it is exact for the tier the object needs; wasm is the fallback. §5 evaluates capability-constraints' sets in the page. §6 reports a measured figure per decode beside Media Capabilities' powerEfficient, which stays the user agent's word. §7–§10 define contentBinding { alg, digest, receipt?, tier } on every decoded frame, its life through clone(), transfer, copyTo() and re-encode, the side table a page keeps until a registry lists the member, and the state of the frame on screen. §11 is the reasons table.
  • wai/proposals/webcodecs-content-binding.md (new; draft, not filed). The registry requirements read against the proposal, a draft registry row, draft IDL and semantics, the issue text ready to paste, what was tested, and what is open. Filing is the owner's decision.
  • capability-constraints.md §3.1: the wai: parameter register the extension needed and never had, and a pointer to its evaluation (browser-sink §5).
  • wai-rs: module browser_sink (feature browser_sink = jwp_hash + ed25519; builds for wasm32), corpus generator wai_browser_sink_vectors, tests/browser_sink.rs.
  • Corpus wai/live-conformance/browser/ (vectors.json, int_motion.wai, README with **Provenance.**). It holds routes (energy entries among them), constraint sets, bindings, frame tiers, a chain of hand-signed legacy group receipts whose objects bind, frame states and side-table sequences. The second implementation is browser_sink_verify.py (python3 -I, standard library). It reads §4.1, §4.2, §11, §2's tier table and capability-constraints §3.1 from the texts, and each capability's class from registry/capabilities.json.
  • wai-live-web: src/browser.rs exports routeReport, evaluateConstraints, readBinding, makeBinding, frameTier and frameState. With the new feature int_video it also exports decodeBound, the wasm-route decoder: it verifies a wai.video.int_motion envelope against its group and object receipts (jwp_delivery), decodes it, and binds every frame. An object that does not verify is not decoded. One decoded without receipts is left unbound. Native tests hold these functions to the corpus.
  • joule-moq-web:
    • The verifier's object() now also returns content_hash and group_receipt (the link hash).
    • web/room/binding.js (new) holds the side table, the two-pass capability report and the state helpers.
    • The room routes each track before decoding it, binds every frame its video decoders output, and shows each tile's state with ?debug=binding.
    • web/binding/ is the test page.
    • e2e/energy.py reads the host's per-process counter.
    • e2e/run.sh builds wai_live_web_decode and runs web/room/binding.test.mjs.
    • The e2e scenario binding is in the default list.
  • Registry: contentBinding is a wire identifier of the new kind frame-metadata, status proposed. The export is regenerated and registry export --check passes.
  • SPEC §4 paragraph for a sink in a web page. CHANGELOG entry. README of joule-moq-web and the live-conformance index.
  • CI: a wai-conformance.yml step "browser sink · wai-rs, wai-live-web and browser_sink_verify.py", with paths for the two extensions and wai/wai-live-web/**. joule-moq-web.yml now watches wai/live-conformance/browser/** and wai/wai-rs/tests/vectors/int_video/**, and its header describes the scenario.

PR #54: what was kept, changed and dropped (close #54 as superseded)

#54's branch was not merged. Its intent is rebuilt on current main inside browser-sink.md.

  • Kept
    • A browser sink answers §4's "advertises cap" with the platform's own question first (isConfigSupported) and WAI's wasm decoders as the fallback, which depends on nothing the device ships.
    • #54's reconciliation with capability-constraints is now §4.5. isConfigSupported already does a pre-flight, per-configuration constraint check: it answers "unsupported", never a decode error. A page SHOULD configure the decoder with the configuration the answer returns.
    • The stated limit: the WebCodecs vocabulary is conventional media only. The neural, quantum, world and haptic families have no UA tier, and only WAI's constraint sets constrain them.
    • A SPEC §4 paragraph pointing at the browser mapping.
  • Changed
    • Informative became normative. #54 was a mapping with "SHOULD resolve"; the route is now a deterministic function of a capability report, with a reason for every refusal, a corpus and a second implementation.
    • The resolution order. #54 had native → wasm → fallback. It is now UA → host → wasm, and the UA leg is taken only where the decoder is exact for the object's need (the tier from its pin or the track's wai_determinism). A need of decode-equivalence takes the UA decoder only after a probe matches published frame digests (§4.4).
    • The UA table is gated by the Codec Registry (2026-09). #54 listed AV1 and Opus with no registry check; AV2, VVC and IAMF now never reach the user agent.
    • #54 said the wasm decode is "byte-identical"; that now holds only through the registered tier. A wasm route carries the capability's registered tier, so it is decode-equivalence only for StandardDefined and NeuralIntegerExact capabilities.
    • capability-constraints' evaluation, spec-only in #54's day, now has a register and code.
  • Dropped
    • ImageDecoder rows for wai.image.png/jpeg/jxl/avif: a user agent's image decode may apply colour conversion and does not say so.
    • The claim that a browser sink "MAY carry wasm decoders for PNG/FLAC/zstd" for the classical floor: no page here ships them, and a page now lists what its wasm decodes in the report's wasm.
    • The file name webcodecs-negotiation.md, #54's CHANGELOG entry, and its §6 framing.

Decisions the lead should know

  1. The extension's home. I made it a new extension rather than an addition to http-delivery §12. The room is MoQ and the browser sink is transport-independent; http-delivery §12.1's via is filled from the route (§4.6), and http-delivery is not edited.
  2. What digest and receipt name. digest is jwp-receipts §2.1's content hash of the receipted object. In an E2EE room that is the sealed object the receipt signs, not the plaintext decoded from it. Over MoQ, digest and receipt equal the WAI_RECEIPT property's content_hash and group_receipt.
  3. StandardDefined is read as decode-equivalence for this extension's "registered tier", because SPEC §7 gives it the same property. GenerativeExact is held at none. The text says this is a reading. In the room, VP9 frames get tier none because no probe ran, and that is shown as-is.
  4. A frame is bound only when every object its decoder was given since its last key frame was established. Predicted frames depend on their references. The room verifies before decoding, so it meets this by construction.
  5. The energy classes. The brief says "capability-constraints' energy classes", but that extension has none (its §4 says it is not an energy mechanism). I used energy-measurement §2's classes, and capability-constraints is left unchanged on energy.
  6. Reason::ALL became Reason::all(). A pub const array of a type named Reason in a new module made cbindgen add four opaque typedef struct Reason Reason; to include/wai.h, one for each module's Reason. I bisected it on the build host. With the function, a fresh generation of wai.h is byte-identical to main's.
  7. A separate decode wasm build (wai_live_web_decode), so the room's own wai_live_web is unchanged in what it carries.
  8. No consent rule. §13 states what a report reveals and points a service that collects reports to jwp-receipts' lawful-basis claim. It sets no rule for the user.
  9. JSON integers in a JS page. A page cannot keep 1920.0 apart from 1920, so the corpus's malformed-bound case is a true fraction (1920.5). §5 says so.

Findings from the browser

  • The user agent drops the unregistered member. It is the one the e2e drives, headless; its version is in the run's log. metadata() was empty for a frame constructed with init.metadata = { contentBinding, rtpTimestamp }, through both the buffer constructor and the frame constructor, and after clone(). The registered rtpTimestamp was dropped the same way. The proposal's issue text states this.
  • Energy. On the build host, decoding 640×360 VP9 through the user agent's decoder measured 816–858 µJ per decode across eight runs (840, 838, 816, 857, 858, 851, 836 and 839). Each run is about 36 000 decodes over about 14 s, taken as the marginal over idle windows of the same length. It is OnChipCounter with filtering Unknown and coverage Partial, because the per-process counter covers CPU energy only (plus billed and less serviced). The user agent's powerEfficient beside it was true every time. Each run declares a type-A standard uncertainty of 0.3–2.8 % from its own windows, but the spread between runs is about 5 %. The declared figure therefore understates what a repeat would see. A meter that publishes such figures should add a between-run component; the e2e only checks that a figure is carried with its class beside the boolean.

Checks

check command result
browser_sink unit tests cargo test --locked --no-default-features --features browser_sink --lib browser_sink:: (rb) 12 passed
integration test cargo test --locked --no-default-features --features browser_sink,receipt_claims --test browser_sink (rb) 6 passed
corpus regenerates wai_browser_sink_vectors verify ../live-conformance/browser (rb) PASS ×3
second implementation python3 -I live-conformance/browser_sink_verify.py live-conformance/browser 101 cases PASS; unregistered answers changed 3 ways never read; 28 of 28 mutations fail
wai-live-web native cargo test --locked --lib and --features int_video --lib browser_tests (rb) 5 passed; 3 passed (every chain object's frames hash to sha256.json)
joule-moq-web cargo test --locked -p joule-moq-web --lib (rb) 8 passed
side table in node node --test web/room/binding.test.mjs 12 passed
registry wai registry export then --check; cargo test --lib codecs::registry (rb) up to date; 14 passed
header fresh cbindgen run (touch src/container.rs, build) then git diff include/wai.h (rb) unchanged
clippy --all-targets on wai-rs (browser_sink,receipt_claims), wai-live-web (± int_video), joule-moq-web (rb) zero notes in the new code (joule-moq's and int_payload's notes are main's)
full browser e2e e2e/run.sh on the build host, installed browser 188 of 188 checks, every scenario
binding e2e, looped e2e/run.sh --only binding --runs 3, then --runs 2 on the final tree 3 of 3 runs, 90 of 90 checks; 2 of 2, 60 of 60
repo checks check-tests-in-ci.py, check-workflow-concurrency.sh, check-spec-refs.py, check-wai-section-refs.py, check-no-vanity-metrics.sh, test-public-text-checks.sh, check-comparative-prose.sh, check-orphan-sources.py all exit 0
public names check-public-names.sh over wai/, and file by file over every file this branch changes every changed file clean. The hits are main's: live_provenance.rs ×3 and the existing BROWSER_BIN path in joule-moq-web.yml
after merging origin/main (#397, #399, #403) registry export --check, codecs::registry, browser_sink lib and --test browser_sink, int_kernels_vectors every_decoded_output_states_a_sha256, corpus_provenance, the verifier, the repo checks (rb and local) up to date; all pass. The conflicts were in five files where both sides added (registry wire identifiers, the live-conformance index, the digest-exclusion reason, the CHANGELOG, the workflow), and each kept both

Merge with main after SYN-2 (wai-mark:tent64), and what it changed

origin/main at df94e4af8 was merged into this branch (merge df1004f18). It brings SYN-2 #407, CO1 #403, E6 #404, ACC #396, E7 #398, CO2 and CO3 #406, shared-lists #416, #423 and the rest. I reset the worktree to the pushed head first and merged fresh. Every conflict was resolved by meaning, and rerere's recorded resolutions were discarded, not reused.

What the merge changed:

  • synthesis::Case now holds both the exact-upscale priors and SYN-2's unmarked frames (§7.6).
    • The synthesis corpus serialises and reads both.
    • A literal of synthesis's Policy in this branch's corpus now takes ..Default::default(), so it carries the new second_layer_proof.
  • synthesis.md §11.1 lists both inputs (priors, then unmarked frames) and the three policy members. The rest of synthesis.md, both §7 and §4, merged without conflict, and its header and §13 keep both changes.
  • The synthesis-mark corpus (synthesis_mark/corpus.rs): its cases hold no prior (priors: vec![]). Its JSON is unchanged and its corpus regenerates byte for byte. synthesis_mark_verify.py states case["priors"] = [] for the same reason.
  • exact-upscale passes synthesis's third policy member through.
    • upscale/vectors.json policies now carry "second_layer".
    • upscale_verify.py gives synthesis its unmarked input (none) and second_layer.
    • §7.1 now reads "synthesis §11.1's members and one more".
  • Text and CI:
    • SPEC §5 and REGISTRY take main's new record lists, which point at the registry table.
    • The live-conformance table gains the upscale/ row in its generator column.
    • Both CI steps (the synthesis watermark and exact upscale) and both CHANGELOG entries are kept.
    • The synthesis corpus README now names second_layer and unmarked.
  • Regenerated, never merged by hand: synthesis/vectors.json (wai_synthesis_vectors gen), upscale/vectors.json (wai_upscale_vectors gen), and the registry export (wai registry export).

Checks after the merge (all through rb at 23c9c09ae):

check result
--lib int_sr:: 5 passed
--lib exact_upscale:: 3 passed
--lib synthesis:: (synthesis features) 17 passed
--lib synthesis:: (exact_upscale features) 17 passed
--test exact_upscale 5 passed
--test synthesis 4 passed
--lib -- synthesis::mark synthesis_mark:: 13 passed
--release --test synthesis_mark 5 passed
--lib codecs:: 53 passed
codecs::registry with exact_upscale 14 passed
WAI_OPS_GPU_REQUIRE=1 --release --test exact_upscale_gpu graph_runs=12 acc32=12 cpu=0 differ=0 errors=0 equal
registry export --check up to date
synthesis_verify.py exit 0, 45 vectors, every mutation caught
upscale_verify.py exit 0, 30 vectors, every mutation caught
synthesis_mark_verify.py exit 0
clippy --all-targets (neural_int, exact_upscale, ops_gpu,exact_upscale, synthesis_mark, exact_upscale,video_int) 0 notes in the files this branch touches
check-tests-in-ci, check-workflow-concurrency, check-spec-refs, check-wai-section-refs, check-no-vanity-metrics, test-public-text-checks, check-comparative-prose, check-orphan-sources all exit 0

Interaction left for later:

  • A frame a sink generates and presents is a samples subject, and §7's markings name objects. The second marking layer therefore does not yet reach sink-made frames, as before.
  • SYN-3 (tent64s, #425) integrates after this lands.

Falsifications (each committed first, broken on the build host or locally, then restored)

  • Rust (wai_browser_sink_vectors verify). Each of these fails (exit 101 from the generator's own asserts, or exit 1 with FAIL lines), and the restored code exits 0:
    • registry gate removed;
    • a timestamp given twice not struck;
    • refused provenance ignored;
    • zero-with-class read as a figure;
    • probe not required;
    • powerEfficient stated off the UA route;
    • receipt dropped from the canonical form.
    • Also, a reason row dropped from §11 fails the_extension_lists_the_reasons.
  • Python (browser_sink_verify.py). Each fails with exit 1, and the restored verifier exits 0:
    • gate dropped;
    • provenance ignored;
    • side table never strikes;
    • an unregistered answer read.
    • The signature check failed no case at first: every signature mutation was also caught by the canonical bytes. I added a mutation that forges a signature and makes the canonical bytes, link hash and binding all agree, and dropping the signature check now fails it (27 of 28, exit 1).
  • JS (binding.test.mjs): with the side table never striking, 2 tests fail.
  • Browser e2e (--only binding), each against an otherwise unchanged tree:
    • the page asks about every codec string: exit 1, and the check "no codec string outside the WebCodecs Codec Registry…" FAILs;
    • the room enters no binding for its chunks: exit 1, and the tile never shows a bound, verified frame (the wait times out);
    • the wasm decoder binds an object that did not verify: exit 1. The changed object is then decoded, its payload no longer decodes, and the page fails at "an object changed after it was receipted is refused and not decoded" and the checks after it.

Deploy note

The live room must be redeployed as one unit after this lands: page assets, relay and room service from one commit. The page assets are web/room/ (with the new binding.js), both rebuilt wasm (joule_moq_web, whose verifier result changed, and wai_live_web, now with browser_sink), and web/binding/ with wai_live_web_decode if /binding/ is served. Nothing was deployed.

Not done

  • No probe objects for UA codecs are published. A need of decode-equivalence therefore never routes to a UA decoder (§4.4 says so), and the room's VP9 frames are tier none.
  • No page implements the host route. It is in the spec and the corpus only.
  • C2's live-provenance verdict enters §10 as an input, and the corpus covers accepted, refused, missing and not_applicable. The room's tracks are WAI-packaged, so their provenance is not_applicable (live-provenance §1). No page verifies a CMAF provenance track in the browser yet.
  • No binding for AudioData, which has no metadata dictionary. Frames decoded for a late joiner's FETCH and for DVR playback are not bound.
  • The energy figure is measured only where the host has a per-process counter (macOS here). Elsewhere the e2e step is reported skipped, not passed.
  • The WebCodecs issue is not filed.
  • wai-live-web's pre-existing native tests (compose and fixture) are still in no workflow; this PR adds only its browser_tests to CI.

🤖 Generated with Claude Code

Closed: Landed in #428 (merge train, merged as aa61ab3b with all 65 checks green).

## What this is The browser sink, as one new extension and its code: how a sink in a web page answers SPEC §4's "the sink advertises `cap`", and what the frame it shows can be checked against. - **`wai/extensions/browser-sink.md`** (new). §4 routes each object to the user agent's decoder, a native host's or WAI's wasm decoder. The user agent is asked only about codec strings the WebCodecs Codec Registry (Registry Draft, 24 September 2026) lists, so AV2, VVC and IAMF are never put to `isConfigSupported` and an answer about them is never read. The UA decoder is taken only where it is exact for the tier the object needs; wasm is the fallback. §5 evaluates capability-constraints' sets in the page. §6 reports a measured figure per decode beside Media Capabilities' `powerEfficient`, which stays the user agent's word. §7–§10 define **`contentBinding`** `{ alg, digest, receipt?, tier }` on every decoded frame, its life through `clone()`, transfer, `copyTo()` and re-encode, the side table a page keeps until a registry lists the member, and the state of the frame on screen. §11 is the reasons table. - **`wai/proposals/webcodecs-content-binding.md`** (new; **draft, not filed**). The registry requirements read against the proposal, a draft registry row, draft IDL and semantics, the issue text ready to paste, what was tested, and what is open. Filing is the owner's decision. - **`capability-constraints.md` §3.1**: the `wai:` parameter register the extension needed and never had, and a pointer to its evaluation (browser-sink §5). - **`wai-rs`**: module `browser_sink` (feature `browser_sink` = `jwp_hash` + `ed25519`; builds for wasm32), corpus generator `wai_browser_sink_vectors`, `tests/browser_sink.rs`. - **Corpus** `wai/live-conformance/browser/` (`vectors.json`, `int_motion.wai`, README with `**Provenance.**`). It holds routes (energy entries among them), constraint sets, bindings, frame tiers, a chain of hand-signed legacy group receipts whose objects bind, frame states and side-table sequences. The second implementation is **`browser_sink_verify.py`** (`python3 -I`, standard library). It reads §4.1, §4.2, §11, §2's tier table and capability-constraints §3.1 from the texts, and each capability's class from `registry/capabilities.json`. - **`wai-live-web`**: `src/browser.rs` exports `routeReport`, `evaluateConstraints`, `readBinding`, `makeBinding`, `frameTier` and `frameState`. With the new feature `int_video` it also exports `decodeBound`, the wasm-route decoder: it verifies a `wai.video.int_motion` envelope against its group and object receipts (`jwp_delivery`), decodes it, and binds every frame. An object that does not verify is not decoded. One decoded without receipts is left unbound. Native tests hold these functions to the corpus. - **`joule-moq-web`**: - The verifier's `object()` now also returns `content_hash` and `group_receipt` (the link hash). - `web/room/binding.js` (new) holds the side table, the two-pass capability report and the state helpers. - The room routes each track before decoding it, binds every frame its video decoders output, and shows each tile's state with `?debug=binding`. - `web/binding/` is the test page. - `e2e/energy.py` reads the host's per-process counter. - `e2e/run.sh` builds `wai_live_web_decode` and runs `web/room/binding.test.mjs`. - The e2e scenario `binding` is in the default list. - **Registry**: `contentBinding` is a wire identifier of the new kind `frame-metadata`, status `proposed`. The export is regenerated and `registry export --check` passes. - **SPEC §4** paragraph for a sink in a web page. **CHANGELOG** entry. README of `joule-moq-web` and the `live-conformance` index. - **CI**: a `wai-conformance.yml` step "browser sink · wai-rs, wai-live-web and browser_sink_verify.py", with paths for the two extensions and `wai/wai-live-web/**`. `joule-moq-web.yml` now watches `wai/live-conformance/browser/**` and `wai/wai-rs/tests/vectors/int_video/**`, and its header describes the scenario. ## PR #54: what was kept, changed and dropped (close #54 as superseded) #54's branch was not merged. Its intent is rebuilt on current main inside `browser-sink.md`. - **Kept** - A browser sink answers §4's "advertises `cap`" with the platform's own question first (`isConfigSupported`) and WAI's wasm decoders as the fallback, which depends on nothing the device ships. - #54's reconciliation with capability-constraints is now §4.5. `isConfigSupported` already does a pre-flight, per-configuration constraint check: it answers "unsupported", never a decode error. A page SHOULD configure the decoder with the configuration the answer returns. - The stated limit: the WebCodecs vocabulary is conventional media only. The neural, quantum, world and haptic families have no UA tier, and only WAI's constraint sets constrain them. - A SPEC §4 paragraph pointing at the browser mapping. - **Changed** - Informative became normative. #54 was a mapping with "SHOULD resolve"; the route is now a deterministic function of a capability report, with a reason for every refusal, a corpus and a second implementation. - The resolution order. #54 had native → wasm → fallback. It is now UA → host → wasm, and the UA leg is taken only where the decoder is exact for the object's need (the tier from its pin or the track's `wai_determinism`). A need of `decode-equivalence` takes the UA decoder only after a probe matches published frame digests (§4.4). - The UA table is gated by the Codec Registry (2026-09). #54 listed AV1 and Opus with no registry check; AV2, VVC and IAMF now never reach the user agent. - #54 said the wasm decode is "byte-identical"; that now holds only through the registered tier. A wasm route carries the capability's registered tier, so it is `decode-equivalence` only for `StandardDefined` and `NeuralIntegerExact` capabilities. - capability-constraints' evaluation, spec-only in #54's day, now has a register and code. - **Dropped** - `ImageDecoder` rows for `wai.image.png/jpeg/jxl/avif`: a user agent's image decode may apply colour conversion and does not say so. - The claim that a browser sink "MAY carry wasm decoders for PNG/FLAC/zstd" for the classical floor: no page here ships them, and a page now lists what its wasm decodes in the report's `wasm`. - The file name `webcodecs-negotiation.md`, #54's CHANGELOG entry, and its §6 framing. ## Decisions the lead should know 1. **The extension's home.** I made it a new extension rather than an addition to http-delivery §12. The room is MoQ and the browser sink is transport-independent; http-delivery §12.1's `via` is filled from the route (§4.6), and http-delivery is not edited. 2. **What `digest` and `receipt` name.** `digest` is jwp-receipts §2.1's content hash of the *receipted* object. In an E2EE room that is the sealed object the receipt signs, not the plaintext decoded from it. Over MoQ, `digest` and `receipt` equal the `WAI_RECEIPT` property's `content_hash` and `group_receipt`. 3. **`StandardDefined` is read as `decode-equivalence`** for this extension's "registered tier", because SPEC §7 gives it the same property. `GenerativeExact` is held at `none`. The text says this is a reading. In the room, VP9 frames get tier **`none`** because no probe ran, and that is shown as-is. 4. **A frame is bound only when every object its decoder was given since its last key frame was established.** Predicted frames depend on their references. The room verifies before decoding, so it meets this by construction. 5. **The energy classes.** The brief says "capability-constraints' energy classes", but that extension has none (its §4 says it is not an energy mechanism). I used energy-measurement §2's classes, and capability-constraints is left unchanged on energy. 6. **`Reason::ALL` became `Reason::all()`.** A `pub const` array of a type named `Reason` in a new module made cbindgen add four opaque `typedef struct Reason Reason;` to `include/wai.h`, one for each module's `Reason`. I bisected it on the build host. With the function, a fresh generation of `wai.h` is byte-identical to main's. 7. **A separate decode wasm build** (`wai_live_web_decode`), so the room's own `wai_live_web` is unchanged in what it carries. 8. **No consent rule.** §13 states what a report reveals and points a service that collects reports to jwp-receipts' `lawful-basis` claim. It sets no rule for the user. 9. **JSON integers in a JS page.** A page cannot keep `1920.0` apart from `1920`, so the corpus's malformed-bound case is a true fraction (`1920.5`). §5 says so. ## Findings from the browser - **The user agent drops the unregistered member.** It is the one the e2e drives, headless; its version is in the run's log. `metadata()` was empty for a frame constructed with `init.metadata = { contentBinding, rtpTimestamp }`, through both the buffer constructor and the frame constructor, and after `clone()`. The registered `rtpTimestamp` was dropped the same way. The proposal's issue text states this. - **Energy.** On the build host, decoding 640×360 VP9 through the user agent's decoder measured **816–858 µJ per decode** across eight runs (840, 838, 816, 857, 858, 851, 836 and 839). Each run is about 36 000 decodes over about 14 s, taken as the marginal over idle windows of the same length. It is `OnChipCounter` with filtering `Unknown` and coverage `Partial`, because the per-process counter covers CPU energy only (plus billed and less serviced). The user agent's `powerEfficient` beside it was `true` every time. **Each run declares a type-A standard uncertainty of 0.3–2.8 % from its own windows, but the spread between runs is about 5 %. The declared figure therefore understates what a repeat would see.** A meter that publishes such figures should add a between-run component; the e2e only checks that a figure is carried with its class beside the boolean. ## Checks | check | command | result | |---|---|---| | browser_sink unit tests | `cargo test --locked --no-default-features --features browser_sink --lib browser_sink::` (rb) | 12 passed | | integration test | `cargo test --locked --no-default-features --features browser_sink,receipt_claims --test browser_sink` (rb) | 6 passed | | corpus regenerates | `wai_browser_sink_vectors verify ../live-conformance/browser` (rb) | PASS ×3 | | second implementation | `python3 -I live-conformance/browser_sink_verify.py live-conformance/browser` | 101 cases PASS; unregistered answers changed 3 ways never read; 28 of 28 mutations fail | | wai-live-web native | `cargo test --locked --lib` and `--features int_video --lib browser_tests` (rb) | 5 passed; 3 passed (every chain object's frames hash to `sha256.json`) | | joule-moq-web | `cargo test --locked -p joule-moq-web --lib` (rb) | 8 passed | | side table in node | `node --test web/room/binding.test.mjs` | 12 passed | | registry | `wai registry export` then `--check`; `cargo test --lib codecs::registry` (rb) | up to date; 14 passed | | header | fresh cbindgen run (`touch src/container.rs`, build) then `git diff include/wai.h` (rb) | unchanged | | clippy | `--all-targets` on wai-rs (`browser_sink,receipt_claims`), wai-live-web (± `int_video`), joule-moq-web (rb) | zero notes in the new code (joule-moq's and int_payload's notes are main's) | | full browser e2e | `e2e/run.sh` on the build host, installed browser | **188 of 188 checks**, every scenario | | binding e2e, looped | `e2e/run.sh --only binding --runs 3`, then `--runs 2` on the final tree | **3 of 3 runs, 90 of 90 checks; 2 of 2, 60 of 60** | | repo checks | `check-tests-in-ci.py`, `check-workflow-concurrency.sh`, `check-spec-refs.py`, `check-wai-section-refs.py`, `check-no-vanity-metrics.sh`, `test-public-text-checks.sh`, `check-comparative-prose.sh`, `check-orphan-sources.py` | all exit 0 | | public names | `check-public-names.sh` over `wai/`, and file by file over every file this branch changes | every changed file clean. The hits are main's: `live_provenance.rs` ×3 and the existing `BROWSER_BIN` path in `joule-moq-web.yml` | | after merging origin/main (#397, #399, #403) | registry export `--check`, `codecs::registry`, `browser_sink` lib and `--test browser_sink`, `int_kernels_vectors every_decoded_output_states_a_sha256`, `corpus_provenance`, the verifier, the repo checks (rb and local) | up to date; all pass. The conflicts were in five files where both sides added (registry wire identifiers, the live-conformance index, the digest-exclusion reason, the CHANGELOG, the workflow), and each kept both | ## Merge with main after SYN-2 (`wai-mark:tent64`), and what it changed `origin/main` at `df94e4af8` was merged into this branch (merge `df1004f18`). It brings SYN-2 #407, CO1 #403, E6 #404, ACC #396, E7 #398, CO2 and CO3 #406, shared-lists #416, #423 and the rest. I reset the worktree to the pushed head first and merged fresh. Every conflict was resolved by meaning, and rerere's recorded resolutions were discarded, not reused. What the merge changed: - **`synthesis::Case`** now holds both the exact-upscale `priors` and SYN-2's `unmarked` frames (§7.6). - The synthesis corpus serialises and reads both. - A literal of synthesis's `Policy` in this branch's corpus now takes `..Default::default()`, so it carries the new `second_layer_proof`. - **`synthesis.md` §11.1** lists both inputs (priors, then unmarked frames) and the three policy members. The rest of synthesis.md, both §7 and §4, merged without conflict, and its header and §13 keep both changes. - **The synthesis-mark corpus** (`synthesis_mark/corpus.rs`): its cases hold no prior (`priors: vec![]`). Its JSON is unchanged and its corpus regenerates byte for byte. `synthesis_mark_verify.py` states `case["priors"] = []` for the same reason. - **exact-upscale** passes synthesis's third policy member through. - `upscale/vectors.json` policies now carry `"second_layer"`. - `upscale_verify.py` gives synthesis its `unmarked` input (none) and `second_layer`. - §7.1 now reads "synthesis §11.1's members and one more". - **Text and CI**: - SPEC §5 and REGISTRY take main's new record lists, which point at the registry table. - The live-conformance table gains the `upscale/` row in its generator column. - Both CI steps (the synthesis watermark and exact upscale) and both CHANGELOG entries are kept. - The synthesis corpus README now names `second_layer` and `unmarked`. - **Regenerated, never merged by hand**: `synthesis/vectors.json` (`wai_synthesis_vectors gen`), `upscale/vectors.json` (`wai_upscale_vectors gen`), and the registry export (`wai registry export`). Checks after the merge (all through `rb` at `23c9c09ae`): | check | result | |---|---| | `--lib int_sr::` | 5 passed | | `--lib exact_upscale::` | 3 passed | | `--lib synthesis::` (`synthesis` features) | 17 passed | | `--lib synthesis::` (`exact_upscale` features) | 17 passed | | `--test exact_upscale` | 5 passed | | `--test synthesis` | 4 passed | | `--lib -- synthesis::mark synthesis_mark::` | 13 passed | | `--release --test synthesis_mark` | 5 passed | | `--lib codecs::` | 53 passed | | `codecs::registry` with `exact_upscale` | 14 passed | | `WAI_OPS_GPU_REQUIRE=1 --release --test exact_upscale_gpu` | `graph_runs=12 acc32=12 cpu=0 differ=0 errors=0 equal` | | `registry export --check` | up to date | | `synthesis_verify.py` | exit 0, 45 vectors, every mutation caught | | `upscale_verify.py` | exit 0, 30 vectors, every mutation caught | | `synthesis_mark_verify.py` | exit 0 | | clippy `--all-targets` (`neural_int`, `exact_upscale`, `ops_gpu,exact_upscale`, `synthesis_mark`, `exact_upscale,video_int`) | 0 notes in the files this branch touches | | `check-tests-in-ci`, `check-workflow-concurrency`, `check-spec-refs`, `check-wai-section-refs`, `check-no-vanity-metrics`, `test-public-text-checks`, `check-comparative-prose`, `check-orphan-sources` | all exit 0 | Interaction left for later: - A frame a sink generates and presents is a `samples` subject, and §7's markings name objects. The second marking layer therefore does not yet reach sink-made frames, as before. - SYN-3 (`tent64s`, #425) integrates after this lands. ## Falsifications (each committed first, broken on the build host or locally, then restored) - **Rust** (`wai_browser_sink_vectors verify`). Each of these fails (exit 101 from the generator's own asserts, or exit 1 with FAIL lines), and the restored code exits 0: - registry gate removed; - a timestamp given twice not struck; - refused provenance ignored; - zero-with-class read as a figure; - probe not required; - `powerEfficient` stated off the UA route; - receipt dropped from the canonical form. - Also, a reason row dropped from §11 fails `the_extension_lists_the_reasons`. - **Python** (`browser_sink_verify.py`). Each fails with exit 1, and the restored verifier exits 0: - gate dropped; - provenance ignored; - side table never strikes; - an unregistered answer read. - The signature check failed **no** case at first: every signature mutation was also caught by the canonical bytes. I added a mutation that forges a signature and makes the canonical bytes, link hash and binding all agree, and dropping the signature check now fails it (27 of 28, exit 1). - **JS** (`binding.test.mjs`): with the side table never striking, 2 tests fail. - **Browser e2e** (`--only binding`), each against an otherwise unchanged tree: - the page asks about every codec string: exit 1, and the check "no codec string outside the WebCodecs Codec Registry…" FAILs; - the room enters no binding for its chunks: exit 1, and the tile never shows a bound, verified frame (the wait times out); - the wasm decoder binds an object that did not verify: exit 1. The changed object is then decoded, its payload no longer decodes, and the page fails at "an object changed after it was receipted is refused and not decoded" and the checks after it. ## Deploy note **The live room must be redeployed as one unit after this lands**: page assets, relay and room service from one commit. The page assets are `web/room/` (with the new `binding.js`), both rebuilt wasm (`joule_moq_web`, whose verifier result changed, and `wai_live_web`, now with `browser_sink`), and `web/binding/` with `wai_live_web_decode` if `/binding/` is served. Nothing was deployed. ## Not done - No probe objects for UA codecs are published. A need of `decode-equivalence` therefore never routes to a UA decoder (§4.4 says so), and the room's VP9 frames are tier `none`. - No page implements the host route. It is in the spec and the corpus only. - C2's live-provenance verdict enters §10 as an input, and the corpus covers `accepted`, `refused`, `missing` and `not_applicable`. The room's tracks are WAI-packaged, so their provenance is `not_applicable` (live-provenance §1). No page verifies a CMAF provenance track in the browser yet. - No binding for `AudioData`, which has no metadata dictionary. Frames decoded for a late joiner's FETCH and for DVR playback are not bound. - The energy figure is measured only where the host has a per-process counter (macOS here). Elsewhere the e2e step is reported **skipped**, not passed. - The WebCodecs issue is not filed. - `wai-live-web`'s pre-existing native tests (compose and fixture) are still in no workflow; this PR adds only its `browser_tests` to CI. 🤖 Generated with [Claude Code](https://claude.com/claude-code) --- **Closed:** Landed in #428 (merge train, merged as aa61ab3b with all 65 checks green). 
The int_sr module (WSR1 prior, wai-upscale:sr2/sr4 lowered to one wai-ops/1 graph), the
exact_upscale module (upscale record with residual and target, presentation statement),
synthesis's samples subject and SR evidence, the corpus generator, the second implementation,
the training tool and its trained priors.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
Co-Authored-By: Claude Opus 5.5 <[email protected]>
Co-Authored-By: Claude Opus 5.5 <[email protected]>
Co-Authored-By: Claude Opus 5.5 <[email protected]>
Co-Authored-By: Claude Opus 5.5 <[email protected]>
wai: exact upscale — the entry's size bound and §3.3's limits at step 4.3, stated
All checks were successful
WAI conformance / cargo-class corpora · linux (pull_request) Successful in 19m2s
Details
wai-ops integer operators / wasm32-wasip2 (wasmtime) (pull_request) Successful in 30s
Details
wai-ops integer operators / wasm32 under Node (pull_request) Successful in 23s
Details
wai-ops integer operators / independent checker (Python) (pull_request) Successful in 14s
Details
wai-pack / wai-pack · linux (pull_request) Successful in 1m43s
Details
wai-quantum studio and desktop / studio · browser (pull_request) Successful in 2m38s
Details
wai-web wasm32 / wai-web wasm32 build · corpus, menu and writers (pull_request) Successful in 4m18s
Details
joule-moq-web browser client / joule-moq-web · browser end-to-end (macos) (pull_request) Successful in 7m49s
Details
standards workspaces build + test / joule-code-rs (joule-code/joule-code-rs, joule-code-rs, linux-arm64, true) (pull_request) Successful in 4m34s
Details
WAI conformance / v2 vocabulary · manifests validate as stated (pull_request) Successful in 35s
Details
WAI conformance / delivery telemetry · CMCD and CMSD against an independent library (pull_request) Successful in 25s
Details
WAI conformance / cross-arch byte-equality · linux-arm64 (pull_request) Successful in 26s
Details
WAI conformance / C2PA manifest emission · signed, read back, validated (pull_request) Successful in 1m45s
Details
WAI conformance / cargo-class corpora · linux-arm64 (pull_request) Successful in 21m1s
Details
wai-ops integer operators / linux-aarch64 (pull_request) Successful in 1m59s
Details
WAI energy receipts / energy receipts · linux-arm64 (pull_request) Successful in 10m15s
Details
wai-pack / wai-pack · linux-arm64 (pull_request) Successful in 2m2s
Details
wai-pack / wai-pack · the floor through a media prober (pull_request) Successful in 1m54s
Details
wai-pack / wai-pack · dependency licences (pull_request) Successful in 1m39s
Details
wai-pack / wai-pack · independent verifier (Python) (pull_request) Successful in 5m42s
Details
WAI conformance / independent verifiers · claims, JWP, key discovery and integer payloads (Python) (pull_request) Successful in 32m28s
Details
standards workspaces build + test / macOS energy backends (pull_request) Successful in 8m31s
Details
WAI conformance / cargo-class corpora · macos (pull_request) Successful in 14m0s
Details
WAI conformance / cross-arch byte-equality · macos (pull_request) Successful in 39s
Details
WAI conformance / codec wrappers · fuzz smoke, text decode limits, C entry points, provenance, the semantic path (pull_request) Successful in 48s
Details
WAI energy receipts / energy receipts · macos (pull_request) Successful in 6m6s
Details
wai-ops-gpu integer operators on GPUs / Metal GPU and browser WebGPU (macos) (pull_request) Successful in 5m1s
Details
wai-ops integer operators / macos-arm64 (pull_request) Successful in 55s
Details
wai-pack / wai-pack · macos (pull_request) Successful in 1m4s
Details
wai-quantum studio and desktop / desktop app · macos (pull_request) Successful in 2m3s
Details
aa47120215 
Co-Authored-By: Claude Opus 5.5 <[email protected]>
Resolved by meaning: synthesis::Case holds both the priors (exact-upscale) and the unmarked
frames (§7.6); synthesis §11.1 lists both inputs and the three policy members; the
synthesis-mark corpus's cases hold no prior; exact-upscale passes synthesis's second_layer
policy through; the live-conformance table gains the upscale row in its generator column; both
CI steps and both CHANGELOG entries are kept. Generated files are main's here and are
regenerated in the next commit.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
wai: regenerate the synthesis and exact-upscale corpora and the registry export after the merge
Some checks failed
wai-ops-gpu integer operators on GPUs / does this change touch it (pull_request) Has been cancelled
Details
wai-ops-gpu integer operators on GPUs / builds · no adapter is not a pass (linux) (pull_request) Has been cancelled
Details
wai-ops-gpu integer operators on GPUs / Metal GPU and browser WebGPU (macos) (pull_request) Has been cancelled
Details
WAI energy receipts / does this change touch it (pull_request) Has been cancelled
Details
WAI energy receipts / energy receipts · linux (pull_request) Has been cancelled
Details
WAI energy receipts / energy receipts · linux-arm64 (pull_request) Has been cancelled
Details
WAI energy receipts / energy receipts · macos (pull_request) Has been cancelled
Details
WAI conformance / does this change touch it (pull_request) Has been cancelled
Details
WAI conformance / cargo-class corpora · linux (pull_request) Has been cancelled
Details
WAI conformance / cargo-class corpora · linux-arm64 (pull_request) Has been cancelled
Details
WAI conformance / cargo-class corpora · macos (pull_request) Has been cancelled
Details
WAI conformance / quantum core and corpus under wasm32-wasip2 (wasmtime) (pull_request) Has been cancelled
Details
WAI conformance / v2 vocabulary · manifests validate as stated (pull_request) Has been cancelled
Details
WAI conformance / independent verifiers · claims, JWP, key discovery and integer payloads (Python) (pull_request) Has been cancelled
Details
WAI conformance / delivery telemetry · CMCD and CMSD against an independent library (pull_request) Has been cancelled
Details
WAI conformance / cross-arch byte-equality · linux (pull_request) Has been cancelled
Details
WAI conformance / cross-arch byte-equality · linux-arm64 (pull_request) Has been cancelled
Details
WAI conformance / cross-arch byte-equality · macos (pull_request) Has been cancelled
Details
WAI conformance / codec wrappers · fuzz smoke, text decode limits, C entry points, provenance, the semantic path (pull_request) Has been cancelled
Details
WAI conformance / C2PA manifest emission · signed, read back, validated (pull_request) Has been cancelled
Details
standards workspaces build + test / which workspaces this change touches (pull_request) Has been cancelled
Details
standards workspaces build + test / macOS energy backends (pull_request) Has been cancelled
Details
repository checks / lockfiles, tests in CI, workflow hygiene, spec and section references, public text (pull_request) Has been cancelled
Details
joule-moq-web browser client / joule-moq-web · does this change touch it (pull_request) Has been cancelled
Details
joule-moq-web browser client / joule-moq-web · browser end-to-end (macos) (pull_request) Has been cancelled
Details
joule-moq-room room service / joule-moq-room · does this change touch it (pull_request) Has been cancelled
Details
joule-moq-room room service / joule-moq-room, every target (pull_request) Has been cancelled
Details
joule-code conformance gate / Build + round-trip C1-C4 vectors (pull_request) Has been cancelled
Details
joule-code conformance gate / joule-moq-relay, every target (pull_request) Has been cancelled
Details
joule-code conformance gate / joule-moq-ingest, every target (pull_request) Has been cancelled
Details
23c9c09ae7 
The synthesis corpus's cases carry both priors and unmarked frames; the exact-upscale corpus's
policies carry synthesis's second_layer member. Written by wai_synthesis_vectors gen,
wai_upscale_vectors gen and wai registry export.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
dcharlot closed this pull request 2026-10-22 12:02:10 -04:00
Some checks failed
wai-web wasm32 / does this change touch it (pull_request) Has been cancelled
Details
wai-web wasm32 / wai-web wasm32 build · corpus, menu and writers (pull_request) Has been cancelled
Details
wai-web reach builds / wai-web reach · does this change touch it (pull_request) Has been cancelled
Details
wai-web reach builds / wai-web reach builds · the package of record, every output as recorded (pull_request) Has been cancelled
Details
wai-web reach builds / wai-web reach builds · this tree's wai-web, every rung on its engine line (pull_request) Has been cancelled
Details
wai-quantum studio and desktop / does this change touch it (pull_request) Has been cancelled
Details
wai-quantum studio and desktop / studio · browser (pull_request) Has been cancelled
Details
wai-quantum studio and desktop / desktop app · macos (pull_request) Has been cancelled
Details
wai-pack / wai-pack · does this change touch it (pull_request) Has been cancelled
Details
wai-pack / wai-pack · linux (pull_request) Has been cancelled
Details
wai-pack / wai-pack · linux-arm64 (pull_request) Has been cancelled
Details
wai-pack / wai-pack · macos (pull_request) Has been cancelled
Details
wai-pack / wai-pack · independent verifier (Python) (pull_request) Has been cancelled
Details
wai-pack / wai-pack · the floor through a media prober (pull_request) Has been cancelled
Details
wai-pack / wai-pack · dependency licences (pull_request) Has been cancelled
Details
wai-ops integer operators / does this change touch it (pull_request) Has been cancelled
Details
wai-ops integer operators / linux-aarch64 (pull_request) Has been cancelled
Details
wai-ops integer operators / linux-x86_64 (pull_request) Has been cancelled
Details
wai-ops integer operators / macos-arm64 (pull_request) Has been cancelled
Details
wai-ops integer operators / wasm32-wasip2 (wasmtime) (pull_request) Has been cancelled
Details
wai-ops integer operators / wasm32 under Node (pull_request) Has been cancelled
Details
wai-ops integer operators / independent checker (Python) (pull_request) Has been cancelled
Details
wai-ops-gpu integer operators on GPUs / does this change touch it (pull_request) Has been cancelled
Details
wai-ops-gpu integer operators on GPUs / builds · no adapter is not a pass (linux) (pull_request) Has been cancelled
Details
wai-ops-gpu integer operators on GPUs / Metal GPU and browser WebGPU (macos) (pull_request) Has been cancelled
Details
WAI energy receipts / does this change touch it (pull_request) Has been cancelled
Details
WAI energy receipts / energy receipts · linux (pull_request) Has been cancelled
Details
WAI energy receipts / energy receipts · linux-arm64 (pull_request) Has been cancelled
Details
WAI energy receipts / energy receipts · macos (pull_request) Has been cancelled
Details
WAI conformance / does this change touch it (pull_request) Has been cancelled
Details
WAI conformance / cargo-class corpora · linux (pull_request) Has been cancelled
Details
WAI conformance / cargo-class corpora · linux-arm64 (pull_request) Has been cancelled
Details
WAI conformance / cargo-class corpora · macos (pull_request) Has been cancelled
Details
WAI conformance / quantum core and corpus under wasm32-wasip2 (wasmtime) (pull_request) Has been cancelled
Details
WAI conformance / v2 vocabulary · manifests validate as stated (pull_request) Has been cancelled
Details
WAI conformance / independent verifiers · claims, JWP, key discovery and integer payloads (Python) (pull_request) Has been cancelled
Details
WAI conformance / delivery telemetry · CMCD and CMSD against an independent library (pull_request) Has been cancelled
Details
WAI conformance / cross-arch byte-equality · linux (pull_request) Has been cancelled
Details
WAI conformance / cross-arch byte-equality · linux-arm64 (pull_request) Has been cancelled
Details
WAI conformance / cross-arch byte-equality · macos (pull_request) Has been cancelled
Details
WAI conformance / codec wrappers · fuzz smoke, text decode limits, C entry points, provenance, the semantic path (pull_request) Has been cancelled
Details
WAI conformance / C2PA manifest emission · signed, read back, validated (pull_request) Has been cancelled
Details
standards workspaces build + test / which workspaces this change touches (pull_request) Has been cancelled
Details
standards workspaces build + test / macOS energy backends (pull_request) Has been cancelled
Details
repository checks / lockfiles, tests in CI, workflow hygiene, spec and section references, public text (pull_request) Has been cancelled
Details
joule-moq-web browser client / joule-moq-web · does this change touch it (pull_request) Has been cancelled
Details
joule-moq-web browser client / joule-moq-web · browser end-to-end (macos) (pull_request) Has been cancelled
Details
joule-moq-room room service / joule-moq-room · does this change touch it (pull_request) Has been cancelled
Details
joule-moq-room room service / joule-moq-room, every target (pull_request) Has been cancelled
Details
joule-code conformance gate / Build + round-trip C1-C4 vectors (pull_request) Has been cancelled
Details
joule-code conformance gate / joule-moq-relay, every target (pull_request) Has been cancelled
Details
joule-code conformance gate / joule-moq-ingest, every target (pull_request) Has been cancelled
Details

Pull request closed

Please reopen this pull request to perform a merge.
Sign in to join this conversation.
Reviewers
No reviewers
Labels
Clear labels
No items
No labels
Milestone
Clear milestone
No items
No milestone
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
Transaction-Science/open-standards!408
No description provided.