Transaction-Science/open-standards
Watch
1
0
Fork
You've already forked open-standards
 0
Code Issues Pull requests Projects Releases 17 Packages Wiki Activity Actions 1

wai: staged-delivery measurement — counted bytes, integer quality, a proved stall bound, and delivery telemetry #169

Merged
dcharlot merged 51 commits from wai/w5-staged-measurement into main 2026-10-06 05:13:29 -04:00
Conversation 0 Commits 51 Files changed 197 +22451 -99
197 changed files with 22451 additions and 99 deletions
Download patch file Download diff file Expand all files Collapse all files

77
.github/workflows/wai-conformance.yml vendored
View file

@ -141,11 +141,14 @@ jobs:
# is in the set because the reference meters (wai_meter, wai_world_meter) write a
# measurement report by default and require it: without it they would drop out
# of this build on every leg of this matrix, without a word. staged and the integer
# media features build the staged-delivery core, its corpus tool and `wai stage`.
run: cargo build --no-default-features --features world,audio_scene,haptic,splat4d,score,provenance,neural_int,film,receipt_claims,staged,video_int,audio_int,splat_int
# media features build the staged-delivery core, its corpus tool and `wai stage`;
# stage_measure builds the staged-measurement core and wai_stage_measure; telemetry
# the delivery-telemetry mapping and wai_telemetry.
run: cargo build --no-default-features --features world,audio_scene,haptic,splat4d,score,provenance,neural_int,film,receipt_claims,staged,video_int,audio_int,splat_int,stage_measure,telemetry
# video_package runs the package receipt's dedup tests, which content_dedup feeds.
- name: Library unit tests (the fixed-point determinism floors)
working-directory: wai/wai-rs
run: cargo test --lib --no-default-features --features world,audio_scene,haptic,splat4d,score,provenance,neural_int,film,oer,receipt_claims,key_discovery,staged,video_int,audio_int,splat_int
run: cargo test --lib --no-default-features --features world,audio_scene,haptic,splat4d,score,provenance,neural_int,film,oer,receipt_claims,key_discovery,staged,video_int,audio_int,splat_int,stage_measure,telemetry,video_package
# The JPEG AI exact decoder reproduces across platforms only if no platform math
# library is on its path. Two guards, each failing the step: clippy's
# disallowed-methods (wai-rs/clippy.toml), which each module on the decode path
@ -201,7 +204,11 @@ jobs:
# own binary, a counting allocator) checks that the staged sink's work and memory per
# layer grow linearly in the layer's units, that a layer past the chain budget is
# refused before any stream is decoded, that what no unit holds is dropped, and that a
# WIR1 parse holds no more than its declared working bytes. Cargo.toml declares them, and the other
# WIR1 parse holds no more than its declared working bytes. tests/stage_measure.rs counts every
# committed staged envelope as exactly its length, refuses JSON, base64 and another capability's
# format, and evaluates a measured title read from a map of files to its golden, refusing a
# changed byte of a layer, the base, the source or a parameter-set file (extensions/
# staged-measurement.md). Cargo.toml declares them, and the other
# feature-gated targets named here, with required-features, so if this step ever loses a
# feature, cargo stops with an error instead of compiling an empty binary that reports ok.
# Each committed vector's decoded output is also checked against the SHA-256 in
@ -211,24 +218,32 @@ jobs:
# wai.generate.spectral vector, which `generate` enables.
- name: Fast integer kernels ≡ reference kernels (overflow checks on)
working-directory: wai/wai-rs
run: cargo test --no-default-features --features neural_int,video_int,audio_int,world_int,splat_int,generate,staged --lib --test int_kernels_vectors --test int_kernels_geometry --test int_mlicpp --test int_mlicpp_preflight --test int_audio --test int_splat --test generate --test int_payload --test int_payload_preflight --test int_payload_cost --test int_video_enc_memory --test cli_stage --test staged_resources
run: cargo test --no-default-features --features neural_int,video_int,audio_int,world_int,splat_int,generate,staged,stage_measure --lib --test int_kernels_vectors --test int_kernels_geometry --test int_mlicpp --test int_mlicpp_preflight --test int_audio --test int_splat --test generate --test int_payload --test int_payload_preflight --test int_payload_cost --test int_video_enc_memory --test cli_stage --test staged_resources --test stage_measure --test stage_measure_bounds
- name: Fast integer kernels ≡ reference kernels (release, overflow checks off)
working-directory: wai/wai-rs
run: cargo test --release --no-default-features --features neural_int,video_int,audio_int,world_int,splat_int,generate,staged --lib --test int_kernels_vectors --test int_kernels_geometry --test int_mlicpp --test int_mlicpp_preflight --test int_audio --test int_splat --test generate --test int_payload --test int_payload_preflight --test int_payload_cost --test int_video_enc_memory --test cli_stage --test staged_resources
run: cargo test --release --no-default-features --features neural_int,video_int,audio_int,world_int,splat_int,generate,staged,stage_measure --lib --test int_kernels_vectors --test int_kernels_geometry --test int_mlicpp --test int_mlicpp_preflight --test int_audio --test int_splat --test generate --test int_payload --test int_payload_preflight --test int_payload_cost --test int_video_enc_memory --test cli_stage --test staged_resources --test stage_measure --test stage_measure_bounds
# The integer decode path builds under each of its features alone, so a sink can take
# one medium without the others, and for wasm32 (a browser sink); `int_payload` arms a
# medium only under its feature, and none needs `neural`.
# medium only under its feature, and none needs `neural`. So does the staged-measurement
# core (`stage_measure`), which pulls no ONNX and reads no file on wasm32.
- name: Each integer feature alone, and wasm32 (unix)
if: matrix.os != 'windows'
shell: bash
working-directory: wai/wai-rs
run: |
set -euo pipefail
for f in neural_int video_int audio_int splat_int staged; do
for f in neural_int video_int audio_int splat_int staged stage_measure telemetry; do
cargo check --lib --no-default-features --features "$f"
done
# The decode bench prints its machine line from bench_env; no other step builds it.
cargo build --no-default-features --features neural_int,video_int,audio_int,world_int,splat_int --bin wai_decode_bench
rustup target add --toolchain 1.98.0 wasm32-unknown-unknown
cargo check --lib --target wasm32-unknown-unknown --no-default-features --features video_int,audio_int,splat_int,staged
# The staged-measurement core is wasm32-clean: a browser computes the same figures.
# Built, not only checked, so codegen and linking of the library run for wasm32.
cargo build --lib --target wasm32-unknown-unknown --no-default-features --features stage_measure
# So is delivery telemetry: a browser sink writes the same reports.
cargo build --lib --target wasm32-unknown-unknown --no-default-features --features telemetry
# carriage: every registered capability round-trips byte-exact. cli_wrap_intent and
# cli_registry run the `wai` binary, which builds without the codecs feature: the
# intent rule of `wai wrap`, its refusal of candidate capabilities (SPEC §5), its media
@ -299,7 +314,7 @@ jobs:
working-directory: wai/wai-rs
run: |
set -euo pipefail
F=world,audio_scene,haptic,splat4d,score,provenance,neural_int,film,quantum,quantum_ops,oer,receipt_claims,key_discovery,video_int,audio_int,splat_int,staged
F=world,audio_scene,haptic,splat4d,score,provenance,neural_int,film,quantum,quantum_ops,oer,receipt_claims,key_discovery,video_int,audio_int,splat_int,staged,stage_measure,telemetry
v() { echo "::group::$1"; cargo run -q --no-default-features --features "$F" --bin "$2" -- verify "$3"; echo "::endgroup::"; }
v worlds wai_world_vectors ../world-conformance
v spatial-audio wai_audio_vectors ../audio-conformance
@ -319,6 +334,8 @@ jobs:
v signers wai_signer_vectors ../signer-conformance
v int-payload wai_int_payload_vectors ../int-payload-conformance
v staged wai_staged_vectors ../staged-conformance
v staged-measure wai_stage_measure ../staged-measure-conformance
v telemetry wai_telemetry ../telemetry-conformance
# The pin corpus's second implementation: Python standard library only, sharing no
# code with wai-rs. One leg is enough — it checks digests and listing rules, which
# do not depend on the architecture — and the macOS host has python3.
@ -347,7 +364,7 @@ jobs:
shell: pwsh
working-directory: wai/wai-rs
run: |
$F = "world,audio_scene,haptic,splat4d,score,provenance,neural_int,film,quantum,quantum_ops,oer,receipt_claims,key_discovery,video_int,audio_int,splat_int,staged"
$F = "world,audio_scene,haptic,splat4d,score,provenance,neural_int,film,quantum,quantum_ops,oer,receipt_claims,key_discovery,video_int,audio_int,splat_int,staged,stage_measure,telemetry"
function v($grp, $bin, $path) {
Write-Output "::group::$grp"
cargo run -q --no-default-features --features $F --bin $bin -- verify $path
@ -372,6 +389,8 @@ jobs:
v signers wai_signer_vectors ../signer-conformance
v int-payload wai_int_payload_vectors ../int-payload-conformance
v staged wai_staged_vectors ../staged-conformance
v staged-measure wai_stage_measure ../staged-measure-conformance
v telemetry wai_telemetry ../telemetry-conformance
# The generated v2 vocabulary (wai/vocab/v2.*) is checked as a consumer uses it:
# sample manifests, typed and untyped, WAI1 and WAI2, are expanded through the
@ -457,6 +476,22 @@ jobs:
working-directory: wai
shell: bash
run: python3 staged-conformance/staged_verify.py staged-conformance
# The staged-measurement corpus's second implementation (staged-measure-conformance/README):
# it counts every object, applies every layer from its base's stage 0, computes the quality
# integers, the output tree, the prefixes, the overhead points and the dedup totals itself,
# and requires its decode to be the fixture tool's and its figures to be goldens.json's.
# Standard library only.
- name: measure_verify.py
working-directory: wai
shell: bash
run: python3 staged-measure-conformance/measure_verify.py staged-measure-conformance
# The delivery simulator's second implementation: Appendix B from the text, every case
# of staged-measure-conformance/sim/ reproduced and its claim checked, with a
# counter-case required for each part of the stall bound a schedule can lose.
- name: sim_verify.py
working-directory: wai
shell: bash
run: python3 staged-measure-conformance/sim_verify.py staged-measure-conformance
- name: claim_verify.py, differential.py, jwp_verify.py, signer_verify.py
working-directory: wai
shell: bash
@ -470,6 +505,28 @@ jobs:
"$GITHUB_WORKSPACE/wai/wai-rs/target/debug/wai_signer_vectors" verdicts signer-conformance/vectors.json > "$RUNNER_TEMP/signer-rs.tsv"
diff "$RUNNER_TEMP/signer-py.tsv" "$RUNNER_TEMP/signer-rs.tsv"
# Delivery telemetry's second implementation is an independent open-source CMCD and CMSD
# library at locked versions (telemetry-conformance/cml/package-lock.json). The
# differential builds each observation's report from the extension's tables, encodes it
# with the library, requires the bytes of expected/cmcd.json and expected/cmsd.json, reads
# the expected strings back as dictionary, query and header shards, and validates every
# report with no error. One platform: string encoding, not arithmetic.
telemetry-differential:
name: delivery telemetry · CMCD and CMSD against an independent library
runs-on: linux
steps:
- uses: actions/checkout@v5
- uses: actions/setup-node@v4
with:
node-version: '22'
- name: npm ci, then the differential
working-directory: wai/telemetry-conformance/cml
shell: bash
run: |
set -euo pipefail
npm ci --no-audit --no-fund
node differential.mjs ..
byte-exact:
name: cross-arch byte-equality · ${{ matrix.os }}
runs-on: ${{ matrix.os }}

104
wai/CHANGELOG.md
View file

@ -227,6 +227,110 @@ wire format may change and such changes are recorded here.
`rdiv` and the rate indices. `rdiv` and the rate indices are computed in checked
64-bit arithmetic.
### Staged-delivery measurement: counted bytes, integer quality, the output tree
- **The extension** [staged-measurement](extensions/staged-measurement.md): how a
figure about staged delivery is measured and published. The counted bytes of an
object are its envelope to the byte (`12 + man_len + P` for `WAI1`,
`8 + man_len + 2 + 8n + Σ P` for `WAI2`), and an object is countable only where WAI
checks the format of every payload it carries: the pinned integer formats and `WIR1`,
by the parse a sink runs, for the capability and for a `WAI1` fallback with a pinned
format. Any other capability, a classical codestream or an unregistered string among
them, is not countable (`no-format-check`), and neither is a `WAI2` component whose
`sha256` is not its payload's, nor an encrypted payload; no estimate stands in for a
figure that is not countable. A figure carries the framing label `registered`. Quality is integers per
presentation and per prefix, against the source and against the final prefix: SSE
over RGB and over an integer BT.601 luma, a Q32 structural similarity over clipped
8×8 windows at stride 4 with an edge-aligned last window on each axis, so every pixel
is in a window (bounded in 128-bit integers), PCM SSE and reference energy,
and tensor SSE and largest difference, each pooled over units; decibels are display
only. First presentable, layering overhead as two rate–quality points (never a
ratio), delivery simulation, decode time and energy (only a clean attribution verdict is a figure; a rail
that is not live is unmetered), deduplication, dated records, what CI asserts, and a
media-class matrix are stated. The output tree (Appendix C) commits to every
presentation, one SHA-256 leaf per (unit, layers), under RFC 6962's tree hash, by its
root and leaf count together, with audit paths checked as RFC 9162 checks them under
the committed count (`output_tree::OutputCommitment`).
- **Measured titles** (Appendix D): a staged set over a staged-conformance base, read
where it is, with only what the stage model does not carry beside it (unit duration,
capture times, the source, single-layer references, the fixture tool's own
reconstruction digests), and a content identity over the measured files.
- **Reference implementation.** `wai-rs` feature `stage_measure` (pure integer, no
ONNX, wasm32-clean; CI builds it for wasm32): `measure_framing`, `output_tree`,
`stage_measure::{title, decode, run}`, which decodes each base through the sink's own
base decode, `staged::decode_base_pinned` (which also gives the pin it verified),
under the cost limit, and applies the layers through `staged::StageSink`
under the chain budget, handing each stage's output over as it is reached. The tools'
stated limits are `staged::TOOL_DEFAULT_LIMIT` and `staged::TOOL_DEFAULT_CHAIN_BUDGET`
(staged-measurement Appendix D states each figure and why); `wai stage` (`apply`,
`refine`, `base`), `wai_stage_measure eval` and `wai_stage_bench` decode every base
under them, hold every layer and chain to them, and take
`--limit OUT,SYMBOLS,WORKING,MACS`. A title's `limits` lower a limit and never raise
one: each figure is the lower of the title's and the tool's.
The corpus's `refusals/` hold titles a tool must refuse, with the code: a base past its
chain budget, the two declared-clip bound vectors (refused before anything is
allocated for them; a counting-allocator test shows it), the 1 GiB one again under a
title that states limits of 2^40, two inert envelopes, and a layer past its title's
cost limit. `tests/vectors/staged_bounds/` gains a base whose up2 layer declares
67,092,480 output bytes, which `wai stage apply` and `wai_stage_bench` refuse with no
flag. Always on:
`quality` and `content_dedup` (the package receipt's dedup figures now go through
it; a map that gives one content hash two lengths no longer verifies); native only:
`bench_env`, the machine record, which `wai_decode_bench` now prints from. The
`wai_stage_measure` tool evaluates a title (`eval`), verifies the corpus (`verify`)
and writes it (`gen-fixtures`, `generate-goldens`).
- **The corpus** [`staged-measure-conformance/`](staged-measure-conformance/): a
picture, a clip, an audio clip and a splat attribute tensor, each with its figures in
`goldens.json`; `wai_stage_measure verify` and an independent Python verifier,
`measure_verify.py`, both reproduce every figure, and CI runs both.
- **The delivery simulator** (§10, Appendix B; `stage_measure::sim`,
`wai_stage_measure sim`): a discrete-event model in integer microseconds of a sender
putting a title's objects on one link (`strict-priority` or `fifo`; `datagram` or
`subgroup` carriage with delivery timeouts that drop or reset; a layer cap; a VOD
window or live capture) and a sink decoding and presenting them (workers, `base-first`
or `fifo` decode, playout deadlines). Its figures are labelled simulated. Bases are
sent, and decoded under `base-first`, in the order they became available. A
refinement's stall of its base is then bounded by one chunk (strict priority) or a
timeout and one chunk (`fifo`), plus one refinement decode (`base-first`): §10.4
proves it, for every plan with no base timeout, a bounded link schedule and
`base-first` decode. It is also checked on generated plans drawn over that whole
range and on the corpus's plans, and for each unbounded part a counter-plan's stall
passes the bound of the same plan with that part restored. The corpus `sim/` and an independent
`sim_verify.py` reproduce every result.
- **Wire.** No envelope, payload, receipt, claim, pin or corpus file of an earlier
revision changes bytes.
### Delivery telemetry: CMCD version 2 and CMSD for WAI objects
- **The extension** [delivery-telemetry](extensions/delivery-telemetry.md): an unsigned,
advisory observation about WAI delivery (no joule figure, no acquisition class, no
quality figure; content and receipt hashes only as pointers), read strictly with
exact members; its CMCD version 2 report, in Request mode for requests and Event mode
otherwise (`e=rr` for responses, play states, custom events for switches, dispatch
outcomes, layer-cap changes and dropped frames), with header shards; custom keys
under `science.transaction.wai-`, each carrying a String; switch reasons; CMSD-Static
and CMSD-Dynamic for a WAI object; session metrics from one session's log (video
start time, exit before start, start failure, stalls, engagement, units, counted
bytes); and privacy rules: a content id is a content hash or a short token, and an
emitter puts no address, query, token or personal data in it; an object URL holding a
`?`, `#`, `@` or `\` is refused rather than stripped, so no query, signed-URL token or
user information enters an observation however a parser splits the URL; and a
Request-mode report goes only to the host serving the object. Cohort assignment and capable share are not in this revision.
- **Reference implementation.** `wai-rs` feature `telemetry` (no dependency;
wasm32-clean): `telemetry::{sf, observation, cmcd, cmsd, qoe}`, with the RFC 9651
subset the reports use; the `wai_telemetry` tool.
- **The corpus** [`telemetry-conformance/`](telemetry-conformance/): observations of
every event kind with their reports, refused observations, CMSD descriptions and
session logs. `wai_telemetry verify` requires every value; a differential encodes
every report with an independent open-source CMCD and CMSD library at locked versions
and requires the same bytes, reads them back, and validates them; CI runs both.
- **Other text.** [moq-streaming-format](extensions/moq-streaming-format.md) §5 requires
that telemetry **MUST NOT** restate the per-object property's figure, class or evidence;
[energy-exposure-mapping](research/energy-exposure-mapping.md) gains a dated
addendum.
- **Wire.** No envelope, payload, receipt, claim or corpus file of an earlier revision
changes bytes.
### The integer codecs' payloads pinned in place: `WIV1`, `WIH1`, `WIA1`, `WIS1`
- **Pinned in place** (SPEC §5, *Payloads of the WAI-native integer codecs*, and the

248
wai/extensions/delivery-telemetry.md Normal file
View file

@ -0,0 +1,248 @@
# WAI Extension: Delivery Telemetry
> Status: Draft. An **unsigned, advisory** observation about WAI delivery that a sink,
> edge or origin emits; its mapping onto the keys of CMCD version 2 (CTA-5004-B) and
> CMSD (CTA-5006), written as RFC 9651 Structured Field Values; and the session
> metrics a log of observations gives. Reference impl: `wai-rs` —
> `telemetry::{sf, observation, cmcd, cmsd, qoe}`; the `wai_telemetry` tool; the corpus
> [`telemetry-conformance/`](../telemetry-conformance/), with a differential against an
> independent open-source CMCD and CMSD library. Keywords **MUST**, **MUST NOT**,
> **SHOULD**, **MAY** are RFC 2119/8174.
## 1. Status of an observation
§1.1 No signature covers an observation. A relying party **MUST NOT** treat one as
evidence of delivery, presentation, quality or energy. What a session did is attested,
where it is, by receipts and claims ([jwp-receipts](jwp-receipts.md)), never by
telemetry.
§1.2 An observation **MUST NOT** carry a joule figure, an acquisition class or a quality
figure. It **MAY** carry content hashes and link hashes as pointers.
§1.3 Telemetry binds no receipt to CMCD or CMSD, and no receipt to telemetry.
## 2. The observation
§2.1 An observation is a JSON object with exactly these members; a member that does not
apply is `null`. Integers are unsigned and at most 2⁵³ − 1, and a member a report
carries as an Integer (§3.3: `sn`, `ts_ms`, the object's `d_ms`, `dl_ms`, `ttfb_ms`,
`ttlb_ms` and `rc`, and the metrics) at most 10¹⁵ − 1, the largest RFC 9651 Integer, so
every observation can be reported. A string a report carries is printable ASCII.
| member | type | meaning |
|---|---|---|
| `v` | integer | `1` |
| `sid` | string | 32 lowercase hexadecimal digits, fresh per session, derived from no identifier (§8) |
| `sn` | integer | the observation's sequence number in the session |
| `ts_ms` | integer or `null` | Unix time of the event in milliseconds; required for every event reported in Event mode (§3.1) |
| `event` | string | `request`, `response`, `start`, `present`, `switch`, `layer-change`, `stall-start`, `stall-end`, `frame-drop`, `capable`, `abandon` or `end` |
| `cid` | string or `null` | the entry object's content hash (64 lowercase hexadecimal digits), or a deployer's content id of 1–64 letters, digits, `.`, `_` and `-` |
| `st` | string | stream type: `v` (on demand), `l` (live) or `ll` (low-latency live) |
| `sf` | string | `o` for WAI objects over the HTTP binding; `h` or `d` for a WAI rendition referenced from HLS or DASH |
| `object` | object or `null` | for `request` and `response`: the object (§2.2) |
| `present` | object or `null` | for `present`: `{ unit, layers, first, late_ms, decode_us }`; `first` is `true` only for the session's first presentation; `layers` is at least 1 |
| `switch` | object or `null` | for `switch`: `{ unit, from, to, reason }`; `from` and `to` are `primary` or `fallback` and differ; `reason` is a §5 term |
| `layer_change` | object or `null` | for `layer-change`: `{ from, to }`, a change of `layer_cap`; `null` is no cap |
| `capable` | object or `null` | for `capable`: `{ outcome, reason, classical_floor }` (§2.3) |
| `cohort` | string or `null` | `control` or `wai`: the arm the content's service assigned the session to |
| `metrics` | object or `null` | `{ msd_ms, bsa, bsda_ms, dfa }`: media start delay, buffer starvations and their total duration since the session began, and dropped frames since the session began |
| `claim` | 64 hexadecimal digits or `null` | on `end` only: the link hash of the session's `reconstruct-session` claim, where the sink sealed one |
§2.2 **The object** has exactly the members `ot`, `url`, `content_hash`, `capability`,
`rendition`, `layer`, `layers_total`, `layer_cap`, `counted`, `transferred`, `d_ms`,
`dl_ms`, `ttfb_ms`, `ttlb_ms`, `rc`, `path` and `receipt`, each `null` where it does
not apply:
- `ot` is a CTA-5004-B object type token (`m`, `a`, `v`, `av`, `i`, `c`, `tt`, `k`,
`o`); `content_hash` the object's content hash and `receipt` a receipt's link hash,
each 64 lowercase hexadecimal digits; `capability` at most 64 characters;
`rendition` the `WAI2` rendition index; `layer` and `layers_total` the object's
stage and its stage set's stage count; `layer_cap` the cap the request asks for;
`counted` its counted bytes and `transferred` its transferred bytes
([staged-measurement](staged-measurement.md) §3); `path` `primary` or `fallback`;
`url` the object's address, at most 2048 characters, holding no `?`, `#`, `@` or `\`
(§8.2).
- The request keys `ot`, `d_ms`, `dl_ms` and `layer_cap` are `null` on a `response`;
the response keys `rc`, `ttfb_ms`, `ttlb_ms` and `url` are `null` on a `request`.
§2.3 **A `capable` observation** reports one WAI-arm dispatch outcome: `not-reached`
(no dispatch decision for a WAI object was reached), `missing-capability`,
`pin-refused`, `module-refused`, `below-rate` or `capable`. `reason` is SPEC §3.1's pin
refusal for `pin-refused` (one of the terms §5 lists before `missing-capability`), and
`null` otherwise. `classical_floor` is the platform's answer for the classical
rendition's codec string, `supported` or `unsupported`, where the sink asked. The
outcome comes from WAI's own dispatch: advertised capabilities, pin verification and a
timed decode. A sink **MUST NOT** derive it from a platform media-capability query,
which answers only for the platform's own decoders; it reports that answer separately,
as `classical_floor`. How sessions are assigned to arms, and the shares computed over
outcomes, are not specified in this revision.
§2.4 An observation that breaks §2.1–§2.3 is refused as a whole: a member missing or
not listed, a value of another type or outside its range, an object on an event that
has none, or none on one that has one.
## 3. CMCD version 2 mapping
§3.1 The modes:
- a `request` observation is reported in Request mode;
- every other observation is reported in Event mode, with `ts`: a `response` as an
event report with `e=rr`, whose response keys (`rc`, `ttfb`, `ttlb`, `url`) are never
carried in Request mode or in a header shard;
- a `present` observation with `first: false` is not reported in CMCD.
Every report carries `v=2`, `sid`, `sn`, `st` and `sf`, and `cid` where it is not `null`.
| observation | CMCD | mode |
|---|---|---|
| `sid`, `sn`, `cid`, `st`, `sf` | same keys | every report |
| `object.ot`, `object.d_ms`, `object.dl_ms` | `ot`, `d`, `dl` | request |
| `object.rc`, `object.ttfb_ms`, `object.ttlb_ms`, `object.url` | `rc`, `ttfb`, `ttlb`, `url`, with `e=rr` | event |
| `ts_ms` | `ts` | event |
| `start` / `end` / `abandon` | `e=ps`, `sta=s` / `e` / `q` | event |
| `present` with `first: true` | `e=ps`, `sta=p` | event |
| `stall-start` / `stall-end` | `e=ps`, `sta=r` / `p` | event |
| `metrics.msd_ms`, `metrics.dfa` | `msd`, `dfa` | any report |
| `metrics.bsa`, `metrics.bsda_ms` | `bsa`, `bsda`, each an inner list of one Integer | any report |
| `switch`, `capable`, `layer-change`, `frame-drop` | `e=ce`, `cen` = `"wai-switch"`, `"wai-capable"`, `"wai-layer-change"`, `"wai-frame-drop"` | event |
§3.2 An emitter **MUST NOT** put counted bytes, a layer or a capability into `br`, `tb`,
`pb` or any other bitrate key.
§3.3 A report is an RFC 9651 Dictionary: Integers for counts and times, Tokens for
`st`, `sf`, `ot`, `e` and `sta`, Strings for `sid`, `cid`, `url` and `cen`. Its keys are
written in bytewise order and without optional whitespace, and a key is written only
when its value is not `null`. The corpus fixes the output byte for byte.
## 4. Custom keys
§4.1 Custom keys have the form `science.transaction.wai-<x>`: a lowercase first letter,
then lowercase letters, digits, `.` and `-`, with an inner hyphen (CTA-5004-B's
custom-key rule). Every custom key's value is a String, the type CTA-5004-B admits for
custom keys besides a Token; an integer member is written as its decimal digits.
| key | from | report | header shard (Request mode) |
|---|---|---|---|
| `science.transaction.wai-b` | `object.counted` | request, `e=rr` | `CMCD-Object` |
| `science.transaction.wai-tx` | `object.transferred` | request, `e=rr` | `CMCD-Object` |
| `science.transaction.wai-ch` | `object.content_hash` | request, `e=rr` | `CMCD-Object` |
| `science.transaction.wai-cap` | `object.capability` | request, `e=rr` | `CMCD-Object` |
| `science.transaction.wai-ri` | `object.rendition` | request, `e=rr` | `CMCD-Object` |
| `science.transaction.wai-l` | `object.layer` | request, `e=rr` | `CMCD-Object` |
| `science.transaction.wai-lt` | `object.layers_total` | request, `e=rr` | `CMCD-Object` |
| `science.transaction.wai-p` | `object.path`, or `switch.to`: `"p"` or `"f"` | request, `e=rr`, `wai-switch` | `CMCD-Object` |
| `science.transaction.wai-rcpt` | `object.receipt` | request, `e=rr` | `CMCD-Object` |
| `science.transaction.wai-lc` | `object.layer_cap`, or `layer_change.to` | request, `wai-layer-change` | `CMCD-Request` |
| `science.transaction.wai-lcf` | `layer_change.from` | `wai-layer-change` | — |
| `science.transaction.wai-coh` | `cohort` | every report | `CMCD-Session` |
| `science.transaction.wai-u` | `present.unit`, or `switch.unit` | `sta=p` (first presentation), `wai-switch` | — |
| `science.transaction.wai-pl`, `-late`, `-dec` | `present.layers`, `present.late_ms`, `present.decode_us` | `sta=p` (first presentation) | — |
| `science.transaction.wai-sw` | `switch.reason` | `wai-switch` | — |
| `science.transaction.wai-co`, `-cr`, `-cf` | `capable.outcome`, `capable.reason`, `capable.classical_floor` | `wai-capable` | — |
| `science.transaction.wai-clm` | `claim` | `sta=e` | — |
§4.2 In Request mode the report also goes in header shards: each key to the shard
CTA-5004-B assigns it (`v`, `sid`, `cid`, `st`, `sf` and `msd` to `CMCD-Session`; `sn`,
`dl` and `dfa` to `CMCD-Request`; `ot` and `d` to `CMCD-Object`; `bsa` and `bsda` to
`CMCD-Status`), and each custom key to the shard of §4.1. Each shard is a Dictionary
written as §3.3 writes a report; a shard with no key is not sent.
## 5. Switch reasons
§5.1 A switch reason is one of:
- SPEC §3.1's pin refusals, in the order of its checks: `pin-malformed`,
`form-not-registered`, `form-unreadable`, `form-required`, `prior-revoked`,
`prior-not-authorised` (check 5, at a sink that holds a key log), `prior-not-held`,
`digest-superseded`, `set-malformed`, `part-mismatch`, `set-incomplete`;
- `missing-capability`;
- `module-unverified`, `module-revoked`, `trap`, `fuel-exhausted`, `memory-exhausted`,
`deadline-threshold`, `unit-hash-mismatch`, `budget-exhausted`;
- `recovered`.
A later extension adds its reasons here.
## 6. CMSD mapping
§6.1 An origin or edge **MAY** describe a WAI object in `CMSD-Static` with `at`, `d`,
`ht`, `nor`, `nrr`, `ot`, `sf`, `st` and `su`, and the custom keys
`science.transaction.wai-ch` (its content hash), `-l` and `-lt` (its layer and its
stage set's stage count) and `-rcpt` (a delivery receipt's link hash), each a String.
`v` is not written for version 1.
§6.2 A deriving node adds `science.transaction.wai-op`, its operation token, as a Token.
§6.3 `CMSD-Dynamic` **MAY** state, for each node, `etp`, `rtt`, `mb`, `rd` and `du`, and
`science.transaction.wai-drop` (a String): the number of objects the node discarded on
delivery timeout since its previous response to the same session. Each node adds its
own entry, in order; within an entry the parameters are in bytewise order.
§6.4 Keys and parameters are written as §3.3 writes a report: in bytewise order, a key
only when its value is not `null`, and a Boolean (`su`, `du`) only when true.
§6.5 CMSD carries no joule figure. An edge's energy travels in a claim.
## 7. Session metrics
§7.1 A **log** is one session's observations in `sn` order: one `sid`, `sn` strictly
increasing, timestamps never decreasing, and nothing after `end` or `abandon`.
§7.2 A log gives, in integer milliseconds and bytes:
- `started`: the log has a `start`;
- `vst_ms`, the video start time: the first presentation's `ts_ms` less the `start`'s,
`null` without both;
- `exit_before_start`: an `abandon` before any first presentation;
- `start_failed`: a session that started, never presented, and ended with `end`;
- `stalls` and `stall_ms`: the stalls begun after the first presentation, each from its
`stall-start` to its `stall-end`, or to `end` or `abandon` where it is still open,
and their total duration;
- `engaged_ms`: from the first presentation to `end` or `abandon`;
- `units`: the distinct units presented;
- `counted`: the counted bytes of every `response`;
- `claim`: the `end` observation's `claim`.
§7.3 These are statements of the emitter, as the observations are (§1.1).
## 8. Privacy
§8.1 `sid` is fresh per session. An emitter **MUST NOT** add a persistent identifier, or
send telemetry to a party other than the content's own service, without the person's
agreement.
§8.2 Neither a URL's query nor a signed URL's token enters an observation. A reader
refuses an `object.url` that holds a `?`, a `#`, a `@` or a `\`: a query, a fragment,
user information, or a separator some URL parsers read as `/` and so a way to user
information past a check for it. It refuses rather than strips them, and a path that
holds a `@` is refused with the rest. An emitter whose URLs carry a token in their
path **MUST** report `url` as `null`. `cid`'s syntax (§2.1) admits a content hash or
a short identifier, and no syntax keeps an address, a token or personal data out of an
identifier: an emitter **MUST NOT** put an address, a query, a token or personal data
in `cid`, and reports the content hash where it has no identifier free of them.
§8.3 A Request-mode report travels with its request, to the host that serves the object
and to no other party. Event-mode reports go to the content's own service (§8.1).
§8.4 A published aggregate covers at least 50 sessions per cell. A cell with fewer is
suppressed, not rounded.
## 9. Conformance
§9.1 [`telemetry-conformance/`](../telemetry-conformance/) holds:
- observations of every event kind, with their expected CMCD reports and header shards
(`expected/cmcd.json`);
- observations every reader refuses (`invalid/`);
- CMSD-Static and CMSD-Dynamic descriptions with their expected fields
(`expected/cmsd.json`);
- session logs with their expected metrics (`expected/qoe.json`).
§9.2 `wai_telemetry verify` requires every expected value byte for byte, and every
`invalid/` observation refused.
§9.3 `cml/differential.mjs` builds each report from this extension's tables, encodes it
with an independent open-source CMCD and CMSD library at the versions
`cml/package-lock.json` locks, and requires the expected bytes; reads the
expected strings back as a dictionary, as a query string and as header shards, and
requires the report's members; and validates every report, and the Event-mode reports
together, with no error.

3
wai/extensions/moq-streaming-format.md
View file

@ -145,6 +145,9 @@ commitment:
claim, a relay that rewrites this property included.
- A relay **MUST** preserve this property unchanged, per `jwp-receipts` §4.1.
A relay that recomputes it is asserting something it did not measure.
- Telemetry about objects on a WAI track is
[delivery-telemetry](delivery-telemetry.md)'s. It is unsigned and advisory, and it
**MUST NOT** restate this property's figure, class or evidence.
- Absence of the property is not evidence of absence of a receipt; receipts MAY
ride elsewhere (§1 of `jwp-receipts` lists sidecar and trailer carriage). A
sink **MUST NOT** infer that an object without the property is unattested.

671
wai/extensions/staged-measurement.md Normal file
View file

@ -0,0 +1,671 @@
# WAI Extension: Staged-Delivery Measurement
> Status: Draft. How a figure about staged delivery
> ([staged-delivery](staged-delivery.md)) is measured and published: which bytes are
> counted, how quality is computed per prefix, what "first presentable" means, how
> layering overhead is stated, how decode time and energy are reported, how
> deduplication is computed, how delivery is simulated, and what a record carries.
> Reference impl: `wai-rs` — `measure_framing`, `quality`, `output_tree`,
> `content_dedup`, `bench_env`, `stage_measure::{title, decode, run, sim}`; the
> `wai_stage_measure` tool; the corpus
> [`staged-measure-conformance/`](../staged-measure-conformance/). Keywords **MUST**,
> **MUST NOT**, **SHOULD**, **MAY** are RFC 2119/8174.
## 1. Scope
§1.1 This extension binds any party that publishes a figure about WAI staged or split
delivery: bytes, time, quality, overhead, decode time, energy or deduplication. It
binds the reference tools. It does not bind a sink's decode behaviour, which
[staged-delivery](staged-delivery.md) states.
§1.2 A figure that does not meet this extension **MUST NOT** be presented as a WAI
measurement.
§1.3 This extension states no price, and a record **MUST NOT** carry one. A deployer
that derives a cost from transferred bytes does so outside WAI.
## 2. Terms
| term | meaning |
|---|---|
| **title** | What a session plays: one or more bases and the layers that refine them ([staged-delivery](staged-delivery.md) §1). |
| **unit** | A presentable output, as staged-delivery §2 fixes it per capability: a frame, a picture, an audio clip, an attribute tensor. A title numbers its units globally: a base's unit `i` is the title's unit `first_unit + i`. |
| **stage** | A unit's stage: 0 after its base decodes, `k` after a layer whose `refines.layer` is `k` applies to it. |
| **prefix** | The stages `0…k` of a title: every object whose layer is at most `k`. Its **layers** count is `k + 1`. |
| **presentation** | A (unit, layers) pair a sink handed to output, with the output it presented. |
| **counted bytes** | Bytes under §3. Apart from transferred bytes, the only byte figure this extension reports. |
| **transferred bytes** | Bytes of WAI objects as a sink received them, whole or as byte ranges, without transport or receipts. |
A stage's position is its layer (0 for the base). A prototype MoQ binding in this
repository carries it as `mrl_layer`, a stage set's stage count as `mrl_total_layers`,
and a subscriber's upper bound on the layers it accepts as `max_mrl_layer`; this
extension names that bound `layer_cap`. Staged-delivery §13 says which of them a sink
reads.
## 3. Byte accounting
§3.1 **Counted bytes of an object.** For a `WAI1` envelope, `12 + man_len + P`; for a
`WAI2` envelope, `8 + man_len + 2 + 8·n + Σ Pᵢ`. Here `man_len` is the manifest's
length as written, `n` is the component count, and `P` or `Pᵢ` is each payload's
length (SPEC §2, §7.1). These are the envelope's bytes, exactly: an object with bytes
after its last payload is not the envelope its lengths describe, and is not counted.
Transport bytes (HTTP, QUIC, MoQ object headers, TLS) are not counted bytes; a figure
that includes them **MUST** state them separately.
§3.2 **Only a checked format is counted.** An object is countable only where WAI checks
the format of every payload it carries: a capability SPEC §5 pins to a binary format
(`WIV1`, `WIH1`, `WIA1`, `WIS1`, and `WIR1` for a refinement-only capability; Appendix
A) whose payload parses as that format, by the parse a sink runs. Where a `WAI1`
manifest names a fallback with a pinned format, the payload must parse as the
fallback's format too, since the fallback decodes the same payload (SPEC §4 step 5).
Otherwise the object is **not countable**, with a code:
- `no-format-check`: a capability WAI checks no format for, a classical codestream or
a string WAI does not register among them. Nothing in WAI tells such a payload's bytes
from a text encoding of them (a JSON text, base64, JSON arrays of byte values), which
**MUST NOT** be counted, as stored or as an estimate of what it would be in binary;
- the format parse's code (`bad_magic`, `format_capability_mismatch`, …:
[integer-payloads](integer-payloads.md) §8, staged-delivery A.6), for a payload that
is not its capability's or its fallback's format. A JSON form of a pinned format, its
base64 text, a JSON string, or the format's bytes behind a byte-order mark all fail
here;
- `sha256-mismatch`: a `WAI2` component whose `sha256` is not its payload's (SPEC §7.1);
- `encrypted`: an encrypted payload, whose plaintext's format cannot be checked.
No byte figure is published for an object that is not countable. The JSON that the
integer codecs' encode and export tools write is a converter input
([integer-payloads](integer-payloads.md) §10); its counted bytes are those of the
envelope that carries the converted payload.
§3.3 **Framing label.** A byte figure carries a framing label. A figure is
`registered` when every payload it counts is in its capability's registered format,
which §3.2 requires of every figure this revision defines. A later revision that
counts a payload under another framing names its label. Two figures with different
framing labels **MUST NOT** be compared, summed or placed on one curve.
§3.4 A record **MUST** state:
- the counted bytes of each object, split into envelope header, manifest and payload;
- the sums per (base, stage) and per (base, prefix), and over the title per stage and
per prefix;
- the framing label;
- the bytes of each base's parameter set, separately (staged-delivery §11).
§3.5 Receipt and claim bytes are counted separately, as receipt bytes.
## 4. Quality per prefix
§4.1 Quality is stated per presentation and per prefix against a **reference**: the
source, where the title pins it by SHA-256, and the final prefix's output, labelled
"against the final prefix". A figure against the final prefix **MUST NOT** be
presented as distortion against the source. A unit whose shape at a stage differs from
the reference's (below an up2 or a rate layer) has no figure at that stage; a figure
pooled over a prefix covers the units that have one, and says how many.
§4.2 **Pictures and frames** (`rgb8`). For two RGB8 buffers of width `W ≥ 1` and
height `H ≥ 1`:
- `sse_rgb` = Σ (aᵢ − bᵢ)² over the `3·W·H` bytes;
- `sse_y` is the same sum over the luma planes, with
`Y = (77·R + 150·G + 29·B + 128) >> 8`;
- `ssim_y_q32` is the mean structural similarity of the luma planes in Q32 fixed
point (`2³²` is 1).
The similarity's windows measure `ww = min(8, W)` by `wh = min(8, H)` pixels. Their
left edges are at every multiple of 4 up to `W − ww`, and at `W − ww` itself when it is
not a multiple of 4; their top edges are at every multiple of 4 up to `H − wh`, and at
`H − wh` itself when it is not one. A window is at every pair of a left and a top edge,
so every pixel is in at least one window. For a window of `n = ww·wh` pixels, with `Sx = Σx`,
`Sy = Σy`, `Sxx = Σx²`, `Syy = Σy²` and `Sxy = Σxy` over the reference's luma `x` and
the test's luma `y`:
```
A1 = 20000·Sx·Sy + 65025·n²
A2 = 20000·(n·Sxy − Sx·Sy) + 585225·n²
B1 = 10000·(Sx² + Sy²) + 65025·n²
B2 = 10000·((n·Sxx − Sx²) + (n·Syy − Sy²)) + 585225·n²
s_w = ⌊ A1·A2·2³² / (B1·B2) ⌋ (floor toward −∞)
ssim_y_q32 = ⌊ (Σ_w s_w) / (window count) ⌋ (floor toward −∞)
```
This is the structural similarity of Wang et al. (2004) with K1 = 0.01, K2 = 0.03, a
peak of 255, uniform windows at stride 4 with an edge-aligned last window on each
axis, and population statistics, multiplied out so
that no floating-point operation occurs. Because `n ≤ 64`, `|A1·A2·2³²| < 2¹¹⁵` and
`B1·B2 > 0` (the reference implementation's `quality` module proves both), so the
computation fits signed 128-bit integers.
§4.3 **Audio** (`pcm16`). For two sequences of `i16` samples, channel-interleaved and
in output order: `sse` = Σ (aᵢ − bᵢ)² and `ref_energy` = Σ aᵢ² over the reference.
Both are unsigned integers of at least 96 bits, written in records as decimal
strings. Sequences of different lengths are refused.
§4.4 **Tensors** (`tensor-i64`), for splat attributes, volumetric attributes and
haptic samples: `sse` over the capability's canonical integer output, written as a
decimal string, and `max_abs` = max |aᵢ − bᵢ|.
§4.5 **Pooling.** A figure over several units (a clip, or a prefix) sums each sum over
the units; for pictures, `ssim_y_q32` is the mean of every window of every unit, one
division over the pooled `Σ s_w` and window count, and `max_abs` is the largest.
§4.6 **Exact outputs** (`state-exact`), for worlds, feeds, scores, avatars, quantum
and every capability whose output is a state: a prefix is `equal` or `differs`, by
SPEC §7 *Digests*. It is never stated as a distance.
§4.7 PSNR = `10·log10(255²·N / sse)` and SNR = `10·log10(ref_energy / sse)` are derived
for display only. A record **MUST** carry the integers. An `sse` of 0 has no finite
PSNR, and is shown as exact.
§4.8 A record **MAY** add another full-reference metric. It names the metric's
implementation and version, and the metric is informative only.
## 5. First presentable
§5.1 `bfpf_min` is the sum of the counted bytes of the smallest set of objects needed
to present unit 0 at one layer: the base that holds it, and every index or manifest
object a sink reads first. It is computed from the title alone. A base's parameter
set is stated beside it (§3.4), not in it.
§5.2 `bfpf_sched` is the counted bytes that had arrived when unit 0 was first
presented, under a stated delivery schedule (§10). It is at least `bfpf_min`.
§5.3 `tfpf` is the time from the session's first request to the first presentation of
unit 0, in microseconds. Under §10 it is virtual and deterministic. Measured, it is a distribution over repeated runs, with the
clock and the presentation event named; in a browser that event is the frame callback
after the canvas write.
§5.4 `tffs_k` is the time to the first presentation of unit 0 at `k + 1` layers, stated
as §5.3 states `tfpf`.
## 6. Layering overhead
§6.1 A figure that states a saving from staging **MUST** be accompanied by its layering
overhead.
§6.2 The **single-layer reference** is the same content coded by the same staged form
as one layer at the final layer's operating point: the title's bases and, over each,
one layer straight from stage 0 to the source, with the final layer's step and the
predictor that reaches the final shape. A reference at further operating points is
the same with another step.
§6.3 The overhead is two rate–quality points: (counted bytes of the full title, quality
of its final prefix against the source) and (counted bytes of the reference, its
quality against the source). A record **MUST NOT** reduce them to a ratio, and
**MUST NOT** say "at equal quality" unless the two quality integers are equal. It
**SHOULD** add the reference at further operating points as a curve, and state which
two reference points bracket the quality of the final prefix.
## 7. Decode time
§7.1 Decode time is measured per (unit, stage) on a stated machine and build. A record
states the runs; the minimum, median, 10th and 90th percentiles; the order in which
cases ran; and the discarded warm-up runs.
§7.2 Cases **MUST** be interleaved, with the order rotated each round.
§7.3 A run whose output digest differs from the reference is not a timing; the bench
aborts.
§7.4 A record states the machine (its processor, cores, operating system and
architecture, and the build profile), its load before and after each block, its power
source, and whether it was otherwise idle. Continuous integration **MUST NOT** assert
any timing.
## 8. Energy
§8.1 An energy figure **MUST** follow [energy-measurement](energy-measurement.md):
an acquisition class (§2); for a figure obtained under its §3, the bracket a
`measurement` claim publishes (§6); the build (§5); and a recomputable work unit. For
staged decode, the work unit is the (unit, stage) decodes of one completion.
§8.2 A figure the meter could not obtain is **unmetered**: it is written as `null`
with a reason, never as 0.
§8.3 Only a figure whose attribution verdict (energy-measurement §4, §6.2) is clean is
published as a figure. A bracket usable for relative comparison **MAY** be shown,
labelled "relative comparison only, not a figure", and never as an absolute joule
figure. Noisy and unusable brackets are kept in the record and not shown.
§8.4 A figure covers decode on the measured machine's processor rails. A record **MUST
NOT** state, imply or chart energy for displays, radios, networks or other devices. A
reading from a whole-system rail that includes a display is not published.
§8.5 A rail that is not live in a bracket (energy-measurement §3.1) gives no figure; the
decode is unmetered.
## 9. Deduplication
§9.1 Over a set of objects:
- `objects` is their count, a repeated object counted each time;
- `unique` is the count of distinct content hashes ([jwp-receipts](jwp-receipts.md)
§2.1);
- `referenced_bytes` is the sum of their counted bytes;
- `stored_bytes` is the sum over one object per content hash.
All four are recomputed, never declared. One content hash has one length; a set that
gives one hash two lengths has no totals.
§9.2 Deduplication is stated per title, over every object the title and its
single-layer references reference (a reference's bases counted again), and, where
content repeats across titles, across the record.
## 10. Delivery simulation
§10.1 A deterministic delivery figure comes from the **staged delivery simulator**, a
discrete-event model in integer microseconds defined by its model and plan
(Appendix B). Its figures are labelled "simulated" with the plan. A simulated figure
is never a measurement of a network, and is never presented as one.
§10.2 **Delivery timeout.** A plan gives each object a carriage, after the
`DELIVERY TIMEOUT` of MoQ Transport (draft-ietf-moq-transport):
- `datagram`: each object is one datagram, and one not started by its deadline (its
availability plus its timeout) is **dropped**.
- `subgroup`: each (base, layer) is one subgroup, whose objects are sent in order. When
an object of a subgroup is not fully sent by its deadline, the subgroup is reset: the
bytes already sent of it and of every later object of the subgroup are **abandoned**,
and the bytes not yet sent are **dropped**, whatever the later objects' own
deadlines.
Dropped and abandoned bytes are never delivered, and every offered byte is delivered,
dropped or abandoned.
§10.3 **Layer cap.** An object whose layer exceeds `layer_cap` is never sent and never
counted as offered.
§10.4 **A refinement never stalls its base beyond a bound.** A plan's *base-only*
model removes every refinement. The **stall** of a unit is its base presentation time
under the plan minus the same time under the base-only model. The stall bound is
`S = S_link + S_decode`:
- `S_link`: under `strict-priority`, the serialisation time of one chunk; under `fifo`
with a refinement timeout `T`, `T` plus one chunk; otherwise there is no bound.
- `S_decode`: under `base-first` decode, one refinement's decode time; otherwise there
is no bound.
**Preconditions.** The bound holds for every model and every plan with no base
timeout (`delivery_timeout_us.base` is `null`), a bounded link schedule (`strict-priority`,
or `fifo` with a refinement timeout) and `base-first` decode, whatever else the plan
states (carriage, window, live captures, layer cap, workers, decode times, playout
deadlines). A plan outside them has no bound: a base its refinements delay past its own
timeout is not presented at all, and the two unbounded parts above have counter-plans.
**Proof.** Write `c` for one chunk's serialisation time, `⌈chunk_bytes·1000 /
bytes_per_ms⌉`, the longest any chunk takes; `L = c` under `strict-priority` and
`L = T + c` under `fifo`; and `d_r` for one refinement's decode time. A *base* below is
one layer-0 object (a base split into several layer-0 objects contributes each).
Number the bases `b_1, b_2, …` in the order the sender sends them. That order is
greedy under subgroup precedence: whenever the sender starts a base, it takes, of the
bases available then and not held behind an unfinished earlier object of their
subgroup (§10.2; under `subgroup`, a base's layer-0 objects form one subgroup, sent in
title order), the least by the order key (Appendix B, *Order*), whose first part is the
availability under either schedule. It is the same order under the plan and under its
base-only model. By induction on `k`: when either starts its `k`-th base, the bases
finished are `b_1, …, b_{k−1}` in both, so the same bases are held. The base-only
model starts it at the first time after `b_{k−1}` ends at which a base is available;
the plan starts it no earlier, since `b_{k−1}` ends no earlier under the plan and it
too needs a base available. A base that becomes available after the base-only model's
start has a later availability, and so a larger key, than every base eligible then, so
the least is `b_k` in both. The order can differ from the order of the keys alone:
a base listed after another of its subgroup waits for it, even when it is available
earlier. A base's service time `s_k` (the sum of its chunks' times) is the same in
both.
*The link.* Let `a_k` be `b_k`'s availability, `start_k`, `end_k` its first chunk's start
and last chunk's end under the plan, and `start⁰_k`, `end⁰_k` under the base-only model.
1. A base, once started, is sent to its end without a chunk of another object between.
At each boundary while `b_k` is sent, the eligible objects are those eligible when
it started, less any dropped or reset, and those that became available since, whose
availability, and so whose key under either schedule, is larger than `b_k`'s. None
becomes eligible by its subgroup in between: an earlier object of a subgroup is
finished only by being sent, which does not happen while `b_k` is sent, and a reset
ends the whole tail. `b_k` was the least eligible object when it started (under
`strict-priority` every base precedes every refinement; under `fifo` it was chosen),
so it stays the least. So `end_k = start_k + s_k`.
2. In the base-only model, `start⁰_k = max(a_k, end⁰_{k−1})`: the bases are sent in
order, and the link is idle only when no base is available.
3. Under the plan, at `t_k = max(a_k, end_{k−1})` base `b_k` is eligible (the bases
its subgroup holds it behind are among `b_1, …, b_{k−1}`) and, by the order above,
the eligible base of least key. Under `strict-priority`, a base precedes every refinement, so `b_k` starts
at the first boundary at or after `t_k`; a chunk in flight at `t_k` is a refinement's
(step 1), and ends within `c`. So `start_k ≤ t_k + c`. Under `fifo`, objects that
precede `b_k` are bases (all before it in the order) and refinements available no
later than `a_k`, whose deadlines are therefore at most `a_k + T`. At the first
boundary after `a_k + T`, which comes within `c` of it, every such refinement not
yet finished has been dropped or reset (§10.2); one already finished took no
longer. So `start_k ≤ max(end_{k−1}, a_k + T + c)`, and in both cases
`start_k ≤ max(a_k + L, end_{k−1})`, with `end_{k−1}` exact when it is the larger:
a base that ends at a boundary where `b_k` is eligible is followed by `b_k`.
4. By induction on `k`, `start_k ≤ start⁰_k + L`: with step 3, step 1 and the
hypothesis, `start_k ≤ max(a_k + L, end⁰_{k−1} + L) = start⁰_k + L`. Each base
therefore arrives at most `L` later than in the base-only model, and the bases
arrive in the same order in both.
*The decode.* Number the base decode jobs in arrival order, which is the order of step
4. Under `base-first` the sink takes bases in the order they became ready (Appendix B,
*Workers*), which is that order, and before any refinement. Let `r_k ≤ r⁰_k + L` be
`b_k`'s arrival under the plan and the base-only model, `W` the workers and `d_b` a
base's decode time.
5. In the base-only model, with equal decode times and jobs started in order,
`u⁰_k = max(r⁰_k, u⁰_{k−1}, u⁰_{k−W} + d_b)` is `b_k`'s start (terms with an index
below 1 omitted): at that time `b_{k−1}` has started, every base up to `b_{k−W}` has
finished, so at most `W − 1` workers are busy.
6. Under the plan, no refinement starts while a base is ready and waiting. A
refinement that holds a worker after `r_k + d_r` therefore started after `r_k` and
so after `b_k` started. At `τ = max(r_k + d_r, u_{k−1}, u_{k−W} + d_b)`, if `b_k` has
not started, it is the waiting base of least order, no refinement holds a worker,
and at most `W − 1` bases do; so `u_k ≤ τ`.
7. By induction, `u_k ≤ u⁰_k + L + d_r`: each term of `τ` is at most the matching term
of step 5 plus `L + d_r`.
So every base is presented at most `S_link + S_decode = L + d_r` after its time in the
base-only model, and a late base is presented all the same (Appendix B,
*Deadlines*). ∎
The reference implementation checks the bound on generated models and plans drawn
over the whole range of the preconditions (groups and captures in any order, objects
listed in any order, slow links, zero and short timeouts, caps, one to four workers),
and on draws aimed at the orders the bound depends on (captures on and beside chunk
boundaries and out of order, bases split into two layer-0 objects, chunks of one byte),
which fail on their own when the send or the decode order of bases is not the one
above; the corpus checks it on every bounded plan it holds. For each part of the bound a
plan can lose, the corpus holds a counter-plan whose stall passes the bound of the
same plan with that part restored: a `fifo` link without a refinement timeout against
the same `fifo` link with one, and `fifo` decode against the same plan with
`base-first` decode.
## 11. Records
§11.1 A published figure **MUST** cite a dated record in
`wai/research/staged-measurements/` by path and commit. The record carries:
- the date and the commit;
- the title's content identity (Appendix D) and the framing label;
- the machine and build;
- every deterministic figure of §3–§6 and §9, and every measured figure of §5.3–§5.4,
§7 and §8 with its method;
- the link hashes of its claims.
§11.2 Records are immutable. A correction is a new record that names the one it
corrects.
§11.3 A classical baseline is a reference curve of points (counted bytes, quality per
§4) from a named encoder at stated versions and settings, on the same source. A
record **MUST NOT** state a multiplier, percentage or ratio between a WAI figure and a
baseline.
## 12. What continuous integration asserts
§12.1 Only values that are the same on every machine:
- counted bytes, `bfpf_min` and simulator outputs;
- the integers of §4, output digests and the output tree's root and leaf count;
- deduplication totals and the overhead points;
- every figure a conformance corpus states.
§12.2 Measured times and energy are stored as artifacts and never asserted.
§12.3 The corpus [`staged-measure-conformance/`](../staged-measure-conformance/) holds
titles (Appendix D) and, for each, every figure of §12.1 (`goldens.json`) and the
fixture tool's own reconstruction digests (`expect.json`). A conforming implementation
reproduces every figure and every digest. The corpus's README names its second,
independent implementation.
## 13. Media-class matrix
The last column follows §3.2: an object is countable only under a capability whose
format WAI checks (`count_object` gives `no-format-check` for any other), so a class
whose staged form or base has no checked format is not countable in this revision,
whatever else it supports.
| class | unit | staged form | edge operation | quality kind | in this revision |
|---|---|---|---|---|---|
| image | a picture | pinned integer base (`wai.neural.int_hyper`) + `WIR1` layers (`wai.image.int_refine`: identity, up2) | serve a prefix of layers | `rgb8` | measured (corpus); a classical base (PNG and the like) is not countable in this revision |
| video, WAI-native | a frame | `wai.video.int_motion` or `wai.video.int_hyper` base + `WIR1` layers, per frame or per span (`wai.video.int_refine`) | drop layers above a cap | `rgb8` per frame, pooled per clip | measured (corpus) |
| audio | the clip | `wai.audio.int_codec` base + `WIR1` layers (`wai.audio.int_refine`: rate, identity) | drop layers above a cap | `pcm16` | measured (corpus); a `wai.audio.flac` base is not countable in this revision |
| splat | the attribute tensor | `wai.splat.int_codec` base + `WIR1` layers (`wai.splat.int_refine`) | drop layers above a cap | `tensor-i64` | measured (corpus) |
| video, enhancement layer | a frame | registered base + `wai.video.lcevc` companion (SPEC §7.1) | drop the companion | `rgb8` after the sink's decode | not countable in this revision (the companion's format is not checked); quality needs a sink-supplied decoder |
| image, progressive passes | a picture | byte prefixes of one progressive codestream | serve a byte prefix | `rgb8` after the sink's decode | a staged form not yet registered (staged-delivery §6.2); not countable in this revision |
| audio, codebook prefix | the clip | the further codebooks of a residual-vector-quantised code | drop codebooks above a cap | `pcm16` | a staged form not yet registered; not countable in this revision |
| volumetric (4D splat) | a keyframe span | keyframe refinement | serve a prefix | `tensor-i64` | a staged form not yet registered; not countable in this revision |
| world / feed / film / avatar | a segment | snapshot and operation deltas | snapshot and tail | `state-exact` | a staged form not yet registered; not countable in this revision |
| score | a section | none | none | `state-exact` (mixdown digest) | not countable in this revision |
| haptics | an envelope | keyframe refinement | serve a prefix | `tensor-i64` | a staged form not yet registered; not countable in this revision |
| quantum | a circuit | none | none | `state-exact` | not countable in this revision |
| text and records | an object | none | none | none | not countable in this revision |
## Appendix A. Counted formats
The formats §3.2 counts, and how the reference implementation
(`measure_framing::count_object`) checks each before it counts it. A payload that fails
its check is not countable, with the check's code.
| capabilities | format | check |
|---|---|---|
| `wai.video.int_motion` (keyframe 0), `wai.video.int_hyper` (keyframe 1) | `WIV1` | [integer-payloads](integer-payloads.md) §2–§3, in its order |
| `wai.neural.int_hyper` | `WIH1` | integer-payloads §2, §4 |
| `wai.audio.int_codec` | `WIA1` | integer-payloads §2, §5 |
| `wai.splat.int_codec` | `WIS1` | integer-payloads §2, §6 |
| `wai.image.int_refine`, `wai.video.int_refine`, `wai.audio.int_refine`, `wai.splat.int_refine` | `WIR1` | staged-delivery Appendix A.1–A.3, for the capability's unit kind |
| every other capability | — | none: not countable (`no-format-check`) |
A `WAI1` fallback with a pinned format is checked as its capability is. Parameter sets
(SPEC §3.1) are not objects and are never counted bytes; §3.4 states them separately.
The corpus's `counting.json` holds an object for each outcome.
## Appendix B. Delivery simulator `wai-sim/1`
A **model** is a title's objects in title order, the object index, as delivery sees
them (`wai-sim-model/1`):
```json
{ "format": "wai-sim-model/1", "unit_us": 40000,
"objects": [ { "id": "base.wai", "base": 0, "layer": 0, "units": [0, 5], "bytes": 2519, "capture_us": null }, … ] }
```
`units` is `[from, count]` over the title's units and `bytes` the object's counted
bytes (§3). A model is well formed when every object has at least one byte and one
unit, no two objects cover one unit at one layer, and every unit an object of layer
`k ≥ 1` covers is covered at layer `k − 1` by an object of the same base. A measured
title's model is its objects with their counted bytes, and its bases' capture times.
A **plan** (every member required, none other allowed):
```json
{ "plan": "wai-sim/1", "mode": "vod" | "live",
"link": { "rtt_us": 40000, "bytes_per_ms": 2500, "chunk_bytes": 1200 },
"sender": { "schedule": "strict-priority" | "fifo", "carriage": "datagram" | "subgroup",
"layer_cap": null, "delivery_timeout_us": { "base": null, "refinement": 30000 },
"vod_window_us": null },
"sink": { "workers": 1, "decode_order": "base-first" | "fifo",
"decode_us": { "base": 4000, "refinement": 2000 },
"playout_start_us": null, "late_refinement": "discard" | "present" } }
```
`bytes_per_ms`, `chunk_bytes`, `workers` and both decode times are at least 1. Every
integer of a model or plan is at most 2⁵³ − 1. All arithmetic is in integer
microseconds, and `h = ⌊rtt_us / 2⌋`; a time or a byte sum past 64 bits is refused
(`overflow`), never wrapped.
**The sender.**
- *Availability.* The sink requests at 0 and the sender holds the request at `h`. An
object is available at `h + max(0, from·unit_us − vod_window_us)` on demand (`h`
with no window), and at `max(h, capture_us)` live; a live plan over an object with no
capture time is refused.
- *Deadline.* An object's availability plus its layer's timeout (`base` for layer 0,
`refinement` otherwise); none when that timeout is `null`.
- *Cap.* An object above `layer_cap` is capped: never sent, never offered.
- *Datagrams.* Under `datagram`, an object of more than `chunk_bytes` that is not capped
is refused (`datagram-too-large`).
- *Link.* The link carries one chunk at a time. At each **boundary** — time 0, and the
end of each chunk — the sender first applies the timeouts, then sends the next chunk:
- *Timeouts.* In object order, each object still pending whose deadline is before the
boundary's time: under `datagram` it is dropped; under `subgroup` its subgroup is
reset from it (§10.2).
- *Eligibility.* An object is eligible when it is pending, available, and, under
`subgroup`, no earlier object of its subgroup is pending.
- *Order.* Of the eligible objects, the least by (layer, then for a base its
availability, `from`, object index) under `strict-priority`, and by
(availability, object index) under `fifo`: under either, bases go in the order
they became available.
- *Chunk.* `c = min(chunk_bytes, bytes not yet sent)`, occupying the link
`⌈c·1000 / bytes_per_ms⌉` µs from the boundary, and arriving `h` after it ends. An
object's `sent_first_us` is its first chunk's start. An object whose last chunk
ends at `e` is delivered, arriving at `e + h`, under `datagram`, with no deadline,
or when `e` is at most its deadline; otherwise it stays pending and the next
boundary resets its subgroup.
- With nothing eligible, the time moves to the next availability of a pending
object, which is a boundary too; the run ends when nothing is pending.
**The sink.**
- *Ready.* A delivered object is ready when it has arrived and, for layer `k ≥ 1`,
every unit it covers is at stage `k − 1`; it became ready at the later of its arrival
and the time the last of those units reached that stage.
- *Workers.* The sink has `workers` non-preemptive workers. At each event time it first
completes the jobs ending then, then gives each free worker the least ready job: by
(layer, then for a base the time it became ready, `from`, object index) under
`base-first`, so bases are decoded in the order they arrived, and by (the time it
became ready, object index) under `fifo`. A job lasts `decode_us.base` or `decode_us.refinement`,
once per object whatever its units. The next event is the next job end or the next
time a job becomes ready.
- *Presentation.* When a job ends at `t`, every unit it covers is presented at
`layers = k + 1` at `t`, and reaches stage `k`.
- *Deadlines.* With `playout_start_us` set, unit `u`'s deadline is
`playout_start_us + u·unit_us`. Each unit a job presents after its deadline counts
one **missed**: a base is presented all the same; a refinement is presented under
`present`, and under `discard` is not presented and the unit stays at its stage.
**Outputs.**
- per object: `sent_first_us`, `arrived_us`, and its state: `delivered`, `dropped`,
`abandoned` or `capped`;
- the presentations `(unit, layers, t_us)`, ordered by unit, then layers;
- `bfpf_sched`, the bytes of the chunks arrived by unit 0's first presentation;
`tfpf_us`, that presentation's time; and `tffs_us`, unit 0's first presentation at
each layer count, `null` where none;
- `offered`, `delivered`, `dropped`, `abandoned` and `never_sent_capped`, in bytes, with
`offered = delivered + dropped + abandoned`;
- `missed`;
- each unit's stall (§10.4), `null` where the plan or the base-only model never presents
its base.
A **case** of the corpus names a model, a plan, and its claim: `within-bound` (every
stall at most the plan's bound), `exceeds:<plan>` (the plan has no bound, and its
largest stall passes the named plan's), or `unbounded` (the plan has no bound, and only
its figures are checked).
## Appendix C. Output tree
A commitment to every presentation of a session or a title: one leaf per (unit,
layers), so that a record or a claim commits to its outputs by the pair (root,
`leaf_count`) and any one output is provable by an inclusion path.
```
out_digest = SHA-256(canonical output) SPEC §7 Digests: RGB8, s16le PCM, the attribute tensor, a 32-byte state hash
leaf = SHA-256("wai:out-leaf\x01" ‖ u64_be(unit) ‖ u16_be(layers) ‖ u16_be(decoder) ‖ out_digest)
node = SHA-256("wai:out-node\x01" ‖ left ‖ right)
```
- `decoder` is the index of the decoder entry that produced the output: entries are
(capability, pin) pairs in first-use order, presentations taken by (layers, unit).
For a title, the base's capability with the pin it verified, then each refinement
capability.
- Leaves are ordered by (unit, layers), and a (unit, layers) given twice is refused.
- The root over `n ≥ 1` leaves is RFC 6962's Merkle tree hash: one leaf is itself;
otherwise `node(MTH(D[0:k]), MTH(D[k:n]))`, with `k` the largest power of two below
`n`. No leaf is ever duplicated, so two leaf lists never give one root.
- A commitment is the root with its leaf count; a root alone is not one. A record or a
claim that commits to an output tree states both.
- An inclusion proof is RFC 6962's audit path, checked as RFC 9162 §2.1.3.2 checks one
under the leaf count the commitment states, never a count the proof's holder
supplies. An audit path does not fix the tree's size: leaf 0's path in a tree of 7
leaves also walks to that root when 8 is taken as its size.
- `layers` is 16 bits, so it holds any stage count.
The JWP group root ([jwp-receipts](jwp-receipts.md)) is a separate, fixed wire format
and is not this tree.
## Appendix D. Measured titles
A measured title is a fixture directory with a `title.json`:
```json
{ "format": "wai-measure-title/1", "title": "still", "about": "…",
"unit_us": 40000,
"bases": [ { "case": "staged-conformance/cases/image_int_hyper_2l", "base": 0,
"first_unit": 0, "capture_us": null } ],
"layers": [ "L01.wai", "L02.wai", "L03.wai" ],
"source": { "file": "source.bin", "sha256": "<64 hex>" },
"references": [ { "label": "one layer, q 1", "layers": [ "R0_00.wai" ] } ],
"expect": "expect.json" }
```
- The stage structure is the staged set's own. Each base is a base of a
[staged-conformance](../staged-conformance/) case, named by the case directory and
its index there: the case states its envelope, the parameter set it decodes against
(by member and path) and its stage-0 output. Each layer's `refines` names its base,
producers, layer and span (staged-delivery §3). `title.json` adds only what that
model does not carry.
- `unit_us` is one unit's duration, and `capture_us` a base's capture time for a live
delivery plan (`null` for one delivered on demand).
- `source` holds each unit's canonical buffer at its final shape, in unit order, with
its SHA-256.
- `references` are the single-layer references of §6.2: each is the title's bases with
the layers it lists.
- `expect.json` holds, from the fixture tool's own encoder, the SHA-256 of every
unit's output at every stage, and of every reference's final units. An evaluation
**MUST** reproduce them.
- A base **MAY** instead be named by its envelope alone, `{ "envelope": <path>, "set":
{ <member>: <path> }, "first_unit", "capture_us" }`: it has no stage-0 output or case
to check its decode against.
- `limits` (optional) lowers the limits the title is measured under: `cost`, four
figures (output bytes, symbols, working bytes, multiply-accumulates), and
`chain_budget`, each `null` for the tool's limit below. A title lowers a limit and
never raises one (below).
- `refused` (optional) names the code the title's evaluation **MUST** be refused with:
a title that exists to be refused has no figures.
- Every path is below `wai/`; a layer, source and expectation file is a file name in
the fixture directory.
A title's **content identity** is BLAKE3 over `"wai:measure-title\x01"` and then, in
path order, each measured file's path and bytes, each prefixed by its length as a
64-bit big-endian integer. The measured files are its objects (bases, layers and the
references' layers), its bases' parameter-set files and its source; labels,
`title.json`, stage-0 outputs and `expect.json` are not part of it.
**Evaluation.** Each base is decoded as a sink decodes one (SPEC §4: its capability,
then its fallback, a refinement-only or companion-only capability making the envelope
inert), against the parameter set its pin names (SPEC §3.1), and under the cost
limit, which its payload's declared cost (integer-payloads §7) is compared with
before any symbol is decoded or any output allocated. It must give the case's stage 0,
and the capability, fallback, tier and pin the case states. Its layers are applied as a
sink applies them (staged-delivery §4) under the chain budget, with the same
limit on each layer's declared cost (Appendix A.5), one layer index at a time and,
within one, in the order the title lists them; every layer **MUST** apply to every unit
it covers, as in a set delivered whole and in order. The figures of §3–§6 and §9 and
the output tree follow from the outputs at every stage. A refusal on the way names its
code: the base decode's (`cost_over_limit`, `missing_capability`, a pin refusal) or the
sink's (`chain-over-limit`, a layer's §4.2 reason). This composition of library
functions is for measurement, not a sink's dispatch path.
**The tools' limits.** The reference tools (`wai stage`, `wai_stage_measure`,
`wai_stage_bench`) decode every base under these cost figures, hold every layer to
their output, symbols and working bytes (staged-delivery Appendix A.5), and hold every
chain to the chain budget. A tool's `--limit OUT,SYMBOLS,WORKING,MACS` replaces the four
cost figures. A title is measured under the lower, figure by figure, of its own
`limits` and the tool's: a title lowers a limit and **MUST NOT** raise one, because
whoever writes a title is not whoever runs the tool, and raising a limit takes the
tool's own `--limit`.
| figure | default | why |
|---|---|---|
| output bytes | 2²⁵ (32 MiB) | five 1920 × 1080 RGB8 frames, or minutes of 48 kHz stereo PCM: more than one stage set's base carries, and below the declared clips a tool must refuse |
| symbols | 2²⁵ | one entropy-decode step per output byte at the output limit |
| working bytes | 2²⁸ (256 MiB) | eight bytes per network activation, for networks a few times the output's size |
| multiply-accumulates | 2³⁸ | a learned decode of several megapixels |
| chain budget | 2²⁸ (256 MiB) | eight times the output limit, for a base's units and the layers retained over them |

9
wai/research/energy-exposure-mapping.md
View file

@ -648,3 +648,12 @@ Read on 2026-09-27 unless noted.
11. [2026-09-global-sweep](2026-09-global-sweep.md), for the CM-EE announcement
(§2.4), the ISO/IEC 23001-19 stage (§2.1) and the annexes of C(2026) 3472
(§5.2).
## 2026-10 addendum: delivery telemetry
CMSD headers and 5GMS remain outside WAI's receipt layer. Since this note was
written, [delivery-telemetry](../extensions/delivery-telemetry.md) maps an unsigned,
advisory observation about WAI delivery onto CMCD version 2 and CMSD keys. It carries
content hashes and receipt link hashes as pointers, never a joule figure, an
acquisition class or a quality figure, and it binds no receipt to CMCD or CMSD. The
text above is unchanged.

2
wai/staged-measure-conformance/.gitattributes vendored Normal file
View file

@ -0,0 +1,2 @@
# Hashed files: a line-ending conversion on checkout would change every digest.
* -text

121
wai/staged-measure-conformance/README.md Normal file
View file

@ -0,0 +1,121 @@
# staged-measure-conformance — staged-delivery measurement
The corpus of [`extensions/staged-measurement.md`](../extensions/staged-measurement.md)
§12.3: measured titles, and every figure continuous integration may assert about them
(§12.1).
**What a conforming implementation reproduces.** For each title: the counted bytes of
every object, split into envelope header, manifest and payload (§3), and their sums per
stage and per prefix, per base and over the title; the bytes of each base's parameter
set; the first-presentable bytes (§5.1); every presentation's output digest (SPEC §7
*Digests*), with its quality integers against the source and against the final prefix
(§4), and every prefix's pooled quality; the decoder entries and the output tree's root
and leaf count (Appendix C); the layering overhead's rate–quality points, the full title's and each
single-layer reference's (§6); the deduplication totals (§9); and the title's content
identity (Appendix D). Across the corpus: the record-wide deduplication totals (§9.2).
- `fixtures/<name>/` — one title (Appendix D): `title.json`; its layers (`L*.wai`) and
its single-layer references' layers (`R*.wai`); `source.bin`, each unit's canonical
buffer at its final shape; `expect.json`, the fixture tool's own reconstruction
digests. Each title's base is a base of a [staged-conformance](../staged-conformance/)
case, read where it is, with the case's stage 0 and parameter set.
- `goldens.json` — every title's figures, in fixture order, and the record-wide
totals.
- `quality.json` — §4.2's integers for picture pairs drawn from a stated formula, at
sizes that place the similarity's windows every way the rule does: narrower or
shorter than a window, offsets that reach the edge at a multiple of 4, and an
edge-aligned last window on one axis or both.
- `counting/` and `counting.json` — objects whose counting §3.2 decides, each with its
counted bytes or the code it is not countable with: a `WIH1` base, a `WIR1` layer and
a `WAI2` (counted); the JSON form of `WIH1`, its base64 text, a JSON string, and
`WIH1` behind a byte-order mark (`bad_magic`); a PNG base and an unregistered
capability (`no-format-check`); `WIH1` under a `wai.video.int_motion` fallback
(`format_capability_mismatch`) and under a fallback WAI checks no format for
(counted); a `WAI2` component with another payload's `sha256` (`sha256-mismatch`); an
encrypted payload; and bytes after an envelope (`trailing`).
- `refusals/<name>/title.json` — titles a tool must refuse, each with the code it states
(`refused`): a base whose units pass the chain budget its title states
(`chain-over-limit`, over the staged-delivery corpus's `base_over_budget` case); the
two bound vectors under `../wai-rs/tests/vectors/staged_bounds/`, whose headers declare
64 MiB and 1 GiB of output, past the tools' stated limit (`cost_over_limit`, refused
before anything is allocated for them), the second also under a title whose `limits`
state 2^40 for every figure, which a title cannot raise; and the two envelopes there whose capability
is refinement-only or companion-only (`missing_capability`: inert, the fallback never
reached); and the still title's up2 layer under a cost limit its base passes and the
layer's declared output does not (`cost-over-limit`).
- `sim/` — the delivery simulator's corpus (§10, Appendix B): `models/` (a synthetic
model of three groups of five units, each a base and two layers per unit, captured as
each group ends), `plans/` (`wai-sim/1`), `cases.json` (a model — a file, or
`fixture:<name>` for a title's objects and counted bytes — a plan, and the case's
claim: `within-bound`, `exceeds:<plan>` or `unbounded`) and `results.json` (every
case simulated: each object's delivery, the presentations, first presentable, the
byte totals, the missed deadlines, each unit's stall, the plan's bound and the
largest stall). The plans cover both carriages and both bounded link schedules, a slow
link on which refinements miss their timeouts (with a window, so that a reset takes a
subgroup's tail before its own deadlines), a layer cap, live playout with late
refinements discarded or presented, two models on which bases were once sent or
decoded out of the order they became available (`reorder-link`, `reorder-decode`),
and a counter-plan for each part of §10.4's bound a schedule can lose: a `fifo` link
with no refinement timeout, whose stall passes the bound of the same `fifo` link
with a short timeout (`fifo-timeout-short`), and `fifo` decode on one worker, whose
stall passes the bound of the same plan with `base-first` decode (`strict-window`).
**The content is synthetic.** Each base is a committed vector under
[`../wai-rs/tests/vectors/`](../wai-rs/tests/vectors/), as the staged-delivery corpus
wraps it: a `wai.neural.int_hyper` picture, a `wai.video.int_hyper` clip, a
`wai.audio.int_codec` clip and a `wai.splat.int_codec` attribute tensor, each pinned to
its parameter set. Each source is the base's output with fine detail added by integer
functions of position, channel and a seed (for audio, two triangle waves), raised to the
final shape by the layer's own predictor where the title changes the shape. The layers
are `WIR1` payloads coded toward the source by the reference encoder
(`staged::wir1::encode`): up2 then identity for the picture; one identity layer per
frame, twice, for the clip; a rate layer to twice the sample rate, then identity, for
the audio; identity for the attribute tensor. Each title's single-layer references code
one layer straight from stage 0 at the final step and, for some titles, at further
steps (the curve of §6.3).
**Verifiers.**
- `wai_stage_measure verify .` (from `../wai-rs`, feature `stage_measure`): regenerates
every fixture and refusal title in memory and requires it byte for byte; decodes each
base through the sink's own base decode (`staged::decode_base_pinned`) under the
title's limits and requires the case's stage 0, capability, fallback, tier and pin;
applies the layers through the reference sink (`staged::StageSink`) under the title's
chain budget; requires every refusal title refused with its code; requires every
presentation's digest and
every reference's final digests to be the fixture tool's (`expect.json`, written from
the encoder's own reconstruction); and requires `goldens.json` byte for byte.
`wai_stage_measure generate-goldens .` and `wai_stage_measure gen-fixtures .` write
them; `wai_stage_measure eval fixtures/<name>` prints one title's figures.
- `wai_stage_measure verify .` also regenerates the simulator's models, plans and cases
byte for byte, simulates every case (`stage_measure::sim`), checks each claim, and
requires `sim/results.json` byte for byte; `wai_stage_measure sim <model> <plan>`
runs one.
- `sim_verify.py .`: pure Python with the standard library, sharing no code with
`wai-rs`: Appendix B implemented from the text, with each fixture's model taken from
`measure_verify.py`'s own evaluation. It reproduces every result, checks every claim,
and requires a counter-case for each part of the bound a schedule can lose.
- `measure_verify.py .`: pure Python with the standard library, sharing no code with
`wai-rs`. It reimplements, from the extension and SPEC: the byte rule and the formats
it counts; each base's units from its case's stage 0; every layer applied in layer
order to every unit it covers, with its predecessor checked; the integer luma, SSE,
the Q32 structural similarity with its window placement (Python's `//` floors toward
−∞, as §4.2 states), checked over `quality.json` as well, PCM and tensor sums and
their pooling; the output tree; the prefixes, first presentable,
overhead points, deduplication and content identity. It requires its decode to be the
fixture tool's and every figure to be `goldens.json`'s. Before it uses a base it runs
its own base checks (SPEC §4's dispatch, the output and symbols the payload's header
declares against the title's cost limit, the units against its chain budget), and it
requires every refusal title refused, by those checks, with the code the title
states. It takes BLAKE3 from
[`../claim-conformance/_blake3.py`](../claim-conformance/_blake3.py), the integer
payload parsers from
[`../int-payload-conformance/int_payload_verify.py`](../int-payload-conformance/int_payload_verify.py),
and the `WAI1` reader, `refines`, `WIR1` and its predictors from
[`../staged-conformance/staged_verify.py`](../staged-conformance/staged_verify.py),
each written from its specification. Each base's stage 0 is the staged-delivery
corpus's, which `staged_verify.py` checks against the vectors' sidecars.
Every figure here is integer, and the same on every machine. The simulator's times are
virtual and labelled simulated; no measured time and no energy is stated: those are
measured, never asserted (§12.2).

110
wai/staged-measure-conformance/counting.json Normal file
View file

@ -0,0 +1,110 @@
{
"format": "wai-measure-counting/1",
"about": "Objects whose counting extensions/staged-measurement.md §3.2 decides: each object's counted bytes, or the code it is not countable with (and the WAI2 component). Written by wai_stage_measure gen-fixtures; required by wai_stage_measure verify and measure_verify.py.",
"cases": [
{
"file": "staged-measure-conformance/counting/wih1.wai",
"counted": {
"header": 12,
"manifest": 158,
"payload": 88,
"total": 258
},
"refused": null,
"component": null
},
{
"file": "staged-measure-conformance/fixtures/still/L01.wai",
"counted": {
"header": 12,
"manifest": 323,
"payload": 616,
"total": 951
},
"refused": null,
"component": null
},
{
"file": "staged-measure-conformance/counting/json-form.wai",
"counted": null,
"refused": "bad_magic",
"component": null
},
{
"file": "staged-measure-conformance/counting/base64.wai",
"counted": null,
"refused": "bad_magic",
"component": null
},
{
"file": "staged-measure-conformance/counting/json-string.wai",
"counted": null,
"refused": "bad_magic",
"component": null
},
{
"file": "staged-measure-conformance/counting/bom.wai",
"counted": null,
"refused": "bad_magic",
"component": null
},
{
"file": "staged-conformance/cases/image_png_up2/base.wai",
"counted": null,
"refused": "no-format-check",
"component": null
},
{
"file": "staged-measure-conformance/counting/unregistered-capability.wai",
"counted": null,
"refused": "no-format-check",
"component": null
},
{
"file": "staged-measure-conformance/counting/fallback-mismatch.wai",
"counted": null,
"refused": "format_capability_mismatch",
"component": null
},
{
"file": "staged-measure-conformance/counting/fallback-unchecked.wai",
"counted": {
"header": 12,
"manifest": 185,
"payload": 88,
"total": 285
},
"refused": null,
"component": null
},
{
"file": "staged-measure-conformance/counting/wai2.wai",
"counted": {
"header": 18,
"manifest": 212,
"payload": 88,
"total": 318
},
"refused": null,
"component": null
},
{
"file": "staged-measure-conformance/counting/wai2-sha256-mismatch.wai",
"counted": null,
"refused": "sha256-mismatch",
"component": 0
},
{
"file": "staged-measure-conformance/counting/encrypted.wai",
"counted": null,
"refused": "encrypted",
"component": null
},
{
"file": "staged-measure-conformance/counting/trailing.wai",
"counted": null,
"refused": "trailing",
"component": null
}
]
}

BIN
wai/staged-measure-conformance/counting/base64.wai Normal file
View file

Binary file not shown.

BIN
wai/staged-measure-conformance/counting/bom.wai Normal file
View file

Binary file not shown.

BIN
wai/staged-measure-conformance/counting/encrypted.wai Normal file
View file

Binary file not shown.

BIN
wai/staged-measure-conformance/counting/fallback-mismatch.wai Normal file
View file

Binary file not shown.

BIN
wai/staged-measure-conformance/counting/fallback-unchecked.wai Normal file
View file

Binary file not shown.

BIN
wai/staged-measure-conformance/counting/json-form.wai Normal file
View file

Binary file not shown.

BIN
wai/staged-measure-conformance/counting/json-string.wai Normal file
View file

Binary file not shown.

BIN
wai/staged-measure-conformance/counting/trailing.wai Normal file
View file

Binary file not shown.

BIN
wai/staged-measure-conformance/counting/unregistered-capability.wai Normal file
View file

Binary file not shown.

BIN
wai/staged-measure-conformance/counting/wai2-sha256-mismatch.wai Normal file
View file

Binary file not shown.

BIN
wai/staged-measure-conformance/counting/wai2.wai Normal file
View file

Binary file not shown.

BIN
wai/staged-measure-conformance/counting/wih1.wai Normal file
View file

Binary file not shown.

BIN
wai/staged-measure-conformance/fixtures/audio/L01.wai Normal file
View file

Binary file not shown.

BIN
wai/staged-measure-conformance/fixtures/audio/L02.wai Normal file
View file

Binary file not shown.

BIN
wai/staged-measure-conformance/fixtures/audio/L03.wai Normal file
View file

Binary file not shown.

BIN
wai/staged-measure-conformance/fixtures/audio/R0_00.wai Normal file
View file

Binary file not shown.

BIN
wai/staged-measure-conformance/fixtures/audio/R1_00.wai Normal file
View file

Binary file not shown.

40
wai/staged-measure-conformance/fixtures/audio/expect.json Normal file
View file

@ -0,0 +1,40 @@
{
"format": "wai-measure-expect/1",
"about": "The fixture tool's own reconstruction: the SHA-256 of each unit's canonical buffer at each stage it coded, and of each reference's final units, from the encoder's reconstruction loop.",
"presentations": [
{
"unit": 0,
"layers": 1,
"out_sha256": "b8ff83a442c112661ec7a41d6e51cb32559510a699a0e9bb37fa301f8952978a"
},
{
"unit": 0,
"layers": 2,
"out_sha256": "7cf4ee1daabc1f8779252bb3e02f16ba2ab2d8a65b374c8cbb1c3c3354121cdd"
},
{
"unit": 0,
"layers": 3,
"out_sha256": "02c3e3540210f1e5fce2b9f75d38e74d9b877cf17f2233b7825a8a10b823c64c"
},
{
"unit": 0,
"layers": 4,
"out_sha256": "8405e88c07e18cd93aecd578493cb3ff060c98be1790ebca0edeb0d1af1c512a"
}
],
"references": [
{
"label": "one layer, q 1",
"out_sha256": [
"8405e88c07e18cd93aecd578493cb3ff060c98be1790ebca0edeb0d1af1c512a"
]
},
{
"label": "one layer, q 8",
"out_sha256": [
"9b34a3dcbc301e180a7a675b3304ed57ddee43858863e113a55455a926d614e8"
]
}
]
}

BIN
wai/staged-measure-conformance/fixtures/audio/source.bin Normal file
View file

Binary file not shown.

38
wai/staged-measure-conformance/fixtures/audio/title.json Normal file
View file

@ -0,0 +1,38 @@
{
"format": "wai-measure-title/1",
"title": "audio",
"about": "a pinned wai.audio.int_codec clip (8192 samples, mono); a rate layer to twice the sample rate at q 64, then identity layers at q 8 and q 1; references: one rate layer at q 1 and at q 8",
"unit_us": 512000,
"bases": [
{
"case": "staged-conformance/cases/audio_int_codec_1l",
"base": 0,
"first_unit": 0,
"capture_us": null
}
],
"layers": [
"L01.wai",
"L02.wai",
"L03.wai"
],
"source": {
"file": "source.bin",
"sha256": "8405e88c07e18cd93aecd578493cb3ff060c98be1790ebca0edeb0d1af1c512a"
},
"references": [
{
"label": "one layer, q 1",
"layers": [
"R0_00.wai"
]
},
{
"label": "one layer, q 8",
"layers": [
"R1_00.wai"
]
}
],
"expect": "expect.json"
}

BIN
wai/staged-measure-conformance/fixtures/clip/L01.wai Normal file
View file

Binary file not shown.

BIN
wai/staged-measure-conformance/fixtures/clip/L02.wai Normal file
View file

Binary file not shown.

BIN
wai/staged-measure-conformance/fixtures/clip/L03.wai Normal file
View file

Binary file not shown.

BIN
wai/staged-measure-conformance/fixtures/clip/L04.wai Normal file
View file

Binary file not shown.

BIN
wai/staged-measure-conformance/fixtures/clip/L05.wai Normal file
View file

Binary file not shown.

BIN
wai/staged-measure-conformance/fixtures/clip/L06.wai Normal file
View file

Binary file not shown.

BIN
wai/staged-measure-conformance/fixtures/clip/L07.wai Normal file
View file

Binary file not shown.

BIN
wai/staged-measure-conformance/fixtures/clip/L08.wai Normal file
View file

Binary file not shown.

BIN
wai/staged-measure-conformance/fixtures/clip/L09.wai Normal file
View file

Binary file not shown.

BIN
wai/staged-measure-conformance/fixtures/clip/L10.wai Normal file
View file

Binary file not shown.

BIN
wai/staged-measure-conformance/fixtures/clip/R0_00.wai Normal file
View file

Binary file not shown.

BIN
wai/staged-measure-conformance/fixtures/clip/R0_01.wai Normal file
View file

Binary file not shown.

BIN
wai/staged-measure-conformance/fixtures/clip/R0_02.wai Normal file
View file

Binary file not shown.

BIN
wai/staged-measure-conformance/fixtures/clip/R0_03.wai Normal file
View file

Binary file not shown.

BIN
wai/staged-measure-conformance/fixtures/clip/R0_04.wai Normal file
View file

Binary file not shown.

BIN
wai/staged-measure-conformance/fixtures/clip/R1_00.wai Normal file
View file

Binary file not shown.

BIN
wai/staged-measure-conformance/fixtures/clip/R1_01.wai Normal file
View file

Binary file not shown.

BIN
wai/staged-measure-conformance/fixtures/clip/R1_02.wai Normal file
View file

Binary file not shown.

BIN
wai/staged-measure-conformance/fixtures/clip/R1_03.wai Normal file
View file

Binary file not shown.

BIN
wai/staged-measure-conformance/fixtures/clip/R1_04.wai Normal file
View file

Binary file not shown.

103
wai/staged-measure-conformance/fixtures/clip/expect.json Normal file
View file

@ -0,0 +1,103 @@
{
"format": "wai-measure-expect/1",
"about": "The fixture tool's own reconstruction: the SHA-256 of each unit's canonical buffer at each stage it coded, and of each reference's final units, from the encoder's reconstruction loop.",
"presentations": [
{
"unit": 0,
"layers": 1,
"out_sha256": "dce7282d46f88f03d5dc0fd42f325756d310fe1fc982b6e1792a1fbfbeb482cf"
},
{
"unit": 0,
"layers": 2,
"out_sha256": "71639c87086d0fe69eb0e9ba8a976e882cb7ac38056a3c388d901dd3f0ac4fd2"
},
{
"unit": 0,
"layers": 3,
"out_sha256": "7655e2d7055f1ea34084785b13e0becbec58301023ee9beff3e75a11abb7b621"
},
{
"unit": 1,
"layers": 1,
"out_sha256": "eb22eaca420f922c3e65f7583e4ba15195a013a4694d3d704c1c393477279197"
},
{
"unit": 1,
"layers": 2,
"out_sha256": "5ca748b74ae00febec767d871f336c61cb1f67db1d0451a02112191f7f9c664f"
},
{
"unit": 1,
"layers": 3,
"out_sha256": "06e01123126a93ebd5f845883f66c959a57270662fc7982e32d0768637b48722"
},
{
"unit": 2,
"layers": 1,
"out_sha256": "904e9064df9f95582e2ac4b9ef594a27810922c325c5144acd4d17e08294ecdf"
},
{
"unit": 2,
"layers": 2,
"out_sha256": "804b060fcb5fe3ea6a1588c5768a4520eb25bb67df51cbdc4de392253bcfd387"
},
{
"unit": 2,
"layers": 3,
"out_sha256": "1e946bd550123521b8d6e67485c2058159e6b3d82b58f709ad23ceb223c6bb0c"
},
{
"unit": 3,
"layers": 1,
"out_sha256": "ba33b5f7ffe2dd451106b43596f7be854ba0a5e011039a66bc8f221b4321c2df"
},
{
"unit": 3,
"layers": 2,
"out_sha256": "14eb7bbe7d69932210f029c7319f19f2e911f43c2222b08fc345f9a04069264f"
},
{
"unit": 3,
"layers": 3,
"out_sha256": "7b577471db6e6c1b9f301c34470678407d6c6d74e44d89ca6152d1c12f177fe6"
},
{
"unit": 4,
"layers": 1,
"out_sha256": "38ccbd2a62233c72087b136588551fdb742ae98a1e78a5e4f6b68c25b7778e3b"
},
{
"unit": 4,
"layers": 2,
"out_sha256": "0261a1b9ae00788df786e4cc542d36629cff822c52195ce8be5c12d33ff66905"
},
{
"unit": 4,
"layers": 3,
"out_sha256": "6d6ff21e4b9654c757a98fafc949e760e01ea22953f201affd4a738a15be6726"
}
],
"references": [
{
"label": "one layer, q 2",
"out_sha256": [
"7655e2d7055f1ea34084785b13e0becbec58301023ee9beff3e75a11abb7b621",
"06e01123126a93ebd5f845883f66c959a57270662fc7982e32d0768637b48722",
"1e946bd550123521b8d6e67485c2058159e6b3d82b58f709ad23ceb223c6bb0c",
"7b577471db6e6c1b9f301c34470678407d6c6d74e44d89ca6152d1c12f177fe6",
"6d6ff21e4b9654c757a98fafc949e760e01ea22953f201affd4a738a15be6726"
]
},
{
"label": "one layer, q 6",
"out_sha256": [
"71639c87086d0fe69eb0e9ba8a976e882cb7ac38056a3c388d901dd3f0ac4fd2",
"5ca748b74ae00febec767d871f336c61cb1f67db1d0451a02112191f7f9c664f",
"804b060fcb5fe3ea6a1588c5768a4520eb25bb67df51cbdc4de392253bcfd387",
"14eb7bbe7d69932210f029c7319f19f2e911f43c2222b08fc345f9a04069264f",
"0261a1b9ae00788df786e4cc542d36629cff822c52195ce8be5c12d33ff66905"
]
}
]
}

BIN
wai/staged-measure-conformance/fixtures/clip/source.bin Normal file
View file

Binary file not shown.

53
wai/staged-measure-conformance/fixtures/clip/title.json Normal file
View file

@ -0,0 +1,53 @@
{
"format": "wai-measure-title/1",
"title": "clip",
"about": "a pinned wai.video.int_hyper clip (5 frames, 16×16); one identity layer per frame at q 6, then one per frame at q 2; references: one layer per frame at q 2 and at q 6",
"unit_us": 40000,
"bases": [
{
"case": "staged-conformance/cases/video_int_hyper_keyframe",
"base": 0,
"first_unit": 0,
"capture_us": null
}
],
"layers": [
"L01.wai",
"L02.wai",
"L03.wai",
"L04.wai",
"L05.wai",
"L06.wai",
"L07.wai",
"L08.wai",
"L09.wai",
"L10.wai"
],
"source": {
"file": "source.bin",
"sha256": "9148bb9adeb87d45530d07ded7a4f648861f119af5347ab0d84962f75762175d"
},
"references": [
{
"label": "one layer, q 2",
"layers": [
"R0_00.wai",
"R0_01.wai",
"R0_02.wai",
"R0_03.wai",
"R0_04.wai"
]
},
{
"label": "one layer, q 6",
"layers": [
"R1_00.wai",
"R1_01.wai",
"R1_02.wai",
"R1_03.wai",
"R1_04.wai"
]
}
],
"expect": "expect.json"
}

BIN
wai/staged-measure-conformance/fixtures/splat/L01.wai Normal file
View file

Binary file not shown.

BIN
wai/staged-measure-conformance/fixtures/splat/L02.wai Normal file
View file

Binary file not shown.

BIN
wai/staged-measure-conformance/fixtures/splat/R0_00.wai Normal file
View file

Binary file not shown.

29
wai/staged-measure-conformance/fixtures/splat/expect.json Normal file
View file

@ -0,0 +1,29 @@
{
"format": "wai-measure-expect/1",
"about": "The fixture tool's own reconstruction: the SHA-256 of each unit's canonical buffer at each stage it coded, and of each reference's final units, from the encoder's reconstruction loop.",
"presentations": [
{
"unit": 0,
"layers": 1,
"out_sha256": "f07679de87ff7c3d4fc4890b80919ef857fb85bd4d9e4cfa16105211d580427f"
},
{
"unit": 0,
"layers": 2,
"out_sha256": "4603b50563302461ce2baeaa1810268c9ebe834599f62790d7acf01da33c66fd"
},
{
"unit": 0,
"layers": 3,
"out_sha256": "9a177a9b1879bc66804f7c96d98755d39ee16aca5d8e042b31706aff4c539bee"
}
],
"references": [
{
"label": "one layer, q 1",
"out_sha256": [
"9a177a9b1879bc66804f7c96d98755d39ee16aca5d8e042b31706aff4c539bee"
]
}
]
}

BIN
wai/staged-measure-conformance/fixtures/splat/source.bin Normal file
View file

Binary file not shown.

31
wai/staged-measure-conformance/fixtures/splat/title.json Normal file
View file

@ -0,0 +1,31 @@
{
"format": "wai-measure-title/1",
"title": "splat",
"about": "a pinned wai.splat.int_codec attribute tensor (14 planes of 32×32); an identity layer at q 6, then one at q 1; reference: one identity layer at q 1",
"unit_us": 40000,
"bases": [
{
"case": "staged-conformance/cases/splat_attr_1l",
"base": 0,
"first_unit": 0,
"capture_us": null
}
],
"layers": [
"L01.wai",
"L02.wai"
],
"source": {
"file": "source.bin",
"sha256": "9a177a9b1879bc66804f7c96d98755d39ee16aca5d8e042b31706aff4c539bee"
},
"references": [
{
"label": "one layer, q 1",
"layers": [
"R0_00.wai"
]
}
],
"expect": "expect.json"
}

BIN
wai/staged-measure-conformance/fixtures/still/L01.wai Normal file
View file

Binary file not shown.

BIN
wai/staged-measure-conformance/fixtures/still/L02.wai Normal file
View file

Binary file not shown.

BIN
wai/staged-measure-conformance/fixtures/still/L03.wai Normal file
View file

Binary file not shown.

BIN
wai/staged-measure-conformance/fixtures/still/R0_00.wai Normal file
View file

Binary file not shown.

BIN
wai/staged-measure-conformance/fixtures/still/R1_00.wai Normal file
View file

Binary file not shown.

BIN
wai/staged-measure-conformance/fixtures/still/R2_00.wai Normal file
View file

Binary file not shown.

46
wai/staged-measure-conformance/fixtures/still/expect.json Normal file
View file

@ -0,0 +1,46 @@
{
"format": "wai-measure-expect/1",
"about": "The fixture tool's own reconstruction: the SHA-256 of each unit's canonical buffer at each stage it coded, and of each reference's final units, from the encoder's reconstruction loop.",
"presentations": [
{
"unit": 0,
"layers": 1,
"out_sha256": "dce7282d46f88f03d5dc0fd42f325756d310fe1fc982b6e1792a1fbfbeb482cf"
},
{
"unit": 0,
"layers": 2,
"out_sha256": "827c831aef7df8d700616cd896dcb5d2cde2a926afaea503d85487026836a34a"
},
{
"unit": 0,
"layers": 3,
"out_sha256": "952ebbf5a937f327c0d056684ed88fef9d9489be4735a456ae8ecdd2d299ace6"
},
{
"unit": 0,
"layers": 4,
"out_sha256": "fef81da98166b22c30e66ce6dda9a835aa95a7d1aa278c6046dd01a8c56f0efd"
}
],
"references": [
{
"label": "one layer, q 1",
"out_sha256": [
"fef81da98166b22c30e66ce6dda9a835aa95a7d1aa278c6046dd01a8c56f0efd"
]
},
{
"label": "one layer, q 3",
"out_sha256": [
"8181c12af11507de87344cbf5a3f9217025c4f2f946ac1a12011e531806e449d"
]
},
{
"label": "one layer, q 8",
"out_sha256": [
"827c831aef7df8d700616cd896dcb5d2cde2a926afaea503d85487026836a34a"
]
}
]
}

BIN
wai/staged-measure-conformance/fixtures/still/source.bin Normal file
View file

Binary file not shown.

44
wai/staged-measure-conformance/fixtures/still/title.json Normal file
View file

@ -0,0 +1,44 @@
{
"format": "wai-measure-title/1",
"title": "still",
"about": "a pinned wai.neural.int_hyper picture (16×16); an up2 layer to 32×32 at q 8, then identity layers at q 3 and q 1; references: one up2 layer at q 1, 3 and 8",
"unit_us": 40000,
"bases": [
{
"case": "staged-conformance/cases/image_int_hyper_2l",
"base": 0,
"first_unit": 0,
"capture_us": null
}
],
"layers": [
"L01.wai",
"L02.wai",
"L03.wai"
],
"source": {
"file": "source.bin",
"sha256": "fef81da98166b22c30e66ce6dda9a835aa95a7d1aa278c6046dd01a8c56f0efd"
},
"references": [
{
"label": "one layer, q 1",
"layers": [
"R0_00.wai"
]
},
{
"label": "one layer, q 3",
"layers": [
"R1_00.wai"
]
},
{
"label": "one layer, q 8",
"layers": [
"R2_00.wai"
]
}
],
"expect": "expect.json"
}

1744
wai/staged-measure-conformance/goldens.json Normal file
View file

File diff suppressed because it is too large Load diff

660
wai/staged-measure-conformance/measure_verify.py Executable file
View file

@ -0,0 +1,660 @@
#!/usr/bin/env python3
"""Independent verifier of the staged-measurement corpus (extensions/staged-measurement.md).
Written from the extension text and SPEC §2, §7. It shares no code with wai-rs, which
writes the corpus. It takes BLAKE3 from ../claim-conformance/_blake3.py, and the
WAI1 reader, `refines`, the WIR1 parse, its stream checks and its decode (the three
predictors) from ../staged-conformance/staged_verify.py, and the integer payload
parsers from ../int-payload-conformance/int_payload_verify.py, all written from their
specifications. It implements the rest itself:
- the byte rule (§3.1): a WAI1 envelope is 12 + man_len + P bytes and a WAI2 envelope
8 + man_len + 2 + 8n + Σ P, and a payload is counted only where WAI checks its
capability's format (§3.2): WIR1 for a refinement, the pinned integer formats, by
their parsers, the capability's and a checked fallback's; anything else is
`no-format-check`, and a WAI2 component whose sha256 is not its payload's is
`sha256-mismatch`; every case of counting.json is required;
- each base's stage 0 from its staged-conformance case (whose stage0.bin
staged_verify.py checks against the vectors' sidecars), and every layer applied in
layer order, each to every unit it covers, with its predecessor checked;
- the quality integers (§4): integer BT.601 luma, SSE, the Q32 structural similarity
over clipped 8×8 windows at stride 4 with an edge-aligned last window on each axis
(Python's // floors toward −∞, as the method states), PCM and tensor sums, each pooled over several units;
- the output tree (Appendix C): RFC 6962's tree hash over SHA-256 leaves;
- the prefixes' counted bytes, first presentable, the layering overhead's points,
deduplication (§9), and the title's content identity.
Before it uses a base it runs its own base checks, in the order the measurement
refuses: SPEC §4's dispatch (a refinement-only or companion-only capability makes the
envelope inert, `missing_capability`, and its fallback is never reached); the figures a
payload's header fixes, its output bytes and its symbols, against the title's cost
limit (`cost_over_limit`; the working bytes and multiply-accumulates need the
parameter set's networks, which it does not read); and the units' bytes against the
title's chain budget (`chain-over-limit`). A title's limits are its `limits` member's,
within the tools' stated defaults: each figure the lower of the two (staged-measurement
Appendix D).
For every fixture it requires its own decode's digests to equal the fixture tool's
(expect.json), and its whole evaluation to equal the fixture's block of goldens.json;
it requires the record-wide dedup totals; it requires §4.2's integers for every
picture pair of quality.json; and it requires every title under `refusals/` to be
refused, by its own checks, with the code the title states.
python3 measure_verify.py <staged-measure-conformance dir>
Exit status 0 only if every fixture verifies.
"""
import hashlib
import json
import os
import struct
import sys
HERE = os.path.dirname(os.path.abspath(__file__))
WAI = os.path.normpath(os.path.join(HERE, ".."))
sys.path.insert(0, os.path.join(WAI, "claim-conformance"))
sys.path.insert(0, os.path.join(WAI, "int-payload-conformance"))
sys.path.insert(0, os.path.join(WAI, "staged-conformance"))
from _blake3 import blake3 # noqa: E402
import int_payload_verify as IPV # noqa: E402
import staged_verify as SV # noqa: E402
# SPEC §5: the media class of each base capability the corpus uses, which fixes its
# quality kind (§4.2–§4.4).
UNIT_CLASS = {
"wai.neural.int_hyper": "rgb8",
"wai.video.int_hyper": "rgb8",
"wai.video.int_motion": "rgb8",
"wai.audio.int_codec": "pcm16",
"wai.splat.int_codec": "tensor-i64",
}
KIND = {"rgb8": "rgb8", "pcm16": "pcm", "tensor-i64": "attr"}
# staged-measurement Appendix D: the tools' stated limits, and the order Rust's
# DeclaredCost::exceeds compares the figures in.
TOOL_COST = [1 << 25, 1 << 25, 1 << 28, 1 << 38]
TOOL_CHAIN_BUDGET = 1 << 28
INT_PATH = set(IPV.CAP_MAGIC) | {"wai.neural.int_synth", "wai.neural.int_mlicpp"}
class Refusal(Exception):
def __init__(self, code, detail):
super().__init__(f"{code}: {detail}")
self.code = code
def companion_only(cap):
# SPEC §5: the enhancement layer and the HDR metadata descriptors.
return cap == "wai.video.lcevc" or cap.startswith("wai.meta.")
def limits_of(t):
"""The limits a title is measured under: each figure the lower of the title's and
the tools' stated one. A title lowers a limit and never raises one."""
l = t.get("limits") or {}
cost = [min(a, b) for a, b in zip(l.get("cost") or TOOL_COST, TOOL_COST)]
budget = l.get("chain_budget")
return cost, TOOL_CHAIN_BUDGET if budget is None else min(budget, TOOL_CHAIN_BUDGET)
def header_cost(cap, payload):
"""The output bytes and symbols a payload's header fixes (integer-payloads §7);
None for a figure the header alone does not fix."""
v = IPV.parse(cap, payload)
magic = IPV.CAP_MAGIC[cap]
if magic == b"WIV1":
frame = v["h"] * v["w"] * 3
if v["keyframe"] == 0:
return frame * v["n_frames"], frame * v["n_frames"]
k = IPV.parse_wih1(v["wih1"])
return frame * v["n_frames"], frame * (v["n_frames"] - 1) + k["n_z"] * k["zh"] * k["zw"] + k["m"] * k["sh"] * k["sw"]
if magic == b"WIH1":
return None, v["n_z"] * v["zh"] * v["zw"] + v["m"] * v["sh"] * v["sw"]
if magic == b"WIA1":
return 2 * v["t_out"], v["n_ch"] * v["lat_len"]
return v["attr"] * v["s"] * v["s"], v["c"] * v["sh"] * v["sw"]
def base_checks(env, cost):
"""SPEC §4's dispatch and the header's cost: the capability a base decodes through,
or a Refusal."""
m, _, payload = SV.read_wai1(env)
cap = m["model_requirement"]["capability"]
fb = m["model_requirement"].get("fallback")
if cap in SV.UNIT_KIND or companion_only(cap):
raise Refusal("missing_capability", f"{cap} is never dispatched; its fallback is not reached")
for via in [cap, fb]:
if via is None or via not in INT_PATH:
continue
if via in IPV.CAP_MAGIC:
out, sym = header_cost(via, payload)
if (out is not None and out > cost[0]) or sym > cost[1]:
# A cost past the limit makes the capability unsupported; the fallback is
# tried next (SPEC §4).
continue
return via
if cap in IPV.CAP_MAGIC:
raise Refusal("cost_over_limit", "the header declares a decode past the limit")
raise Refusal("missing_capability", "no capability this decode takes")
def sha(b):
return hashlib.sha256(b).hexdigest()
def read(rel):
with open(os.path.join(WAI, rel), "rb") as f:
return f.read()
# ---------------------------------------------------------------- §3, the byte rule
class NotCountable(Exception):
def __init__(self, code, component=None):
super().__init__(code)
self.code = code
self.component = component
def has_format_check(cap):
return cap in IPV.CAP_MAGIC or cap in SV.UNIT_KIND
def format_check(cap, payload):
"""§3.2: a payload counts only in a format WAI checks for its capability."""
if not has_format_check(cap):
raise NotCountable("no-format-check")
try:
if cap in SV.UNIT_KIND:
SV.parse_wir1(payload, SV.UNIT_KIND[cap])
else:
IPV.parse(cap, payload)
except (SV.Refused, IPV.Refused) as e:
raise NotCountable(e.code)
def count_any(b):
"""The counted bytes of a WAI1 or WAI2 envelope (§3.1), or NotCountable with the
code: `not-an-envelope`, `encrypted`, a format's code or `no-format-check` (the
capability's, then a WAI1 fallback's that WAI checks), `sha256-mismatch` (a WAI2
component), `trailing`."""
if b[:4] == b"WAI1":
try:
m, text, payload = SV.read_wai1(b)
except SV.NotWai1:
raise NotCountable("not-an-envelope")
if not SV.plaintext(m):
raise NotCountable("encrypted")
mr = m["model_requirement"]
format_check(mr["capability"], payload)
fb = mr.get("fallback")
if fb is not None and has_format_check(fb):
format_check(fb, payload)
header, man, pl = 12, len(text), len(payload)
elif b[:4] == b"WAI2":
try:
text, entries, payloads = SV.read_wai2(b)
except SV.NotWai2:
raise NotCountable("not-an-envelope")
sec = json.loads(text).get("security")
if sec is not None and sec.get("encryption") not in (None, "none"):
raise NotCountable("encrypted")
for i, (e, p) in enumerate(zip(entries, payloads)):
if e.get("sha256") is not None and e["sha256"] != sha(p):
raise NotCountable("sha256-mismatch", i)
try:
format_check(e["capability"], p)
except NotCountable as x:
raise NotCountable(x.code, i)
header, man, pl = 8 + 2 + 8 * len(entries), len(text.encode()), sum(len(p) for p in payloads)
else:
raise NotCountable("not-an-envelope")
total = header + man + pl
if total != len(b):
raise NotCountable("trailing")
return {"header": header, "manifest": man, "payload": pl, "total": total}
def count(env):
"""A measured WAI1 object: its manifest, payload and counted bytes."""
counted = count_any(env)
m, _, payload = SV.read_wai1(env)
return m, payload, counted
# ---------------------------------------------------------------- §4, quality
def luma(px):
return [(77 * px[i] + 150 * px[i + 1] + 29 * px[i + 2] + 128) >> 8 for i in range(0, len(px), 3)]
def starts(length, win):
"""A window's offsets along an axis: every multiple of 4 up to length - win, then
length - win itself when it is not one, so every pixel is in a window."""
last = length - win
return list(range(0, last + 1, 4)) + ([last] if last % 4 else [])
def image_parts(h, w, a, b):
sse_rgb = sum((x - y) ** 2 for x, y in zip(a, b))
ya, yb = luma(a), luma(b)
sse_y = sum((x - y) ** 2 for x, y in zip(ya, yb))
ww, wh = min(8, w), min(8, h)
n = ww * wh
tot = cnt = 0
for y0 in starts(h, wh):
for x0 in starts(w, ww):
sx = sy = sxx = syy = sxy = 0
for yy in range(y0, y0 + wh):
row = yy * w
for xx in range(x0, x0 + ww):
p, q = ya[row + xx], yb[row + xx]
sx += p
sy += q
sxx += p * p
syy += q * q
sxy += p * q
a1 = 20000 * sx * sy + 65025 * n * n
a2 = 20000 * (n * sxy - sx * sy) + 585225 * n * n
b1 = 10000 * (sx * sx + sy * sy) + 65025 * n * n
b2 = 10000 * ((n * sxx - sx * sx) + (n * syy - sy * sy)) + 585225 * n * n
tot += (a1 * a2 * (1 << 32)) // (b1 * b2)
cnt += 1
return len(a), sse_rgb, sse_y, tot, cnt
def picture(w, h, seed, mul):
"""quality.json's picture: byte c of pixel (x, y)."""
return [(x * mul + y * (mul + 3) + 29 * c + 17 * seed + 5 * ((x * y + seed) % 13)) % 256
for y in range(h) for x in range(w) for c in range(3)]
def s16(b):
return list(struct.unpack(f"<{len(b) // 2}h", b))
def pooled(cls, pairs):
"""pairs: (test samples, test shape, reference canonical bytes, reference shape).
Pooled over the pairs of equal shape; None when there is none."""
pairs = [(t, s, r) for t, s, r, rs in pairs if s == rs]
if not pairs:
return None
if cls == "rgb8":
n = sr = sy = tot = cnt = 0
for t, s, r in pairs:
a, b, c, d, e = image_parts(s[0], s[1], list(r), t)
n, sr, sy, tot, cnt = n + a, sr + b, sy + c, tot + d, cnt + e
return {"kind": "rgb8", "n_samples": n, "sse_rgb": str(sr), "sse_y": str(sy), "ssim_y_q32": tot // cnt, "windows": cnt}
if cls == "pcm16":
n = e = en = 0
for t, s, r in pairs:
ref = s16(r)
n += len(ref)
e += sum((x - y) ** 2 for x, y in zip(ref, t))
en += sum(x * x for x in ref)
return {"kind": "pcm16", "n_samples": n, "sse": str(e), "ref_energy": str(en)}
n = e = mx = 0
for t, s, r in pairs:
n += len(r)
for x, y in zip(r, t):
d = abs(x - y)
e += d * d
mx = max(mx, d)
return {"kind": "tensor-i64", "n": n, "sse": str(e), "max_abs": mx}
def canonical(cls, t):
return struct.pack(f"<{len(t)}h", *t) if cls == "pcm16" else bytes(t)
# ---------------------------------------------------------------- Appendix C
def leaf(unit, layers, dec, digest):
return hashlib.sha256(b"wai:out-leaf\x01" + struct.pack(">QHH", unit, layers, dec) + bytes.fromhex(digest)).digest()
def mth(d):
if len(d) == 1:
return d[0]
k = 1
while k * 2 < len(d):
k *= 2
return hashlib.sha256(b"wai:out-node\x01" + mth(d[:k]) + mth(d[k:])).digest()
# ---------------------------------------------------------------- a title
class Fixture:
def __init__(self, corpus_rel, name, sub="fixtures"):
self.dir = f"{corpus_rel}/{sub}/{name}"
self.t = json.loads(read(f"{self.dir}/title.json"))
if self.t["format"] != "wai-measure-title/1":
raise ValueError("title format")
self.read = {} # path -> bytes, every file the content identity covers
def obj(self, rel, measured=True):
b = read(rel)
if measured:
self.read[rel] = b
return b
def replay(bases, layers, cost=None):
"""bases: list of dicts (hash, cap, pin, first, units: [(shape, samples)]);
layers: list of (path, env). Returns per unit (global, base index) the list of
stages (layers count, capability, pin, shape, samples)."""
traces = {}
producer = {}
for bi, b in enumerate(bases):
for u, (shape, s) in enumerate(b["units"]):
g = b["first"] + u
traces[g] = [(1, b["cap"], b["pin"], shape, s)]
producer[g] = b["hash"]
parsed = []
for path, env in layers:
m, _, payload = SV.read_wai1(env)
cap = m["model_requirement"]["capability"]
ref = SV.parse_refines(m["refines"])
bi = next(i for i, b in enumerate(bases) if b["hash"] == ref["base"])
frm, cnt = SV.units_of(ref, len(bases[bi]["units"]))
w = SV.parse_wir1(payload, SV.UNIT_KIND[cap])
out, sym, work = SV.cost(w)
if cost is not None and (out > cost[0] or sym > cost[1] or work > cost[2]):
raise Refusal("cost-over-limit", f"{path}: the layer's declared cost passes the limit")
SV.check_streams(w)
if w["n_units"] != cnt:
raise ValueError(f"{path}: n_units is not the span")
parsed.append((path, cap, ref, bi, frm, cnt, w, SV.content_hash(env)))
top = max([p[2]["layer"] for p in parsed], default=0)
for k in range(1, top + 1):
for path, cap, ref, bi, frm, cnt, w, h in parsed:
if ref["layer"] != k:
continue
for i in range(cnt):
g = bases[bi]["first"] + frm + i
stage = len(traces[g]) - 1
want = SV.producer_for(ref, i)
if stage != k - 1 or producer[g] != want:
raise ValueError(f"{path}: unit {g} is not ready for layer {k}")
_, _, _, shape, s = traces[g][-1]
if not SV.accepts_below(w, shape):
raise ValueError(f"{path}: unit {g}'s shape")
out = SV.apply_unit(w, i, s, shape)
traces[g].append((k + 1, cap, None, w["d"], out))
producer[g] = h
return [traces[g] for g in sorted(traces)]
def evaluate(f):
t = f.t
bases = []
objects = []
base_extra = []
cost, budget = limits_of(t)
for bi, b in enumerate(t["bases"]):
if b.get("envelope") is not None:
# A base by its envelope alone: the corpus names one only to be refused.
env = f.obj(b["envelope"])
base_checks(env, cost)
raise ValueError(f"{b['envelope']}: a base with no stage 0 to check passes the base checks")
case = json.loads(read(f"{b['case']}/case.json"))
cb = case["bases"][b["base"]]
path = f"{b['case']}/{cb['file']}"
env = f.obj(path)
via = base_checks(env, cost)
m, payload, counted = count(env)
cap = m["model_requirement"]["capability"]
if SV.content_hash(env) != cb["object"] or via != cb["decoded_via"] or (via != cap) != cb["via_fallback"]:
raise ValueError(f"{path}: not the case's base")
stage0 = read(f"{b['case']}/{cb['stage0']}")
if sha(stage0) != cb["stage0_sha256"]:
raise ValueError(f"{path}: stage0")
set_bytes = 0
for _, sp in sorted(cb["decode"]["set"].items()):
set_bytes += len(f.obj(sp))
units, at = [], 0
for u in cb["units"]:
shape = tuple(u["shape"])
n = shape[0] * shape[1] * shape[2]
if u["elem"] == "i16":
units.append((shape, s16(stage0[at:at + 2 * n])))
at += 2 * n
else:
units.append((shape, list(stage0[at:at + n])))
at += n
if sum(len(x) * (2 if u["elem"] == "i16" else 1) for x, u in zip((v for _, v in units), cb["units"])) > budget:
raise Refusal("chain-over-limit", "the base's units pass the chain budget")
prior = (m.get("conditioning") or {}).get("prior")
pin = prior["sha256"] if cb["pin_resolved"] and prior else None
bases.append({"hash": cb["object"], "cap": cap, "pin": pin, "first": b["first_unit"], "units": units})
base_extra.append({"object": cb["object"], "capability": cap, "pin": pin, "tier": cb["tier"],
"first_unit": b["first_unit"], "capture_us": b["capture_us"], "units": len(units),
"param_set_bytes": set_bytes})
objects.append({"file": path, "object": cb["object"], "capability": cap, "base": bi, "layer": 0,
"units": [b["first_unit"], len(units)], "counted": counted})
cls = UNIT_CLASS[bases[0]["cap"]]
layers = []
for name in t["layers"]:
path = f"{f.dir}/{name}"
env = f.obj(path)
m, _, counted = count(env)
ref = SV.parse_refines(m["refines"])
bi = next(i for i, b in enumerate(bases) if b["hash"] == ref["base"])
frm, cnt = SV.units_of(ref, len(bases[bi]["units"]))
objects.append({"file": path, "object": SV.content_hash(env), "capability": m["model_requirement"]["capability"],
"base": bi, "layer": ref["layer"], "units": [bases[bi]["first"] + frm, cnt], "counted": counted})
layers.append((path, env))
traces = replay(bases, layers, cost)
finals = [tr[-1] for tr in traces]
src = None
if t.get("source") is not None:
sb = f.obj(f"{f.dir}/{t['source']['file']}")
if sha(sb) != t["source"]["sha256"]:
raise ValueError("source sha256")
src, at = [], 0
for fin in finals:
n = len(canonical(cls, fin[4]))
src.append(sb[at:at + n])
at += n
if at != len(sb):
raise ValueError("source length")
# decoders, by first use as presentations run by (layers, unit)
order = sorted(((st[0], tr_i, s_i) for tr_i, tr in enumerate(traces) for s_i, st in enumerate(tr)))
decoders = []
for _, tr_i, s_i in order:
st = traces[tr_i][s_i]
key = (st[1], st[2])
if key not in decoders:
decoders.append(key)
units_g = sorted(bases[bi]["first"] + u for bi in range(len(bases)) for u in range(len(bases[bi]["units"])))
presentations = []
leaves = []
for tr_i, tr in enumerate(traces):
fin = finals[tr_i]
fin_bytes = canonical(cls, fin[4])
for st in tr:
digest = sha(canonical(cls, st[4]))
dec = decoders.index((st[1], st[2]))
presentations.append({
"unit": units_g[tr_i], "layers": st[0], "decoder": dec, "shape": list(st[3]), "out_sha256": digest,
"vs_source": pooled(cls, [(st[4], st[3], src[tr_i], fin[3])]) if src is not None else None,
"vs_final": pooled(cls, [(st[4], st[3], fin_bytes, fin[3])]),
})
leaves.append(((units_g[tr_i], st[0]), leaf(units_g[tr_i], st[0], dec, digest)))
leaves.sort()
root = mth([l for _, l in leaves]).hex() if leaves else None
top = max(o["layer"] for o in objects)
prefixes = []
cum = [0] * len(bases)
for k in range(top + 1):
per_base = []
for bi in range(len(bases)):
s = sum(o["counted"]["total"] for o in objects if o["base"] == bi and o["layer"] == k)
cum[bi] += s
per_base.append({"stage_counted": s, "counted": cum[bi]})
at = [tr[min(k, len(tr) - 1)] for tr in traces]
prefixes.append({
"layers": k + 1,
"stage_counted": sum(p["stage_counted"] for p in per_base),
"counted": sum(p["counted"] for p in per_base),
"per_base": per_base,
"units": sum(1 for tr in traces if len(tr) > k),
"vs_source": pooled(cls, [(st[4], st[3], src[i], finals[i][3]) for i, st in enumerate(at)]) if src is not None else None,
"vs_final": pooled(cls, [(st[4], st[3], canonical(cls, finals[i][4]), finals[i][3]) for i, st in enumerate(at)]),
})
bfpf = next((o["counted"]["total"] for o in objects if o["layer"] == 0 and o["units"][0] == 0 and o["units"][1] >= 1), None)
overhead = []
items = [(o["object"], o["counted"]["total"]) for o in objects]
base_total = sum(o["counted"]["total"] for o in objects if o["layer"] == 0)
ref_items = []
if src is not None:
overhead.append({"label": "staged", "counted": sum(o["counted"]["total"] for o in objects),
"vs_source": pooled(cls, [(fin[4], fin[3], src[i], fin[3]) for i, fin in enumerate(finals)]),
"out_sha256": [sha(canonical(cls, fin[4])) for fin in finals]})
for r in t["references"]:
rl = []
counted = base_total
for name in r["layers"]:
path = f"{f.dir}/{name}"
env = f.obj(path)
counted += count(env)[2]["total"]
rl.append((path, env))
ref_items.append((SV.content_hash(env), count(env)[2]["total"]))
ref_items.extend((o["object"], o["counted"]["total"]) for o in objects if o["layer"] == 0)
rt = replay(bases, rl, cost)
rf = [tr[-1] for tr in rt]
overhead.append({"label": r["label"], "counted": counted,
"vs_source": pooled(cls, [(x[4], x[3], src[i], finals[i][3]) for i, x in enumerate(rf)]),
"out_sha256": [sha(canonical(cls, x[4])) for x in rf]})
items += ref_items
dedup = dedup_totals(items)
# §11.1's content identity: the measured content, by path.
h = b"wai:measure-title\x01"
for p in sorted(f.read):
b = f.read[p]
h += struct.pack(">Q", len(p)) + p.encode() + struct.pack(">Q", len(b)) + b
return {
"title": t["title"],
"content_id": blake3(h).hex(),
"framing": "registered",
"unit_class": cls,
"unit_us": t["unit_us"],
"bases": base_extra,
"objects": objects,
"bfpf_min": bfpf,
"prefixes": prefixes,
"decoders": [{"capability": c, "pin": p} for c, p in decoders],
"presentations": presentations,
"output_root": root,
"leaf_count": len(presentations),
"overhead": overhead,
"dedup": dedup,
}, items
def dedup_totals(items):
seen = {}
d = {"objects": 0, "unique": 0, "referenced_bytes": 0, "stored_bytes": 0}
for h, n in items:
d["objects"] += 1
d["referenced_bytes"] += n
if h in seen:
if seen[h] != n:
raise ValueError(f"object {h} has two lengths")
else:
seen[h] = n
d["unique"] += 1
d["stored_bytes"] += n
return d
def diff(a, b, at, out):
"""Where two JSON values differ, for the report."""
if type(a) is not type(b) or (not isinstance(a, (dict, list)) and a != b):
out.append(f"{at}: {json.dumps(a)[:80]} != {json.dumps(b)[:80]}")
elif isinstance(a, dict):
for k in sorted(set(a) | set(b)):
diff(a.get(k), b.get(k), f"{at}.{k}", out)
elif isinstance(a, list):
if len(a) != len(b):
out.append(f"{at}: {len(a)} items != {len(b)}")
for i, (x, y) in enumerate(zip(a, b)):
diff(x, y, f"{at}[{i}]", out)
def main(d):
d = os.path.abspath(d)
rel = os.path.relpath(d, WAI)
goldens = json.load(open(os.path.join(d, "goldens.json")))
names = sorted(n for n in os.listdir(os.path.join(d, "fixtures")) if os.path.isfile(os.path.join(d, "fixtures", n, "title.json")))
fails = []
got_titles = [g["title"] for g in goldens["fixtures"]]
if got_titles != names:
fails.append(f"goldens.json covers {got_titles}; the fixtures are {names}")
all_items = []
for name in names:
f = Fixture(rel, name)
try:
ev, items = evaluate(f)
except (ValueError, Refusal, NotCountable, SV.Refused, SV.Malformed, IPV.Refused, KeyError, StopIteration) as e:
fails.append(f"{name}: {e!r}")
continue
all_items += items
exp = json.load(open(os.path.join(d, "fixtures", name, f.t["expect"])))
mine = [{"unit": p["unit"], "layers": p["layers"], "out_sha256": p["out_sha256"]} for p in ev["presentations"]]
if exp["presentations"] != mine:
fails.append(f"{name}: this decode is not the fixture tool's reconstruction (expect.json)")
refs = [{"label": o["label"], "out_sha256": o["out_sha256"]} for o in ev["overhead"] if o["label"] != "staged"]
if exp["references"] != refs:
fails.append(f"{name}: the references' decodes are not the fixture tool's reconstruction")
g = next((x for x in goldens["fixtures"] if x["title"] == name), None)
out = []
diff(ev, g, name, out)
fails += out[:20]
print(f" {name}: {len(ev['objects'])} objects, {ev['leaf_count']} presentations, root {ev['output_root']}"
f"{'' if not out else ' — DIFFERS'}")
# The counting cases: each object's counted bytes, or its code, as counting.json
# states them.
for c in json.load(open(os.path.join(d, "counting.json")))["cases"]:
b = read(c["file"])
try:
got = {"counted": count_any(b), "refused": None, "component": None}
except NotCountable as e:
got = {"counted": None, "refused": e.code, "component": e.component}
want = {k: c[k] for k in ("counted", "refused", "component")}
if got != want:
fails.append(f"counting {c['file']}: {got} != {want}")
# The quality cases: §4.2's integers for each picture pair quality.json states.
for c in json.load(open(os.path.join(d, "quality.json")))["cases"]:
w, h = c["w"], c["h"]
n, sse, ssey, tot, cnt = image_parts(h, w, picture(w, h, *c["reference"]), picture(w, h, *c["test"]))
got = {"n_samples": n, "sse_rgb": str(sse), "sse_y": str(ssey), "ssim_y_q32": tot // cnt, "windows": cnt}
want = {k: c[k] for k in got}
if got != want:
fails.append(f"quality {w}x{h}: {got} != {want}")
refusals = os.path.join(d, "refusals")
for name in sorted(os.listdir(refusals)) if os.path.isdir(refusals) else []:
f = Fixture(rel, name, "refusals")
want = f.t.get("refused")
try:
evaluate(f)
fails.append(f"refusals/{name}: evaluated, though it must be refused {want}")
except Refusal as e:
if e.code != want:
fails.append(f"refusals/{name}: refused {e.code}, the title states {want}")
else:
print(f" refusals/{name}: refused {e.code}")
except (ValueError, SV.Refused, SV.Malformed, SV.NotWai1, IPV.Refused, KeyError) as e:
fails.append(f"refusals/{name}: {e!r}")
rec = dedup_totals(all_items)
if goldens["record"]["dedup"] != rec:
fails.append(f"record dedup {rec} != goldens {goldens['record']['dedup']}")
if fails:
for x in fails:
print("FAIL", x)
sys.exit(1)
print("staged-measure: every fixture verifies independently")
if __name__ == "__main__":
main(sys.argv[1] if len(sys.argv) > 1 else HERE)

244
wai/staged-measure-conformance/quality.json Normal file
View file

@ -0,0 +1,244 @@
{
"format": "wai-measure-quality/1",
"about": "Picture pairs and their §4.2 integers (extensions/staged-measurement.md), written by wai_stage_measure generate-goldens and required by wai_stage_measure verify and measure_verify.py. Byte c of pixel (x, y) of the picture [seed, mul] is (x*mul + y*(mul + 3) + 29*c + 17*seed + 5*((x*y + seed) mod 13)) mod 256.",
"cases": [
{
"w": 1,
"h": 1,
"reference": [
1,
5
],
"test": [
2,
7
],
"n_samples": 3,
"sse_rgb": "1452",
"sse_y": "484",
"ssim_y_q32": 3986842579,
"windows": 1
},
{
"w": 5,
"h": 5,
"reference": [
3,
11
],
"test": [
3,
13
],
"n_samples": 75,
"sse_rgb": "65392",
"sse_y": "2145",
"ssim_y_q32": 4227018163,
"windows": 1
},
{
"w": 3,
"h": 100,
"reference": [
1,
7
],
"test": [
2,
7
],
"n_samples": 900,
"sse_rgb": "4705154",
"sse_y": "532902",
"ssim_y_q32": 2841543333,
"windows": 24
},
{
"w": 8,
"h": 8,
"reference": [
2,
9
],
"test": [
4,
9
],
"n_samples": 192,
"sse_rgb": "1321821",
"sse_y": "175220",
"ssim_y_q32": 1280208992,
"windows": 1
},
{
"w": 9,
"h": 9,
"reference": [
5,
3
],
"test": [
5,
4
],
"n_samples": 243,
"sse_rgb": "256360",
"sse_y": "6844",
"ssim_y_q32": 4122222193,
"windows": 4
},
{
"w": 11,
"h": 11,
"reference": [
1,
5
],
"test": [
2,
7
],
"n_samples": 363,
"sse_rgb": "1366484",
"sse_y": "201849",
"ssim_y_q32": 3172402069,
"windows": 4
},
{
"w": 13,
"h": 5,
"reference": [
6,
2
],
"test": [
7,
2
],
"n_samples": 195,
"sse_rgb": "425640",
"sse_y": "33092",
"ssim_y_q32": 2427457119,
"windows": 3
},
{
"w": 12,
"h": 20,
"reference": [
4,
9
],
"test": [
5,
9
],
"n_samples": 720,
"sse_rgb": "4072752",
"sse_y": "548256",
"ssim_y_q32": 2404129353,
"windows": 8
},
{
"w": 16,
"h": 16,
"reference": [
1,
5
],
"test": [
1,
6
],
"n_samples": 768,
"sse_rgb": "3329152",
"sse_y": "245154",
"ssim_y_q32": 3542145100,
"windows": 9
},
{
"w": 17,
"h": 9,
"reference": [
2,
3
],
"test": [
5,
3
],
"n_samples": 459,
"sse_rgb": "2191911",
"sse_y": "500794",
"ssim_y_q32": 1115121540,
"windows": 8
},
{
"w": 9,
"h": 33,
"reference": [
8,
13
],
"test": [
9,
14
],
"n_samples": 891,
"sse_rgb": "8244787",
"sse_y": "1469842",
"ssim_y_q32": 1291000068,
"windows": 16
},
{
"w": 19,
"h": 23,
"reference": [
3,
1
],
"test": [
3,
2
],
"n_samples": 1311,
"sse_rgb": "3944806",
"sse_y": "322676",
"ssim_y_q32": 3344287242,
"windows": 20
},
{
"w": 40,
"h": 24,
"reference": [
3,
7
],
"test": [
3,
8
],
"n_samples": 2880,
"sse_rgb": "19201376",
"sse_y": "3078475",
"ssim_y_q32": 2198626257,
"windows": 45
},
{
"w": 41,
"h": 27,
"reference": [
9,
4
],
"test": [
1,
4
],
"n_samples": 3321,
"sse_rgb": "50241520",
"sse_y": "11485039",
"ssim_y_q32": -168294012,
"windows": 60
}
]
}

23
wai/staged-measure-conformance/refusals/base-over-budget/title.json Normal file
View file

@ -0,0 +1,23 @@
{
"format": "wai-measure-title/1",
"title": "base-over-budget",
"about": "a base whose units' output passes the chain budget the title states",
"unit_us": 40000,
"bases": [
{
"case": "staged-conformance/cases/base_over_budget",
"base": 0,
"first_unit": 0,
"capture_us": null
}
],
"layers": [],
"source": null,
"references": [],
"expect": null,
"limits": {
"cost": null,
"chain_budget": 383
},
"refused": "chain-over-limit"
}

28
wai/staged-measure-conformance/refusals/clip-1gib-limits-raised/title.json Normal file
View file

@ -0,0 +1,28 @@
{
"format": "wai-measure-title/1",
"title": "clip-1gib-limits-raised",
"about": "the 1 GiB clip under a title that states limits of 2^40: a title lowers a limit and never raises one, so the tools' stated limit refuses it before anything is allocated",
"unit_us": 40000,
"bases": [
{
"envelope": "wai-rs/tests/vectors/staged_bounds/clip_1gib.wai",
"set": {},
"first_unit": 0,
"capture_us": null
}
],
"layers": [],
"source": null,
"references": [],
"expect": null,
"limits": {
"cost": [
1099511627776,
1099511627776,
1099511627776,
1099511627776
],
"chain_budget": 1099511627776
},
"refused": "cost_over_limit"
}

20
wai/staged-measure-conformance/refusals/clip-1gib/title.json Normal file
View file

@ -0,0 +1,20 @@
{
"format": "wai-measure-title/1",
"title": "clip-1gib",
"about": "a wai.video.int_motion base whose header declares 1 GiB of output, refused before anything is allocated for it",
"unit_us": 40000,
"bases": [
{
"envelope": "wai-rs/tests/vectors/staged_bounds/clip_1gib.wai",
"set": {},
"first_unit": 0,
"capture_us": null
}
],
"layers": [],
"source": null,
"references": [],
"expect": null,
"limits": null,
"refused": "cost_over_limit"
}

20
wai/staged-measure-conformance/refusals/clip-64mib/title.json Normal file
View file

@ -0,0 +1,20 @@
{
"format": "wai-measure-title/1",
"title": "clip-64mib",
"about": "a wai.video.int_motion base whose header declares 64 MiB of output, past the tools' stated limit",
"unit_us": 40000,
"bases": [
{
"envelope": "wai-rs/tests/vectors/staged_bounds/clip_64mib.wai",
"set": {},
"first_unit": 0,
"capture_us": null
}
],
"layers": [],
"source": null,
"references": [],
"expect": null,
"limits": null,
"refused": "cost_over_limit"
}

20
wai/staged-measure-conformance/refusals/inert-companion-primary/title.json Normal file
View file

@ -0,0 +1,20 @@
{
"format": "wai-measure-title/1",
"title": "inert-companion-primary",
"about": "a companion-only capability with an integer fallback: inert, the fallback never reached",
"unit_us": 40000,
"bases": [
{
"envelope": "wai-rs/tests/vectors/staged_bounds/inert_companion_primary.wai",
"set": {},
"first_unit": 0,
"capture_us": null
}
],
"layers": [],
"source": null,
"references": [],
"expect": null,
"limits": null,
"refused": "missing_capability"
}

20
wai/staged-measure-conformance/refusals/inert-refinement-primary/title.json Normal file
View file

@ -0,0 +1,20 @@
{
"format": "wai-measure-title/1",
"title": "inert-refinement-primary",
"about": "a refinement-only capability with an integer fallback: inert, the fallback never reached",
"unit_us": 40000,
"bases": [
{
"envelope": "wai-rs/tests/vectors/staged_bounds/inert_refinement_primary.wai",
"set": {},
"first_unit": 0,
"capture_us": null
}
],
"layers": [],
"source": null,
"references": [],
"expect": null,
"limits": null,
"refused": "missing_capability"
}

BIN
wai/staged-measure-conformance/refusals/layer-over-limit/L01.wai Normal file
View file

Binary file not shown.

30
wai/staged-measure-conformance/refusals/layer-over-limit/title.json Normal file
View file

@ -0,0 +1,30 @@
{
"format": "wai-measure-title/1",
"title": "layer-over-limit",
"about": "the still title's up2 layer under a cost limit its base passes and its layer does not: the layer's declared output, 3072 bytes, passes 1024",
"unit_us": 40000,
"bases": [
{
"case": "staged-conformance/cases/image_int_hyper_2l",
"base": 0,
"first_unit": 0,
"capture_us": null
}
],
"layers": [
"L01.wai"
],
"source": null,
"references": [],
"expect": null,
"limits": {
"cost": [
1024,
1024,
268435456,
274877906944
],
"chain_budget": null
},
"refused": "cost-over-limit"
}

186
wai/staged-measure-conformance/sim/cases.json Normal file
View file

@ -0,0 +1,186 @@
{
"format": "wai-sim-cases/1",
"about": "Delivery simulator cases (extensions/staged-measurement.md §10, Appendix B): a model (a file, or fixture:<name> for a measured title's objects and counted bytes), a plan, and what the plan must show: within-bound, exceeds:<plan> (its largest stall passes that plan's bound), or unbounded.",
"cases": [
{
"name": "gop3/strict-subgroup",
"model": "models/gop3.json",
"plan": "plans/strict-subgroup.json",
"claim": "within-bound"
},
{
"name": "gop3/strict-slow-timeout",
"model": "models/gop3.json",
"plan": "plans/strict-slow-timeout.json",
"claim": "within-bound"
},
{
"name": "gop3/fifo-timeout-subgroup",
"model": "models/gop3.json",
"plan": "plans/fifo-timeout-subgroup.json",
"claim": "within-bound"
},
{
"name": "gop3/fifo-timeout-window",
"model": "models/gop3.json",
"plan": "plans/fifo-timeout-window.json",
"claim": "within-bound"
},
{
"name": "gop3/fifo-timeout-datagram",
"model": "models/gop3.json",
"plan": "plans/fifo-timeout-datagram.json",
"claim": "within-bound"
},
{
"name": "gop3/fifo-timeout-short",
"model": "models/gop3.json",
"plan": "plans/fifo-timeout-short.json",
"claim": "within-bound"
},
{
"name": "gop3/fifo-no-timeout",
"model": "models/gop3.json",
"plan": "plans/fifo-no-timeout.json",
"claim": "exceeds:fifo-timeout-short"
},
{
"name": "gop3/strict-window",
"model": "models/gop3.json",
"plan": "plans/strict-window.json",
"claim": "within-bound"
},
{
"name": "gop3/fifo-decode",
"model": "models/gop3.json",
"plan": "plans/fifo-decode.json",
"claim": "exceeds:strict-window"
},
{
"name": "gop3/capped",
"model": "models/gop3.json",
"plan": "plans/capped.json",
"claim": "within-bound"
},
{
"name": "gop3/live-playout",
"model": "models/gop3.json",
"plan": "plans/live-playout.json",
"claim": "within-bound"
},
{
"name": "gop3/live-playout-present",
"model": "models/gop3.json",
"plan": "plans/live-playout-present.json",
"claim": "within-bound"
},
{
"name": "reorder-link",
"model": "models/reorder-link.json",
"plan": "plans/reorder-link.json",
"claim": "within-bound"
},
{
"name": "reorder-decode",
"model": "models/reorder-decode.json",
"plan": "plans/reorder-decode.json",
"claim": "within-bound"
},
{
"name": "audio/strict-subgroup",
"model": "fixture:audio",
"plan": "plans/strict-subgroup.json",
"claim": "within-bound"
},
{
"name": "audio/fifo-timeout-subgroup",
"model": "fixture:audio",
"plan": "plans/fifo-timeout-subgroup.json",
"claim": "within-bound"
},
{
"name": "audio/fifo-timeout-datagram",
"model": "fixture:audio",
"plan": "plans/fifo-timeout-datagram.json",
"claim": "within-bound"
},
{
"name": "audio/fifo-no-timeout",
"model": "fixture:audio",
"plan": "plans/fifo-no-timeout.json",
"claim": "unbounded"
},
{
"name": "clip/strict-subgroup",
"model": "fixture:clip",
"plan": "plans/strict-subgroup.json",
"claim": "within-bound"
},
{
"name": "clip/fifo-timeout-subgroup",
"model": "fixture:clip",
"plan": "plans/fifo-timeout-subgroup.json",
"claim": "within-bound"
},
{
"name": "clip/fifo-timeout-datagram",
"model": "fixture:clip",
"plan": "plans/fifo-timeout-datagram.json",
"claim": "within-bound"
},
{
"name": "clip/fifo-no-timeout",
"model": "fixture:clip",
"plan": "plans/fifo-no-timeout.json",
"claim": "unbounded"
},
{
"name": "splat/strict-subgroup",
"model": "fixture:splat",
"plan": "plans/strict-subgroup.json",
"claim": "within-bound"
},
{
"name": "splat/fifo-timeout-subgroup",
"model": "fixture:splat",
"plan": "plans/fifo-timeout-subgroup.json",
"claim": "within-bound"
},
{
"name": "splat/fifo-timeout-datagram",
"model": "fixture:splat",
"plan": "plans/fifo-timeout-datagram.json",
"claim": "within-bound"
},
{
"name": "splat/fifo-no-timeout",
"model": "fixture:splat",
"plan": "plans/fifo-no-timeout.json",
"claim": "unbounded"
},
{
"name": "still/strict-subgroup",
"model": "fixture:still",
"plan": "plans/strict-subgroup.json",
"claim": "within-bound"
},
{
"name": "still/fifo-timeout-subgroup",
"model": "fixture:still",
"plan": "plans/fifo-timeout-subgroup.json",
"claim": "within-bound"
},
{
"name": "still/fifo-timeout-datagram",
"model": "fixture:still",
"plan": "plans/fifo-timeout-datagram.json",
"claim": "within-bound"
},
{
"name": "still/fifo-no-timeout",
"model": "fixture:still",
"plan": "plans/fifo-no-timeout.json",
"claim": "unbounded"
}
]
}

369
wai/staged-measure-conformance/sim/models/gop3.json Normal file
View file

@ -0,0 +1,369 @@
{
"format": "wai-sim-model/1",
"unit_us": 40000,
"objects": [
{
"id": "g0-base",
"base": 0,
"layer": 0,
"units": [
0,
5
],
"bytes": 5000,
"capture_us": 200000
},
{
"id": "g0-l1-u0",
"base": 0,
"layer": 1,
"units": [
0,
1
],
"bytes": 1500,
"capture_us": 200000
},
{
"id": "g0-l1-u1",
"base": 0,
"layer": 1,
"units": [
1,
1
],
"bytes": 1537,
"capture_us": 200000
},
{
"id": "g0-l1-u2",
"base": 0,
"layer": 1,
"units": [
2,
1
],
"bytes": 1574,
"capture_us": 200000
},
{
"id": "g0-l1-u3",
"base": 0,
"layer": 1,
"units": [
3,
1
],
"bytes": 1611,
"capture_us": 200000
},
{
"id": "g0-l1-u4",
"base": 0,
"layer": 1,
"units": [
4,
1
],
"bytes": 1648,
"capture_us": 200000
},
{
"id": "g0-l2-u0",
"base": 0,
"layer": 2,
"units": [
0,
1
],
"bytes": 900,
"capture_us": 200000
},
{
"id": "g0-l2-u1",
"base": 0,
"layer": 2,
"units": [
1,
1
],
"bytes": 937,
"capture_us": 200000
},
{
"id": "g0-l2-u2",
"base": 0,
"layer": 2,
"units": [
2,
1
],
"bytes": 974,
"capture_us": 200000
},
{
"id": "g0-l2-u3",
"base": 0,
"layer": 2,
"units": [
3,
1
],
"bytes": 1011,
"capture_us": 200000
},
{
"id": "g0-l2-u4",
"base": 0,
"layer": 2,
"units": [
4,
1
],
"bytes": 1048,
"capture_us": 200000
},
{
"id": "g1-base",
"base": 1,
"layer": 0,
"units": [
5,
5
],
"bytes": 5700,
"capture_us": 400000
},
{
"id": "g1-l1-u0",
"base": 1,
"layer": 1,
"units": [
5,
1
],
"bytes": 1511,
"capture_us": 400000
},
{
"id": "g1-l1-u1",
"base": 1,
"layer": 1,
"units": [
6,
1
],
"bytes": 1548,
"capture_us": 400000
},
{
"id": "g1-l1-u2",
"base": 1,
"layer": 1,
"units": [
7,
1
],
"bytes": 1585,
"capture_us": 400000
},
{
"id": "g1-l1-u3",
"base": 1,
"layer": 1,
"units": [
8,
1
],
"bytes": 1622,
"capture_us": 400000
},
{
"id": "g1-l1-u4",
"base": 1,
"layer": 1,
"units": [
9,
1
],
"bytes": 1659,
"capture_us": 400000
},
{
"id": "g1-l2-u0",
"base": 1,
"layer": 2,
"units": [
5,
1
],
"bytes": 911,
"capture_us": 400000
},
{
"id": "g1-l2-u1",
"base": 1,
"layer": 2,
"units": [
6,
1
],
"bytes": 948,
"capture_us": 400000
},
{
"id": "g1-l2-u2",
"base": 1,
"layer": 2,
"units": [
7,
1
],
"bytes": 985,
"capture_us": 400000
},
{
"id": "g1-l2-u3",
"base": 1,
"layer": 2,
"units": [
8,
1
],
"bytes": 1022,
"capture_us": 400000
},
{
"id": "g1-l2-u4",
"base": 1,
"layer": 2,
"units": [
9,
1
],
"bytes": 1059,
"capture_us": 400000
},
{
"id": "g2-base",
"base": 2,
"layer": 0,
"units": [
10,
5
],
"bytes": 6400,
"capture_us": 600000
},
{
"id": "g2-l1-u0",
"base": 2,
"layer": 1,
"units": [
10,
1
],
"bytes": 1522,
"capture_us": 600000
},
{
"id": "g2-l1-u1",
"base": 2,
"layer": 1,
"units": [
11,
1
],
"bytes": 1559,
"capture_us": 600000
},
{
"id": "g2-l1-u2",
"base": 2,
"layer": 1,
"units": [
12,
1
],
"bytes": 1596,
"capture_us": 600000
},
{
"id": "g2-l1-u3",
"base": 2,
"layer": 1,
"units": [
13,
1
],
"bytes": 1633,
"capture_us": 600000
},
{
"id": "g2-l1-u4",
"base": 2,
"layer": 1,
"units": [
14,
1
],
"bytes": 1670,
"capture_us": 600000
},
{
"id": "g2-l2-u0",
"base": 2,
"layer": 2,
"units": [
10,
1
],
"bytes": 922,
"capture_us": 600000
},
{
"id": "g2-l2-u1",
"base": 2,
"layer": 2,
"units": [
11,
1
],
"bytes": 959,
"capture_us": 600000
},
{
"id": "g2-l2-u2",
"base": 2,
"layer": 2,
"units": [
12,
1
],
"bytes": 996,
"capture_us": 600000
},
{
"id": "g2-l2-u3",
"base": 2,
"layer": 2,
"units": [
13,
1
],
"bytes": 1033,
"capture_us": 600000
},
{
"id": "g2-l2-u4",
"base": 2,
"layer": 2,
"units": [
14,
1
],
"bytes": 1070,
"capture_us": 600000
}
]
}

50
wai/staged-measure-conformance/sim/models/reorder-decode.json Normal file
View file

@ -0,0 +1,50 @@
{
"format": "wai-sim-model/1",
"unit_us": 40000,
"objects": [
{
"id": "B0",
"base": 0,
"layer": 0,
"units": [
0,
1
],
"bytes": 10,
"capture_us": null
},
{
"id": "R0",
"base": 0,
"layer": 1,
"units": [
0,
1
],
"bytes": 10,
"capture_us": null
},
{
"id": "Bb",
"base": 2,
"layer": 0,
"units": [
2,
1
],
"bytes": 100000,
"capture_us": null
},
{
"id": "Ba",
"base": 1,
"layer": 0,
"units": [
1,
1
],
"bytes": 10,
"capture_us": null
}
]
}

50
wai/staged-measure-conformance/sim/models/reorder-link.json Normal file
View file

@ -0,0 +1,50 @@
{
"format": "wai-sim-model/1",
"unit_us": 40000,
"objects": [
{
"id": "B0",
"base": 0,
"layer": 0,
"units": [
0,
1
],
"bytes": 10,
"capture_us": 0
},
{
"id": "R0",
"base": 0,
"layer": 1,
"units": [
0,
1
],
"bytes": 1000,
"capture_us": 0
},
{
"id": "Bb",
"base": 2,
"layer": 0,
"units": [
2,
1
],
"bytes": 10,
"capture_us": 15
},
{
"id": "Ba",
"base": 1,
"layer": 0,
"units": [
1,
1
],
"bytes": 50000,
"capture_us": 20
}
]
}

29
wai/staged-measure-conformance/sim/plans/capped.json Normal file
View file

@ -0,0 +1,29 @@
{
"plan": "wai-sim/1",
"mode": "vod",
"link": {
"rtt_us": 40000,
"bytes_per_ms": 2500,
"chunk_bytes": 1200
},
"sender": {
"schedule": "strict-priority",
"carriage": "subgroup",
"layer_cap": 1,
"delivery_timeout_us": {
"base": null,
"refinement": null
},
"vod_window_us": null
},
"sink": {
"workers": 1,
"decode_order": "base-first",
"decode_us": {
"base": 4000,
"refinement": 2000
},
"playout_start_us": null,
"late_refinement": "discard"
}
}

29
wai/staged-measure-conformance/sim/plans/fifo-decode.json Normal file
View file

@ -0,0 +1,29 @@
{
"plan": "wai-sim/1",
"mode": "vod",
"link": {
"rtt_us": 40000,
"bytes_per_ms": 2500,
"chunk_bytes": 1200
},
"sender": {
"schedule": "strict-priority",
"carriage": "subgroup",
"layer_cap": null,
"delivery_timeout_us": {
"base": null,
"refinement": null
},
"vod_window_us": 180000
},
"sink": {
"workers": 1,
"decode_order": "fifo",
"decode_us": {
"base": 4000,
"refinement": 2000
},
"playout_start_us": null,
"late_refinement": "discard"
}
}

29
wai/staged-measure-conformance/sim/plans/fifo-no-timeout.json Normal file
View file

@ -0,0 +1,29 @@
{
"plan": "wai-sim/1",
"mode": "vod",
"link": {
"rtt_us": 40000,
"bytes_per_ms": 2500,
"chunk_bytes": 1200
},
"sender": {
"schedule": "fifo",
"carriage": "subgroup",
"layer_cap": null,
"delivery_timeout_us": {
"base": null,
"refinement": null
},
"vod_window_us": null
},
"sink": {
"workers": 1,
"decode_order": "base-first",
"decode_us": {
"base": 4000,
"refinement": 2000
},
"playout_start_us": null,
"late_refinement": "discard"
}
}

29
wai/staged-measure-conformance/sim/plans/fifo-timeout-datagram.json Normal file
View file

@ -0,0 +1,29 @@
{
"plan": "wai-sim/1",
"mode": "vod",
"link": {
"rtt_us": 40000,
"bytes_per_ms": 400,
"chunk_bytes": 12000
},
"sender": {
"schedule": "fifo",
"carriage": "datagram",
"layer_cap": null,
"delivery_timeout_us": {
"base": null,
"refinement": 30000
},
"vod_window_us": null
},
"sink": {
"workers": 1,
"decode_order": "base-first",
"decode_us": {
"base": 4000,
"refinement": 2000
},
"playout_start_us": null,
"late_refinement": "discard"
}
}

29
wai/staged-measure-conformance/sim/plans/fifo-timeout-short.json Normal file
View file

@ -0,0 +1,29 @@
{
"plan": "wai-sim/1",
"mode": "vod",
"link": {
"rtt_us": 40000,
"bytes_per_ms": 2500,
"chunk_bytes": 1200
},
"sender": {
"schedule": "fifo",
"carriage": "subgroup",
"layer_cap": null,
"delivery_timeout_us": {
"base": null,
"refinement": 1000
},
"vod_window_us": null
},
"sink": {
"workers": 1,
"decode_order": "base-first",
"decode_us": {
"base": 4000,
"refinement": 2000
},
"playout_start_us": null,
"late_refinement": "discard"
}
}

29
wai/staged-measure-conformance/sim/plans/fifo-timeout-subgroup.json Normal file
View file

@ -0,0 +1,29 @@
{
"plan": "wai-sim/1",
"mode": "vod",
"link": {
"rtt_us": 40000,
"bytes_per_ms": 400,
"chunk_bytes": 1200
},
"sender": {
"schedule": "fifo",
"carriage": "subgroup",
"layer_cap": null,
"delivery_timeout_us": {
"base": null,
"refinement": 30000
},
"vod_window_us": null
},
"sink": {
"workers": 1,
"decode_order": "base-first",
"decode_us": {
"base": 4000,
"refinement": 2000
},
"playout_start_us": null,
"late_refinement": "discard"
}
}

29
wai/staged-measure-conformance/sim/plans/fifo-timeout-window.json Normal file
View file

@ -0,0 +1,29 @@
{
"plan": "wai-sim/1",
"mode": "vod",
"link": {
"rtt_us": 40000,
"bytes_per_ms": 400,
"chunk_bytes": 1200
},
"sender": {
"schedule": "fifo",
"carriage": "subgroup",
"layer_cap": null,
"delivery_timeout_us": {
"base": null,
"refinement": 30000
},
"vod_window_us": 100000
},
"sink": {
"workers": 1,
"decode_order": "base-first",
"decode_us": {
"base": 4000,
"refinement": 2000
},
"playout_start_us": null,
"late_refinement": "discard"
}
}

29
wai/staged-measure-conformance/sim/plans/live-playout-present.json Normal file
View file

@ -0,0 +1,29 @@
{
"plan": "wai-sim/1",
"mode": "live",
"link": {
"rtt_us": 40000,
"bytes_per_ms": 2500,
"chunk_bytes": 1200
},
"sender": {
"schedule": "strict-priority",
"carriage": "subgroup",
"layer_cap": null,
"delivery_timeout_us": {
"base": null,
"refinement": null
},
"vod_window_us": null
},
"sink": {
"workers": 2,
"decode_order": "base-first",
"decode_us": {
"base": 4000,
"refinement": 2000
},
"playout_start_us": 215000,
"late_refinement": "present"
}
}

29
wai/staged-measure-conformance/sim/plans/live-playout.json Normal file
View file

@ -0,0 +1,29 @@
{
"plan": "wai-sim/1",
"mode": "live",
"link": {
"rtt_us": 40000,
"bytes_per_ms": 2500,
"chunk_bytes": 1200
},
"sender": {
"schedule": "strict-priority",
"carriage": "subgroup",
"layer_cap": null,
"delivery_timeout_us": {
"base": null,
"refinement": 100000
},
"vod_window_us": null
},
"sink": {
"workers": 2,
"decode_order": "base-first",
"decode_us": {
"base": 4000,
"refinement": 2000
},
"playout_start_us": 215000,
"late_refinement": "discard"
}
}

29
wai/staged-measure-conformance/sim/plans/reorder-decode.json Normal file
View file

@ -0,0 +1,29 @@
{
"plan": "wai-sim/1",
"mode": "vod",
"link": {
"rtt_us": 0,
"bytes_per_ms": 1000,
"chunk_bytes": 1000
},
"sender": {
"schedule": "fifo",
"carriage": "subgroup",
"layer_cap": null,
"delivery_timeout_us": {
"base": null,
"refinement": 20
},
"vod_window_us": null
},
"sink": {
"workers": 1,
"decode_order": "base-first",
"decode_us": {
"base": 100000,
"refinement": 100
},
"playout_start_us": null,
"late_refinement": "present"
}
}

29
wai/staged-measure-conformance/sim/plans/reorder-link.json Normal file
View file

@ -0,0 +1,29 @@
{
"plan": "wai-sim/1",
"mode": "live",
"link": {
"rtt_us": 0,
"bytes_per_ms": 1000,
"chunk_bytes": 1000
},
"sender": {
"schedule": "strict-priority",
"carriage": "subgroup",
"layer_cap": null,
"delivery_timeout_us": {
"base": null,
"refinement": null
},
"vod_window_us": null
},
"sink": {
"workers": 1,
"decode_order": "base-first",
"decode_us": {
"base": 100,
"refinement": 100
},
"playout_start_us": null,
"late_refinement": "present"
}
}

29
wai/staged-measure-conformance/sim/plans/strict-slow-timeout.json Normal file
View file

@ -0,0 +1,29 @@
{
"plan": "wai-sim/1",
"mode": "vod",
"link": {
"rtt_us": 40000,
"bytes_per_ms": 400,
"chunk_bytes": 1200
},
"sender": {
"schedule": "strict-priority",
"carriage": "subgroup",
"layer_cap": null,
"delivery_timeout_us": {
"base": null,
"refinement": 30000
},
"vod_window_us": null
},
"sink": {
"workers": 1,
"decode_order": "base-first",
"decode_us": {
"base": 4000,
"refinement": 2000
},
"playout_start_us": null,
"late_refinement": "discard"
}
}

29
wai/staged-measure-conformance/sim/plans/strict-subgroup.json Normal file
View file

@ -0,0 +1,29 @@
{
"plan": "wai-sim/1",
"mode": "vod",
"link": {
"rtt_us": 40000,
"bytes_per_ms": 2500,
"chunk_bytes": 1200
},
"sender": {
"schedule": "strict-priority",
"carriage": "subgroup",
"layer_cap": null,
"delivery_timeout_us": {
"base": null,
"refinement": null
},
"vod_window_us": null
},
"sink": {
"workers": 1,
"decode_order": "base-first",
"decode_us": {
"base": 4000,
"refinement": 2000
},
"playout_start_us": null,
"late_refinement": "discard"
}
}

29
wai/staged-measure-conformance/sim/plans/strict-window.json Normal file
View file

@ -0,0 +1,29 @@
{
"plan": "wai-sim/1",
"mode": "vod",
"link": {
"rtt_us": 40000,
"bytes_per_ms": 2500,
"chunk_bytes": 1200
},
"sender": {
"schedule": "strict-priority",
"carriage": "subgroup",
"layer_cap": null,
"delivery_timeout_us": {
"base": null,
"refinement": null
},
"vod_window_us": 180000
},
"sink": {
"workers": 1,
"decode_order": "base-first",
"decode_us": {
"base": 4000,
"refinement": 2000
},
"playout_start_us": null,
"late_refinement": "discard"
}
}

7176
wai/staged-measure-conformance/sim/results.json Normal file
View file

File diff suppressed because it is too large Load diff

329
wai/staged-measure-conformance/sim_verify.py Executable file
View file

@ -0,0 +1,329 @@
#!/usr/bin/env python3
"""Independent verifier of the delivery simulator's corpus (extensions/staged-measurement.md
§10, Appendix B).
Written from the extension text; it shares no code with wai-rs, which writes the
corpus. It implements Appendix B itself: availability, eligibility and the sender's
order, the link chunk by chunk, delivery timeouts (a datagram not started by its
deadline is dropped; a subgroup is reset at the deadline of an object not fully sent,
its sent bytes abandoned and the rest dropped), the layer cap, the sink's workers and
decode order, playout deadlines, the outputs, conservation, the base-only plan and each
unit's stall, and §10.4's bound. A fixture's model comes from measure_verify.py's own
evaluation of the title (each object's counted bytes, base, layer and units).
For every case of sim/cases.json it requires its own result, bound and largest stall
to equal sim/results.json's, and the case's claim to hold: `within-bound` (every stall
within the plan's bound), `exceeds:<plan>` (the largest stall passes that plan's bound),
or `unbounded` (the plan has none). It also requires the corpus to hold, for each part
of the bound a plan can lose (a fifo link with no refinement timeout; fifo decode), a
case whose stall passes a bounded plan's bound.
python3 sim_verify.py <staged-measure-conformance dir>
Exit status 0 only if every case verifies.
"""
import json
import os
import sys
HERE = os.path.dirname(os.path.abspath(__file__))
sys.path.insert(0, HERE)
import measure_verify as MV # noqa: E402
def ceil_div(a, b):
return -(-a // b)
def check_model(m):
covered = {}
for o in m["objects"]:
frm, cnt = o["units"]
if o["bytes"] < 1 or cnt < 1:
raise ValueError(f"{o['id']}: no bytes or no units")
for u in range(frm, frm + cnt):
if (u, o["layer"]) in covered:
raise ValueError(f"{o['id']}: unit {u} covered twice")
covered[(u, o["layer"])] = o["base"]
for o in m["objects"]:
if o["layer"] >= 1:
frm, cnt = o["units"]
for u in range(frm, frm + cnt):
if covered.get((u, o["layer"] - 1)) != o["base"]:
raise ValueError(f"{o['id']}: nothing below unit {u}")
def run_link(m, p):
objs = m["objects"]
n = len(objs)
snd, lnk = p["sender"], p["link"]
h = lnk["rtt_us"] // 2
ser = lambda c: ceil_div(c * 1000, lnk["bytes_per_ms"]) # noqa: E731
cap = snd["layer_cap"]
capped = [cap is not None and o["layer"] > cap for o in objs]
avail = []
for o in objs:
if p["mode"] == "vod":
w = snd["vod_window_us"]
avail.append(h + (0 if w is None else max(0, o["units"][0] * m["unit_us"] - w)))
else:
if o["capture_us"] is None:
raise ValueError(f"{o['id']}: no capture time")
avail.append(max(h, o["capture_us"]))
to = snd["delivery_timeout_us"]
deadline = [None if (to["base"] if o["layer"] == 0 else to["refinement"]) is None
else avail[i] + (to["base"] if o["layer"] == 0 else to["refinement"]) for i, o in enumerate(objs)]
datagram = snd["carriage"] == "datagram"
if datagram and any(not capped[i] and o["bytes"] > lnk["chunk_bytes"] for i, o in enumerate(objs)):
raise ValueError("datagram-too-large")
state = ["capped" if c else None for c in capped]
pending = [not c for c in capped]
sent = [0] * n
first = [None] * n
arrived = [None] * n
chunks = []
dropped = abandoned = 0
sub = [(o["base"], o["layer"]) for o in objs]
t = 0
while True:
for i in range(n):
if not pending[i] or deadline[i] is None or deadline[i] >= t:
continue
if datagram:
pending[i] = False
state[i] = "dropped"
dropped += objs[i]["bytes"]
else:
for j in range(i, n):
if pending[j] and sub[j] == sub[i]:
pending[j] = False
abandoned += sent[j]
dropped += objs[j]["bytes"] - sent[j]
state[j] = "abandoned" if sent[j] > 0 else "dropped"
def eligible(i):
if not pending[i] or avail[i] > t:
return False
return datagram or not any(pending[j] and sub[j] == sub[i] for j in range(i))
el = [i for i in range(n) if eligible(i)]
if not el:
later = [avail[i] for i in range(n) if pending[i] and avail[i] > t]
if later:
t = min(later)
continue
if any(pending):
raise ValueError("stuck")
break
if snd["schedule"] == "strict-priority":
# Bases in the order they became available (Appendix B, Order).
i = min(el, key=lambda i: (objs[i]["layer"], avail[i] if objs[i]["layer"] == 0 else 0, objs[i]["units"][0], i))
else:
i = min(el, key=lambda i: (avail[i], i))
c = min(lnk["chunk_bytes"], objs[i]["bytes"] - sent[i])
if sent[i] == 0:
first[i] = t
end = t + ser(c)
sent[i] += c
chunks.append((end + h, c))
if sent[i] == objs[i]["bytes"] and (datagram or deadline[i] is None or end <= deadline[i]):
pending[i] = False
state[i] = "delivered"
arrived[i] = end + h
t = end
return state, first, arrived, chunks, dropped, abandoned
def run_once(m, p):
objs = m["objects"]
n = len(objs)
sink = p["sink"]
state, first, arrived, chunks, dropped, abandoned = run_link(m, p)
stage = {} # unit -> (stage, when)
done = [False] * n
running = [] # (end, i)
pres = []
missed = 0
jobs = [i for i in range(n) if state[i] == "delivered"]
def ready_at(i):
o = objs[i]
r = arrived[i]
if o["layer"] >= 1:
frm, cnt = o["units"]
for u in range(frm, frm + cnt):
s = stage.get(u)
if s is None or s[0] != o["layer"] - 1:
return None
r = max(r, s[1])
return r
t = 0
while True:
running.sort()
while running and running[0][0] <= t:
end, i = running.pop(0)
o = objs[i]
frm, cnt = o["units"]
for u in range(frm, frm + cnt):
ps = sink["playout_start_us"]
late = ps is not None and end > ps + u * m["unit_us"]
if late:
missed += 1
if late and o["layer"] >= 1 and sink["late_refinement"] == "discard":
continue
pres.append((u, o["layer"] + 1, end))
stage[u] = (o["layer"], end)
while len(running) < sink["workers"]:
cands = []
for i in jobs:
if done[i]:
continue
r = ready_at(i)
if r is not None and r <= t:
cands.append((i, r))
if not cands:
break
if sink["decode_order"] == "base-first":
# Bases in the order they became ready (Appendix B, Workers).
i, _ = min(cands, key=lambda x: (objs[x[0]]["layer"], x[1] if objs[x[0]]["layer"] == 0 else 0, objs[x[0]]["units"][0], x[0]))
else:
i, _ = min(cands, key=lambda x: (x[1], x[0]))
done[i] = True
d = sink["decode_us"]["base"] if objs[i]["layer"] == 0 else sink["decode_us"]["refinement"]
running.append((t + d, i))
nexts = [e for e, _ in running]
nexts += [r for i in jobs if not done[i] for r in [ready_at(i)] if r is not None and r > t]
if not nexts:
break
t = min(nexts)
pres.sort()
t0 = min((x[2] for x in pres if x[0] == 0 and x[1] == 1), default=None)
top = max(o["layer"] for o in objs)
tffs = [min((x[2] for x in pres if x[0] == 0 and x[1] == k + 1), default=None) for k in range(top + 1)]
bfpf = None if t0 is None else sum(c for a, c in chunks if a <= t0)
return {
"objects": [{"id": o["id"], "sent_first_us": first[i], "arrived_us": arrived[i], "state": state[i]} for i, o in enumerate(objs)],
"presentations": [list(x) for x in pres],
"bfpf_sched": bfpf,
"tfpf_us": t0,
"tffs_us": tffs,
"offered": sum(o["bytes"] for i, o in enumerate(objs) if state[i] != "capped"),
"delivered": sum(o["bytes"] for i, o in enumerate(objs) if state[i] == "delivered"),
"dropped": dropped,
"abandoned": abandoned,
"never_sent_capped": sum(o["bytes"] for i, o in enumerate(objs) if state[i] == "capped"),
"missed": missed,
}
def simulate(m, p):
check_model(m)
full = run_once(m, p)
base = run_once({"unit_us": m["unit_us"], "objects": [o for o in m["objects"] if o["layer"] == 0]}, p)
def first_base(r):
out = {}
for u, l, t in r["presentations"]:
if l == 1 and u not in out:
out[u] = t
return out
fb, bb = first_base(full), first_base(base)
units = sorted(u for o in m["objects"] if o["layer"] == 0 for u in range(o["units"][0], o["units"][0] + o["units"][1]))
full["stall_us"] = [[u, fb[u] - bb[u] if u in fb and u in bb else None] for u in units]
return full
def bound(p):
snd, sink = p["sender"], p["sink"]
if snd["delivery_timeout_us"]["base"] is not None:
return None
chunk = ceil_div(p["link"]["chunk_bytes"] * 1000, p["link"]["bytes_per_ms"])
if snd["schedule"] == "strict-priority":
link = chunk
elif snd["delivery_timeout_us"]["refinement"] is not None:
link = snd["delivery_timeout_us"]["refinement"] + chunk
else:
return None
if sink["decode_order"] != "base-first":
return None
return link + sink["decode_us"]["refinement"]
def max_stall(r):
m = 0
for _, s in r["stall_us"]:
if s is None:
return None
m = max(m, s)
return m
def fixture_model(d, name):
rel = os.path.relpath(d, MV.WAI)
ev, _ = MV.evaluate(MV.Fixture(rel, name))
caps = [b["capture_us"] for b in ev["bases"]]
return {"unit_us": ev["unit_us"], "objects": [
{"id": o["file"].rsplit("/", 1)[-1], "base": o["base"], "layer": o["layer"], "units": o["units"],
"bytes": o["counted"]["total"], "capture_us": caps[o["base"]]} for o in ev["objects"]]}
def main(d):
d = os.path.abspath(d)
cases = json.load(open(os.path.join(d, "sim", "cases.json")))["cases"]
results = {r["name"]: r for r in json.load(open(os.path.join(d, "sim", "results.json")))["results"]}
fails = []
shown = {"link": False, "decode": False}
if set(results) != {c["name"] for c in cases}:
fails.append("results.json does not cover exactly the cases")
for c in cases:
name = c["name"]
if c["model"].startswith("fixture:"):
m = fixture_model(d, c["model"].split(":", 1)[1])
else:
m = json.load(open(os.path.join(d, "sim", c["model"])))
p = json.load(open(os.path.join(d, "sim", c["plan"])))
try:
r = simulate(m, p)
except ValueError as e:
fails.append(f"{name}: {e}")
continue
if r["offered"] != r["delivered"] + r["dropped"] + r["abandoned"]:
fails.append(f"{name}: bytes are not conserved")
b, w = bound(p), max_stall(r)
claim = c["claim"]
if claim == "within-bound":
if b is None or w is None or w > b:
fails.append(f"{name}: stall {w} not within the bound {b}")
elif claim == "unbounded":
if b is not None:
fails.append(f"{name}: the plan has a bound")
elif claim.startswith("exceeds:"):
other = json.load(open(os.path.join(d, "sim", "plans", claim.split(":", 1)[1] + ".json")))
ob = bound(other)
if b is not None or ob is None or (w is not None and w <= ob):
fails.append(f"{name}: stall {w} does not pass {claim}'s bound {ob}")
else:
if p["sender"]["schedule"] == "fifo" and p["sender"]["delivery_timeout_us"]["refinement"] is None:
shown["link"] = True
if p["sink"]["decode_order"] == "fifo":
shown["decode"] = True
else:
fails.append(f"{name}: claim {claim}")
g = results.get(name)
if g is None:
continue
out = []
MV.diff({"stall_bound_us": b, "max_stall_us": w, "result": r},
{"stall_bound_us": g["stall_bound_us"], "max_stall_us": g["max_stall_us"], "result": g["result"]}, name, out)
fails += out[:10]
for part, ok in shown.items():
if not ok:
fails.append(f"no case shows a stall past a bounded plan's bound for an unbounded {part} schedule")
if fails:
for x in fails:
print("FAIL", x)
sys.exit(1)
print(f"staged-measure sim: {len(cases)} cases verify independently")
if __name__ == "__main__":
main(sys.argv[1] if len(sys.argv) > 1 else HERE)

2
wai/telemetry-conformance/.gitattributes vendored Normal file
View file

@ -0,0 +1,2 @@
# Hashed files: a line-ending conversion on checkout would change every digest.
* -text

49
wai/telemetry-conformance/README.md Normal file
View file

@ -0,0 +1,49 @@
# telemetry-conformance — delivery telemetry
The corpus of [`extensions/delivery-telemetry.md`](../extensions/delivery-telemetry.md)
§9: observations, their CMCD version 2 reports and CMSD fields, and session logs with
their metrics. Every value here is a string or an integer, the same on every machine.
- `observations/` — one observation (§2) each, every event kind among them: requests
with the object's members, a response, the play states, the first presentation and a
later one (which CMCD does not report), stalls with their metrics, a switch, a layer
change, a dropped frame, dispatch outcomes, an abandon, an end with its claim pointer,
a low-latency live session, and strings RFC 9651 escapes (a quote and a backslash in
a capability in a custom key, a quote in a URL in an Event-mode report).
- `expected/cmcd.json` — for each observation: its mode (`request`, `event` or `none`),
its report (§3, §4), and for Request mode its header shards (§4.2).
- `invalid/` — observations every reader refuses (§2.4): a member unknown or missing,
the wrong version, a session id that is not hexadecimal, an Event-mode observation
with no time, an integer past 2⁵³ − 1, a time past an RFC 9651 Integer, a claim
before the end, an unknown event, a response key on a request and a request key on a
response, a switch to its own path or for an unknown reason, a pin refusal with no
reason, an unknown object type, a content id past 64 characters or holding an address
or a URL, an object URL with a query, a fragment, user information (behind `//`, behind
backslashes, behind no slash, or behind a mixed separator), a backslash, or a `@` in
its path (§8.2), a presentation at no layer,
an unknown cohort.
`expected/refused.json` lists them.
- `cmsd/` — CMSD-Static descriptions of a base and of a derived layer, and a
CMSD-Dynamic field from two nodes (§6); `expected/cmsd.json`, their fields.
- `logs/` — session logs (§7.1), one observation per line: a session with stalls, one
open at its end; an exit before start; a start failure. `expected/qoe.json`, their
metrics (§7.2).
- `cml/` — the differential (§9.3).
**Verifiers.**
- `wai_telemetry verify .` (from `../wai-rs`, feature `telemetry`): reads every input
from disk, requires each observation to write back as given, recomputes every
expected file and requires it byte for byte, reads every report back, and requires
every `invalid/` observation refused. `wai_telemetry gen .` writes the corpus.
- `cml/differential.mjs` (`npm ci && node differential.mjs ..` in `cml/`): builds each
report from the extension's tables in JavaScript, not from `wai-rs`'s output; encodes
it with an independent open-source CMCD and CMSD library at the versions
`package-lock.json` locks, in the mode the event fixes, with the header shards for
Request mode; requires the bytes of `expected/cmcd.json` and `expected/cmsd.json`;
reads the expected strings back with the library's dictionary, query and header
readers and requires the members; and validates every report, and the Event-mode
reports together, requiring no error.
The content is synthetic: session ids, content hashes and timestamps are fixed values
chosen for the corpus.

Some files were not shown because too many files have changed in this diff Show more