open-standards/wai/telemetry-conformance
Repository files (latest commit first)
Filename Latest commit message Latest commit date
Transaction Science 3fac029a71 wai: every corpus states the provenance of its content
Each *-conformance/README.md now has a line that begins **Provenance.**:
the generator that writes the corpus and its input, and for an input not
generated here its source and licence. OER/2's corpus is settled and left
as it is. byte-exact-conformance and quantum-ops-conformance get a README.
wai-rs/tests/vectors/README.md gives a row per vector directory. Where the
staged-delivery, staged-measurement and telemetry READMEs called their
content "synthetic", that sentence is now their provenance line.

Two vectors are recorded as not generated, from the provenance audit:
int_video's video_real (a pan over a third-party picture) and
neural-conformance's photo (a third-party picture, with a decoder trained on
unrecorded images). Their lines say so and that they are to be replaced;
the photo bullet no longer names the picture's source product.

tests/corpus_provenance.rs requires the line in every corpus and a row for
every vector directory; CI runs it with the carriage tests. CONTRIBUTING
states the rule: corpus content is generated here, or recorded with a
licence that permits redistribution, and a word like "synthetic" is not a
provenance line.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-07 10:09:21 -04:00
..
cml wai telemetry: name the differential's library by what it is; fix stale section refs 2026-10-05 09:48:41 -04:00
cmsd wai: wai_telemetry and telemetry-conformance, with a differential against an independent library 2026-10-05 08:07:16 -04:00
expected wai telemetry: no ?, #, @ or backslash in an object URL; cid privacy is the emitter's 2026-10-05 10:59:09 -04:00
invalid wai telemetry: no ?, #, @ or backslash in an object URL; cid privacy is the emitter's 2026-10-05 10:59:09 -04:00
logs wai: wai_telemetry and telemetry-conformance, with a differential against an independent library 2026-10-05 08:07:16 -04:00
observations wai telemetry: no ?, #, @ or backslash in an object URL; cid privacy is the emitter's 2026-10-05 10:59:09 -04:00
.gitattributes wai: wai_telemetry and telemetry-conformance, with a differential against an independent library 2026-10-05 08:07:16 -04:00
README.md wai: every corpus states the provenance of its content 2026-10-07 10:09:21 -04:00

telemetry-conformance — delivery telemetry

The corpus of extensions/delivery-telemetry.md §9: observations, their CMCD version 2 reports and CMSD fields, and session logs with their metrics. Every value here is a string or an integer, the same on every machine.

  • observations/ — one observation (§2) each, every event kind among them: requests with the object's members, a response, the play states, the first presentation and a later one (which CMCD does not report), stalls with their metrics, a switch, a layer change, a dropped frame, dispatch outcomes, an abandon, an end with its claim pointer, a low-latency live session, and strings RFC 9651 escapes (a quote and a backslash in a capability in a custom key, a quote in a URL in an Event-mode report).
  • expected/cmcd.json — for each observation: its mode (request, event or none), its report (§3, §4), and for Request mode its header shards (§4.2).
  • invalid/ — observations every reader refuses (§2.4): a member unknown or missing, the wrong version, a session id that is not hexadecimal, an Event-mode observation with no time, an integer past 2⁵³ − 1, a time past an RFC 9651 Integer, a claim before the end, an unknown event, a response key on a request and a request key on a response, a switch to its own path or for an unknown reason, a pin refusal with no reason, an unknown object type, a content id past 64 characters or holding an address or a URL, an object URL with a query, a fragment, user information (behind //, behind backslashes, behind no slash, or behind a mixed separator), a backslash, or a @ in its path (§8.2), a presentation at no layer, an unknown cohort. expected/refused.json lists them.
  • cmsd/ — CMSD-Static descriptions of a base and of a derived layer, and a CMSD-Dynamic field from two nodes (§6); expected/cmsd.json, their fields.
  • logs/ — session logs (§7.1), one observation per line: a session with stalls, one open at its end; an exit before start; a start failure. expected/qoe.json, their metrics (§7.2).
  • cml/ — the differential (§9.3).

Verifiers.

  • wai_telemetry verify . (from ../wai-rs, feature telemetry): reads every input from disk, requires each observation to write back as given, recomputes every expected file and requires it byte for byte, reads every report back, and requires every invalid/ observation refused. wai_telemetry gen . writes the corpus.
  • cml/differential.mjs (npm ci && node differential.mjs .. in cml/): builds each report from the extension's tables in JavaScript, not from wai-rs's output; encodes it with an independent open-source CMCD and CMSD library at the versions package-lock.json locks, in the mode the event fixes, with the header shards for Request mode; requires the bytes of expected/cmcd.json and expected/cmsd.json; reads the expected strings back with the library's dictionary, query and header readers and requires the members; and validates every report, and the Event-mode reports together, requiring no error.

Provenance. Generated here: session ids, content hashes and timestamps are fixed values chosen for the corpus. No recorded traffic.