open-standards/wai/telemetry-conformance/README.md

telemetry-conformance — delivery telemetry

The corpus of extensions/delivery-telemetry.md §9: observations, their CMCD version 2 reports and CMSD fields, and session logs with their metrics. Every value here is a string or an integer, the same on every machine.

• observations/ — one observation (§2) each, every event kind among them: requests with the object's members, a response, the play states, the first presentation and a later one (which CMCD does not report), stalls with their metrics, a switch, a layer change, a dropped frame, dispatch outcomes, an abandon, an end with its claim pointer, a low-latency live session, and strings RFC 9651 escapes (a quote and a backslash in a capability in a custom key, a quote in a URL in an Event-mode report).
• expected/cmcd.json — for each observation: its mode (request, event or none), its report (§3, §4), and for Request mode its header shards (§4.2).
• invalid/ — observations every reader refuses (§2.4): a member unknown or missing, the wrong version, a session id that is not hexadecimal, an Event-mode observation with no time, an integer past 2⁵³ − 1, a time past an RFC 9651 Integer, a claim before the end, an unknown event, a response key on a request and a request key on a response, a switch to its own path or for an unknown reason, a pin refusal with no reason, an unknown object type, a content id past 64 characters or holding an address or a URL, an object URL with a query, a fragment, user information (behind //, behind backslashes, behind no slash, or behind a mixed separator), a backslash, or a @ in its path (§8.2), a presentation at no layer, an unknown cohort. expected/refused.json lists them.
• cmsd/ — CMSD-Static descriptions of a base and of a derived layer, and a CMSD-Dynamic field from two nodes (§6); expected/cmsd.json, their fields.
• cmsd-read/cases.json — CMSD fields to read (§6.6): fields as §6.1 to §6.3 write them, the same with their keys in another order, with v=1 and with a key or parameter of another sender, and fields refused with each code of §6.6 (not a Dictionary or List, a duration or throughput as a String, a layer not in decimal or with a leading zero, a short hash, an operation as a String, a server as a Token, a drop count not in decimal, version 2); expected/cmsd-read.json, each one's description or refusal.
• logs/ — session logs (§7.1), one observation per line: a session with stalls, one open at its end; an exit before start; a start failure. expected/qoe.json, their metrics (§7.2).
• cml/ — the differential (§9.3).

Verifiers.

• wai_telemetry verify . (from ../wai-rs, feature telemetry): reads every input from disk, requires each observation to write back as given, recomputes every expected file and requires it byte for byte, reads every report back, and requires every invalid/ observation refused. wai_telemetry gen . writes the corpus.
• cmsd_read_verify.py (python3 -I cmsd_read_verify.py .): the second implementation of §6.6, written from that text and RFC 9651 and sharing no code with wai-rs: it parses every field of cmsd-read/ with its own RFC 9651 reader and requires each outcome, reads every field of expected/cmsd.json back to its description, requires §6.6's table to be the codes the corpus gives, and requires every mutated copy of the corpus to fail.
• cml/differential.mjs (npm ci && node differential.mjs .. in cml/): builds each report from the extension's tables in JavaScript, not from wai-rs's output; encodes it with an independent open-source CMCD and CMSD library at the versions package-lock.json locks, in the mode the event fixes, with the header shards for Request mode; requires the bytes of expected/cmcd.json and expected/cmsd.json; reads the expected strings back with the library's dictionary, query and header readers and requires the members; and validates every report, and the Event-mode reports together, requiring no error.

Provenance. Generated here: session ids, content hashes and timestamps are fixed values chosen for the corpus. No recorded traffic.