open-standards/wai/telemetry-conformance

telemetry-conformance — delivery telemetry

The corpus of extensions/delivery-telemetry.md §9: observations, their CMCD version 2 reports and CMSD fields, and session logs with their metrics. Every value here is a string or an integer, the same on every machine.

• observations/ — one observation (§2) each, every event kind among them: requests with the object's members, a response, the play states, the first presentation and a later one (which CMCD does not report), stalls with their metrics, a switch, a layer change, a dropped frame, dispatch outcomes, an abandon, an end with its claim pointer, a low-latency live session, and strings RFC 9651 escapes (a quote and a backslash in a capability in a custom key, a quote in a URL in an Event-mode report).
• expected/cmcd.json — for each observation: its mode (request, event or none), its report (§3, §4), and for Request mode its header shards (§4.2).
• invalid/ — observations every reader refuses (§2.4): a member unknown or missing, the wrong version, a session id that is not hexadecimal, an Event-mode observation with no time, an integer past 2⁵³ − 1, a time past an RFC 9651 Integer, a claim before the end, an unknown event, a response key on a request and a request key on a response, a switch to its own path or for an unknown reason, a pin refusal with no reason, an unknown object type, a content id past 64 characters or holding an address or a URL, an object URL with a query, a fragment, user information (behind //, behind backslashes, behind no slash, or behind a mixed separator), a backslash, or a @ in its path (§8.2), a presentation at no layer, an unknown cohort. expected/refused.json lists them.
• cmsd/ — CMSD-Static descriptions of a base and of a derived layer, and a CMSD-Dynamic field from two nodes (§6); expected/cmsd.json, their fields.
• logs/ — session logs (§7.1), one observation per line: a session with stalls, one open at its end; an exit before start; a start failure. expected/qoe.json, their metrics (§7.2).
• cml/ — the differential (§9.3).

Verifiers.

• wai_telemetry verify . (from ../wai-rs, feature telemetry): reads every input from disk, requires each observation to write back as given, recomputes every expected file and requires it byte for byte, reads every report back, and requires every invalid/ observation refused. wai_telemetry gen . writes the corpus.
• cml/differential.mjs (npm ci && node differential.mjs .. in cml/): builds each report from the extension's tables in JavaScript, not from wai-rs's output; encodes it with an independent open-source CMCD and CMSD library at the versions package-lock.json locks, in the mode the event fixes, with the header shards for Request mode; requires the bytes of expected/cmcd.json and expected/cmsd.json; reads the expected strings back with the library's dictionary, query and header readers and requires the members; and validates every report, and the Event-mode reports together, requiring no error.

The content is synthetic: session ids, content hashes and timestamps are fixed values chosen for the corpus.