Transaction-Science/open-standards
Watch
1
0
Fork
You've already forked open-standards
 0
Code Issues Pull requests Projects Releases 17 Packages Wiki Activity Actions 96
open-standards/wai/extensions/delivery-telemetry.md
Transaction Science 40796bb37b wai: the reconstruct-session claim class, its authorities and its figure profile (text) 
jwp-receipts §2.4.12 registers reconstruct-session, the next free number after
W1-5's stage (§2.4.10) and delivery-gap (§2.4.11): a sink's statement of one
session — decoders and host, paths and switches, late presentations, the output
tree, counted and transferred bytes, the classical rendition it did not need,
the stage claims it signed, and its decode energy where metered. The session
names its sets' stage claims rather than restating their objects and units;
step 6 checks it against its outputs, objects, entry envelopes and those stage
claims. §3.1 gains seventeen reasons, §5.11 the corpus, §7.5 the authority
and the energy figure as a second item. energy-measurement §6.1 widens the
claim-link profile; delivery-telemetry's claim member now names a registered
class; staged-measurement §8.6-§8.7 and Appendix C say how a tool checks the
rail and seals a session's figure.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-06 08:55:52 -04:00

248 lines
15 KiB
Markdown
Raw Permalink Blame History

This file contains ambiguous Unicode characters

This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.

# WAI Extension: Delivery Telemetry
> Status: Draft. An **unsigned, advisory** observation about WAI delivery that a sink,
> edge or origin emits; its mapping onto the keys of CMCD version 2 (CTA-5004-B) and
> CMSD (CTA-5006), written as RFC 9651 Structured Field Values; and the session
> metrics a log of observations gives. Reference impl: `wai-rs` —
> `telemetry::{sf, observation, cmcd, cmsd, qoe}`; the `wai_telemetry` tool; the corpus
> [`telemetry-conformance/`](../telemetry-conformance/), with a differential against an
> independent open-source CMCD and CMSD library. Keywords **MUST**, **MUST NOT**,
> **SHOULD**, **MAY** are RFC 2119/8174.
## 1. Status of an observation
§1.1 No signature covers an observation. A relying party **MUST NOT** treat one as
evidence of delivery, presentation, quality or energy. What a session did is attested,
where it is, by receipts and claims ([jwp-receipts](jwp-receipts.md)), never by
telemetry.
§1.2 An observation **MUST NOT** carry a joule figure, an acquisition class or a quality
figure. It **MAY** carry content hashes and link hashes as pointers.
§1.3 Telemetry binds no receipt to CMCD or CMSD, and no receipt to telemetry.
## 2. The observation
§2.1 An observation is a JSON object with exactly these members; a member that does not
apply is `null`. Integers are unsigned and at most 2⁵³ − 1, and a member a report
carries as an Integer (§3.3: `sn`, `ts_ms`, the object's `d_ms`, `dl_ms`, `ttfb_ms`,
`ttlb_ms` and `rc`, and the metrics) at most 10¹⁵ − 1, the largest RFC 9651 Integer, so
every observation can be reported. A string a report carries is printable ASCII.
| member | type | meaning |
|---|---|---|
| `v` | integer | `1` |
| `sid` | string | 32 lowercase hexadecimal digits, fresh per session, derived from no identifier (§8) |
| `sn` | integer | the observation's sequence number in the session |
| `ts_ms` | integer or `null` | Unix time of the event in milliseconds; required for every event reported in Event mode (§3.1) |
| `event` | string | `request`, `response`, `start`, `present`, `switch`, `layer-change`, `stall-start`, `stall-end`, `frame-drop`, `capable`, `abandon` or `end` |
| `cid` | string or `null` | the entry object's content hash (64 lowercase hexadecimal digits), or a deployer's content id of 1–64 letters, digits, `.`, `_` and `-` |
| `st` | string | stream type: `v` (on demand), `l` (live) or `ll` (low-latency live) |
| `sf` | string | `o` for WAI objects over the HTTP binding; `h` or `d` for a WAI rendition referenced from HLS or DASH |
| `object` | object or `null` | for `request` and `response`: the object (§2.2) |
| `present` | object or `null` | for `present`: `{ unit, layers, first, late_ms, decode_us }`; `first` is `true` only for the session's first presentation; `layers` is at least 1 |
| `switch` | object or `null` | for `switch`: `{ unit, from, to, reason }`; `from` and `to` are `primary` or `fallback` and differ; `reason` is a §5 term |
| `layer_change` | object or `null` | for `layer-change`: `{ from, to }`, a change of `layer_cap`; `null` is no cap |
| `capable` | object or `null` | for `capable`: `{ outcome, reason, classical_floor }` (§2.3) |
| `cohort` | string or `null` | `control` or `wai`: the arm the content's service assigned the session to |
| `metrics` | object or `null` | `{ msd_ms, bsa, bsda_ms, dfa }`: media start delay, buffer starvations and their total duration since the session began, and dropped frames since the session began |
| `claim` | 64 hexadecimal digits or `null` | on `end` only: the link hash of the session's `reconstruct-session` claim ([jwp-receipts](jwp-receipts.md) §2.4.12), where the sink sealed one |
§2.2 **The object** has exactly the members `ot`, `url`, `content_hash`, `capability`,
`rendition`, `layer`, `layers_total`, `layer_cap`, `counted`, `transferred`, `d_ms`,
`dl_ms`, `ttfb_ms`, `ttlb_ms`, `rc`, `path` and `receipt`, each `null` where it does
not apply:
- `ot` is a CTA-5004-B object type token (`m`, `a`, `v`, `av`, `i`, `c`, `tt`, `k`,
`o`); `content_hash` the object's content hash and `receipt` a receipt's link hash,
each 64 lowercase hexadecimal digits; `capability` at most 64 characters;
`rendition` the `WAI2` rendition index; `layer` and `layers_total` the object's
stage and its stage set's stage count; `layer_cap` the cap the request asks for;
`counted` its counted bytes and `transferred` its transferred bytes
([staged-measurement](staged-measurement.md) §3); `path` `primary` or `fallback`;
`url` the object's address, at most 2048 characters, holding no `?`, `#`, `@` or `\`
(§8.2).
- The request keys `ot`, `d_ms`, `dl_ms` and `layer_cap` are `null` on a `response`;
the response keys `rc`, `ttfb_ms`, `ttlb_ms` and `url` are `null` on a `request`.
§2.3 **A `capable` observation** reports one WAI-arm dispatch outcome: `not-reached`
(no dispatch decision for a WAI object was reached), `missing-capability`,
`pin-refused`, `module-refused`, `below-rate` or `capable`. `reason` is SPEC §3.1's pin
refusal for `pin-refused` (one of the terms §5 lists before `missing-capability`), and
`null` otherwise. `classical_floor` is the platform's answer for the classical
rendition's codec string, `supported` or `unsupported`, where the sink asked. The
outcome comes from WAI's own dispatch: advertised capabilities, pin verification and a
timed decode. A sink **MUST NOT** derive it from a platform media-capability query,
which answers only for the platform's own decoders; it reports that answer separately,
as `classical_floor`. How sessions are assigned to arms, and the shares computed over
outcomes, are not specified in this revision.
§2.4 An observation that breaks §2.1–§2.3 is refused as a whole: a member missing or
not listed, a value of another type or outside its range, an object on an event that
has none, or none on one that has one.
## 3. CMCD version 2 mapping
§3.1 The modes:
- a `request` observation is reported in Request mode;
- every other observation is reported in Event mode, with `ts`: a `response` as an
event report with `e=rr`, whose response keys (`rc`, `ttfb`, `ttlb`, `url`) are never
carried in Request mode or in a header shard;
- a `present` observation with `first: false` is not reported in CMCD.
Every report carries `v=2`, `sid`, `sn`, `st` and `sf`, and `cid` where it is not `null`.
| observation | CMCD | mode |
|---|---|---|
| `sid`, `sn`, `cid`, `st`, `sf` | same keys | every report |
| `object.ot`, `object.d_ms`, `object.dl_ms` | `ot`, `d`, `dl` | request |
| `object.rc`, `object.ttfb_ms`, `object.ttlb_ms`, `object.url` | `rc`, `ttfb`, `ttlb`, `url`, with `e=rr` | event |
| `ts_ms` | `ts` | event |
| `start` / `end` / `abandon` | `e=ps`, `sta=s` / `e` / `q` | event |
| `present` with `first: true` | `e=ps`, `sta=p` | event |
| `stall-start` / `stall-end` | `e=ps`, `sta=r` / `p` | event |
| `metrics.msd_ms`, `metrics.dfa` | `msd`, `dfa` | any report |
| `metrics.bsa`, `metrics.bsda_ms` | `bsa`, `bsda`, each an inner list of one Integer | any report |
| `switch`, `capable`, `layer-change`, `frame-drop` | `e=ce`, `cen` = `"wai-switch"`, `"wai-capable"`, `"wai-layer-change"`, `"wai-frame-drop"` | event |
§3.2 An emitter **MUST NOT** put counted bytes, a layer or a capability into `br`, `tb`,
`pb` or any other bitrate key.
§3.3 A report is an RFC 9651 Dictionary: Integers for counts and times, Tokens for
`st`, `sf`, `ot`, `e` and `sta`, Strings for `sid`, `cid`, `url` and `cen`. Its keys are
written in bytewise order and without optional whitespace, and a key is written only
when its value is not `null`. The corpus fixes the output byte for byte.
## 4. Custom keys
§4.1 Custom keys have the form `science.transaction.wai-<x>`: a lowercase first letter,
then lowercase letters, digits, `.` and `-`, with an inner hyphen (CTA-5004-B's
custom-key rule). Every custom key's value is a String, the type CTA-5004-B admits for
custom keys besides a Token; an integer member is written as its decimal digits.
| key | from | report | header shard (Request mode) |
|---|---|---|---|
| `science.transaction.wai-b` | `object.counted` | request, `e=rr` | `CMCD-Object` |
| `science.transaction.wai-tx` | `object.transferred` | request, `e=rr` | `CMCD-Object` |
| `science.transaction.wai-ch` | `object.content_hash` | request, `e=rr` | `CMCD-Object` |
| `science.transaction.wai-cap` | `object.capability` | request, `e=rr` | `CMCD-Object` |
| `science.transaction.wai-ri` | `object.rendition` | request, `e=rr` | `CMCD-Object` |
| `science.transaction.wai-l` | `object.layer` | request, `e=rr` | `CMCD-Object` |
| `science.transaction.wai-lt` | `object.layers_total` | request, `e=rr` | `CMCD-Object` |
| `science.transaction.wai-p` | `object.path`, or `switch.to`: `"p"` or `"f"` | request, `e=rr`, `wai-switch` | `CMCD-Object` |
| `science.transaction.wai-rcpt` | `object.receipt` | request, `e=rr` | `CMCD-Object` |
| `science.transaction.wai-lc` | `object.layer_cap`, or `layer_change.to` | request, `wai-layer-change` | `CMCD-Request` |
| `science.transaction.wai-lcf` | `layer_change.from` | `wai-layer-change` | — |
| `science.transaction.wai-coh` | `cohort` | every report | `CMCD-Session` |
| `science.transaction.wai-u` | `present.unit`, or `switch.unit` | `sta=p` (first presentation), `wai-switch` | — |
| `science.transaction.wai-pl`, `-late`, `-dec` | `present.layers`, `present.late_ms`, `present.decode_us` | `sta=p` (first presentation) | — |
| `science.transaction.wai-sw` | `switch.reason` | `wai-switch` | — |
| `science.transaction.wai-co`, `-cr`, `-cf` | `capable.outcome`, `capable.reason`, `capable.classical_floor` | `wai-capable` | — |
| `science.transaction.wai-clm` | `claim` | `sta=e` | — |
§4.2 In Request mode the report also goes in header shards: each key to the shard
CTA-5004-B assigns it (`v`, `sid`, `cid`, `st`, `sf` and `msd` to `CMCD-Session`; `sn`,
`dl` and `dfa` to `CMCD-Request`; `ot` and `d` to `CMCD-Object`; `bsa` and `bsda` to
`CMCD-Status`), and each custom key to the shard of §4.1. Each shard is a Dictionary
written as §3.3 writes a report; a shard with no key is not sent.
## 5. Switch reasons
§5.1 A switch reason is one of:
- SPEC §3.1's pin refusals, in the order of its checks: `pin-malformed`,
`form-not-registered`, `form-unreadable`, `form-required`, `prior-revoked`,
`prior-not-authorised` (check 5, at a sink that holds a key log), `prior-not-held`,
`digest-superseded`, `set-malformed`, `part-mismatch`, `set-incomplete`;
- `missing-capability`;
- `module-unverified`, `module-revoked`, `trap`, `fuel-exhausted`, `memory-exhausted`,
`deadline-threshold`, `unit-hash-mismatch`, `budget-exhausted`;
- `recovered`.
A later extension adds its reasons here.
## 6. CMSD mapping
§6.1 An origin or edge **MAY** describe a WAI object in `CMSD-Static` with `at`, `d`,
`ht`, `nor`, `nrr`, `ot`, `sf`, `st` and `su`, and the custom keys
`science.transaction.wai-ch` (its content hash), `-l` and `-lt` (its layer and its
stage set's stage count) and `-rcpt` (a delivery receipt's link hash), each a String.
`v` is not written for version 1.
§6.2 A deriving node adds `science.transaction.wai-op`, its operation token, as a Token.
§6.3 `CMSD-Dynamic` **MAY** state, for each node, `etp`, `rtt`, `mb`, `rd` and `du`, and
`science.transaction.wai-drop` (a String): the number of objects the node discarded on
delivery timeout since its previous response to the same session. Each node adds its
own entry, in order; within an entry the parameters are in bytewise order.
§6.4 Keys and parameters are written as §3.3 writes a report: in bytewise order, a key
only when its value is not `null`, and a Boolean (`su`, `du`) only when true.
§6.5 CMSD carries no joule figure. An edge's energy travels in a claim.
## 7. Session metrics
§7.1 A **log** is one session's observations in `sn` order: one `sid`, `sn` strictly
increasing, timestamps never decreasing, and nothing after `end` or `abandon`.
§7.2 A log gives, in integer milliseconds and bytes:
- `started`: the log has a `start`;
- `vst_ms`, the video start time: the first presentation's `ts_ms` less the `start`'s,
`null` without both;
- `exit_before_start`: an `abandon` before any first presentation;
- `start_failed`: a session that started, never presented, and ended with `end`;
- `stalls` and `stall_ms`: the stalls begun after the first presentation, each from its
`stall-start` to its `stall-end`, or to `end` or `abandon` where it is still open,
and their total duration;
- `engaged_ms`: from the first presentation to `end` or `abandon`;
- `units`: the distinct units presented;
- `counted`: the counted bytes of every `response`;
- `claim`: the `end` observation's `claim`.
§7.3 These are statements of the emitter, as the observations are (§1.1).
## 8. Privacy
§8.1 `sid` is fresh per session. An emitter **MUST NOT** add a persistent identifier, or
send telemetry to a party other than the content's own service, without the person's
agreement.
§8.2 Neither a URL's query nor a signed URL's token enters an observation. A reader
refuses an `object.url` that holds a `?`, a `#`, a `@` or a `\`: a query, a fragment,
user information, or a separator some URL parsers read as `/` and so a way to user
information past a check for it. It refuses rather than strips them, and a path that
holds a `@` is refused with the rest. An emitter whose URLs carry a token in their
path **MUST** report `url` as `null`. `cid`'s syntax (§2.1) admits a content hash or
a short identifier, and no syntax keeps an address, a token or personal data out of an
identifier: an emitter **MUST NOT** put an address, a query, a token or personal data
in `cid`, and reports the content hash where it has no identifier free of them.
§8.3 A Request-mode report travels with its request, to the host that serves the object
and to no other party. Event-mode reports go to the content's own service (§8.1).
§8.4 A published aggregate covers at least 50 sessions per cell. A cell with fewer is
suppressed, not rounded.
## 9. Conformance
§9.1 [`telemetry-conformance/`](../telemetry-conformance/) holds:
- observations of every event kind, with their expected CMCD reports and header shards
(`expected/cmcd.json`);
- observations every reader refuses (`invalid/`);
- CMSD-Static and CMSD-Dynamic descriptions with their expected fields
(`expected/cmsd.json`);
- session logs with their expected metrics (`expected/qoe.json`).
§9.2 `wai_telemetry verify` requires every expected value byte for byte, and every
`invalid/` observation refused.
§9.3 `cml/differential.mjs` builds each report from this extension's tables, encodes it
with an independent open-source CMCD and CMSD library at the versions
`cml/package-lock.json` locks, and requires the expected bytes; reads the
expected strings back as a dictionary, as a query string and as header shards, and
requires the report's members; and validates every report, and the Event-mode reports
together, with no error.
Reference in a new issue View git blame Copy permalink