- The falsifications found the check that refuses an acknowledgement the record does not name guarded by no vector in either implementation: the corpus gains ack_not_named (audit_bundle_invalid), and the Python verifier can drop that check too, which the corpus now catches. - The collector's default port moves to 9694: 9690 is the stitched presentation tool's. - stream-audit §6.3 states that this revision claims every marked ad with WPC1, and that a stitched ad is claimed with the stitched presentation record where a deployment holds it, its bundle then carrying the manifests its claims name. Co-Authored-By: Claude Opus 5.5 <[email protected]>
609 lines
37 KiB
Markdown
609 lines
37 KiB
Markdown
# WAI Extension: Stream Audit
|
||
|
||
> Status: Draft. Registers the record `wai.delivery.audit`: evidence a content owner gathers from
|
||
> the players of a stream it already distributes over HTTP, with no change to the service that
|
||
> packages and delivers it. For each playback session the player makes a fresh key, the
|
||
> **session key**, and signs with it: [settlement](settlement.md) acknowledgements, `WDA1`, of
|
||
> every object it fetched, each named by plain SHA-256 under settlement §14, one chain per
|
||
> origin; a presentation claim, `WPC1` of [ad-presentation](ad-presentation.md), for each ad the
|
||
> stream marks; and a **session record**, `WAS1`, which binds them: the channel, the owner's
|
||
> collector, the spans of time it presented and whether each was in view and audible, its
|
||
> start time and stalls (recomputed from its [delivery-telemetry](delivery-telemetry.md) log),
|
||
> its sink energy per role with each figure's acquisition class, how the session ended, and the
|
||
> digests of every record it names. The player sends the records to the owner's **collector**,
|
||
> which verifies them, stores them, and states a **channel report** per period: time presented,
|
||
> in view and audible; ads presented and the quartile each reached; start time and stalls once
|
||
> enough sessions hold them; acknowledged bytes per origin and object class; and joules per role
|
||
> and acquisition class, never summed across classes. A party holding the stored records
|
||
> recomputes the report byte for byte. No new transport, and no price, tariff or amount. The
|
||
> reference implementation is the `stream_audit` module of `wai-rs` (feature `stream_audit`);
|
||
> the collector is `wai/wai-audit`, whose `web/` directory holds the player integration; the
|
||
> corpus is [`live-conformance/audit/`](../live-conformance/audit/), and its second
|
||
> implementation is `live-conformance/stream_audit_verify.py`.
|
||
> Keywords **MUST**, **MUST NOT**, **SHOULD**, **MAY** are RFC 2119/8174.
|
||
>
|
||
> Texts this revision relies on: the WAI texts it cites; RFC 9110 §8.4 (content codings) and
|
||
> §14 (range requests); RFC 9530 §2 and §3 (`Content-Digest`, `Repr-Digest`); RFC 6454 §6.2
|
||
> (the ASCII serialization of an origin); RFC 8032; FIPS 180-4; the WHATWG Fetch Standard's
|
||
> CORS-safelisted response-header names and `Access-Control-Expose-Headers`; and
|
||
> `draft-pantos-hls-rfc8216bis-22` §4.4.5.1 (`EXT-X-DATERANGE`) as
|
||
> [live-ads](live-ads.md) reads it, with its `EXTINF`, `EXT-X-PROGRAM-DATE-TIME` and
|
||
> `EXT-X-BYTERANGE` tags.
|
||
|
||
## 1. Scope
|
||
|
||
In scope: the channel and the session key (§3); what a player acknowledges and how (§4); the
|
||
session record (§5); the ads it presented (§6); its telemetry (§7); its energy and how the
|
||
session ended (§8); the bundle a player sends (§9); verification (§10); the channel report and
|
||
its recomputation (§11); and what a collector does (§12).
|
||
|
||
Out of scope: a transport; how a player reaches the collector, beyond §12; the service that
|
||
delivers the stream, which this extension asks for nothing; the authenticity of what was
|
||
delivered, since nothing the delivering service signs is read here (§15); the markers of a DASH
|
||
`EventStream` and of an in-band event, which this revision does not read; and any price, tariff,
|
||
amount or invoice: a report states what was presented and delivered, never what it is worth.
|
||
|
||
Relationship to the core spec: `wai.delivery.audit` is a **record** ([SPEC](../SPEC.md) §5): a
|
||
signed record a WAI tool emits and verifies. It names no payload format and is not
|
||
dispatchable.
|
||
|
||
## 2. Terms
|
||
|
||
- **Owner**: the party a stream's content belongs to, which runs the collector. Its **key** is
|
||
the collector's Ed25519 key (RFC 8032).
|
||
- **Channel**: a stream the owner names, live or on demand, under one name (§3.1).
|
||
- **Player**: the software that fetches and presents the stream and signs this extension's
|
||
records, with an integration of §13 or one of its own.
|
||
- **Session**: one playback of one channel by one player, from the player's first request to
|
||
its end (§8.2).
|
||
- **Object**: one HTTP response body the player fetched for the presentation, as settlement §14
|
||
defines it: a playlist, a manifest, an initialization segment, a media segment, a timed-text
|
||
segment.
|
||
- **Origin**: the origin (RFC 6454) of the URL an object came from, after redirects.
|
||
- **Time**: microseconds since 1970-01-01T00:00:00Z, as settlement §2's. A span from `start`
|
||
(included) to `end` (excluded).
|
||
- *u64*, *u32*, *u16*, *u8*: little-endian unsigned integers. A **digest** is SHA-256 (FIPS
|
||
180-4), 32 bytes. **Zero** is 32 zero bytes. Ed25519 is verified strictly
|
||
([SPEC](../SPEC.md) §7 *Signatures*): a key that fails step 1 or 2 is `bad_key`, a signature
|
||
that fails a later step `bad_signature`.
|
||
|
||
## 3. The channel and the session key
|
||
|
||
### 3.1 The channel
|
||
|
||
A channel's **name** is 1 to 255 bytes of printable ASCII (0x21 to 0x7E), which the owner
|
||
chooses and configures its players with. Its **id** is SHA-256 over `wai:audit-channel` (ASCII),
|
||
0x01 and the name. A name is the owner's label, never a URL: a URL can carry a token, and a
|
||
channel outlives the URLs its stream is served from.
|
||
|
||
### 3.2 The session key and the session
|
||
|
||
A player **MUST** make a fresh Ed25519 key for each session, from its platform's random
|
||
generator, and **MUST NOT** use it for another session or derive it from an identifier of the
|
||
viewer, the device or an account: ad-presentation §3's rule, for a channel in place of a room.
|
||
|
||
The **session** is the first 16 bytes of SHA-256 over `wai:ad-presentation-session` (ASCII),
|
||
0x01, the channel id and the session key: ad-presentation §3's derivation with the channel id in
|
||
the room id's place, so that the claims of §6 name it exactly as a room's claims name theirs.
|
||
|
||
## 4. Acknowledgements
|
||
|
||
### 4.1 What a player acknowledges
|
||
|
||
A player acknowledges every object of the presentation it receives, under settlement §14: each
|
||
named by the SHA-256 of its bytes as received, from the upstream named by its origin. It does not
|
||
acknowledge a key or a licence response: their bytes are secrets, and their digest would let a
|
||
party holding a candidate key confirm it.
|
||
|
||
Each acknowledgement (settlement §3) has `mode` 1, `role` 2 (a sink), `upstream` the origin's
|
||
upstream (settlement §14.2), `downstream` the session key, which signs it, and zero `grant` and
|
||
`proof` in every entry. Its entries are by object class (settlement §3.3):
|
||
|
||
- class 1 (`live`): the initialization and media segments of a live presentation (a playlist
|
||
with no `EXT-X-ENDLIST`);
|
||
- class 2 (`fetch`): those of an on-demand presentation, a retrieval no grant was presented for;
|
||
- class 4 (`ancillary`): every other object: playlists, manifests (each refresh is an object),
|
||
timed text.
|
||
|
||
Every item is a whole object: `start` 0 and `end` its length. A byte-range segment is an object
|
||
of its own (settlement §14.1).
|
||
|
||
### 4.2 Windows and chains
|
||
|
||
A player cuts its session into **windows** of *W* microseconds from the session's `start`: the
|
||
*k*-th from `start` + *kW* to `start` + (*k* + 1)*W*, the last ending at the session's `end`.
|
||
*W* is the player's choice, 10 s by default and at least 1 s. For each origin and each window in
|
||
which it received an object from that origin, it signs one acknowledgement of the objects
|
||
received in that window, in the order received; an object's time is when its body was complete.
|
||
An origin's acknowledgements form one chain (settlement §3.7), each naming in `prev` the digest
|
||
of the one before it, the first naming zero.
|
||
|
||
A window's item list names an object once. An object the window already lists for that origin,
|
||
received again with the same bytes (a retried request, a playlist refreshed unchanged) is a
|
||
**repeat**: the session record counts repeats, their objects and bytes (§5.1), and lists none.
|
||
|
||
### 4.3 What an acknowledgement states
|
||
|
||
That the holder of the session key received those bytes from that origin in that window. Not
|
||
that the bytes are what the owner published, nor that they were presented: presentation is the
|
||
session record's spans (§5.2) and the claims (§6).
|
||
|
||
## 5. The session record
|
||
|
||
### 5.1 Layout
|
||
|
||
When a session ends, the player signs one record:
|
||
|
||
| field | size | meaning |
|
||
|---|---|---|
|
||
| `magic` | 4 | `WAS1` (ASCII) |
|
||
| `version` | 1 | 1 in this revision |
|
||
| `channel` | 32 | the channel id (§3.1) |
|
||
| `owner` | 32 | the owner's key: the collector the session reports to |
|
||
| `player` | 32 | the session key, which signs the record |
|
||
| `session` | 16 | the session (§3.2) |
|
||
| `presentation` | 1 | 1 live, 2 on demand |
|
||
| `start`, `end` | 8 each | u64: the session's span |
|
||
| `ended` | 1 | how the session ended (§8.2) |
|
||
| `origins` | 1 + … | a u8 count *n*, at least 1, then each origin: a u16 length, 1 to 2048, and its ASCII serialization (RFC 6454 §6.2) |
|
||
| `acks` | 2 + 32 × *n* | a u16 count, at least 1, then the digests of the session's acknowledgements |
|
||
| `claims` | 2 + 32 × *n* | a u16 count, then the digests of its presentation claims (§6) |
|
||
| `spans` | 4 + 11 × *n* | a u32 count, then each span (§5.2) |
|
||
| `startup_ms` | 4 | u32: the start time, delivery-telemetry's `vst_ms` (§7.1); 4 294 967 295 when nothing was presented |
|
||
| `stalls` | 4 | u32: the stalls begun after the first presentation |
|
||
| `stall_ms` | 8 | u64: their total duration |
|
||
| `repeat_objects` | 4 | u32: the repeats (§4.2) |
|
||
| `repeat_bytes` | 8 | u64: their bytes |
|
||
| `log` | 32 | the digest of the session's observation log (§7.1) |
|
||
| `notes` | 32 | the digest of its notes document (§7.3), or zero for none |
|
||
| `energy` | 1 + … | a u8 count, then each energy entry (§5.3) |
|
||
| `attachments` | 2 + 32 × *n* | a u16 count, then the digests of the records it attaches (§8.3) |
|
||
| `sig` | 64 | §5.4 |
|
||
|
||
Each list of digests is in strictly ascending order, and so are the origins, bytewise.
|
||
|
||
### 5.2 Spans
|
||
|
||
A span is a stretch of the session's time during which the player presented the stream at normal
|
||
speed; 11 bytes:
|
||
|
||
| field | size | meaning |
|
||
|---|---|---|
|
||
| `start_ms` | 4 | u32: where it starts, in milliseconds after the session's `start` |
|
||
| `length_ms` | 4 | u32: its length, at least 1 |
|
||
| `visible_bp` | 2 | u16: the least fraction of the player's pixels in view it observed over the span, in basis points (0 to 10 000) |
|
||
| `flags` | 1 | bit 0 **audible**: audible throughout; bit 1 **ad**: within a marked ad (§6) |
|
||
|
||
A player observes at least every 200 ms and begins a new span whenever playback stops or
|
||
resumes, its quantised visible fraction or audible state changes, or a marked ad begins or ends,
|
||
by ad-presentation §10's rules: a fraction of 0 while the page or application is hidden, out of
|
||
focus or locked; audible only when unmuted at a volume above 0. The spans are in order and do not
|
||
overlap.
|
||
|
||
### 5.3 Energy entries
|
||
|
||
An entry states the energy of one **role** of the session:
|
||
|
||
| field | size | meaning |
|
||
|---|---|---|
|
||
| `role` | 1 | 1 **origin**, 2 **delivery** (the network between origin and player), 3 **sink decode**, 4 **display** |
|
||
| `joules` | 8 | u64: microjoules over the session; 0 is the unmetered marker |
|
||
| `class` | 1 + … | its acquisition class: energy-measurement §2.7's tag byte, then that tag's evidence |
|
||
|
||
A role with a figure states it with its class; a role the player knows took part and did not
|
||
measure states the unmetered marker (tag 0 and 0 J, live-metering §2); a role it knows nothing
|
||
of has no entry (§8.1).
|
||
|
||
### 5.4 Digest and signature
|
||
|
||
The **digest** of a record is SHA-256 over its bytes before `sig`. Its **signature** is Ed25519
|
||
by `player` over the 18 bytes `wai:audit-session` (ASCII) and 0x01, followed by the bytes the
|
||
digest covers.
|
||
|
||
### 5.5 Reading and rules
|
||
|
||
A reader reads exactly the bytes it is given as one record, in the order of §5.1, and refuses at
|
||
the first of these that holds: fewer than 4 bytes, `truncated`; a magic other than `WAS1`,
|
||
`malformed`; no fifth byte, `truncated`; a version other than 1, `unknown_version`; the bytes end
|
||
before a field the layout requires, `truncated`; a class whose tag 5 reference is longer than 255
|
||
bytes, `malformed`; bytes after `sig`, `malformed`. It then refuses as `audit_session_invalid` a
|
||
record that breaks any of these:
|
||
|
||
- `presentation` is 1 or 2, `start` is below `end`, and `ended` is 1 to 5;
|
||
- there is at least one origin, each 1 to 2048 bytes of printable ASCII beginning `http://` or
|
||
`https://`, with no `/`, `?`, `#` or `@` after those, in strictly ascending order;
|
||
- there is at least one acknowledgement; `acks`, `claims` and `attachments` are each in strictly
|
||
ascending order;
|
||
- each span's `length_ms` is at least 1, its `visible_bp` at most 10 000 and its `flags` at most
|
||
3; each starts at or after the end of the one before; the last ends at or before
|
||
⌊(`end` − `start`) / 1000⌋;
|
||
- `startup_ms` is 4 294 967 295 exactly when there is no span; `stall_ms` is 0 when `stalls` is;
|
||
`repeat_bytes` is 0 when `repeat_objects` is, and at least it otherwise;
|
||
- `log` is not zero;
|
||
- the energy entries are in strictly ascending order of `role`, each 1 to 4;
|
||
|
||
and as `energy_invalid` one whose energy entry breaks live-metering §3.6's rules for a figure
|
||
and its class (the tag is 0 exactly when `joules` is; no tag 1; the hardware classes' evidence
|
||
and resolution floor). Then the signature (§5.4): `bad_key`, `bad_signature`.
|
||
|
||
## 6. Ads presented
|
||
|
||
### 6.1 Marked ads and their markers
|
||
|
||
An ad is **marked** when the media playlist the player presents carries an `EXT-X-DATERANGE`
|
||
tag (`draft-pantos-hls-rfc8216bis-22` §4.4.5.1) over it with a `SCTE35-OUT` or `SCTE35-CMD`
|
||
attribute, or with a `CLASS` the player is configured to treat as an ad. Its **marker** is the
|
||
tag's line as the playlist carries it, from `#` to the last byte before the line terminator, and
|
||
its **marker digest** is SHA-256 over `wai:audit-marker` (ASCII), 0x01 and the marker. The marked
|
||
span runs from the tag's `START-DATE` for its `DURATION` (or `PLANNED-DURATION`), placed in the
|
||
playlist by its `EXT-X-PROGRAM-DATE-TIME` tags.
|
||
|
||
### 6.2 The creative
|
||
|
||
The ad's **creative** (ad-presentation §4) is the media segments of the presented playlist
|
||
whose span meets the marked span, in order, each by the SHA-256 of its bytes as received
|
||
(settlement §14.1) and its `EXTINF` duration in milliseconds, rounded half up and at least 1. Its
|
||
media time starts at the first such segment's start. A creative's objects are among the
|
||
objects the session acknowledged.
|
||
|
||
### 6.3 The claim
|
||
|
||
For each marked ad it presented, the player signs a claim of ad-presentation §5, with `room` the
|
||
channel id, `cue` the marker digest, `session` the session, `slot` 0, `creative_hash` the
|
||
creative digest, `player` the session key, and spans over the creative's media time by
|
||
ad-presentation §10's rules. A player that plays into the ad from before it, with no pause or
|
||
seek between its last observation before the ad and its first within it, starts the ad's first
|
||
span at media time 0; one that plays out of it the same way ends its last span where it left, at
|
||
most the creative's duration: playback was continuous across the boundary, which fell between two
|
||
observations. A claim states what ad-presentation §13 says one states, and here no
|
||
decider's receipt stands beside it: its quartile and viewability are computed (§10.2 step 9) and
|
||
reported, not compared with anyone's count.
|
||
|
||
This revision claims every marked ad with `WPC1` under the channel binding above. A record of
|
||
its own for ads another party stitches into a stream, `wai.video.stitched_presentation`, names an
|
||
ad by its marker in a manifest whose SHA-256 it states; where a deployment holds that record, a
|
||
stitched ad is claimed with it, the bundle then carries the bytes of each manifest its claims
|
||
name, and `WPC1` stays for a marked ad that is not stitched. This revision does not read it.
|
||
|
||
## 7. Telemetry
|
||
|
||
### 7.1 The observation log
|
||
|
||
The player keeps a delivery-telemetry log of the session (delivery-telemetry §2 and its §7.1),
|
||
whose `sid` is the session as 32 lowercase hexadecimal digits: fresh per session and derived
|
||
from no identifier, as delivery-telemetry §8 requires. Its `object` members report `url` as
|
||
`null`, since a third party's URL can carry a token in its path, and `content_hash` as the
|
||
object's SHA-256. The **observation log** is live-metering §6.1's logs document holding exactly
|
||
this one log, as UTF-8 bytes. The record's `startup_ms`, `stalls` and `stall_ms` are the log's
|
||
`vst_ms`, `stalls` and `stall_ms` (delivery-telemetry §7).
|
||
|
||
### 7.2 CMCD
|
||
|
||
The log's Event-mode reports, CMCD version 2 (CTA-5004-B) as delivery-telemetry §3 maps them,
|
||
go to the owner's collector as `application/cmcd` batches (live-metering §6.3) while the session
|
||
runs. Request-mode reports **MAY** ride on the player's requests to an origin where the owner
|
||
configures it; they are off by default, since a query added to a third party's URL can break the
|
||
signature that URL carries. A batch is advisory, as every observation is (delivery-telemetry
|
||
§1); what the report reads is the log the record binds.
|
||
|
||
### 7.3 CMSD and the notes document
|
||
|
||
Where a response exposes `CMSD-Static` or `CMSD-Dynamic` (CTA-5006) to the player (in a page,
|
||
only when the origin lists them in `Access-Control-Expose-Headers`; they are not CORS-safelisted
|
||
names), the player records them. The **notes document** is a JSON array, as UTF-8 bytes, of one
|
||
object per response that exposed either, in the order received, with exactly the members
|
||
`content_hash` (the object's SHA-256, 64 lowercase hexadecimal digits), `cmsd_static` and
|
||
`cmsd_dynamic` (each the field's value as received, a string, or `null`). The values are the
|
||
origin's statements, recorded, not parsed or checked. The record names the document by its
|
||
digest, zero when no response exposed either field.
|
||
|
||
## 8. Energy, the end of a session and attachments
|
||
|
||
### 8.1 Energy
|
||
|
||
The entries of §5.3 follow one rule: a figure with its class, the unmetered marker where a role
|
||
took part unmeasured, no entry where the player knows nothing. A page has no meter: it states
|
||
roles 3 and 4 as the unmetered marker and has no entry for roles 1 and 2. A native host or a
|
||
sink with an energy profile states its figure with the class and evidence its counter or model
|
||
gives (energy-measurement §2.7). Nothing sums figures of two classes: the report totals each
|
||
role and class apart (§11.2), as live-metering §4.5 totals a session's hops.
|
||
|
||
### 8.2 How the session ended
|
||
|
||
| `ended` | name | when |
|
||
|---|---|---|
|
||
| 1 | `ended` | the presentation reached its end |
|
||
| 2 | `user` | the viewer stopped it, or left the page or application |
|
||
| 3 | `hidden` | the player ended the video because nobody could see it for its grace period |
|
||
| 4 | `idle` | the player ended it after its idle bound passed with no interaction |
|
||
| 5 | `error` | playback failed |
|
||
|
||
### 8.3 Attachments
|
||
|
||
A player may attach other signed records of the session (a termination record of
|
||
`wai.live.termination`, a sink energy receipt of `wai.sink.energy`, each where the player keeps
|
||
one), each named in `attachments` by the SHA-256 over all its bytes. This revision binds an
|
||
attachment to the session and does not read it; a collector stores it with the session.
|
||
|
||
## 9. The bundle
|
||
|
||
A player sends a session to the collector as one **bundle**: a JSON object (RFC 8259) with
|
||
exactly these members:
|
||
|
||
| member | value |
|
||
|---|---|
|
||
| `v` | 1 |
|
||
| `session` | the record (§5), in lowercase hexadecimal |
|
||
| `acks` | an array, one per digest of `acks`: `{ "record": hex, "items": [hex, …] }`, the acknowledgement and each of its entries' item lists in entry order |
|
||
| `claims` | an array, one per digest of `claims`: `{ "claim": hex, "marker": string, "creative": [{ "content_hash": hex, "duration_ms": integer }, …] }` |
|
||
| `log` | the observation log (§7.1), a string whose UTF-8 bytes are the document |
|
||
| `notes` | the notes document (§7.3) as such a string, or `null` |
|
||
| `attachments` | an array: `{ "name": string, "record": hex }`, the record's registered name and its bytes |
|
||
|
||
A bundle that is not such an object, holds a member not listed or lacks one, a value of another
|
||
type, hexadecimal that is not lowercase or not whole bytes, a creative object whose `duration_ms`
|
||
is not 1 to 4 294 967 295 or whose `content_hash` is not 32 bytes, or a creative with no object,
|
||
is refused `audit_bundle_invalid`.
|
||
|
||
## 10. Verification
|
||
|
||
### 10.1 Inputs
|
||
|
||
A verifier is given a bundle, the owner's key, and the names of the channels the owner serves.
|
||
|
||
### 10.2 Steps
|
||
|
||
It runs these steps and stops at the first refusal, which it reports with the reason and where:
|
||
|
||
1. **The bundle** (§9), at `bundle`: `audit_bundle_invalid`.
|
||
2. **The record** (§5.5), at `session`: `truncated`, `malformed`, `unknown_version`,
|
||
`audit_session_invalid`, `energy_invalid`, `bad_key`, `bad_signature`.
|
||
3. **Its binding**, at `session`: `owner` is the owner's key, `audit_wrong_owner`; `channel` is
|
||
the id of a channel the owner serves, `audit_unknown_channel`; `session` is the session of
|
||
`channel` and `player` (§3.2), `audit_session_unbound`.
|
||
4. **Each acknowledgement**, in the record's order, at `ack` and its index:
|
||
1. one of the bundle's acknowledgements has the digest (settlement §8.2: SHA-256 over all but
|
||
the last 64 bytes): otherwise `record_missing`;
|
||
2. it reads and verifies (settlement §3.6): `truncated`, `malformed`, `unknown_version`,
|
||
`ack_invalid`, `self_acknowledged`, `bad_key`, `bad_signature`;
|
||
3. its `mode` is 1 and its `downstream` is `player`: otherwise `audit_ack_unbound`;
|
||
4. its `upstream` is the upstream of one of the record's origins (settlement §14.2):
|
||
otherwise `audit_origin_unknown`;
|
||
5. its window is within the session, the session's `start` ≤ its `start` and its `end` ≤ the
|
||
session's `end`: otherwise `ack_outside_period`;
|
||
6. its `role` is 2, every entry's `grant` and `proof` are zero, and every entry's class is 4 or
|
||
the class of the record's `presentation` (§4.1): otherwise `audit_ack_invalid`;
|
||
7. each entry, at `entry` and its index: its item list is given (`record_missing`), reads
|
||
(`items_invalid`) and recomputes (`items_mismatch`) as settlement §3.6 says, and every
|
||
item's `start` is 0 (`audit_ack_invalid`); an item list beyond its entries is
|
||
`audit_bundle_invalid`.
|
||
|
||
A bundle acknowledgement whose digest the record does not name, or that repeats one, is
|
||
`audit_bundle_invalid`, at `ack` and its index in the bundle.
|
||
5. **The chains**, at `ack` and an index: each origin's acknowledgements ordered by (`start`,
|
||
index), as settlement §8.2 step 4 orders a chain: the first with a `prev` other than zero,
|
||
`ack_chain_broken`; a later one whose window overlaps the one before it, `ack_overlap`, or
|
||
whose `prev` is not that one's digest, `ack_chain_broken`. Of every such break, the one at the
|
||
acknowledgement first in the record's order is reported.
|
||
6. **The log**, at `log`: the SHA-256 of the bundle's `log` bytes is `log`, otherwise
|
||
`audit_document_mismatch`; it reads as a logs document of exactly one log (live-metering
|
||
§6.1), otherwise `qoe_invalid`; and its `sid` is the session's and its `vst_ms`, `stalls` and
|
||
`stall_ms` are the record's `startup_ms` (`null` for 4 294 967 295), `stalls` and `stall_ms`,
|
||
otherwise `audit_qoe_mismatch`.
|
||
7. **The notes**, at `notes`: with `notes` zero, the bundle's `notes` is `null`; otherwise its
|
||
SHA-256 is `notes`: otherwise `audit_document_mismatch`. A notes document that does not read
|
||
as §7.3 says, or that names a `content_hash` no item of the session's acknowledgements has,
|
||
is `audit_notes_invalid`.
|
||
8. **The attachments**, at `attachment` and its index: each digest the record names is an
|
||
attachment's (`record_missing`), and each attachment's digest is one the record names
|
||
(`audit_document_mismatch`).
|
||
9. **Each claim**, in the record's order, at `claim` and its index: a bundle claim with the digest
|
||
(SHA-256 over all but its last 64 bytes) is given, otherwise `record_missing`, and a bundle
|
||
claim whose digest the record does not name is `audit_bundle_invalid`. Then each claim gets a
|
||
verdict, **presented** or the reason of the first of these it fails:
|
||
1. it reads (ad-presentation §5.4): `claim_malformed`;
|
||
2. its signature verifies under its `player` (ad-presentation §5.3): `bad_key`,
|
||
`bad_signature`;
|
||
3. its `room` is the channel id, its `session` the session and its `player` the session key:
|
||
`audit_claim_unbound`;
|
||
4. its `cue` is the digest of the marker beside it (§6.1): `audit_marker_mismatch`;
|
||
5. no claim before it that passed steps 1 to 4 has its `cue` and `slot`: `claim_duplicate`;
|
||
6. its `creative_hash` is the digest of the creative beside it, every span ends within the
|
||
creative's duration, and its `objects` are exactly the content hashes of the creative's
|
||
objects its spans touch, in order (ad-presentation §9.2 step 6): `claim_asset_mismatch`;
|
||
7. every one of its `objects` is the content hash of an item of one of the session's
|
||
acknowledgements: `audit_ad_not_acknowledged`.
|
||
|
||
A claim's verdict does not refuse the session. A presented claim's facts are the quartile it
|
||
reaches (ad-presentation §6.3, with the creative's duration), and its run and viewable time
|
||
under the video rule (ad-presentation §6.2, §6.1 `viewability` 1), and whether that run is at
|
||
least 2 000 ms.
|
||
|
||
### 10.3 Outcome
|
||
|
||
- **refuse**: the reason and where.
|
||
- **accept**: the record's digest; the channel; `presentation`, `start`, `end` and `ended`; the
|
||
time presented (the sum of the spans' `length_ms`), of it in view (spans whose `visible_bp` is at
|
||
least 5 000), audible, and within ads; for each origin and object class, the acknowledged
|
||
objects and bytes, and the repeats; the start time, stalls and stall time; each energy entry
|
||
with its role, class label and figure, or the unmetered marker; each claim's verdict and, for a
|
||
presented claim, its facts; the notes' count of responses; and the attachments' count.
|
||
|
||
## 11. The channel report
|
||
|
||
### 11.1 Its sessions
|
||
|
||
A report covers one channel of one owner over a **period** from `from` to `to`. Its sessions are
|
||
the accepted bundles of that channel and owner whose record's `start` lies in the period. Two
|
||
records with one `session` are refused `audit_session_duplicate`, at `report`: a session reports
|
||
once.
|
||
|
||
### 11.2 The document
|
||
|
||
The report is a JSON object with exactly these members:
|
||
|
||
| member | value |
|
||
|---|---|
|
||
| `v` | 1 |
|
||
| `channel`, `owner` | the channel id and the owner's key, hexadecimal |
|
||
| `from`, `to` | the period |
|
||
| `sessions` | the records' digests, hexadecimal, in ascending order |
|
||
| `presented` | `{ "presented_ms", "viewable_ms", "audible_ms", "ad_ms" }`: §10.3's sums over the sessions |
|
||
| `ended` | `{ "ended", "user", "hidden", "idle", "error" }`: the sessions by `ended` |
|
||
| `ads` | `{ "claims", "presented", "quartiles", "viewable", "refused" }`: the claims, those presented, an array of five counts (the presented claims reaching quartiles 1 to 5), those whose run under the video rule is at least 2 000 ms, and an object of each refusing verdict's count by its code |
|
||
| `rebuffering` | `{ "sessions", "suppressed", "presented", "startup_ms", "stalls", "stall_ms" }`: the sessions; `true` when they are fewer than 50, and then every other member `null` (delivery-telemetry §8); otherwise `false`, the sessions with a span, the sum of their `startup_ms`, and the sums of `stalls` and `stall_ms` over all |
|
||
| `delivery` | an array of `{ "origin", "class", "objects", "bytes" }`, one per origin and object class with an acknowledged object (class by settlement §3.3's name), in ascending order of origin then class |
|
||
| `repeats` | `{ "objects", "bytes" }`: the sums of the records' repeats |
|
||
| `energy` | an array of `{ "role", "class", "figures", "joules_micro" }`, one per role and acquisition class among the sessions' metered entries (role `origin`, `delivery`, `sink_decode` or `display`; class live-metering §4.5's label), in ascending order of role number then tag |
|
||
| `unmetered` | an object with a member per role name: the sessions whose entry for the role is the unmetered marker |
|
||
| `absent` | an object with a member per role name: the sessions with no entry for the role |
|
||
|
||
Joules are summed within one role and one class and never across: a role's figures of two classes
|
||
are two members of `energy`, and a session whose entry is the unmetered marker counts in
|
||
`unmetered`, never as zero joules.
|
||
|
||
### 11.3 Its bytes and signature
|
||
|
||
The report's **bytes** are its JSON with every object's members in bytewise order of their
|
||
names, no white space, integers in decimal, and strings with no escape but `\"` and `\\`: the
|
||
form RFC 8785 gives such a document. The collector signs them: Ed25519 by the owner's key over
|
||
the 17 bytes `wai:audit-report` (ASCII) and 0x01, followed by the report's bytes. A served report
|
||
is `{"report": <the report>, "sig": <hex>}`.
|
||
|
||
### 11.4 Recomputation
|
||
|
||
A party given the bundles, the channel's name, the owner's key and the served report recomputes
|
||
the report over the report's own period, in this order: each bundle verifies as §10 verifies it,
|
||
otherwise its refusal, at `bundle`, its index and its place; a served report that is not an object
|
||
of `report` and `sig`, or whose `from`, `to` or `sessions` do not read, is
|
||
`audit_report_mismatch`; two of the sessions it reads with one `session` are
|
||
`audit_session_duplicate`; a digest the report names that no given bundle has is `record_missing`,
|
||
at `report`, `session` and its index; a report whose bytes are not the recomputed report's bytes
|
||
is `audit_report_mismatch`; and one whose signature does not verify under the owner's key is
|
||
`bad_signature`, each at `report` where no other place is named.
|
||
|
||
## 12. The collector
|
||
|
||
A collector:
|
||
|
||
- **MUST** verify a bundle (§10) before it stores it, and refuses with its reason one that does
|
||
not verify;
|
||
- stores an accepted bundle under its record's digest, and accepts the same bundle again with no
|
||
change; a record naming the `session` of a stored record that differs from it is refused
|
||
`audit_session_duplicate`;
|
||
- serves a channel's report for a period (§11), signed with its key, and its key;
|
||
- **MAY** accept Event-mode CMCD batches (§7.2), which it stores apart and never reads into a
|
||
report;
|
||
- names nothing a player has not sent: no cookie, no account and no identifier of a viewer.
|
||
|
||
The reference collector, `wai-audit serve`, keeps one directory per channel, a file per session,
|
||
and answers on loopback; its README states its routes and limits.
|
||
|
||
## 13. The player integration
|
||
|
||
The reference integration is `wai/wai-audit/web/audit.js`, a module with no dependency, and its
|
||
adapters: one that wraps the page's `fetch` and `XMLHttpRequest` for any player that fetches over
|
||
them, and one for a player whose loader interface a page can wrap. The module hashes each object
|
||
with WebCrypto's SHA-256, writes and signs the acknowledgements, the claims and the record with
|
||
WebCrypto's Ed25519, computes jwp-receipts §2.2's tree root with a BLAKE3 of its own for inputs of
|
||
one chunk, observes the media element every 200 ms as ad-presentation §10 says, writes the
|
||
observation log and its Event-mode CMCD, and sends the bundle when the session ends. Its tests
|
||
drive it with the corpus's inputs and require the corpus's records, byte for byte.
|
||
|
||
## 14. Reasons
|
||
|
||
The codes this extension adds, each with where it is raised:
|
||
|
||
| reason | raised at |
|
||
|---|---|
|
||
| `audit_bundle_invalid` | §9, §10.2 steps 1, 4 and 9 |
|
||
| `audit_session_invalid` | §5.5, §10.2 step 2 |
|
||
| `audit_wrong_owner` | §10.2 step 3 |
|
||
| `audit_unknown_channel` | §10.2 step 3 |
|
||
| `audit_session_unbound` | §10.2 step 3 |
|
||
| `audit_ack_unbound` | §10.2 step 4 |
|
||
| `audit_origin_unknown` | §10.2 step 4 |
|
||
| `audit_ack_invalid` | §10.2 step 4 |
|
||
| `audit_document_mismatch` | §10.2 steps 6 to 8 |
|
||
| `audit_qoe_mismatch` | §10.2 step 6 |
|
||
| `audit_notes_invalid` | §10.2 step 7 |
|
||
| `audit_claim_unbound` | §10.2 step 9 |
|
||
| `audit_marker_mismatch` | §10.2 step 9 |
|
||
| `audit_ad_not_acknowledged` | §10.2 step 9 |
|
||
| `audit_session_duplicate` | §11.1, §12 |
|
||
| `audit_report_mismatch` | §11.4 |
|
||
|
||
A verifier of this extension also raises `truncated`, `malformed` and `unknown_version` for a
|
||
record that does not read; `bad_key` and `bad_signature`; `record_missing` (live-metering §9);
|
||
`energy_invalid` and `qoe_invalid` (live-metering §9); settlement §11's `ack_invalid`,
|
||
`self_acknowledged`, `items_invalid`, `items_mismatch`, `ack_outside_period`, `ack_overlap` and
|
||
`ack_chain_broken`; and ad-presentation §11's `claim_malformed`, `claim_duplicate` and
|
||
`claim_asset_mismatch`, as claim verdicts.
|
||
|
||
## 15. Conformance
|
||
|
||
[`live-conformance/audit/`](../live-conformance/audit/) holds the corpus, written from fixed
|
||
seeds by `wai_stream_audit_vectors gen` as `vectors.json`. It holds the owner's key, the channel
|
||
names, the objects' bytes, and:
|
||
|
||
- **sessions**, each a bundle with the outcome §10.3 gives it. **Accepted**: a live session from
|
||
two origins with a marked ad presented in view throughout and one presented half out of view,
|
||
repeats, CMSD notes and a sink-decode figure of the `Estimator` class beside the unmetered
|
||
display; an on-demand session ended `hidden` with an `OnChipCounter` figure and an attachment;
|
||
a session that presented nothing and ended `error`; and a session whose claims get every claim
|
||
verdict of §14 and of ad-presentation's that §10.2 step 9 names. **Refused**: each reason of §14
|
||
that a session raises, and each code it borrows, each vector a control of one defect.
|
||
- **reports**, each a channel, a period, the sessions it reads and the outcome §11.4 gives it:
|
||
a report over the accepted sessions, whose rebuffering is suppressed; a report over a channel
|
||
of 52 sessions, whose rebuffering is stated; and refused, a report whose bytes do not
|
||
recompute, one whose signature fails, one naming a session not given, and one over two records
|
||
of one session.
|
||
|
||
A conforming verifier gives every session and every report the outcome, the reason, the place and
|
||
every figure the corpus states. `wai_stream_audit_vectors verify` and the test
|
||
`tests/stream_audit.rs` check that the file regenerates byte for byte, that `wai-rs` gives every
|
||
vector its outcome, that every reason of §14 and every borrowed code refuses a vector, and that §14
|
||
lists exactly the reference's codes. `live-conformance/stream_audit_verify.py` is the second
|
||
implementation, written from this text: it shares no code with `wai-rs`; it reads acknowledgements
|
||
and item lists with the settlement corpus's second implementation (`settlement_verify.py`), claims
|
||
with the ad-presentation corpus's (`ad_presentation_verify.py`), and logs and energy classes with
|
||
the live-metering corpus's (`live_metering_verify.py`), verifies every signature with WAI's strict
|
||
rule (`claim-conformance/_ed25519.py`), recomputes every report itself, gives every vector the same
|
||
outcome, requires every mutated copy of the corpus to fail, and requires the corpus to fail a copy
|
||
of itself with each of its checks dropped.
|
||
|
||
## 16. Security and privacy considerations
|
||
|
||
- **What the evidence shows, and to whom.** The relying party is the owner. A session's records
|
||
show what the holder of a fresh key received, from which origins, what it presented and how
|
||
that went; they show nothing of the delivering service's own logs, and they need nothing from
|
||
it. A report's figures are recomputable from the stored records, so the owner can show a party
|
||
it deals with (an ad buyer, a rights holder) a figure that party recomputes.
|
||
- **What the evidence does not show.** That the bytes were what the owner published: nothing the
|
||
delivering service signs is read, so a stream altered on the way is acknowledged as received.
|
||
An owner that also signs what it publishes (jwp-receipts) can compare its own content hashes with
|
||
the acknowledged ones. That a person watched (ad-presentation §13). That the player is honest: a
|
||
modified player signs what it is told to, and a party can make keys and sign sessions it never
|
||
played. The records make each session attributable to one key and each figure recomputable; they
|
||
do not make fabrication impossible, and an owner weighs sessions from keys it cannot place as
|
||
ad-presentation §13 weighs claims without attestation.
|
||
- **Collusion between the session key's holder and an origin** can produce acknowledgements of
|
||
bytes that never moved, as settlement §13 says of any downstream party; repeats and the log's
|
||
timing make a replayed session's figures stand out, and nothing here detects one.
|
||
- **What a session reveals.** To the collector: the session's objects by digest, their origins,
|
||
its timing, what it presented and its energy entries. The session key is fresh and the session
|
||
derived from it (§3.2), so sessions do not link to each other or to a viewer, device or account;
|
||
`sid` is the session (§7.1); the log reports no URL; the record names the channel by the owner's
|
||
label, never by a URL. A digest of a public segment names a position in the stream, which is
|
||
what an acknowledgement is for; a key response is never acknowledged (§4.1). The collector sends
|
||
records to no one; a report states sums over sessions and suppresses rebuffering below 50
|
||
sessions (delivery-telemetry §8). This extension sets no gate of its own: sending the records is
|
||
part of playing the owner's stream, as acknowledging is part of a delivery (settlement §13).
|
||
Where a regime asks the operator to state the lawful basis on which it processes these records,
|
||
the operator states it as a `lawful-basis` declaration (jwp-receipts §2.4.6), which this
|
||
extension does not restate or check.
|
||
- **Replay.** A record names its owner and channel, and its session derives from its key: a record
|
||
sent to another owner's collector is `audit_wrong_owner`, and a record for one session counts
|
||
once in a report (§11.1).
|
||
- **Size.** A collector bounds a bundle's bytes and a client's rate; the reference's limits are in
|
||
its README.
|