open-standards/wai/wai-audit/README.md

# wai-audit

The stream audit kit of [`extensions/stream-audit.md`](../extensions/stream-audit.md): evidence
a content owner gathers from the players of a stream another service packages and delivers over
HTTP, with nothing asked of that service.

- **The player integration** (`web/`): for each playback session the player makes a fresh key,
  hashes every playlist and segment it fetches, and signs settlement acknowledgements (`WDA1`) of
  them by plain SHA-256, one chain per origin; a presentation claim (`WPC1`) for each ad the
  playlist marks; and the session record (`WAS1`): the spans it presented, in view and audible,
  its start time and stalls, its energy per role with each figure's class, and how it ended. It
  sends Event-mode CMCD version 2 to the collector as it plays, and the session's bundle when it
  ends.
- **The collector** (`wai-audit serve`): verifies each bundle, stores the accepted ones one per
  session, takes the CMCD batches, and serves each channel's report, signed: time presented, ads
  presented and the quartile each reached, start time and stalls once 50 sessions hold them,
  acknowledged bytes per origin and class, and joules per role and class, never summed across
  classes. Anyone holding the stored bundles recomputes the report byte for byte
  (`wai-audit check`).

No price, tariff or amount is computed anywhere here.

## Run the collector

```sh
cargo build --release
WAI_AUDIT_CHANNELS=news-east,film-library \
WAI_AUDIT_ALLOW_ORIGINS=https://player.example \
  ./target/release/wai-audit serve
```

| variable | default | meaning |
|---|---|---|
| `WAI_AUDIT_CHANNELS` | (required) | the channel names served, comma-separated, each 1 to 255 bytes of printable ASCII (§3.1) |
| `WAI_AUDIT_ADDR` | `127.0.0.1:9694` | where it listens |
| `WAI_AUDIT_DATA` | `~/.local/share/wai-audit` | where it stores sessions and CMCD batches |
| `WAI_AUDIT_KEY` | `~/.config/wai-audit/signing.key` | the owner's Ed25519 key, 32 bytes, created readable by its owner alone if absent |
| `WAI_AUDIT_ALLOW_ORIGINS` | none | the page origins a browser may send from (CORS), or `*` |
| `WAI_AUDIT_TRUSTED_PROXIES` | `127.0.0.1,::1` | peers whose `X-Forwarded-For` names the client |
| `WAI_AUDIT_PER_MINUTE` | `120` | requests a client may make a minute (an IPv4 address, or an IPv6 /64) |
| `WAI_AUDIT_MAX_BUNDLE` | `4194304` | the largest bundle, bytes |
| `WAI_AUDIT_MAX_BATCH` | `262144` | the largest CMCD batch, bytes |

## Routes

| route | answer |
|---|---|
| `GET /v1` | the collector: its key, its channels and their ids, its limits |
| `GET /v1/health` | `{"ok": true}` |
| `GET /v1/key` | the owner's key, hexadecimal, as text: what a player's records name as `owner` |
| `POST /v1/sessions` | a bundle (§9), any media type: 201 stored, 200 stored before, 409 another record of a stored session (`audit_session_duplicate`), 413 too large, 422 refused, 429 too many; with the outcome (§10.3) |
| `POST /v1/cmcd` | an Event-mode CMCD batch (`application/cmcd`, live-metering §6.3): 204, or 422 naming the first line that is not an Event-mode report of version 2 |
| `GET /v1/channels/{name}/report?from=&to=` | the channel's report over the period (microseconds since 1970), `{"report", "sig"}` (§11.3) |
| `GET /metrics` | the collector's counters in the text exposition format (`../wai-metrics`): requests by route, method and status (`wai_audit_requests_total`), session bundles verified by outcome (`wai_audit_sessions_total`: stored, stored again, duplicate, refused, error), CMCD batches and reports by outcome, and the channels served. No channel, session, key or client is a label. Through a front: 404 |

Storage: `<data>/channels/<channel id>/<session>.json`, the bundle as the player sent it, written
to a temporary name and renamed; `<data>/cmcd/<UTC date>.cmcd`, the batches as they came. A report
verifies every stored session of its channel again, so a stored file that no longer verifies
(another key, a channel no longer served) is an error, not a silent omission.

## As a library

The collector and its routes are the crate's library, for a service that hosts them; the binary
reads its configuration from the environment and calls them:

```rust
let key = wai_audit::load_or_create(&key_path)?;             // the owner's key, as `serve` keeps it
let collector = Arc::new(wai_audit::collector::Collector::new(config, key));
let app = Arc::new(wai_audit::http::App::new(collector, trusted_proxies));
axum::serve(listener, wai_audit::http::router(app).into_make_service_with_connect_info::<SocketAddr>()).await?;
```

## The command line

```sh
wai-audit key [PATH]                          # the owner's key at PATH (created if absent)
WAI_AUDIT_OWNER=<hex> wai-audit verify bundle.json news-east
WAI_AUDIT_KEY=<path>  wai-audit report <data> news-east <from> <to>
WAI_AUDIT_OWNER=<hex> wai-audit check served.json news-east bundle-1.json bundle-2.json …
```

`verify` prints a bundle's outcome; `report` recomputes a channel's report from a data directory
and signs it; `check` recomputes a served report from the bundles it reads (§11.4) and checks
its signature.

## Put it in a page

```html
<script type="module">
  import { AuditSession, attach, CLASS, ROLE } from "./audit.js";
  import { tap } from "./adapter-tap.js";
  const session = await AuditSession.start({ channel: "news-east", owner: "<the collector's key>", collector: "https://audit.example/" });
  session.energy(ROLE.SINK_DECODE, 0, CLASS.UNMETERED); // a page has no meter (§8.1)
  session.energy(ROLE.DISPLAY, 0, CLASS.UNMETERED);
  tap(session, { manifest: src });                      // before the player starts fetching
  // ... start the page's own player on `src` ...
  attach(video, session, { programMs: () => session.programAt(video.currentTime) });
</script>
```

- `adapter-tap.js` wraps the page's `fetch` and `XMLHttpRequest` and hands the session every
  response the presentation names: for any web player that fetches over them.
- `adapter-hlsjs.js` wraps the loader class of a player whose configuration takes one, and gives
  the program time of the position from the player itself, which a live stream needs. <!-- allow-name: the adapter's file is named for the open-source player whose loader interface it wraps -->
- `attach` observes the media element every 200 ms and on its events, logs the first
  presentation, stalls and the end, and closes and sends the session when playback ends or the
  page goes away.

The origin serving the stream must answer cross-origin requests (players need that already),
and must list `CMSD-Static` and `CMSD-Dynamic` in `Access-Control-Expose-Headers` for the kit to
record them. Ad markers are read from HLS `EXT-X-DATERANGE` tags with `SCTE35-OUT`, `SCTE35-CMD`
or a configured `CLASS`; DASH events are not read in this revision. A bundle sent as a page goes
away rides on a keep-alive request, which a user agent may refuse past its quota (64 KiB in the
Fetch Standard); the kit sends at the end of playback first.

## Tests

- `cargo test --release`: every session of the corpus `live-conformance/audit/` posted gets the
  corpus's outcome, and the stored sessions report exactly the corpus's served reports, byte for
  byte and signature; CORS, body and request limits, CMCD intake; `GET /metrics` read strictly as
  the text format, its counters moved by those requests, and none of their keys, digests,
  sessions, channels, origins or addresses in it.
- `node --test web/audit.test.mjs`: a session driven with the corpus's inputs signs the corpus's
  records byte for byte, its log and CMCD batch too; the playlist reader, the tap and the loader
  adapter.
- `e2e/run.sh`: a real browser plays an HLS stream from another origin through an open-source web
  player, audited through the loader adapter and through the tap, each session verified, stored and
  reported by a collector on loopback, the report recomputed by `wai-audit check`. It needs the
  host's media tool and a browser (`BROWSER_BIN`). The browser first plays the stream once,
  untimed, through the player alone (`e2e/page/warm.html`): what a browser starts for its first
  media (its audio service, and on macOS the system's check of a helper program's first run) is
  then not inside a timed session, whose start time, held to 10 s, is the kit's, the player's and
  the stream's. `--hold-playlist-ms N` holds the stream's playlist N ms in each timed session, a
  planted slow start that check must fail.

## Deploy

One process per owner key, on loopback behind the site's reverse proxy, which terminates TLS and
strips a path prefix (`/audit/` → `/`); the proxy is the trusted peer whose `X-Forwarded-For`
names the client. Keep `WAI_AUDIT_DATA` on storage that is backed up: the stored bundles are the
evidence a report is recomputed from. Back up the key apart from the data, readable by the
service's user alone. Run it under the host's service manager with restart on failure, and check
`GET /v1/health`. `GET /metrics` is for the operator's scraper, at the service's own address: a
request that came through the proxy, one carrying `X-Forwarded-For` or another of the fields a
front sets, is answered `404`, and a proxy that sets none of them is configured not to pass the
path on.