open-standards/wai/extensions/native-sink.md

# WAI Extension: Native Sink

> Status: Draft. What a **native host**, an application that plays a stream through its
> platform's own player, does so that the stream is verified where it carries WAI's receipts and
> evidenced as [stream-audit](stream-audit.md) evidences a page's playback. The platform's player
> decodes on the platform's media engine and is not changed; the **sink** sits on the path its
> requests take and, for one playback session: verifies a media playlist against its playlist
> receipt and the chain of its live revisions ([http-delivery](http-delivery.md) §8.6), holds an
> object whose name states its content hash to that hash, and tells the host, object by object,
> whether it may hand the bytes to the player (§4); signs, under a fresh key per session, what
> stream-audit has a player sign: settlement acknowledgements of every object it received, a
> presentation claim for each ad the stream marks, and the session record, with the energy of
> each role and its acquisition class or the unmetered marker (§5, §7); ends a session nobody can
> see, or whose viewer has gone idle (§6); and hands the host the bundle for the owner's
> collector. One C interface serves every native platform (§9). No new record, no new transport
> and no change to the wire format. The reference implementation is the `native_sink` module of
> `wai-rs` and its C interface `ffi_sink` (feature `native_sink`, header `include/wai.h`); the
> Swift package `wai/wai-sink-swift` integrates it with the native player of the platforms whose
> language it is, and the Kotlin library `wai/wai-sink-android` with the media player library of
> the phone and television platform whose applications are written in Kotlin; the corpus is [`live-conformance/native-sink/`](../live-conformance/native-sink/),
> and its second implementation is `live-conformance/native_sink_verify.py`.
> Keywords **MUST**, **MUST NOT**, **SHOULD**, **MAY** are RFC 2119/8174.
>
> Texts this revision relies on: the WAI texts it cites; `draft-pantos-hls-rfc8216bis-22` §4.4
> (the playlist tags a sink reads: `EXTINF`, `EXT-X-MAP`, `EXT-X-KEY`, `EXT-X-SESSION-KEY`,
> `EXT-X-PROGRAM-DATE-TIME`, `EXT-X-DATERANGE`, `EXT-X-ENDLIST`, `EXT-X-STREAM-INF`) and §4.2
> (attribute lists); RFC 3339 §5.6 (date-times); RFC 3986 §5.2 and §6.2.2 (resolution and
> normalization); RFC 6454 §6.2 (the ASCII serialization of an origin); RFC 9110 §8.4 (content
> codings); RFC 8032; FIPS 180-4; and the BLAKE3 specification. A section number in this text is
> this extension's unless it names another document.

## 1. Scope

In scope: the session and its configuration (§3); what a sink reads of a playlist, how it
verifies a playlist and an object, and what a host does with the answer (§4); the observations a
host gives and the log they make (§5); the rule that ends a session nobody can see (§6); the
energy a native host states (§7); the reasons (§8); the C interface (§9); and conformance (§10).

Out of scope: a transport; the records the sink signs, their verification and the channel report,
which stream-audit defines; the playlist receipt and its chain, which http-delivery §8.6 defines;
the player itself and its decoder; the collector; and how a host routes its player's requests
through the sink, which each platform's integration states (§4.4).

Relationship to the core spec: this extension registers nothing. It is a profile of stream-audit
for a native host, with verification added from http-delivery and packaging, and the C interface
that every native integration shares.

## 2. Terms

- **Native host**: an application on a phone or tablet, a television or set-top box, or a
  computer, that plays a stream with the player its platform provides.
- **Player**: the platform's player. It requests the playlists and segments of an HTTP
  presentation, decodes and presents them; this extension changes nothing in it.
- **Sink**: the code of this extension, in the host's process, on the path the player's requests
  take: the player asks the sink, the sink fetches, verifies and answers (§4.4).
- **Session**, **channel**, **owner**, **session key**, **object**, **origin**: as
  stream-audit §2 and §3 define them.
- **URL key**: an object's URL after redirects, resolved against itself and normalized as
  http-delivery §11.5 does (RFC 3986 §5.2 and §6.2.2), without its fragment. Two spellings of one
  URL are one key.
- **Content hash**: of a WAI object, jwp-receipts §2.1's, `BLAKE3("moq-jwp:object\x01" ‖
  envelope)`; of any other object, its SHA-256 (settlement §14.1).
- Time is microseconds since 1970-01-01T00:00:00Z on the host's clock; a media playlist's
  program time is milliseconds on the same scale.

## 3. The session

### 3.1 Configuration

A host opens a session with a JSON object of these members; `channel` and `owner` are required,
every other has the default shown, and a member not listed, or a value of another type, is refused
(`native_config_invalid`):

| member | value | default |
|---|---|---|
| `channel` | the channel's name, as stream-audit §3.1 has it | — |
| `owner` | the owner's key, 64 lowercase hexadecimal digits | — |
| `presentation` | `"live"`, `"on_demand"`, or `null` to take it from the first media playlist taken: on demand when it has `EXT-X-ENDLIST` | `null` |
| `window_us` | the acknowledgement window of stream-audit §4.2, at least 1 000 000 | 10 000 000 |
| `ad_classes` | `EXT-X-DATERANGE` classes the owner treats as ads (stream-audit §6.1) | `[]` |
| `content_addressed` | `true` when the owner's packager names segments by the SHA-256 of their bytes (packaging §7), so that such names hold under a playlist with no receipt (§4.3) | `false` |
| `require` | `"nothing"`, or `"signed"`: every media playlist sealed by its receipt, and every object listed by one (§4.2) | `"nothing"` |
| `unseen_grace_ms` | how long a video stream stays unseen before the session ends as `hidden`, 0 for never (§6.1) | 10 000 |
| `idle_ms` | how long with no interaction before it ends as `idle`, 0 for never (§6.1) | 0 |

### 3.2 The session key

A sink **MUST** make a fresh Ed25519 key for each session from its platform's random generator, as
stream-audit §3.2 requires, and **MUST NOT** derive it from an identifier of the viewer, the device
or an account. The C interface takes a 32-byte seed in its place for conformance vectors and tests
only (§9); a host **MUST NOT** pass one for a viewer's session. The session is stream-audit §3.2's,
over the channel id and the key.

### 3.3 Lifetime

The session's log opens with a `start` observation at the time the session opens (§5.2). The
session ends when the host ends it (§6), and then signs its records once. Every call after that is
refused (`native_session_ended`), so no record can be signed twice for one session.

## 4. Verification

### 4.1 Reading a playlist

A sink reads a playlist as a player does, line by line (a line ends at LF; a CR, space or tab
before it is not part of it), and refuses (`playlist_malformed`) one that is not UTF-8, is longer
than 16 MiB, or whose first line, after an optional byte order mark, does not begin `#EXTM3U`. A
playlist with an `EXT-X-STREAM-INF` or `EXT-X-I-FRAME-STREAM-INF` line is a **multivariant**
playlist, and the sink takes nothing from it but that. Of a media playlist it reads:

- each media segment: the URI line after its `EXTINF`, resolved against the playlist's URL key
  (§2); its duration, the `EXTINF` value as a decimal number of seconds read exactly and rounded
  half up to the millisecond (a value that does not read, or reads as 0, is 1 ms); and its program
  time, the last `EXT-X-PROGRAM-DATE-TIME` before it plus the durations of the segments since;
- the URI of `EXT-X-MAP`, and the URIs of `EXT-X-KEY` and `EXT-X-SESSION-KEY`, resolved so;
- each marked ad of stream-audit §6.1: an `EXT-X-DATERANGE` with `SCTE35-OUT` or `SCTE35-CMD`, or a
  `CLASS` the configuration names, whose `START-DATE` reads and which has a `DURATION` (or a
  `PLANNED-DURATION`), its marker the line as the playlist carries it;
- whether it has `EXT-X-ENDLIST`.

An attribute list is read as `draft-pantos-hls-rfc8216bis-22` §4.2 writes it: a name of upper-case
letters, digits and `-`, `=`, and a quoted string or a value up to the next comma. A date-time is
RFC 3339 §5.6's `date-time` with an upper-case or lower-case `T` and `Z`, its fraction truncated to
the millisecond. These readings are the ones stream-audit's integration reads, so that the page and
the native host name the same segments and the same ads.

### 4.2 A playlist and its receipt

Before it hands a media playlist to the player, the host fetches the playlist's receipt file,
`mr/<BLAKE3 of the playlist's bytes>.json` resolved against the playlist (http-delivery §8.6), and
gives the sink both; a receipt that is not served is given as none. Then the sink, in order:

1. reads the playlist (§4.1);
2. with a receipt: checks the playlist against it as http-delivery §8.6 checks one
   (`playlist_receipt_malformed`, `playlist_receipt_mismatch`, `playlist_receipt_invalid`), and
   then takes it into the chain of receipts it holds for the playlist's URL key, as that section
   takes a live revision (`playlist_receipt_unchained`, `playlist_receipt_parent_missing`,
   `receipt_name_mismatch`, `playlist_receipt_walk_exceeded`, `playlist_listing_rebound`). Where
   the chain needs a parent receipt the sink does not hold, it answers with that parent's name,
   `mr/<hash>.json`, and takes nothing: the host fetches it relative to the playlist, gives it to
   the sink (or says it was not served), and gives the playlist again;
3. with no receipt: refuses it (`playlist_unsigned`) when the configuration requires signing;
4. takes it: the playlist is **signed** when its receipt verified and chained, and **listed**
   otherwise. Its segments, its initialization section, its key URIs and its marked ads join the
   session, each segment and initialization section listed by its URL key, as signed or listed.

A refused playlist joins nothing, and the host **MUST NOT** hand it to the player. A multivariant
playlist is listed, never signed: http-delivery §8.6 signs media playlists only.

### 4.3 An object

The host gives the sink every response body the player's requests received, with its content
codings removed (RFC 9110 §8.4), its URL after redirects, the time its body was complete, its
status, and whether it is media where it knows (an initialization or media segment; otherwise the
playlists it took say so). The sink:

1. **keys**: an object whose URL key is a key URI a playlist named is a key or licence response:
   it is not acknowledged, not logged and not checked (stream-audit §4.1);
2. **what its name states**: the object's content hash is checked against the hash its URL states,
   and an object whose bytes are another's is refused (`object_digest_mismatch`):
   - a **cid**: the one the playlist line that lists the object states, its fragment's `cid`
     (http-delivery §3.1) or else a last path segment `<64 hex>.wai` (http-delivery §5.2); else
     the one the object's own URL states in the same way; against the object's jwp-receipts §2.1
     content hash;
   - a **SHA-256**: a last path segment of 64 lowercase hexadecimal digits, a dot and an extension
     of letters and digits other than `wai`, the name a floor fragment has in packaging §7
     (`floor/<id>/<SHA-256>.m4s`),
     against the SHA-256 of the bytes, where a signed playlist lists the object or the
     configuration says `content_addressed`. Under a listed playlist and without it, such a name
     binds nothing: a stream another service packages is never refused for what its names look
     like;
3. **signing required**: under a configuration that requires signing, an object that no signed
   playlist lists, and that is not itself a playlist the session took, is refused
   (`object_unsigned`);
4. **a playlist's body**: an object that is the body of a playlist the session was given (by URL
   key and BLAKE3) is listed as that playlist was, and refused when it was refused;
5. **received**: every object but a key, refused or not, is acknowledged and logged as stream-audit
   §4 and §7 say. An acknowledgement states what was received, not what was presented.

The answer states the object's SHA-256; its cid, where its URL states one; how it is listed
(`signed`, `listed` or `unlisted`); what bound its bytes (`cid`, `sha256` or `none`); whether it is
media; whether it is a key; and the refusal, or none.

### 4.4 What a host does with an answer

A host routes the player's requests through the sink: a server on the loopback interface that
the player is given in place of the origin, or the platform's hook for loading a player's
resources, whichever the platform lets the player use for every request of an HTTP presentation.
For an answer that refuses, the host **MUST NOT** hand the bytes to the player; it answers the
player's request with a failure (the loopback server with status 502), and the player reacts as it
reacts to a failed request. A host **MAY** fetch the object again from another origin the
presentation names; the second response is an object of its own. A host **MUST NOT** answer the
player with bytes the sink was not given.

What a verification establishes is http-delivery §11.6's: an object listed by a signed playlist
and bound by its name is what the playlist's signer published; one listed, only what the origin
that served the playlist named. Nothing here makes a third party's stream signed: such a stream
is acknowledged, and evidenced, as stream-audit says.

## 5. Observations and the log

### 5.1 Observations

While the player presents, the host gives the sink an observation at least every 200 ms, and on
each change of playback, visibility, audibility, rate or position, as JSON:

| member | value |
|---|---|
| `t_us` | when it observed |
| `playing` | presenting at normal speed: not paused, not stalled, not ended, rate 1 |
| `visible_bp` | the fraction of the player's area in view, in basis points (0 to 10 000, a number rounded to an integer): 0 while the application is not shown (§6.2) |
| `audible` | unmuted at a volume above 0 |
| `program_ms` | the program time of the position, or `null` |
| `seeked` | the position moved by a seek since the last observation |
| `interaction` | the viewer interacted since the last observation (§6.2) |

The sink builds the record's spans from them as stream-audit §5.2 does, and each marked ad's claim
spans as ad-presentation §10 and stream-audit §6.3 do. An observation with a member not listed, or
`t_us` or `visible_bp` missing, is refused (`native_input_invalid`).

### 5.2 The log

The sink keeps stream-audit §7.1's observation log. It opens with `start` when the session opens
(§3.3); the host logs `present` when the player shows its first frame, `stall-start` and
`stall-end` around each stall, and `end` or `abandon` when it stops. Where the host logged no
`present`, the first observation that is `playing` logs it; where it logged neither `end` nor
`abandon`, ending the session logs `end` for a session that ended `ended` and `abandon` for any
other. An event of another name is refused (`native_input_invalid`). A log's times never
decrease and nothing follows its `end` or `abandon` (delivery-telemetry §7.1): an entry made at a
time before the last entry's, which a host's player and its requests, on threads of their own,
make, is logged at the last entry's time; and after `end` or `abandon` the log takes nothing,
while an object received then is still verified and acknowledged. The host sends the log's
Event-mode CMCD reports to the owner's collector as it plays (stream-audit §7.2).

## 6. Ending

### 6.1 The termination rule

A sink ends a session that nobody can see or that nobody attends to, by the host's configuration:

- **hidden**: `unseen_grace_ms` is above 0, and the stream has been unseen (an observation that is
  `playing` with `visible_bp` 0, and none since with `visible_bp` above 0) for at least
  `unseen_grace_ms`;
- **idle**: `idle_ms` is above 0, and the last observation with `interaction`, or the session's
  start where there is none, is at least `idle_ms` before.

The answer to the observation that meets a rule names it. The host then **MUST** stop the player's
requests for the stream (it stops the player or removes its item, so no further segment is
fetched) and end the session with that reason (stream-audit §8.2). A host that keeps the audio of a
presentation playing while its video is hidden ends the video's session and keeps the audio's.

### 6.2 What a native host observes

| what | where a native host reads it |
|---|---|
| shown | the platform's state of the application and its window: in the foreground and active, not minimised, not behind the lock screen; `visible_bp` is 0 otherwise. A window the platform shows without the input focus (one of a split screen, a picture-in-picture window, one under a translucent window) is shown: "active" is the platform's state of being presented to the viewer, not of holding the focus |
| in view | the player's area against the window's visible area, and the platform's report that the window is fully covered, where it makes one |
| display | the platform's notice that the display slept, which makes the application not shown |
| interaction | the platform's input events to the application; at most one observation with `interaction` per second, carrying nothing of the input |
| audible | the player's mute state and volume, and the platform's audio route where it reports none |

A host states what its platform reports and claims nothing it does not: a platform that does not
report occlusion has its window's place on the screen, not what other windows cover.

## 7. Energy

A session opens with the unmetered marker for the sink decode (role 3) and the display (role 4),
and no entry for the origin (role 1) or the delivery (role 2): stream-audit §8.1's rule for a sink
with no meter. A host states a role's figure with its acquisition class only when a counter or a
model covers that role's work (energy-measurement §2), replaces a marker with it, or removes a
role's entry where it knows nothing of the role. A figure of 0 with a class, or above 0 with none,
and a class energy-measurement §2.7 refuses, are refused (`native_input_invalid`).

The platform's player decodes on the platform's fixed-function decoder, in a process of the
platform's, which no energy counter an application can read covers. So a native host that plays
through it states the sink decode as the unmetered marker, never as a figure of its own process's
counter: that counter covers the host's own work (its requests, its hashing, its signing), not the
decode. A host that decodes in its own process, a WAI-native capability decoded through `wai-rs`
for example, states the reading of a counter that covers that process, with its class and
evidence. The display is the marker unless the host holds a figure for the panel with its class.

## 8. Reasons

| reason | refused when |
|---|---|
| `native_config_invalid` | the configuration does not read (§3.1) |
| `playlist_malformed` | a playlist is not UTF-8, is longer than 16 MiB, or does not begin `#EXTM3U` (§4.1) |
| `playlist_receipt_malformed` | the receipt file does not read (http-delivery §8.6) |
| `playlist_receipt_mismatch` | the playlist is not the one its receipt renders and hashes (http-delivery §8.6) |
| `playlist_receipt_invalid` | the receipt does not verify (http-delivery §8.6) |
| `playlist_receipt_unchained` | a live revision's receipt does not chain to the head (http-delivery §8.6) |
| `playlist_receipt_parent_missing` | a parent receipt the chain needs is not served (http-delivery §8.6) |
| `receipt_name_mismatch` | a parent receipt's hash is not the name it was fetched by (http-delivery §8.1, §8.6) |
| `playlist_listing_rebound` | a revision lists another object under a URL or sequence number the chain bound (http-delivery §11.5) |
| `playlist_unsigned` | a media playlist has no receipt where the configuration requires one (§4.2) |
| `object_digest_mismatch` | an object's bytes are not the content hash its URL states (§4.3) |
| `object_unsigned` | an object no signed playlist lists, where the configuration requires one (§4.3) |
| `native_input_invalid` | an observation, an event, an object's description or an energy entry does not read (§4.3, §5, §7) |
| `native_session_ended` | a call after the session ended (§3.3) |

A sink gives every other refusal of http-delivery §8.6 by its code (`playlist_receipt_walk_exceeded`
among them). A refusal of a playlist or an object is an answer, not an error: the call succeeds and
the answer says the bytes are not to be presented.

## 9. The C interface

`include/wai.h` declares it; `libwai` built with feature `native_sink` exports it, for every
native target, with no codec library (`cargo build --release --no-default-features --features
native_sink`). A session is an opaque `wai_sink_t`. Inputs are bytes, NUL-terminated UTF-8 strings
and JSON; outputs are JSON or bytes in a `wai_buffer_t` the caller frees with `wai_buffer_free`.
Every function returns a status: `WAI_OK` (0); `WAI_ERR_INVALID_ARG` (1) for a null pointer or a
length that does not fit; `WAI_ERR_PANIC` (4), never an unwinding across the boundary;
`WAI_ERR_SINK_INPUT` (8) for an input that does not read, `wai_last_error` beginning with its reason;
and `WAI_ERR_SINK_ENDED` (9) after the session ended. A session is safe to call from any thread:
each call holds its lock. `WAI_SINK_ABI_VERSION` is 1 in this revision; `wai_sink_abi_version()`
returns the library's, and a host checks it before it opens a session. A later revision adds
members to the JSON it writes and functions to the interface, and does not change those of a
version.

| function | does |
|---|---|
| `wai_sink_open(config_json, seed, seed_len, start_us, &out)` | opens a session (§3); `seed` null for a fresh key |
| `wai_sink_free(s)` | frees it, signing nothing |
| `wai_sink_info(s, &out)` | `{"channel", "owner", "player", "session", "start_us", "presentation", "ended"}` |
| `wai_sink_receipt_name(playlist, len, &out)` | `mr/<BLAKE3>.json`, the receipt file to fetch beside a playlist |
| `wai_sink_playlist(s, url, bytes, len, receipt, receipt_len, &out)` | §4.2; `receipt` null for none. `{"kind", "listing", "receipt", "receipt_name", "step", "need_parent", "refused", "present", "segments", "ads"}` |
| `wai_sink_playlist_parent(s, url, name, bytes, len)` | hands over the parent `need_parent` named; `bytes` null when it was not served |
| `wai_sink_object(s, meta_json, bytes, len, &out)` | §4.3; `meta_json` `{"url", "t_us", "media", "status", "cmsd_static", "cmsd_dynamic"}`. `{"sha256", "cid", "listing", "bound", "media", "key", "acknowledged", "refused", "present"}` |
| `wai_sink_event(s, name, t_us)` | §5.2 |
| `wai_sink_observe(s, obs_json, &out)` | §5.1; `{"end": null}` or `{"end": "hidden" or "idle"}` (§6.1) |
| `wai_sink_energy(s, entry_json)` | §7; `{"role", "joules_micro", "energy_provenance", "energy_uncertainty", "energy_counter_filtering", "energy_calibration_ref"}`, the class as jwp-receipts §2.4.5 writes it, or `{"role", "absent": true}` |
| `wai_sink_attach(s, name, bytes, len)` | attaches a record (stream-audit §8.3) |
| `wai_sink_take_cmcd(s, &out)` | the Event-mode CMCD batch of the log since the last call |
| `wai_sink_end(s, ended, t_us, &out)` | ends the session (§6): the bundle of stream-audit §9 |
| `wai_audit_verify_bundle(bundle, len, owner, channels_json, &out)` | stream-audit §10's outcome, as the owner's collector reaches it |
| `wai_sink_now_us()` | the host's clock, for a host with none of its own to hand |

## 10. Conformance

- A conforming sink gives every call of the corpus the answer it states, and ends every case with
  the bundle it states, byte for byte; the bundle verifies under stream-audit §10 with the outcome
  the case states.
- The corpus holds, from fixed seeds: a signed on-demand presentation played to its end; a fragment
  changed in transit; a session that requires signing given none; receipts that do not hold; a live
  chain walked through a parent, under another key, with a parent not served and one served under
  another name; WAI objects held to their cids; a live revision that rebinds an alias; a stream of
  another service, with a key, a marked ad and a CMSD note; content addressing by the owner's word,
  and names that bind nothing without it; sessions ended `hidden` and `idle`; energy entries; the
  log's order and its close; a case
  the page's integration of stream-audit replays call for call, signing the same bundle byte for
  byte; and refused calls and configurations. Every reason of §8 refuses at least one of them.
- Its second implementation, `native_sink_verify.py`, replays every case from this text, verifies
  every bundle with stream-audit's second implementation, and catches every mutation of the corpus
  and every one of its checks dropped.
- A native integration's own tests drive it through its platform's player and require that every
  object the player was handed is one the sink presented, and that the bundle verifies.
- `wai conform sink --suite native-sink` runs the corpus against a sink, each case with its answers
  withheld, and writes the run's outcome as a signed `conform-result/1` record
  ([conformance-results](conformance-results.md)). WAI's own sink is run so in CI, through the
  reference module and through the C interface of §9, and each record must be admitted
  (conformance-results §4.1).

## 11. Security and privacy considerations

- **What verification proves.** A signed playlist binds the segments it lists by URL; a name that
  states a content hash binds the bytes. A segment that is neither is what the origin served, and
  its acknowledgement says so and no more. A host that hands the player bytes a refusal withheld
  defeats the check, and nothing downstream can tell.
- **The loopback server.** A host's server for its player listens on the loopback interface alone,
  on a port the platform assigns, and answers only the paths the session named, so that another
  application cannot fetch through it what the session fetched. Where the platform refuses
  unencrypted HTTP by default, the host permits it to the loopback address alone (`127.0.0.1`), never
  to a name or another address, so the exception reaches no origin.
- **The session key** names no viewer, device or account, as stream-audit §3.2 and §16 say; a
  seed is for vectors only.
- **What the bundle tells.** The URLs a session fetched are not in it: objects are named by their
  hashes, origins by their serialization, as stream-audit §7.1 has it. The observations' cadence
  and the spans tell how the stream was watched; a host sends them to the owner's collector and to
  no one else.