open-standards/wai/extensions/platform-encode.md

# WAI Extension: Platform and Hardware Encoding

> Status: Draft. Registers the record `wai.video.encode`. A sink or a packager that needs an
> AVC (ISO/IEC 14496-10) or HEVC (ISO/IEC 23008-2) rendition encodes it with an encoder the
> device already has: the operating system's media framework and its hardware media engine,
> a GPU's encode engine through the video acceleration API or through the GPU driver's own
> encode interface, or a software AVC library the user obtains separately and names at run
> time. WAI writes no AVC or HEVC encoder. This extension fixes the shape the encoder's output
> is held to (§4), what names the encoder (§5), and the **encode record**, a signed statement
> of who made an elementary stream, from what source, with which settings and at what
> measured cost, and, where its signer decoded the stream, the quality of the pictures it
> decodes to (§6 to §8). A verifier holds the stream to its digest and its shape; it never
> re-encodes. Decoding the stream itself and holding the source, it recomputes the quality;
> pictures handed to it show only that the figures are theirs (§7). An
> encode is never a deterministic step: no tier and no `replicate` claim covers
> the encoded pictures against the source or against a base. An encode made over a package's
> pieces is framed as one floor object per piece, each naming its encoder in `derived_from`
> as [packaging](packaging.md) §7 requires in its paragraph 7.5 (§9). No new transport. The reference
> implementation is the `platform_encode` module of `wai-rs` (feature `platform_encode`), the
> `wai encode avc|hevc`, `wai encoders` and `wai encode-record verify` commands, and
> `wai-pack encode-rung` and `wai-pack verify-rung`; the corpus is
> [`encode-conformance/`](../encode-conformance/), and its second implementation is
> `encode-conformance/encode_verify.py`.
> Keywords **MUST**, **MUST NOT**, **SHOULD**, **MAY** are RFC 2119/8174.

## 1. Scope

In scope: what an encode is handed and how a ladder rung's pictures are made (§4.1, §4.2);
the shape of the elementary stream (§4.3); the encoder interfaces by role and the identity
each reports (§3, §5); the encode record and its signature (§6); its verification (§7); the
energy and the quality of an encode (§8); an encode over a package's pieces (§9); conformance
and measurement (§10).

Out of scope: choosing an encoder's coding tools, which is the encoder's; any claim about an
encoder's rate and distortion beyond the quality of one stream (§8.3), which a record states
and never compares; any encoder written for WAI. The record states what the encoder made, and a
verifier checks the bytes against it.

## 2. What WAI writes, and what it does not

**2.1** WAI writes no AVC or HEVC encoder and ships none. The reference implementation links
no encoder: it loads the operating system's media framework, the video acceleration API's
library, the GPU driver's encode library, or a software library at a path the user names, each
at run time, and a build of it runs on a host without any of them and says which is missing.

**2.2** *Patent position, by mechanism.* Both standards' essential patents are licensed for
royalties (SPEC §5, *Broadcast and streaming codecs*: `patent-pool`). An encoder an operating
system or a GPU driver provides is supplied by the device's maker with the device, under that
maker's terms. A software library the user names is obtained by the user from its distributor,
separately from WAI, under the terms that distributor states, which may cover the royalties for
some uses and not for others. This extension states which encoder made a stream (§5); it states
nothing about whether a use of it is licensed. A deployer decides that from the encoder's
supplier's terms and its own use.

**2.3** *Header syntax under the video acceleration API.* That interface leaves the parameter
sets and the slice headers to the application, and the drivers reached for this revision emit
none unless the application hands them over. The reference adapter writes them (ISO/IEC
14496-10 §7.3.2.1, §7.3.2.2, §7.3.3; ISO/IEC 23008-2 §7.3.2.1 to §7.3.2.3, §7.3.6.1) from the
values it gives the driver in its parameter buffers, and chooses each picture's QP, which it
writes in both the parameter buffer and the slice header so the two agree. No picture sample
passes through it: the driver's engine codes every slice's data.

**2.4** *Not a deterministic step.* An encoder is not a pure function of its input
([packaging](packaging.md) §11): a hardware engine's output is fixed by no specification, and
the media framework's hardware encoder was measured making other bytes from the same pictures
in each round (§10.3). So no step of a pack or a verify is an encode, no verdict depends on an
encoder's output beyond the digest a record states, and an encoded rendition promises only its
capability's decoder conformance (SPEC §7: `wai.video.avc` and `wai.video.hevc` are
`StandardDefined`, so every conforming decoder outputs the same pictures from the stream).

## 3. Encoder interfaces

An encoder is reached through one of four interfaces. Each record names the one it used in
`encoder.api`.

| `api` | what it drives | where |
|---|---|---|
| `platform` | the operating system's media framework: its hardware media engine, or its software encoder where the engine is refused | the desktop, phone and TV operating systems that ship the framework |
| `va` | the video acceleration API, whose driver drives a GPU's encode engine | Linux, through a render node |
| `gpu-driver` | the encode interface a GPU's driver exports from its own library | Linux |
| `software` | a software AVC encoder library loaded at run time from a path the user names | any Unix |

**3.1** A `va` or `gpu-driver` encode is a hardware encode. A `software` encode is not, and
codes AVC only in this revision. A `platform` encode is either, and says which (§5).

**3.2** *The phone and TV operating system's codec interface (informative).* On the mobile and
TV operating system whose media codec interface exposes hardware and software encoders by
name, an adapter would: open an encoder for the codec's MIME type by name, read the name, the
canonical name and whether the interface reports it hardware-accelerated for `encoder.name` and
`encoder.hardware`; configure the codec's flexible 4:2:0 input, the target bit rate, no
B-pictures and no periodic key frame; request a key frame for exactly the pictures the plan
names; and read the device's hardware and system-on-chip identifiers and the operating system's
release for `encoder.device` and `encoder.driver`. Its output (the codec's configuration buffer,
then one buffer per access unit) is an Annex B stream held to §4.3 like any other. This revision
builds no adapter for it.

## 4. The encode

### 4.1 The source

A source is a YUV4MPEG2 stream: 8-bit 4:2:0 (`C420`, `C420jpeg`, `C420paldv`, `C420mpeg2`, or no
`C` tag), progressive (`Ip`, `I?` or no `I` tag), each side even and at most 16384, its frame rate
`F<num>:<den>` both non-zero, and at least one frame. `XCOLORRANGE=FULL` makes it full range;
anything else is the video range. Each frame is its luma plane, then its Cb and Cr planes, each
row-major with no padding. The source's **digest** is SHA-256 (FIPS 180-4) over every frame's
three planes, one frame after another, with no frame header.

### 4.2 A ladder rung's pictures

A rung asks for a size `W × H` (each even, from 16 to 8192), a frame rate (the source's), a
target average bit rate in kilobits per second (1 to 1000000) and a range (the source's). Where
the size is the source's, the encoder is handed the source's frames. Otherwise each plane is
resampled by **`wai.resample.two-tap`**, chroma at half the luma size: for an axis of `n_in`
samples resampled to `n_out`, output sample `o` reads the source at

    p = ((2·o + 1) · n_in · 128) div n_out − 128

in 1/256ths of a sample, clamped to `0 ..= 256 · (n_in − 1)`; with `x0 = p >> 8`, `f = p & 255`
and `x1 = min(x0 + 1, n_in − 1)`, the sample is `(s[x0] · (256 − f) + s[x1] · f + 128) >> 8`.
Rows are resampled first, then columns, each pass rounding to 8 bits. The resampler is a pure
function in integers: every host hands its encoder the same samples.

The **IDR plan** is the frame indices at which the stream has an IDR access unit: sorted, the
first 0, each below the source's frame count. A package's plan is its pieces' starts (§9).

### 4.3 The stream

An encoder's output **MUST** be, and a producer **MUST** refuse to record one that is not:

1. an Annex B byte stream (ISO/IEC 14496-10 Annex B; ISO/IEC 23008-2 Annex B): it begins with
   zero bytes and a start code prefix; a NAL unit ends where the next three bytes are
   `00 00 00` or `00 00 01`; no NAL unit is empty, holds `00 00 02`, or has its forbidden bit set;
   an HEVC NAL unit's `nuh_temporal_id_plus1` is not 0; every sequence parameter set reads, an
   AVC one through its VUI (Annex E.1.1) where it has one and an HEVC one through its sub-layer
   ordering information (`sps_max_latency_increase_plus1`); and the start of every slice header
   this section reads (below) reads (`encode-stream-malformed`);
2. exactly one access unit per source frame, in presentation order (`encode-frame-count`). An
   access unit begins at the first of an access unit delimiter, a parameter set, a prefix SEI or
   the other non-VCL types the standard lists after the previous picture's slices (ISO/IEC
   14496-10 §7.4.1.2.3, its types 6 to 9 and 14 to 18; ISO/IEC 23008-2 §7.4.2.4.4, its types 32
   to 35, 39, 41 to 44 and 48 to 55), or at a slice whose `first_mb_in_slice` is 0 (AVC; the
   first syntax element of its slice header) or whose `first_slice_segment_in_pic_flag` is 1
   (HEVC; the first bit). NAL units after the last picture's slices that begin an access unit
   holding no slice are `encode-stream-malformed`, checked with item 1;
3. an access unit's slices all IDR or none (`encode-stream-malformed`); no random access point but
   an IDR (HEVC `nal_unit_type` 16 to 23 other than 19 and 20); and an IDR access unit (AVC
   `nal_unit_type` 5; HEVC 19 or 20) at exactly the plan's frames (`encode-keyframe-mismatch`);
4. every IDR access unit carrying the sequence and picture parameter sets (and the video
   parameter set for HEVC), the same bytes in the same order as the first IDR access unit's; no
   other access unit carrying parameter set bytes the first IDR's do not; no access unit carrying
   a sequence parameter set other than the stream's first; and no access unit before the first
   sequence parameter set (`encode-parameter-sets`);
5. 8-bit 4:2:0 in one colour plane; progressive, which for AVC is `frame_mbs_only_flag` 1 (this
   revision reads no HEVC field signalling); a single layer: no AVC NAL unit of type 14, 15, 20
   or 21, and HEVC `nuh_layer_id` 0 throughout; and AVC `pic_order_cnt_type` 0 or 2, the types
   this revision reads (`encode-stream-unsupported`);
6. with no reordering: HEVC `sps_max_num_reorder_pics` of the highest sub-layer 0; AVC the VUI's
   `max_num_reorder_frames` 0 where the SPS has one, and, for `pic_order_cnt_type` 0, each access
   unit's top field order count (§8.2.1.1: `PicOrderCntMsb + pic_order_cnt_lsb`, from the first
   slice's `pic_order_cnt_lsb` and the previous reference picture's, both reset to 0 at each IDR)
   above the one before it since the last IDR (`encode-stream-reordered`). The slice header is
   read only as far as `pic_order_cnt_lsb`; this revision does not read `dec_ref_pic_marking`, so
   a stream whose pictures carry `memory_management_control_operation` 5 may be refused here;
7. of the rung's size: the sequence parameter set's cropped picture is `W × H` (AVC frame
   cropping, HEVC conformance window), and where an AVC VUI signals `video_full_range_flag` it is
   the rung's range (`encode-sps-mismatch`).

Item 1 and item 2's count are checked over the whole stream first; then items 3 to 6 access
unit by access unit, in stream order, each access unit's items in that order; then item 7. The
first that fails is the reason. They hold the stream to what a piece of a package needs: each piece starts at an IDR
access unit that carries its parameter sets, so it decodes with nothing outside it, and access
unit *i* is the picture of frame *i*.

## 5. The encoder's identity

**5.1** An encode names its maker with six strings and one flag, each as the interface reports it
at run time, read as the adapter documents:

| member | meaning |
|---|---|
| `api` | the interface (§3) |
| `name` | the encoder, as the interface names it: the framework's encoder identifier and display name; the profile and entrypoint the driver offered, with the driver's vendor string; the codec and preset the GPU driver's interface names; or the library's file name with the settings it was opened with |
| `version` | the encoder's version: the framework's bundle version; the video acceleration API's version; the encode interface's version and the highest the driver supports; or the library's own version |
| `hardware` | `yes` when a fixed-function engine coded the pictures, `no` otherwise |
| `device` | the device the encoder ran on: the processor's name and model; the render node and its PCI identifiers; the GPU's name, ordinal and PCI bus |
| `driver` | the driver, or for a platform or software encoder the operating system's release and build, that ran it |
| `library_sha256` | for `software`, the SHA-256 of the file loaded; otherwise absent (`null`) |

**5.2** A producer **MUST NOT** state an identity it did not read from the interface. Where an
interface reports nothing for a member, the adapter writes what it read instead and says so in
the string; no member is empty.

**5.3** A runtime-loaded library is refused at load unless it is of the interface version the
adapter was written for: its structures may differ between versions. The record names the file
loaded by its SHA-256, so a relying party can hold it to the distributor's published digest.

## 6. The encode record

### 6.1 Members

The record is a JSON object (RFC 8259) read as [contribution](contribution.md) §2 reads a
record: strict JSON (no member named twice in one object, at most 32 levels), integers unsigned,
decimal, with no sign, fraction, exponent or leading zero and at most 2^53 − 1, no other number,
no boolean; byte strings in lowercase hexadecimal; a **term** 1 to 64 bytes of `a`–`z`, `0`–`9`,
`-`, `_` and `.`; a **date-time** RFC 3339 in UTC, `YYYY-MM-DDTHH:MM:SS`, at most nine fraction
digits, then `Z`, its day one its month has in the proleptic Gregorian calendar and its second
at most 60; a **decimal** a string of 1 to 39 decimal digits with no leading zero but `0`
itself, an integer below 2^128; a **signed decimal** a decimal, or `-` and a decimal other than
`0`, an integer from −2^32 to 2^32. It has exactly these members, at every level, none missing
and none added, but `quality`: a record with no such member, as one written before the member
was, reads as a record whose `quality` is `null` does, and neither states any figure (§8.3.2). A
record that does not hold to this, or a member of another type or encoding (any `quality` that
is neither `null` nor an object of §8.3's members among them), or outside the fixed words the
table gives `kind`, `hardware`, `range` and `tier`, is `record_malformed`. Every other set or
range of values is a §7 step's, and refused with that step's reason.

| member | type | meaning |
|---|---|---|
| `kind` | string | `"wai-encode"` |
| `capability` | string | `wai.video.avc` or `wai.video.hevc` |
| `encoder` | object | §5.1's members: `api` a term; `name`, `version`, `device`, `driver` strings of at most 1024 bytes; `hardware` `"yes"` or `"no"`; `library_sha256` 32 bytes or `null` |
| `settings` | object | `width`, `height`, `fps_num`, `fps_den`, `kbps` integers; `range` `"video"` or `"full"`; `resample` a term, `none` or `wai.resample.two-tap` (§4.2) |
| `source` | object | `frames`, `width`, `height` integers; `sha256` 32 bytes, the source's digest (§4.1) |
| `idr` | array | the IDR plan (§4.2), integers |
| `stream` | object | `sha256` 32 bytes, the stream's SHA-256; `bytes`, `units` integers; `profile_idc`, `level_idc` integers, the SPS's (`general_profile_idc` and `general_level_idc` for HEVC); `tier` `"main"` or `"high"`, HEVC's `general_tier_flag`, and `"main"` for AVC, which has none (held to the stream at §7 step 8) |
| `energy` | object or `null` | §8 |
| `wall_us` | integer | the encode's wall time, from the first picture handed over to the last byte back |
| `package` | object or `null` | §9 |
| `quality` | object or `null` | §8.3: the quality of the pictures the stream decodes to, `null` when the signer did not measure it |
| `issued_at` | date-time | when the signer signed it, self-asserted |
| `signer_pubkey` | 32 bytes | an Ed25519 key (RFC 8032) |
| `sig` | 64 bytes | §6.2 |

### 6.2 Signature and digest

```
sig    = Ed25519(signer_key, "wai:video-encode\x01" || JCS(record without "sig"))
digest = SHA-256("wai:video-encode-record\x01" || JCS(record))
```

JCS is RFC 8785. The signature is verified strictly ([SPEC](../SPEC.md) §7 *Signatures*): a key
that fails step 1 or 2 is `bad_key`, a signature that fails a later step `bad_signature`. The
record's text as the reference writes it is its JCS form. The signer is whoever ran the encode;
the record is that signer's statement of what the interface reported (§5.2).

## 7. Verification

A verifier checks, in order, and the first that fails is the reason:

1. the record reads (§6.1) (`record_malformed`);
2. its signature (§6.2) (`bad_key`, `bad_signature`);
3. `capability` is `wai.video.avc` or `wai.video.hevc` (`encode-capability`);
4. the identity: `name`, `version`, `device` and `driver` each hold a character other than
   Unicode White_Space (`encode-identity-incomplete`); `api` is one of §3's; `library_sha256` is
   present exactly for `software`; a `software` encode is `hardware` `"no"` and of
   `wai.video.avc`; a `va` or `gpu-driver` encode is `hardware` `"yes"`
   (`encode-identity-inconsistent`);
5. the settings and the source: the rung's width and height even and from 16 to 8192, the
   source's even and from 2 to 16384; both rate members non-zero and at most 2^32 − 1; `kbps` from 1 to 1000000; at least one source frame; and `resample`
   `none` exactly where the rung's size is the source's, `wai.resample.two-tap` otherwise
   (`encode-settings-invalid`);
6. the IDR plan: its first entry 0, each above the one before, the last below `source.frames`;
   and, where `package` is not `null`, its pieces' starts equal to the plan, each piece of at
   least one frame and starting where the one before it ended, their frames summing to
   `source.frames` (`encode-plan-invalid`);
7. the energy, where it is not `null` (§8.1) (`encode-energy-invalid`, `encode-energy-coverage`);
8. with the stream held: its SHA-256 and length are `stream.sha256` and `stream.bytes`
   (`encode-stream-mismatch`); it holds to §4.3 for `source.frames` pictures and the plan, its
   reasons as there; its access units number `stream.units` (`encode-frame-count`); its SPS's
   cropped size is the settings' `width × height`, its profile, level and tier `stream`'s, and an
   AVC VUI's signalled range the settings' (`encode-sps-mismatch`);
9. with the source held: it reads as §4.1 says, its digest is `source.sha256`, its width, height
   and frame count are `source`'s, its frame rate's numerator and denominator, as its header
   writes them, are the settings' `fps_num` and `fps_den`, and its range is the settings'
   (`encode-source-mismatch`);
10. with the package and the floor objects held, §9.3 (`encode-package-mismatch`);
11. with the source and pictures held, where `quality` is not `null`: the pictures are
    `source.frames` pictures of the settings' `width × height`, their SHA-256 is
    `quality.decoded_sha256`, and each of `sse_y`, `sse_cb`, `sse_cr` and `ssim_y_q32` is the
    figure §8.3 computes from them and the pictures the encoder was handed
    (`encode-quality-mismatch`).

Steps 8 to 11 are made only where the verifier holds what they read; a verdict names which it
made. Nothing in them re-encodes: the source is held to its digest, the stream to its digest
and its shape, and the pictures to their digest.

Step 11 reads pictures, and only a decode ties pictures to a stream: `quality.decoded_sha256`
is the signer's statement of what the stream decodes to, and a signer can state any figures and
supply pictures that give them. So a verdict on a record stating figures also names what they
rest on:

| verdict | when | what it says of the figures |
|---|---|---|
| `decoded` | step 11 was made over pictures the verifier decoded itself, with a conforming decoder, from the stream step 8 held | they are the stream's |
| `handed` | step 11 was made over any other pictures, whoever made them | they are those pictures', and nothing of the stream |
| `stated` | step 11 was not made | they are the signer's statement |

A relying party **MUST NOT** read a figure as the stream's on a verdict other than `decoded`. A
record stating no figures has no step 11 and no such word, whatever pictures are held.

## 8. Energy and quality

### 8.1 The figure

`energy` restates a measured figure with the members [jwp-receipts](jwp-receipts.md) §2.4.5
gives an energy claim, and three of this extension's own:

| member | type | meaning |
|---|---|---|
| `joules_micro` | integer | the figure, microjoules, for the whole encode the record names |
| `energy_provenance` | string | an acquisition class of [energy-measurement](energy-measurement.md) §2 |
| `energy_uncertainty` | object or `null` | `{ "relative_ppm", "window_us" }`, integers, as §2.3 there |
| `energy_counter_filtering` | string or `null` | an `OnChipCounter`'s filtering state |
| `energy_calibration_ref` | string or `null` | a `CalibratedInstrument`'s calibration reference |
| `coverage` | string | `Whole` or `Partial` ([energy-binding](energy-binding.md) §4) |
| `counter` | term | the counter read (§8.2) |
| `attribution` | string | `task`, `rail` or `bracket` (§8.2) |
| `fabrics` | array of strings | what the counter's reading covers, each of `cpu`, `gpu`, `media`, `npu`, at least one, in that order, none twice |

It is held to jwp-receipts §2.4.5's rules 1 to 3, read as written there: a non-zero figure; a
class of energy-measurement §2 other than `HwShunt`; the class's evidence whole, including the
resolution floor `joules_micro × relative_ppm ≥ 288 676`, an `OnChipCounter`'s filtering state
of `On`, `Off` or `Unknown`, and a `CalibratedInstrument`'s non-blank calibration reference; and
to a `coverage` of `Whole` or `Partial`, an `attribution` of the three, and `fabrics` as the
table says (`encode-energy-invalid`). A **hardware** encode's figure whose `fabrics` names
neither `gpu` nor `media` counts none of the engine that coded the pictures, so it **MUST** be
`Partial` (`encode-energy-coverage`). A relying party **MUST NOT** report a `Partial` figure as
the whole cost of the encode.

### 8.2 How the reference measures it

The figure is the median of an odd number of rounds (at least three) of the same encode, after
one unmetered round that loads and warms the encoder. A round's figure is a counter's advance
over the encode (`task`), or energy-measurement §3's marginal: the advance over the encode less
the mean power over an idle window just before it, times the encode's length (`bracket`); a
round whose marginal is not above zero, or whose counter fails a read, makes the whole figure
absent rather than estimated. The declared uncertainty is the rounds' spread read as rectangular,
`(max − min) / (2·√3)` relative to the median, rounded up, or energy-measurement §2.3's rounding
floor if that is larger; its window is the median round's. The class is `OnChipCounter` with
filtering `Unknown`, and the coverage `Partial`: no counter the reference reads sees all of a
host's energy.

| `counter` | what it reads | `attribution` | `fabrics` |
|---|---|---|---|
| `task` | the operating system's account of the encoding process's own energy, with the work other processes did for it billed to it | `task` | `cpu` |
| `rapl` | the processor package zones' energy counters, summed | `bracket` | `cpu` |
| `gpu-board` | a discrete GPU driver's total-energy counter for its device | `bracket` | `gpu`, `media` |
| `rapl-and-gpu-board` | both, summed | `bracket` | `cpu`, `gpu`, `media` |

The table is the reference's practice, not a rule a verifier applies: §8.1 is. No counter is
divided among its users by busy time. A `task` figure counts none of a hardware
engine's draw, since the operating system gives an unprivileged process no account of it; a
`bracket` figure on a host other work shares is machine-wide, and its rounds' spread says how
much that work moved it.

### 8.3 Quality

A record whose signer decoded the stream states the quality of the decoded pictures against
the pictures the encoder was handed (§4.2) in `quality`, an object of exactly:

| member | type | meaning |
|---|---|---|
| `decoded_sha256` | 32 bytes | the SHA-256 of the decoded pictures: each picture's luma plane, then its Cb and Cr planes, row-major with no padding, one picture after another in output order, as §4.1 digests a source |
| `sse_y`, `sse_cb`, `sse_cr` | decimal | Σ (aᵢ − bᵢ)² over that plane of every picture, `a` the samples the encoder was handed and `b` the decoded ones |
| `ssim_y_q32` | signed decimal | the structural similarity of the luma planes: [staged-measurement](staged-measurement.md) §4's, its windows and integers as its paragraph 4.2 gives them, with each plane's samples read as its luma, pooled over every picture as its paragraph 4.5 pools units, one mean over every window of every picture, in Q32 (2^32 is 1) |

The decoded pictures are `source.frames` pictures of `settings.width × settings.height`, 8-bit
4:2:0. The handed pictures are the source's frames where `resample` is `none`, and their
`wai.resample.two-tap` resampling to the rung's size (§4.2) otherwise.

**8.3.1** *A figure of the stream.* Every conforming decoder outputs the same pictures from the
stream (§2.4: `StandardDefined`), and the figures are integers computed from those pictures and
the handed ones alone. So the figures a stream gives are the same whichever conforming decoder
made the pictures, and a verifier that decodes the stream itself recomputes them exactly (§7, the
verdict `decoded`). A record signs them for that reason. Pictures handed to a verifier show only
that the record's figures are theirs: nothing but a decode ties them to the stream (§7, the
verdict `handed`). The time and the energy of an encode are figures of the encoder, the host,
its load and its counter, and §8.1, §8.2 and §10.3 state them as such.

**8.3.2** A record whose signer did not measure the stream's pictures, because it did not decode
the stream or its decode gave no pictures to measure, states `quality` `null`: the encode stands,
and the record says nothing of its quality. The reference's `wai_encode_measure` writes the figures
from the pictures the platform's decoder makes, or from pictures another conforming decoder made,
which it is handed (§10.3). PSNR, `10·log10(255²·N / sse)`
over a plane's `N` samples, is derived from the sums for display only, as staged-measurement
§4 derives it in its paragraph 4.7, and a sum of 0 has no finite PSNR and shows as `exact`; a
record carries the integers. The figures state the distortion of one stream against the
pictures it was made from; this extension compares no figure with another.

**8.3.3** A record given figures after it was signed is a new statement: its signer signs it
again, and its `issued_at` is when (§6.1). Another key does not add to a record: the record is its
signer's (§6.2).

## 9. An encode over a package's pieces

### 9.1 The rendition

An **encoded rendition** of a package ([packaging](packaging.md) §11) is one encode of a video
component's source whose IDR plan is the component's pieces' starts, as its stage index lists
them (§6.2 there), and whose rung is named by a term, its **rung id**. Each piece is framed as
one floor object (packaging §5.2, §7.2): a `WAI1` envelope (SPEC §2) whose manifest is the JSON
text, with no whitespace and its members in this order,

    {"wai":"1.0","media":"video","intent":"replicate","model_requirement":{"capability":C},
     "conditioning":{"kind":K},"derived_from":{"motif":M,"motif_capability":MC,
     "renderer":R,"settings":D},"target":{"frames":F,"h":H,"w":W}}

with `C` the capability, `K` `avc` or `hevc`, `F` the piece's frames, `H` and `W` the rung's
height and width, each string written as JCS writes it (RFC 8785 §3.2.2.2), and the
`derived_from` members:

- `motif`: the piece's base object's content hash, the stage index's entry of role base and
  layer 0 for that piece, as 64 lowercase hexadecimal digits; `motif_capability`: its capability;
- `renderer`: `<encoder.name>@<encoder.version>`;
- `settings`: `BLAKE3("wai:render-settings\x01" ‖ JCS(S))`, with `S` the object
  `{"source": "encoded", "api", "hardware", "device", "driver", "library_sha256",
  "stream_sha256"}`, each member but `source` the record's `encoder` member of that name, and
  `stream_sha256` the record's `stream.sha256`; the digest written as 64 lowercase hexadecimal
  digits.

So each object names the encoder and its version in `renderer` (packaging §7.5) and binds its
device and driver through `settings`, which any verifier holding the record recomputes. Its
payload is its piece's access units as NAL units each after the four bytes `00 00 00 01`, in the
order the stream holds them: the stream's own NAL units, re-framed with one start code length,
beginning with the piece's IDR access unit and its parameter sets.

### 9.2 The record's `package`

| member | type | meaning |
|---|---|---|
| `receipt_sha256` | 32 bytes | the SHA-256 of the package receipt's file (packaging §9) |
| `rung` | term | the rung id |
| `pieces` | array | one object per piece, in order: `start` and `frames` integers, the piece's first frame and frame count; `motif` 32 bytes, the base it presents; `object` 32 bytes, its floor object's content hash ([jwp-receipts](jwp-receipts.md) §2.1) |

The objects are written `objects/<object>.wai`, the stream `<rung id>.h264` or `<rung id>.h265`,
and the record `<rung id>.encode.json`, beside the package and not inside it: the package
receipt does not list them, and a package made before them is unchanged.

### 9.3 Verification against the package

With the package receipt's file, the component's stage index and the floor objects held, a
verifier checks, in order, every failure `encode-package-mismatch`: the receipt file's SHA-256
is `receipt_sha256`; the stage index's pieces, in order, have the record's starts and frame
counts (a video component's timebase unit is one frame, packaging §2), and each piece's one
entry of role base and layer 0 has the record's `motif`; and each piece's object is present,
its content hash is `object`, it is a `WAI1` envelope with exactly §9.1's manifest for this
record and piece, and its payload is §9.1's bytes from the held stream.

A sink that decodes an encoded rendition decodes each object's payload as its capability's
stream; the record tells it who made the bytes, and nothing more.

## 10. Conformance and measurement

**10.1** *The corpus.* [`encode-conformance/`](../encode-conformance/) holds small elementary
streams made by encoders through each interface (fixtures, pinned by their SHA-256 in the
corpus's manifest and never regenerated, since no encoder is a pure function), the source they
were made from, the pictures a conforming decoder made from one of them (a fixture too, pinned
in `decoded.json`), and records generated reproducibly over them from a fixed-seed key with a
fixed `issued_at`: accepted cases, among them a record with `quality` over a resampled rung, one
stating `null`, and records with no such member, and negative controls each refused with its
reason, among them that record with each figure one away from the pictures' and with another's
pictures; and an encoded rendition of a packaging golden with its objects, accepted and mutated. A
case with pictures states how the verifier came by them, `decoded` or `handed`, and an accepted
record stating figures states §7's verdict. The
pinned pictures stand in for a verifier's own decode in a case of `decoded`, since §10.2 holds
them to what a conforming decoder makes of the stream, and a verifier with a decoder may decode
the stream instead. The corpus records what pictures handed over cannot show: a record stating
the perfect score of the encoder's own input, which its stream does not give, is accepted as
`handed` with that input (`handed/`, generated) and refused as `decoded`. `wai-rs`
`tests/platform_encode.rs` requires the generator to reproduce the corpus byte for byte and every
case to get its outcome and its verdict; `wai-pack` `tests/encode_rung.rs` does the same for the
rendition. The second implementation, `encode-conformance/encode_verify.py`, is written from this
text: it shares no code with `wai-rs` or `wai-pack`, reads the Annex B stream, its access units
and its parameter sets itself, decodes nothing, resamples the source and recomputes every quality
figure itself, verifies signatures with WAI's strict Ed25519 (`claim-conformance/_ed25519.py`) and
content hashes with BLAKE3 (`claim-conformance/_blake3.py`), and mutates copies of the corpus and
requires every copy to fail.

**10.2** *Decoder conformance.* Each corpus stream, and each stream the reference encoders made
for §10.3, decodes in two independent conforming decoders, a hardware decoder and a software
one, to the same pictures byte for byte, as `StandardDefined` requires.

**10.3** *Measurement.* `wai_encode_measure` encodes a source through one interface at a rung,
rounds as §8.2, and decodes the result with the platform's decoder, or reads the pictures
another conforming decoder made, to write the record's `quality` (§8.3), from which it prints
PSNR for display (luma, each chroma plane, and the 6:1:1 weighted mean of their squared
errors) and which pictures it read. With no pictures to measure it writes the record with
`quality` `null` and says why (§8.3.2). Its time and energy figures are informative: each is of
the encoder, the host, its load and its counter at the time it was taken, and none is compared
with a figure taken under another counter.

## 11. Relationship to other extensions

- [packaging](packaging.md) §7, in its paragraph 7.5, requires a floor made by an encoder that is not a pure function
  of its input to name the encoder and its version; §9 here is how an AVC or HEVC rendition over
  a package's pieces does it. Packaging §7.1's floor remains a capability of SPEC §5's mandatory
  floor or recommended set: an encoded rendition is beside it, not in its place.
- [energy-measurement](energy-measurement.md) gives the acquisition classes, the declared
  uncertainty and the bracketed marginal (§2, §3); [jwp-receipts](jwp-receipts.md) §2.4.5 gives
  the members and rules §8.1 restates.
- SPEC §5, *Broadcast and streaming codecs*, registers `wai.video.avc` and `wai.video.hevc`, their
  payload (the Annex B byte stream) and their licence class.

## 12. Reasons

| reason | step | meaning |
|---|---|---|
| `record_malformed` | §7 step 1 | the record does not read as §6.1 says |
| `bad_key` | step 2 | the signer's key fails SPEC §7's strict rule |
| `bad_signature` | step 2 | the signature fails it |
| `encode-capability` | step 3 | not `wai.video.avc` or `wai.video.hevc` |
| `encode-identity-incomplete` | step 4 | an identity member is blank |
| `encode-identity-inconsistent` | step 4 | the interface, `hardware`, `library_sha256` and capability disagree |
| `encode-settings-invalid` | step 5 | a size, rate, bit rate, frame count or `resample` out of range or inconsistent |
| `encode-plan-invalid` | step 6 | the IDR plan, or the package's pieces, out of order or out of range |
| `encode-energy-invalid` | step 7 | the figure's members break §8.1 |
| `encode-energy-coverage` | step 7 | a hardware encode's figure that counts no engine is stated `Whole` |
| `encode-stream-mismatch` | step 8 | the stream's SHA-256 or length is not the record's |
| `encode-stream-malformed` | §4.3 items 1 to 3 | not an Annex B stream, NAL units after the last picture that hold no slice, or an access unit mixing IDR and other slices |
| `encode-frame-count` | §4.3 item 2, step 8 | another number of access units |
| `encode-keyframe-mismatch` | §4.3 item 3 | an IDR off the plan, a planned picture not an IDR, or another random access point |
| `encode-parameter-sets` | §4.3 item 4 | parameter sets missing or changed |
| `encode-stream-unsupported` | §4.3 item 5 | not 8-bit 4:2:0 progressive single-layer, or a picture order type this revision reads not |
| `encode-stream-reordered` | §4.3 item 6 | the stream reorders pictures |
| `encode-sps-mismatch` | §4.3 item 7, step 8 | the coded size, profile, level, tier or range is not the record's |
| `encode-source-mismatch` | step 9 | the source's digest is not the record's |
| `encode-package-mismatch` | step 10 | the rendition does not match the package, the stream or the record |
| `encode-quality-mismatch` | step 11 | the pictures held are not the record's, or a quality figure is not the one they give |
| `encoder-unavailable` | producer | no encoder of the interface for the codec here (not a record reason) |
| `encoder-refused`, `encoder-failed` | producer | the interface refused the settings, or failed an encode (not record reasons) |

## 13. Security considerations

- **A record is its signer's statement.** It names what the interface reported; a signer can
  name an encoder that did not run. The stream is checked, not the claim of who made it: a
  relying party that must know the maker trusts the signer, or runs the encode itself.
- **A runtime-loaded library is code the user chose.** The adapter refuses a library of another
  interface version and states the digest of the file it loaded; it does not check the file
  against any list. A deployer that must hold the library to its distributor's published digest
  compares `library_sha256` with it.
- **Quality is the signer's statement until a verifier decodes the stream.** A verifier that
  decodes the stream it holds with a conforming decoder recomputes every figure from what it
  decoded and the source, and its verdict is `decoded` (§7). Pictures handed to it, whose digest
  is `decoded_sha256` and which give every figure, show only that the figures are theirs: a
  signer can state any figures and supply such pictures, so that verdict is `handed`, and of the
  stream the figures stay their signer's.
- **A stream is checked before it is recorded.** §4.3 is applied by the producer before it signs,
  and again by every verifier holding the stream, so a record never names a stream whose pieces
  do not each start at a self-contained IDR access unit.