open-standards/wai/extensions/direction.md

# WAI Extension: Direction Records

> Status: Draft. Registers the record `wai.media.direction`: a person's direction of generated
> or edited video, as signed evidence bound to the output. A director, a person named by an
> Ed25519 key (as a `did:key`, the same key), signs **direction records** at four **levels** of
> control: a **storyboard** fixes the shots, each with its duration, its subjects by reference
> picture and its framing (the **shot** level), and may fix their order (the **sequence**
> level); **keyframes** fix, at stated times of a shot, a subject's box on screen, its size
> class, its screen position and its angle, relative to the subject, and a camera window, with a
> pinned interpolation between keys (the **frame** level), and **regions** hold samples of
> stated frames to a reference picture (the **pixel** level); a **sequence** fixes the edit:
> which shots, which part of each, in which order, cut or dissolved (the sequence level). Each
> record states the levels it sets. A tool that makes a video under direction signs a
> **directed output** record naming the direction records, the layout of every frame of every
> shot, the timeline of the edit and the output's frames by digest. Where the output was made
> by the pinned render of this extension from layers that `wai.generate.spectral` regenerates
> or that the verifier holds, every frame recomputes, and the checks are **exact**: each cut
> falls at its directed frame, each duration and the order are the directed ones, each subject
> is where the keyframes and their interpolation put it under a pinned integer **framing
> measure**, and each held region holds. Where a model made the frames in floating point, the
> same checks run on the layout the tool states, and the outcome is **attested** with the class
> of what stated it, never exact. Every directed element **governs** the output frames that show
> what it directed, and the record states them: the per-frame map shows which person directed
> each frame's framing, timing, held samples and place in the sequence, whoever made its
> samples, and each element maps onto a `direct` step of the contribution record and onto a
> C2PA action. WAI decides no legal question. No new transport. The reference implementation is
> the `direction` module of `wai-rs` (feature `direction`); the corpus is
> [`live-conformance/direction/`](../live-conformance/direction/), and its second
> implementation is `live-conformance/direction_verify.py`.
> Keywords **MUST**, **MUST NOT**, **SHOULD**, **MAY** are RFC 2119/8174.
>
> Texts read for this revision, each pinned to the version named: the C2PA Technical
> Specification, version 2.4, §6.4, §18.2 (§18.2.2.1, §18.2.2.4 and §18.2.3 among it), §18.15.1,
> §18.15.2, §18.15.4.1 and §18.15.4.6; the Creator Assertions Working Group's Identity
> Assertion, version 1.2, §5.1, §5.1.2 and §6.1; the serialized schemas of OpenTimelineIO
> release 0.18.1 (`Timeline.1`, `Stack.1`, `Track.1`, `Clip.2`, `Transition.1`,
> `RationalTime.1`, `TimeRange.1`) and its transition types; the abstract of arXiv 2606.01900;
> and *Copyright and Artificial Intelligence, Part 2: Copyrightability*, a report of the United
> States Copyright Office (January 2025), its Part II.F and its Part V. A section number of one
> of those texts in this extension is that version's.

## 1. Scope

### 1.1 Why

Direction of generated video works at four levels: the pixel, the frame, the shot and the
sequence. A director decides what each shot shows and how it is framed, where a subject sits on
screen and how large, how the camera moves, which samples stay fixed, where each cut falls and
in which order the shots run. arXiv 2606.01900 states framing as functions of the subject's pose
and motion (shot size, angle and composition) and directs a virtual camera with sparse keyframes
deterministically interpolated into a trajectory. A generator then makes the samples.

The cited report concludes that outputs made with generative AI may be protected in whole or in
part where a person was able to determine the expressive elements they contain, that prompts
alone are unlikely to suffice at this stage, and that the questions are decided case by case
(its Part V); and that a person may select, coordinate and arrange generated material, or modify
it, in a way that contributes authorship (its Part II.F). Whether a director's choices are such
a contribution is a question for a registration authority or a court, under its own rule.

This extension makes the director's choices signed facts, bound to the output they governed, and
checks them: exactly, frame by frame, where the output recomputes; as the tool's statement, with
its class, where it does not. It states, for every frame, who directed what. It does not say what
any of that makes anyone.

### 1.2 In scope and out of scope

In scope: the records and their encodings (§2); the levels and the elements of direction (§3);
the storyboard (§4); keyframes, their interpolation and held regions (§5); the sequence and the
timeline arithmetic (§6); the framing measure (§7); the directed output (§8); the render (§9);
governance, the per-frame map and the contribution record's `direct` act (§10); the mappings
onto C2PA actions, identity assertions and an open timeline interchange format (§11);
verification (§12).

Out of scope:

- **Legal questions.** WAI decides none. A record states which person directed which frames at
  which level, and how much of that a verifier checked; a reader applies its own rule to those
  facts. Registering a work, and contacting any authority about it, are its owner's decisions.
- **How a generator follows direction.** A model conditioned on a storyboard, a keyframe or a
  pose is the tool's; this extension checks what came out.
- **Measuring a subject in arbitrary pixels.** Where the subject is a known layer, the render of
  §9 places it and its box is exact. A detector or tracker that finds a subject in a generated
  frame is a method of the tool's, and what it states is attested with its class (§8.1).
- **Audio** and its direction, and direction of a live room's composition, which
  [live-compose](live-compose.md) lays out from the room's events.
- **Camera paths in world space.** The camera of this revision is a window over a plate (§5.1);
  a path through a scene is a later revision's.

### 1.3 Relationships

Relationship to the core spec: `wai.media.direction` is a **record** ([SPEC](../SPEC.md) §5):
signed records a WAI tool emits and verifies. It names no payload format and is not
dispatchable.

Relationship to `wai.generate.spectral`: a layer of a directed output may be a picture that
generator makes; a verifier holding the model file regenerates it (SPEC §5, *Payload of
`wai.generate.spectral`*).

Relationship to [deriving-edge](deriving-edge.md): the render of §9 places layers as that
extension's `wai.derive.overlay_composition` does, opaquely at integer offsets, and resamples by
the nearest sample, the reference rule of its `wai.derive.scaling` (§4.2 there), which this
extension pins for its own render.

Relationship to [synthesis](synthesis.md): a directed output's origin claim is that extension's;
direction changes no class. A video whose layers or frames a model generated is synthesized,
however closely a person directed it.

Relationship to the contribution record, `wai.media.contribution`: that record reserves the act
`direct` for this extension. §10.3 defines the step, its parameters and its dependency rule.

Relationship to the exploration record, `wai.media.exploration`: a subject's view, a shot's
reference or a plate may be a picture an exploration archive holds, chosen by a pick; this
extension names it by its samples digest, and the archive and the pick are that record's.

## 2. Records

Every record of this extension is a JSON object (RFC 8259) under the encodings of
[jwp-receipts](jwp-receipts.md) §2.4.3, read as synthesis §2 reads a record: strict JSON, no
member named twice, at most 32 levels of nesting, byte strings in lowercase hexadecimal,
integers unsigned and at most 2^53 − 1, no other number, no boolean, **terms** and
**date-times** as defined there. Each record has exactly the members its section lists, no
member missing and none added, at any level, and a `kind` naming it; one that does not is
`record_malformed`. So is a member outside the values or ranges its section gives, or one that
breaks a rule its section states of the record alone.

Every record carries `signer_pubkey` (32 bytes, an Ed25519 key of RFC 8032) and `sig` (64
bytes):

```
sig       = Ed25519(signer_key, "wai:direction\x01" || JCS(record without "sig"))
link_hash = BLAKE3("wai:direction-link\x01" || JCS(record))
```

JCS is RFC 8785. The signature is verified strictly ([SPEC](../SPEC.md) §7 *Signatures*): a key
that fails step 1 or 2 is `bad_key`, and a signature that fails a later step `bad_signature`. A
record is named by its link hash wherever another names it. `issued_at` holds the instant its
signer signed it, self-asserted.

Values used throughout:

- A **signed integer** is a string: `0`, or an optional `-` followed by one to seven of the
  digits `0`–`9` of which the first is not `0` (no `+`, no space, no `-0`), with a value from
  −1 048 576 to 1 048 576. The encodings admit no negative number, and a subject's box may
  begin left of or above the frame.
- A **box** is `{"x", "y", "w", "h"}`: `x` and `y` signed integers, the box's left and top edge
  in pixels from the frame's top-left corner; `w` and `h` integers from 1 to 1 048 576. It may
  extend past any edge of the frame.
- A **rect** is `{"x", "y", "w", "h"}`, four integers, `w` and `h` at least 1: a rectangle of
  pixels within a picture, `x` + `w` at most its width and `y` + `h` at most its height, checked
  where the picture is known (§8.5, §12.2 step 2).
- A **picture** is `width` × `height` pixels as the `RGB8` buffer of
  [reconstruction-binding](reconstruction-binding.md) §2 (row-major, three bytes a pixel, no row
  padding), each side from 1 to 4096 and `width` · `height` at most 2^23, the sizes
  `wai.generate.spectral` makes. Its **samples digest** is SHA-256 (FIPS 180-4) over the
  buffer. A **picture reference** is `{"samples", "width", "height"}`: a samples digest and the
  size it is of. A pixel is addressed (*x*, *y*), *x* to the right and *y* down from the
  top-left.
- A **frame range** is a pair `[a, b]` of integers, *a* < *b*: the frames *a* to *b* − 1. A
  **frame set** is an array of frame ranges in ascending order, each beginning after the one
  before it ends and not where it ends (`b` of one below `a` of the next): its one encoding.
- ⌊*a* / *b*⌋, for *b* above 0, rounds toward negative infinity. Every quantity of this
  extension is an integer; none needs more than 127 bits with its sign.

## 3. Levels and elements

A **level** is one of `pixel`, `frame`, `shot` and `sequence`, in that order. Each direction
record has a member `levels`: the levels it sets, in that order, none twice. A record whose
`levels` is not exactly the levels its members set (§4, §5, §6) is `record_malformed`.

An **element** is one directed choice of a record, named by a **path**:

| level | element | path | what it fixes | its record |
|---|---|---|---|---|
| `shot` | a shot | `["shot", i]` | the shot's duration, its subjects and the framing it opens on | storyboard (§4) |
| `frame` | a shot's keys | `["keyframes", e]` | each subject's box and classes, and the camera, at each key and between keys | keyframes (§5) |
| `pixel` | a held region | `["region", j]` | the samples inside rects of stated frames | keyframes (§5.3) |
| `sequence` | the order and cuts | `["sequence"]` | which shots in which order, which part of each, each cut and dissolve | storyboard (§4) or sequence (§6) |

*i* is a shot's position in the storyboard, *e* an entry's and *j* a region's position in their
keyframes record. A record's **elements** are, in this order: a storyboard's shots by position,
then `["sequence"]` where it sets that level; a keyframes record's entries by position, then its
regions by position; a sequence record's `["sequence"]`.

## 4. The storyboard

A director signs a **storyboard**, `kind` `"wai-storyboard"`:

| member | type | meaning |
|---|---|---|
| `kind` | string | `"wai-storyboard"` |
| `levels` | array | `["shot"]`, or `["shot", "sequence"]` when `order` is not `null` |
| `width`, `height` | integer | every frame's size: a picture's sides (§2) |
| `rate` | object | `{"num", "den"}`, each from 1 to 1 048 576: frames per second, as *num* / *den* |
| `subjects` | array | 0 to 16 subjects (§4.1) |
| `shots` | array | 1 to 256 shots (§4.2) |
| `order` | array or `null` | 1 to 1024 shot positions, in the order the shots run, each a shot whole, joined by cuts; `null` where the storyboard does not fix an order |
| `issued_at` | date-time | when the director signed it |
| `signer_pubkey`, `sig` | | §2: the director's key |

### 4.1 Subjects

A **subject** is `{"views"}`: 1 to 8 **views**, each `{"angle", "samples", "width", "height"}`,
a term naming the angle the subject is seen from and a picture reference: the subject's
reference picture as seen from that angle, its whole extent edge to edge. No angle appears twice
in one subject. Angles are the director's terms; `front`, `three-quarter`, `profile`, `back`,
`high` and `low` are examples, not a list. A subject is named by its position.

### 4.2 Shots

A **shot** is `{"duration", "subjects", "framing", "reference"}`:

- `duration`: its number of frames, from 1 to 65 536;
- `subjects`: 0 to 16 subject positions, none twice, in drawing order, the first drawn first;
- `framing`: one entry per subject of `subjects`, in its order: `{"size", "position",
  "angle"}`, each `null` or a class: `size` and `position` the classes of §7, `angle` an angle
  of that subject's views. It is the framing the shot opens on: the measure of its first frame
  (§7);
- `reference`: a picture reference (§2), a reference picture for the shot, or `null`.

A position in `subjects` or `order` that is not a subject's or a shot's, or an `angle` that is
not one of the subject's views', is `record_malformed`.

## 5. Keyframes

A director signs a **keyframes** record, `kind` `"wai-keyframes"`:

| member | type | meaning |
|---|---|---|
| `kind` | string | `"wai-keyframes"` |
| `storyboard` | 32 B hex | the link hash of the storyboard it directs |
| `levels` | array | `["frame"]` when only `keyframes` is not `null`, `["pixel"]` when only `regions` is not, `["pixel", "frame"]` when neither is `null` |
| `keyframes` | array or `null` | 1 to 256 entries, each `{"shot", "keys"}`: a shot's position and 1 to 1024 keys (§5.1); no shot twice |
| `regions` | array or `null` | 1 to 256 held regions (§5.3) |
| `issued_at` | date-time | when the director signed it |
| `signer_pubkey`, `sig` | | §2: the director's key |

A record whose `keyframes` and `regions` are both `null` is `record_malformed`.

### 5.1 Keys

A **key** is `{"t", "interp", "camera", "subjects"}`:

- `t`: the frame of the shot it is at, from 0; the keys of an entry in strictly ascending `t`;
- `interp`: `hold`, `linear` or `ease`: how its values move to the next key's (§5.2);
- `camera`: `null`, or a rect: the **camera window**, the part of the shot's plate the frame
  shows, scaled to the frame (§9.1);
- `subjects`: one entry per subject of the shot, in the shot's order, each `{"box", "size",
  "position", "angle"}`: `box` `null` or a box (§2), the subject's whole extent on screen, actor
  relative: where the subject is and how large; `size`, `position` and `angle` each `null` or a
  class (§7).

Within an entry, every key's `subjects` has one length, and for each subject either every key
gives a box or none does; likewise the camera. Otherwise `record_malformed`.

A subject's boxes, and the cameras, are **tracks**. A track begins at the entry's first key.
The classes of a key are checked at its `t` alone.

### 5.2 Interpolation

A track's value at frame *t* of the shot, from the first key's `t` to the shot's last frame, is
computed member by member (`x`, `y`, `w`, `h`): for *t* from a key *k* (at *t*ₖ, value *v*ₖ)
up to the next key (at *t*ₖ₊₁, value *v*ₖ₊₁), with *n* = *t* − *t*ₖ and *D* = *t*ₖ₊₁ −
*t*ₖ, by key *k*'s `interp`:

| `interp` | value |
|---|---|
| `hold` | *v*ₖ |
| `linear` | *v*ₖ + ⌊(*v*ₖ₊₁ − *v*ₖ) · *n* / *D*⌋ |
| `ease` | *v*ₖ + ⌊(*v*ₖ₊₁ − *v*ₖ) · *n*² · (3*D* − 2*n*) / *D*³⌋ |

From the last key to the shot's end the value is the last key's. `ease` is the cubic that
starts and ends with zero speed, *s*²(3 − 2*s*) of *s* = *n* / *D*, in integers. A value lies
between its two keys' values, so `w` and `h` stay at least 1 and a camera window between two
windows inside the plate stays inside it.

### 5.3 Held regions

A **held region** is `{"shot", "from", "to", "rects", "source"}`: a shot's position; its frames
`from` up to, not including, `to` (`from` < `to`); 1 to 64 rects within the frame; and a picture
reference of the frame's size, the **source**. In those frames, every pixel inside a rect is the
source's pixel at the same position. A region holds a reference picture, a drawn matte, a fixed
element or a locked background to exact samples, whatever made the rest of the frame.

## 6. The sequence

A director (an editor directs the sequence) signs a **sequence**, `kind` `"wai-sequence"`:

| member | type | meaning |
|---|---|---|
| `kind` | string | `"wai-sequence"` |
| `storyboard` | 32 B hex | the link hash of the storyboard it edits |
| `levels` | array | `["sequence"]` |
| `events` | array | 1 to 1024 events (§6.1) |
| `issued_at` | date-time | when the director signed it |
| `signer_pubkey`, `sig` | | §2: the director's key |

### 6.1 Events

An **event** is `{"shot", "in", "out", "transition"}`: a shot's position; the frames of the shot
it shows, `in` up to, not including, `out` (`in` < `out` ≤ 65 536), its **length** *L* = `out`
− `in`; and the transition into it: `{"kind": "cut"}`, or `{"kind": "dissolve", "frames"}`, a
dissolve over `frames` ≥ 1 frames. With *d*ᵢ the dissolve frames into event *i* (0 for a cut):
the first event's transition is a cut, and *d*ᵢ + *d*ᵢ₊₁ ≤ *L*ᵢ for every event *i* (with
*d* = 0 past the last), so no frame shows more than two events. Otherwise `record_malformed`.

A storyboard's `order` is the events `{"shot": i, "in": 0, "out": duration of i, "transition":
{"kind": "cut"}}`, one for each position *i* of `order`, in its order.

### 6.2 The timeline

Events run in order. Event *i* **starts** at output frame *S*ᵢ: *S*₀ = 0 and *S*ᵢ = *S*ᵢ₋₁ +
*L*ᵢ₋₁ − *d*ᵢ, and occupies the output frames *S*ᵢ up to, not including, *S*ᵢ + *L*ᵢ. The
output has *N* = *S*ₘ₋₁ + *L*ₘ₋₁ frames, *m* the number of events. A dissolve into event *i*
overlaps the last *d*ᵢ frames of event *i* − 1 with the first *d*ᵢ of event *i*.

Output frame *f* **shows** frame `in` + (*f* − *S*ᵢ) of event *i*'s shot for each event *i* that
occupies it: one event, or two in a dissolve. The **cuts** are *S*₁, …, *S*ₘ₋₁: the frames where
each event after the first begins to show, cut or dissolved.

## 7. The framing measure

The **framing measure** of a subject in a frame of `width` *W* and `height` *H* is computed from
its box (*x*, *y*, *w*, *h*) and the angle of the view it is drawn from:

- **size**: with *q* = ⌊1000 · *h* / *H*⌋, the subject's height in thousandths of the frame's:

  | class | *q* |
  |---|---|
  | `extreme_wide` | below 250 |
  | `wide` | 250 to 699 |
  | `full` | 700 to 1099 |
  | `medium_wide` | 1100 to 1499 |
  | `medium` | 1500 to 2199 |
  | `medium_close` | 2200 to 3199 |
  | `close` | 3200 to 4999 |
  | `extreme_close` | 5000 or more |

  The box is the subject's whole extent, so a close shot's box runs far past the frame. The
  bounds are this measure's: a director's vocabulary maps onto them, and they define no
  craft's terms.
- **position**: the cell of a three-by-three grid holding the box's centre: column *c* = min(2,
  max(0, ⌊3 · (2*x* + *w*) / (2*W*)⌋)) and row *r* = min(2, max(0, ⌊3 · (2*y* + *h*) /
  (2*H*)⌋)), named `R-C` with *R* `top`, `middle` or `bottom` for *r* 0, 1, 2 and *C* `left`,
  `centre` or `right` for *c* 0, 1, 2: `top-left` … `middle-centre` … `bottom-right`.
- **angle**: the angle of the view.

A directed class **matches** when the measure gives it. A directed box matches when it equals the
subject's box member for member; a directed camera window when it equals the frame's window.

## 8. The directed output

### 8.1 The record

The tool that made the output signs the **directed output**, `kind` `"wai-directed"`:

| member | type | meaning |
|---|---|---|
| `kind` | string | `"wai-directed"` |
| `direction` | array | 1 to 64 link hashes of the direction records it was made under, the storyboard first; none twice |
| `directors` | array | 1 to 64 keys (32 B hex), the people who directed it; none twice, none the tool's key |
| `check` | string | `replicate` or `attest` |
| `attestation` | object or `null` | `null` for `replicate`; for `attest`, `{"generator", "class"}`: a term naming what made the frames, and a term naming the class of method that stated the layout (a tracker, a detector, the generator's own conditioning) |
| `layers` | array | 0 to 256 layers (§8.2) |
| `clips` | array | 1 to 256 clips (§8.3) |
| `timeline` | array | 1 to 1024 events (§6.1): the edit as made |
| `output` | object | `{"width", "height", "frames", "digest", "content_hash"}`: the output's frame size, its number of frames (1 to 2^20), the digest of its frames (§8.4), and the content hash (jwp-receipts §2.1) of the object it was published as, or `null` |
| `governs` | array | 0 to 1024 entries (§10.1) |
| `issued_at` | date-time | when the tool signed it |
| `signer_pubkey`, `sig` | | §2: the **tool's key** |

`replicate` states that every frame is the render of §9 over the layers; `attest` that something
else made the frames and the layout is the tool's statement, of the class it names. The record
reads only if its timeline does (§6.1).

### 8.2 Layers

A **layer** is a picture reference with its source, `{"samples", "width", "height",
"source"}`; no samples digest twice. Its `source` is one of:

- `{"generator", "model", "prompt", "seed", "steps"}` with `generator`
  `"wai.generate.spectral"`: the picture that generator makes with the model file whose SHA-256
  is `model` (32 B hex), the prompt (0 to 256 bytes, each from 0x20 to 0x7E other than `"` and
  `\`), the seed (0 to 2^32 − 1) and the steps (1 to 64), at the layer's size: SPEC §5,
  *Payload of `wai.generate.spectral`*, `generate(model, prompt, seed, steps, height, width)`;
- `{"generator", "model"}`, `generator` any other term: a picture another generator made, with
  the model whose digest is `model`;
- `null`: a picture given as it is, such as a person's drawing.

A layer is named by its samples digest wherever a clip or a direction record names a picture.

### 8.3 Clips

A **clip** is the layout of every frame of one shot as made, `{"frames", "plate", "camera",
"subjects", "holds"}`; the clips are the storyboard's shots, in order:

- `frames`: its number of frames, from 1 to 65 536;
- `plate`: the samples digest of the picture the frames are drawn over, or `null` (§8.5);
- `camera`: `null`, or one rect per frame: the camera window over the plate;
- `subjects`: 0 to 16 entries, in drawing order, each `{"subject", "views", "boxes"}`: a
  subject's position; one view position per frame (the view drawn); and one box per frame;
- `holds`: 0 to 64 entries `{"from", "to", "rects", "source"}`: frames `from` up to, not
  including, `to` (`from` < `to` ≤ `frames`), 1 to 64 rects and a samples digest: the regions
  held, as made.

A `camera`, `views` or `boxes` whose length is not `frames` is `record_malformed`.

### 8.4 The frames' digest

```
digest = BLAKE3("wai:direction-frames\x01" || D_0 || D_1 || … || D_{N−1})
```

with *D*ₖ the samples digest of output frame *k*, `width` × `height` `RGB8` (§2), in order. A
frame is named by its samples digest, as a frame of a `wai.video.int_motion` clip is checked by
its state.

### 8.5 Layout rules

Against the storyboard, a directed output whose layout breaks one of these is `layout_invalid`,
or `layer_missing` where a picture it needs is not one of its layers:

1. There is one clip per shot of the storyboard.
2. Each clip's subjects are subjects of the storyboard, and each view position is one of that
   subject's views.
3. For `replicate`: each clip's `plate` is not `null`, and is a layer (`layer_missing`); with no
   camera, the plate
   is the frame's size; each camera window is a rect within the plate; each view drawn is a
   layer of the size the storyboard states for that view (`layer_missing` where it is not a
   layer). For `attest`: each `plate` is `null`.
4. Each hold's rects are within the frame; for `replicate` its source is a layer
   (`layer_missing`) of the frame's size.
5. Each timeline event names a shot of the storyboard and `out` is at most its clip's
   `frames`.
6. The output's `width` and `height` are the storyboard's, and its `frames` is the timeline's
   *N* (§6.2).

## 9. The render

The render is pinned: every byte of every frame it makes is a pure function of the layers and
the record, integer throughout.

### 9.1 A clip's frame

Frame *t* of a clip, `width` *W* by `height` *H*, is drawn in this order:

1. **The plate.** With no camera, out(*X*, *Y*) = plate(*X*, *Y*). With the camera window
   (*cx*, *cy*, *cw*, *ch*) of frame *t*, out(*X*, *Y*) = plate(*cx* + ⌊*X* · *cw* / *W*⌋, *cy* +
   ⌊*Y* · *ch* / *H*⌋): the window resampled to the frame by the nearest sample.
2. **The subjects**, in the clip's order. With the box (*bx*, *by*, *bw*, *bh*) of frame *t* and
   the view *V* of frame *t*, *vw* × *vh*: every pixel (*X*, *Y*) of the frame with *bx* ≤ *X*
   < *bx* + *bw* and *by* ≤ *Y* < *by* + *bh* is *V*(⌊(*X* − *bx*) · *vw* / *bw*⌋, ⌊(*Y* −
   *by*) · *vh* / *bh*⌋): the view scaled to its box by the nearest sample and drawn opaquely,
   the part off the frame dropped.
3. **The holds**, in order: for each hold whose frames include *t*, every pixel inside each of
   its rects is the source's pixel at the same position.

### 9.2 An output frame

Output frame *f* that shows one event is that event's clip frame (§6.2). One that shows events
*i* − 1 and *i*, in a dissolve of *d* = *d*ᵢ frames, is a blend of their clip frames *A* and *B*:
with *j* = *f* − *S*ᵢ + 1 (1 to *d*), each byte is

```
out = ⌊(A · (d + 1 − j) + B · j + ⌊(d + 1) / 2⌋) / (d + 1)⌋
```

so *B*'s weight rises by 1/(*d* + 1) each frame and neither picture is ever shown alone inside
the dissolve.

## 10. Governance

### 10.1 What an element governs

An element (§3) **governs** the output frames that show what it directed, on the timeline the
directed output states:

| element | the output frames it governs |
|---|---|
| `["shot", i]` | each frame that shows any frame of shot *i* |
| `["keyframes", e]` | each frame that shows a frame *t* of the entry's shot with *t* in its **keyed frames**: from its first key's `t` to the shot's end where it has a track, and otherwise the `t` of each key that gives a class |
| `["region", j]` | each frame that shows a frame of the region's shot from `from` up to, not including, `to` |
| `["sequence"]` | every output frame |

Within those frames, a region governs the samples inside its rects, and every other element every
sample. A frame that shows two events is governed by what governs either.

The directed output states this in `governs`: one entry for each element of each record it
names, in the order of `direction` and then of each record's elements (§3), each `{"record",
"element", "frames"}`: the record's position in `direction`, the element's path and the frame
set (§2) it governs, `[]` for none. An entry that is not that element's, or not its frame set, is
`governance_mismatch`. So a tool cannot claim for a director frames the director's choice did not
govern, and cannot leave out frames it did.

### 10.2 The per-frame map

The **frame map** gives each output frame: the frames it shows, `[shot, t]` for each event
occupying it (§6.2); and `directed_by`, an object of four members, `pixel`, `frame`, `shot` and
`sequence`, each the positions in `directors` of the signers of the elements of that level that
govern it, ascending, none twice. It shows, frame by frame, which person directed the framing,
the timing, the held samples and the order, beside whatever made the frame's samples: the
layers' sources (§8.2) for `replicate`, the attested generator for `attest`.

### 10.3 The contribution record's `direct` act

The contribution record, `wai.media.contribution`, reserves the act `direct` for this
extension. Each element that governs at least one frame becomes one step of that record's session:

| member of the step | value |
|---|---|
| `party` | the director: a `human` party whose key is the element's record's signer |
| `act` | `direct` |
| `inputs` | none |
| `output`, `region` | `null` |
| `params` | `{"record", "element", "frames"}`: the direction record's link hash, the element's path, and the frame set it governs (§10.1) |
| `check` | `replicate` when the directed output's `check` is `replicate`, so every check of §12.2 the element has ran on recomputed frames; `attest` otherwise |
| signer | the director |

Its dependency rule: the step introduces no sample and changes none; it adds the director to
`directed_by` of every sample the element governs (§10.1), in every frame of `frames`, and
leaves `made_by` as it is. A verifier of the contribution record holding the direction records and
the directed output recomputes `frames` by §10.1 and refuses a step whose `frames` differ, as
§12.2 refuses a `governs` entry. So the authorship map shows, per frame, that a person directed
framing and timing where a model made the samples. The contribution record's revision of
2026-10-21 maps still pictures only; the step applies, sample by sample, in the revision that maps
video frame by frame, and to a still picture as an output of one frame.

## 11. Mappings

### 11.1 C2PA actions

For each director with at least one element that governs a frame, an accepted case gives one
C2PA statement, `{"actions": A}`, written in JCS: `A` is an actions assertion's actions (C2PA 2.4
§18.15.1), one for each of that director's elements that governs a frame, in the order of §10.1:

```
{"action": "c2pa.edited", "changes": C, "description": T}
```

- `T`, the action's description (C2PA 2.4 §18.15.4.1), by level: `"directed shot: framing,
  duration and subjects"`, `"directed keyframes: framing over time"`, `"directed region: samples
  held to a reference"` and `"directed sequence: order, cuts and transitions"`.
- `C`, its `changes` (C2PA 2.4 §18.15.4.6): one region map (§18.2) for each frame range of the
  element's frame set, in order, each `{"region": [{"type": "frame", "frame": {"start": a,
  "end": b − 1}}]}`, a frame range of §18.2.2.4, whose end is inclusive. For a held region, one
  region map for each frame range and each rect, by range and then by rect, whose `region`
  holds that frame range and then `{"type": "spatial", "shape": {"type": "rectangle", "unit":
  "pixel", "origin": {"x": X, "y": Y}, "width": W, "height": H}}`, the rect, the coordinates as
  integers (a manifest in CBOR writes them as the floats §18.2.3 types them).

C2PA 2.4 §18.2.2.4 recommends temporal ranges for media over time. Direction is exact in frames,
and a frame's time at a rate such as 24000/1001 has no exact decimal form, so the statement uses
frame ranges; a manifest that also wants times derives them from `rate`.

Each director's statement is one actions assertion of the output's manifest, labelled by C2PA
2.4 §6.4's rule for several assertions of one type. It is never the first actions assertion,
which begins with the manifest's `c2pa.created` or `c2pa.opened` action (C2PA 2.4 §18.15.2):
the generation's, or the contribution record's statement where there is one. Embedding the
statements in a manifest, and signing it, is C2PA's.

### 11.2 Identity assertions (informative)

A director may give the output's manifest an identity assertion (Identity Assertion 1.2 §5.1,
label `cawg.identity`) whose referenced assertions include the actions assertion of §11.1 that
holds that director's actions, which §6.1 there gives as an example of an assertion that lets an
actor describe its relationship to the content. A role of §5.1.2 there is the director's own
statement; this extension derives none. The director's key in the records is an Ed25519 key, and
the relation between it and the assertion's credential is the director's to state.

### 11.3 An open timeline interchange format (informative)

A sequence (§6) maps onto OpenTimelineIO's serialized schemas, release 0.18.1: a `Timeline.1`
whose `tracks` is a `Stack.1` holding one `Track.1` of `kind` `"Video"`, whose children are, for
each event *i* in order, a `Clip.2` whose `source_range` is a `TimeRange.1` from `start_time`
`in` to a `duration` of *L*ᵢ − *d*ᵢ₊₁ frames, each a `RationalTime.1` at `rate` *num* / *den*,
naming the shot's clip as its media; and, before each event entered by a dissolve, a
`Transition.1` of `transition_type` `"SMPTE_Dissolve"` with `in_offset` 0 and `out_offset` *d*ᵢ
frames. The cut point is then *S*ᵢ, the outgoing clip's range ends there, and the dissolve
draws its last *d*ᵢ frames from past that range, as this extension's timeline shows them. An edit
decision list that records each event's source in and out points and its record in point (*S*ᵢ),
with a cut or a dissolve of a duration, carries the same values. Neither format carries the
framing of §5 or the governance of §10, and neither is signed: they are the edit's interchange,
and the direction records are its evidence.

## 12. Verification

### 12.1 Inputs

A verifier is given: the directed output record; the direction records it holds, in a list; the
model files it holds, each named by its SHA-256; the pictures it holds, each a buffer named by
its samples digest; and a **policy** of one member:

| member | values | default | meaning |
|---|---|---|---|
| `require` | `attested`, `exact` | `attested` | the lowest status it accepts (`not_exact`) |

### 12.2 Steps

The verifier runs these steps and stops at the first refusal, which it reports with its reason
and its place: `["output"]`, `["record", i]`, `["direction", k]`, `["layer", i]`, `["clip", i]`,
`["clip", i, t]`, `["timeline"]`, `["timeline", i]`, `["governs", i]` or `["policy"]`, each *i*
a position in its list (a record's among the records given, a direction record's *k* among
`direction`), and *t* a frame of a clip.

1. **Reading** (§2, §4–§6, §8): the directed output at `["output"]`, then each record given at
   `["record", i]`: `record_malformed`, `bad_key`, `bad_signature`.
2. **The direction**, each link of `direction` in order at `["direction", k]`:
   1. a record given has that link hash (`record_unknown`);
   2. the first is a storyboard and no other is; each keyframes or sequence record's
      `storyboard` is the first's link hash (`storyboard_mismatch`);
   3. its signer is one of `directors` (`signer_not_party`);
   4. no second record sets the sequence level, and no keyframes entry directs a shot an
      earlier record's entry directs (`direction_conflict`);
   5. against the storyboard, every shot, subject and angle it names exists; every key's `t` is
      below its shot's duration; every key has one subject entry per subject of its shot; every
      region's `to` is at most its shot's duration, its rects are within the frame and its
      source is the frame's size; every event's `out` is at most its shot's duration
      (`direction_inconsistent`).
3. **The check**, at `["output"]`: a `replicate` record with a layer of another generator
   than `wai.generate.spectral` (§8.2) is `check_mismatch`: a float generator's picture is
   never claimed exact.
4. **The layout** (§8.5): rule 1 at `["output"]`; rules 2 to 4 clip by clip at `["clip", i]`;
   rule 5 event by event at `["timeline", i]`; rule 6 at `["output"]`: `layout_invalid`,
   `layer_missing`.
5. **The layers**, for `replicate`, each in order at `["layer", i]`: a generated layer whose
   model file the verifier does not hold, or a given layer whose buffer it does not hold, is
   `layer_missing`; a held buffer that is not `width` · `height` · 3 bytes is
   `samples_mismatch`; a generated layer whose picture's samples digest is not `samples` is
   `regeneration_mismatch`.
6. **The frames**, for `replicate`, at `["output"]`: the verifier renders every clip frame (§9.1)
   and every output frame (§9.2) and computes the digest of §8.4; another digest than
   `output.digest` is `frame_mismatch`.
7. **The direction against the layout**: each element of each direction record, in the order of
   §10.1, against the clips and the timeline the directed output states:
   - `["shot", i]`: clip *i*'s `frames` is the shot's duration (`duration_mismatch`, at
     `["clip", i]`); its subjects are the shot's, in its order (`subject_mismatch`, at
     `["clip", i]`); each class of the shot's `framing` matches the measure (§7) of clip frame
     0 (`framing_mismatch`, at `["clip", i, 0]`);
   - `["keyframes", e]`: for each keyed frame *t* of the entry's shot in ascending order, each
     subject's box is its track's value (§5.2), the camera window is the camera track's, and,
     at a key, each class it gives matches the measure (`framing_mismatch`, at `["clip", s,
     t]`, *s* the shot);
   - `["region", j]`: the shot's clip has a hold with the region's `from`, `to`, rects and source
     (`region_mismatch`, at `["clip", s]`);
   - `["sequence"]`: with the directed events (§6.1), the timeline holds the same shots in the
     same order (`order_mismatch`, at `["timeline"]`); then, event by event, each starts at its
     directed frame (`cut_mismatch`, at `["timeline", i]`); then each has its directed
     transition (`transition_mismatch`); then its directed `in` and `out`
     (`duration_mismatch`).
8. **Governance** (§10.1), each entry in order at `["governs", i]`, and a missing entry at the
   position it would have: `governance_mismatch`.
9. **Policy**: an `attest` record under `require` `exact` is `not_exact`, at `["policy"]`.

A sequence level no record sets is not checked: the order is then the tool's own, and no element
governs it.

### 12.3 Status

The outcome's **status** is `exact` for `replicate`: the frames recompute, so the layout the
checks of step 7 read is what the frames hold, cut for cut and pixel for pixel. It is `attested`
for `attest`, with the attestation's generator and class: the same checks ran on what the tool
stated, which nothing recomputed. An `attest` record is never reported exact.

### 12.4 What an accepted case states

- `output`: the directed output's link hash;
- `status`, `generator` and `class`: §12.3, the last two `null` when exact;
- `records`: each record of `direction`, in order, its `kind`, its link hash, its signer's
  position in `directors` and its `levels`;
- `elements`: each element in the order of §10.1, its `record`, `element`, `level`, `director`
  and governed `frames`;
- `cuts`: the cuts of the stated timeline (§6.2);
- `frames`: the frame map (§10.2), one entry per output frame;
- `frame_sha256`: for `exact`, the samples digest of each output frame, in order; `null`
  otherwise;
- `layers`: each layer's status, in order: `regenerated` or `held` for `exact`, `stated` for
  `attested`;
- `c2pa`: the statements of §11.1, each `{"director", "statement"}`, by director.

## 13. Reasons

The codes this extension adds, each with where it is raised:

| reason | raised at |
|---|---|
| `record_malformed` | §2, §3, §4, §5, §6.1, §8, §12.2 step 1 |
| `bad_key` | §2 |
| `bad_signature` | §2 |
| `record_unknown` | §12.2 step 2 |
| `storyboard_mismatch` | §12.2 step 2 |
| `signer_not_party` | §12.2 step 2 |
| `direction_conflict` | §12.2 step 2 |
| `direction_inconsistent` | §12.2 step 2 |
| `check_mismatch` | §8.1, §12.2 step 3 |
| `layout_invalid` | §8.5 |
| `layer_missing` | §8.5, §12.2 step 5 |
| `samples_mismatch` | §12.2 step 5 |
| `regeneration_mismatch` | §8.2, §12.2 step 5 |
| `frame_mismatch` | §8.4, §12.2 step 6 |
| `duration_mismatch` | §4.2, §6.1, §12.2 step 7 |
| `subject_mismatch` | §4.2, §12.2 step 7 |
| `framing_mismatch` | §5, §7, §12.2 step 7 |
| `region_mismatch` | §5.3, §12.2 step 7 |
| `order_mismatch` | §6, §12.2 step 7 |
| `cut_mismatch` | §6.2, §12.2 step 7 |
| `transition_mismatch` | §6.1, §12.2 step 7 |
| `governance_mismatch` | §10.1, §12.2 step 8 |
| `not_exact` | §12.1, §12.3 |

## 14. Conformance

[`live-conformance/direction/`](../live-conformance/direction/) holds the corpus, in one file,
`vectors.json`, written from fixed seeds by `wai_direction_vectors gen`. It holds the model file
of `wai.generate.spectral` its layers are generated with, the pictures given as they are, the
**measures** (the framing measure of §7 alone: boxes at both sides of every size bound and every
column and row bound, past every edge and at the limits of a signed integer, each with the classes
it measures), the **interpolations** (§5.2 alone: each mode, rising and falling, at the keys,
between them and with floors below zero, each with its value), and the **vectors**: each a case
of §12.1, its outcome and, accepted, everything §12.4 states; refused, its reason and its place.
Its frames are small (64 × 36 pixels at 24 frames a second), so every one recomputes. It has:

- **Accepted, `exact`**: a storyboard of three shots and two subjects, a cinematographer's
  keyframes (a camera that eases across a plate in one shot, a push in from a medium to a close
  framing in another) and a held region of a drawn reference, and an editor's sequence that
  reorders the shots, trims one and dissolves into it, every frame recomputed; the same
  storyboard fixing its own order, with no other record; a storyboard and keyframes with no
  sequence level, the order then the tool's own; and the first under a policy that requires
  exact.
- **Accepted, `attested`**: the first direction with frames a float generator made, the layout
  stated by a tracker.
- **Refused**: each reason of §13 at least once, each a control of one defect: among them a cut
  one frame from its directed frame, a keyframe box one pixel from its track, a size class the
  box does not measure, a direction record signed by a key that is not a director's, the edit's
  shots in another order, a float generator's plate claimed exact, frames that do not recompute,
  a governance entry claiming frames its element does not govern, and a missing held region.

A conforming verifier gives every vector the outcome, the reason, the place and the statements
the corpus states. `wai_direction_vectors verify` and the test `tests/direction.rs` check that
the file regenerates byte for byte, that `wai-rs` gives every vector its outcome, that the corpus
refuses with every reason of §13, and that §13 lists exactly the reference's codes.
`live-conformance/direction_verify.py` is the second implementation, written from this text: it
shares no code with `wai-rs`; it runs the generator in its own arithmetic and checks it first
against the generator's conformance vector (`wai-rs/tests/vectors/generate/sha256.json`),
renders every frame, interpolates every track and measures every box itself, writes every
statement and C2PA statement itself, reads records with synthesis's second implementation's
JSON reader, verifies every signature with WAI's strict rule (`claim-conformance/_ed25519.py`)
and every link hash with BLAKE3 written from its specification (`claim-conformance/_blake3.py`);
it mutates copies of the corpus and requires every copy to fail; and it drops each of its own
checks in turn and requires the corpus to catch the weakened verifier.

## 15. Security and privacy considerations

- **What is checked and what is stated.** A direction record is its director's signed
  statement of a choice; WAI checks the output against it, not whether the director made the
  choice unaided. Under `replicate` the frames recompute, so the layout the checks read is what
  the frames hold: a cut, a box, a held sample is exact. Under `attest` the layout is the tool's
  statement, of the class it names, and the outcome says so.
- **No enrolment without a signature.** A person is credited with direction only for elements of
  records that person signed, and a directed output names its directors; a record by any other
  key is refused. A tool cannot credit or blame a director for frames the director's elements do
  not govern (§10.1).
- **Direction is not making.** Directing a frame does not make its samples. The frame map keeps
  the two apart: `directed_by` per level beside the layers' or the generator's making, never one
  merged figure, and an origin claim keeps its class ([synthesis](synthesis.md) §3.2).
- **Not a judgment.** The records state facts frame by frame; they rank no contribution. What
  any of it means for authorship is decided elsewhere, by others, under their own rules.
- **Privacy.** Records name directors by keys, never by name; subjects by reference pictures,
  which a director discloses as it chooses. A layer's prompt is in the clear in the directed
  output; a tool that keeps prompts private names the layer as given (`source` `null`) and
  commits the prompt in a contribution record instead. This extension sets no consent
  requirement for a service's own mechanics: rendering under a director's records, measuring a
  frame, stating governance. Using a distributor's WAI service is the person's agreement to how
  that service works. Where a regime asks for a lawful basis for processing a director's key or
  a subject's reference picture, the service records it in a `lawful-basis` claim
  ([jwp-receipts](jwp-receipts.md) §2.4.6).