open-standards/wai/SPEC.md

# WAI v1.0 — Web AI Media Transport & Execution

Status: **Draft Standard**. Reference implementation: `wai-rs/` (Rust
lib + cdylib + staticlib, C FFI). Two paths open every conforming
file:

- the **neural condition** — the sink regenerates the media from a
  compact conditioning payload against a *shared ambient prior* (a
  model). The model is a **requirement of existence**, never
  transported in the envelope that decodes against it, exactly as `.mp3`
  does not ship its decoder. Its identity MAY be pinned: the manifest MAY
  carry a *pin*, the SHA-256 of the whole parameter set the payload was coded
  against (§3.1), and a sink whose parameter set does not match the pin does
  not decode against it (§4, §7.1). The pinned set's wire form MAY be declared
  ([prior-carriage](extensions/prior-carriage.md)).
- the **zeroth condition** — a *menu of registered standard codecs*
  (AVIF, JPEG XL, PNG, Opus, FLAC, AV1, zstd, XZ and the others §5
  registers), each decoded by the sink with the library it already has.

WAI dispatches registered standard codecs and also defines WAI-native
capability classes (§5): integer codecs, worlds, scores, films and the
other extensions, each with its own payload format and conformance
criterion.

The capability a file requires is *named*, not supplied. Sinks that
have the named capability use it; sinks that don't fall back to a
declared `fallback` capability.

Keywords **MUST**, **MUST NOT**, **SHOULD**, **MAY** are RFC 2119.

---

## 1. Scope and model

WAI is a **container + capability dispatch standard**. It is
media-general; its floor and recommended set are royalty-free. Its
substance is the envelope, the dispatch rules and the registry's classes:
a standard codec is dispatched to a sink's existing decoder, not
re-implemented.

A WAI file declares (in its manifest):
- what media it carries,
- what *capability* a sink must have to decode it,
- a fallback capability the sink can use if it lacks the primary,
- the conditioning bytes for that capability,
- and, OPTIONALLY, for a capability that decodes against a parameter set, the
  pin of the set the payload was coded against (§3.1).

Conforming sinks MUST implement the container envelope (§2), the
manifest schema (§3), the capability-dispatch algorithm (§4), and at
least the **mandatory floor** capabilities (§5). Other capabilities are
OPTIONAL; §5 says which are RECOMMENDED.

---

## 2. Container

```
+--------+----------------+-------------------+----------------+-----------+
| "WAI1" | u32  man_len   | manifest (JSON)   | u32 payload_len| payload   |
| 4 B    | little-endian  | man_len bytes     | little-endian  | bytes     |
+--------+----------------+-------------------+----------------+-----------+
```

- Bytes 0..4 MUST be ASCII `WAI1`. A reader MUST reject any file whose
  first 4 bytes differ.
- `man_len` and `payload_len` are unsigned 32-bit little-endian.
- The manifest is UTF-8 JSON, serialized with no insignificant
  whitespace, preserving key insertion order. Readers MUST NOT depend
  on key order for *semantics*; encoders SHOULD preserve order so files
  round-trip byte-identically.
- Member names are unique within each object of a manifest, at any
  depth: a manifest that names a member twice in one object is malformed,
  and a reader rejects it as it rejects any manifest that does not parse.
  Names compare after their escapes are read.
- An integer member of a manifest, or of any JSON this specification or
  its extensions define, is a JSON number written as an integer: digits,
  with a minus sign only before a non-zero value, and no fraction or
  exponent. `-0`, `1.0` and `1e0` are not integers, whatever their value,
  and a member that must be an integer and is written so has the wrong
  type.
- The payload is opaque codec bytes whose interpretation is fully
  determined by the manifest's `model_requirement.capability` and, for a
  capability that decodes against a parameter set, by that set, which the
  manifest MAY pin (§3.1).

All WAI multi-byte integers are little-endian unless stated otherwise.

An envelope delivered over HTTP is one object of the proposed media type
`application/wai`, named by its receipt content hash, and fetched whole or, for a
`WAI2`, one verified component at a time
([http-delivery](extensions/http-delivery.md)).

---

## 3. Manifest schema

```json
{
  "wai": "1.0",
  "media": "image" | "audio" | "video" | "text" | "volumetric" | "haptic"
         | "metadata" | "interactive" | "binary",
  "intent": "replicate" | "create" | "improve" | "...",
  "model_requirement": { "capability": "<capability-string>",
                         "fallback":   "<capability-string>" | null },
  "conditioning": { "kind": "<codec-id>", "prior": { ... } },
  "security": { "encryption": "<scheme>" | "none", "kid": <u64> },
  "refines":  { ... },
  "target":   { ... }
}
```

- `wai` — spec version. This document defines `"1.0"`. Unknown major →
  reject. Unknown minor → accept. A reader ignores a member it does not
  understand, whatever the minor (§8).
- `media` — media class; informational. The capability is authoritative.
  The values are the registry's media classes (`codecs::capability_media`;
  [`registry/capabilities.json`](registry/capabilities.json)): `image`,
  `audio`, `video`, `text`, `volumetric` (meshes, point clouds, Gaussian
  splats), `haptic`, `metadata` (data applied with another payload),
  `interactive` (worlds, feeds, avatars) and `binary` (anything else). A
  reader MUST accept a value it does not know: the field is informational,
  and a MINOR revision may add a value (§8).
- `intent` — generative intent. `replicate` (reconstruct the source) is
  the only intent with normative reconstruction semantics in v1.0. What it
  promises is set by the capability's registry class (§7, *Replicate-soundness*):
  a byte-exact reconstruction — every conforming sink that decodes the payload
  produces the same bytes (decode-equivalence) — where the capability's
  reconstruction is specified exactly; and, for a standard codec whose named
  standard holds decoders to a criterion around a reference rather than to
  identical output (Opus's thresholds, RFC 6716 §6, are one), no more than that
  standard's decoder conformance, so conforming sinks' reconstructions may differ
  as far as that criterion allows; for a standard codec §7 has not yet classified,
  what its standard's decoder conformance states, which the registry does not
  assert is byte identity.
  For a **neural** capability, `replicate` promises decode-equivalence and is sound
  only under the integer-entropy contract (§7), with a pin (§3.1, §7): a capability whose decode runs
  float, or whose entropy decode alone is integer, MUST NOT claim it. That is
  WAI's rule, and this revision applies it to `wai.image.jpegai` as well; it is
  not a finding that JPEG AI's standard leaves the reconstruction unbounded (§7,
  `NeuralEntropyExact`).
- `model_requirement.capability` — the capability string a sink MUST
  resolve. See §5 for the registered set.
- `model_requirement.fallback` — a second capability the sink MAY use
  if it lacks the primary, or if the envelope's pin does not resolve for the
  primary at the sink (§4 step 3). `null` means the file is inert without the
  primary. An encoder whose primary capability decodes with a module a
  sink obtains apart from the envelope does not write `null` (§4).
- `conditioning.kind` — short codec id matching the capability. Helps
  multiplexers route bytes without parsing the full capability string.
- `conditioning.prior` — OPTIONAL. The pin (§3.1) of the parameter set the
  payload was coded against. It applies to whichever of
  `model_requirement.capability` and `model_requirement.fallback` decodes the
  payload (§4), where that capability decodes against a parameter set. Absent, or
  `null`, the key is not written, and a decode through such a capability is
  unpinned: it runs against whatever parameter set the sink holds, and it cannot
  back `intent = replicate` over a `NeuralIntegerExact` capability (§7).
- `security` — OPTIONAL. Absent, `null`, or `encryption: "none"` ⇒ the
  payload is plaintext (the default). Otherwise the payload is encrypted
  and MUST be recovered before dispatch; see §6. The capability, media,
  and conditioning fields always describe the **plaintext** codec.
- `refines` — OPTIONAL. Carried exactly when `model_requirement.capability` is
  refinement-only (§5, *Refinement capabilities*): what the payload refines, by
  content hash ([staged-delivery](extensions/staged-delivery.md) §3). A manifest
  that carries it has a `null` `fallback`.
- `target` — codec-specific reconstruction parameters (dimensions,
  sample rate, duration, frame count). Informational; the payload is
  self-describing per the codec's own format.

### 3.1 Parameter-set pin (`prior`)

A capability **decodes against a parameter set** when its registry class (§7) is
`NeuralIntegerExact`, `NeuralEntropyExact`, `NeuralFloat` or
`GenerativeDeterministic`. Its **parameter set** is every byte, other than the
payload, that its decode reads and that its output depends on: networks, entropy
tables, normative constants. The sink supplies it; the envelope never carries it
(§1). §5 registers each such capability's parameter set as **one file** or
**several files**; a capability registered with neither, and a capability this
revision does not register, is treated as several files.

A manifest names the parameter set a payload was coded against with a **pin**:

```json
"prior": {
  "id":          "<a name for people>",
  "sha256":      "<64 lowercase hexadecimal digits>",
  "determinism": "<how the decode is made reproducible>",
  "form":        "<a registered prior form>",
  "tier":        "decode-equivalence",
  "supersedes":  { "sha256": "<64 lowercase hexadecimal digits>", "form": "<a registered prior form>" }
}
```

| member | presence | meaning |
|---|---|---|
| `id` | REQUIRED, string | A name for people and logs. A sink MUST NOT select, resolve or accept a parameter set by it. |
| `sha256` | REQUIRED, string | The SHA-256 (FIPS 180-4) of the parameter set in its form (below), as 64 lowercase hexadecimal digits. |
| `determinism` | REQUIRED, string | How the decode is made reproducible, in words. Informative, except where §7 reads it for a pin without `tier`. |
| `form` | OPTIONAL, string | A registered prior form (§5; [prior-carriage](extensions/prior-carriage.md)). |
| `tier` | OPTIONAL, string | The reproducibility tier ([determinism-tiers](extensions/determinism-tiers.md) §2) the sender claims for the decode, from a closed vocabulary: `decode-equivalence`, `entropy-consistency`, or `none` for neither. |
| `supersedes` | OPTIONAL, object | Informative. The parameter set this one replaces, named as a pin names one: its `sha256` (REQUIRED) and, if it is pinned in a declared form, that `form`. |

A `prior`, or a member of it, whose JSON type is not the one given makes the
manifest malformed, and a reader rejects it as it rejects any manifest that does
not parse. A mistyped member is never read as absent: a `form` dropped that way
would turn a set-form pin into a form-less one, which a single file can satisfy. `null` in an OPTIONAL member is the same as the member being absent.
A member this section does not define is ignored (§8). An optional member that is
absent is not written, so a pin written without it keeps its bytes.

**What the digest covers.** A pin identifies the whole parameter set.

- With `form` absent, `sha256` is the SHA-256 of the one file that holds the set.
  That is admissible only for a capability registered with a one-file set.
- With `form` present, `sha256` is the SHA-256 of the set serialized in exactly
  that form ([prior-carriage](extensions/prior-carriage.md) §2). The form
  `wai.prior.set` serializes a set of one or more files as its **listing**, so a
  pin in that form covers every file of the set:
  - The listing has one line for each file: the file's SHA-256 as 64 lowercase
    hexadecimal digits, two spaces (0x20 0x20), the file's path, and a line feed
    (0x0A).
  - A path names the file below the set's root: one or more segments joined by
    `/` (0x2F). A segment begins with one of `A`–`Z`, `a`–`z`, `0`–`9`, `_` and
    `-`, continues with any of those and `.`, and does not end with `.`.
  - The lines are in strictly ascending order of their paths compared as byte
    strings. No two paths are equal once `A`–`Z` are mapped to `a`–`z`, and no
    path followed by `/` begins another path.
  - The listing is its lines concatenated. It has at least one line and nothing
    else: no header, no byte-order mark, no carriage return, no blank line.

  A listing is the text common checksum utilities write and check, so a third
  party can check a set against its listing, and a listing against a pin, without
  WAI software.

**Verification.** A sink that is to decode through a capability, for an envelope
whose pin applies to that capability, MUST verify the pin first. A pin applies to
a capability that decodes against a parameter set, and to a capability this
revision does not register when the sink advertises it (such a string is
dispatchable, §4): such a capability is verified as several files, so a form-less pin over it is refused
(`form_required`). The pin **resolves** when the sink holds a parameter set for the
capability whose digest, in the pin's form, equals `sha256`: in the form
`wai.prior.set`, a listing whose SHA-256 is `sha256`, together with every file the
listing names below the listing's root, each with the SHA-256 its line gives.
Nothing else satisfies a pin: not a matching `id`, not a set that `supersedes`
names or that names this one, and not a set in another form whose unpacked
parameters compare equal. A sink MAY hold several parameter sets for one
capability; the pin resolves when one of them satisfies it.

A pin that does not resolve is refused with one reason. The checks run in this
order, and the first that fails is the reason reported:

1. `pin_malformed`: `sha256` is not 64 lowercase hexadecimal digits.
2. `form_not_registered`: `form` names no registered prior form.
3. `form_unreadable`: `form` is registered and the sink has no reader for it.
   This is the sink's gap, and it is never reported as a mismatch.
4. `form_required`: `form` is absent and the capability is not registered with a
   one-file set.
5. `prior_revoked` or `prior_not_authorised`: the sink holds a key log
   ([jwp-receipts](extensions/jwp-receipts.md) §7), and it refuses the set this
   pin names, which it identifies by the pin's `sha256` in the pin's `form`. The
   reason is `prior_revoked` when that digest is in the `revoked_modules` of the
   head the sink reaches from any anchor it holds, or, where none can be reached,
   of that anchor revision (§7.9 check 5). A revocation needs no grant, status or
   clock, so this part runs for every capability a pin applies to, and whichever
   check of §7.9 a grant fails first. Where the sink also holds the capability's
   pinned parameter sets to §7.9 (§4, *Module authorisation*), §7.9 authorises the
   set as a decoder module, and the reason is `prior_not_authorised` when §7.9 does
   not hold it for any other reason — no grant from one of the sink's module
   origins, a granting key §7.4 does not confirm, no fresh status because the sink
   is offline or has no clock it trusts, a grant outside its window or for another
   capability — or the sink holds no grant for the set. The sink reports §7.9's
   check and reason with it. A sink that holds no key log skips this check.
   Check 5 covers every identity of the bytes the decode reads, not only the
   pin's: once checks 6 to 8 have found the set, the sink runs this check again
   over each digest a pin over those bytes could name — the listing of the files
   the decode reads, and, for a capability registered with a one-file set, that
   file's SHA-256 — so a set refused under one identity is not reached through a
   pin that names it by another (a set-form pin over a listing of one revoked file,
   or a listing that adds a file the decode does not read). Authorisation needs
   one of them to be held; the pin's own digest, which passed above, is one.
6. `prior_not_held`: the sink holds no parameter set for the capability whose
   digest, in the pin's form, equals `sha256`; a file the sink cannot read is not
   held. A sink that holds the set a well-formed `supersedes` names — a file when
   `supersedes.form` is absent, a listing when it is `wai.prior.set` — reports
   `digest_superseded` instead, so that its operator can tell it holds the set
   this one replaces.
7. `set_malformed`: in the form `wai.prior.set`, the listing whose SHA-256 is
   `sha256` breaks a rule above.
8. `part_mismatch`: in the form `wai.prior.set`, in every copy of that listing
   the sink holds, a file the listing names is missing, cannot be read, or has a
   SHA-256 other than its line gives.

These checks read the envelope, the registry, the forms the sink can read and the
bytes it holds, and nothing that depends on how the sink was built, so two
conforming sinks that read the same forms and hold the same bytes report the same
reason. Check 5 also reads what the sink holds under jwp-receipts §7.9 and its
clock, so two sinks that hold the same key-log material and agree on the time
report the same reason there. Refusal reasons are snake_case; tier names and class labels are
kebab-case.

**Decoding against a verified set.** A pinned decode MUST read its parameters
only from the set it verified. A decode that needs a file the verified set does
not contain is refused with `set_incomplete`; the sink presents no output from it,
and dispatch treats it as a pin that did not resolve. A file a model names for
part of its own data, such as the file an ONNX tensor stored as external data
names, is a file the decode needs: when the decode cannot take it from the
verified set, it is refused the same way. A sink SHOULD decode from
the very bytes it verified: one that reads a file again after verifying it can
decode bytes it did not verify.

**A refused pin** makes its capability **unsupported for that envelope** at that
sink. It is not a decode error and not a reason to decode anyway, as
[capability-constraints](extensions/capability-constraints.md) §2 treats an unmet
constraint. Dispatch continues as §4 and §7.1 state. A sink MUST NOT decode a
payload through a capability whose pin it refused.

**Tier.** `tier` is the sender's claim, from the registered vocabulary
`decode-equivalence`, `entropy-consistency` and `none`. An encoder MUST write a
registered token. A sink reads a token this revision does not register as `none`,
and a verifier reports it (`tier_unregistered`). The tier a registry class gives
is `decode-equivalence` for `NeuralIntegerExact`, `entropy-consistency` for
`NeuralEntropyExact`, and `none` for every other class. With the tiers ordered
`none` < `entropy-consistency` < `decode-equivalence`, a `tier` above the class's
does not conform (`tier_exceeds_class`): the `replicate` verdict (§7) does not
rest on it, and a verifier reports it. The decode itself is unaffected. A `tier`
at or below the class's is honoured as stated.

**Updates.** `supersedes` is informative: it records that the set a pin names
replaces another, changes no decision, and changes only the reason check 6
reports. A `supersedes` whose `sha256` is not 64 lowercase hexadecimal digits or
equals `sha256`, or whose `form` is not a registered prior form, is malformed: a
verifier reports it (`supersedes_malformed`), and a sink ignores the member and so
never reports `digest_superseded` for that pin. Content coded against a set pins
that set, and a successor never satisfies its predecessor's pin, so content coded
against a predecessor decodes at a sink exactly as long as that sink still holds
the predecessor. A sink can hold both without a second copy of the files they
share, by keeping the predecessor's listing beside the successor's. An update
delivered as a change to an earlier set, such as an NNC incremental update, is not
a parameter set and is never pinned; the self-contained set it yields is
([prior-carriage](extensions/prior-carriage.md) §2).

**Where a pin sits.** In a `WAI1` manifest, `conditioning.prior` (§3). In a `WAI2`
manifest, a rendition entry's `prior`, which pins that entry's capability and is
part of the entry's identity (§7.1). A pin where no capability it would apply to
decodes against a parameter set, and every such capability is registered, means
nothing: an encoder MUST NOT write one, a verifier reports it
(`prior_not_applicable`), and a sink ignores it.

**Sinks of earlier revisions.** A sink that implements an earlier text of this
specification may do one of two things with a pin (§8). It may reject the
envelope as carrying a member it does not understand, which fails closed. Or it
may ignore `prior`, and then the pin fails open: such a sink may decode a pinned
payload against another parameter set. The rules of this section bind sinks that
implement this revision; an encoder that must not be decoded against another set
at an earlier sink has no means to require that within `1.x` other than a
capability string earlier sinks do not advertise.

**Envelopes these rules make non-conforming.** The rules enforce the definition of
`prior` as it was made — a lowercase hexadecimal SHA-256 of the artifact that
determines the decode's output, which a `replicate` over an integer-exact neural
capability requires — and change no payload format and no role (§8). Each of these
envelopes but the last still parses, and does not conform:

- one whose pin's `sha256` is not 64 lowercase hexadecimal digits (the reference
  implementation compared digests without regard to case or length);
- one whose pin has no `form` for a capability registered with several files: its
  digest covered one file of the set or, where the registered artifact was a
  directory, nothing a sink could hash;
- one that carries a pin where no capability it would apply to decodes against a
  parameter set;
- one with `intent = replicate` whose decode capability is `NeuralIntegerExact`
  and which pins no parameter set for it, or pins one that does not declare
  `decode-equivalence` (§7); the reference implementation already refused these;
- one that carries a member this revision defines — `conditioning.prior` or a
  rendition entry's `prior`, a pin member, or a rendition entry's `id`, `sha256` or
  `companion_of_id` (§7.1) — with a JSON type other than the one defined. A reader
  of an earlier revision may have ignored the member; a reader of this one rejects the
  manifest as malformed, so this envelope no longer parses. §8 states this as the
  one exception this revision makes to the `1.x` freeze.

---

## 4. Capability dispatch

A conforming sink MUST implement this algorithm:

1. Read the manifest. If `wai` major version is unknown → reject.
2. Let `cap = model_requirement.capability`.
3. If `cap` is dispatchable (below), the sink advertises `cap` — holding under
   jwp-receipts §7.9 any module it obtained apart from the envelope to decode
   `cap` (*Module authorisation*, below) — and, when a pin applies to `cap`
   (§3.1) and the manifest carries `conditioning.prior`, the pin resolves at the
   sink for `cap`, decode `payload` via the codec registered for `cap` (§5),
   against the verified set, and return. A pin that does not resolve makes `cap`
   unsupported for this envelope: keep its reason and continue at step 4.
4. Else, let `fb = model_requirement.fallback`.
5. If `fb != null`, `fb` is dispatchable and the sink advertises `fb` on the same
   terms, and, when a pin applies to `fb` and the manifest carries
   `conditioning.prior`, the pin resolves at the sink for `fb`, decode via `fb`'s
   codec and return. A pin that does not resolve: keep its reason.
6. Else the file is **inert at this sink** — return an error to the calling
   application that names why: the reason kept at step 3, else the one kept at
   step 5, else `missing_capability`.

`conditioning.prior` pins the parameter set the payload was coded against, so it
applies to whichever capability decodes the payload. A pin that does not resolve
is never a reason to decode against another parameter set: a learned decode
against the wrong parameters yields plausible output, not an error. An encoder
that needs an alternative for sinks that hold another parameter set carries one
pinned rendition per set in a `WAI2` envelope (§7.1).

A capability is **dispatchable** unless this revision registers it as something
a sink does not decode on its own: a candidate (§5, *Candidate capabilities*), a
companion-only capability (§5, *Companion-only capabilities*), a refinement-only
capability (§5, *Refinement capabilities*), a record or a derivation (§5, *Kinds
of registered string*). A sink MUST NOT take step 3 or
step 5 with a capability that is not dispatchable, whatever it advertises. A
string this revision does not register is dispatchable: a later MINOR revision
may register it (§8), and a sink that has a codec for it says so by advertising
it. A `WAI1` whose `cap` is companion-only does not conform and is inert: its
payload is companion data, so step 5 is not taken either (§5).

A WAI **payload always corresponds to exactly one capability**. v1.0
does not support multi-codec multiplexed payloads.

Dispatch goes by `cap`: a `WAI1` whose `cap` is not refinement-only is dispatched as
above whatever else its manifest carries, a `refines` member included, which such a
manifest MUST NOT carry and a sink ignores
([staged-delivery](extensions/staged-delivery.md) §3). A sink never takes step 3 or
step 5 with a capability §5 registers as **refinement-only**, and a `WAI1` whose `cap`
is refinement-only takes neither:
its payload refines the output of a base, and
[staged-delivery](extensions/staged-delivery.md) §4 processes it. A sink that does
not implement that extension finds the capability unsupported, and the envelope is
inert.

If the manifest declares `security.encryption` other than `"none"`
(§6), the sink MUST recover the plaintext payload before step 3 and run
this algorithm on the plaintext; see §6.4.

**Module authorisation.** A module is code a sink obtains apart from the envelope
and runs to decode a capability, or to rebuild a reconstruction from what a decoder
produced: a decoder or reconstruct module
([`extensions/jwp-receipts.md`](extensions/jwp-receipts.md) §7.9). A sink that
decodes `cap` with one MUST hold it under §7.9 for the envelope before it runs it, from one of the sink's module origins (§7.9): the origins its
relying party names, and the origin under which it confirmed the envelope's
delivery. A module that is not held — not granted by a module origin, revoked, or
not checkable because the sink is offline or has no clock it trusts — leaves `cap`
not advertised for that envelope at steps 3 and 5. A sink that holds a key log
refuses a parameter set whose digest the key log revokes (§3.1 check 5,
`prior_revoked`). A sink MAY also hold the parameter sets envelopes pin to §7.9, by
its relying party's choice per capability; for such a capability §3.1's check 5
also refuses a pinned set §7.9 does not hold (`prior_not_authorised`). A refused
pin does not resolve. An envelope that pins no set is decoded against the set the
sink holds for `cap`, and a sink that holds a key log runs check 5 over that set
too, identified by each digest a pin over it could name: it does not decode `cap`
for that envelope when any of them is revoked, or, for a capability it holds to
§7.9, when §7.9 holds the set under none of them. It then decodes from the bytes
check 5 ran over alone, as a pinned decode reads only what its pin verified (§3.1):
read once, so a file changed after the check is not what the decode reads, and, for
a capability it holds to §7.9, only the part §7.9 holds (the set, the files a held
listing names, or one held file); a file the decode needs outside them is
`set_incomplete`. Either way
the algorithm above runs unchanged, so a sink MUST NOT report a file inert because
of module authorisation while a fallback it can decode without that module remains,
and an encoder emitting a capability whose decode needs a module MUST declare such a
fallback (§3). A sink configured with no key log holds no module under §7.9, so it
runs no module obtained apart from an envelope, and it does not run check 5. The
reference dispatcher in `wai-rs` runs no module obtained apart from an envelope; it
runs check 5 when the sink is configured with a key log (`key_log::SinkKeyLog`,
feature `key_discovery`, given to `pin::verify_with`, to `int_payload::resolve`
(which runs it for a pinned decode and over the files an unpinned integer decode
reads) or to `neural::ModelRegistry::with_key_log`, which also runs it before an
unpinned decode, over the bytes that decode then reads, read once: an integer
decode's members, and otherwise the registered file of a one-file set, or every
file below the directory the decoder reads from, identified by the listing
`pin::listing_of_dir` derives, each file's digest and each listing beside them that
matches): the revocation part for
every capability a pin applies to, and §7.9's authorisation for the capabilities the
sink holds to it. `wai-web` and the C ABI take no key log.

---

## 5. Registered capabilities (the zeroth menu)

WAI v1.0 registers the following capabilities. Each names a standard
codec or a WAI-native class. The mandatory floor and the recommended
set are royalty-free; every payload and companion capability carries a
licence class (see the end of this section), and one that is not
royalty-free is never a default. The "payload format" column is the
on-the-wire payload bytes. For a standard codec it is the canonical
bytes that codec normally emits (a JXL file, an Opus stream, etc.) —
WAI does not wrap or re-frame them. For a WAI-native class it is the
format the class's registration or extension defines; for the
WAI-native integer codecs, the format *Payloads of the WAI-native
integer codecs* (below) pins.

The registry is also published in machine-readable form, generated from
the reference implementation: [`registry/capabilities.json`](registry/capabilities.json)
and its schema, with the policy that governs it in [`REGISTRY.md`](REGISTRY.md).
Where the two differ, this section is normative and the generated files are
wrong.

**Kinds of registered string.** Every string this section registers has one
kind:

- a **payload** capability: an envelope's `capability`, `fallback` or `WAI2`
  rendition names the codec its payload is decoded with;
- a **companion** capability: companion-only (*Companion-only capabilities*,
  below), carried only as a `WAI2` companion;
- a **refinement** capability: refinement-only (*Refinement capabilities*,
  below), applied to the output of the stage below it and never decoded on its
  own;
- a **derivation** operation (`wai.derive.*`): an operation on samples another
  capability decoded, which names no payload format;
- a **record**: a signed record a WAI tool emits and verifies (a receipt kind —
  `wai.asset.provenance`, `wai.quantum.calibration`, `wai.quantum.job`, and the
  delivery records `wai.video.manifest`, `wai.video.ssai`, `wai.video.keys`,
  `wai.video.package` and `wai.video.channel`), which names no payload format.

Only payload, companion and refinement capabilities have a media class, a
fidelity tier, a replicate class and a role. A derivation or record string is not a payload
capability and is not dispatchable (§4): step 3 does not take an envelope's
`capability` that names one, and step 5 may still take a usable fallback. A row of the tables below that
registers a record says what the record attests, not a payload format. Named
sub-identifiers this section uses — `wai.det.fixed64`, `wai.world.sim.feed0`,
`wai.world.sim.skeletal0` — name parts of a capability's definition and are
not capabilities.

### Mandatory floor

Every conforming sink MUST implement:

| capability | payload | role |
|---|---|---|
| `wai.image.png` | PNG (W3C Recommendation, *Portable Network Graphics (PNG) Specification*, Third Edition) | universal lossless image |
| `wai.audio.flac` | FLAC (RFC 9639) | universal lossless audio |
| `wai.text.zstd` | Zstandard (RFC 8878) | universal compressed bytes |

These three guarantee that any sink with the standard system codec
libraries can open a file in the image, audio and text classes. The floor
has no video capability.

### Recommended modern set

Every conforming sink SHOULD implement:

| capability | payload | role |
|---|---|---|
| `wai.image.jxl` | JPEG XL (ISO/IEC 18181) | lossless + lossy |
| `wai.image.avif` | AVIF (the AOM AVIF specification: an AV1 image format based on HEIF, conforming to MIAF) | lossy (AV1-based) |
| `wai.image.jpeg` | JPEG (ITU-T T.81) | compatibility with existing JPEG decoders |
| `wai.audio.opus` | Opus (RFC 6716) | lossy speech and music |
| `wai.video.av1` | AV1 (AOMedia) | lossy |
| `wai.video.av1.lossless` | AV1 (AOMedia), `quantizer=0` | mathematically lossless YUV (lossy in RGB due to color conversion) |
| `wai.text.xz` | XZ/LZMA2 | maximum classical text ratio |

### Neural capabilities (OPTIONAL, sink advertises if its ML runtime supports them)

Each names a model the sink MAY have installed. Payload is a compact
conditioning representation specific to the capability (latent tokens,
prompt embeddings, etc.). The standard does not specify the byte
layout of these payloads — the capability owner does — except where this
section pins one: the integer codecs' payloads are pinned below
(*Payloads of the WAI-native integer codecs*).

Each row names the model and the operating point its own publication
reports (bitrate, sample rate) or the property WAI's integer construction
gives it; this section states no comparison between models or with the
zeroth codecs. The decode route for each is implemented in the reference
sink (see the `source` column / `neural/*.rs`); the model weights are
sink-supplied.

A capability that decodes against a parameter set (§3.1) is registered with that
set's shape. **One file:** `wai.neural.int_synth` (its model specification,
`int_synth.json`); `wai.neural.int_mlicpp` (its `WMB1` bundle, read from a set as
`model.wmb`); `wai.audio.int_codec` (its `WAM1` synthesis, read from a set as
`model.wam`); `wai.splat.int_codec` (its `WHM1` synthesis, read from a set as
`g_s.whm`); and `wai.neural.encodec32`, `wai.neural.dac`, `wai.neural.mimi`,
`wai.neural.wavtokenizer`, `wai.neural.snac` and `wai.neural.video_bmshj2018` (a
self-contained ONNX model; one that keeps tensors in external files is several
files). **Several files:** every other capability that decodes against a parameter
set; `wai.neural.int_hyper` and `wai.video.int_hyper` read `g_s.whm`, `h_s.whm` and
`entropy.json` from theirs. `wai.audio.int_codec` and `wai.splat.int_codec` were
registered as several files until this revision named their one-file formats. A later revision MAY register a capability of several files as one file when
it defines a one-file format that holds the whole set; that widens the
registration and makes no conforming envelope non-conforming (§8).

*Audio:*

| capability | model | operating point | source |
|---|---|---|---|
| `wai.neural.encodec32` | EnCodec at 32 kHz, 4 codebooks | 32 kHz; 4 codebooks of 2048 at 50 Hz (2.2 kbps) | EnCodec (arXiv 2210.13438) |
| `wai.neural.dac` | DAC at 44.1 kHz | 44.1 kHz; 9 codebooks at 86 Hz (about 8 kbps) | DAC (arXiv 2306.06546) |
| `wai.neural.mimi` | Mimi, 12.5 Hz frame rate | 24 kHz; 8 codebooks of 2048 at 12.5 Hz (1.1 kbps), real-time speech | Mimi (arXiv 2410.00037) |
| `wai.neural.wavtokenizer` | WavTokenizer single-codebook VQ | 24 kHz; one codebook at 40 or 75 tokens per second (under 1 kbps) | WavTokenizer (arXiv 2408.16532) |
| `wai.neural.snac` | SNAC multi-resolution RVQ (hierarchical code levels, own sample rate) | hierarchical code levels at the model's own sample rate | SNAC (arXiv 2410.14411), `neural/snac.rs` |

*Image:*

| capability | model | operating point or property | source |
|---|---|---|---|
| `wai.neural.bmshj2018` | bmshj2018-factorized + rANS (byte-exact CompressAI port) | about 0.26 bpp at 29 dB PSNR (quality 3) | CompressAI |
| `wai.neural.mbt2018_mean` | mean-scale hyperprior (Gaussian-conditional entropy) | the conditional-entropy family (arXiv 1809.02736) | `neural/hyperprior.rs` |
| `wai.neural.int_synth` | integer fixed-point synthesis (all-i64 transposed-conv, no float/ONNX) | bit-identical on every machine — soundly backs `intent = replicate` | `neural/int_synth.rs` (QAT weights sink-supplied) |
| `wai.neural.int_hyper` | integer fixed-point **conditional** codec — whole bitstream→image path integer (integer hyper-decoder σ → integer-σ entropy → integer synthesis) | the integer-entropy contract fully assembled: a PTQ of mbt2018_mean, bit-identical on every machine, whose integer decode CI holds within 1 dB of the same model's float decode on a pinned photograph | payload `WIH1`; `int_hyper_synth.rs` (tables sink-supplied) |
| `wai.neural.int_mlicpp` | integer fixed-point **MLIC++ entropy topology** — `K` channel slices of the latent, each decoded as a checkerboard anchor pass then a non-anchor pass, with an entropy network over concat[latent decoded so far, hyper] re-run before every pass to give σ‖μ; hyper-decoder, entropy network and synthesis all integer | the channel-autoregressive × checkerboard entropy model at **decode-equivalence**: the encoder and every decoder compute the same σ, so the same distribution for every element, at any thread count; bit-identical on every machine. The committed conformance vector has synthetic weights, so it demonstrates decode-equivalence and states no quality figure | `int_mlicpp.rs` (one-file model bundle sink-supplied; `tools/wai_int_mlicpp_export.py`) |
| `wai.image.jpegai` | JPEG AI (ISO/IEC 6048-1:2025) codestream → RGB — integer entropy front end (mANS, with an integer scale-index derivation) + 16-bit-PTQ integer hyper-decoder, context-model and synthesis networks + dequantisation, latent refinement, post-filters (the eICCI CNN among them) and colour conversion in IEEE-754 float with a fixed evaluation order. Scope: one configuration — 256×256 4:4:4, `model_id` 0 (tools_0), `beta_displacement_log` 0, the high-operation-point synthesis, the eICCI model pair the weight bundle carries, and the other header conditions in [determinism-tiers](extensions/determinism-tiers.md) §5; both decode paths refuse any other codestream. Within it, a sink without the integer weight bundle, or in a non-default floating-point environment, takes a float ONNX decode instead. Only checkpoint constants are sink-supplied | **entropy-consistency** (registry class `NeuralEntropyExact`): the entropy decode, including the scale-index derivation, is integer on both decode paths, so the decoded symbols reproduce on every machine. The exact decoder also pins one reconstruction to RGB, reproducible across machines under the build and floating-point conditions in [determinism-tiers](extensions/determinism-tiers.md) §5–§6 (it checks the floating-point environment before decoding) — a property of that decoder, not of the capability, so `wai.image.jpegai` does not back `replicate` (§7). On one reference codestream the exact decoder stays within max\|Δ\| ≤ 2 of the reference decoder (recorded max\|Δ\| = 1 on ~4.4% of samples); evidence in [determinism-tiers](extensions/determinism-tiers.md) §6. **Downgraded in v0.4.0:** this row previously claimed decode-equivalence, and the registry classed the capability `StandardDefined` | `jpegai_decode.rs` (+ `jpegai_{bitstream,mans,pipeline,synthesis,icci}.rs`) |
| `wai.generate.spectral` | deterministic integer GENERATIVE synthesis (`intent=create`) — prompt+seed → integer-LUT spectral composition, no float/ONNX | byte-identical on every machine, pinned to (model, prompt, seed, steps); the create-side of "conformance is a hash" (generation-equivalence) | `generate.rs` (small pinned model bundle) |
| `wai.neural.elic` | checkerboard context model (hyperprior + 2-pass spatial context) | the checkerboard-context entropy class (arXiv 2203.10886) | `neural/context.rs` |
| `wai.neural.mlicpp` | MLIC++ multi-reference entropy (channel-AR slices + checkerboard + global attention) | the multi-reference entropy model (arXiv 2307.15421) | research code, `neural/mlicpp.rs` |
| `wai.neural.mlicv2` | MLICv2 / MLICv2+ (2025), MetaFormer transform | shares the mlicpp route | research code, `neural/mlicpp.rs` |
| `wai.neural.cmic` | Content-Adaptive Mamba (ICLR 2026), SSM transform + SCCTX entropy | shares the mlicpp route | arXiv 2508.02192, `neural/mlicpp.rs` |
| `wai.neural.glc` | Generative Latent Coding — categorical / VQ entropy | ultra-low-bitrate image coding | research code, `neural/glc.rs` |
| `wai.semantic.clip` | a CLIP-embedding package regenerated by any conforming sink generator (the generative-semantic path, `semantic.rs`) | conforms by similarity above a declared threshold, never by bytes (`Behavioral`, tier `Perceptual`); generators are sink-supplied | `semantic.rs` |
| `wai.semantic.perco2` | PerCoV2 ultra-low-bitrate generative image coding | as `wai.semantic.clip`; its model licence is non-commercial (CC BY-NC-SA), so it is licence-flagged and never a default | `semantic.rs` |

*Video:*

| capability | model | operating point or property | source |
|---|---|---|---|
| `wai.neural.video_bmshj2018` | per-frame bmshj2018 | per-frame image coding; decodable in a browser on WebGPU | CompressAI |
| `wai.neural.dcvc_rt` | DCVC-RT (2025), inter-frame with reference-feature propagation | inter-frame neural video; its paper reports real-time 1080p and 4K decode on a GPU | research code, `neural/video_dcvc.rs` |
| `wai.neural.dcvc_fm` | DCVC-FM (feature modulation), wide single-model quality range (RGB + YUV) | a wide single-model quality range; shares the inter-frame route | research code, `neural/video_dcvc.rs` |
| `wai.video.int_motion` | DETERMINISTIC INTEGER inter-frame — I-frame + integer block motion-compensated P-frames + integer-quantized residual over rANS, no float/ONNX; motion is whole-pel or **half-pel** (`mv_precision`), the latter predicting at the half-grid by integer bilinear interpolation (`(a+b+1)>>1`, `(a+b+c+d+2)>>2`) so real footage — which pans a non-integer number of pixels per frame — costs fewer residual bits while staying byte-exact | byte-identical frame-by-frame on every machine (*frame-state-equivalence*, a per-frame portable hash); the temporal extension of the integer contract, where a float inter-frame decode desyncs down the GOP | payload `WIV1`, keyframe 0; `int_video.rs` (the encode tools write a JSON form a converter reads) |
| `wai.video.int_hyper` | NEURAL-KEYFRAME inter-frame — the I-frame is the LEARNED integer codec `int_hyper` decoded byte-exact, the P-frames are integer block-MC residuals on top | a learned keyframe you can put in a receipt + integer inter-frames; whole decode integer, byte-identical everywhere | payload `WIV1`, keyframe 1 (a nested `WIH1`); `int_video::decode_pframes_typed` ∘ `int_hyper_synth::decode_conditional_checked` |
| `wai.video.manifest` | ATTESTED PER-SESSION MANIFEST — the OTT delivery layer, not a codec: a personalised HLS manifest (ads stitched at `EXT-X-DISCONTINUITY`, blackouts + geo enforced) rendered as a **pure function of the session *class*** `(content, entitlement, geo, blackout set, ad pod, window)` rather than the viewer, and sealed in an Ed25519 receipt that binds the JCP grant (capability, joule ceiling, `funds_ceiling` = ad budget), the composition, the stitched ads and the policy applied. A receipt over a WAI playlist MAY carry `render` and `extras`, the rendering parameters and per-segment lines that playlist adds, both bound through `manifest_hash` and through a render digest the signature covers ([http-delivery](extensions/http-delivery.md) §8.6) | **manifest-equivalence**: `manifest_hash == hash(render(segments))` and `session_class_hash` are recomputable by anyone holding the bytes, so the ads, blackouts and compose-work stop being dashboard claims; class-determinism means one generation serves every viewer of the class, and `uj_per_viewer` is the amortised energy that a 1:1-unique generator cannot report below its per-viewer cost. Only the measurement — `joules_micro`, its acquisition class and the class's evidence — is measured-attested: signed, and not re-measurable | `wai_video_manifest.rs` |
| `wai.video.ssai` | ATTESTED SERVER-SIDE AD INSERTION — a **deterministic** ad decision (greedy by price within the avail duration and the funds remaining) plus a per-impression Ed25519 receipt binding the candidate pod, the manifest it belongs to, the served `creative_hash`es and the VAST quartile each reached | **spend-equivalence**: `budget_spent_micros` is recomputed from the *billable* impressions and rejected if it exceeds the grant's `funds_ceiling` — an overspent receipt fails verification; every creative and quartile is checkable, so a buyer and seller reconcile impressions from the same bytes instead of two logs that only a dashboard joins. Only the measurement — `joules_micro`, its acquisition class and the class's evidence — is measured-attested: signed, and not re-measurable | `wai_video_ssai.rs` |
| `wai.video.keys` | ATTESTED CONTENT-KEY RELEASE — the DRM licensing decision as a record: every key release, and every refusal, is an Ed25519 receipt binding the KID, the DRM system, the entitlement demanded and the window enforced, with a BLAKE3 `key_commit` proving *which* key was released without carrying it | **decision-equivalence**: `verify()` re-runs the release policy over the receipt's own `(grant capability, entitlement, issued, expiry)` and rejects any misreported outcome, and enforces granted-iff-committed — so "keys went only to entitled, unexpired sessions" is checkable against the signed log, not asserted. Only the measurement — `joules_micro`, its acquisition class and the class's evidence — is measured-attested: signed, and not re-measurable | `wai_video_keys.rs` |
| `wai.video.package` | DETERMINISTIC PACKAGING + ATTESTED SEGMENTMAP — a source sliced into content-hashed segments on GOP boundaries; the ordered `(index → content_hash)` map is a signed Merkle commitment, and identical content (ads, bumpers, replays, simulcast overlap) dedups to one stored/encoded copy | **map-equivalence + verifiable dedup**: `segment_map_root` lets a `wai.video.manifest` reference be checked against the package that produced it, and `verify()` recomputes `unique_segments`/`stored_bytes` from the segments so the storage/encode saving cannot be overstated; `uj_per_unique_segment` is the encode-once energy. Only the measurement — `joules_micro`, its acquisition class and the class's evidence — is measured-attested: signed, and not re-measurable. Version 2 ([packaging](extensions/packaging.md) §9) binds the policy digest and one stage index per component, names each object by its jwp-receipts §2.1 content hash, and covers every media class a packager packs; the string keeps `video` for history | `wai_video_package.rs`, `package.rs` |
| `wai.video.channel` | VIRTUAL LINEAR / FAST ASSEMBLY — a continuous schedule of assets over wall-clock time, sliced into attested playback windows; windows chain via `parent_receipt_hash`, so a channel's history is a hash chain of program-as-run records | **schedule-continuity + recomputable program-as-run**: `verify()` rejects a schedule with a gap or overlap (no dead air, no double-booking) and recomputes `aired` from the schedule and the window, so what actually aired is checkable by a licensor, advertiser or regulator against the signed chain rather than an operator-held play-out log. Only the measurement — `joules_micro`, its acquisition class and the class's evidence — is measured-attested: signed, and not re-measurable | `wai_video_channel.rs` |

**Sink-architecture note:** the inter-frame neural video codecs (DCVC-RT,
DCVC-FM) decode through `ort` and are **CPU-capable** for native sinks —
their real-time figures come from GPU execution, but a GPU is not required
to decode. They are heavier than per-frame image codecs (a temporal net
plus reference-feature state), so a browser sink that cannot host the
temporal net should declare only `wai.neural.video_bmshj2018` (or another
browser-decodable neural video codec) for the `video` media class. The
choice is the deployer's, advertised per sink — the envelope is identical.

**Scope of the neural rows.** A neural image or video capability is an
option a deployer may declare and a sink may advertise; none is in the
floor or the recommended set, and none is a default.

Each capability owns its payload format. Decoder implementations
(Python source-side, ONNX-export, browser onnxruntime-web) live in
`tools/wai_<capability>_*.py` and `wai-web/demo/`. New neural
capabilities are added by registration; the WAI envelope is unchanged.

See [`extensions/jwp-receipts.md`](extensions/jwp-receipts.md) for the
receipt-chain sub-spec covering per-object content hashes, per-group
Merkle roots, Ed25519 group signatures, and cross-hop preservation; an
object's figure, acquisition class and origin bound into its Merkle leaf,
and a group total's class signed under a labelled group payload; the
typed receipt classes beside delivery (`energy`, `lawful-basis`,
`log-inclusion`, `measurement`, `operating-point`, `stage`, `delivery-gap`,
`reconstruct-session`) carried as `claim` receipts; one receipt chain per publisher key and track, and
an object not delivered accounted for by its receipt, verified by inclusion; and a second, weaker binding class (a registry-named soft
binding with a resolution endpoint) whose recovered receipts attest the
ancestor, never the object in hand.
See [`extensions/energy-measurement.md`](extensions/energy-measurement.md) for
how a receipt's `joules_micro` is acquired: the required acquisition class
(`CalibratedInstrument` / `OnChipCounter` / `ModelBased` / `Estimator`, with the
legacy `HwShunt` still accepted when read), the declared uncertainty — a
standard relative uncertainty and its integration window, never less than the
`0.5/√3` µJ (≈ 0.289 µJ) that rounding the figure to whole microjoules adds, so
`joules_micro × relative_ppm ≥ 288 676` — that both hardware classes must carry, the counter filtering state and calibration reference that
distinguish them, the bracketed marginal protocol (a burst whose window opens
from rest, divided by the work the window paid for, from brackets in which the
power reader's rail was live), the attribution-quality rule that decides whether
a figure may be published at all, the signed measurement report that publishes a
figure's bracket, build and power reader beside the receipt it names, and how the
receipt families that sealed a figure before the class existed carry it as an
optional signed label that leaves every issued receipt's bytes unchanged. A
`joules_micro` of `0` is the unmetered marker, never a figure of zero joules.
See [`extensions/confidentiality.md`](extensions/confidentiality.md) for the
private-channel sub-spec: MLS (RFC 9420) group keys + SFrame (RFC 9605)
object encryption, the byte-level framing behind §6.
See [`extensions/reconstruction-binding.md`](extensions/reconstruction-binding.md)
for the reproducible-reconstruction binding: a C2PA / JPEG-Trust hard binding
over the *deterministically decoded pixels* of a neural-coded asset (SHA-256 over
the canonical RGB plus the descriptor that pins the reconstruction) — the
reconstruct-side companion to `provenance`, applicable to the byte-exact
`wai.neural.int_*` codecs and, over the exact decoder's pinned reconstruction
only, to `wai.image.jpegai`.
See [`extensions/energy-binding.md`](extensions/energy-binding.md) for the
companion energy assertion: the joules a reconstruction cost, bound to that
reconstruction, with the acquisition class and coverage C2PA 2.4's
`c2pa.environmental-sustainability` has no field for.

### Prior artifacts, general tensors and derivation operations

| capability | payload | status · extension |
|---|---|---|
| `wai.prior.nnc` | a trained network's parameters as an ISO/IEC 15938-17:2024 (NNC, 2nd edition) bitstream | a registered wire form a pin MAY declare (`prior.form`); its digest then covers the coded bitstream. Optional, not recommended: royalty status is not established, and cross-decoder exactness of the reconstruction is not established, so a `replicate` over a neural capability cannot rest on a prior read out of it. As a payload it takes the replicate class of a standard whose decoder conformance the registry has not checked · [prior-carriage](extensions/prior-carriage.md) |
| `wai.prior.set` | the listing of a parameter set of one or more files: each file's SHA-256 and path (§3.1) | a registered wire form a pin MAY declare (`prior.form`), and the form in which a set of several files is pinned: the pin covers the listing, and each line covers one file. Reading a set through its listing compares digests and hands the decoder each file's bytes unchanged, so the registry lists it as established exact, and a `replicate` MAY rest on a pin in it. WAI-native, royalty-free; as a payload its output is its own bytes (`StandardDefined`) · [prior-carriage](extensions/prior-carriage.md) |
| `wai.tensor.nnc` | general tensorial data (split-inference feature maps, 3D/4D scene parameters) coded with NNC | **candidate**: general-tensor coding is in an unpublished amendment, so the payload format is not fixed and MUST NOT be emitted until pinned (*Candidate capabilities*, below) · [prior-carriage](extensions/prior-carriage.md) |

The ten derivation operations of ISO/IEC 23001-16:2021 (derived visual tracks)
are registered as `wai.derive.*` strings, each with a per-operation determinism
tier. They are **not payload capabilities**: an operation acts on samples another
capability decoded. WAI does not composite; the registration states what a
reproducibility claim over a derived sample needs. The geometric conditions a
tier lists for each operation are read from the operation's title: clause 8 of
ISO/IEC 23001-16, which defines each operation, was not read · [derived-tracks](extensions/derived-tracks.md).

### Interactive, temporal & computed content (extensions)

Media that consumes input, moves over time, or is *computed* from a
compact instruction set. Each registers its own payload format and a
deterministic `*-equivalence` conformance criterion over the
`wai.det.fixed64` fixed-point floor — the reconstruct half of the
standard. See the linked extension for each.

| capability | payload | conformance · extension |
|---|---|---|
| `wai.world.scene` / `wai.world.replay` / `wai.world.live` | WWLD container (op log) | replay-equivalence · [interactive-worlds](extensions/interactive-worlds.md) |
| `wai.world.model` | generative-world transport: a signed **action trace** (the world CausalOp log) + a declared `model_id` (`world_model.rs`) | the wire carries the *action trace*, not the pixels — it replays under canonical total order to one **session hash** on every machine (replay-equivalence), the generated frames vouched for by a signed, joules-accounted provenance receipt. The neural sibling of `.replay`; with an `int_*` renderer the frames hash-converge too (a generative interactive world you can put in a receipt) |
| `wai.mesh.glb` | glTF 2.0 Binary (Khronos) | asset, content-addressed (decoder sink-supplied) |
| `wai.splat.spz` | SPZ (an open-source Gaussian-splat format) | asset, content-addressed; presentation only |
| `wai.audio.scene` | WSCN scene (objects + trajectories) | mixdown-equivalence · [spatial-audio](extensions/spatial-audio.md) |
| `wai.audio.score` | WSCR container (note events) | music as instructions; sink synthesizes, mixdown-equivalence · [score](extensions/score.md) |
| `wai.audio.int_codec` | `WIA1` — a LEARNED 1-D transform codec: integer transposed-conv synthesis + integer IGDN over a factorized-rANS latent (`tools/wai_int_audio_export.py` writes the model and a JSON form a converter reads) | **sample-equivalence**: int16 PCM byte-identical on every machine (a portable PCM hash); the audio analog of `wai.neural.int_hyper`, where a float learned-audio decode desyncs on ~1e-5 round-off. Pure i64, no float/ONNX (`int_audio.rs`) |
| `wai.haptic.signal` | WHAP envelope (intensity/sharpness keyframes) | sample-equivalence · [haptics](extensions/haptics.md) |
| `wai.feed.state` | WWLD, sim `wai.world.sim.feed0` | replay-equivalence · [state-feeds](extensions/state-feeds.md) |
| `wai.avatar.pose` | WWLD, sim `wai.world.sim.skeletal0` | pose-equivalence · [avatars](extensions/avatars.md) |
| `wai.splat.4d` | W4DS container (per-Gaussian keyframed tracks) | frame-state-equivalence · [volumetric](extensions/volumetric.md) |
| `wai.splat.int_codec` | `WIS1` — a LEARNED Gaussian-splat ATTRIBUTE codec: a learned transform over the `[14,S,S]` attribute image (pos/scale/rot/color/opacity), integer synthesis (int_hyper machinery, 14 channels) over a factorized-rANS latent (`int_splat.rs`; `tools/wai_int_splat_export.py` writes the model and a JSON form a converter reads) | **splat-equivalence**: the decoded attribute tensor is byte-identical on every machine (a portable cloud hash) — the int_hyper of 3D, where a float splat-attribute decode desyncs. The float rasterizer stays sink-supplied presentation; the *attributes* are the contract. Pure i64, no float/ONNX |
| `wai.film.linear` | WFLM container (sealed world + camera track + frame schedule) | film-equivalence · [linear-film](extensions/linear-film.md) |
| `wai.film.reel` | WREL container (sealed world + a sequence of shots = camera track + schedule each) | reel-equivalence (film-equivalence with cuts) · [linear-film](extensions/linear-film.md) |
| `wai.ternary.tt` | WQTT container (balanced-ternary MPS cores — packed trits + integer per-column scales) | a **quantum-inspired** (tensor-network) transform ported from the ternary/MPS fabric with ONE change: the per-column scale is an **integer**, so the whole decode is `i64`/`i128` and the reconstruction is byte-identical on every machine — **reconstruction-equivalence** (a portable recon hash), receiptable. Scope: a **provenance** transform, measured at 13.3 dB SNR for 3.8× with six residual layers (`Lossy` tier; the extension's §5). Encoder-side float SVD only · [ternary-tt](extensions/ternary-tt.md) |
| `wai.quantum.circuit` | WQC container (gate op-log — Clifford+T + dyadic phase) | the wire carries a **circuit** (kilobytes, linear in gate count), the sink CLASSICALLY simulates it in a fixed-point **complex** floor (`Amp` = i64 re/im), and the reconstructed `2^n`-amplitude statevector is byte-identical on every machine — **statevector-equivalence** (a portable statevector hash); measurement is sampled by the pinned `splitmix64` so a shot histogram at a pinned `(seed, shots)` converges too (**shot-histogram-equivalence**). The dyadic phase ladder gives the **QFT** ("quantum FFT"), transported as a circuit and reconstructed byte-exact. No quantum hardware — NISQ noise breaks the determinism the standard rests on; this makes a quantum *computation* transportable + receiptable, with **no** speedup claimed · [quantum-sim](extensions/quantum-sim.md) |

And the verify half — a signed, energy-accounted lineage for opaque
captured content WAI does **not** reconstruct (content-addressing +
receipts + measured joules): [`extensions/provenance.md`](extensions/provenance.md)
(`wai.asset.provenance`).

The same verify half, specialized to **quantum operations** — a signed,
content-addressed, joule-metered, grant-authorized receipt of a calibration run
(`wai.quantum.calibration`) or a dispatched QPU job (`wai.quantum.job`), where a
job binds the exact calibration state it ran under and `verify()` enforces the
grant's joule ceiling (gate ⊗ meter ⊗ receipt in one object). WAI owns the
receipt; the sink's control system runs the hardware:
[`extensions/quantum-ops.md`](extensions/quantum-ops.md).

### Payloads of the WAI-native integer codecs

This revision pins a payload format to each WAI-native integer codec whose row named
its decoder and no byte layout. The rows of `wai.video.int_motion`,
`wai.audio.int_codec` and `wai.splat.int_codec` also named the tool that wrote a
bitstream, a JSON document, which this section's opening called the bytes the codec
normally emits; those of `wai.neural.int_hyper` and `wai.video.int_hyper` named their
decoders only. The pin completes these registrations as §8 permits (*Completing a
registration*). [integer-payloads](extensions/integer-payloads.md) §3–§6 defines
each layout and its decode, §7 the checks a sink makes, and §8 the closed list of
codes it refuses with. The rows above keep their headings.

| capability | payload format | parameter set |
|---|---|---|
| `wai.video.int_motion` | `WIV1`, keyframe 0 | none |
| `wai.video.int_hyper` | `WIV1`, keyframe 1 (a nested `WIH1`) | several files: `g_s.whm`, `h_s.whm`, `entropy.json` |
| `wai.neural.int_hyper` | `WIH1` | several files: `g_s.whm`, `h_s.whm`, `entropy.json` |
| `wai.audio.int_codec` | `WIA1` | one file: `model.wam` |
| `wai.splat.int_codec` | `WIS1` | one file: `g_s.whm` |

From this revision on, the format of each is fixed as §8 states. The JSON the encode
and export tools write is a converter input
([integer-payloads](extensions/integer-payloads.md) §10), never a payload: an encoder
MUST NOT carry it under these strings, and a sink refuses it (`bad_magic`).

*Envelopes the pin makes non-conforming* (they still parse): a `WAI1` whose
`capability` or `fallback` is one of these strings and whose payload is not that
string's format, a tool's JSON form among them; and a `WAI2` with a component of one
of these strings whose bytes are not its format. A sink refuses such a payload when it
decodes it, and does not try the fallback
([integer-payloads](extensions/integer-payloads.md) §11).

### Immersive audio, cinema, volumetric, haptics, HDR (registered, sink-supplied)

The same "name the capability, the sink supplies the decoder" pattern extends
to the standard carriers of the immersive / theatrical / volumetric media a
full distributor needs. Each names an open reference where one exists; the
proprietary and SEP-pool entries are flagged, never defaults. Payload is the
named standard's canonical bytes.

| class | capabilities |
|---|---|
| immersive audio | `wai.audio.iamf` (IAMF, royalty-free) · `wai.audio.ac4` (AC-4, ETSI TS 103 190, proprietary) · `wai.audio.ivas` (3GPP IVAS, SEP-pool) · `wai.audio.mpegh` (MPEG-H 3D Audio, SEP-pool) · `wai.audio.avs3` (AVS3-P3 / Audio Vivid, SEP-pool) — the last two with the broadcast codecs below |
| cinema mastering | `wai.image.jp2` · `wai.video.j2k` (JPEG 2000 / DCP, OpenJPEG) |
| volumetric / 3D | `wai.mesh.draco` (royalty-free) · `wai.pc.gpcc` (G-PCC, ISO/IEC 23090-9) · `wai.pc.vpcc` (V-PCC, ISO/IEC 23090-5) — MPEG point clouds · `wai.splat.sog` (Gaussian-splat compression; a **candidate**, no payload format pinned — *Candidate capabilities*, below) |
| haptics | `wai.haptic.mihs` (MPEG-I Haptics, ISO/IEC 23090-31; its 23090-33 ships bit-exact conformance vectors) |
| HDR dynamic metadata | `wai.meta.hdr10plus` (SMPTE ST 2094-40 dynamic metadata, royalty-free) · `wai.meta.dovi` (SMPTE ST 2094-10 dynamic metadata, proprietary) — **companion descriptors** to a video payload, not standalone media |

**Point clouds: what the two registrations name.** `wai.pc.gpcc` names G-PCC,
ISO/IEC 23090-9, and `wai.pc.vpcc` names V-PCC, ISO/IEC 23090-5; each payload is
a bitstream of the part it names. A bitstream of another part of MPEG-I is not a
payload of either unless it is also a bitstream of that part. Enhanced G-PCC,
ISO/IEC 23090-38, is such another part, with its own conformance part, so it
would be registered, if at all, under a capability string of its own (§8). Neither
registration names an edition, so each carries the edition of its part current
when it was registered, on 2026-06-17 (§8, *Editions*). Which edition that was is
unresolved: the research registry records it as open, as it does whether
`wai.pc.vpcc` carries the V3C bitstream of MIV. Until a revision records the
edition, what either registration admits beyond the tools of its part's first
edition is unspecified: a sink that advertises either capability is not known to
decode a bitstream that uses a later tool, and an encoder that emits one makes no
claim a sink can rely on. The amendments in preparation that add Gaussian splats
to both parts were not published on the registration date, so each needs a
capability string of its own (§8). Whether V-DMC, which the fourth edition of
ISO/IEC 23090-5 makes a V3C application, is admitted turns on the unresolved
edition.

**Candidate capabilities.** A *candidate* is a registered capability string whose
payload format no revision has pinned. The candidates are `wai.tensor.nnc`
(general-tensor coding is in an unpublished NNC amendment; *Prior artifacts*
above) and `wai.splat.sog` (registered for a family of Gaussian-splat formats; no
specification of the format is cited). A conforming encoder's payload is the
canonical byte stream of its capability (§7), and a candidate has none. So, until a
revision pins a candidate's payload format:

- an encoder MUST NOT emit it: not as a `WAI1` `capability` or `fallback`, and
  not as a `WAI2` component in either role;
- a sink MUST treat it as unsupported: it takes neither step 3 nor step 5 of §4
  with a candidate, and §7.1's selection never picks a component of one.

The revision that pins a candidate states its payload format, and §8 fixes that
format from that revision on. This rule enforces the candidate registrations as
they were made (§8); it changes no payload format and no role. It makes non-conforming
a `WAI1` envelope that names a candidate as its `capability` or its `fallback`, and
a `WAI2` envelope with a component of a candidate; such envelopes still parse. The
reference implementation lists the candidates in `codecs::is_candidate`, reports
such envelopes from `Wai::check_candidates` and `WaiMulti::check_candidates`, and
never dispatches or selects a candidate: `Wai::dispatch_verified`,
`WaiMulti::pick`, `WaiMulti::primary` and the C `wai_envelope_select` take only a
capability `codecs::is_dispatchable` accepts (§4's dispatchable). Its `wai wrap` tool, its C
envelope packers (`wai_envelope_pack`, `wai_envelope_pack_multi`) and its browser
writers (`pack`, `packMulti`) refuse to write one.

### Refinement capabilities (staged delivery)

| capability | staged form | payload | registry class |
|---|---|---|---|
| `wai.image.int_refine` | residual | `WIR1` ([staged-delivery](extensions/staged-delivery.md) Appendix A.1), units RGB8 | StandardDefined |
| `wai.video.int_refine` | residual | `WIR1`, units RGB8 frames | StandardDefined |
| `wai.audio.int_refine` | residual | `WIR1`, units `i16` PCM | StandardDefined |
| `wai.splat.int_refine` | residual | `WIR1`, units attribute tensors | StandardDefined |

Each is **refinement-only** (§8): its payload is applied, in a declared order, to
the output of the stage below it, bound to that output's producer by content hash,
and never decoded on its own. It is carried only as a `WAI1` with `refines` (§3),
or as a `WAI2` component with `role: "refinement"` (§7.1). It is never a
`capability` or `fallback` that §4 dispatches, a selectable rendition or a
companion. Each is royalty-free and WAI-native, its payload format is pinned by
this registration, it reads no parameter set, and its tier is `Lossy`: its residual
is quantised. Its next stage is exact given the stage below, so its class is
StandardDefined and its output is as exact as the base it refines
([staged-delivery](extensions/staged-delivery.md) §5).

### Base admission

A capability applied to the output of a base — a refinement-only capability, or an
enhancement layer — applies only over the bases listed for it here. A later
revision MAY add a base, which widens the capability (§8).

| capability | admitted bases |
|---|---|
| `wai.image.int_refine` | `wai.neural.int_hyper`, `wai.neural.int_mlicpp`, `wai.neural.int_synth`, `wai.image.png` |
| `wai.video.int_refine` | `wai.video.int_motion`, `wai.video.int_hyper` |
| `wai.audio.int_refine` | `wai.audio.int_codec`, `wai.audio.flac` |
| `wai.splat.int_refine` | `wai.splat.int_codec` |
| `wai.video.lcevc` | `wai.video.avc`, `wai.video.hevc`, `wai.video.vvc`, `wai.video.evc`, `wai.video.evc.baseline`, `wai.video.vp9`, `wai.video.av1`, `wai.video.av1.lossless` (*Enhancement layers (LCEVC)*, below) |

A refinement is admitted over a base when its `refines.base` names an envelope that
a sink decoded through one of these capabilities. The reference implementation
holds this table once (`codecs::BASE_ADMISSION`), and every admission check reads
it.

### Broadcast and streaming codecs (registered, sink-supplied)

The elementary video and audio bitstreams that terrestrial and satellite
broadcast, streaming and device playback carry. Registering them lets one
envelope name the bitstream a distribution chain already holds, and dispatch it
with a declared fallback or a sibling rendition. Every entry is
**sink-supplied** — the reference implementation bundles no decoder for any of
them — and **none is in the mandatory floor or the recommended set**:
registration never makes a capability a default. Payload is the named
standard's canonical elementary bitstream in the framing given; WAI does not
re-frame it.

| capability | standard | payload (canonical bytes) | licence |
|---|---|---|---|
| `wai.video.av2` | AV2 (AOMedia) — AV2 Bitstream & Decoding Process Specification v1.0.0, 28 May 2026 | the length-delimited bitstream of that specification's Annex B (each OBU preceded by its `leb128` byte length) | royalty-free |
| `wai.video.vvc` | VVC — ITU-T H.266 \| ISO/IEC 23090-3 | H.266 Annex B byte stream | SEP-pool |
| `wai.video.hevc` | HEVC — ITU-T H.265 \| ISO/IEC 23008-2 | H.265 Annex B byte stream | SEP-pool |
| `wai.video.avc` | AVC — ITU-T H.264 \| ISO/IEC 14496-10 | H.264 Annex B byte stream | SEP-pool |
| `wai.video.vp9` | VP9 Bitstream & Decoding Process Specification | VP9 frames in the IVF frame container (FourCC `VP90`) | royalty-free |
| `wai.video.evc` | EVC — MPEG-5 Part 1, ISO/IEC 23094-1, any profile | raw bitstream file storage format, ISO/IEC 23094-1 Annex B | SEP-pool (Main-profile tools) |
| `wai.video.evc.baseline` | EVC, Baseline profile only | as `wai.video.evc`, Baseline-conformant | royalty-free by design |
| `wai.video.avs3` | AVS3 video — AVS3-P2, T/AI 109.2-2021, IEEE 1857.10 | AVS3-P2 elementary bitstream | SEP-pool |
| `wai.video.lcevc` | LCEVC — MPEG-5 Part 2, ISO/IEC 23094-2 | LCEVC enhancement as the ISO/IEC 23094-2 Annex B byte stream (start-code-delimited NAL units) — **companion-only**, see below | SEP-pool |
| `wai.audio.mpegh` | MPEG-H 3D Audio — ISO/IEC 23008-3 | MHAS packet stream (MPEG-H 3D Audio Stream) | SEP-pool |
| `wai.audio.avs3` | AVS3 audio / Audio Vivid — AVS3-P3, T/AI 109.3-2023; GY/T 363-2023; GB/T 46271-2025 | AVS3-P3 elementary bitstream | SEP-pool |
| `wai.audio.aac` | AAC-LC, HE-AAC, HE-AAC v2 — ISO/IEC 14496-3 | ADTS stream (ISO/IEC 13818-7 / 14496-3); Extended HE-AAC is not this capability | SEP-pool |

*SEP-pool* means the standard's essential patents are licensed for royalties —
through patent pools, a single standard-essential-patent licensor, or FRAND
holders; it is the `patent-pool` licence class (*Licence classes*, below).
Per-entry sources and caveats are in the research registry linked
below. Each is tiered `Lossy` (its distribution role; the lossless modes some
of them have are signalled in the bitstream, as for AVIF). No model WAI runs
is involved in any of them. For replication (§7) they fall in three classes:

- **`StandardDefined`** — AVC, HEVC, VVC and VP9. Each standard's decoder
  conformance requires every sample a conforming decoder outputs to equal the
  value its decoding process produces (H.264 §C.4; H.265 and H.266 §C.5.1; VP9
  §8.1), so every conforming decoder outputs the same cropped decoded pictures.
  The guarantee stops at those pictures: film-grain synthesis an SEI message
  signals, colour conversion and other post-decoding processing are outside it.
- **`StandardTolerance`** — AV2. As for AV1, the output is identical unless the
  payload signals film grain; then grain synthesis is held only to the absence
  of "perceptually significant differences" from the reference (AV2 v1.0.0
  §7.1).
- **`StandardUnclassified`** — EVC (both entries), AVS3 video, LCEVC, MPEG-H 3D
  Audio, AVS3 audio and AAC. Their conformance documents were not read for this
  revision, so the registry promises no byte identity for them. For AAC, three
  decoders on one host gave three different PCM streams from one ADTS stream.
  LCEVC's decoding process includes a signalled dither step (ISO/IEC 23094-2,
  clause 8.11). Any of them moves to `StandardDefined` only with the conformance
  clause that fixes its output.

The clauses behind each class, what was not read, and the cross-decoder
measurements are in the research registry.

**Dispatch.** These dispatch exactly as §4 describes. Because none is a floor or
recommended capability, no sink can be assumed to hold one: an encoder
addressing sinks it does not control SHOULD ship such a payload in a `WAI2`
envelope (§7.1) together with a recommended-set rendition of the same content,
and that rendition is what keeps the file from being inert at a conforming
sink. A v1.0 `fallback` alone does not: dispatch step 5 decodes the *same*
payload with the fallback's codec (§4), so a fallback helps only where that
codec's decoder accepts the primary's bitstream (`wai.video.evc.baseline` →
`wai.video.evc` is such a pair; an HEVC, VVC or AV2 bitstream is not an AV1
bitstream). A deployer that must not ship pool-licensed decode filters on the
licence class below, not on a hand-kept list.

**AV2 placement.** AV2 is licensed royalty-free by its standards body (footnote ⁶
of the research registry records third-party assertions), with a published
specification and a reference decoder (AVM v1.0.0). It is registered here and **not** in the
recommended set, because that set states what every conforming sink SHOULD
implement, and in September 2026 that is not yet a reasonable ask: AVM is a
reference model, a production software decoder is still in development, and
hardware decode in shipping devices is not confirmed. Promotion to the
recommended set is a MINOR revision (§8) once deployable decode exists; the
capability string and its payload framing do not change. Until then an AV2
payload SHOULD travel in a `WAI2` envelope with a `wai.video.av1` rendition of
the same content.

**Enhancement layers (LCEVC).** An LCEVC enhancement is not a picture: it holds
residuals that an LCEVC decoder adds to the upscaled output of **one specific
base encode**. This registration names its **base codecs**: AVC, HEVC, VVC,
EVC, VP9 and AV1, as the capabilities that name those bitstreams —
`wai.video.avc`, `wai.video.hevc`, `wai.video.vvc`, `wai.video.evc`,
`wai.video.evc.baseline`, `wai.video.vp9`, `wai.video.av1` and
`wai.video.av1.lossless` — and not AV2, AVS3, JPEG 2000 or the WAI-native
integer codecs. The list is WAI's, not the LCEVC standard's: ISO/IEC
23094-2:2021 takes its decoded base picture from "a base decoder" (definition
3.17) and names no base codec among its normative references. A later revision
MAY add a base, which widens what the enhancement layer applies to (§8). WAI
carries it one of two ways.

- **Embedded.** The enhancement rides inside the base bitstream (SEI /
  registered user data). The payload is the base codec's bytes, under the base
  capability; a sink without LCEVC decodes the base and ignores the enhancement
  under the base codec's own rules. No `wai.video.lcevc` component exists.
- **Separate.** The enhancement, in the byte-stream framing given below, is a
  `WAI2` companion (§7.1): `role = "companion"`, `companion_of` naming the base
  capability. This is the shape of LCEVC's dual-track storage in ISO/IEC
  14496-15, where an enhancement track refers to exactly one base track, and the
  companion mechanism fits it with one added rule that binds in both directions:
  `companion_of` MUST name one of its base codecs and resolve to **exactly
  one** selectable rendition of it in the envelope, and that rendition MUST NOT
  be the base of any other enhancement layer. `companion_of` names a capability,
  not a component, so with two renditions of the base capability it cannot say
  which encode the residuals belong to; and a sink applies every companion bound
  to the rendition it picks (§7.1), so two enhancement layers on one base would
  add two sets of residuals to one picture. Either way the picture is
  corrupted. Such an envelope does not conform; a sink MUST NOT apply its
  enhancement layers, and MAY still present a base rendition.

The separate carriage's payload is the enhancement in the byte-stream form of
ISO/IEC 23094-2 Annex B, where a start code prefix (`0x000001`) precedes every
NAL unit. ISO/IEC 23094-2 defines an LCEVC bitstream as either a NAL unit
stream or a byte stream (definition 3.2); the byte stream is the form that
delimits its own NAL units, so it is the one `wai.video.lcevc` names. A NAL
unit stream without start code prefixes (for example with each NAL unit
preceded by its length) is not a `wai.video.lcevc` payload; carrying that form
would need its own capability string (§8).

**Companion-only capabilities.** `wai.video.lcevc` and the HDR dynamic-metadata
descriptors `wai.meta.hdr10plus` and `wai.meta.dovi` are **companion-only**:
their payload means nothing without the base it accompanies. An HDR descriptor
applies to a rendition whose payload is video (§7.1, rule 2), `wai.video.lcevc`
to one rendition of one of its base codecs (above). Every other registered
capability is **standalone media** and is never carried as a companion (§7.1).
A companion-only
capability MUST NOT be a v1.0 `capability` (a `WAI1` carries one payload, so
there is nothing for it to accompany) or a v1.0 `fallback` (dispatch step 5
would decode it on its own), nor a v1.1 selectable rendition; a sink never
dispatches it standalone. A `WAI1` naming one as its `capability` is inert: its
payload is companion data. A `WAI1` naming one only as its `fallback` does not
conform, and a sink MUST NOT take dispatch step 5 with it; step 3 is unaffected,
so a sink that advertises the envelope's `capability` still decodes it (§4).
The reference implementation checks these rules in
`Wai::check_component_roles` (v1.0) and `WaiMulti::check_component_roles`
(v1.1: the companion rules of §7.1, including both enhancement-layer rules
above), dispatches `WAI1`
envelopes through `Wai::dispatch_verified`, which applies both sentences
above, never returns a companion-only component from `pick` / `primary`, and
its `wai wrap` tool refuses to write an envelope that fails either check. Parsing stays
format-agnostic: a non-conforming envelope still parses, and the check is what
reports it.

The companion binding carries no per-picture timing. The framing above fixes
the bytes of a separately carried enhancement stream, not how its access units
align picture by picture with its base outside a timed container; this
revision does not pin that alignment, so the embedded form is the unambiguous
one until it does.

**Licence classes.** Every payload and companion capability carries a licence
class — `codecs::capability_license(cap)` ∈ {royalty-free, patent-pool,
proprietary, non-commercial}, with `is_royalty_free(cap)` as the predicate. It is
a registry property (not a per-file manifest claim — a codec's licence can't be
honestly asserted in an envelope's header), so a deployer can audit a title or a
whole catalogue in code: encode only to royalty-free capabilities, or flag every
pool/proprietary one. The classification is exhaustive with no default class: a capability registered
without an explicit class is a registry defect the reference tests reject, so no
codec becomes royalty-free by omission. Where a capability's royalty status is
not established, it is classified conservatively (patent-pool), never
royalty-free; a candidate's class is provisional until a revision pins its
payload format. The same tests hold that every floor and recommended-set
capability is royalty-free.

See [`research/media-codec-registry.md`](research/media-codec-registry.md) for
the per-capability standard / spec number / verified royalty status / reference
library, what each named standard's decoder conformance fixes (§7), and the
remaining spec follow-ups.

---

## 6. Confidentiality (OPTIONAL — encrypted payloads / private channels)

By default a WAI payload is **plaintext**: any sink that advertises the
named capability (§4–§5) can decode it. This is correct for public
media — a shared world, a broadcast film. Some channels are instead
**private**: only an intended group may recover the content, and any
relay or multiplexer that carries it MUST be able to route, meter, and
attest the bytes **without** reading them.

Confidentiality is an OPTIONAL layer that is **orthogonal to the
capability**. It wraps the payload bytes; the capability still names how
to decode the *plaintext* once recovered. A sink decrypts first, then
runs capability dispatch (§4) on the plaintext.

### 6.1 Manifest field

A manifest MAY carry a `security` object:

```json
"security": { "encryption": "mls-sframe", "kid": 0 }
```

- `security.encryption` — encryption scheme id. Absent, `null`, or
  `"none"` ⇒ the payload is plaintext (the default; §1–§5 unchanged).
  `"mls-sframe"` ⇒ the payload is an SFrame (RFC 9605) ciphertext object
  whose group key is agreed by MLS (RFC 9420), following the construction
  of `draft-jennings-moq-e2ee-mls`, an individual Internet-Draft that has
  since expired ([confidentiality](extensions/confidentiality.md) §1).
- `security.kid` — the SFrame key id selecting which sender/epoch key
  opens the object. It is also carried in the SFrame header; the
  manifest copy lets a multiplexer route without parsing the payload.

`media`, `intent`, `conditioning.kind`, and
`model_requirement.capability` continue to describe the **plaintext**
codec. They are NOT encrypted and remain visible for routing —
confidentiality protects the *content*, not the *shape*.

### 6.2 Keys are a requirement of existence

Exactly as the model behind a neural capability is never transported
(§1), the **group key is never transported in the WAI file**. Group
membership is established out of band via MLS; each member derives the
32-byte SFrame base key from the group's exporter secret under the label
`"moq-jwp sframe base key"`. A sink that is not a group member cannot
derive the key and the file is **inert at that sink** — the
confidentiality analog of a missing capability.

This is the content key. Checking a receipt needs no content key, only
the signer's verifying key, which the receipt itself carries
(`signer_pubkey`): where a signer publishes its verifying keys, and how
a verifier confirms whose a key is, is
[`extensions/jwp-receipts.md`](extensions/jwp-receipts.md) §7.

### 6.3 Relay transparency and attestation

A WAI relay or multiplexer that carries an encrypted payload MUST treat
it as opaque: it MUST NOT require the plaintext to route, meter, or
attest, and MUST NOT attempt capability dispatch on ciphertext. The JWP
receipt chain ([`extensions/jwp-receipts.md`](extensions/jwp-receipts.md))
commits over the **ciphertext** content hash, so each object's integrity,
and its delivery under the publisher's signature, are attested end-to-end
without any relay holding the content key. Each object's figure, class and
origin are signed where its leaf is labelled, and the group total, its class,
its coverage and the parent link are signed under the labelled group payload
(jwp-receipts §2.2–§2.3). An object's figure and origin beside a legacy leaf
are the publisher's statement, unsigned beside the signed bytes; such an
object's figure is signed only where an `energy` claim restates it
(jwp-receipts §2.4.5). A legacy group payload signs the total, but no class,
coverage or parent link.
Content secrecy (SFrame/MLS) and delivery attestation (JWP) are independent
layers over the same bytes and compose: a relay forwards the exact sealed
bytes and their receipts unaltered, signs nothing in the publisher's name
(jwp-receipts §4.3), and never learns what the bytes say. Checking the
attestation needs the publisher's verifying key, which the receipt carries,
never the content key; whose key it is, jwp-receipts §7 confirms.

### 6.4 Decrypt-then-dispatch

A confidentiality-aware sink MUST, before running §4:

1. If `security.encryption` is absent / `null` / `"none"`, proceed to §4
   on the payload unchanged.
2. Else, if the sink does not implement the named scheme, or holds no
   key for `security.kid`, the file is **inert at this sink** — return a
   clear "no key / unsupported encryption" error. It MUST NOT feed
   ciphertext to a codec. (The reference dispatchers in `wai-rs` implement
   no scheme and report `unsupported_encryption`, for a `WAI2` envelope
   against every entry.)
3. Else recover the plaintext payload via the named scheme (for
   `mls-sframe`: SFrame `open` under the member's base key), then run §4
   on the recovered plaintext.

A sink that does not implement confidentiality at all MUST refuse an
envelope whose manifest carries a non-`none` `security.encryption`
cleanly, rather than mis-decode ciphertext.

Byte-level SFrame framing, the SFrame key schedule, and the MLS binding
(including per-sender keying for multi-sender channels) are specified in
[`extensions/confidentiality.md`](extensions/confidentiality.md). Its
implementation is a companion implementation, outside this tree and under
its own licence: the `moq-sframe` (SFrame) and `joule-comm-mls` (MLS
exporter → base key) crates, with browser bindings in
`joule-comm-mls-wasm`; `Track::e2e_encrypted` is the switch that selects
this path. Live: https://wai.transaction.science/private-channel

---

## 7. Conformance

- A **conforming sink** MUST: implement the container (§2), capability
  dispatch (§4), and the mandatory floor (§5).
- A **conforming encoder** MUST emit containers (§2) that any
  conforming sink can read; the payload MUST be the canonical byte
  stream for the registered capability.
- A decode is conformant by the criterion its capability's registry class
  (below) sets. A `StandardDefined` or `NeuralIntegerExact` capability is
  verified by **bit-exact decode-equivalence**: against the in-crate decoder or
  wrapped reference library for the capabilities the reference implementation
  decodes (`codecs::sink_capabilities()` and the in-crate neural and WAI-native
  decoders), and against the named standard's own conformance bitstreams and
  decoded outputs for a sink-supplied codec. A `GenerativeDeterministic`
  capability is verified by bit-exact generation-equivalence at its pin. A
  `StandardTolerance` capability conforms by its named standard's own
  decoder-conformance criterion around a reference, not by bit-exact equality
  with one library's output; a `StandardUnclassified` capability conforms by its
  named standard's criterion, which this revision has not classified; WAI adds
  no bit-exact requirement to either. A `NeuralEntropyExact` capability
  conforms, as WAI registers it, at the entropy-decoder output (for
  `wai.image.jpegai`, the criterion its named standard sets on the
  reconstruction is not read in this revision).
  `NeuralFloat` capabilities make no claim that their reconstruction is
  bit-exact across sinks, and `Behavioral` ones conform by similarity. The WAI
  Rust reference impl in `wai-rs/` ships a test suite that carries every
  registered capability through the full WAI envelope (`cargo test --lib`,
  `tests/carriage.rs`).

### Replicate-soundness — the integer-entropy contract (normative)

`intent = replicate` promises the reconstruction the capability's registry class
fixes (the classes are below). For every class that backs it except two, that is
a **byte-exact reconstruction**: every conforming sink that decodes the payload
produces the same bytes of the medium (*decode-equivalence*,
[determinism-tiers](extensions/determinism-tiers.md) §2). The two exceptions are
the standard codecs whose named standard fixes less than the bytes
(`StandardTolerance`) or has not been checked (`StandardUnclassified`); over those,
`replicate` promises that standard's decoder conformance and no more (*What
`replicate` promises over a standard codec*, below).

For a **neural** capability `replicate` promises decode-equivalence, and this
revision does not substitute a standard's decoder conformance for it. The one
registered neural capability with a named external standard is
`wai.image.jpegai` (ISO/IEC 6048-1:2025). That standard's conformance part may set a
criterion on the reconstruction — an overview of the standard says the part
"considers the possibility that a decoder is conformant to the standard without
having bit-exact reconstruction" — but this revision has not read it, so holding
`wai.image.jpegai` to the neural rule is a conservative choice pending that
reading, not a finding that its standard leaves the reconstruction unbounded
(`NeuralEntropyExact`, below).
Decode-equivalence is only sound if the decode is byte-identical on
every machine — and that is **not** automatic. A learned
codec's entropy decode desyncs the instant the decoder's scale-bucket / σ
computation drifts from the encoder's by one float ULP: the bucket assignment
flips and the rest of the stream corrupts. So a neural decode that runs **float**
inference through a runtime — which chooses kernels, accumulation order, vector
width and fused multiply-adds per device — is reproducible only on the *same
build*, never guaranteed across hardware.

IEEE-754 basic operations are themselves correctly rounded. Float reproduces
across machines only when one implementation fixes everything a runtime leaves
free: the operations and their order, their precision, no contraction or
reassociation, no library function whose last-place error can reach the output —
and runs in the default floating-point environment (round-to-nearest-even, no
flush-to-zero, no denormals-are-zero), which a host process can change. These are
properties of one build and one process that neither the envelope nor the
registry can state, which is why the rule below requires integer arithmetic.

Therefore a neural capability MAY back `intent = replicate` **only** if its
entire decode path — entropy parameterization *and* synthesis — is
integer / fixed-point, with no float operation anywhere (the **integer-entropy
contract**). That is the only construction by which a neural capability is
registered at `decode-equivalence`. Two weaker properties MUST NOT back
`replicate`:

- **`entropy-consistency`** — the entropy decode (the decoded symbols, and the
  parameters they are decoded with) reproduces on every machine, but later stages
  run float, so the reconstruction may differ between conforming sinks. It is
  necessary for a sound neural `replicate` — without it the decode desyncs — and
  not sufficient, because `replicate` promises the reconstruction, not the
  symbols.
- **A reconstruction pinned by one implementation** — float stages in a fixed
  evaluation order, reproducible across machines only under that
  implementation's build and the process's floating-point environment. Those
  conditions are the ones the envelope and the registry cannot state, so a
  `replicate` resting on them promises something no sink can check from what it
  received.

The reference impl factors the halves of the contract into ORT-free, pure-i64
modules a codec composes: `int_entropy` (the deterministic scale→bucket step
that removes the float-σ desync), `int_synth` (integer synthesis), and the
integer rANS — assembled into a full conditional decode by `int_hyper_synth`
(`wai.neural.int_hyper`: factorized rANS-decode `z` → integer hyper-decoder
`h_s` → σ,μ → integer-σ bucket → rANS-decode residual → `y_hat = r + μ` →
integer synthesis — both bitstreams entropy-coded, no float anywhere). This is
a property of the capability, classified in the
registry as `codecs::capability_replicate_class(cap) -> ReplicateClass`:

- **`NeuralIntegerExact`** — integer by construction; byte-identical on every
  machine; registered at `decode-equivalence`, and the only neural class that
  backs `replicate`. (`wai.neural.int_synth` is synthesis only;
  `wai.neural.int_hyper` is the full conditional codec — σ source, entropy, and
  synthesis all integer; `wai.neural.int_mlicpp` is the channel-autoregressive ×
  checkerboard entropy model on integer networks, so the σ it re-derives before
  each of its `2·K` passes is integer too; `wai.video.int_hyper`,
  `wai.audio.int_codec` and `wai.splat.int_codec` are integer throughout as well.)
- **`NeuralEntropyExact`** — the entropy decode is integer by construction, so
  the decoded symbols are byte-identical on every machine; later stages run float.
  Registered at `entropy-consistency`. An encoder MUST NOT emit `intent =
  replicate` for one of these. (`wai.image.jpegai`: integer mANS entropy decode
  and integer scale-index derivation on both reference decode paths; float
  dequantisation, latent refinement, post-filters and colour conversion. What
  its exact decoder pins, and under which conditions, is stated in
  [determinism-tiers](extensions/determinism-tiers.md) §5. This class is the
  neural rule applied to it, not a statement about its standard's conformance on
  the reconstruction, which this revision has not read; see *`wai.image.jpegai`
  and the standard classes*, below.)
- **`GenerativeDeterministic`** — the *create*-side of byte-exact conformance.
  A generator (`intent = create`) whose entire pipeline is integer/fixed-point
  and pinned to `(model, prompt, seed, steps)`, so the **generated** content is
  byte-identical on every machine. There is no original to reconstruct, yet the
  output hash is portable and receiptable — *generation-equivalence*. The
  fixed-point sampler is mandatory: a float RNG or float sampling step desyncs
  the stream exactly as a float σ does. (`wai.generate.spectral`: a pure-integer
  spectral synthesis — prompt+seed draw integer-LUT sinusoid components by a
  `splitmix64` PRNG; the float `sin()` that bakes the LUT runs once at
  model-build, never at generation.) Distinct from `Behavioral`, which conforms
  by similarity and never reproduces bytes. It generates rather than
  reconstructs, so it does not back `replicate`.
- **`NeuralFloat`** — runs float inference; exact only on the same build. An
  encoder MUST NOT emit `intent = replicate` for one of these; `perceptual` /
  "exact on this build" is the honest claim. The registry claims no determinism
  tier for these.
- **`StandardDefined`** — the reconstruction is specified exactly: by the named
  standard, for the lossless formats whose decoded output it determines
  (`wai.image.png`, `wai.audio.flac`, `wai.text.zstd`, `wai.text.xz`) and for the
  video codecs whose decoder conformance requires every output sample to equal
  the specified decoding process's (`wai.video.avc`, `wai.video.hevc`,
  `wai.video.vvc`, `wai.video.vp9`; clauses in the table below), or by the
  WAI extension that defines the capability's `*-equivalence` over integer /
  fixed-point state (world, score, film, quantum, ternary, the haptic signal, the
  integer inter-frame codec and the other WAI-native capabilities). `replicate`
  promises the same bytes on every conforming sink. An extension's equivalence
  holds given its inputs: where it composes other capabilities' decoded output (a
  `wai.audio.scene` source decoded through `wai.audio.opus`, say), it is only as
  exact as those capabilities' classes. `wai.audio.scene` is the one registered
  capability whose output depends on envelopes it references rather than
  carries, so the reference registry does not report it byte-exact from the
  capability alone (below). No neural decode is in this class, and a
  capability enters it only with a recorded basis — the conformance clause or
  extension that fixes its output (the reference registry tests pin one per
  member); without one, a standard codec is `StandardUnclassified`.
- **`StandardTolerance`** — a standard codec whose named standard, for at least
  some conforming payloads, holds decoders to a criterion around a reference
  rather than to identical output. For Opus and JPEG XL the clauses read state it
  as numeric thresholds. For JPEG and JPEG 2000 the text read points to a
  criterion without stating its bound (T.81 leaves the accuracy of dequantisation
  and IDCT to its Part 2, which was not read; only the table of contents and terms
  of ISO/IEC 15444-4 were read), so this revision states none. For IVAS, an
  implementation based on its floating-point reference code whose output is not
  bit-exact is tested by a non-bit-exact procedure instead: a per-frame
  loudness-difference corridor for stereo and immersive operation, and for mono
  operation the procedure of TS 26.444, which was not read (3GPP TS 26.252
  §7.2.1). For AV1, AVIF and AV2 when film grain is signalled it is the absence
  of "perceptually significant differences" from the reference grain synthesis,
  which the AV1 specification leaves undefined (§7.2: its definition "is beyond
  the scope of this specification") and the AV2 specification leaves to others
  to define (§7.1). `replicate` promises what that conformance promises: every
  conforming sink reconstructs within the standard's criterion — for film-grain
  AV1 and AV2, a criterion no document WAI cites quantifies. It does not promise
  byte identity across sinks.
- **`StandardUnclassified`** — a standard codec whose named standard's
  decoder-conformance clauses this revision has not checked. `replicate` promises
  what that conformance promises; the registry does not say whether that is byte
  identity.
- **`Behavioral`** — a sink-generator path that conforms by similarity, not
  bytes (`wai.semantic.*`).

#### What `replicate` promises over a standard codec

A standard codec's reconstruction is what its named standard's decoder
conformance fixes, in that standard's own output domain; `replicate` promises that
and no more. The registry classes each one by what the standard fixes (the clauses
cited are the ones read; the evidence, and what was not read, is in
[`research/media-codec-registry.md`](research/media-codec-registry.md)):

| class | capabilities | what the named standard fixes |
|---|---|---|
| `StandardDefined` | `wai.image.png`, `wai.audio.flac`, `wai.text.zstd`, `wai.text.xz` | the decoded output exactly: lossless formats (PNG, W3C Recommendation, Third Edition — its filters are exact integer operations; FLAC, RFC 9639) and lossless byte formats |
| | `wai.prior.set` | its bytes exactly: a listing (§3.1) is its own output |
| | `wai.video.avc`, `wai.video.hevc`, `wai.video.vvc` | every sample of every cropped decoded picture a conforming decoder outputs equals the value the specified decoding process produces (ITU-T H.264 (V16, 06/2026) §C.4; H.265 (V11, 01/2026) §C.5.1; H.266 (V4, 01/2026) §C.5.1); processing an SEI message only signals, such as film grain synthesis, is outside that decoding process |
| | `wai.video.vp9` | output frames "identical in all respects", in the same order, to those of the specified decoding process (VP9 Bitstream & Decoding Process Specification v0.6 §8.1) |
| `StandardTolerance` | `wai.audio.opus` | output within `opus_compare`'s thresholds of the reference decoder, with the same final range-decoder state (RFC 6716 §6) |
| | `wai.image.jpeg` | decoding "with appropriate accuracy", the accuracy of dequantisation and IDCT specified in Part 2 (ITU-T T.81 §7, A.3.3); Part 2 (ITU-T T.83) was not read, so the bound is not stated here |
| | `wai.image.jxl` | decoded samples within per-test peak-error and RMSE thresholds of reference images (ISO/IEC 18181-3 §4.1–§4.3) |
| | `wai.image.jp2`, `wai.video.j2k` | decoded components compared with reference components by error metrics, and a reversible path its definitions describe as lossless (ISO/IEC 15444-4); only the table of contents and terms of its public preview were read, so the metrics and thresholds are not stated here |
| | `wai.video.av1`, `wai.video.av1.lossless`, `wai.image.avif` | identical output frames, unless the payload signals film grain; then identical frames before grain synthesis, and output frames that "do not have perceptually significant differences" from the reference grain synthesis, a criterion whose definition the specification places outside its scope (AV1 §7.2) |
| | `wai.video.av2` | as AV1: identical output unless film grain is signalled, then one of the same two options (AV2 v1.0.0 §7.1) |
| | `wai.audio.ivas` | output bit-exact to the test sequences, or, for an implementation based on the floating-point reference code whose output is not, a non-bit-exact procedure: TS 26.444's for mono operation, a per-frame loudness-difference corridor for stereo and immersive operation (3GPP TS 26.252 V19.2.0 §7.1, §7.2.1) |
| | `wai.audio.iamf` | its substreams include Opus (above), and renderers are out of its scope |
| `StandardUnclassified` | `wai.video.evc`, `wai.video.evc.baseline`, `wai.video.avs3`, `wai.video.lcevc`, `wai.audio.mpegh`, `wai.audio.avs3`, `wai.audio.ac4`, `wai.mesh.draco`, `wai.pc.gpcc`, `wai.pc.vpcc`, `wai.splat.sog`, `wai.haptic.mihs`, `wai.meta.hdr10plus`, `wai.meta.dovi`, `wai.prior.nnc`, `wai.tensor.nnc` | not checked in this revision (all sink-supplied; `wai.splat.sog` and `wai.tensor.nnc` are candidates, with no payload format pinned — §5) |
| | `wai.audio.aac` | not checked in this revision; three decoders on one host gave three different PCM streams from one ADTS stream, so it is not placed where byte identity would be promised |
| | `wai.mesh.glb`, `wai.splat.spz` | the payload bytes pass through intact; what a sink decodes from them is not classified here (a glTF binary may carry JPEG images in its buffer and compressed geometry through extensions, and glTF 2.0 specifies an asset format, not a decoder conformance over what is rendered; SPZ stores quantised attributes) |

A conversion a sink applies after the standard's decode is not part of the
promise. A standard fixes its decoded samples at their own depth, layout and
colour representation, and none of those above fixes a conversion to another: PNG
sample-depth reduction (the PNG specification gives more than one method), YUV→RGB for AV1 and
AVIF, JPEG's colour space (outside T.81's scope, §4.1), rendering an IAMF mix to a
loudspeaker layout. The reference sink returns `RGB8` images and f32 PCM, so its
output buffers include such conversions; they are the reference implementation's,
and this revision does not specify them.

#### `wai.image.jpegai` and the standard classes

`wai.image.jpegai` has a named standard, ISO/IEC 6048, but its registry class
(`NeuralEntropyExact`) comes from the neural rule above, not from what that
standard's decoder conformance fixes. What this revision has established, and what
it has not:

- JPEG AI sets its bit-exact conformance point at the entropy-decoder output, and
  one codestream may be reconstructed through up to three synthesis transforms,
  which a profile or the encoder can restrict (an overview of the standard, arXiv
  2510.13867; [determinism-tiers](extensions/determinism-tiers.md) §1).
- The same overview says the conformance part, ISO/IEC 6048-4, "specifies the
  requirements for generating a JPEG AI-compliant reconstruction" through a
  testing suite for each profile and level pair, and "considers the possibility
  that a decoder is conformant to the standard without having bit-exact
  reconstruction", which it also calls "the possibility of standard compliance
  with a certain leeway"; outputs of the processes after entropy decoding "may
  diverge slightly from the reference". Read together, these passages point to a
  criterion on the reconstruction around a reference, the kind
  `StandardTolerance` records. The overview does not state such a criterion.
- ISO/IEC 6048-4 itself has not been read (the overview describes it as a draft
  when it was written), so this revision does not state whether it sets such a
  criterion, of what form, or how wide.

Classing `wai.image.jpegai` as `NeuralEntropyExact`, so that `replicate` over it
is refused, is therefore a conservative choice pending a reading of Part 4. It is
not a finding that the standard leaves the reconstruction unbounded. If Part 4
states a criterion on the reconstruction, `wai.image.jpegai` is a candidate for
`StandardTolerance`, over which `replicate` would promise that criterion; moving
it there would also take an exception to the neural rule above, which admits a
neural capability to `replicate` only by the integer construction.

This is not the choice made for the standard codecs. Over a `StandardUnclassified`
codec, whose conformance clauses are unread, and over AV1 or AVIF with film grain,
whose criterion the AV1 specification leaves undefined, `replicate` is honoured,
with its promise narrowed to the standard's conformance. Those capabilities keep
the behaviour they had before this revision. `wai.image.jpegai` had been
registered at `decode-equivalence`, which its float stages do not meet, and it was
moved to a class that does not back `replicate`. The two treatments differ because
of those choices, not because of anything the standards' texts were found to say.

**Registry change in v0.4.0 — a downgrade.** `wai.image.jpegai` was classed
`StandardDefined`, and §5 listed it at `decode-equivalence`. Its decode has float
stages, and a sink without the integer weight bundle decodes it through a float
ONNX Runtime path, so neither held as a property of the capability. It is now
`NeuralEntropyExact`, registered at `entropy-consistency`, and it no longer backs
`replicate`; before, the reference strict check passed a `replicate` envelope over
it whenever the prior declared an integer contract.

**Registry change in v0.4.0 — the standard codecs.** Every standard codec was
classed `StandardDefined`, and the first form of this revision's §3 promised the
same bytes on every conforming sink over all of them. The previously registered
capabilities in the `StandardTolerance` and `StandardUnclassified` rows above are
now in those classes. `replicate` over them is still honoured; what it promises is
narrower — the named standard's decoder conformance, not byte identity across
sinks. No other previously registered capability changes class. The broadcast and
streaming codecs this revision registers (§5) enter with the classes in the table.

A conforming sink that receives `intent = replicate` for a capability whose class
is `NeuralEntropyExact`, `NeuralFloat`, `GenerativeDeterministic` or
`Behavioral` MUST refuse the replicate claim (decode as perceptual or reject). The
capability is the one it decodes through: `model_requirement.capability`, or the
fallback when it decodes via §4 step 5, or, in a `WAI2` envelope, the capability
of the rendition it picks (§7.1).

**The pin under `replicate`.** Over a `NeuralIntegerExact` capability a conforming
sink MUST also refuse the replicate claim unless a pin applies to that capability
(§3.1), declares `decode-equivalence`, and declares no form other than one the
registry lists as established exact ([prior-carriage](extensions/prior-carriage.md)
§5). A pin declares a tier by `tier`. A pin without `tier` declares
`decode-equivalence` exactly when its `determinism`, compared in ASCII lower case,
contains `int`, `fixed` or `integer` and none of `f32`, `f64` and `float`, and
declares no tier otherwise. That legacy reading exists so that a pin written
before `tier` keeps its verdict. It reads a declaration and can misread one (a
contract naming `fp16` contains no float token); the registry class, checked
first, is what guarantees the arithmetic, so a misread declaration cannot admit a
float decode. An encoder SHOULD write `tier`, and SHOULD write a `determinism`
that this reading classes the same way, so that a sink of an earlier revision
reaches the same verdict. The refusals are named `replicate_unregistered` (the
capability is not registered), `replicate_class` (its class cannot back
`replicate`), `replicate_unpinned`, `replicate_tier` (the pin does not declare
`decode-equivalence`) and `replicate_form_not_exact`. They read only the manifest
and the registry, so a sink MAY make them before it verifies the pin; refusing the
claim does not excuse the pin, which every decode of a pinned payload verifies
(§3.1).

The
reference impl checks this per-envelope in `decode_envelope_strict`
(`codecs::replicate_refusal`) in two steps. First the capability's registry
class: only `NeuralIntegerExact` and the three standard classes pass, whatever the
envelope declares (`ReplicateClass::replicate_promise` states what each
promises). Then, for a `NeuralIntegerExact` capability, the pin: it must be present and
declare `decode-equivalence` (`Prior::declared_tier`), and a pin that declares a
form passes only if the registry lists that form as established exact
([prior-carriage](extensions/prior-carriage.md) §5): `wai.prior.set` is listed,
and no coded form is. The prior check reads the
sender's declaration; the class check is what ties it to the capability, so an
envelope whose prior declares an integer contract over a float or entropy-only
capability is refused. `decode_envelope_strict` applies the rule to the capability
it decodes through — `model_requirement.capability`, or the fallback when dispatch
reaches step 5 — and, when the rule refuses every capability the sink could
decode through, refuses before it verifies a pin or reads a file;
`decode_multi_strict` considers only the renditions this rule admits, and reports
each one it leaves out with that rendition's `replicate_*` reason (§7.1 step 5). On the encoder
side, `wai wrap`, which pins no prior, writes `intent = replicate` only where that
check accepts it unpinned — over the three standard classes — for the capability
and for any fallback it declares, or, in the `WAI2` companion form (§7.1), for the
base payload's capability and each companion's, and otherwise refuses unless
given another intent (`codecs::unpinned_envelope_intent`). The registry class is
also the *static* audit of the same property: `wai capabilities` prints it per
row and reports the count of neural integer-exact capabilities, so a deployer can
require "dispatch `replicate` only to a cross-hardware-sound decode", or only to
a capability whose output is the same bytes on every conforming sink
(`codecs::is_cross_decoder_exact`), in code, exactly as it audits royalty
status. The class's promise
(`ReplicateClass::replicate_promise`) and `is_cross_decoder_exact` say the same
about byte identity for every registered capability but one. A capability whose
output is computed from other envelopes' decoded output
(`codecs::composes_decoded_sources`: `wai.audio.scene`) has the class
`StandardDefined` and the promise `ReplicatePromise::ByteExact`, because its mix
is exact given the PCM its sources decode to; `is_cross_decoder_exact` answers
from the capability alone and is false for it, because a source decoded through
`wai.audio.opus`, say, may decode differently on two conforming sinks. A deployer
auditing for the same bytes uses `is_cross_decoder_exact`, not the promise. Once
the capabilities of the envelopes a scene references are known,
`codecs::is_cross_decoder_exact_with_sources` is true when the scene's class and
every source's promise the same bytes; it answers false when it cannot tell — no
source given, or a source outside `wai.audio.*`, which a scene may not reference.
The `replicate` check above covers one envelope, so `replicate` over a scene is
honoured and promises the same mix only given the same source PCM; each source is
an envelope of its own, checked by its own class. That promise covers the output
the standard or extension specifies, before any conversion a sink applies
afterwards (above), so a hash over a converted output is portable only between
sinks that also agree on the conversion. The classification is exhaustive with no
default class: a capability registered without one has no class, and the registry
tests reject it.

This makes the cross-hardware property explicit per capability: a neural
capability whose decode is *integer all the way down* can put its reconstruction
in a receipt that verifies on another machine, and the registry records which
capabilities are.

Reproducibility is a **spectrum**, not a single point, and the `ReplicateClass`
values above span it. The neural classes map onto the two tiers
[`extensions/determinism-tiers.md`](extensions/determinism-tiers.md) names.
`NeuralIntegerExact` capabilities are registered at **`decode-equivalence`** —
the whole decode to the final medium reproduces on every conforming sink, so the
rendered result has a portable hash; this is the tier `replicate` over a neural
capability promises, and the precondition for a hard
[reconstruction binding](extensions/reconstruction-binding.md).
`NeuralEntropyExact` capabilities are registered at **`entropy-consistency`** —
the entropy decode reproduces across devices (integer entropy arithmetic or
explicit parameter transmission), the tier at which JPEG AI sets its bit-exact
conformance point; it does not back `replicate`. `NeuralFloat` capabilities claim
neither tier. A pin MAY state its decode's tier (§3.1, `tier`); a stated tier above
the class's does not conform.

### Digests (normative)

WAI names one digest algorithm for each purpose:

- **Parameter-set identity** — a pin's `sha256` and `supersedes.sha256`, and each
  line of a listing (§3.1): SHA-256 (FIPS 180-4), written as 64 lowercase
  hexadecimal digits.
- **Component identity** — a `WAI2` entry's `sha256`, the digest of its payload
  bytes (§7.1): SHA-256, written the same way. The content hash a receipt chain
  takes over a whole envelope ([jwp-receipts](extensions/jwp-receipts.md) §2.1)
  is a different object.
- **Decoded output in conformance material** — a digest of the decoded output of
  a capability that decodes against a parameter set (§3.1), or of a standard codec
  (§5), in a published conformance vector or corpus, in a golden a conformance test
  compares with, or in a reconstruction or energy binding
  ([reconstruction-binding](extensions/reconstruction-binding.md) §2,
  [energy-binding](extensions/energy-binding.md)): SHA-256 over the capability's
  canonical output buffer, written as 64 lowercase hexadecimal digits.
- **Receipt content and WAI-native state** — an envelope's content hash in a
  receipt chain, and, for a WAI-native capability that decodes against no
  parameter set, its state hash and the digests of its rendered output: the
  algorithm the defining extension names ([jwp-receipts](extensions/jwp-receipts.md)
  §2.1; the extensions §5 links).

A conformance vector or corpus that states a digest of decoded output in the third
purpose MUST state its SHA-256 and MUST name the buffer it covers:

- an image, or one frame of video: `RGB8` as
  [reconstruction-binding](extensions/reconstruction-binding.md) §2 defines it;
- audio: the samples as signed 16-bit little-endian integers in output order,
  channels interleaved;
- a sequence of frames: each frame's digest, and a clip digest, the SHA-256 of the
  frames' 32-byte SHA-256 digests concatenated in output order;
- any other output (an attribute tensor, an intermediate stage): the buffer the
  vector's documentation names, with its element type, byte order and layout.

A vector MAY state other digests of the same buffer, such as a 64-bit FNV-1a or a
domain-separated BLAKE3, to locate a divergence. A verifier MUST NOT accept a
decode on such a digest alone. A hash a capability uses as a step of its own
algorithm (the prompt fold of `wai.generate.spectral`) is part of that capability,
not a digest of its output, and this rule does not reach it. A frozen digest
SHOULD be re-derived on more than one instruction-set architecture before it is
published.

### Signatures (normative)

Every Ed25519 signature (RFC 8032, pure Ed25519) that WAI or one of its extensions
defines is verified strictly: on a receipt of any family, a claim, a key-log document,
a module grant or status, an op, and a world's seal. A verifier MUST refuse a signature
unless each of these holds, in this order:

1. **The key is canonical.** The public key `A` is the one encoding of its point that
   RFC 8032 §5.1.3 allows: `y` < p, and no sign bit when `x` is 0. A decoder that
   reduces `y` modulo p, or reads the sign bit beside `x` = 0, reads a second spelling
   of a point; that key is refused.
2. **The key is not of small order.** `A`'s order does not divide 8. Under such a key
   the equation can hold for a signature made without any private key: under the
   identity, `(R, S) = (B, 1)` verifies over every message.
3. **`S` < L**, the order of the base point B (RFC 8032 §5.1.7).
4. **`R` is not of small order.** `R` decodes to a point whose order does not divide 8.
5. **The cofactorless equation holds.** `[S]B − [k]A`, with
   `k = SHA-512(R ‖ A ‖ M) mod L` over the bytes of `R` and `A` as the signature and the
   key give them, encodes to `R` byte for byte; an `R` that is not its point's
   canonical encoding never does.

A key that fails step 1 or 2 is a defect of the key, and an extension MAY report it
apart from a defect of the signature (jwp-receipts §3.1 step 3: `bad_signer_key` and
`signature_verify_failed`). The rule fixes what RFC 8032 leaves to the verifier, so
every conforming verifier accepts the same signatures.

OER/2 is the one exception: oer-receipts §5 settles its chain signature, which refuses
a key by steps 1 and 2 and checks `R` as RFC 8032 does.

### Machine-checked totality (informative)

The cross-architecture matrix establishes byte-exactness by **sampling**: it
decodes fixed vectors on every OS/arch leg of the byte-equality matrix
(`.github/workflows/wai-byte-equality.yml`). `wai-verify/` supplies
the complementary half by **reasoning** — a model checker (Kani/CBMC) over the
same `#[path]`-included sources, for every input in a bounded window.

What it discharges is the property the byte-exactness claim rests on:
**arithmetic totality**. Rust integer overflow *panics in debug and wraps in
release*, so an overflowing decoder is not one implementation but two, and which
one a sink runs depends on how it was compiled. A decode proved free of
overflow, out-of-bounds access and non-termination is a decode whose bitstream
denotes exactly one symbol sequence.

`matmul`'s accumulation is total for fixed-point operands of realistic magnitude (±2^31 at Q16). This one is a *conditional* result and is stated as such: the product is formed in i128 then truncated to i64 and accumulated with a plain `+=`, which genuinely does overflow for large enough operands. Nothing in the signature constrains magnitude, so callers inherited that safe-operating range silently; the harness turns it into a written, checked precondition rather than reporting an unstated contract as a defect.

Currently proved for the integer transformer kernels (`int_transform`): `softmax` is total over the whole fixed-point path (ln2 reduction, i128 polynomial, shift, division) and its output **sums to exactly `2^PROB_FRAC`** with no negative probability — the normalisation the entropy stage depends on, since a denormalised distribution becomes a denormalised CDF. The checker also shows the degenerate uniform fallback to be *unreachable*; that branch would violate the same invariant if it ever fired, so the property currently holds by unreachability rather than by construction, and the code says so at the site.

Currently proved for the scale→bucket primitive (`int_entropy`): `det_build_index` always returns an index addressing a real scale-table entry, and is **monotone in σ** — a larger σ never selects a smaller bucket. The monotonicity is proved for an *arbitrary* edge table rather than only an ascending one, so it does not rest on a precondition a caller might not maintain; it is the property the bucketing exists to guarantee, and a non-monotone bucketing would mis-order the table without ever panicking.

Currently proved for the integer rANS coder: `RansDecoder::new` is total for
every input slice; `decode_categorical` is total and returns an in-alphabet
symbol for all admissible CDFs and all streams in the window; and `decode`'s
bypass escape is total. Each harness carries a `kani::cover` obligation, so a
proof cannot pass vacuously on assumptions that admit no state.

This is not redundant with the matrix. The matrix decodes *well-formed* vectors
and therefore cannot see defects only a malformed stream reaches; four such
defects in the bypass escape — a stream-controlled shift distance, two
saturating-arithmetic gaps, an unbounded read — were found this way and fixed,
with the conformance vectors confirming the fix changed nothing a conforming
encoder can produce.

The `wai.video.ssai` decision policy is **test-backed, not proof-backed**: the natural theorem — that `decide` never proposes a fill exceeding the funds, the avail duration or the slot count — did not close (three configurations, up to 900s each). The obstacle is that the decision sorts on a heap-allocated key, so the model carries `memcmp` and the allocator machinery rather than the budget arithmetic. Stubbing that away would prove a property of the stub. What *is* proved there is narrower: a fresh decision bills nothing until its beacons arrive, so a fill cannot be mistaken for delivery.

`layernorm` closes only at ±2^10 — *below* realistic Q16 activation magnitude (at ±2^20 it did not close in 25 minutes). Its guarded divide and surrounding arithmetic are therefore established for small operands only, not over the kernel's working range. The obstacle is `isqrt` over a symbolic u128, which is a limit of bounded model checking rather than a property of the code. `attention`, `linear`, `igdn` and the composed `TransformerBlock` are **not covered at all**. The technique does not reach everything, and this section records where it stops rather than listing only what it reached.

**Scope, stated plainly:** these are *bounded* proofs — small symbolic CDFs,
fixed-length symbolic streams, explicit unwind limits — not universal ones, and
they cover the stream-controlled paths rather than the whole crate. A CDF
supplied by internal tables is still assumed well-formed. The verification
claims what it checks and no more.

---

## 7.1. Multi-rendition envelope (v1.1 extension)

The v1.0 single-payload envelope is sufficient when one source produces
one rendition for one capability. v1.1 adds an OPTIONAL multi-payload
form for cases where the source emits the SAME content in multiple
renditions and lets the sink pick — typically because the sink's
compute budget, bandwidth budget, or runtime preference (browser-
native vs WebGPU vs WebNN vs native ML) differs at decode time.

The `model_requirement.fallback` field in v1.0 is **declarative** — it
names a capability the sink CAN look for. v1.1 makes fallback (and
multi-rendition selection generally) **executable** by carrying every
declared capability's payload in one envelope.

### Self-contained-ecosystem note

In a controlled-deployment ecosystem (every sink ships from one
authority and therefore has a known capability set) the multi-rendition
form is NOT a "graceful fallback for missing capability" — every sink
has every required capability. Renditions exist for **policy selection**:
high-quality on a desktop sink with WebGPU; low-quality on a phone sink;
quality-vs-latency by bandwidth headroom; native vs WebNN by which
runtime is initialized. The sink picks by deployer-defined policy, not
by which capability happens to be installed.

### v1.1 wire format

```
+--------+----------------+-------------------+----------------+----...---+
| "WAI2" | u32  man_len   | manifest (JSON)   | rendition table| payloads |
+--------+----------------+-------------------+----------------+----...---+
```

- Magic is `WAI2` (distinct from v1.0's `WAI1`; sinks dispatch by magic).
- Manifest is JSON with a single `renditions` array of self-contained
  rendition entries, in deployer-preferred order:
  ```json
  {
    "wai":    "1.1",
    "media":  "audio",
    "intent": "replicate",
    "renditions": [
      { "capability": "wai.neural.encodec32", "kind": "encodec_tokens" },
      { "capability": "wai.audio.opus",       "kind": "opus"           }
    ],
    "target": { "sr": 48000, "dur": 5.0 }
  }
  ```
  Each entry has its own `capability` (the dispatch key) and `kind`
  (the codec id matching the bytes in the corresponding payload). The manifest
  MAY carry `security` (§6.1), which covers every payload: a sink applies §6.4
  before step 2 of the selection below, and an encoder writes only a security
  object there. An entry MAY
  carry `prior`, the pin (§3.1) of the parameter set its payload was coded
  against. Two entries MAY name one capability with different pins: the same
  content coded against two parameter sets, such as a set and its successor:
  ```json
  "renditions": [
    { "capability": "wai.neural.int_mlicpp", "kind": "int_mlicpp",
      "prior": { "id": "int_mlicpp/b", "sha256": "<B>", "determinism": "int-rans/p16+int-mlicpp/i64",
                 "tier": "decode-equivalence", "supersedes": { "sha256": "<A>" } } },
    { "capability": "wai.neural.int_mlicpp", "kind": "int_mlicpp",
      "prior": { "id": "int_mlicpp/a", "sha256": "<A>", "determinism": "int-rans/p16+int-mlicpp/i64",
                 "tier": "decode-equivalence" } },
    { "capability": "wai.image.avif", "kind": "avif" }
  ]
  ```
  A sink that holds `A` decodes the second entry; one that holds `B`, the first;
  one that holds neither, the third.
- **Component identity.** An entry MAY carry an `id`, a string of 1 to 64 of
  `a`–`z`, `0`–`9`, `_` and `-`, unique within the envelope (it contains no `.`,
  so it never equals a capability string), and a `sha256`, the SHA-256 of the
  entry's payload bytes as 64 lowercase hexadecimal digits. A component's identity
  is its `id`, its payload digest `sha256` and its parameter-set pin `prior`
  (§3.1). These are one construct: no extension defines another per-component
  digest, and a later mechanism that addresses a component (a staged layer, a
  byte-range or referenced rendition, a per-rendition track) addresses it by `id`
  and verifies it by `sha256` and `prior`. A sink MUST check an entry's `sha256` against the entry's payload bytes
  before it decodes or applies the entry: a selectable entry that does not match is
  skipped (`component_digest_mismatch`), as an entry whose pin is refused is, and a
  companion that does not match is not applied. An entry whose `id`, `sha256` or
  `companion_of_id` is not a string (`null` is the same as absent) makes the
  manifest malformed, as a mistyped pin member does (§3.1), and the reader rejects
  the envelope. An entry's `capability` names a payload format: a capability §5
  registers as a record or a derivation (*Kinds of registered string*) names none,
  so an entry of one, in any role, does not conform
  (`record-or-derivation-as-component`).
- **Exploding a component.** The exploded envelope of entry *i* is the `WAI1`
  envelope (§2) whose payload is entry *i*'s rendition bytes. Its manifest is the
  entry's `envelope_manifest` when it carries one, written exactly as it is in the
  `WAI2` manifest; otherwise it is
  `{"wai":W,"media":M,"intent":I,"model_requirement":{"capability":C,"fallback":null},"conditioning":{"kind":K[,"prior":P]}[,"security":S][,"refines":R][,"target":T]}`,
  with no other bytes, where `W`, `M`, `I`, `S` and `T` are the `WAI2` manifest's
  `wai`, `media`, `intent`, `security` and `target`, and `C`, `K` and `P` the entry's
  `capability`, `kind` and `prior`, each written exactly as it is in the `WAI2`
  manifest; `R` is the entry's `refines` (*Refinement components*, below) with each
  component id replaced by the content hash of that entry's exploded envelope, written
  as [staged-delivery](extensions/staged-delivery.md) §3 orders its members (`base`,
  `prev`, `layer`, `span`; a run's `producer`, then `count`) with no whitespace. A
  bracketed part is present exactly when its member is present and not `null`. A
  component and its exploded
  envelope are the same object for every rule that names an object by content hash
  ([jwp-receipts](extensions/jwp-receipts.md) §2.1). An entry MAY carry
  `envelope_manifest`, a JSON object: the exact manifest of the entry as a `WAI1`
  envelope. Its `model_requirement.capability`, `conditioning.kind` and
  `conditioning.prior` MUST equal the entry's, its `refines` MUST be the entry's
  with each id replaced as above, or both absent, and, where the entry's capability is
  refinement-only, its `model_requirement.fallback` MUST be `null`
  (`component-manifest-mismatch`). A `WAI1` envelope packed into a `WAI2` with its
  manifest copied verbatim into `envelope_manifest` explodes back to the same bytes,
  and so keeps its content hash, when its manifest has no bytes before or after its
  JSON value (whitespace around a member's value is not part of it, so an explode
  cannot write it back) and nothing follows its payload; a packer refuses any other
  envelope, and refuses an output that does not explode back to its sources (one
  whose manifest, nested in the `WAI2` manifest, passes a reader's depth limit). Entry
  *i*'s exploded envelope is defined by entry *i* and the entries its `refines` names,
  so an entry that cannot be exploded leaves every entry that does not name it intact.
  A tool that re-serialises a `WAI2` manifest may change these bytes, and with them
  the content hashes of the exploded envelopes.
- Rendition table: `u16 n_renditions | n × (u32 offset, u32 length)`,
  offsets relative to the start of the payloads block. The table's
  i-th entry corresponds to `renditions[i]` in the manifest.
- Payloads block: the rendition byte ranges, packed contiguously.
- `n_renditions` MUST equal the manifest's `renditions.length`.
- The bytes from offset 0 to the end of the rendition table are the envelope's
  **head**. They give every component's byte range and, with each entry's `sha256`,
  its digest, so a sink can fetch and verify one component without the others
  ([http-delivery](extensions/http-delivery.md) §7).

### Selection algorithm

A v1.1-aware sink MUST:

1. Read the manifest and take its ordered `renditions` entries.
2. Walk the entries in order, and pick the first that is not a companion (below),
   whose capability is dispatchable (§4), that the sink either advertises
   OR whose declared backend the sink prefers under its deployer policy, whose
   `sha256`, when present, matches its payload bytes, and whose `prior`, when the
   entry carries one and a pin applies to its capability (§3.1), resolves at the
   sink. An entry whose payload digest does not match, or whose pin is refused, is
   skipped as if the sink did not advertise its capability, and the sink keeps the
   reason. A deployer policy MAY reorder or restrict the entries the sink
   considers; it MUST NOT pick an entry skipped for its digest or its pin.
3. Read the corresponding rendition's bytes from the payload block.
4. Dispatch as in v1.0 §4, with the entry's `capability` and `prior` in place of
   `model_requirement.capability` and `conditioning.prior`, and no fallback. A
   decode refused with `set_incomplete` (§3.1) is handled as a skipped entry:
   continue at step 2 with the next entry.
5. If no entry is picked, the envelope is inert at this sink: return an error
   that names, for each entry, its index and why it was skipped:
   `not_selectable` (a companion, or a capability that is not dispatchable,
   which step 2 never picks), `missing_capability`, `component_digest_mismatch`,
   a pin reason (§3.1), or, where a deployer policy restricts step 2 to the entries §7's
   `replicate` rule admits, the `replicate_*` reason the rule gives for an entry
   it leaves out; an envelope inert under §6.4 step 2 names that reason for every
   entry.

A sink MUST NOT decode one entry's payload against the parameter set another
entry pins. The envelope's `intent` applies to the rendition the sink picks, and
§7's `replicate` rule reads that entry's capability and pin; a deployer policy
that serves only sound `replicate` claims MAY restrict step 2 to the entries that
rule admits.

A v1.0 sink that encounters a `WAI2` envelope MUST refuse it cleanly (unknown
magic, §2). No silent fallback to the first payload — that would hide the
version mismatch.

### Companion components (descriptors bound to a payload)

A `WAI2` component carries an OPTIONAL `role`. Absent ⇒ `"rendition"` (a
selectable alternative of the same content — the default above).
`"companion"` marks a component **applied *with* the chosen rendition, not
instead of it**, naming the capability it binds to in `companion_of`. It
carries the payloads that are not standalone media — the capabilities §5
registers as **companion-only**: the HDR dynamic-metadata descriptors
`wai.meta.hdr10plus` / `wai.meta.dovi` (SMPTE ST 2094), which describe a video
payload, and the enhancement layer `wai.video.lcevc`, which adds residuals to
one. Carried as companions, the companion and its base ride in the **same
envelope**, so the binding is intrinsic — not a spoofable sidecar. This
revision registers no capability for immersive-audio mix metadata;
`wai.audio.iamf` carries its own audio substreams, so it is standalone media
and travels as a rendition.

Selection: a sink picks a rendition among the
`role != "companion"` entries by the algorithm above, then applies every companion
bound to the chosen component — by `companion_of_id` when the companion carries
one, otherwise every companion whose `companion_of` names the chosen component's
capability (e.g. the HDR10+ block for the AV1
video the sink picked). `pick`/`primary` never return a companion, nor a
component whose capability is not dispatchable (§4): a companion-only
capability, a candidate, a record or a derivation. A component whose capability
this revision does not register may be picked when the sink advertises it. Static HDR
characteristics (ST 2086 mastering display, HLG transfer) stay in-band in the
video bitstream / a manifest color field and need no companion.

A component with `role = "companion"` conforms only if:

1. its capability is companion-only, or is not registered in this revision. A
   capability registered in any other kind (§5, *Kinds of registered string*) is
   not a companion: a payload capability is standalone media — its payload decodes
   on its own — and MUST be carried as a rendition, and a record or a derivation is
   no component in any role (*Component identity*, above);
2. `companion_of` is present and names a capability the companion applies to.
   An HDR dynamic-metadata descriptor, registered as a companion to a video
   payload, applies to a registered capability whose payload is video — the
   `video` media class (§3), as the capability's registration places it: every
   `wai.video.*` capability that is not companion-only, the neural video codecs
   `wai.neural.video_bmshj2018`, `wai.neural.dcvc_rt` and `wai.neural.dcvc_fm`
   (§5, *Video:*), and the films `wai.film.linear` and `wai.film.reel`
   ([linear-film](extensions/linear-film.md) gives a linear film's `media` the
   value `"video"`, and a reel is a film with cuts). A capability registered as
   video in a later revision joins that class.
   `wai.video.lcevc` applies to one of the base codecs §5 lists for it. No
   companion applies to a companion-only capability. This revision constrains
   no base for a companion whose capability it does not register, nor an HDR
   descriptor's base whose capability it does not register;
3. a selectable rendition of that capability is present in the envelope — for
   an **enhancement layer**, exactly one, the base encode its residuals were
   produced against, and that rendition is the base of no other enhancement
   layer (§5). Several descriptor companions MAY bind to one base, and a
   descriptor MAY bind to several renditions of one capability.

A companion MAY carry `companion_of_id`, the `id` of the one entry it binds to.
That entry MUST exist, MUST have the capability the companion's `companion_of`
names, and MUST be a selectable rendition; the companion then binds to exactly that
entry, and rule 3's "at most one enhancement layer per base" is checked by entry.
`companion_of` keeps its meaning, so a sink that does not read `companion_of_id`
binds by capability: for an enhancement layer over two renditions of one base
capability it finds the base not unique, and does not apply the layer.

An envelope with a companion that breaks any of these does not conform (the
reference `WaiMulti::check_component_roles` reports the first break). A sink
MUST NOT apply a companion that breaks rule 1 or 2, nor one whose capability it
does not recognise; a companion that breaks rule 3 binds to nothing, or, for an
enhancement layer, is handled as §5 states. The sink MAY still present the
rendition it picked.

### Refinement components

A component with `role: "refinement"` is a layer of a stage set
([staged-delivery](extensions/staged-delivery.md)). It carries `refines` (§3), whose
`base` and producers are the `id`s of entries in the same envelope (§7.1, *Component
identity*); its exploded envelope carries the same `refines` with those ids replaced
by the content hashes of the named entries' exploded envelopes (*Exploding a
component*, above), so it is the same object as the layer carried on its own.

- **Placement.** A refinement component MUST follow every selectable entry in
  `renditions`.
- **Selection.** Selection never picks one (step 2: it is not selectable). A sink
  picks a rendition as above, then offers to staged-delivery §4, in `renditions`
  order, the exploded envelopes of the refinements whose `refines.base` is the picked
  entry's `id`, up to the stage its deployer policy chooses. It offers them only from
  an envelope every component of which conforms — each refinement below, each
  companion (*Companion components*), each `id` (*Component identity*) and each
  capability in the role §5 gives it (the reference `WaiMulti::check_component_roles`,
  which reads the manifest alone): an envelope one component of which breaks a rule
  offers none of its refinements. It MAY still present the rendition it picked. A
  refinement whose payload does not match its `sha256` is not offered. A refinement's
  exploded envelope a sink receives on its own, as a `WAI1` object
  ([staged-delivery](extensions/staged-delivery.md) §9), is a separate delivery, which
  staged-delivery §4 processes as any layer, whatever the envelope it was exploded
  from.
- **Conformance.** A refinement component conforms only if, in this order: its
  capability is refinement-only (§5); it carries `id` and `sha256`; its `refines`
  is well formed (staged-delivery §3, with ids in place of hashes); `refines.base`
  names a selectable entry, which carries `id` and `sha256`; its capability admits
  that entry's (§5, *Base admission*); every producer in `prev` names a selectable
  entry or a refinement before it, each carrying `id` and `sha256`; and no selectable
  entry follows it. No other component carries `refines`, and a refinement-only
  capability is carried with no other role. The reference
  `WaiMulti::check_component_roles` reports the first break.

A sink of an earlier revision does not advertise a refinement-only capability, so it
never picks a refinement by capability; and since refinements follow every selectable
entry, a sink that picks the first entry it can decode picks a base. If it picks one
by another policy, it finds the capability unsupported and the component inert.

### Derived renditions

A `WAI1` manifest (§3), or a rendition entry, MAY carry `derived_from`. It states that
its payload presents another object, the **motif**, and was made from it by a named
renderer or encoder:

```json
"derived_from": { "motif": "<64 hex>", "motif_capability": "<capability>",
                  "renderer": "<id>@<version>", "settings": "<64 hex>" }
```

- `motif` is the motif's content hash ([jwp-receipts](extensions/jwp-receipts.md)
  §2.1), 64 lowercase hexadecimal digits.
- `motif_capability` is the motif's capability, and `renderer` the renderer or encoder
  that made the payload; each a non-empty string.
- `settings` is `BLAKE3("wai:render-settings\x01" ‖ JCS(settings))`, over the settings
  the renderer was run with as RFC 8785 canonical JSON, 64 lowercase hexadecimal
  digits.

A sink MUST NOT present such a payload's output as meeting the motif's equivalence
criterion, its tier or a `replicate` intent: the payload presents the motif; it does not
reconstruct it. A verifier checks `motif` against a motif it holds and reports every
other member as the producer's statement, except where a profile fixes them:
[packaging](extensions/packaging.md) §7 fixes a supplied floor's `renderer` and
`settings`, and a package's verifier holds them to it. A value of another form makes the member
malformed; the envelope still parses (§8). A reader of an earlier revision ignores the
member and loses only that statement. [packaging](extensions/packaging.md) §7 writes it
on every floor object.

### v1.0 compatibility

v1.0 envelopes (`WAI1`) remain valid and their format is unchanged, except that
a `WAI1` naming a companion-only capability as its `capability` or `fallback`
does not conform (§5; below). Encoders
SHOULD emit `WAI1` when only one rendition is present (simpler form,
wider sink support). `WAI2` is for sources that genuinely benefit from
shipping multiple renditions per file.

### Envelopes the companion rules make non-conforming

The companion rules of §5 and this section change no capability's payload
format and no capability's role (§8). Each enforces something fixed before the
rules were stated:

- **a registration.** `wai.meta.hdr10plus` and `wai.meta.dovi` were registered
  as companion descriptors to a video payload, not standalone media.
  `wai.video.lcevc` was registered companion-only, with its base codecs, in the
  same revision as these rules. Every other registered capability is standalone
  media.
- **the definition of a companion.** When this section introduced
  `role = "companion"`, it defined a companion as a component applied with the
  chosen rendition and naming, in `companion_of`, the component it binds to.

Before these rules were stated, the text rejected none of the envelopes below,
and the reference implementation had no check that reported them (its `pick`
and `primary` could select an HDR descriptor). These do not conform:

- a `WAI1` that names an HDR descriptor as its `capability` or `fallback`,
  carrying it as standalone media (§5);
- a `WAI2` that carries an HDR descriptor as a selectable rendition instead of
  with `role = "companion"`. `pick` and `primary` no longer select it
  (*Selection*, above);
- a `WAI2` that binds an HDR descriptor to a registered capability outside the
  `video` media class (rule 2) — an audio, image or text payload, or any other
  registered capability rule 2 does not list, a volumetric or world payload
  among them;
- a `WAI2` that marks a registered standalone capability as a companion
  (rule 1). This includes `wai.audio.iamf` carried as immersive-audio mix
  metadata: the earlier text of this section named immersive-audio metadata
  beside the HDR descriptors as a payload that accompanies another, and a
  reference test carried `wai.audio.iamf` that way. That use did not follow
  the registration — `wai.audio.iamf` was registered as an immersive-audio
  carrier, standalone media, and no capability for mix metadata alone is
  registered — and rule 1 follows the registration;
- a `WAI2` with a companion, of any capability — one this revision does not
  register included — that has no `companion_of`, or whose `companion_of`
  names no selectable rendition in the envelope (rules 2 and 3).

The first four enforce the registrations; the last enforces the definition of
a companion. Each still parses, and §5 and the rules above state what a sink
does with it. No earlier revision registered `wai.video.lcevc`; this one
registers it with these rules. An envelope that used that string while no
revision registered it is covered by §8's rule for a new registration, as is an
envelope that uses any string a later revision registers.

---

## 8. Versioning

- The `wai` manifest field is `MAJOR.MINOR`. A reader MUST reject an
  unknown MAJOR. A reader MUST accept an unknown MINOR by ignoring
  fields it does not understand, provided MAJOR matches.
- A reader MUST ignore a manifest member it does not understand, whatever the
  MINOR. Earlier text required this only for an unknown MINOR, so a reader of an
  earlier revision may instead reject an envelope that carries such a member,
  which fails closed. Or it ignores the member, and then a pin fails open (§3.1).
- The container format (§2) is frozen for all of `1.x`. The manifest schema (§3,
  and a §7.1 rendition entry) is frozen for all of `1.x` in this sense: no member
  is removed or renamed, none changes type or meaning, and none becomes REQUIRED.
  A MINOR revision MAY define an OPTIONAL member, at the top level or inside an
  existing object, whose absence leaves every envelope's bytes and meaning as they
  were, and the revision states what it means for a reader of an earlier
  revision (above). This revision defines `conditioning.prior`, a rendition
  entry's `prior`, `id`, `sha256` and `companion_of_id`, and the pin's members
  `form`, `tier` and `supersedes` this way (§3.1, §7.1); envelopes that use them
  keep `wai` `"1.0"` or `"1.1"`. The reference implementation wrote `prior`, with
  `id`, `sha256`, `determinism` and `form`, before this revision defined it; an
  envelope written that way keeps its bytes and its meaning, except as §3.1 lists.
- **The one exception to the freeze.** This revision makes one exception to the
  sentence above, deliberately. A member it defines, carried with another JSON
  type, makes the manifest malformed rather than being ignored, because a reader
  that ignored a mistyped pin member could decode against a set the pin excludes
  (a mistyped `form`, read as absent, turns a set-form pin into a form-less one).
  The rule covers every member listed above, the rendition entry's `id`, `sha256`
  and `companion_of_id` included, so one rule decides whether a manifest parses.
  An envelope that carried such a value before the member was defined, and parsed
  then, does not parse now; §3.1 lists it. Nothing in the envelope signals the
  change: its `wai` value is the same, so a reader of this revision rejects the
  whole envelope where an earlier reader ignored the member.
- This revision also defines the manifest's `refines` (§3) and a rendition entry's
  `refines` and `envelope_manifest` (§7.1) as OPTIONAL members under the MINOR rule
  above; envelopes that use them keep their `wai` value. The exception does not cover
  them: an envelope whose `refines` is malformed still parses, and
  [staged-delivery](extensions/staged-delivery.md) §4 reports it
  (`refines-malformed`); one whose `envelope_manifest` does not match its entry still
  parses, and §7.1 reports it (`component-manifest-mismatch`). `derived_from`, on a
  manifest and on a rendition entry (§7.1, *Derived renditions*), is defined the same
  way: an envelope whose `derived_from` is malformed still parses.
- New capabilities (§5) MAY be added in any MINOR revision. Existing
  capabilities MUST NOT be redefined — once a capability string is
  registered for a codec, its payload format is fixed forever; for a registration
  completed as the next item says, from the revision that completes it.
- **Completing a registration.** A WAI-native string whose §5 row named the decoder
  that reads its payload and fixed no byte layout (naming, at most, a tool that wrote
  it) has a registration with no payload format to hold. A MINOR revision MAY
  complete such a registration by pinning a byte layout, which is not a
  redefinition. Envelopes that carried another form under the string, a tool's
  output among them, become non-conforming; they still parse, and the completing
  revision lists them in §5. From that revision on, the string's payload format is
  fixed as the item above states. §5, *Payloads of the WAI-native integer codecs*,
  completes the registrations of `wai.video.int_motion`, `wai.video.int_hyper`,
  `wai.neural.int_hyper`, `wai.audio.int_codec` and `wai.splat.int_codec` this way.
- **Editions.** A registration that names an edition (or version) of its
  standard carries that edition. A registration that names none carries the
  edition of its named standard current on the date it was registered, and no
  later edition or amendment. A later edition or amendment that changes the
  bitstream a decoder must accept is registered, if at all, under a capability
  string of its own; it never widens an existing string. The machine-readable
  registry records both (`edition`, `registered`;
  [`registry/capabilities.json`](registry/capabilities.json)). Where which
  edition was current on the registration date has not been established, the
  registration says so (§5), and what it admits beyond the tools of the
  standard's first edition is unspecified until a revision records the edition. Citing a later
  edition of the same specification as the reference for a format, where the
  bitstream a registration admits does not change, is not a redefinition.
- A MINOR revision MAY add a value of an informational manifest field (the
  `media` classes of §3); a reader accepts a value it does not know.
- A registration also fixes the capability's **role** (§5, §7.1): standalone
  media, or companion-only — applied with a base rendition, never on its own.
  Changing a capability's role, or narrowing the renditions a companion-only
  capability applies to, redefines it. Widening them does not: no conforming
  envelope stops conforming, and a reader of an earlier revision that reports
  the new binding as not conforming does not apply that companion and can
  still present its base.
- This revision adds a third role, **refinement-only**: a payload applied, in a
  declared order, to the output of the stage below it, bound to that output's
  producer by content hash ([staged-delivery](extensions/staged-delivery.md)), and
  never decoded on its own. It differs from companion-only, which is unordered,
  names its base by capability and travels only inside a `WAI2`: a refinement is
  ordered, names its base by content and travels as its own object. Changing a
  capability into or out of either role redefines it; admitting another base (§5,
  *Base admission*) widens it. The manifest member `refines` (§3) is an OPTIONAL
  member defined under the rule above for MINOR revisions: an envelope that does
  not carry it keeps its bytes and its meaning. A reader of an earlier revision
  does not advertise a refinement-only capability, so it finds such an envelope's
  capability unsupported and the envelope inert.
- A registration of a capability that decodes against a parameter set also fixes
  that set's shape (§5, §3.1). Changing one file to several narrows it and
  requires a new MAJOR; several to one widens it, which a MINOR revision MAY do.
- Removing or repurposing an existing capability — changing its payload format
  or its role — requires a new MAJOR.
- A MINOR revision MAY add a conformance rule that enforces a registration,
  or the definition of a manifest field, as it was made, changing no payload
  format and no role. Such a rule makes non-conforming only envelopes that used
  a capability against its registration or a field against its definition,
  and the revision lists every kind of envelope it makes non-conforming; they
  still parse. The companion rules of §5 and §7.1 are such rules, and §7.1
  lists the envelopes they affect (*Envelopes the companion rules make
  non-conforming*). §3.1's pin rules are such rules, and §3.1 lists the
  envelopes they affect.
- Registering a capability gives a string a payload format and a role that no
  earlier revision gave it, and the rules of §5 and §7.1 then apply to it. An
  envelope that used the string before then used a capability no revision
  defined, and no rule checked that use against a registration it did not yet
  have: §7.1 constrains an HDR descriptor's base only when the base's
  capability is registered, and holds a companion of an unregistered
  capability only to naming a base that is present (rules 2 and 3). Leaving a
  use unchecked does not make it conform to a registration made later, so the
  registering revision MAY make such an envelope non-conforming where it uses
  the string against the new registration. This narrows nothing a registration
  fixed and is not a redefinition. The kinds of envelope are listed here, once,
  for every registration, so a registering revision need not list them again:
  registering a standalone capability makes non-conforming a `WAI2` that
  carried it as a companion (rule 1) and, when it is outside the `video` media
  class, one that bound an HDR descriptor to it (rule 2); registering a
  companion-only capability makes non-conforming an envelope that carried it as
  a `WAI1` `capability` or `fallback` or as a selectable rendition, that bound
  it to a base or a number of bases its registration does not admit, or that
  bound another companion to it (no companion applies to a companion-only
  capability); registering a refinement-only capability makes non-conforming an
  envelope that carried it as a `WAI1` `capability` without `refines`, as a
  `fallback`, or as a selectable rendition or a companion, and one that bound a
  companion to it. Such envelopes still parse.

---

## Appendix A — Reference implementation

`wai-rs/` (Rust). Builds as `lib`, `cdylib`, `staticlib` so the C ABI
in `wai-rs/src/ffi.rs` drops in as `libwai.dylib` / `.so` / `.dll` and
can be called from any language. The crate wraps:

- **image**: `image` crate (PNG, JPEG, AVIF via `ravif`), `jpegxl-rs`
  (libjxl 0.11.x).
- **audio**: `opus` (libopus 1.x), `claxon` + `flac-bound` (libFLAC).
- **video**: `rav1e` (AV1 encode) + `dav1d` (AV1 decode).
- **text**: `zstd` (libzstd), `xz2` (liblzma).

Run `cargo test --lib` (with `RUSTFLAGS="-L /opt/homebrew/opt/flac/lib"`
on macOS, or via the committed `.cargo/config.toml`).

Decode speed is an implementation property, not a conformance property: the
integer decoders run faster kernels (SIMD, threads) built to give the same
integer for every output element as the retained scalar reference kernels. That
equivalence is argued by construction and tested, not machine-proved (the
harnesses of §7 do not reach these kernels) — see
[`research/decode-throughput.md`](research/decode-throughput.md) for the
argument, its tests, its limits and measured throughput.

## Appendix B — Why a *capability menu*

For a format a standard already specifies, WAI names the standard and the sink
decodes it with the library it already has: the zeroth condition's purpose is
**availability**, so a file opens wherever that library is present. Where WAI
needs a property no registered standard specifies — a reconstruction that is
byte-identical on every machine, a world or a film that replays to the same
state — it defines a WAI-native capability class with its own payload format
and its own conformance criterion (§5, §7). The envelope, the dispatch rules,
the registry's classes and the shared-prior model are what WAI itself
specifies.

## Appendix C — License

Apache-2.0. The standard and the reference implementation are open;
royalty-free implementation is a precondition for adoption, so every default
capability (§5) is royalty-free. Companion implementations outside this tree
keep their own licences.