open-standards/wai/SPEC.md

WAI v1.0 — Web AI Media Transport & Execution

Status: Draft Standard. Reference implementation: wai-rs/ (Rust lib + cdylib + staticlib, C FFI). Two paths open every conforming file:

• the neural condition — the sink regenerates the media from a compact conditioning payload against a shared ambient prior (a model). The model is a requirement of existence, never transported in the envelope that decodes against it, exactly as .mp3 does not ship its decoder. Its identity MAY be pinned: the manifest MAY carry a pin, the SHA-256 of the whole parameter set the payload was coded against (§3.1), and a sink whose parameter set does not match the pin does not decode against it (§4, §7.1). The pinned set's wire form MAY be declared (prior-carriage).
• the zeroth condition — a menu of registered standard codecs (AVIF, JPEG XL, PNG, Opus, FLAC, AV1, zstd, XZ and the others §5 registers), each decoded by the sink with the library it already has.

WAI dispatches registered standard codecs and also defines WAI-native capability classes (§5): integer codecs, worlds, scores, films and the other extensions, each with its own payload format and conformance criterion.

The capability a file requires is named, not supplied. Sinks that have the named capability use it; sinks that don't fall back to a declared fallback capability.

Keywords MUST, MUST NOT, SHOULD, MAY are RFC 2119.

---

1. Scope and model

WAI is a container + capability dispatch standard. It is media-general; its floor and recommended set are royalty-free. Its substance is the envelope, the dispatch rules and the registry's classes: a standard codec is dispatched to a sink's existing decoder, not re-implemented.

A WAI file declares (in its manifest):

• what media it carries,
• what capability a sink must have to decode it,
• a fallback capability the sink can use if it lacks the primary,
• the conditioning bytes for that capability,
• and, OPTIONALLY, for a capability that decodes against a parameter set, the pin of the set the payload was coded against (§3.1).

Conforming sinks MUST implement the container envelope (§2), the manifest schema (§3), the capability-dispatch algorithm (§4), and at least the mandatory floor capabilities (§5). Other capabilities are OPTIONAL; §5 says which are RECOMMENDED.

---

2. Container

+--------+----------------+-------------------+----------------+-----------+
| "WAI1" | u32  man_len   | manifest (JSON)   | u32 payload_len| payload   |
| 4 B    | little-endian  | man_len bytes     | little-endian  | bytes     |
+--------+----------------+-------------------+----------------+-----------+

• Bytes 0..4 MUST be ASCII WAI1. A reader MUST reject any file whose first 4 bytes differ.
• man_len and payload_len are unsigned 32-bit little-endian.
• The manifest is UTF-8 JSON, serialized with no insignificant whitespace, preserving key insertion order. Readers MUST NOT depend on key order for semantics; encoders SHOULD preserve order so files round-trip byte-identically.
• Member names are unique within each object of a manifest, at any depth: a manifest that names a member twice in one object is malformed, and a reader rejects it as it rejects any manifest that does not parse. Names compare after their escapes are read.
• An integer member of a manifest, or of any JSON this specification or its extensions define, is a JSON number written as an integer: digits, with a minus sign only before a non-zero value, and no fraction or exponent. -0, 1.0 and 1e0 are not integers, whatever their value, and a member that must be an integer and is written so has the wrong type.
• The payload is opaque codec bytes whose interpretation is fully determined by the manifest's model_requirement.capability and, for a capability that decodes against a parameter set, by that set, which the manifest MAY pin (§3.1).

All WAI multi-byte integers are little-endian unless stated otherwise.

Reader limits. A reader MUST NOT refuse a manifest for its length, its nesting or the number of its values while it is at most 134 217 728 bytes (man_len ≤ 2²⁷), its arrays and objects nest at most 127 deep, the manifest object itself counting as one, and it holds at most 4 194 304 JSON values (2²²), counting the manifest object and every value nested in it at any depth; a member's name is not a value. It MAY refuse a longer or deeper manifest, or one that holds more values. The limits hold the largest manifest a conforming layer has: a prev (staged-delivery §3) that names a different producer for each of the 2²⁰ units a WIR1 layer may cover (staged-delivery Appendix A.3, n_units ≤ 2²⁰) is about 94 MB, and three values a run (the run, its producer and its count), 3·2²⁰ and a few more in all. A reader that refuses one past the length limit does so from man_len alone, before it reads or requests the manifest (manifest_too_large). A reader that refuses one past the value limit does so while it counts the values, before it builds anything from the manifest (manifest_too_many_values). The reference readers refuse all three. The same limits apply to the WAI2 manifest (§7.1), whose renditions lists no more entries than its table holds. They move no byte of the layout above, which stays frozen (§8). Since a conforming encoder emits only containers every conforming sink can read (§7), a conforming encoder writes no manifest past any of these limits; the reference writers refuse one past any of them.

Together the length and value limits bound what parsing a manifest takes, which the length limit alone does not: a parsed value takes more memory than its text, and many small values can take many times their length. The reference readers count a manifest's values before they build anything from it, then build each array and object at the size counted, and on a 64-bit target allocate at most 128 bytes for each value and one for each byte of the manifest, with, for a WAI2, a rendition entry of 768 bytes for each it lists: at most 640 MiB for a WAI1 manifest at both limits, whatever its shape, and 48 MiB more for a WAI2's entries (informative; SECURITY.md).

A wai.text.zstd or wai.text.xz payload (§5) has reader limits of its own, since neither stream format bounds what a payload may ask of its decoder. A reader MUST NOT refuse one for the length it decodes to, or for the history its decoder must keep, while it decodes to at most 134 217 728 bytes (2²⁷) and every frame or block of it declares at most 2²⁷ bytes of history: for Zstandard the frame's window size, which for a single-segment frame is its content size (RFC 8878 §3.1.1.1); for XZ the LZMA2 dictionary size. It MAY refuse one that decodes to more (text_decoded_too_long) or declares more (text_window_too_large). A reader that refuses for length does so when its output would pass the limit, in a buffer it never sizes past the limit; where a Zstandard frame declares a content size past the limit, it does so from the declaration, before it decodes anything. A reader that refuses for the history does so from the declaration, before its decoder allocates it. The reference readers refuse both. A decode then takes its output, at most the length limit, the decoder's history, at most the other, and the decoder's fixed state. As for a manifest, a conforming encoder writes no payload past either limit, and the reference writers refuse one; bytes longer than the length limit are carried as more than one object.

A picture, audio or video payload of the mandatory floor or the recommended set (§5) has reader limits too, since its format bounds what it may ask of a decoder loosely or not at all. A reader MUST NOT refuse a picture (a wai.image.png, wai.image.jpeg, wai.image.avif or wai.image.jxl payload, or a frame of a wai.video.av1 or wai.video.av1.lossless one) for its size while it is at most 65 536 pixels wide and tall and holds at most 134 217 728 pixels (2²⁷, a 16 384 × 8 192 picture). It MUST NOT refuse a wai.audio.flac or wai.audio.opus payload for its length while it decodes to at most 134 217 728 samples (2²⁷), counting every channel: 46 minutes of one channel at 48 kHz. It MUST NOT refuse a wai.video.av1 or wai.video.av1.lossless payload for its length while its framing header (§5, Payload framing) declares at most 1 048 576 temporal units (2²⁰), 4.9 hours at 60 frames a second. It MAY refuse one past any of these: a picture (image_too_large), audio (audio_decoded_too_long) or video (video_too_large). A reader that refuses a picture does so from the size its payload declares, before it decodes a pixel: the PNG IHDR, the JPEG frame header, the ispe property of each image item of an AVIF, the JPEG XL size header, the AV1 framing header's h and w. One that refuses audio does so when its output would pass the limit, in a buffer it never sizes past the limit, and where a FLAC STREAMINFO declares a total past the limit, from the declaration, before it decodes. One that refuses video does so from the framing header. A picture or audio decode then takes its output and the decoder's state; a video decode that hands on each frame as it decodes takes one frame and the decoder's reference frames, whatever the payload's length. (A reader that keeps every frame at once needs n · h · w pixels; the reference library's whole-output entry points refuse a payload past a budget, 4 GiB by default, and decode one frame at a time otherwise; informative, SECURITY.md.) As for text, a conforming encoder writes no payload past these limits, and the reference writers refuse one; longer audio and video are carried as more than one object.

An envelope delivered over HTTP is one object of the proposed media type application/wai, named by its receipt content hash, and fetched whole or, for a WAI2, one verified component at a time (http-delivery).

---

3. Manifest schema

{
  "wai": "1.0",
  "media": "image" | "audio" | "video" | "text" | "volumetric" | "haptic"
         | "metadata" | "interactive" | "binary",
  "intent": "replicate" | "create" | "improve" | "...",
  "model_requirement": { "capability": "<capability-string>",
                         "fallback":   "<capability-string>" | null },
  "conditioning": { "kind": "<codec-id>", "prior": { ... } },
  "security": { "encryption": "<scheme>" | "none", "kid": <u64> },
  "refines":  { ... },
  "target":   { ... }
}

• wai — spec version. This document defines "1.0". Unknown major → reject. Unknown minor → accept. A reader ignores a member it does not understand, whatever the minor (§8).
• media — media class; informational. The capability is authoritative. The values are the registry's media classes (codecs::capability_media; registry/capabilities.json): image, audio, video, text, volumetric (meshes, point clouds, Gaussian splats), haptic, metadata (data applied with another payload), interactive (worlds, feeds, avatars) and binary (anything else). A reader MUST accept a value it does not know: the field is informational, and a MINOR revision may add a value (§8).
• intent — generative intent. replicate (reconstruct the source) is the only intent with normative reconstruction semantics in v1.0. What it promises is set by the capability's registry class (§7, Replicate-soundness): a byte-exact reconstruction — every conforming sink that decodes the payload produces the same bytes (decode-equivalence) — where the capability's reconstruction is specified exactly; and, for a standard codec whose named standard holds decoders to a criterion around a reference rather than to identical output (Opus's thresholds, RFC 6716 §6, are one), no more than that standard's decoder conformance, so conforming sinks' reconstructions may differ as far as that criterion allows; for a standard codec §7 has not yet classified, what its standard's decoder conformance states, which the registry does not assert is byte identity. For a neural capability, replicate promises decode-equivalence and is sound only under the integer-entropy contract (§7), with a pin (§3.1, §7): a capability whose decode runs float, or whose entropy decode alone is integer, MUST NOT claim it. That is WAI's rule, and this revision applies it to wai.image.jpegai as well; it is not a finding that JPEG AI's standard leaves the reconstruction unbounded (§7, NeuralEntropyExact).
• model_requirement.capability — the capability string a sink MUST resolve. See §5 for the registered set.
• model_requirement.fallback — a second capability the sink MAY use if it lacks the primary, or if the envelope's pin does not resolve for the primary at the sink (§4 step 3). null means the file is inert without the primary. An encoder whose primary capability decodes with a module a sink obtains apart from the envelope does not write null (§4).
• conditioning.kind — short codec id matching the capability. Helps multiplexers route bytes without parsing the full capability string.
• conditioning.prior — OPTIONAL. The pin (§3.1) of the parameter set the payload was coded against. It applies to whichever of model_requirement.capability and model_requirement.fallback decodes the payload (§4), where that capability decodes against a parameter set. Absent, or null, the key is not written, and a decode through such a capability is unpinned: it runs against whatever parameter set the sink holds, and it cannot back intent = replicate over a NeuralIntegerExact capability (§7).
• security — OPTIONAL. Absent, null, or encryption: "none" ⇒ the payload is plaintext (the default). Otherwise the payload is encrypted and MUST be recovered before dispatch; see §6. The capability, media, and conditioning fields always describe the plaintext codec.
• refines — OPTIONAL. Carried exactly when model_requirement.capability is refinement-only (§5, Refinement capabilities): what the payload refines, by content hash (staged-delivery §3). A manifest that carries it has a null fallback.
• target — codec-specific reconstruction parameters (dimensions, sample rate, duration, frame count). Informational; the payload is self-describing per the codec's own format.

3.1 Parameter-set pin (prior)

A capability decodes against a parameter set when its registry class (§7) is NeuralIntegerExact, NeuralEntropyExact, NeuralFloat or GenerativeDeterministic. Its parameter set is every byte, other than the payload, that its decode reads and that its output depends on: networks, entropy tables, normative constants. The sink supplies it; the envelope never carries it (§1). §5 registers each such capability's parameter set as one file or several files; a capability registered with neither, and a capability this revision does not register, is treated as several files.

A manifest names the parameter set a payload was coded against with a pin. One capability is the exception: the payload of wai.generate.spectral names its own parameter set, the model file, by SHA-256, and an envelope that names it carries no pin (§5, Payload of wai.generate.spectral).

"prior": {
  "id":          "<a name for people>",
  "sha256":      "<64 lowercase hexadecimal digits>",
  "determinism": "<how the decode is made reproducible>",
  "form":        "<a registered prior form>",
  "tier":        "decode-equivalence",
  "supersedes":  { "sha256": "<64 lowercase hexadecimal digits>", "form": "<a registered prior form>" }
}

member	presence	meaning
id	REQUIRED, string	A name for people and logs. A sink MUST NOT select, resolve or accept a parameter set by it.
sha256	REQUIRED, string	The SHA-256 (FIPS 180-4) of the parameter set in its form (below), as 64 lowercase hexadecimal digits.
determinism	REQUIRED, string	How the decode is made reproducible, in words. Informative, except where §7 reads it for a pin without tier.
form	OPTIONAL, string	A registered prior form (§5; prior-carriage).
tier	OPTIONAL, string	The reproducibility tier (determinism-tiers §2) the sender claims for the decode, from a closed vocabulary: decode-equivalence, entropy-consistency, or none for neither.
supersedes	OPTIONAL, object	Informative. The parameter set this one replaces, named as a pin names one: its sha256 (REQUIRED) and, if it is pinned in a declared form, that form.

A prior, or a member of it, whose JSON type is not the one given makes the manifest malformed, and a reader rejects it as it rejects any manifest that does not parse. A mistyped member is never read as absent: a form dropped that way would turn a set-form pin into a form-less one, which a single file can satisfy. null in an OPTIONAL member is the same as the member being absent. A member this section does not define is ignored (§8). An optional member that is absent is not written, so a pin written without it keeps its bytes.

What the digest covers. A pin identifies the whole parameter set.

• With form absent, sha256 is the SHA-256 of the one file that holds the set. That is admissible only for a capability registered with a one-file set.

• With form present, sha256 is the SHA-256 of the set serialized in exactly that form (prior-carriage §2). The form wai.prior.set serializes a set of one or more files as its listing, so a pin in that form covers every file of the set:

  • The listing has one line for each file: the file's SHA-256 as 64 lowercase hexadecimal digits, two spaces (0x20 0x20), the file's path, and a line feed (0x0A).
  • A path names the file below the set's root: one or more segments joined by / (0x2F). A segment begins with one of A–Z, a–z, 0–9, _ and -, continues with any of those and ., and does not end with ..
  • The lines are in strictly ascending order of their paths compared as byte strings. No two paths are equal once A–Z are mapped to a–z, and no path followed by / begins another path.
  • The listing is its lines concatenated. It has at least one line and nothing else: no header, no byte-order mark, no carriage return, no blank line.

  A listing is the text common checksum utilities write and check, so a third party can check a set against its listing, and a listing against a pin, without WAI software.

Verification. A sink that is to decode through a capability, for an envelope whose pin applies to that capability, MUST verify the pin first. A pin applies to a capability that decodes against a parameter set, and to a capability this revision does not register when the sink advertises it (such a string is dispatchable, §4): such a capability is verified as several files, so a form-less pin over it is refused (form_required). The pin resolves when the sink holds a parameter set for the capability whose digest, in the pin's form, equals sha256: in the form wai.prior.set, a listing whose SHA-256 is sha256, together with every file the listing names below the listing's root, each with the SHA-256 its line gives. Nothing else satisfies a pin: not a matching id, not a set that supersedes names or that names this one, and not a set in another form whose unpacked parameters compare equal. A sink MAY hold several parameter sets for one capability; the pin resolves when one of them satisfies it.

A pin that does not resolve is refused with one reason. The checks run in this order, and the first that fails is the reason reported:

1. pin_malformed: sha256 is not 64 lowercase hexadecimal digits.
2. form_not_registered: form names no registered prior form.
3. form_unreadable: form is registered and the sink has no reader for it. This is the sink's gap, and it is never reported as a mismatch.
4. form_required: form is absent and the capability is not registered with a one-file set.
5. prior_revoked or prior_not_authorised: the sink holds a key log (jwp-receipts §7), and it refuses the set this pin names, which it identifies by the pin's sha256 in the pin's form. The reason is prior_revoked when that digest is in the revoked_modules of the head the sink reaches from any anchor it holds, or, where none can be reached, of that anchor revision (§7.9 check 5). A revocation needs no grant, status or clock, so this part runs for every capability a pin applies to, and whichever check of §7.9 a grant fails first. Where the sink also holds the capability's pinned parameter sets to §7.9 (§4, Module authorisation), §7.9 authorises the set as a decoder module, and the reason is prior_not_authorised when §7.9 does not hold it for any other reason — no grant from one of the sink's module origins, a granting key §7.4 does not confirm, no fresh status because the sink is offline or has no clock it trusts, a grant outside its window or for another capability — or the sink holds no grant for the set. The sink reports §7.9's check and reason with it. A sink that holds no key log skips this check. Check 5 covers every identity of the bytes the decode reads, not only the pin's: once checks 6 to 8 have found the set, the sink runs this check again over each digest a pin over those bytes could name — the listing of the files the decode reads, and, for a capability registered with a one-file set, that file's SHA-256 — so a set refused under one identity is not reached through a pin that names it by another (a set-form pin over a listing of one revoked file, or a listing that adds a file the decode does not read). Authorisation needs one of them to be held; the pin's own digest, which passed above, is one.
6. prior_not_held: the sink holds no parameter set for the capability whose digest, in the pin's form, equals sha256; a file the sink cannot read is not held. A sink that holds the set a well-formed supersedes names — a file when supersedes.form is absent, a listing when it is wai.prior.set — reports digest_superseded instead, so that its operator can tell it holds the set this one replaces.
7. set_malformed: in the form wai.prior.set, the listing whose SHA-256 is sha256 breaks a rule above.
8. part_mismatch: in the form wai.prior.set, in every copy of that listing the sink holds, a file the listing names is missing, cannot be read, or has a SHA-256 other than its line gives.

These checks read the envelope, the registry, the forms the sink can read and the bytes it holds, and nothing that depends on how the sink was built, so two conforming sinks that read the same forms and hold the same bytes report the same reason. Check 5 also reads what the sink holds under jwp-receipts §7.9 and its clock, so two sinks that hold the same key-log material and agree on the time report the same reason there. Refusal reasons are snake_case; tier names and class labels are kebab-case.

Decoding against a verified set. A pinned decode MUST read its parameters only from the set it verified. A decode that needs a file the verified set does not contain is refused with set_incomplete; the sink presents no output from it, and dispatch treats it as a pin that did not resolve. A file a model names for part of its own data, such as the file an ONNX tensor stored as external data names, is a file the decode needs: when the decode cannot take it from the verified set, it is refused the same way. A sink SHOULD decode from the very bytes it verified: one that reads a file again after verifying it can decode bytes it did not verify.

A refused pin makes its capability unsupported for that envelope at that sink. It is not a decode error and not a reason to decode anyway, as capability-constraints §2 treats an unmet constraint. Dispatch continues as §4 and §7.1 state. A sink MUST NOT decode a payload through a capability whose pin it refused.

Tier. tier is the sender's claim, from the registered vocabulary decode-equivalence, entropy-consistency and none. An encoder MUST write a registered token. A sink reads a token this revision does not register as none, and a verifier reports it (tier_unregistered). The tier a registry class gives is decode-equivalence for NeuralIntegerExact, entropy-consistency for NeuralEntropyExact, and none for every other class. With the tiers ordered none < entropy-consistency < decode-equivalence, a tier above the class's does not conform (tier_exceeds_class): the replicate verdict (§7) does not rest on it, and a verifier reports it. The decode itself is unaffected. A tier at or below the class's is honoured as stated.

Updates. supersedes is informative: it records that the set a pin names replaces another, changes no decision, and changes only the reason check 6 reports. A supersedes whose sha256 is not 64 lowercase hexadecimal digits or equals sha256, or whose form is not a registered prior form, is malformed: a verifier reports it (supersedes_malformed), and a sink ignores the member and so never reports digest_superseded for that pin. Content coded against a set pins that set, and a successor never satisfies its predecessor's pin, so content coded against a predecessor decodes at a sink exactly as long as that sink still holds the predecessor. A sink can hold both without a second copy of the files they share, by keeping the predecessor's listing beside the successor's. An update delivered as a change to an earlier set, such as an NNC incremental update, is not a parameter set and is never pinned; the self-contained set it yields is (prior-carriage §2).

Where a pin sits. In a WAI1 manifest, conditioning.prior (§3). In a WAI2 manifest, a rendition entry's prior, which pins that entry's capability and is part of the entry's identity (§7.1). A pin where no capability it would apply to decodes against a parameter set, and every such capability is registered, means nothing: an encoder MUST NOT write one, a verifier reports it (prior_not_applicable), and a sink ignores it.

Sinks of earlier revisions. A sink that implements an earlier text of this specification may do one of two things with a pin (§8). It may reject the envelope as carrying a member it does not understand, which fails closed. Or it may ignore prior, and then the pin fails open: such a sink may decode a pinned payload against another parameter set. The rules of this section bind sinks that implement this revision; an encoder that must not be decoded against another set at an earlier sink has no means to require that within 1.x other than a capability string earlier sinks do not advertise.

Envelopes these rules make non-conforming. The rules enforce the definition of prior as it was made — a lowercase hexadecimal SHA-256 of the artifact that determines the decode's output, which a replicate over an integer-exact neural capability requires — and change no payload format and no role (§8). Each of these envelopes but the last still parses, and does not conform:

• one whose pin's sha256 is not 64 lowercase hexadecimal digits (the reference implementation compared digests without regard to case or length);
• one whose pin has no form for a capability registered with several files: its digest covered one file of the set or, where the registered artifact was a directory, nothing a sink could hash;
• one that carries a pin where no capability it would apply to decodes against a parameter set;
• one with intent = replicate whose decode capability is NeuralIntegerExact and which pins no parameter set for it, or pins one that does not declare decode-equivalence (§7); the reference implementation already refused these;
• one that carries a member this revision defines — conditioning.prior or a rendition entry's prior, a pin member, or a rendition entry's id, sha256 or companion_of_id (§7.1) — with a JSON type other than the one defined. A reader of an earlier revision may have ignored the member; a reader of this one rejects the manifest as malformed, so this envelope no longer parses. §8 states this as the one exception this revision makes to the 1.x freeze.

---

4. Capability dispatch

A conforming sink MUST implement this algorithm:

1. Read the manifest. If wai major version is unknown → reject.
2. Let cap = model_requirement.capability.
3. If cap is dispatchable (below), the sink advertises cap — holding under jwp-receipts §7.9 any module it obtained apart from the envelope to decode cap (Module authorisation, below) — and, when a pin applies to cap (§3.1) and the manifest carries conditioning.prior, the pin resolves at the sink for cap, decode payload via the codec registered for cap (§5), against the verified set, and return. A pin that does not resolve makes cap unsupported for this envelope: keep its reason and continue at step 4.
4. Else, let fb = model_requirement.fallback.
5. If fb != null, fb is dispatchable and the sink advertises fb on the same terms, and, when a pin applies to fb and the manifest carries conditioning.prior, the pin resolves at the sink for fb, decode via fb's codec and return. A pin that does not resolve: keep its reason.
6. Else the file is inert at this sink — return an error to the calling application that names why: the reason kept at step 3, else the one kept at step 5, else missing_capability.

conditioning.prior pins the parameter set the payload was coded against, so it applies to whichever capability decodes the payload. A pin that does not resolve is never a reason to decode against another parameter set: a learned decode against the wrong parameters yields plausible output, not an error. An encoder that needs an alternative for sinks that hold another parameter set carries one pinned rendition per set in a WAI2 envelope (§7.1).

A capability is dispatchable unless this revision registers it as something a sink does not decode on its own: a candidate (§5, Candidate capabilities), a companion-only capability (§5, Companion-only capabilities), a refinement-only capability (§5, Refinement capabilities), a record or a derivation (§5, Kinds of registered string). A sink MUST NOT take step 3 or step 5 with a capability that is not dispatchable, whatever it advertises. A string this revision does not register is dispatchable: a later MINOR revision may register it (§8), and a sink that has a codec for it says so by advertising it. A WAI1 whose cap is companion-only does not conform and is inert: its payload is companion data, so step 5 is not taken either (§5).

A WAI payload always corresponds to exactly one capability. v1.0 does not support multi-codec multiplexed payloads.

Dispatch goes by cap: a WAI1 whose cap is not refinement-only is dispatched as above whatever else its manifest carries, a refines member included, which such a manifest MUST NOT carry and a sink ignores (staged-delivery §3). A sink never takes step 3 or step 5 with a capability §5 registers as refinement-only, and a WAI1 whose cap is refinement-only takes neither: its payload refines the output of a base, and staged-delivery §4 processes it. A sink that does not implement that extension finds the capability unsupported, and the envelope is inert.

If the manifest declares security.encryption other than "none" (§6), the sink MUST recover the plaintext payload before step 3 and run this algorithm on the plaintext; see §6.4.

Module authorisation. A module is code a sink obtains apart from the envelope and runs to decode a capability, or to rebuild a reconstruction from what a decoder produced: a decoder or reconstruct module (extensions/jwp-receipts.md §7.9). A sink that decodes cap with one MUST hold it under §7.9 for the envelope before it runs it, from one of the sink's module origins (§7.9): the origins its relying party names, and the origin under which it confirmed the envelope's delivery. A module that is not held — not granted by a module origin, revoked, or not checkable because the sink is offline or has no clock it trusts — leaves cap not advertised for that envelope at steps 3 and 5. A sink that holds a key log refuses a parameter set whose digest the key log revokes (§3.1 check 5, prior_revoked). A sink MAY also hold the parameter sets envelopes pin to §7.9, by its relying party's choice per capability; for such a capability §3.1's check 5 also refuses a pinned set §7.9 does not hold (prior_not_authorised). A refused pin does not resolve. An envelope that pins no set is decoded against the set the sink holds for cap, and a sink that holds a key log runs check 5 over that set too, identified by each digest a pin over it could name: it does not decode cap for that envelope when any of them is revoked, or, for a capability it holds to §7.9, when §7.9 holds the set under none of them. It then decodes from the bytes check 5 ran over alone, as a pinned decode reads only what its pin verified (§3.1): read once, so a file changed after the check is not what the decode reads, and, for a capability it holds to §7.9, only the part §7.9 holds (the set, the files a held listing names, or one held file); a file the decode needs outside them is set_incomplete. Either way the algorithm above runs unchanged, so a sink MUST NOT report a file inert because of module authorisation while a fallback it can decode without that module remains, and an encoder emitting a capability whose decode needs a module MUST declare such a fallback (§3). A sink configured with no key log holds no module under §7.9, so it runs no module obtained apart from an envelope, and it does not run check 5. The reference dispatcher in wai-rs runs no module obtained apart from an envelope; it runs check 5 when the sink is configured with a key log (key_log::SinkKeyLog, feature key_discovery, given to pin::verify_with, to int_payload::resolve (which runs it for a pinned decode and over the files an unpinned integer decode reads) or to neural::ModelRegistry::with_key_log, which also runs it before an unpinned decode, over the bytes that decode then reads, read once: an integer decode's members, and otherwise the registered file of a one-file set, or every file below the directory the decoder reads from, identified by the listing pin::listing_of_dir derives, each file's digest and each listing beside them that matches): the revocation part for every capability a pin applies to, and §7.9's authorisation for the capabilities the sink holds to it. wai-web and the C ABI take no key log.

---

5. Registered capabilities (the zeroth menu)

WAI v1.0 registers the following capabilities. Each names a standard codec or a WAI-native class. The mandatory floor and the recommended set are royalty-free; every payload and companion capability carries a licence class (see the end of this section), and one that is not royalty-free is never a default. The "payload format" column is the on-the-wire payload bytes. For a standard codec it is the canonical bytes that codec normally emits (a JXL file, an Opus stream, etc.) — WAI does not wrap or re-frame them. For a WAI-native class it is the format the class's registration or extension defines; for the WAI-native integer codecs, the format Payloads of the WAI-native integer codecs (below) pins.

The registry is also published in machine-readable form, generated from the reference implementation: registry/capabilities.json and its schema, with the policy that governs it in REGISTRY.md. Where the two differ, this section is normative and the generated files are wrong.

Kinds of registered string. Every string this section registers has one kind:

• a payload capability: an envelope's capability, fallback or WAI2 rendition names the codec its payload is decoded with;
• a companion capability: companion-only (Companion-only capabilities, below), carried only as a WAI2 companion;
• a refinement capability: refinement-only (Refinement capabilities, below), applied to the output of the stage below it and never decoded on its own;
• a derivation operation (wai.derive.*): an operation on samples another capability decoded, which names no payload format;
• a record: a signed record a WAI tool emits and verifies (a receipt kind — wai.asset.provenance, wai.quantum.calibration, wai.quantum.job, and the delivery records wai.video.manifest, wai.video.ssai, wai.video.keys, wai.video.package and wai.video.channel), which names no payload format.

Only payload, companion and refinement capabilities have a media class, a fidelity tier, a replicate class and a role. A derivation or record string is not a payload capability and is not dispatchable (§4): step 3 does not take an envelope's capability that names one, and step 5 may still take a usable fallback. A row of the tables below that registers a record says what the record attests, not a payload format. Named sub-identifiers this section uses — wai.det.fixed64, wai.det.amp64, wai.world.sim.feed0, wai.world.sim.skeletal0 — name parts of a capability's definition and are not capabilities.

Mandatory floor

Every conforming sink MUST implement:

capability	payload	role
wai.image.png	PNG (W3C Recommendation, Portable Network Graphics (PNG) Specification, Third Edition)	universal lossless image
wai.audio.flac	FLAC (RFC 9639)	universal lossless audio
wai.text.zstd	Zstandard (RFC 8878)	universal compressed bytes

These three guarantee that any sink with the standard system codec libraries can open a file in the image, audio and text classes. The floor has no video capability.

Recommended modern set

Every conforming sink SHOULD implement:

capability	payload	role
wai.image.jxl	JPEG XL (ISO/IEC 18181)	lossless + lossy
wai.image.avif	AVIF (the AOM AVIF specification: an AV1 image format based on HEIF, conforming to MIAF)	lossy (AV1-based)
wai.image.jpeg	JPEG (ITU-T T.81)	compatibility with existing JPEG decoders
wai.audio.opus	Opus (RFC 6716)	lossy speech and music
wai.video.av1	AV1 (AOMedia)	lossy
wai.video.av1.lossless	AV1 (AOMedia), quantizer=0	mathematically lossless YUV (lossy in RGB due to color conversion)
wai.text.xz	XZ/LZMA2	maximum classical text ratio

Payload framing. A wai.image.* payload is one file of its format as the named specification defines it, and a reader decodes it as that format only: PNG bytes are not a wai.image.jpeg payload. A wai.audio.flac payload is a FLAC stream (RFC 9639), its fLaC marker and STREAMINFO first. A wai.audio.opus payload is an Ogg Opus stream (RFC 7845); its OpusHead input sample rate is informational (RFC 7845 §5.1), so a reader decodes at a rate of its own choosing when the declared one is not an Opus rate. A wai.video.av1 or wai.video.av1.lossless payload is five u32 little-endian values, n, h, w, fps_num and fps_den (the number of temporal units, the largest frame's height and width, and the frame rate as fps_num / fps_den frames a second), then each temporal unit as a u32 little-endian length and its bytes: the temporal unit in the low-overhead bitstream format (AV1 §5), every OBU with its size field. A payload holding more temporal units than n, or a frame larger than h × w, is malformed. packaging §7 writes its floor objects in this framing. A sink decodes AV1 in any profile, layout and depth AV1 allows.

Neural capabilities (OPTIONAL, sink advertises if its ML runtime supports them)

Each names a model the sink MAY have installed. Payload is a compact conditioning representation specific to the capability (latent tokens, prompt embeddings, etc.). The standard does not specify the byte layout of these payloads — the capability owner does — except where this section pins one: the integer codecs' payloads are pinned below (Payloads of the WAI-native integer codecs).

Each row names the model and the operating point its own publication reports (bitrate, sample rate) or the property WAI's integer construction gives it; this section states no comparison between models or with the zeroth codecs. The decode route for each is implemented in the reference sink (see the source column / neural/*.rs); the model weights are sink-supplied.

A capability that decodes against a parameter set (§3.1) is registered with that set's shape. One file: wai.neural.int_synth (its model specification, int_synth.json); wai.neural.int_mlicpp (its WMB1 bundle, read from a set as model.wmb); wai.audio.int_codec (its WAM1 synthesis, read from a set as model.wam); wai.splat.int_codec (its WHM1 synthesis, read from a set as g_s.whm); and wai.neural.encodec32, wai.neural.dac, wai.neural.mimi, wai.neural.wavtokenizer, wai.neural.snac and wai.neural.video_bmshj2018 (a self-contained ONNX model; one that keeps tensors in external files is several files). Several files: every other capability that decodes against a parameter set; wai.neural.int_hyper and wai.video.int_hyper read g_s.whm, h_s.whm and entropy.json from theirs. wai.audio.int_codec and wai.splat.int_codec were registered as several files until this revision named their one-file formats. A later revision MAY register a capability of several files as one file when it defines a one-file format that holds the whole set; that widens the registration and makes no conforming envelope non-conforming (§8).

Audio:

capability	model	operating point	source
wai.neural.encodec32	EnCodec at 32 kHz, 4 codebooks	32 kHz; 4 codebooks of 2048 at 50 Hz (2.2 kbps)	EnCodec (arXiv 2210.13438)
wai.neural.dac	DAC at 44.1 kHz	44.1 kHz; 9 codebooks at 86 Hz (about 8 kbps)	DAC (arXiv 2306.06546)
wai.neural.mimi	Mimi, 12.5 Hz frame rate	24 kHz; 8 codebooks of 2048 at 12.5 Hz (1.1 kbps), real-time speech	Mimi (arXiv 2410.00037)
wai.neural.wavtokenizer	WavTokenizer single-codebook VQ	24 kHz; one codebook at 40 or 75 tokens per second (under 1 kbps)	WavTokenizer (arXiv 2408.16532)
wai.neural.snac	SNAC multi-resolution RVQ (hierarchical code levels, own sample rate)	hierarchical code levels at the model's own sample rate	SNAC (arXiv 2410.14411), neural/snac.rs

Image:

capability	model	operating point or property	source
wai.neural.bmshj2018	bmshj2018-factorized + rANS (byte-exact CompressAI port)	about 0.26 bpp at 29 dB PSNR (quality 3)	CompressAI
wai.neural.mbt2018_mean	mean-scale hyperprior (Gaussian-conditional entropy)	the conditional-entropy family (arXiv 1809.02736)	neural/hyperprior.rs
wai.neural.int_synth	integer fixed-point synthesis (all-i64 transposed-conv, no float/ONNX)	bit-identical on every machine — soundly backs intent = replicate	payload and set: integer-payloads §13; decoded through its lowering, integer-operators §13.1 (int_synth/1); int_synth.rs (integer weights sink-supplied; the reference tools make them by training in float, then quantising after training with per-layer shifts calibrated on measured activations, tools/wai_int_synth_train.py. Of the committed vectors only photo comes from a trained model, tools/wai_int_synth_real.py; gradient, bloom, facade and the cross-architecture golden use seeded integer weights)
wai.neural.int_hyper	integer fixed-point conditional codec — whole bitstream→image path integer (integer hyper-decoder σ → integer-σ entropy → integer synthesis)	the integer-entropy contract fully assembled: a PTQ of mbt2018_mean, bit-identical on every machine, whose integer decode CI holds within 1 dB of the same model's float decode on a pinned photograph	payload WIH1; int_hyper_synth.rs (tables sink-supplied). Payload: integer-payloads §4; its networks decoded through their lowering, integer-operators §13.2 (whm1/1)
wai.neural.int_mlicpp	integer fixed-point MLIC++ entropy topology — K channel slices of the latent, each decoded as a checkerboard anchor pass then a non-anchor pass, with an entropy network over concat[latent decoded so far, hyper] re-run before every pass to give σ‖μ; hyper-decoder, entropy network and synthesis all integer	the channel-autoregressive × checkerboard entropy model at decode-equivalence: the encoder and every decoder compute the same σ, so the same distribution for every element, at any thread count; bit-identical on every machine. The committed conformance vector has synthetic weights, so it demonstrates decode-equivalence and states no quality figure	int_mlicpp.rs (one-file model bundle sink-supplied; tools/wai_int_mlicpp_export.py). Payload and bundle: integer-payloads §14; its networks decoded through their lowering, integer-operators §13.4 (wmb1/1)
wai.image.jpegai	JPEG AI (ISO/IEC 6048-1:2025) codestream → RGB — integer entropy front end (mANS, with an integer scale-index derivation) + 16-bit-PTQ integer hyper-decoder, context-model and synthesis networks + dequantisation, latent refinement, post-filters (the eICCI CNN among them) and colour conversion in IEEE-754 float with a fixed evaluation order. Scope: one configuration — 256×256 4:4:4, model_id 0 (tools_0), beta_displacement_log 0, the high-operation-point synthesis, the eICCI model pair the weight bundle carries, and the other header conditions in determinism-tiers §5; both decode paths refuse any other codestream. Within it, a sink without the integer weight bundle, or in a non-default floating-point environment, takes a float ONNX decode instead. Only checkpoint constants are sink-supplied	entropy-consistency (registry class NeuralEntropyExact): the entropy decode, including the scale-index derivation, is integer on both decode paths, so the decoded symbols reproduce on every machine. The exact decoder also pins one reconstruction to RGB, reproducible across machines under the build and floating-point conditions in determinism-tiers §5–§6 (it checks the floating-point environment before decoding) — a property of that decoder, not of the capability, so wai.image.jpegai does not back replicate (§7). On one reference codestream the exact decoder stays within max|Δ| ≤ 2 of the reference decoder (recorded max|Δ| = 1 on ~4.4% of samples); evidence in determinism-tiers §6. Downgraded in v0.4.0: this row previously claimed decode-equivalence, and the registry classed the capability StandardDefined	jpegai_decode.rs (+ jpegai_{bitstream,mans,pipeline,synthesis,icci}.rs)
wai.generate.spectral	deterministic integer GENERATIVE synthesis (intent=create) — prompt+seed → integer-LUT spectral composition, no float/ONNX	byte-identical on every machine, pinned to (model, prompt, seed, steps); the create-side of "conformance is a hash" (generation-equivalence). Payload: the instruction object of form v0, which names the model by its SHA-256 and lists every size a sink makes, each output checked by its SHA-256 (Payload of wai.generate.spectral, below)	generate.rs (small pinned model bundle)
wai.neural.elic	checkerboard context model (hyperprior + 2-pass spatial context)	the checkerboard-context entropy class (arXiv 2203.10886)	neural/context.rs
wai.neural.mlicpp	MLIC++ multi-reference entropy (channel-AR slices + checkerboard + global attention)	the multi-reference entropy model (arXiv 2307.15421)	research code, neural/mlicpp.rs
wai.neural.mlicv2	MLICv2 / MLICv2+ (2025), MetaFormer transform	shares the mlicpp route	research code, neural/mlicpp.rs
wai.neural.cmic	Content-Adaptive Mamba (ICLR 2026), SSM transform + SCCTX entropy	shares the mlicpp route	arXiv 2508.02192, neural/mlicpp.rs
wai.neural.glc	Generative Latent Coding — categorical / VQ entropy	ultra-low-bitrate image coding	research code, neural/glc.rs
wai.semantic.clip	a CLIP-embedding package regenerated by any conforming sink generator (the generative-semantic path, semantic.rs)	conforms by similarity above a declared threshold, never by bytes (Behavioral, tier Perceptual); generators are sink-supplied	semantic.rs
wai.semantic.perco2	PerCoV2 ultra-low-bitrate generative image coding	as wai.semantic.clip; its model licence is non-commercial (CC BY-NC-SA), so it is licence-flagged and never a default	semantic.rs

Video:

capability	model	operating point or property	source
wai.neural.video_bmshj2018	per-frame bmshj2018	per-frame image coding; decodable in a browser on WebGPU	CompressAI
wai.neural.dcvc_rt	DCVC-RT (2025), inter-frame with reference-feature propagation	inter-frame neural video; its paper reports real-time 1080p and 4K decode on a GPU	research code, neural/video_dcvc.rs
wai.neural.dcvc_fm	DCVC-FM (feature modulation), wide single-model quality range (RGB + YUV)	a wide single-model quality range; shares the inter-frame route	research code, neural/video_dcvc.rs
wai.video.int_motion	DETERMINISTIC INTEGER inter-frame — I-frame + integer block motion-compensated P-frames + integer-quantized residual over rANS, no float/ONNX; motion is whole-pel or half-pel (mv_precision), the latter predicting at the half-grid by integer bilinear interpolation ((a+b+1)>>1, (a+b+c+d+2)>>2) so real footage — which pans a non-integer number of pixels per frame — costs fewer residual bits while staying byte-exact	byte-identical frame-by-frame on every machine (frame-state-equivalence, a per-frame portable hash); the temporal extension of the integer contract, where a float inter-frame decode desyncs down the GOP	payload WIV1, keyframe 0; int_video.rs (the encode tools write a JSON form a converter reads)
wai.video.int_hyper	NEURAL-KEYFRAME inter-frame — the I-frame is the LEARNED integer codec int_hyper decoded byte-exact, the P-frames are integer block-MC residuals on top	a learned keyframe you can put in a receipt + integer inter-frames; whole decode integer, byte-identical everywhere	payload WIV1, keyframe 1 (a nested WIH1); int_video::decode_pframes_typed ∘ int_hyper_synth::decode_conditional_checked. Payload: integer-payloads §3, its keyframe §4; the keyframe's networks decoded through their lowering, integer-operators §13.2 (whm1/1)
wai.video.manifest	ATTESTED PER-SESSION MANIFEST — the OTT delivery layer, not a codec: a personalised HLS manifest (ads stitched at EXT-X-DISCONTINUITY, blackouts + geo enforced) rendered as a pure function of the session class (content, entitlement, geo, blackout set, ad pod, window) rather than the viewer, and sealed in an Ed25519 receipt that binds the JCP grant (capability, joule ceiling, funds_ceiling = ad budget), the composition, the stitched ads and the policy applied. A receipt over a WAI playlist MAY carry render and extras, the rendering parameters and per-segment lines that playlist adds, both bound through manifest_hash and through a render digest the signature covers (http-delivery §8.6)	manifest-equivalence: manifest_hash == hash(render(segments)) and session_class_hash are recomputable by anyone holding the bytes, so the ads, blackouts and compose-work stop being dashboard claims; class-determinism means one generation serves every viewer of the class, and uj_per_viewer is the amortised energy that a 1:1-unique generator cannot report below its per-viewer cost. Only the measurement — joules_micro, its acquisition class and the class's evidence — is measured-attested: signed, and not re-measurable	wai_video_manifest.rs
wai.video.ssai	ATTESTED SERVER-SIDE AD INSERTION — a deterministic ad decision (greedy by price within the avail duration and the funds remaining) plus a per-impression Ed25519 receipt binding the candidate pod, the manifest it belongs to, the served creative_hashes and the VAST quartile each reached	spend-equivalence: budget_spent_micros is recomputed from the billable impressions and rejected if it exceeds the grant's funds_ceiling — an overspent receipt fails verification; every creative and quartile is checkable, so a buyer and seller reconcile impressions from the same bytes instead of two logs that only a dashboard joins. Only the measurement — joules_micro, its acquisition class and the class's evidence — is measured-attested: signed, and not re-measurable	wai_video_ssai.rs
wai.video.keys	ATTESTED CONTENT-KEY RELEASE — the DRM licensing decision as a record: every key release, and every refusal, is an Ed25519 receipt binding the KID, the DRM system, the entitlement demanded and the window enforced, with a BLAKE3 key_commit proving which key was released without carrying it. It is one of Lane A's records (§6.5): it covers the release decision, never a frame the key opens	decision-equivalence: verify() re-runs the release policy over the receipt's own (grant capability, entitlement, issued, expiry) and rejects any misreported outcome, and enforces granted-iff-committed — so "keys went only to entitled, unexpired sessions" is checkable against the signed log, not asserted. Only the measurement — joules_micro, its acquisition class and the class's evidence — is measured-attested: signed, and not re-measurable	wai_video_keys.rs
wai.video.package	DETERMINISTIC PACKAGING + ATTESTED SEGMENTMAP — a source sliced into content-hashed segments on GOP boundaries; the ordered (index → content_hash) map is a signed Merkle commitment, and identical content (ads, bumpers, replays, simulcast overlap) dedups to one stored/encoded copy	map-equivalence + verifiable dedup: segment_map_root lets a wai.video.manifest reference be checked against the package that produced it, and verify() recomputes unique_segments/stored_bytes from the segments so the storage/encode saving cannot be overstated; uj_per_unique_segment is the encode-once energy. Only the measurement — joules_micro, its acquisition class and the class's evidence — is measured-attested: signed, and not re-measurable. Version 2 (packaging §9) binds the policy digest and one stage index per component, names each object by its jwp-receipts §2.1 content hash, and covers every media class a packager packs; the string keeps video for history	wai_video_package.rs, package.rs
wai.video.channel	VIRTUAL LINEAR / FAST ASSEMBLY — a continuous schedule of assets over wall-clock time, sliced into attested playback windows; windows chain via parent_receipt_hash, so a channel's history is a hash chain of program-as-run records	schedule-continuity + recomputable program-as-run: verify() rejects a schedule with a gap or overlap (no dead air, no double-booking) and recomputes aired from the schedule and the window, so what actually aired is checkable by a licensor, advertiser or regulator against the signed chain rather than an operator-held play-out log. Only the measurement — joules_micro, its acquisition class and the class's evidence — is measured-attested: signed, and not re-measurable	wai_video_channel.rs

Sink-architecture note: the inter-frame neural video codecs (DCVC-RT, DCVC-FM) decode through ort and are CPU-capable for native sinks — their real-time figures come from GPU execution, but a GPU is not required to decode. They are heavier than per-frame image codecs (a temporal net plus reference-feature state), so a browser sink that cannot host the temporal net should declare only wai.neural.video_bmshj2018 (or another browser-decodable neural video codec) for the video media class. The choice is the deployer's, advertised per sink — the envelope is identical.

Scope of the neural rows. A neural image or video capability is an option a deployer may declare and a sink may advertise; none is in the floor or the recommended set, and none is a default.

Each capability owns its payload format. Decoder implementations (Python source-side, ONNX-export, browser onnxruntime-web) live in tools/wai_<capability>_*.py and wai-web/demo/. New neural capabilities are added by registration; the WAI envelope is unchanged.

See extensions/jwp-receipts.md for the receipt-chain sub-spec covering per-object content hashes, per-group Merkle roots, Ed25519 group signatures, and cross-hop preservation; an object's figure, acquisition class and origin bound into its Merkle leaf, and a group total's class signed under a labelled group payload; the typed receipt classes beside delivery (energy, lawful-basis, log-inclusion, measurement, operating-point, stage, delivery-gap, reconstruct-session) carried as claim receipts; one receipt chain per publisher key and track, and an object not delivered accounted for by its receipt, verified by inclusion; and a second, weaker binding class (a registry-named soft binding with a resolution endpoint) whose recovered receipts attest the ancestor, never the object in hand. See extensions/energy-measurement.md for how a receipt's joules_micro is acquired: the required acquisition class (CalibratedInstrument / OnChipCounter / ModelBased / Estimator, with the legacy HwShunt still accepted when read), the declared uncertainty — a standard relative uncertainty and its integration window, never less than the 0.5/√3 µJ (≈ 0.289 µJ) that rounding the figure to whole microjoules adds, so joules_micro × relative_ppm ≥ 288 676 — that both hardware classes must carry, the counter filtering state and calibration reference that distinguish them, the bracketed marginal protocol (a burst whose window opens from rest, divided by the work the window paid for, from brackets in which the power reader's rail was live), the attribution-quality rule that decides whether a figure may be published at all, the signed measurement report that publishes a figure's bracket, build and power reader beside the receipt it names, and how the receipt families that sealed a figure before the class existed carry it as an optional signed label that leaves every issued receipt's bytes unchanged. A joules_micro of 0 is the unmetered marker, never a figure of zero joules. See extensions/confidentiality.md for the private-channel sub-spec: MLS (RFC 9420) group keys + SFrame (RFC 9605) object encryption, the byte-level framing behind §6. See extensions/reconstruction-binding.md for the reproducible-reconstruction binding: a C2PA / JPEG-Trust hard binding over the deterministically decoded pixels of a neural-coded asset (SHA-256 over the canonical RGB plus the descriptor that pins the reconstruction) — the reconstruct-side companion to provenance, applicable to the byte-exact wai.neural.int_* codecs and, over the exact decoder's pinned reconstruction only, to wai.image.jpegai. See extensions/energy-binding.md for the companion energy assertion: the joules a reconstruction cost, bound to that reconstruction, with the acquisition class and coverage C2PA 2.4's c2pa.environmental-sustainability has no field for.

Prior artifacts, general tensors and derivation operations

capability	payload	status · extension
wai.prior.nnc	a trained network's parameters as an ISO/IEC 15938-17:2024 (NNC, 2nd edition) bitstream	a registered wire form a pin MAY declare (prior.form); its digest then covers the coded bitstream. Optional, not recommended: royalty status is not established, and cross-decoder exactness of the reconstruction is not established, so a replicate over a neural capability cannot rest on a prior read out of it. As a payload it takes the replicate class of a standard whose decoder conformance the registry has not checked · prior-carriage
wai.prior.set	the listing of a parameter set of one or more files: each file's SHA-256 and path (§3.1)	a registered wire form a pin MAY declare (prior.form), and the form in which a set of several files is pinned: the pin covers the listing, and each line covers one file. Reading a set through its listing compares digests and hands the decoder each file's bytes unchanged, so the registry lists it as established exact, and a replicate MAY rest on a pin in it. WAI-native, royalty-free; as a payload its output is its own bytes (StandardDefined) · prior-carriage
wai.tensor.nnc	general tensorial data (split-inference feature maps, 3D/4D scene parameters) coded with NNC	candidate: general-tensor coding is in an unpublished amendment, so the payload format is not fixed and MUST NOT be emitted until pinned (Candidate capabilities, below) · prior-carriage

The ten derivation operations of ISO/IEC 23001-16:2021 (derived visual tracks) are registered as wai.derive.* strings, each with a per-operation determinism tier. They are not payload capabilities: an operation acts on samples another capability decoded. WAI does not composite; the registration states what a reproducibility claim over a derived sample needs. The geometric conditions a tier lists for each operation are read from the operation's title: clause 8 of ISO/IEC 23001-16, which defines each operation, was not read · derived-tracks.

Interactive, temporal & computed content (extensions)

Media that consumes input, moves over time, or is computed from a compact instruction set. Each registers its own payload format and a deterministic *-equivalence conformance criterion over the wai.det.fixed64 fixed-point floor — the reconstruct half of the standard. See the linked extension for each.

capability	payload	conformance · extension
wai.world.scene / wai.world.replay / wai.world.live	WWLD container (op log)	replay-equivalence · interactive-worlds
wai.world.model	generative-world transport: a signed action trace (the world CausalOp log) + a declared model_id (world_model.rs)	the wire carries the action trace, not the pixels — it replays under canonical total order to one session hash on every machine (replay-equivalence), the generated frames vouched for by a signed, joules-accounted provenance receipt. The neural sibling of .replay; with an int_* renderer the frames hash-converge too (a generative interactive world you can put in a receipt). The integer renderer (int_world.rs) renders through its lowering, integer-operators §13.5 (world/1, informative: no payload text yet binds a trace to a renderer's set, so the capability carries no ops member)
wai.mesh.glb	glTF 2.0 Binary (Khronos)	asset, content-addressed (decoder sink-supplied)
wai.splat.spz	SPZ (an open-source Gaussian-splat format)	asset, content-addressed; presentation only
wai.audio.scene	WSCN scene (objects + trajectories)	mixdown-equivalence · spatial-audio
wai.audio.score	WSCR container (note events)	music as instructions; sink synthesizes, mixdown-equivalence · score
wai.audio.int_codec	WIA1 — a LEARNED 1-D transform codec: integer transposed-conv synthesis + integer IGDN over a factorized-rANS latent (tools/wai_int_audio_export.py writes the model and a JSON form a converter reads)	sample-equivalence: int16 PCM byte-identical on every machine (a portable PCM hash); the audio analog of wai.neural.int_hyper, where a float learned-audio decode desyncs on ~1e-5 round-off. Pure i64, no float/ONNX (int_audio.rs). Payload: integer-payloads §5; decoded through its lowering, integer-operators §13.3 (wam1/1)
wai.haptic.signal	WHAP envelope (intensity/sharpness keyframes)	sample-equivalence · haptics
wai.feed.state	WWLD, sim wai.world.sim.feed0	replay-equivalence · state-feeds
wai.avatar.pose	WWLD, sim wai.world.sim.skeletal0	pose-equivalence · avatars
wai.splat.4d	W4DS container (per-Gaussian keyframed tracks)	frame-state-equivalence · volumetric
wai.splat.int_codec	WIS1 — a LEARNED Gaussian-splat ATTRIBUTE codec: a learned transform over the [14,S,S] attribute image (pos/scale/rot/color/opacity), integer synthesis (int_hyper machinery, 14 channels) over a factorized-rANS latent (int_splat.rs; tools/wai_int_splat_export.py writes the model and a JSON form a converter reads)	splat-equivalence: the decoded attribute tensor is byte-identical on every machine (a portable cloud hash) — the int_hyper of 3D, where a float splat-attribute decode desyncs. The float rasterizer stays sink-supplied presentation; the attributes are the contract. Pure i64, no float/ONNX. Payload: integer-payloads §6; decoded through its lowering, integer-operators §13.2 (whm1/1)
wai.film.linear	WFLM container (sealed world + camera track + frame schedule)	film-equivalence · linear-film
wai.film.reel	WREL container (sealed world + a sequence of shots = camera track + schedule each)	reel-equivalence (film-equivalence with cuts) · linear-film
wai.ternary.tt	WQTT container (balanced-ternary MPS cores — packed trits + integer per-column scales)	a quantum-inspired (tensor-network) transform ported from the ternary/MPS fabric with ONE change: the per-column scale is an integer, so the whole decode is i64/i128 and the reconstruction is byte-identical on every machine — reconstruction-equivalence (a portable recon hash), receiptable. Scope: a provenance transform, measured at 13.3 dB SNR for 3.8× with six residual layers (Lossy tier; the extension's §5). Encoder-side float SVD only · ternary-tt
wai.quantum.circuit	WQC container (gate op-log — Clifford+T + dyadic phase)	the wire carries a circuit (kilobytes, linear in gate count), the sink CLASSICALLY simulates it in a fixed-point complex floor (Amp = i64 re/im), and the reconstructed 2^n-amplitude statevector is byte-identical on every machine — statevector-equivalence (a portable statevector hash); measurement is sampled by the pinned splitmix64 so a shot histogram at a pinned (seed, shots) converges too (shot-histogram-equivalence). The dyadic phase ladder transports the QFT ("quantum FFT") as a circuit, reconstructed byte-exact; the phase table is frozen as existing content was made with it, so P(k) deviates from the ideal phase for k ≥ 8 and applies none for k ≥ 19, and new content uses wai.quantum.circuit2. No quantum hardware — NISQ noise breaks the determinism the standard rests on; this makes a quantum computation transportable + receiptable, with no speedup claimed · quantum-sim
wai.quantum.circuit2	WQC container under magic WQC2 (the same gate op-log)	the successor of wai.quantum.circuit, with the same statevector- and shot-histogram-equivalence: a correctly rounded phase table (each component of every P(k) within half a unit of 2^30·e^{2πi/2^k}), so the dyadic phase ladder gives the QFT; a byte-canonical contract; an op-log that ends exactly after its last op. A new capability, not a change to the old one, because a registered capability's payload format is fixed (§8): a reader that predates it rejects a WQC2 payload at the magic · quantum-sim

The contract of a wai.quantum.circuit2 payload names its arithmetic wai.det.amp64: complex amplitudes as i64 pairs at scale 2^frac, each product exact and then rounded half up, each sum exact. Inside a wai.quantum.circuit contract, wai.det.fixed64 names that same arithmetic at frac 30. quantum-sim §2.1 defines both.

Payloads the quantum-sim reader rules make non-conforming (their envelopes still parse). quantum-sim §2 and §2.1 state reader rules that enforce the registration of wai.quantum.circuit as it was made (§8). A sink refuses, when it decodes it, a wai.quantum.circuit payload whose contract does not carry the registered ext, numeric, frac and gateset, a wai.quantum.circuit2 contract among them; whose n_qubits is not an unsigned integer of at most 255; that carries a known section kind more than once; that has a section whose bounds overflow; or whose contract or measurement repeats a member name within one object, holds a noncharacter, or holds a number of magnitude 2^1024 − 2^970 or more that an earlier reader's float parser let through. No payload a reference writer wrote is among them; quantum-sim §8 lists them with the rules.

And the verify half — a signed, energy-accounted lineage for opaque captured content WAI does not reconstruct (content-addressing + receipts + measured joules): extensions/provenance.md (wai.asset.provenance).

The same verify half, specialized to quantum operations — a signed, content-addressed, joule-metered, grant-authorized receipt of a calibration run (wai.quantum.calibration) or a dispatched QPU job (wai.quantum.job), where a job binds the exact calibration state it ran under and verify() enforces the grant's joule ceiling (gate ⊗ meter ⊗ receipt in one object). WAI owns the receipt; the sink's control system runs the hardware: extensions/quantum-ops.md.

Payloads of the WAI-native integer codecs

This revision pins a payload format to each WAI-native integer codec whose row named its decoder and no byte layout. The rows of wai.video.int_motion, wai.audio.int_codec and wai.splat.int_codec also named the tool that wrote a bitstream, a JSON document, which this section's opening called the bytes the codec normally emits; those of wai.neural.int_hyper and wai.video.int_hyper named their decoders only. The pin completes these registrations as §8 permits (Completing a registration). integer-payloads §3–§6 defines each layout and its decode, §7 the checks a sink makes, and §8 the closed list of codes it refuses with. The rows above keep their headings.

capability	payload format	parameter set
wai.video.int_motion	WIV1, keyframe 0	none
wai.video.int_hyper	WIV1, keyframe 1 (a nested WIH1)	several files: g_s.whm, h_s.whm, entropy.json
wai.neural.int_hyper	WIH1	several files: g_s.whm, h_s.whm, entropy.json
wai.audio.int_codec	WIA1	one file: model.wam
wai.splat.int_codec	WIS1	one file: g_s.whm

From this revision on, the format of each is fixed as §8 states. The JSON the encode and export tools write is a converter input (integer-payloads §10), never a payload: an encoder MUST NOT carry it under these strings, and a sink refuses it (bad_magic).

Envelopes the pin makes non-conforming (they still parse): a WAI1 whose capability or fallback is one of these strings and whose payload is not that string's format, a tool's JSON form among them; and a WAI2 with a component of one of these strings whose bytes are not its format. A sink refuses such a payload when it decodes it, and does not try the fallback (integer-payloads §11).

Payload of wai.generate.spectral: the instruction object (form v0)

This revision completes the registration of wai.generate.spectral as §8 permits (Completing a registration): its row named the generator, generate.rs, and no byte layout. Its payload is an instruction object, form v0: one JSON object (RFC 8259) in UTF-8 with exactly the five members below, written in ascending byte order of their names, with no whitespace, and with each integer in decimal, without sign, fraction or exponent. Nothing precedes or follows the object.

{"model":"sha256:963ee634db1a964d14004618b139fd7e32598f627c023faf9526b14611504f9e","prompt":"a calm spectral horizon","seed":7,"sizes":[[256,128],[640,320],[1280,720],[1920,1080]],"steps":4}

member	value
model	sha256: followed by the SHA-256 of the model file the reference generator reads, as 64 lowercase hexadecimal digits
prompt	0 to 256 bytes, each from 0x20 to 0x7E other than " (0x22) and \ (0x5C), so the string holds no escape
seed	an integer from 0 to 2³² − 1
sizes	1 to 16 pairs [w, h], each side from 1 to 4096 and w·h at most 2²³, in strictly ascending order of w and then h
steps	an integer from 1 to 64

These rules leave one serialization for each object, so two objects that say the same thing are the same bytes, and an envelope's content hash names the instructions. A sink MUST refuse a payload that is not exactly that serialization of such an object — a member added, missing, repeated or out of order, whitespace, an escape, a value out of range — and make nothing from it.

The model file is the generator's parameter set (§3.1), and model is its identity: the object names it, so the envelope does not. An envelope that names wai.generate.spectral, as a WAI1's capability or fallback or as a WAI2 component, carries no prior pin, not even one over the file model names; a sink MUST refuse one that does, whatever the pin names, and make nothing from it. A sink checks the model file it holds against model alone. (This revision registers no one-file shape for this parameter set, so §3.1 would refuse a form-less pin over it with form_required in any case, and a pin in the form wai.prior.set would name the same file twice.)

A sink makes the picture at a listed size [w, h] with the reference generator, generate(model, prompt, seed, steps, h, w): h·w·3 bytes of RGB8 as reconstruction-binding §2 defines it. It MUST NOT make one with a model file whose SHA-256 is not model, or at a size the object does not list. It MAY decline a listed size it cannot afford to make, and then presents nothing for that size. Conformance is generation-equivalence (§7) at each listed size: every conforming sink makes the same bytes. A publisher MAY state each size's SHA-256 beside the payload; a sink that holds such a digest MUST NOT present a picture whose SHA-256 differs from it, and MUST NOT accept one on a 64-bit FNV-1a alone (§7, Digests). The intent of an envelope that names wai.generate.spectral is create: a generator has no source to replicate (it does not back replicate, §7) or to improve.

The conformance vector is wai-rs/tests/vectors/generate/instr-v0.json: the object above and the SHA-256 of each of its four listed sizes; objects at each inclusive bound above, which a sink MUST accept, each with the SHA-256 of its first listed size; and objects a sink MUST refuse, each with the rule the reference reader finds broken first (informative: a sink refuses each, and says why in its own words). wai generate is the reference: --instr makes a listed size from an object, and --write-instr writes one.

Envelopes this makes non-conforming (they still parse): a WAI1 whose capability or fallback is wai.generate.spectral and whose payload is not an instruction object of form v0, whose intent is not create or which carries a prior, and a WAI2 with a component of that string whose bytes are not one, which carries a prior, or in an envelope whose intent is not create.

Immersive audio, cinema, volumetric, haptics, HDR (registered, sink-supplied)

The same "name the capability, the sink supplies the decoder" pattern extends to the standard carriers of the immersive / theatrical / volumetric media a full distributor needs. Each names an open reference where one exists; the proprietary and SEP-pool entries are flagged, never defaults. Payload is the named standard's canonical bytes.

class	capabilities
immersive audio	wai.audio.iamf (IAMF, royalty-free) · wai.audio.ac4 (AC-4, ETSI TS 103 190, proprietary) · wai.audio.ivas (3GPP IVAS, SEP-pool) · wai.audio.mpegh (MPEG-H 3D Audio, SEP-pool) · wai.audio.avs3 (AVS3-P3 / Audio Vivid, SEP-pool) — the last two with the broadcast codecs below
cinema mastering	wai.image.jp2 · wai.video.j2k (JPEG 2000 / DCP, OpenJPEG)
volumetric / 3D	wai.mesh.draco (royalty-free) · wai.pc.gpcc (G-PCC, ISO/IEC 23090-9) · wai.pc.vpcc (V-PCC, ISO/IEC 23090-5) — MPEG point clouds · wai.splat.sog (Gaussian-splat compression; a candidate, no payload format pinned — Candidate capabilities, below)
haptics	wai.haptic.mihs (MPEG-I Haptics, ISO/IEC 23090-31; its 23090-33 ships bit-exact conformance vectors)
HDR dynamic metadata	wai.meta.hdr10plus (SMPTE ST 2094-40 dynamic metadata, royalty-free) · wai.meta.dovi (SMPTE ST 2094-10 dynamic metadata, proprietary) — companion descriptors to a video payload, not standalone media

Point clouds: what the two registrations name. wai.pc.gpcc names G-PCC, ISO/IEC 23090-9, and wai.pc.vpcc names V-PCC, ISO/IEC 23090-5; each payload is a bitstream of the part it names. A bitstream of another part of MPEG-I is not a payload of either unless it is also a bitstream of that part. Enhanced G-PCC, ISO/IEC 23090-38, is such another part, with its own conformance part, so it would be registered, if at all, under a capability string of its own (§8). Neither registration names an edition, so each carries the edition of its part current when it was registered, on 2026-06-17 (§8, Editions). Which edition that was is unresolved: the research registry records it as open, as it does whether wai.pc.vpcc carries the V3C bitstream of MIV. Until a revision records the edition, what either registration admits beyond the tools of its part's first edition is unspecified: a sink that advertises either capability is not known to decode a bitstream that uses a later tool, and an encoder that emits one makes no claim a sink can rely on. The amendments in preparation that add Gaussian splats to both parts were not published on the registration date, so each needs a capability string of its own (§8). Whether V-DMC, which the fourth edition of ISO/IEC 23090-5 makes a V3C application, is admitted turns on the unresolved edition.

Candidate capabilities. A candidate is a registered capability string whose payload format no revision has pinned. The candidates are wai.tensor.nnc (general-tensor coding is in an unpublished NNC amendment; Prior artifacts above) and wai.splat.sog (registered for a family of Gaussian-splat formats; no specification of the format is cited). A conforming encoder's payload is the canonical byte stream of its capability (§7), and a candidate has none. So, until a revision pins a candidate's payload format:

• an encoder MUST NOT emit it: not as a WAI1 capability or fallback, and not as a WAI2 component in either role;
• a sink MUST treat it as unsupported: it takes neither step 3 nor step 5 of §4 with a candidate, and §7.1's selection never picks a component of one.

The revision that pins a candidate states its payload format, and §8 fixes that format from that revision on. This rule enforces the candidate registrations as they were made (§8); it changes no payload format and no role. It makes non-conforming a WAI1 envelope that names a candidate as its capability or its fallback, and a WAI2 envelope with a component of a candidate; such envelopes still parse. The reference implementation lists the candidates in codecs::is_candidate, reports such envelopes from Wai::check_candidates and WaiMulti::check_candidates, and never dispatches or selects a candidate: Wai::dispatch_verified, WaiMulti::pick, WaiMulti::primary and the C wai_envelope_select take only a capability codecs::is_dispatchable accepts (§4's dispatchable). Its wai wrap tool, its C envelope packers (wai_envelope_pack, wai_envelope_pack_multi) and its browser writers (pack, packMulti) refuse to write one.

Refinement capabilities (staged delivery)

capability	staged form	payload	registry class
wai.image.int_refine	residual	WIR1 (staged-delivery Appendix A.1), units RGB8	StandardDefined
wai.video.int_refine	residual	WIR1, units RGB8 frames	StandardDefined
wai.audio.int_refine	residual	WIR1, units i16 PCM	StandardDefined
wai.splat.int_refine	residual	WIR1, units attribute tensors	StandardDefined

Each is refinement-only (§8): its payload is applied, in a declared order, to the output of the stage below it, bound to that output's producer by content hash, and never decoded on its own. It is carried only as a WAI1 with refines (§3), or as a WAI2 component with role: "refinement" (§7.1). It is never a capability or fallback that §4 dispatches, a selectable rendition or a companion. Each is royalty-free and WAI-native, its payload format is pinned by this registration, it reads no parameter set, and its tier is Lossy: its residual is quantised. Its next stage is exact given the stage below, so its class is StandardDefined and its output is as exact as the base it refines (staged-delivery §5).

Base admission

A capability applied to the output of a base — a refinement-only capability, or an enhancement layer — applies only over the bases listed for it here. A later revision MAY add a base, which widens the capability (§8).

capability	admitted bases
wai.image.int_refine	wai.neural.int_hyper, wai.neural.int_mlicpp, wai.neural.int_synth, wai.image.png
wai.video.int_refine	wai.video.int_motion, wai.video.int_hyper
wai.audio.int_refine	wai.audio.int_codec, wai.audio.flac
wai.splat.int_refine	wai.splat.int_codec
wai.video.lcevc	wai.video.avc, wai.video.hevc, wai.video.vvc, wai.video.evc, wai.video.evc.baseline, wai.video.vp9, wai.video.av1, wai.video.av1.lossless (Enhancement layers (LCEVC), below)

A refinement is admitted over a base when its refines.base names an envelope that a sink decoded through one of these capabilities. The reference implementation holds this table once (codecs::BASE_ADMISSION), and every admission check reads it.

Broadcast and streaming codecs (registered, sink-supplied)

The elementary video and audio bitstreams that terrestrial and satellite broadcast, streaming and device playback carry. Registering them lets one envelope name the bitstream a distribution chain already holds, and dispatch it with a declared fallback or a sibling rendition. Every entry is sink-supplied — the reference implementation bundles no decoder for any of them — and none is in the mandatory floor or the recommended set: registration never makes a capability a default. Payload is the named standard's canonical elementary bitstream in the framing given; WAI does not re-frame it.

capability	standard	payload (canonical bytes)	licence
wai.video.av2	AV2 (AOMedia) — AV2 Bitstream & Decoding Process Specification v1.0.0, 28 May 2026	the length-delimited bitstream of that specification's Annex B (each OBU preceded by its leb128 byte length)	royalty-free
wai.video.vvc	VVC — ITU-T H.266 | ISO/IEC 23090-3	H.266 Annex B byte stream	SEP-pool
wai.video.hevc	HEVC — ITU-T H.265 | ISO/IEC 23008-2	H.265 Annex B byte stream	SEP-pool
wai.video.avc	AVC — ITU-T H.264 | ISO/IEC 14496-10	H.264 Annex B byte stream	SEP-pool
wai.video.vp9	VP9 Bitstream & Decoding Process Specification	VP9 frames in the IVF frame container (FourCC VP90)	royalty-free
wai.video.evc	EVC — MPEG-5 Part 1, ISO/IEC 23094-1, any profile	raw bitstream file storage format, ISO/IEC 23094-1 Annex B	SEP-pool (Main-profile tools)
wai.video.evc.baseline	EVC, Baseline profile only	as wai.video.evc, Baseline-conformant	royalty-free by design
wai.video.avs3	AVS3 video — AVS3-P2, T/AI 109.2-2021, IEEE 1857.10	AVS3-P2 elementary bitstream	SEP-pool
wai.video.lcevc	LCEVC — MPEG-5 Part 2, ISO/IEC 23094-2	LCEVC enhancement as the ISO/IEC 23094-2 Annex B byte stream (start-code-delimited NAL units) — companion-only, see below	SEP-pool
wai.audio.mpegh	MPEG-H 3D Audio — ISO/IEC 23008-3	MHAS packet stream (MPEG-H 3D Audio Stream)	SEP-pool
wai.audio.avs3	AVS3 audio / Audio Vivid — AVS3-P3, T/AI 109.3-2023; GY/T 363-2023; GB/T 46271-2025	AVS3-P3 elementary bitstream	SEP-pool
wai.audio.aac	AAC-LC, HE-AAC, HE-AAC v2 — ISO/IEC 14496-3	ADTS stream (ISO/IEC 13818-7 / 14496-3); Extended HE-AAC is not this capability	SEP-pool

SEP-pool means the standard's essential patents are licensed for royalties — through patent pools, a single standard-essential-patent licensor, or FRAND holders; it is the patent-pool licence class (Licence classes, below). Per-entry sources and caveats are in the research registry linked below. Each is tiered Lossy (its distribution role; the lossless modes some of them have are signalled in the bitstream, as for AVIF). No model WAI runs is involved in any of them. For replication (§7) they fall in three classes:

• StandardDefined — AVC, HEVC, VVC and VP9. Each standard's decoder conformance requires every sample a conforming decoder outputs to equal the value its decoding process produces (H.264 §C.4; H.265 and H.266 §C.5.1; VP9 §8.1), so every conforming decoder outputs the same cropped decoded pictures. The guarantee stops at those pictures: film-grain synthesis an SEI message signals, colour conversion and other post-decoding processing are outside it.
• StandardTolerance — AV2. As for AV1, the output is identical unless the payload signals film grain; then grain synthesis is held only to the absence of "perceptually significant differences" from the reference (AV2 v1.0.0 §7.1).
• StandardUnclassified — EVC (both entries), AVS3 video, LCEVC, MPEG-H 3D Audio, AVS3 audio and AAC. Their conformance documents were not read for this revision, so the registry promises no byte identity for them. For AAC, three decoders on one host gave three different PCM streams from one ADTS stream. LCEVC's decoding process includes a signalled dither step (ISO/IEC 23094-2, clause 8.11). Any of them moves to StandardDefined only with the conformance clause that fixes its output.

The clauses behind each class, what was not read, and the cross-decoder measurements are in the research registry.

Dispatch. These dispatch exactly as §4 describes. Because none is a floor or recommended capability, no sink can be assumed to hold one: an encoder addressing sinks it does not control SHOULD ship such a payload in a WAI2 envelope (§7.1) together with a recommended-set rendition of the same content, and that rendition is what keeps the file from being inert at a conforming sink. A v1.0 fallback alone does not: dispatch step 5 decodes the same payload with the fallback's codec (§4), so a fallback helps only where that codec's decoder accepts the primary's bitstream (wai.video.evc.baseline → wai.video.evc is such a pair; an HEVC, VVC or AV2 bitstream is not an AV1 bitstream). A deployer that must not ship pool-licensed decode filters on the licence class below, not on a hand-kept list.

AV2 placement. AV2 is licensed royalty-free by its standards body (footnote ⁶ of the research registry records third-party assertions), with a published specification and a reference decoder (AVM v1.0.0). It is registered here and not in the recommended set, because that set states what every conforming sink SHOULD implement, and in September 2026 that is not yet a reasonable ask: AVM is a reference model, a production software decoder is still in development, and hardware decode in shipping devices is not confirmed. Promotion to the recommended set is a MINOR revision (§8) once deployable decode exists; the capability string and its payload framing do not change. Until then an AV2 payload SHOULD travel in a WAI2 envelope with a wai.video.av1 rendition of the same content.

Enhancement layers (LCEVC). An LCEVC enhancement is not a picture: it holds residuals that an LCEVC decoder adds to the upscaled output of one specific base encode. This registration names its base codecs: AVC, HEVC, VVC, EVC, VP9 and AV1, as the capabilities that name those bitstreams — wai.video.avc, wai.video.hevc, wai.video.vvc, wai.video.evc, wai.video.evc.baseline, wai.video.vp9, wai.video.av1 and wai.video.av1.lossless — and not AV2, AVS3, JPEG 2000 or the WAI-native integer codecs. The list is WAI's, not the LCEVC standard's: ISO/IEC 23094-2:2021 takes its decoded base picture from "a base decoder" (definition 3.17) and names no base codec among its normative references. A later revision MAY add a base, which widens what the enhancement layer applies to (§8). WAI carries it one of two ways.

• Embedded. The enhancement rides inside the base bitstream (SEI / registered user data). The payload is the base codec's bytes, under the base capability; a sink without LCEVC decodes the base and ignores the enhancement under the base codec's own rules. No wai.video.lcevc component exists.
• Separate. The enhancement, in the byte-stream framing given below, is a WAI2 companion (§7.1): role = "companion", companion_of naming the base capability. This is the shape of LCEVC's dual-track storage in ISO/IEC 14496-15, where an enhancement track refers to exactly one base track, and the companion mechanism fits it with one added rule that binds in both directions: companion_of MUST name one of its base codecs and resolve to exactly one selectable rendition of it in the envelope, and that rendition MUST NOT be the base of any other enhancement layer. companion_of names a capability, not a component, so with two renditions of the base capability it cannot say which encode the residuals belong to; and a sink applies every companion bound to the rendition it picks (§7.1), so two enhancement layers on one base would add two sets of residuals to one picture. Either way the picture is corrupted. Such an envelope does not conform; a sink MUST NOT apply its enhancement layers, and MAY still present a base rendition.

The separate carriage's payload is the enhancement in the byte-stream form of ISO/IEC 23094-2 Annex B, where a start code prefix (0x000001) precedes every NAL unit. ISO/IEC 23094-2 defines an LCEVC bitstream as either a NAL unit stream or a byte stream (definition 3.2); the byte stream is the form that delimits its own NAL units, so it is the one wai.video.lcevc names. A NAL unit stream without start code prefixes (for example with each NAL unit preceded by its length) is not a wai.video.lcevc payload; carrying that form would need its own capability string (§8).

Companion-only capabilities. wai.video.lcevc and the HDR dynamic-metadata descriptors wai.meta.hdr10plus and wai.meta.dovi are companion-only: their payload means nothing without the base it accompanies. An HDR descriptor applies to a rendition whose payload is video (§7.1, rule 2), wai.video.lcevc to one rendition of one of its base codecs (above). Every other registered capability is standalone media and is never carried as a companion (§7.1). A companion-only capability MUST NOT be a v1.0 capability (a WAI1 carries one payload, so there is nothing for it to accompany) or a v1.0 fallback (dispatch step 5 would decode it on its own), nor a v1.1 selectable rendition; a sink never dispatches it standalone. A WAI1 naming one as its capability is inert: its payload is companion data. A WAI1 naming one only as its fallback does not conform, and a sink MUST NOT take dispatch step 5 with it; step 3 is unaffected, so a sink that advertises the envelope's capability still decodes it (§4). The reference implementation checks these rules in Wai::check_component_roles (v1.0) and WaiMulti::check_component_roles (v1.1: the companion rules of §7.1, including both enhancement-layer rules above), dispatches WAI1 envelopes through Wai::dispatch_verified, which applies both sentences above, never returns a companion-only component from pick / primary, and its wai wrap tool refuses to write an envelope that fails either check. Parsing stays format-agnostic: a non-conforming envelope still parses, and the check is what reports it.

The companion binding carries no per-picture timing. The framing above fixes the bytes of a separately carried enhancement stream, not how its access units align picture by picture with its base outside a timed container; this revision does not pin that alignment, so the embedded form is the unambiguous one until it does.

Licence classes. Every payload and companion capability carries a licence class — codecs::capability_license(cap) ∈ {royalty-free, patent-pool, proprietary, non-commercial}, with is_royalty_free(cap) as the predicate. It is a registry property (not a per-file manifest claim — a codec's licence can't be honestly asserted in an envelope's header), so a deployer can audit a title or a whole catalogue in code: encode only to royalty-free capabilities, or flag every pool/proprietary one. The classification is exhaustive with no default class: a capability registered without an explicit class is a registry defect the reference tests reject, so no codec becomes royalty-free by omission. Where a capability's royalty status is not established, it is classified conservatively (patent-pool), never royalty-free; a candidate's class is provisional until a revision pins its payload format. The same tests hold that every floor and recommended-set capability is royalty-free.

See research/media-codec-registry.md for the per-capability standard / spec number / verified royalty status / reference library, what each named standard's decoder conformance fixes (§7), and the remaining spec follow-ups.

---

6. Confidentiality (OPTIONAL — encrypted payloads / private channels)

By default a WAI payload is plaintext: any sink that advertises the named capability (§4–§5) can decode it. This is correct for public media — a shared world, a broadcast film. Some channels are instead private: only an intended group may recover the content, and any relay or multiplexer that carries it MUST be able to route, meter, and attest the bytes without reading them.

Confidentiality is an OPTIONAL layer that is orthogonal to the capability. It wraps the payload bytes; the capability still names how to decode the plaintext once recovered. A sink decrypts first, then runs capability dispatch (§4) on the plaintext.

§6.1–§6.4 define that layer. §6.5 sets it beside hardware-protected content: the two lanes protected content travels in, and what a WAI record covers in each.

6.1 Manifest field

A manifest MAY carry a security object:

"security": { "encryption": "mls-sframe", "kid": 0 }

• security.encryption — encryption scheme id. Absent, null, or "none" ⇒ the payload is plaintext (the default; §1–§5 unchanged). "mls-sframe" ⇒ the payload is an SFrame (RFC 9605) ciphertext object whose group key is agreed by MLS (RFC 9420), following the construction of draft-jennings-moq-e2ee-mls, an individual Internet-Draft that has since expired (confidentiality §1).
• security.kid — the SFrame key id selecting which sender/epoch key opens the object. It is also carried in the SFrame header; the manifest copy lets a multiplexer route without parsing the payload.

media, intent, conditioning.kind, and model_requirement.capability continue to describe the plaintext codec. They are NOT encrypted and remain visible for routing — confidentiality protects the content, not the shape.

6.2 Keys are a requirement of existence

Exactly as the model behind a neural capability is never transported (§1), the group key is never transported in the WAI file. Group membership is established out of band via MLS; each member derives the 32-byte SFrame base key from the group's exporter secret under the label "moq-jwp sframe base key". A sink that is not a group member cannot derive the key and the file is inert at that sink — the confidentiality analog of a missing capability.

This is the content key. Checking a receipt needs no content key, only the signer's verifying key, which the receipt itself carries (signer_pubkey): where a signer publishes its verifying keys, and how a verifier confirms whose a key is, is extensions/jwp-receipts.md §7.

6.3 Relay transparency and attestation

A WAI relay or multiplexer that carries an encrypted payload MUST treat it as opaque: it MUST NOT require the plaintext to route, meter, or attest, and MUST NOT attempt capability dispatch on ciphertext. The JWP receipt chain (extensions/jwp-receipts.md) commits over the ciphertext content hash, so each object's integrity, and its delivery under the publisher's signature, are attested end-to-end without any relay holding the content key. Each object's figure, class and origin are signed where its leaf is labelled, and the group total, its class, its coverage and the parent link are signed under the labelled group payload (jwp-receipts §2.2–§2.3). An object's figure and origin beside a legacy leaf are the publisher's statement, unsigned beside the signed bytes; such an object's figure is signed only where an energy claim restates it (jwp-receipts §2.4.5). A legacy group payload signs the total, but no class, coverage or parent link. Content secrecy (SFrame/MLS) and delivery attestation (JWP) are independent layers over the same bytes and compose: a relay forwards the exact sealed bytes and their receipts unaltered, signs nothing in the publisher's name (jwp-receipts §4.3), and never learns what the bytes say. Checking the attestation needs the publisher's verifying key, which the receipt carries, never the content key; whose key it is, jwp-receipts §7 confirms.

6.4 Decrypt-then-dispatch

A confidentiality-aware sink MUST, before running §4:

1. If security.encryption is absent / null / "none", proceed to §4 on the payload unchanged.
2. Else, if the sink does not implement the named scheme, or holds no key for security.kid, the file is inert at this sink — return a clear "no key / unsupported encryption" error. It MUST NOT feed ciphertext to a codec. (The reference dispatchers in wai-rs implement no scheme and report unsupported_encryption, for a WAI2 envelope against every entry.)
3. Else recover the plaintext payload via the named scheme (for mls-sframe: SFrame open under the member's base key), then run §4 on the recovered plaintext.

A sink that does not implement confidentiality at all MUST refuse an envelope whose manifest carries a non-none security.encryption cleanly, rather than mis-decode ciphertext.

Byte-level SFrame framing, the SFrame key schedule, and the MLS binding (including per-sender keying for multi-sender channels) are specified in extensions/confidentiality.md. Its implementation is a companion implementation, outside this tree and under its own licence: the moq-sframe (SFrame) and joule-comm-mls (MLS exporter → base key) crates, with browser bindings in joule-comm-mls-wasm; Track::e2e_encrypted is the switch that selects this path. Live: https://wai.transaction.science/private-channel

6.5 Protected content: two lanes

Content whose licence requires protection travels in one of two lanes. They keep the content from different parties, and a WAI record covers different things in each. A sink, a receipt, a tier claim or a status text MUST NOT present content in one lane as being in the other.

Lane A: hardware-protected content. The media is a bitstream of a capability whose registration names its standard (§5), encrypted for a platform's protected media path: a content decryption module (in a browser, the one the W3C Encrypted Media Extensions interface reaches) and a decoder whose output the application cannot read. Decryption and decode happen inside that path.

• This revision registers no security.encryption value for Lane A, so no WAI envelope carries Lane A media: it travels in the form its protection system defines, and WAI carries records about it. An envelope that names an encryption the sink does not implement is inert at that sink (§6.4 step 2).
• WAI attests about a Lane A title only through its records, among them the manifest a session was served (wai.video.manifest), the package's segment map over the bytes it sliced (wai.video.package), each key-release decision, which binds the released key by commitment and never carries it (wai.video.keys), the ad decision and the per-impression receipts that bind each served creative_hash and the quartile it reached (wai.video.ssai), a channel's program-as-run windows (wai.video.channel), and the content hash of each WAI object delivered (jwp-receipts §2.1). None of them covers a decoded frame or sample.
• A capability that synthesises output from parameters (grain, an enhancement layer, a generative stage) MAY apply to Lane A content only inside the protected path, in a decoder that the protection system certifies. WAI does not certify that decoder and cannot read its output, so such synthesis has no tier and no output digest, and a record about it states at most which capability the sink declared it ran.

Lane B: end-to-end encryption at the application layer. The payload is sealed per object under a group key, mls-sframe (§6.1–§6.4, confidentiality §3–§5). A member's application opens it and runs §4 on the plaintext, so any decoder a sink has can decode it: native code, WebAssembly, or a neural decoder.

• The sink that holds the key decodes the plaintext, so the capability's tier over its decoded output (determinism-tiers §2) and the output digests it states in a reconstruct-session claim (jwp-receipts §2.4.12) hold as they do for plaintext, and the receipt chain covers the ciphertext (§6.3). Not every record carries over: staged-measurement counts no encrypted object and publishes no byte figure for one (staged-measurement §3, Only a checked format is counted, code encrypted), and a verifier of a reconstruct-session claim that holds no key checks none of the rules that need a payload, which stay the signer's statement (jwp-receipts §2.4.12, rule 3). The reference dispatchers implement no scheme (§6.4 step 2), so nothing in the reference states a tier, an output digest or a binding for Lane B content.
• Lane B is not a protected-path robustness level. It keeps the content from relays and from anyone outside the group (§6.2, §6.3). It does not keep it from a member's device: the base key, the plaintext and the decoded output are in the application's memory, readable by the application and by anything that can read that memory. A publisher MUST NOT offer Lane B to meet a licence term that requires a hardware-protected path, and a sink or a receipt MUST NOT describe Lane B content as hardware-protected.

What the exact tier covers. decode-equivalence covers the medium a sink emits and can hash (determinism-tiers §2), on the fabrics determinism-tiers §5 lists: native CPU and WebAssembly in this revision, with no GPU or NPU decoder. It never covers output the application cannot read. A sink MUST NOT state a tier, an output digest, a reconstruction binding or an energy binding for output decoded inside a protected media path, and a reconstruct-session claim (jwp-receipts §2.4.12), whose every presentation carries the SHA-256 of its output, lists no presentation whose output its sink could not hash. In Lane B the tier holds as it does for plaintext; in Lane A there is no output for it to cover.

---

7. Conformance

• A conforming sink MUST: implement the container (§2), capability dispatch (§4), and the mandatory floor (§5).
• A conforming encoder MUST emit containers (§2) that any conforming sink can read; the payload MUST be the canonical byte stream for the registered capability.
• A decode is conformant by the criterion its capability's registry class (below) sets. A StandardDefined or NeuralIntegerExact capability is verified by bit-exact decode-equivalence: against the in-crate decoder or wrapped reference library for the capabilities the reference implementation decodes (codecs::sink_capabilities() and the in-crate neural and WAI-native decoders), and against the named standard's own conformance bitstreams and decoded outputs for a sink-supplied codec. A GenerativeDeterministic capability is verified by bit-exact generation-equivalence at its pin. A StandardTolerance capability conforms by its named standard's own decoder-conformance criterion around a reference, not by bit-exact equality with one library's output; a StandardUnclassified capability conforms by its named standard's criterion, which this revision has not classified; WAI adds no bit-exact requirement to either. A NeuralEntropyExact capability conforms, as WAI registers it, at the entropy-decoder output (for wai.image.jpegai, the criterion its named standard sets on the reconstruction is not read in this revision). NeuralFloat capabilities make no claim that their reconstruction is bit-exact across sinks, and Behavioral ones conform by similarity. The WAI Rust reference impl in wai-rs/ ships a test suite that carries every registered capability through the full WAI envelope (cargo test --lib, tests/carriage.rs).

Replicate-soundness — the integer-entropy contract (normative)

intent = replicate promises the reconstruction the capability's registry class fixes (the classes are below). For every class that backs it except two, that is a byte-exact reconstruction: every conforming sink that decodes the payload produces the same bytes of the medium (decode-equivalence, determinism-tiers §2). The two exceptions are the standard codecs whose named standard fixes less than the bytes (StandardTolerance) or has not been checked (StandardUnclassified); over those, replicate promises that standard's decoder conformance and no more (What replicate promises over a standard codec, below).

For a neural capability replicate promises decode-equivalence, and this revision does not substitute a standard's decoder conformance for it. The one registered neural capability with a named external standard is wai.image.jpegai (ISO/IEC 6048-1:2025). That standard's conformance part may set a criterion on the reconstruction — an overview of the standard says the part "considers the possibility that a decoder is conformant to the standard without having bit-exact reconstruction" — but this revision has not read it, so holding wai.image.jpegai to the neural rule is a conservative choice pending that reading, not a finding that its standard leaves the reconstruction unbounded (NeuralEntropyExact, below). Decode-equivalence is only sound if the decode is byte-identical on every machine — and that is not automatic. A learned codec's entropy decode desyncs the instant the decoder's scale-bucket / σ computation drifts from the encoder's by one float ULP: the bucket assignment flips and the rest of the stream corrupts. So a neural decode that runs float inference through a runtime — which chooses kernels, accumulation order, vector width and fused multiply-adds per device — is reproducible only on the same build, never guaranteed across hardware.

IEEE-754 basic operations are themselves correctly rounded. Float reproduces across machines only when one implementation fixes everything a runtime leaves free: the operations and their order, their precision, no contraction or reassociation, no library function whose last-place error can reach the output — and runs in the default floating-point environment (round-to-nearest-even, no flush-to-zero, no denormals-are-zero), which a host process can change. These are properties of one build and one process that neither the envelope nor the registry can state, which is why the rule below requires integer arithmetic.

Therefore a neural capability MAY back intent = replicate only if its entire decode path — entropy parameterization and synthesis — is integer / fixed-point, with no float operation anywhere (the integer-entropy contract). That is the only construction by which a neural capability is registered at decode-equivalence. Two weaker properties MUST NOT back replicate:

• entropy-consistency — the entropy decode (the decoded symbols, and the parameters they are decoded with) reproduces on every machine, but later stages run float, so the reconstruction may differ between conforming sinks. It is necessary for a sound neural replicate — without it the decode desyncs — and not sufficient, because replicate promises the reconstruction, not the symbols.
• A reconstruction pinned by one implementation — float stages in a fixed evaluation order, reproducible across machines only under that implementation's build and the process's floating-point environment. Those conditions are the ones the envelope and the registry cannot state, so a replicate resting on them promises something no sink can check from what it received.

The reference impl factors the halves of the contract into ORT-free, pure-i64 modules a codec composes: int_entropy (the deterministic scale→bucket step that removes the float-σ desync), int_synth (integer synthesis), and the integer rANS — assembled into a full conditional decode by int_hyper_synth (wai.neural.int_hyper: factorized rANS-decode z → integer hyper-decoder h_s → σ,μ → integer-σ bucket → rANS-decode residual → y_hat = r + μ → integer synthesis — both bitstreams entropy-coded, no float anywhere). This is a property of the capability, classified in the registry as codecs::capability_replicate_class(cap) -> ReplicateClass:

• NeuralIntegerExact — integer by construction; byte-identical on every machine; registered at decode-equivalence, and the only neural class that backs replicate. (wai.neural.int_synth is synthesis only; wai.neural.int_hyper is the full conditional codec — σ source, entropy, and synthesis all integer; wai.neural.int_mlicpp is the channel-autoregressive × checkerboard entropy model on integer networks, so the σ it re-derives before each of its 2·K passes is integer too; wai.video.int_hyper, wai.audio.int_codec and wai.splat.int_codec are integer throughout as well.) A capability of this class whose networks decode through the integer operator set names it in the registry, in its ops member (integer-operators §11): the lowering its sets are lowered by, the profile every conforming set's certificate meets, and the modes of its stage operators. A set whose certificate does not meet that profile is refused. In this revision: wai.neural.int_synth, wai.splat.int_codec, wai.audio.int_codec, wai.neural.int_hyper, wai.video.int_hyper and wai.neural.int_mlicpp.
• NeuralEntropyExact — the entropy decode is integer by construction, so the decoded symbols are byte-identical on every machine; later stages run float. Registered at entropy-consistency. An encoder MUST NOT emit intent = replicate for one of these. (wai.image.jpegai: integer mANS entropy decode and integer scale-index derivation on both reference decode paths; float dequantisation, latent refinement, post-filters and colour conversion. What its exact decoder pins, and under which conditions, is stated in determinism-tiers §5. This class is the neural rule applied to it, not a statement about its standard's conformance on the reconstruction, which this revision has not read; see wai.image.jpegai and the standard classes, below.)
• GenerativeDeterministic — the create-side of byte-exact conformance. A generator (intent = create) whose entire pipeline is integer/fixed-point and pinned to (model, prompt, seed, steps), so the generated content is byte-identical on every machine. There is no original to reconstruct, yet the output hash is portable and receiptable — generation-equivalence. The fixed-point sampler is mandatory: a float RNG or float sampling step desyncs the stream exactly as a float σ does. (wai.generate.spectral: a pure-integer spectral synthesis — prompt+seed draw integer-LUT sinusoid components by a splitmix64 PRNG; the float sin() that bakes the LUT runs once at model-build, never at generation.) Distinct from Behavioral, which conforms by similarity and never reproduces bytes. It generates rather than reconstructs, so it does not back replicate.
• NeuralFloat — runs float inference; exact only on the same build. An encoder MUST NOT emit intent = replicate for one of these; perceptual / "exact on this build" is the honest claim. The registry claims no determinism tier for these.
• StandardDefined — the reconstruction is specified exactly: by the named standard, for the lossless formats whose decoded output it determines (wai.image.png, wai.audio.flac, wai.text.zstd, wai.text.xz) and for the video codecs whose decoder conformance requires every output sample to equal the specified decoding process's (wai.video.avc, wai.video.hevc, wai.video.vvc, wai.video.vp9; clauses in the table below), or by the WAI extension that defines the capability's *-equivalence over integer / fixed-point state (world, score, film, quantum, ternary, the haptic signal, the integer inter-frame codec and the other WAI-native capabilities). replicate promises the same bytes on every conforming sink. An extension's equivalence holds given its inputs: where it composes other capabilities' decoded output (a wai.audio.scene source decoded through wai.audio.opus, say), it is only as exact as those capabilities' classes. wai.audio.scene is the one registered capability whose output depends on envelopes it references rather than carries, so the reference registry does not report it byte-exact from the capability alone (below). No neural decode is in this class, and a capability enters it only with a recorded basis — the conformance clause or extension that fixes its output (the reference registry tests pin one per member); without one, a standard codec is StandardUnclassified.
• StandardTolerance — a standard codec whose named standard, for at least some conforming payloads, holds decoders to a criterion around a reference rather than to identical output. For Opus and JPEG XL the clauses read state it as numeric thresholds. For JPEG and JPEG 2000 the text read points to a criterion without stating its bound (T.81 leaves the accuracy of dequantisation and IDCT to its Part 2, which was not read; only the table of contents and terms of ISO/IEC 15444-4 were read), so this revision states none. For IVAS, an implementation based on its floating-point reference code whose output is not bit-exact is tested by a non-bit-exact procedure instead: a per-frame loudness-difference corridor for stereo and immersive operation, and for mono operation the procedure of TS 26.444, which was not read (3GPP TS 26.252 §7.2.1). For AV1, AVIF and AV2 when film grain is signalled it is the absence of "perceptually significant differences" from the reference grain synthesis, which the AV1 specification leaves undefined (§7.2: its definition "is beyond the scope of this specification") and the AV2 specification leaves to others to define (§7.1). replicate promises what that conformance promises: every conforming sink reconstructs within the standard's criterion — for film-grain AV1 and AV2, a criterion no document WAI cites quantifies. It does not promise byte identity across sinks.
• StandardUnclassified — a standard codec whose named standard's decoder-conformance clauses this revision has not checked. replicate promises what that conformance promises; the registry does not say whether that is byte identity.
• Behavioral — a sink-generator path that conforms by similarity, not bytes (wai.semantic.*).

What replicate promises over a standard codec

A standard codec's reconstruction is what its named standard's decoder conformance fixes, in that standard's own output domain; replicate promises that and no more. The registry classes each one by what the standard fixes (the clauses cited are the ones read; the evidence, and what was not read, is in research/media-codec-registry.md):

class	capabilities	what the named standard fixes
StandardDefined	wai.image.png, wai.audio.flac, wai.text.zstd, wai.text.xz	the decoded output exactly: lossless formats (PNG, W3C Recommendation, Third Edition — its filters are exact integer operations; FLAC, RFC 9639) and lossless byte formats
	wai.prior.set	its bytes exactly: a listing (§3.1) is its own output
	wai.video.avc, wai.video.hevc, wai.video.vvc	every sample of every cropped decoded picture a conforming decoder outputs equals the value the specified decoding process produces (ITU-T H.264 (V16, 06/2026) §C.4; H.265 (V11, 01/2026) §C.5.1; H.266 (V4, 01/2026) §C.5.1); processing an SEI message only signals, such as film grain synthesis, is outside that decoding process
	wai.video.vp9	output frames "identical in all respects", in the same order, to those of the specified decoding process (VP9 Bitstream & Decoding Process Specification v0.6 §8.1)
StandardTolerance	wai.audio.opus	output within opus_compare's thresholds of the reference decoder, with the same final range-decoder state (RFC 6716 §6)
	wai.image.jpeg	decoding "with appropriate accuracy", the accuracy of dequantisation and IDCT specified in Part 2 (ITU-T T.81 §7, A.3.3); Part 2 (ITU-T T.83) was not read, so the bound is not stated here
	wai.image.jxl	decoded samples within per-test peak-error and RMSE thresholds of reference images (ISO/IEC 18181-3 §4.1–§4.3)
	wai.image.jp2, wai.video.j2k	decoded components compared with reference components by error metrics, and a reversible path its definitions describe as lossless (ISO/IEC 15444-4); only the table of contents and terms of its public preview were read, so the metrics and thresholds are not stated here
	wai.video.av1, wai.video.av1.lossless, wai.image.avif	identical output frames, unless the payload signals film grain; then identical frames before grain synthesis, and output frames that "do not have perceptually significant differences" from the reference grain synthesis, a criterion whose definition the specification places outside its scope (AV1 §7.2)
	wai.video.av2	as AV1: identical output unless film grain is signalled, then one of the same two options (AV2 v1.0.0 §7.1)
	wai.audio.ivas	output bit-exact to the test sequences, or, for an implementation based on the floating-point reference code whose output is not, a non-bit-exact procedure: TS 26.444's for mono operation, a per-frame loudness-difference corridor for stereo and immersive operation (3GPP TS 26.252 V19.2.0 §7.1, §7.2.1)
	wai.audio.iamf	its substreams include Opus (above), and renderers are out of its scope
StandardUnclassified	wai.video.evc, wai.video.evc.baseline, wai.video.avs3, wai.video.lcevc, wai.audio.mpegh, wai.audio.avs3, wai.audio.ac4, wai.mesh.draco, wai.pc.gpcc, wai.pc.vpcc, wai.splat.sog, wai.haptic.mihs, wai.meta.hdr10plus, wai.meta.dovi, wai.prior.nnc, wai.tensor.nnc	not checked in this revision (all sink-supplied; wai.splat.sog and wai.tensor.nnc are candidates, with no payload format pinned — §5)
	wai.audio.aac	not checked in this revision; three decoders on one host gave three different PCM streams from one ADTS stream, so it is not placed where byte identity would be promised
	wai.mesh.glb, wai.splat.spz	the payload bytes pass through intact; what a sink decodes from them is not classified here (a glTF binary may carry JPEG images in its buffer and compressed geometry through extensions, and glTF 2.0 specifies an asset format, not a decoder conformance over what is rendered; SPZ stores quantised attributes)

A conversion a sink applies after the standard's decode is not part of the promise. A standard fixes its decoded samples at their own depth, layout and colour representation, and none of those above fixes a conversion to another: PNG sample-depth reduction (the PNG specification gives more than one method), YUV→RGB for AV1 and AVIF, JPEG's colour space (outside T.81's scope, §4.1), rendering an IAMF mix to a loudspeaker layout. The reference sink returns RGB8 images and f32 PCM, so its output buffers include such conversions; they are the reference implementation's, and this revision does not specify them.

wai.image.jpegai and the standard classes

wai.image.jpegai has a named standard, ISO/IEC 6048, but its registry class (NeuralEntropyExact) comes from the neural rule above, not from what that standard's decoder conformance fixes. What this revision has established, and what it has not:

• JPEG AI sets its bit-exact conformance point at the entropy-decoder output, and one codestream may be reconstructed through up to three synthesis transforms, which a profile or the encoder can restrict (an overview of the standard, arXiv 2510.13867; determinism-tiers §1).
• The same overview says the conformance part, ISO/IEC 6048-4, "specifies the requirements for generating a JPEG AI-compliant reconstruction" through a testing suite for each profile and level pair, and "considers the possibility that a decoder is conformant to the standard without having bit-exact reconstruction", which it also calls "the possibility of standard compliance with a certain leeway"; outputs of the processes after entropy decoding "may diverge slightly from the reference". Read together, these passages point to a criterion on the reconstruction around a reference, the kind StandardTolerance records. The overview does not state such a criterion.
• ISO/IEC 6048-4 itself has not been read (the overview describes it as a draft when it was written), so this revision does not state whether it sets such a criterion, of what form, or how wide.

Classing wai.image.jpegai as NeuralEntropyExact, so that replicate over it is refused, is therefore a conservative choice pending a reading of Part 4. It is not a finding that the standard leaves the reconstruction unbounded. If Part 4 states a criterion on the reconstruction, wai.image.jpegai is a candidate for StandardTolerance, over which replicate would promise that criterion; moving it there would also take an exception to the neural rule above, which admits a neural capability to replicate only by the integer construction.

This is not the choice made for the standard codecs. Over a StandardUnclassified codec, whose conformance clauses are unread, and over AV1 or AVIF with film grain, whose criterion the AV1 specification leaves undefined, replicate is honoured, with its promise narrowed to the standard's conformance. Those capabilities keep the behaviour they had before this revision. wai.image.jpegai had been registered at decode-equivalence, which its float stages do not meet, and it was moved to a class that does not back replicate. The two treatments differ because of those choices, not because of anything the standards' texts were found to say.

Registry change in v0.4.0 — a downgrade. wai.image.jpegai was classed StandardDefined, and §5 listed it at decode-equivalence. Its decode has float stages, and a sink without the integer weight bundle decodes it through a float ONNX Runtime path, so neither held as a property of the capability. It is now NeuralEntropyExact, registered at entropy-consistency, and it no longer backs replicate; before, the reference strict check passed a replicate envelope over it whenever the prior declared an integer contract.

Registry change in v0.4.0 — the standard codecs. Every standard codec was classed StandardDefined, and the first form of this revision's §3 promised the same bytes on every conforming sink over all of them. The previously registered capabilities in the StandardTolerance and StandardUnclassified rows above are now in those classes. replicate over them is still honoured; what it promises is narrower — the named standard's decoder conformance, not byte identity across sinks. No other previously registered capability changes class. The broadcast and streaming codecs this revision registers (§5) enter with the classes in the table.

A conforming sink that receives intent = replicate for a capability whose class is NeuralEntropyExact, NeuralFloat, GenerativeDeterministic or Behavioral MUST refuse the replicate claim (decode as perceptual or reject). The capability is the one it decodes through: model_requirement.capability, or the fallback when it decodes via §4 step 5, or, in a WAI2 envelope, the capability of the rendition it picks (§7.1).

The pin under replicate. Over a NeuralIntegerExact capability a conforming sink MUST also refuse the replicate claim unless a pin applies to that capability (§3.1), declares decode-equivalence, and declares no form other than one the registry lists as established exact (prior-carriage §5). A pin declares a tier by tier. A pin without tier declares decode-equivalence exactly when its determinism, compared in ASCII lower case, contains int, fixed or integer and none of f32, f64 and float, and declares no tier otherwise. That legacy reading exists so that a pin written before tier keeps its verdict. It reads a declaration and can misread one (a contract naming fp16 contains no float token); the registry class, checked first, is what guarantees the arithmetic, so a misread declaration cannot admit a float decode. An encoder SHOULD write tier, and SHOULD write a determinism that this reading classes the same way, so that a sink of an earlier revision reaches the same verdict. The refusals are named replicate_unregistered (the capability is not registered), replicate_class (its class cannot back replicate), replicate_unpinned, replicate_tier (the pin does not declare decode-equivalence) and replicate_form_not_exact. They read only the manifest and the registry, so a sink MAY make them before it verifies the pin; refusing the claim does not excuse the pin, which every decode of a pinned payload verifies (§3.1).

The reference impl checks this per-envelope in decode_envelope_strict (codecs::replicate_refusal) in two steps. First the capability's registry class: only NeuralIntegerExact and the three standard classes pass, whatever the envelope declares (ReplicateClass::replicate_promise states what each promises). Then, for a NeuralIntegerExact capability, the pin: it must be present and declare decode-equivalence (Prior::declared_tier), and a pin that declares a form passes only if the registry lists that form as established exact (prior-carriage §5): wai.prior.set is listed, and no coded form is. The prior check reads the sender's declaration; the class check is what ties it to the capability, so an envelope whose prior declares an integer contract over a float or entropy-only capability is refused. decode_envelope_strict applies the rule to the capability it decodes through — model_requirement.capability, or the fallback when dispatch reaches step 5 — and, when the rule refuses every capability the sink could decode through, refuses before it verifies a pin or reads a file; decode_multi_strict considers only the renditions this rule admits, and reports each one it leaves out with that rendition's replicate_* reason (§7.1 step 5). On the encoder side, wai wrap, which pins no prior, writes intent = replicate only where that check accepts it unpinned — over the three standard classes — for the capability and for any fallback it declares, or, in the WAI2 companion form (§7.1), for the base payload's capability and each companion's, and otherwise refuses unless given another intent (codecs::unpinned_envelope_intent). The registry class is also the static audit of the same property: wai capabilities prints it per row and reports the count of neural integer-exact capabilities, so a deployer can require "dispatch replicate only to a cross-hardware-sound decode", or only to a capability whose output is the same bytes on every conforming sink (codecs::is_cross_decoder_exact), in code, exactly as it audits royalty status. The class's promise (ReplicateClass::replicate_promise) and is_cross_decoder_exact say the same about byte identity for every registered capability but one. A capability whose output is computed from other envelopes' decoded output (codecs::composes_decoded_sources: wai.audio.scene) has the class StandardDefined and the promise ReplicatePromise::ByteExact, because its mix is exact given the PCM its sources decode to; is_cross_decoder_exact answers from the capability alone and is false for it, because a source decoded through wai.audio.opus, say, may decode differently on two conforming sinks. A deployer auditing for the same bytes uses is_cross_decoder_exact, not the promise. Once the capabilities of the envelopes a scene references are known, codecs::is_cross_decoder_exact_with_sources is true when the scene's class and every source's promise the same bytes; it answers false when it cannot tell — no source given, or a source outside wai.audio.*, which a scene may not reference. The replicate check above covers one envelope, so replicate over a scene is honoured and promises the same mix only given the same source PCM; each source is an envelope of its own, checked by its own class. That promise covers the output the standard or extension specifies, before any conversion a sink applies afterwards (above), so a hash over a converted output is portable only between sinks that also agree on the conversion. The classification is exhaustive with no default class: a capability registered without one has no class, and the registry tests reject it.

This makes the cross-hardware property explicit per capability: a neural capability whose decode is integer all the way down can put its reconstruction in a receipt that verifies on another machine, and the registry records which capabilities are.

Reproducibility is a spectrum, not a single point, and the ReplicateClass values above span it. The neural classes map onto the two tiers extensions/determinism-tiers.md names. NeuralIntegerExact capabilities are registered at decode-equivalence — the whole decode to the final medium reproduces on every conforming sink, so the rendered result has a portable hash; this is the tier replicate over a neural capability promises, and the precondition for a hard reconstruction binding. NeuralEntropyExact capabilities are registered at entropy-consistency — the entropy decode reproduces across devices (integer entropy arithmetic or explicit parameter transmission), the tier at which JPEG AI sets its bit-exact conformance point; it does not back replicate. NeuralFloat capabilities claim neither tier. A pin MAY state its decode's tier (§3.1, tier); a stated tier above the class's does not conform.

Digests (normative)

WAI names one digest algorithm for each purpose:

• Parameter-set identity — a pin's sha256 and supersedes.sha256, and each line of a listing (§3.1): SHA-256 (FIPS 180-4), written as 64 lowercase hexadecimal digits.
• Component identity — a WAI2 entry's sha256, the digest of its payload bytes (§7.1): SHA-256, written the same way. The content hash a receipt chain takes over a whole envelope (jwp-receipts §2.1) is a different object.
• Decoded output in conformance material — a digest of the decoded output of a capability that decodes against a parameter set (§3.1), or of a standard codec (§5), in a published conformance vector or corpus, in a golden a conformance test compares with, or in a reconstruction or energy binding (reconstruction-binding §2, energy-binding): SHA-256 over the capability's canonical output buffer, written as 64 lowercase hexadecimal digits.
• Receipt content and WAI-native state — an envelope's content hash in a receipt chain, and, for a WAI-native capability that decodes against no parameter set, its state hash and the digests of its rendered output: the algorithm the defining extension names (jwp-receipts §2.1; the extensions §5 links).

A conformance vector or corpus that states a digest of decoded output in the third purpose MUST state its SHA-256 and MUST name the buffer it covers:

• an image, or one frame of video: RGB8 as reconstruction-binding §2 defines it;
• audio: the samples as signed 16-bit little-endian integers in output order, channels interleaved;
• a sequence of frames: each frame's digest, and a clip digest, the SHA-256 of the frames' 32-byte SHA-256 digests concatenated in output order;
• any other output (an attribute tensor, an intermediate stage): the buffer the vector's documentation names, with its element type, byte order and layout.

A vector MAY state other digests of the same buffer, such as a 64-bit FNV-1a or a domain-separated BLAKE3, to locate a divergence. A verifier MUST NOT accept a decode on such a digest alone. A hash a capability uses as a step of its own algorithm (the prompt fold of wai.generate.spectral) is part of that capability, not a digest of its output, and this rule does not reach it. A frozen digest SHOULD be re-derived on more than one instruction-set architecture before it is published.

Signatures (normative)

Every Ed25519 signature (RFC 8032, pure Ed25519) that WAI or one of its extensions defines is verified strictly: on a receipt of any family, a claim, a key-log document, a module grant or status, an op, and a world's seal. A verifier MUST refuse a signature unless each of these holds, in this order:

1. The key is canonical. The public key A is the one encoding of its point that RFC 8032 §5.1.3 allows: y < p, and no sign bit when x is 0. A decoder that reduces y modulo p, or reads the sign bit beside x = 0, reads a second spelling of a point; that key is refused.
2. The key is not of small order. A's order does not divide 8. Under such a key the equation can hold for a signature made without any private key: under the identity, (R, S) = (B, 1) verifies over every message.
3. S < L, the order of the base point B (RFC 8032 §5.1.7).
4. R is not of small order. R decodes to a point whose order does not divide 8.
5. The cofactorless equation holds. [S]B − [k]A, with k = SHA-512(R ‖ A ‖ M) mod L over the bytes of R and A as the signature and the key give them, encodes to R byte for byte; an R that is not its point's canonical encoding never does.

A key that fails step 1 or 2 is a defect of the key, and an extension MAY report it apart from a defect of the signature (jwp-receipts §3.1 step 3: bad_signer_key and signature_verify_failed). The rule fixes what RFC 8032 leaves to the verifier, so every conforming verifier accepts the same signatures.

OER/2 is the one exception: oer-receipts §5 settles its chain signature, which refuses a key by steps 1 and 2 and checks R as RFC 8032 does.

Machine-checked totality (informative)

The cross-architecture matrix establishes byte-exactness by sampling: it decodes fixed vectors on every OS/arch leg of the byte-equality matrix (.github/workflows/wai-byte-equality.yml). wai-verify/ supplies the complementary half by reasoning — a model checker (Kani/CBMC) over the same #[path]-included sources, for every input in a bounded window.

What it discharges is the property the byte-exactness claim rests on: arithmetic totality. Rust integer overflow panics in debug and wraps in release, so an overflowing decoder is not one implementation but two, and which one a sink runs depends on how it was compiled. A decode proved free of overflow, out-of-bounds access and non-termination is a decode whose bitstream denotes exactly one symbol sequence.

matmul's accumulation is total for fixed-point operands of realistic magnitude (±2^31 at Q16). This one is a conditional result and is stated as such: the product is formed in i128 then truncated to i64 and accumulated with a plain +=, which genuinely does overflow for large enough operands. Nothing in the signature constrains magnitude, so callers inherited that safe-operating range silently; the harness turns it into a written, checked precondition rather than reporting an unstated contract as a defect.

Currently proved for the integer transformer kernels (int_transform): softmax is total over the whole fixed-point path (ln2 reduction, i128 polynomial, shift, division) and its output sums to exactly 2^PROB_FRAC with no negative probability — the normalisation the entropy stage depends on, since a denormalised distribution becomes a denormalised CDF. The checker also shows the degenerate uniform fallback to be unreachable; that branch would violate the same invariant if it ever fired, so the property currently holds by unreachability rather than by construction, and the code says so at the site.

Currently proved for the scale→bucket primitive (int_entropy): det_build_index always returns an index addressing a real scale-table entry, and is monotone in σ — a larger σ never selects a smaller bucket. The monotonicity is proved for an arbitrary edge table rather than only an ascending one, so it does not rest on a precondition a caller might not maintain; it is the property the bucketing exists to guarantee, and a non-monotone bucketing would mis-order the table without ever panicking.

Currently proved for the integer rANS coder: RansDecoder::new is total for every input slice; decode_categorical is total and returns an in-alphabet symbol for all admissible CDFs and all streams in the window; and decode's bypass escape is total. Each harness carries a kani::cover obligation, so a proof cannot pass vacuously on assumptions that admit no state.

This is not redundant with the matrix. The matrix decodes well-formed vectors and therefore cannot see defects only a malformed stream reaches; four such defects in the bypass escape — a stream-controlled shift distance, two saturating-arithmetic gaps, an unbounded read — were found this way and fixed, with the conformance vectors confirming the fix changed nothing a conforming encoder can produce.

The wai.video.ssai decision policy is test-backed, not proof-backed: the natural theorem — that decide never proposes a fill exceeding the funds, the avail duration or the slot count — did not close (three configurations, up to 900s each). The obstacle is that the decision sorts on a heap-allocated key, so the model carries memcmp and the allocator machinery rather than the budget arithmetic. Stubbing that away would prove a property of the stub. What is proved there is narrower: a fresh decision bills nothing until its beacons arrive, so a fill cannot be mistaken for delivery.

layernorm closes only at ±2^10 — below realistic Q16 activation magnitude (at ±2^20 it did not close in 25 minutes). Its guarded divide and surrounding arithmetic are therefore established for small operands only, not over the kernel's working range. The obstacle is isqrt over a symbolic u128, which is a limit of bounded model checking rather than a property of the code. attention, linear, igdn and the composed TransformerBlock are not covered at all. The technique does not reach everything, and this section records where it stops rather than listing only what it reached.

Scope, stated plainly: these are bounded proofs — small symbolic CDFs, fixed-length symbolic streams, explicit unwind limits — not universal ones, and they cover the stream-controlled paths rather than the whole crate. A CDF supplied by internal tables is still assumed well-formed. The verification claims what it checks and no more.

---

7.1. Multi-rendition envelope (v1.1 extension)

The v1.0 single-payload envelope is sufficient when one source produces one rendition for one capability. v1.1 adds an OPTIONAL multi-payload form for cases where the source emits the SAME content in multiple renditions and lets the sink pick — typically because the sink's compute budget, bandwidth budget, or runtime preference (browser- native vs WebGPU vs WebNN vs native ML) differs at decode time.

The model_requirement.fallback field in v1.0 is declarative — it names a capability the sink CAN look for. v1.1 makes fallback (and multi-rendition selection generally) executable by carrying every declared capability's payload in one envelope.

Self-contained-ecosystem note

In a controlled-deployment ecosystem (every sink ships from one authority and therefore has a known capability set) the multi-rendition form is NOT a "graceful fallback for missing capability" — every sink has every required capability. Renditions exist for policy selection: high-quality on a desktop sink with WebGPU; low-quality on a phone sink; quality-vs-latency by bandwidth headroom; native vs WebNN by which runtime is initialized. The sink picks by deployer-defined policy, not by which capability happens to be installed.

v1.1 wire format

+--------+----------------+-------------------+----------------+----...---+
| "WAI2" | u32  man_len   | manifest (JSON)   | rendition table| payloads |
+--------+----------------+-------------------+----------------+----...---+

• Magic is WAI2 (distinct from v1.0's WAI1; sinks dispatch by magic).
• Manifest is JSON with a single renditions array of self-contained rendition entries, in deployer-preferred order:

  {
    "wai":    "1.1",
    "media":  "audio",
    "intent": "replicate",
    "renditions": [
      { "capability": "wai.neural.encodec32", "kind": "encodec_tokens" },
      { "capability": "wai.audio.opus",       "kind": "opus"           }
    ],
    "target": { "sr": 48000, "dur": 5.0 }
  }
  

  Each entry has its own capability (the dispatch key) and kind (the codec id matching the bytes in the corresponding payload). The manifest MAY carry security (§6.1), which covers every payload: a sink applies §6.4 before step 2 of the selection below, and an encoder writes only a security object there. An entry MAY carry prior, the pin (§3.1) of the parameter set its payload was coded against. Two entries MAY name one capability with different pins: the same content coded against two parameter sets, such as a set and its successor:

  "renditions": [
    { "capability": "wai.neural.int_mlicpp", "kind": "int_mlicpp",
      "prior": { "id": "int_mlicpp/b", "sha256": "<B>", "determinism": "int-rans/p16+int-mlicpp/i64",
                 "tier": "decode-equivalence", "supersedes": { "sha256": "<A>" } } },
    { "capability": "wai.neural.int_mlicpp", "kind": "int_mlicpp",
      "prior": { "id": "int_mlicpp/a", "sha256": "<A>", "determinism": "int-rans/p16+int-mlicpp/i64",
                 "tier": "decode-equivalence" } },
    { "capability": "wai.image.avif", "kind": "avif" }
  ]
  

  A sink that holds A decodes the second entry; one that holds B, the first; one that holds neither, the third.
• Component identity. An entry MAY carry an id, a string of 1 to 64 of a–z, 0–9, _ and -, unique within the envelope (it contains no ., so it never equals a capability string), and a sha256, the SHA-256 of the entry's payload bytes as 64 lowercase hexadecimal digits. A component's identity is its id, its payload digest sha256 and its parameter-set pin prior (§3.1). These are one construct: no extension defines another per-component digest, and a later mechanism that addresses a component (a staged layer, a byte-range or referenced rendition, a per-rendition track) addresses it by id and verifies it by sha256 and prior. A sink MUST check an entry's sha256 against the entry's payload bytes before it decodes or applies the entry: a selectable entry that does not match is skipped (component_digest_mismatch), as an entry whose pin is refused is, and a companion that does not match is not applied. An entry whose id, sha256 or companion_of_id is not a string (null is the same as absent) makes the manifest malformed, as a mistyped pin member does (§3.1), and the reader rejects the envelope. An entry's capability names a payload format: a capability §5 registers as a record or a derivation (Kinds of registered string) names none, so an entry of one, in any role, does not conform (record-or-derivation-as-component).
• Exploding a component. The exploded envelope of entry i is the WAI1 envelope (§2) whose payload is entry i's rendition bytes. Its manifest is the entry's envelope_manifest when it carries one, written exactly as it is in the WAI2 manifest; otherwise it is {"wai":W,"media":M,"intent":I,"model_requirement":{"capability":C,"fallback":null},"conditioning":{"kind":K[,"prior":P]}[,"security":S][,"refines":R][,"target":T]}, with no other bytes, where W, M, I, S and T are the WAI2 manifest's wai, media, intent, security and target, and C, K and P the entry's capability, kind and prior, each written exactly as it is in the WAI2 manifest; R is the entry's refines (Refinement components, below) with each component id replaced by the content hash of that entry's exploded envelope, written as staged-delivery §3 orders its members (base, prev, layer, span; a run's producer, then count) with no whitespace. A bracketed part is present exactly when its member is present and not null. A component and its exploded envelope are the same object for every rule that names an object by content hash (jwp-receipts §2.1). An entry MAY carry envelope_manifest, a JSON object: the exact manifest of the entry as a WAI1 envelope. Its model_requirement.capability, conditioning.kind and conditioning.prior MUST equal the entry's, its refines MUST be the entry's with each id replaced as above, or both absent, and, where the entry's capability is refinement-only, its model_requirement.fallback MUST be null (component-manifest-mismatch). A WAI1 envelope packed into a WAI2 with its manifest copied verbatim into envelope_manifest explodes back to the same bytes, and so keeps its content hash, when its manifest has no bytes before or after its JSON value (whitespace around a member's value is not part of it, so an explode cannot write it back) and nothing follows its payload; a packer refuses any other envelope, and refuses an output that does not explode back to its sources (one whose manifest, nested in the WAI2 manifest, passes §2's nesting limit). Entry i's exploded envelope is defined by entry i and the entries its refines names, so an entry that cannot be exploded leaves every entry that does not name it intact. A tool that re-serialises a WAI2 manifest may change these bytes, and with them the content hashes of the exploded envelopes.
• Rendition table: u16 n_renditions | n × (u32 offset, u32 length), offsets relative to the start of the payloads block. The table's i-th entry corresponds to renditions[i] in the manifest.
• Payloads block: the rendition byte ranges, packed contiguously.
• n_renditions MUST equal the manifest's renditions.length.
• The bytes from offset 0 to the end of the rendition table are the envelope's head. They give every component's byte range and, with each entry's sha256, its digest, so a sink can fetch and verify one component without the others (http-delivery §7).

Selection algorithm

A v1.1-aware sink MUST:

1. Read the manifest and take its ordered renditions entries.
2. Walk the entries in order, and pick the first that is not a companion (below), whose capability is dispatchable (§4), that the sink either advertises OR whose declared backend the sink prefers under its deployer policy, whose sha256, when present, matches its payload bytes, and whose prior, when the entry carries one and a pin applies to its capability (§3.1), resolves at the sink. An entry whose payload digest does not match, or whose pin is refused, is skipped as if the sink did not advertise its capability, and the sink keeps the reason. A deployer policy MAY reorder or restrict the entries the sink considers; it MUST NOT pick an entry skipped for its digest or its pin.
3. Read the corresponding rendition's bytes from the payload block.
4. Dispatch as in v1.0 §4, with the entry's capability and prior in place of model_requirement.capability and conditioning.prior, and no fallback. A decode refused with set_incomplete (§3.1) is handled as a skipped entry: continue at step 2 with the next entry.
5. If no entry is picked, the envelope is inert at this sink: return an error that names, for each entry, its index and why it was skipped: not_selectable (a companion, or a capability that is not dispatchable, which step 2 never picks), missing_capability, component_digest_mismatch, a pin reason (§3.1), or, where a deployer policy restricts step 2 to the entries §7's replicate rule admits, the replicate_* reason the rule gives for an entry it leaves out; an envelope inert under §6.4 step 2 names that reason for every entry.

A sink MUST NOT decode one entry's payload against the parameter set another entry pins. The envelope's intent applies to the rendition the sink picks, and §7's replicate rule reads that entry's capability and pin; a deployer policy that serves only sound replicate claims MAY restrict step 2 to the entries that rule admits.

A v1.0 sink that encounters a WAI2 envelope MUST refuse it cleanly (unknown magic, §2). No silent fallback to the first payload — that would hide the version mismatch.

Companion components (descriptors bound to a payload)

A WAI2 component carries an OPTIONAL role. Absent ⇒ "rendition" (a selectable alternative of the same content — the default above). "companion" marks a component applied with the chosen rendition, not instead of it, naming the capability it binds to in companion_of. It carries the payloads that are not standalone media — the capabilities §5 registers as companion-only: the HDR dynamic-metadata descriptors wai.meta.hdr10plus / wai.meta.dovi (SMPTE ST 2094), which describe a video payload, and the enhancement layer wai.video.lcevc, which adds residuals to one. Carried as companions, the companion and its base ride in the same envelope, so the binding is intrinsic — not a spoofable sidecar. This revision registers no capability for immersive-audio mix metadata; wai.audio.iamf carries its own audio substreams, so it is standalone media and travels as a rendition.

Selection: a sink picks a rendition among the role != "companion" entries by the algorithm above, then applies every companion bound to the chosen component — by companion_of_id when the companion carries one, otherwise every companion whose companion_of names the chosen component's capability (e.g. the HDR10+ block for the AV1 video the sink picked). pick/primary never return a companion, nor a component whose capability is not dispatchable (§4): a companion-only capability, a candidate, a record or a derivation. A component whose capability this revision does not register may be picked when the sink advertises it. Static HDR characteristics (ST 2086 mastering display, HLG transfer) stay in-band in the video bitstream / a manifest color field and need no companion.

A component with role = "companion" conforms only if:

1. its capability is companion-only, or is not registered in this revision. A capability registered in any other kind (§5, Kinds of registered string) is not a companion: a payload capability is standalone media — its payload decodes on its own — and MUST be carried as a rendition, and a record or a derivation is no component in any role (Component identity, above);
2. companion_of is present and names a capability the companion applies to. An HDR dynamic-metadata descriptor, registered as a companion to a video payload, applies to a registered capability whose payload is video — the video media class (§3), as the capability's registration places it: every wai.video.* capability that is not companion-only, the neural video codecs wai.neural.video_bmshj2018, wai.neural.dcvc_rt and wai.neural.dcvc_fm (§5, Video:), and the films wai.film.linear and wai.film.reel (linear-film gives a linear film's media the value "video", and a reel is a film with cuts). A capability registered as video in a later revision joins that class. wai.video.lcevc applies to one of the base codecs §5 lists for it. No companion applies to a companion-only capability. This revision constrains no base for a companion whose capability it does not register, nor an HDR descriptor's base whose capability it does not register;
3. a selectable rendition of that capability is present in the envelope — for an enhancement layer, exactly one, the base encode its residuals were produced against, and that rendition is the base of no other enhancement layer (§5). Several descriptor companions MAY bind to one base, and a descriptor MAY bind to several renditions of one capability.

A companion MAY carry companion_of_id, the id of the one entry it binds to. That entry MUST exist, MUST have the capability the companion's companion_of names, and MUST be a selectable rendition; the companion then binds to exactly that entry, and rule 3's "at most one enhancement layer per base" is checked by entry. companion_of keeps its meaning, so a sink that does not read companion_of_id binds by capability: for an enhancement layer over two renditions of one base capability it finds the base not unique, and does not apply the layer.

An envelope with a companion that breaks any of these does not conform (the reference WaiMulti::check_component_roles reports the first break). A sink MUST NOT apply a companion that breaks rule 1 or 2, nor one whose capability it does not recognise; a companion that breaks rule 3 binds to nothing, or, for an enhancement layer, is handled as §5 states. The sink MAY still present the rendition it picked.

Refinement components

A component with role: "refinement" is a layer of a stage set (staged-delivery). It carries refines (§3), whose base and producers are the ids of entries in the same envelope (§7.1, Component identity); its exploded envelope carries the same refines with those ids replaced by the content hashes of the named entries' exploded envelopes (Exploding a component, above), so it is the same object as the layer carried on its own.

• Placement. A refinement component MUST follow every selectable entry in renditions.
• Selection. Selection never picks one (step 2: it is not selectable). A sink picks a rendition as above, then offers to staged-delivery §4, in renditions order, the exploded envelopes of the refinements whose refines.base is the picked entry's id, up to the stage its deployer policy chooses. It offers them only from an envelope every component of which conforms — each refinement below, each companion (Companion components), each id (Component identity) and each capability in the role §5 gives it (the reference WaiMulti::check_component_roles, which reads the manifest alone): an envelope one component of which breaks a rule offers none of its refinements. It MAY still present the rendition it picked. A refinement whose payload does not match its sha256 is not offered. A refinement's exploded envelope a sink receives on its own, as a WAI1 object (staged-delivery §9), is a separate delivery, which staged-delivery §4 processes as any layer, whatever the envelope it was exploded from.
• Conformance. A refinement component conforms only if, in this order: its capability is refinement-only (§5); it carries id and sha256; its refines is well formed (staged-delivery §3, with ids in place of hashes); refines.base names a selectable entry, which carries id and sha256; its capability admits that entry's (§5, Base admission); every producer in prev names a selectable entry or a refinement before it, each carrying id and sha256; and no selectable entry follows it. No other component carries refines, and a refinement-only capability is carried with no other role. The reference WaiMulti::check_component_roles reports the first break.

A sink of an earlier revision does not advertise a refinement-only capability, so it never picks a refinement by capability; and since refinements follow every selectable entry, a sink that picks the first entry it can decode picks a base. If it picks one by another policy, it finds the capability unsupported and the component inert.

Derived renditions

A WAI1 manifest (§3), or a rendition entry, MAY carry derived_from. It states that its payload presents another object, the motif, and was made from it by a named renderer or encoder:

"derived_from": { "motif": "<64 hex>", "motif_capability": "<capability>",
                  "renderer": "<id>@<version>", "settings": "<64 hex>" }

• motif is the motif's content hash (jwp-receipts §2.1), 64 lowercase hexadecimal digits.
• motif_capability is the motif's capability, and renderer the renderer or encoder that made the payload; each a non-empty string.
• settings is BLAKE3("wai:render-settings\x01" ‖ JCS(settings)), over the settings the renderer was run with as RFC 8785 canonical JSON, 64 lowercase hexadecimal digits.

A sink MUST NOT present such a payload's output as meeting the motif's equivalence criterion, its tier or a replicate intent: the payload presents the motif; it does not reconstruct it. A verifier checks motif against a motif it holds and reports every other member as the producer's statement, except where a profile fixes them: packaging §7 fixes a supplied floor's renderer and settings, and a package's verifier holds them to it. A value of another form makes the member malformed; the envelope still parses (§8). A reader of an earlier revision ignores the member and loses only that statement. packaging §7 writes it on every floor object.

v1.0 compatibility

v1.0 envelopes (WAI1) remain valid and their format is unchanged, except that a WAI1 naming a companion-only capability as its capability or fallback does not conform (§5; below). Encoders SHOULD emit WAI1 when only one rendition is present (simpler form, wider sink support). WAI2 is for sources that genuinely benefit from shipping multiple renditions per file.

Envelopes the companion rules make non-conforming

The companion rules of §5 and this section change no capability's payload format and no capability's role (§8). Each enforces something fixed before the rules were stated:

• a registration. wai.meta.hdr10plus and wai.meta.dovi were registered as companion descriptors to a video payload, not standalone media. wai.video.lcevc was registered companion-only, with its base codecs, in the same revision as these rules. Every other registered capability is standalone media.
• the definition of a companion. When this section introduced role = "companion", it defined a companion as a component applied with the chosen rendition and naming, in companion_of, the component it binds to.

Before these rules were stated, the text rejected none of the envelopes below, and the reference implementation had no check that reported them (its pick and primary could select an HDR descriptor). These do not conform:

• a WAI1 that names an HDR descriptor as its capability or fallback, carrying it as standalone media (§5);
• a WAI2 that carries an HDR descriptor as a selectable rendition instead of with role = "companion". pick and primary no longer select it (Selection, above);
• a WAI2 that binds an HDR descriptor to a registered capability outside the video media class (rule 2) — an audio, image or text payload, or any other registered capability rule 2 does not list, a volumetric or world payload among them;
• a WAI2 that marks a registered standalone capability as a companion (rule 1). This includes wai.audio.iamf carried as immersive-audio mix metadata: the earlier text of this section named immersive-audio metadata beside the HDR descriptors as a payload that accompanies another, and a reference test carried wai.audio.iamf that way. That use did not follow the registration — wai.audio.iamf was registered as an immersive-audio carrier, standalone media, and no capability for mix metadata alone is registered — and rule 1 follows the registration;
• a WAI2 with a companion, of any capability — one this revision does not register included — that has no companion_of, or whose companion_of names no selectable rendition in the envelope (rules 2 and 3).

The first four enforce the registrations; the last enforces the definition of a companion. Each still parses, and §5 and the rules above state what a sink does with it. No earlier revision registered wai.video.lcevc; this one registers it with these rules. An envelope that used that string while no revision registered it is covered by §8's rule for a new registration, as is an envelope that uses any string a later revision registers.

---

8. Versioning

• The wai manifest field is MAJOR.MINOR. A reader MUST reject an unknown MAJOR. A reader MUST accept an unknown MINOR by ignoring fields it does not understand, provided MAJOR matches.
• A reader MUST ignore a manifest member it does not understand, whatever the MINOR. Earlier text required this only for an unknown MINOR, so a reader of an earlier revision may instead reject an envelope that carries such a member, which fails closed. Or it ignores the member, and then a pin fails open (§3.1).
• The container format (§2) is frozen for all of 1.x. The manifest schema (§3, and a §7.1 rendition entry) is frozen for all of 1.x in this sense: no member is removed or renamed, none changes type or meaning, and none becomes REQUIRED. A MINOR revision MAY define an OPTIONAL member, at the top level or inside an existing object, whose absence leaves every envelope's bytes and meaning as they were, and the revision states what it means for a reader of an earlier revision (above). This revision defines conditioning.prior, a rendition entry's prior, id, sha256 and companion_of_id, and the pin's members form, tier and supersedes this way (§3.1, §7.1); envelopes that use them keep wai "1.0" or "1.1". The reference implementation wrote prior, with id, sha256, determinism and form, before this revision defined it; an envelope written that way keeps its bytes and its meaning, except as §3.1 lists.
• The one exception to the freeze. This revision makes one exception to the sentence above, deliberately. A member it defines, carried with another JSON type, makes the manifest malformed rather than being ignored, because a reader that ignored a mistyped pin member could decode against a set the pin excludes (a mistyped form, read as absent, turns a set-form pin into a form-less one). The rule covers every member listed above, the rendition entry's id, sha256 and companion_of_id included, so one rule decides whether a manifest parses. An envelope that carried such a value before the member was defined, and parsed then, does not parse now; §3.1 lists it. Nothing in the envelope signals the change: its wai value is the same, so a reader of this revision rejects the whole envelope where an earlier reader ignored the member.
• This revision also defines the manifest's refines (§3) and a rendition entry's refines and envelope_manifest (§7.1) as OPTIONAL members under the MINOR rule above; envelopes that use them keep their wai value. The exception does not cover them: an envelope whose refines is malformed still parses, and staged-delivery §4 reports it (refines-malformed); one whose envelope_manifest does not match its entry still parses, and §7.1 reports it (component-manifest-mismatch). derived_from, on a manifest and on a rendition entry (§7.1, Derived renditions), is defined the same way: an envelope whose derived_from is malformed still parses.
• New capabilities (§5) MAY be added in any MINOR revision. Existing capabilities MUST NOT be redefined — once a capability string is registered for a codec, its payload format is fixed forever; for a registration completed as the next item says, from the revision that completes it.
• Completing a registration. A WAI-native string whose §5 row named the decoder that reads its payload and fixed no byte layout (naming, at most, a tool that wrote it) has a registration with no payload format to hold. A MINOR revision MAY complete such a registration by pinning a byte layout, which is not a redefinition. Envelopes that carried another form under the string, a tool's output among them, become non-conforming; they still parse, and the completing revision lists them in §5. From that revision on, the string's payload format is fixed as the item above states. §5, Payloads of the WAI-native integer codecs, completes the registrations of wai.video.int_motion, wai.video.int_hyper, wai.neural.int_hyper, wai.audio.int_codec and wai.splat.int_codec this way, and §5, Payload of wai.generate.spectral, completes that string's.
• Editions. A registration that names an edition (or version) of its standard carries that edition. A registration that names none carries the edition of its named standard current on the date it was registered, and no later edition or amendment. A later edition or amendment that changes the bitstream a decoder must accept is registered, if at all, under a capability string of its own; it never widens an existing string. The machine-readable registry records both (edition, registered; registry/capabilities.json). Where which edition was current on the registration date has not been established, the registration says so (§5), and what it admits beyond the tools of the standard's first edition is unspecified until a revision records the edition. Citing a later edition of the same specification as the reference for a format, where the bitstream a registration admits does not change, is not a redefinition.
• A MINOR revision MAY add a value of an informational manifest field (the media classes of §3); a reader accepts a value it does not know.
• A registration also fixes the capability's role (§5, §7.1): standalone media, or companion-only — applied with a base rendition, never on its own. Changing a capability's role, or narrowing the renditions a companion-only capability applies to, redefines it. Widening them does not: no conforming envelope stops conforming, and a reader of an earlier revision that reports the new binding as not conforming does not apply that companion and can still present its base.
• This revision adds a third role, refinement-only: a payload applied, in a declared order, to the output of the stage below it, bound to that output's producer by content hash (staged-delivery), and never decoded on its own. It differs from companion-only, which is unordered, names its base by capability and travels only inside a WAI2: a refinement is ordered, names its base by content and travels as its own object. Changing a capability into or out of either role redefines it; admitting another base (§5, Base admission) widens it. The manifest member refines (§3) is an OPTIONAL member defined under the rule above for MINOR revisions: an envelope that does not carry it keeps its bytes and its meaning. A reader of an earlier revision does not advertise a refinement-only capability, so it finds such an envelope's capability unsupported and the envelope inert.
• A registration of a capability that decodes against a parameter set also fixes that set's shape (§5, §3.1). Changing one file to several narrows it and requires a new MAJOR; several to one widens it, which a MINOR revision MAY do.
• Removing or repurposing an existing capability — changing its payload format or its role — requires a new MAJOR.
• A MINOR revision MAY add a conformance rule that enforces a registration, or the definition of a manifest field, as it was made, changing no payload format and no role. Such a rule makes non-conforming only envelopes that used a capability against its registration or a field against its definition, and the revision lists every kind of envelope it makes non-conforming; they still parse. The companion rules of §5 and §7.1 are such rules, and §7.1 lists the envelopes they affect (Envelopes the companion rules make non-conforming). §3.1's pin rules are such rules, and §3.1 lists the envelopes they affect.
• Registering a capability gives a string a payload format and a role that no earlier revision gave it, and the rules of §5 and §7.1 then apply to it. An envelope that used the string before then used a capability no revision defined, and no rule checked that use against a registration it did not yet have: §7.1 constrains an HDR descriptor's base only when the base's capability is registered, and holds a companion of an unregistered capability only to naming a base that is present (rules 2 and 3). Leaving a use unchecked does not make it conform to a registration made later, so the registering revision MAY make such an envelope non-conforming where it uses the string against the new registration. This narrows nothing a registration fixed and is not a redefinition. The kinds of envelope are listed here, once, for every registration, so a registering revision need not list them again: registering a standalone capability makes non-conforming a WAI2 that carried it as a companion (rule 1) and, when it is outside the video media class, one that bound an HDR descriptor to it (rule 2); registering a companion-only capability makes non-conforming an envelope that carried it as a WAI1 capability or fallback or as a selectable rendition, that bound it to a base or a number of bases its registration does not admit, or that bound another companion to it (no companion applies to a companion-only capability); registering a refinement-only capability makes non-conforming an envelope that carried it as a WAI1 capability without refines, as a fallback, or as a selectable rendition or a companion, and one that bound a companion to it. Such envelopes still parse.
• Planned breaks. A capability that comes to decode through the integer operator set (integer-operators §13) refuses some inputs it decoded before, or decoded differently by build. Each is a new refusal of input no committed corpus contains, and no committed output changes; integer-operators §15.3 states them, and the changelog records each when it lands. Landed in this revision:
  • wai.neural.int_synth: a payload whose reserved byte is not 0, or whose C is 0, or whose latents exceed 2^26 values; a set with no layers, with a layer whose out_ch, k or stride is 0, with a field of 2^32 or more, or with a shift of 128 or more; a set or payload that a cap refuses (a tensor, or a transposed convolution's uncropped output, past 2^26 elements); and a set whose certificate does not meet acc64. A shift from 64 to 127, which a build used to mask or panic on, now has its exact meaning.
  • wai.splat.int_codec: a set refused at load by its lowering, before any stream is decoded (model_mismatch): a layer with a bias of magnitude 2^63 − 1 or more, or a check range its producer cannot meet, each of which refused every decode (field_range) when the network ran; a network with no layer of tag 0, 1 or 2; and a header shift or gf of 128 or more that no layer uses.
  • wai.audio.int_codec: the same, for its synthesis, whose check ranges include its PCM step's: a set whose last layer cannot give a sample inside that range, which refused every decode at that step, is refused at load.
  • wai.neural.int_hyper and the keyframe of wai.video.int_hyper: the same, for either of its networks, h_s and g_s: such a set is refused after the payload's shapes are checked and before either stream is decoded.
  • wai.neural.int_mlicpp: the same, for any of its three networks, h_s, entropy and g_s: such a bundle is refused after the payload's header and the networks' geometry are checked and before either stream is decoded. Four checks its decoder made when a network ran now come before the streams, with the same code (model_mismatch): a layer that does not fit the channels before it, a network that does not read the payload's channel count, a header shift or gf of 128 or more that a layer uses, and a leaky ReLU's shift of 64 or more; where a stream or a value was refused first, the bundle now is. A bundle with a convolution whose padded side exceeds 2^26 is refused.
  • wai.world.model's integer renderer (informative): what the direct renderer, which checked nothing, rendered from or panicked on is refused before the first frame: a set whose afrac is outside 1..=30 or differs between its two networks, with no action plane, or that breaks a WHM1 network's rules above (model_mismatch); a row that does not carry one value per action plane; T that does not give the latent's shape back, or g_s that does not plan on it (model_mismatch); a latent that is not C·lat·lat values (geometry_mismatch), of side 0, or past 2^26 values; a move whose plane value leaves i64. A value outside a check range is refused (field_range) where the renderer computed past its exact range.

---

Appendix A — Reference implementation

wai-rs/ (Rust). Builds as lib, cdylib, staticlib so the C ABI in wai-rs/src/ffi.rs drops in as libwai.dylib / .so / .dll and can be called from any language. The crate wraps:

• image: image crate (PNG, JPEG, AVIF via ravif), jpegxl-rs (libjxl 0.11.x).
• audio: opus (libopus 1.x), claxon + flac-bound (libFLAC).
• video: rav1e (AV1 encode) + dav1d (AV1 decode).
• text: zstd (libzstd), xz2 (liblzma).

Run cargo test --lib. With the codecs feature, wai-rs/build.rs finds libFLAC through pkg-config (on macOS without pkg-config, in the Homebrew prefix); where it does not, set RUSTFLAGS="-L <the directory holding libFLAC>".

Decode speed is an implementation property, not a conformance property: the integer decoders run faster kernels (SIMD, threads) built to give the same integer for every output element as the retained scalar reference kernels. That equivalence is argued by construction and tested, not machine-proved (the harnesses of §7 do not reach these kernels) — see research/decode-throughput.md for the argument, its tests, its limits and measured throughput.

Appendix B — Why a capability menu

For a format a standard already specifies, WAI names the standard and the sink decodes it with the library it already has: the zeroth condition's purpose is availability, so a file opens wherever that library is present. Where WAI needs a property no registered standard specifies — a reconstruction that is byte-identical on every machine, a world or a film that replays to the same state — it defines a WAI-native capability class with its own payload format and its own conformance criterion (§5, §7). The envelope, the dispatch rules, the registry's classes and the shared-prior model are what WAI itself specifies.

Appendix C — License

Apache-2.0. The standard and the reference implementation are open; royalty-free implementation is a precondition for adoption, so every default capability (§5) is royalty-free. Companion implementations outside this tree keep their own licences.