open-standards/wai/extensions/integer-payloads.md

WAI Extension: Integer Payloads

Status: Draft. Pins the payload formats of the WAI-native integer codecs that SPEC.md §5 registers without one (Payloads of the WAI-native integer codecs): WIV1 for wai.video.int_motion and wai.video.int_hyper, WIH1 for wai.neural.int_hyper, WIA1 for wai.audio.int_codec and WIS1 for wai.splat.int_codec. Defines each layout, its decode, the checks a sink makes and the closed list of refusal codes; names each capability's parameter-set members; and makes the JSON the encode and export tools write a converter input, never a payload. Writes down the payload formats and parameter sets that SPEC §5 keeps for wai.neural.int_synth (§13) and wai.neural.int_mlicpp (WIM1 and WMB1, §14). Pins WIF1 (§16), one frame of WIV1, the payload of the frame forms wai.video.int_motion.frame and wai.video.int_hyper.frame, which frame-stream carries one frame an object. Pins WIC1 and WCF1 (§17), the clip and the frame of wai.video.int_cond and wai.video.int_cond.frame, conditional inter-frame coding at the integer tier, and their one-file parameter set WCB1. Reference impl: wai-rs — int_payload::{wire, decode_int, resolve}, int_payload::{wiv1_from_json, wih1_from_json, wia1_from_json, wis1_from_json}, codecs::PINNED_INT_PAYLOADS, container::check_int_payload_framing. Keywords MUST, MUST NOT, SHOULD, MAY are RFC 2119/8174.

1. Scope

capability	payload	parameter set (§9)
wai.video.int_motion	WIV1 with keyframe 0 (§3)	none
wai.video.int_hyper	WIV1 with keyframe 1, a nested WIH1 (§3)	several files: entropy.json, g_s.whm, h_s.whm
wai.neural.int_hyper	WIH1 (§4)	several files: entropy.json, g_s.whm, h_s.whm
wai.audio.int_codec	WIA1 (§5)	one file: model.wam
wai.splat.int_codec	WIS1 (§6)	one file: g_s.whm
wai.video.int_motion.frame	WIF1 with keyframe 0 (§16)	none
wai.video.int_hyper.frame	WIF1 with keyframe 1, an intra frame a nested WIH1 (§16)	several files: entropy.json, g_s.whm, h_s.whm
wai.video.int_cond	WIC1, a clip of WCF1 frames (§17)	one file: model.wcb
wai.video.int_cond.frame	WCF1 (§17)	one file: model.wcb

The last four are registered with their format (SPEC §5): they are not completions of a registration, and §11 lists no envelope for them.

Why this pin is permitted. Until this revision, these capabilities' SPEC §5 rows named the decoder that read their payload, and no byte layout. The rows of wai.video.int_motion, wai.audio.int_codec and wai.splat.int_codec also named the encode or export tool that wrote a bitstream, a JSON document; those of wai.neural.int_hyper and wai.video.int_hyper named their decoders only. SPEC §5 called every row's payload the bytes its codec normally emits, which for these rows was the tools' JSON. So this revision changes what these strings carry. SPEC §8 permits it under Completing a registration: a revision may complete the registration of a WAI-native string whose row named a decoder and fixed no byte layout by pinning one, and it lists the envelopes that become non-conforming (§11; SPEC §5). SPEC §8's rule that a registered string's payload format is fixed forever applies to these strings from this revision on. The encode and export tools still write JSON that carries the same fields; it is a converter input (§10).

wai.neural.int_synth and wai.neural.int_mlicpp keep the payload formats SPEC §5 gives them, and §13 and §14 write those formats down. A sink decodes them and the five strings above (wai.video.int_motion, wai.video.int_hyper, wai.neural.int_hyper, wai.audio.int_codec, wai.splat.int_codec) through one path (int_payload::decode_int), and §8 names the codes it reports for those two.

2. Common rules

Encoding. Every multi-byte field is little-endian. Offsets are in bytes.

Framing. A payload starts with its four-byte magic and a version byte. This revision defines version 1 for every format. Every reserved field is zero. Nothing follows the last section.

rANS streams. A stream is a u32 length L followed by L bytes. L is a multiple of 4 and at least 8. It is decoded as follows (precision 16, with a bypass escape of 4-bit groups):

• The state x starts as (w1 << 32) | w0, from the stream's first two u32 words. Renormalising: if x < 2^31, then x = (x << 32) | w, where w is the next unread word; if no word is left, the stream has run out and x is left as it is.
• bits(n): v = x mod 2^n, x = x >> n, renormalise, return v.
• Decoding a symbol against a table cdf of length n with offset o: let cum = x mod 2^16 and s be the largest index with cdf[s] ≤ cum (0 ≤ s ≤ n − 2); then x = (cdf[s+1] − cdf[s]) · (x >> 16) + cum − cdf[s], and renormalise. If s < n − 2, the value is s. If s = n − 2 (the escape): let v = bits(4) and count = v; then, at most 8 times, while v = 15 and the stream has not run out, v = bits(4) and count = count + v; then count = min(count, 8) and raw = Σ_{j < count} bits(4) << 4j. The value is −(raw >> 1) − 1 if raw is odd and (raw >> 1) + (n − 2) if it is even, saturated to i32. The symbol is value + o, saturated to i32.
• A decode stops at the first symbol during whose decode its stream runs out, and refuses the payload (stream_exhausted): the value that symbol would give is not used, and nothing after it is decoded. A stream that never runs out decodes every symbol, so a conforming payload is unaffected.
• After the last symbol a decode reads from a stream, the stream ends canonically: every word of it has been read, and the state is 2^31. Otherwise the payload is refused (stream_noncanonical). An encoder starts its state at 2^31 and writes the words it flushes and no others, so its streams end this way. A word after the last symbol's, or a state the encoder did not start from, codes the same symbols in other bytes; the rule closes both, so neither can give one decode several content hashes. A stream that codes no symbol (the residual stream of a one-frame WIV1 with keyframe 1) is the state 2^31 alone: the words 2^31 and 0.

The decode these rules define is integer-operators E1, rans64_decode, with the modes t4097, lenient and canonical.

One clip, several payloads. These rules fix how a payload is read, not the only payload of a decoded output: tables that differ in entries the stream never selects (a symbol the clip does not use, say), or a different escape layout, code the same output in other bytes. A payload's hash therefore identifies the payload, not what it decodes to. Where one output must have one identity, it is identified by the digest of its decoded output: for a clip, its frame-state-equivalence digests (SPEC §7, a video receipt's clip_hash); for the other formats, their capabilities' equivalence digests.

CDF tables. A table is n entries of u32 with 3 ≤ n ≤ 4097, cdf[0] = 0, cdf[n − 1] = 65536 and every entry greater than the one before. These rules are what keep the symbol search and the state update above in range.

Order of the checks. A parser checks, in this order, and refuses with the first that fails:

1. the magic: the capability's (§1), or else format_capability_mismatch if it is another of the seven magics (WIV1, WIH1, WIA1, WIS1, WIF1, WCF1, WIC1), or bad_magic;
2. the version byte: present (stream_length), and 1 (version);
3. the fixed header: present (stream_length);
4. the reserved fields (reserved_nonzero);
5. the header's fields, in the order the format's section lists their rules (field_range), then, for WIV1 and WIF1, the keyframe kind (keyframe_capability_mismatch);
6. the caps of §7 on the sizes the header declares (too_large), in the order the format's section lists them;
7. each later section in order: present (stream_length), then its rules (field_range, cdf_malformed). A section of one table per channel (WIA1, WIS1) is read table by table: each table present, then checked, before the next;
8. nothing after the last section (trailing, or nested_length for the nested keyframe of a WIV1 or a WIF1 intra frame).

A decode then makes the checks of §7, and refuses a stream that runs out or does not end canonically (§2). Two conforming sinks refuse the same bytes with the same code.

3. WIV1 — integer inter-frame video

off	size	field	rule
0	4	magic WIV1
4	1	version	1
5	1	mv_precision	1 (whole-pel) or 2 (half-pel); 1 when n_frames is 1
6	1	keyframe	0 or 1
7	1	reserved	0
8	4	n_frames	≥ 1
12	4	h	≥ 1
16	4	w	≥ 1
20	4	block	≥ 1; 1 when n_frames is 1
24	4	q	1..=65535
28	4	offset	i32
32	2	cdf_len	3..=4097
34	2	reserved	0
36	cdf_len·4	the residual table	§2
…	(n_frames − 1)·nb·4	motion vectors	i16 dy, i16 dx
…	4 + L	the residual stream	§2
…	4 + K	keyframe 1 only: u32 K, then a WIH1 payload	K is exactly the bytes that remain

The header's field rules are checked in the order mv_precision, keyframe, n_frames, h and w, block, q, cdf_len, then, for a one-frame clip, mv_precision 1 and block 1: such a clip codes no motion vector, so neither field would change its decode, and they are fixed so that it has one header. keyframe 0 is wai.video.int_motion's and 1 is wai.video.int_hyper's; the other is keyframe_capability_mismatch. The frame h·w·3, the clip n_frames·h·w·3, the frame count n_frames and the motion-vector field 2·nb·(n_frames − 1) are capped (§7), in that order.

Motion vectors. nb = ⌈h/block⌉·⌈w/block⌉ and nbx = ⌈w/block⌉. The vectors are frames 1 to n_frames − 1 in order, each its block grid row-major: the vector of pixel (y, x) in frame f is entry (f − 1)·nb + (y div block)·nbx + (x div block), in units of 1/mv_precision pel.

Decode. One decoder reads the residual stream for the whole clip, against the residual table at offset. For each frame in order, for each pixel in raster order (y, x), for each channel c of R, G, B: decode the symbol r, and the sample is clamp(pred + r·q, 0, 255). Since |r| < 2^31 and q < 2^16, every step fits i64.

• Frame 0 with keyframe 0: pred = 128.
• Frame 0 with keyframe 1: the frame is the nested WIH1's decode (§4), which reads no symbol of the residual stream. It must be h × w (geometry_mismatch).
• Frame f ≥ 1: pred reads the previous decoded frame R with the pixel's vector (dy, dx), where p(Y, X) is R at row clamp(Y, 0, h − 1), column clamp(X, 0, w − 1), channel c:
  • whole-pel: pred = p(y + dy, x + dx);
  • half-pel: iy = y + (dy >> 1), fy = dy & 1, ix = x + (dx >> 1), fx = dx & 1 (arithmetic shift); then p(iy, ix) when both are 0, (p(iy, ix) + p(iy, ix + 1) + 1) >> 1 when only fx is 1, (p(iy, ix) + p(iy + 1, ix) + 1) >> 1 when only fy is 1, and (p(iy, ix) + p(iy, ix + 1) + p(iy + 1, ix) + p(iy + 1, ix + 1) + 2) >> 2 when both are.

The output is n_frames RGB8 frames of h × w (rows top to bottom, pixels left to right, R, G, B bytes): the buffers of the capability's frame-state-equivalence digests (SPEC §7 Digests).

This decode is the stage operators of integer-operators §5E, with no graph (§13.5 there): the residual stream is E1 rans64_decode with t4097, lenient and canonical, the prediction of frames f ≥ 1 is E10 mc_predict, and the sample is E9 residual_step on [0, 255]. The reference decoder decodes through them.

4. WIH1 — the integer conditional image codec

off	size	field	rule
0	4	magic WIH1
4	1	version	1
5	3	reserved	0
8	4	m	≥ 1: latent channels
12	4	sh	≥ 1: latent height
16	4	sw	≥ 1: latent width
20	4	n_z	≥ 1: hyper-latent channels
24	4	zh	≥ 1: hyper-latent height
28	4	zw	≥ 1: hyper-latent width
32	4 + Lz	the hyper-latent stream	§2
…	4 + Ly	the latent stream	§2

The caps (§7) apply to n_z·zh·zw and to 2·m·sh·sw.

Decode, against the set of §9 (g_s.whm and h_s.whm, two integer networks at one fixed-point scale 2^afrac; entropy.json, the tables):

1. For each hyper-latent channel k and each of its zh·zw elements, decode a symbol s from the hyper-latent stream against channel k's table and offset; the element is s·2^afrac + median_k.
2. h_s on the hyper-latent [n_z, zh, zw] gives [2·m, sh, sw]: its first m channels are the scales σ, the rest the means μ. h_s decodes through its lowering whm1/1 (integer-operators §13.2); the split of its output into σ and μ is a step of this decode, after the shape check, not part of the lowered graph (integer-operators §13.5).
3. For each latent element p in order (channel, then row, then column), its table is b, where, with the scale edges e_0 … e_{N−1} and σ' = max(σ_p, e_0), b = N − 1 − #{ i < N − 1 : σ' ≤ e_i }. Decode a symbol r from the latent stream against table b and its offset; the element is r·2^afrac + μ_p.
4. g_s on the latent [m, sh, sw] gives three channels; each value v is clamped to [0, 2^afrac] and becomes (v·255 + 2^(afrac−1)) >> afrac, as RGB8. g_s decodes through its lowering whm1/1 (integer-operators §13.2), and this is the RGB8 step that ends its lowered graph (§13.0).

Every value is formed in i64 and must fit it (§7).

5. WIA1 — the integer learned audio codec

off	size	field	rule
0	4	magic WIA1
4	1	version	1
5	1	afrac	1..=30; equal to the model's (§7)
6	2	reserved	0
8	4	sr	≥ 1: the sample rate the PCM is presented at, in hertz
12	4	n_ch	≥ 1: latent channels
16	4	lat_len	≥ 1: latent length
20	4	t_out	≥ 1: samples presented
24	4	offset	i32: the symbol offset of every channel's table
28	2	cdf_len	3..=4097
30	2	reserved	0
32	n_ch·8	medians	i64 each, `
…	n_ch·cdf_len·4	one table per channel	§2
…	4 + L	the latent stream	§2

The header's field rules are checked in the order afrac, then sr, n_ch, lat_len and t_out, then cdf_len. The caps (§7) apply to n_ch·lat_len and to t_out.

Decode, against the set of §9 (model.wam, the integer synthesis):

1. For each channel k and each of its lat_len elements, decode a symbol s against channel k's table at offset; the element is s·2^afrac + median_k.
2. The synthesis (one-dimensional integer transposed convolutions and integer inverse GDN) on [n_ch, lat_len] gives one channel of t ≥ t_out samples. It decodes through its lowering wam1/1 (integer-operators §13.3), with t_out as its scalar.
3. The first t_out samples v become (v·32767 + 2^(afrac−1)) >> afrac, clamped to [−32768, 32767]: the PCM step that ends the lowered graph (integer-operators §13.3). A sample for which v·32767 + 2^(afrac−1) leaves i64 is refused (field_range).

The output is t_out samples of i16 PCM, one channel, at sr hertz: the buffer of the capability's sample-equivalence digest (signed 16-bit little-endian samples).

6. WIS1 — the integer learned Gaussian-splat attribute codec

off	size	field	rule
0	4	magic WIS1
4	1	version	1
5	1	afrac	1..=30; equal to the model's (§7)
6	2	reserved	0
8	4	c	≥ 1: latent channels
12	4	sh	≥ 1: latent height
16	4	sw	≥ 1: latent width
20	4	s	≥ 1: the attribute planes' side
24	4	attr	≥ 1: attribute planes
28	4	n_gaussians	1..=s·s
32	attr·8	ranges	per plane, f32 lo then f32 hi (IEEE 754 binary32): finite, lo ≤ hi
…	c·2	table lengths	u16 each, 3..=4097
…	c·4	symbol offsets	i32 each
…	c·8	medians	i64 each, `
…	the lengths' sum ·4	one table per channel	§2
…	4 + L	the latent stream	§2

The header's field rules are checked in the order afrac, the dimensions, n_gaussians. The caps (§7) apply to c·sh·sw and to attr·s·s.

Decode, against the set of §9 (g_s.whm, the integer synthesis):

1. For each channel k and each of its sh·sw elements, decode a symbol s against channel k's table and offset; the element is s·2^afrac + median_k.
2. g_s on [c, sh, sw] gives [attr, s, s] (geometry_mismatch otherwise). g_s decodes through its lowering whm1/1 (integer-operators §13.2).
3. Each value v is clamped to [0, 2^afrac] and becomes (v·255 + 2^(afrac−1)) >> afrac, a byte: the RGB8 step that ends the lowered graph (integer-operators §13.0).

The output, plane after plane, is the buffer of the capability's splat-equivalence digest. The first n_gaussians cells of the planes, in raster order, hold one Gaussian each. The ranges are presentation data: a sink presents a plane's byte u at about lo + (hi − lo)·u/255. They are outside the digest, and this extension does not fix presentation.

7. Caps and the checks a decode makes

Caps, the same on every target. No tensor a payload sizes may exceed 2^26 elements: a frame h·w·3, a WIV1 motion-vector field 2·nb·(n_frames − 1), a latent, a hyper-latent, a network's output, an attribute tensor, a clip's t_out. A WIV1 clip may not exceed 2^30 bytes, nor 2^20 frames.

Planning the networks. Before a decode allocates a latent, it plans every network on the payload's shapes from the layers' geometry and parameter lengths alone. The network's input must be within the cap (too_large); then, layer by layer in order:

1. the layer fits the running channel count (model_mismatch): a convolution's output channels, kernel and stride are non-zero, it has c·out·k·k weights (c·out·k for the audio synthesis) and out multipliers and biases, and its requantisation shift is under 128; an inverse GDN has c·c and c parameters and shifts under 128; a leaky ReLU's shift is under 64;
2. every size the layer forms is within 2^26 (too_large): a convolution's padded input side n + 2·pad, a transposed convolution's uncropped side (n − 1)·stride + k + opad and uncropped plane, and the output;
3. the layer leaves an output (model_mismatch): a kernel no larger than the padded input, a crop 2·pad smaller than the uncropped side.

Every size is formed in 64-bit arithmetic with overflow checked, and a size past 64 bits is past the cap, so a 32-bit sink refuses exactly what a 64-bit one does, with the same code; once a network is planned, every size it forms fits a 32-bit index.

Memory and work. A payload's header declares its decode's output and the work that output takes, and a sink can read both before it decodes anything. A stream does not bound them: a table can make one symbol cost a small fraction of a bit, so a few kilobytes of stream can code a clip at the 2^30-byte cap, as any codec can describe a large output in a small file. The bound is the header's, within the caps:

• the output: a WIV1 clip's n_frames·h·w·3 bytes (one per sample), a WIH1 image's 3·h·w bytes at its synthesis network's size, a WIA1 clip's 2·t_out bytes (i16 samples), a WIS1 attribute tensor's attr·s·s bytes;
• the entropy decode: one step per symbol, n_frames·h·w·3 for a keyframe-0 WIV1 (h·w·3 fewer, plus its keyframe's, for keyframe 1), n_z·zh·zw + m·sh·sw for a WIH1, n_ch·lat_len for a WIA1, c·sh·sw for a WIS1;
• the buffers: latents and network activations of 8 bytes per element (i64); while a network layer runs, its input, its output and, for a transposed convolution, the uncropped plane it accumulates into, each at most 2^26 elements (Planning the networks); a WIV1 decode's frames (one byte per sample) and its motion vectors (4 bytes each);
• each network's work, its multiply-accumulates layer by layer: a convolution's every output times c·k·k (every tap of its padded input), a transposed convolution's every input times out·k·k (out·k for the audio synthesis), an inverse GDN's every element times c; a WIV1 with keyframe 0 runs no network. A kernel may skip a zero term, so it performs at most this many.

A WIA1's latent is the one its t_out needs and no longer: the synthesis of lat_len samples gives at least t_out samples and that of lat_len − 1 fewer (geometry_mismatch), so a payload cannot declare a large latent to present a few samples.

A decode refuses a stream that runs out at its first missing symbol (§2), so a short stream stops early, but a stream that carries its symbols is decoded in full: a sink that will not spend what a header declares refuses before decoding, by its own size policy, applied to the declared cost (int_payload::declared_cost and int_payload::check_cost, with the set; neural::declared_cost over the set a registry resolved; the C ABI's wai_int_payload_declared, given the set's path; wai-web's intPayloadCost; the header alone through container::declared_size), cost_over_limit (§8). The reference neural sink applies one when it is given one (neural::ModelRegistry::with_cost_limit); its C ABI does not decode integer video, PCM or splat attributes, which it has no output kind for. This extension fixes the caps; a sink's policy may be stricter.

Informative. A sink may keep a parameter set lowered and loaded between decodes, keyed by the SHA-256 of its bytes (integer-operators §8.6), so that only the first decode through it pays for the lowering and the bounds pass. What it keeps is the set's, the same for every payload, and is not part of any decode's declared working bytes; the sink's own budget for kept sets bounds it. The reference implementation's store (int_set_cache::LoadedSets) starts with a budget of 256 MiB, evicts the least recently used set first, keeps no set larger than the whole budget, and keeps nothing with a budget of 0 (LoadedSets::set_budget; wai-web's setLoadedSetBudget). A decode that finds its set kept makes every other check of this section, in the order below, and gives the same output and the same refusals.

The parameter set must fit the payload (model_mismatch):

• every network parses (the WHM1 and WAM1 forms, §9), and the networks of one set share one fixed-point scale afrac in 1..=30;
• a WIA1 or WIS1 payload's afrac is the model's;
• the entropy tables of entropy.json are consistent (a table, length and offset per scale edge, at least one edge; a table, length, offset and median per hyper-latent channel) and every table is a CDF of §2 over its length; a payload has no more hyper-latent channels than the tables;
• each layer's parameters have the lengths the running channel count needs, and its geometry leaves an output (Planning the networks);
• the audio synthesis gives one channel; the conditional codec's g_s gives three;
• a network that decodes through its lowering lowers, and its graph loads under acc64 (integer-operators §13.2 and §13.3: for WIH1 and a WIV1's keyframe, h_s and g_s; for WIS1, g_s; for WIA1, the synthesis).

The shapes the payload declares must be the ones the decode gives (geometry_mismatch): h_s's output and [2·m, sh, sw]; a WIV1 keyframe and h × w; the audio synthesis's length and t_out (it may be longer, by less than one latent sample's worth: the synthesis of lat_len − 1 samples gives fewer than t_out); g_s's output and [attr, s, s].

The order of a decode's checks. After the parse (§2), a decode checks in this order and refuses with the first that fails:

1. the set's members, in the order of §9's list, each present (set_incomplete) and parsed (model_mismatch);
2. the parameter set against the payload (model_mismatch): one fixed-point scale, the payload's afrac, the entropy tables, the hyper-latent channels;
3. every network planned, in decode order (h_s then g_s; the audio synthesis; the splat g_s), each layer by the three rules above (model_mismatch, too_large): the caps come before any shape is compared;
4. what the networks give against the payload: for WIH1, h_s's shape (geometry_mismatch), then g_s's three channels (model_mismatch), then h_s and g_s lowered and loaded, in that order (model_mismatch); for WIV1 with keyframe 1, its keyframe's shape and channels as for WIH1, then the keyframe's size against h × w (geometry_mismatch), then the keyframe's networks lowered and loaded (model_mismatch); for WIA1, one channel (model_mismatch), then at least t_out samples, then a latent no longer than that needs (both geometry_mismatch), then the synthesis lowered and loaded (model_mismatch); for WIS1, g_s's shape (geometry_mismatch), then g_s lowered and loaded (model_mismatch);
5. each stream in decode order (WIH1: the hyper-latent's, then, after h_s runs, the latent's; WIV1 with keyframe 1: the keyframe's streams, then the residual stream), symbol by symbol: a symbol that runs its stream out (stream_exhausted) before its value's range (field_range); after its last symbol, its canonical end (stream_noncanonical);
6. each network as it runs, layer by layer (field_range, below).

So a run-out is reported before the value it would have given, and a size past the caps before a shape that disagrees. A sink that applies a cost limit (Memory and work) compares the declared cost with it after step 4, once everything the cost reads has been checked, and before any symbol is decoded (cost_over_limit, §8).

Exactness. A latent rebuilt as s·2^afrac + base must fit i64; before each network layer runs, the decode checks on the values present that no product, partial sum, requantised value or inverse-GDN term can leave the type its arithmetic is formed in. A stream whose values fail either check is refused (field_range). So a decode with overflow checks and one without compute the same integers, and neither panics. A network that decodes through its lowering (integer-operators §13; for WIH1, a WIV1's keyframe and WIS1, §13.2; for WIA1, §13.3) makes the second check as the check ranges of its graph: a value outside one is refused (field_range), and the refusals are the same. So does WIA1's PCM step (§5 step 3), whose range is its graph's last check range.

8. Refusal codes

The list is closed: a sink reports exactly one of these, the first that applies in the order of §2 and §7.

A refusal of the payload or of the parameter set against it, bad_magic through model_mismatch in the table below (and payload_malformed), is final: the sink does not decode the payload by another path, and SPEC §4's fallback is not tried, since it would decode the same bytes. The other codes say the sink cannot decode through the capability at all, and route as SPEC §4 routes such a capability:

• set_incomplete is a pin that did not resolve (SPEC §3.1): a WAI1 continues at SPEC §4 step 4 (or reports it at step 6), and a WAI2 moves on to its next entry (SPEC §7.1);
• not_dispatchable and unsupported_capability name a capability the sink does not decode: SPEC §4 does not take step 3 (or 5) with it, so a WAI1 continues at step 4 and a WAI2 at its next entry, as for a capability the sink does not advertise;
• cost_over_limit is the sink's own policy, not a fault of the payload: a payload whose declared cost exceeds the limit the sink applies (§7, Memory and work) is refused before any symbol is decoded, and routes as unsupported_capability does, so a WAI1 continues at step 4 and a WAI2 at its next entry.

code	when
bad_magic	the payload does not start with any of the seven magics: a tool's JSON form lands here
format_capability_mismatch	the payload is another of the seven formats than the capability's
keyframe_capability_mismatch	a WIV1 or WIF1 keyframe kind that is not the capability's
version	a version this revision does not define
reserved_nonzero	a reserved field that is not zero
field_range	a field of any section outside its range (a header field, a WIS1 range or table length, a median); or a value the stream decodes that leaves the exact integer range (§7)
too_large	a size past the caps of §7
cdf_malformed	a table that breaks the rules of §2
stream_length	the payload ends before a field or section it declares; or a stream length that is not a multiple of 4, or is under 8
nested_length	a keyframe length K (of a WIV1, or of a WIF1 intra frame) that is not the rest of the payload
trailing	bytes after the last section
stream_exhausted	a stream that ran out before its last symbol
stream_noncanonical	a stream that does not end canonically after its last symbol: a word left unread, or a state other than 2^31 (§2)
geometry_mismatch	a shape the payload declares that the decode does not give (§7); a frame of a WIC1 clip, or an inter frame of a wai.video.int_cond.frame group, whose header is not its first frame's (§17)
model_mismatch	a parameter set that does not fit the payload (§7)
set_incomplete	a parameter set without a member the decode reads (§9); a pinned decode reports SPEC §3.1's set_incomplete
not_dispatchable	a capability SPEC §4 does not dispatch
unsupported_capability	a capability the sink has no integer decoder for
payload_malformed	a wai.neural.int_synth or wai.neural.int_mlicpp payload its decoder refuses
cost_over_limit	a payload whose declared cost (§7) exceeds the limit the sink applies before decoding

9. Parameter sets

A capability that decodes against a parameter set reads these members, by the names a set's listing gives them (prior-carriage §7):

capability	shape	members
wai.neural.int_hyper, wai.video.int_hyper	several files	g_s.whm and h_s.whm (networks in WHM1), entropy.json (the tables)
wai.audio.int_codec	one file	model.wam (the synthesis in WAM1)
wai.splat.int_codec	one file	g_s.whm (the synthesis in WHM1)
wai.video.int_cond, wai.video.int_cond.frame	one file	model.wcb (the WCB1 bundle, §17.3)
wai.video.int_motion	none	—

A set of several files is pinned through its listing (wai.prior.set); a one-file set MAY be pinned form-less, by its file's SHA-256 (SPEC §3.1). wai.audio.int_codec and wai.splat.int_codec were registered with sets of several files, because no one-file format had been named for them; this revision registers each as one file, the file its decoder reads, which holds every parameter its output depends on. That widens their registrations, as SPEC §8 allows a MINOR revision to; a set pinned through its listing still verifies.

WHM1 (int_hyper_synth::IntModel::from_bytes): magic, then afrac, gf, shift and the layer count (u32 each), then per layer a tag byte: 0 a transposed convolution and 1 a convolution (input and output channels, kernel, stride, padding and, for 0, output padding as u32; weights as i16; per-output-channel multipliers and biases as i64), 2 an inverse GDN (channels as u32; gamma as i32, beta as i64), 3 a ReLU, 4 a leaky ReLU (slope i64, its shift u32). WAM1 (int_audio::AudioModel::from_bytes): the same header, then layers of tag 0, a one-dimensional transposed convolution, and tag 2, an inverse GDN. In both forms nothing follows the last layer: a file with bytes after it is refused (model_mismatch), so one network has one file and one digest. The layers' integer arithmetic is integer-operators §13.2 (WHM1) and §13.3 (WAM1).

entropy.json, the tables of wai.neural.int_hyper and wai.video.int_hyper: a JSON object with these members. Other members are ignored. An integer is a JSON number with no fraction and no exponent, and -0 is zero. A file that is not a JSON text (RFC 8259, UTF-8, no member named twice, at most 64 levels, nothing after the value) is refused (model_mismatch). The reference decode reads the text with no float parser.

member	value
scale_edges	N ≥ 1 integers, each fitting i64: the edges e_0 … e_{N−1} of the ceil bucket rule (integer-operators E3)
cdfs	N arrays of integers, each fitting i32: the latent tables, one per bucket
cdf_lengths	N non-negative integers: table b's length n_b
offsets	N integers, each fitting i32: table b's offset
z_cdfs	K ≥ 1 arrays of integers, each fitting i32: the hyper-latent tables, one per channel
z_cdf_lengths	K non-negative integers
z_offsets	K integers, each fitting i32
z_median_q	K integers, each fitting i64: channel k's median at 2^afrac

Each table, over its first n_b entries, is a table of §2: 3 ≤ n_b ≤ 4097, n_b no more than its array's length, cdf[0] = 0, cdf[n_b − 1] = 65536, strictly increasing. Entries after n_b are not read. A set whose tables break any of this is refused (model_mismatch). A payload has no more hyper-latent channels than K (§7). Hyper-latent channel k decodes against z_cdfs[k], z_cdf_lengths[k] and z_offsets[k], and its elements are s·2^afrac + z_median_q[k] (§4 step 1). A latent element decodes against table b, the ceil bucket of its σ over scale_edges (§4 step 3).

10. The tools' JSON forms

The encode and export tools under tools/ write a JSON bitstream. It is a converter input and a debugging aid, never a payload. An encoder MUST NOT carry it as the payload of any capability of §1, and a sink refuses it (bad_magic). The converters (int_payload::{wiv1,wih1,wia1,wis1}_from_json, and the independent int-payload-conformance/int_payload_verify.py) make explicit what the JSON leaves implicit:

• WIV1: an absent mv_precision is 1, and only 1 and 2 are converted; a one-frame clip is written with mv_precision 1 and block 1, whatever the JSON states; mvs has one list per frame, and the first is empty; a frame's list shorter than its block grid is padded with (0, 0), the vector the JSON decoder read for a missing block; a list longer than the grid, or a vector outside i16, is refused; rmax is not carried. The keyframe of wai.video.int_hyper is the WIH1 converted from its own JSON.
• WIH1: the fields of cond_bitstream.json, the streams as they are.
• WIA1: sr is the export's sample_rate (in audio.json, which the bitstream does not carry); lo is the symbol offset; every channel's table must have the first table's length; afrac is carried and must be the model's.
• WIS1: the ranges lo and hi must each be a binary32 value, carried exactly. A number spelled as an integer (no fraction, no exponent) is that integer: 2^60 + 1 is refused, not rounded to 2^60, and -0 is zero, so +0. Any other spelling denotes the binary64 value nearest it (ties to even; one that rounds to zero is a zero of its sign, and one past the binary64 range is no value), and that value must be a binary32 value: the export writes each range as the shortest decimal of a binary32 value widened to binary64. The reference converter reads the JSON text in integer arithmetic, so no float parser decides a range. afrac is carried and must be the model's.

A payload and the JSON it is converted from decode to the same bytes; the conformance vectors check that for every committed JSON bitstream.

11. Envelopes this revision makes non-conforming

This revision completes these strings' registrations with a payload format they did not have (SPEC §8, Completing a registration). It makes non-conforming, and they still parse:

• a WAI1 envelope whose capability or fallback is a string of §1 and whose payload is not that string's format: a tool's JSON form among them;
• a WAI2 envelope with a component of a string of §1 whose bytes are not that string's format.

A sink refuses such a payload when it decodes it (§8), with no fallback attempt. An encoder MUST NOT write one; the reference implementation's emit paths (wai wrap, the C packers and the browser writers) parse the payload as a sink does and refuse to. An encrypted envelope's payload (SPEC §6) is ciphertext until a sink decrypts it, so the emit paths do not check it; the sink checks the plaintext when it decodes it (SPEC §6.4). No committed envelope or conformance corpus carried such a payload when this revision was made.

12. Reference implementation and conformance

• int_payload::wire parses and writes the four formats and WIF1 (§16), with no dependency on the rest of the crate; int_payload::decode_int decodes every integer capability from parameter files held in memory, with no ONNX runtime and no filesystem, for native and wasm32 sinks; int_payload::resolve verifies a pin over sets held in memory (pin::verify_bytes_with) and runs SPEC §3.1's check 5, for a pinned decode and over the files an unpinned decode reads (a listing the set carries must match them, and the digests include the listing of the files read); it is the only way to build the parameters a decode reads (IntParams). The neural sink (neural::decode_payload) delegates every integer capability to decode_int, and its registry reads an unpinned integer set once and runs check 5 over those bytes.
• codecs::PINNED_INT_PAYLOADS is the one table of strings and magics; container::check_int_payload_framing parses a payload with int_payload::wire and the capability's keyframe kind, which needs no integer decoder and no feature, so every emit path runs it: Wai::to_emit_bytes and WaiMulti::to_emit_bytes (and so the C packers), wai-web's pack and packMulti, and wai wrap; wai inspect reports it.
• int_video::decode_typed and decode_pframes_typed decode WIV1 clips through wai-ops' stage operators (int_video::ops); the direct decode they replaced is test-only (legacy_int), the reference the differential holds them to.
• video_receipt::VideoReceipt::seal_decode_wiv1 seals a wai.video.int_motion receipt whose identity is BLAKE3 over the WIV1 payload.
• wai-web decodes the four formats in the browser (decodeWiv1, decodeWih1, decodeWia1, decodeWis1).
• Conformance: the payloads beside their JSON sources in wai-rs/tests/vectors/, and the refusal fixtures and index in int-payload-conformance/, written by wai_int_payload_vectors gen and checked by wai_int_payload_vectors verify; the same corpus checked by int_payload_verify.py, which shares no code with wai-rs and decodes the WIV1 keyframe-0 payloads in full; and the cross-architecture goldens of byte-exact-conformance, which parse and decode the WIV1, WIH1, WIA1 and WIS1 payloads on every platform of its matrix.

13. The wai.neural.int_synth payload and set

SPEC §5 keeps this capability's payload format; this section writes it down.

off	size	field	rule
0	4	H	≥ 1: the image's height
4	4	W	≥ 1: the image's width
8	2	C	≥ 1: latent channels
10	1	S_log2	the latent's side is S = 2^S_log2
11	1	reserved	0
12	C·S·S	the latents	one signed 8-bit value each, [C, S, S] in CHW order

The format has no magic and no version. The payload is exactly 12 + C·S·S bytes, and C·S·S ≤ 2^26.

The set is one file, int_synth.json: a JSON object whose member layers is a non-empty array of layer objects. A layer has in_ch, out_ch, k, stride and pad, each a non-negative integer below 2^32, with out_ch, k and stride at least 1; out_pad, a non-negative integer below 2^32, 0 when absent; weight, in_ch·out_ch·k·k integers each fitting i32, in the layout [in_ch][out_ch][k][k]; bias, out_ch integers each fitting i32; shift, in 0..=127, 0 when absent; and relu, a boolean, false when absent. Other members are ignored.

The decode is integer-operators §13.1's graph on the latents. Its output must be [3, H, W], and it is presented as row-major HWC bytes. Every refusal of the payload, the set or the decode is payload_malformed (§8).

The reference decoder (int_synth::decode_bytes) decodes through the lowering. It checks the payload first: its header, a reserved byte of 0, C ≥ 1, C·S·S ≤ 2^26 and its length. It then lowers the set and loads the graph under acc64, which makes the set's checks (integer-operators §13.1: at least one layer, the layers chain, the lengths, fields below 2^32, out_ch, k and stride at least 1, a shift of at most 127, the caps and the certificate), and runs it. A shift from 64 to 127 has its exact meaning. Several of these checks are new refusals of input no committed corpus contains (integer-operators §15.3, SPEC §8 Planned breaks).

14. WIM1 and WMB1 — wai.neural.int_mlicpp

SPEC §5 keeps this capability's payload format and its one-file parameter set, the WMB1 bundle (read from a set as model.wmb); this section writes both down.

off	size	field	rule
0	4	magic WIM1
4	4	H	≥ 1
8	4	W	≥ 1
12	2	Cz	≥ 1: hyper-latent channels
14	2	Zh	≥ 1
16	2	Zw	≥ 1
18	2	Cy	≥ 1: latent channels, a multiple of K
20	2	Yh	≥ 1
22	2	Yw	≥ 1
24	2	K	≥ 1: channel slices
26	2	reserved	0
28	4	Lz	the z stream's length in bytes
32	4	Ly	the y stream's length in bytes
36	Lz	the z stream	§2, with no length prefix
36 + Lz	Ly	the y stream	§2, with no length prefix

A parser checks, in order: at least 36 bytes; the magic; the reserved field; no dimension 0; Cy a multiple of K; Cz·Zh·Zw, 3·Cy·Yh·Yw and 3·H·W each at most 2^26; and the payload exactly 36 + Lz + Ly bytes. Each stream's length is checked (a multiple of 4, at least 8) when the decode reaches it.

The set is one file, a WMB1 bundle: the magic WMB1, then four sections, each a u32 length followed by that many bytes, in the order h_s, entropy, g_s (each a WHM1 network, §9) and the tables. The tables are a JSON object with the members of entropy.json (§9), and afrac, read as §9 reads entropy.json. Nothing follows the fourth section. The bundle's rules: the three networks' afrac are equal, lie in 1..=30, and equal the tables' afrac; 1 ≤ N ≤ 65536; and every table, over its length, follows integer-operators E1's t2 rule (2 ≤ n_b, n_b no more than its array's length, cdf[0] = 0, cdf[n_b − 1] = 65536, strictly increasing), in place of §2's rule.

The decode:

1. Plan, before any stream: Cz is at most the number of z tables; h_s maps [Cz, Zh, Zw] to [2·Cy, Yh, Yw]; entropy maps [3·Cy, Yh, Yw] to [2·Cy, Yh, Yw]; g_s maps [Cy, Yh, Yw] to [3, H, W]; and no tensor any of them forms exceeds 2^26 elements.
2. z. For each channel k, and each of its Zh·Zw elements in raster order, decode a symbol s against z table k, and the element is dequantize(s, afrac, z_median_q[k]) (integer-operators E2). After the last z symbol, the stream must not have run out.
3. hyper = h_s(z).
4. y. Let y be [Cy, Yh, Yw], all 0. For each slice j = 0 … K − 1, and for each pass (the anchors, (row + col) even, then the rest): σ‖μ = entropy(concat(y, hyper)), where every element of y not yet decoded is still 0. Then, for each channel of the slice (j·Cy/K to (j + 1)·Cy/K − 1), and each of its positions in raster order that belongs to the pass: b is the ceil bucket of σ over scale_edges (integer-operators E3); decode r against table b; the element is dequantize(r, afrac, μ). One y stream carries every slice's two passes, in that order. After the last y symbol, the stream must not have run out.
5. g_s(y), then the RGB8 output step (integer-operators §13.0), as row-major HWC bytes.

The streams decode with t2, lenient and final_only (integer-operators E1). A refusal of the payload, of either stream or of a value the decode forms is payload_malformed; a bundle that breaks its rules, or that does not fit the payload (step 1, and the load checks below), is model_mismatch (§8).

The reference decoder (int_mlicpp::decode) runs the three networks through their lowering, wmb1/1 (integer-operators §13.4: each network by whm1/1, g_s with the RGB8 step), each graph loaded under acc64. Step 1 checks the header and plans the networks from their layer geometry, then lowers and loads them, which makes the set's checks of integer-operators §13.2 (a layer of tag 0, 1 or 2 in each network; each layer fits the channels the layer before it gives; header shifts under 128; a leaky shift under 64; no bias of magnitude 2^63 − 1 or more; the certificate), then plans each graph on the payload's shapes (integer-operators §4: its input's channel count, Cz, 3·Cy or Cy, and the caps, a convolution's padded side included), all before either stream. A value outside a graph's check range is refused (payload_malformed): the ranges refuse exactly the values a layer's arithmetic could leave i64 on. Some of these checks are new refusals of input no committed corpus contains, and some now come before the streams where an earlier decoder made them when the network ran (integer-operators §15.3, SPEC §8 Planned breaks).

15. Receipts of wai.video.int_motion

Two signed receipts describe a WIV1 payload of wai.video.int_motion (keyframe 0, §3). The decode receipt (§15.1) is a sink's statement of what the payload reconstructs to; the encode receipt (§15.3) is an encoder's statement of the payload it wrote, the source it wrote it from, its parameters and its error. Each binds a measured energy figure and its acquisition class (energy-measurement §2) beside the figures anyone can recompute, and keeps the two apart. The reference is wai-rs's video_receipt (VideoReceipt, VideoEncodeReceipt; feature video_receipt).

In this section, integers in a signing payload are big-endian and those hashed into a frame or clip commitment little-endian, as the reference writes them; BLAKE3 is the hash of every commitment.

15.1 The decode receipt

field	what it is
codestream_hash	BLAKE3 over the payload's bytes
h, w	the payload's geometry
frames	for each decoded frame i, from 0: BLAKE3 over wai:video-rframe (ASCII), 0x01, i (u64) and the frame's RGB8 bytes
clip_hash	BLAKE3 over wai:video-clip, 0x01, h (u32), w (u32), the number of frames (u64), then each frame's i (u64) and hash
merkle_root	the jwp-receipts §2.2 Merkle root over the frame hashes, in order
work_pixel_ops	frames × h × w × 3, saturating at 2⁶⁴ − 1: the decoded subpixels
joules_micro, the class	the energy the sink spent reconstructing, in microjoules, and its acquisition class; 0 and no class is the unmetered marker
parent_receipt_hash	the receipt hash of the receipt before it, or none
signer_pubkey, signer_id, sig	the signer's Ed25519 key, its id, and the signature

The signing payload is wai:video-receipt and 0x01, then codestream_hash, clip_hash, h (u32), w (u32), merkle_root, the number of frames (u64), work_pixel_ops (u64), joules_micro (u64), the class, the parent (0x00, or 0x01 and the hash), each frame's i (u64) and hash, signer_pubkey, and signer_id's length (u32) and UTF-8 bytes. The class is one tag byte and its evidence: 0 the unmetered marker; 1 HwShunt (legacy), 2 ModelBased, 3 Estimator, with none; 4 OnChipCounter, then relative_ppm (u32), window_us (u64) and the filtering state (u8: 0 Unknown, 1 Off, 2 On); 5 CalibratedInstrument, then relative_ppm (u32), window_us (u64), and the certificate reference's length (u32) and UTF-8 bytes. sig is Ed25519 over the payload, verified strictly (SPEC §7 Signatures). The receipt hash is BLAKE3 over wai:video-receipt-id, 0x01, the payload and sig.

A receipt is consistent when merkle_root, clip_hash and work_pixel_ops are what its frames give, its frames are numbered 0 upwards, its figure and class cohere (0 with no class, or a figure above 0 with a class whose evidence energy-measurement §2.3 requires and whose declared relative_ppm × joules_micro reaches the resolution floor of 288676), and its signature verifies. It is re-derived from a payload when the payload's BLAKE3 is its codestream_hash, the payload decodes (§3), and the decode has its geometry and, frame by frame, its frame hashes; and then it is consistent. The figure is attested by the signature and by nothing else: another sink measures another figure for the same decode.

15.2 The JSON form

A decode receipt's JSON form is one object of these members, in this order: profile ("wai.video.int_motion/receipt"), codestream_hash, clip_hash, h, w, frames (each frame's hash), merkle_root, work_pixel_ops, joules_micro, uj_per_kilopixel_op, energy_provenance, the class's evidence (for OnChipCounter, energy_uncertainty {"relative_ppm", "window_us"} and energy_counter_filtering; for CalibratedInstrument, energy_uncertainty and energy_calibration_ref; for the others, none), parent_receipt_hash, signer_pubkey, signer_id, sig and receipt_hash. Hashes, keys and signatures are lowercase hexadecimal; h, w and the figures are integers in plain decimal; energy_provenance is the class's label, or null when unmetered, and parent_receipt_hash null when there is none. uj_per_kilopixel_op is derived: null when the receipt is unmetered or its work is 0, and otherwise joules_micro × 1000 / work_pixel_ops, computed in IEEE 754 binary64 (the product, then the quotient) and written with six digits after the point, correctly rounded.

A reader reads the text strictly (RFC 8259, no member named twice), requires exactly the members the receipt's class writes, each of its form, reads the class and its evidence as energy-measurement §2.7 reads them, and refuses a uj_per_kilopixel_op that is not, spelled as above, the receipt's own, and a receipt_hash that is not its receipt hash. The members' order and the whitespace between them are not held. A receipt read so is then consistent or not, and re-derived from its payload or not, as §15.1 says.

15.3 The encode receipt

An encode receipt (wai.video.int_motion/encode-receipt) states the payload an encoder wrote and the source it wrote it from:

field	what it is
codestream_hash	BLAKE3 over the payload's bytes
h, w	the source's geometry
source_frames	for each source frame i, from 0: BLAKE3 over wai:video-sframe (ASCII), 0x01, i (u64) and the frame's RGB8 bytes
source_hash	BLAKE3 over wai:video-source, 0x01, h (u32), w (u32), the number of frames (u64), then each frame's i (u64) and hash
block, search, mv_precision, q	the encoder's parameters: the payload's block, mv_precision and q (§3), and the motion search's radius, which the payload does not carry
sse	Σ (source − decode)² over every sample of every frame (u64): the error the payload decodes with
work_pixel_ops	frames × h × w × 3, saturating at 2⁶⁴ − 1: the encoded subpixels
joules_micro, the class	the energy the encoder spent, in microjoules, and its acquisition class; 0 and no class is the unmetered marker
parent_receipt_hash, signer_pubkey, signer_id, sig	as §15.1's

Its signing payload is wai:video-encode-receipt and 0x01, then codestream_hash, source_hash, h (u32), w (u32), the number of frames (u64), block (u32), search (u32), mv_precision (u8), q (u32), sse (u64), work_pixel_ops (u64), joules_micro (u64), the class (§15.1's encoding), the parent, each source frame's i (u64) and hash, signer_pubkey, and signer_id's length (u32) and bytes. Its receipt hash is BLAKE3 over wai:video-encode-receipt-id, 0x01, the payload and sig. It is a profile defined after the hardware class was refined (energy-measurement §2.6, New profiles): its class is never the legacy HwShunt.

An encode receipt is consistent as §15.1 says of a decode receipt, with source_hash for clip_hash and no Merkle root, and the new-profile rule for its class. It is re-derived from a payload and a source when the payload's BLAKE3 is its codestream_hash; the payload reads as WIV1 keyframe 0 (§3) of its h, w and number of frames, with its block, mv_precision and q; each source frame hashes to its own; the payload decodes; and the decode against the source gives its sse; and then it is consistent. Re-deriving it proves the payload decodes to frames within sse of the stated source. It does not prove which encoder wrote it, or that search was the radius used.

Its JSON form is one object of profile ("wai.video.int_motion/encode-receipt"), codestream_hash, source_hash, h, w, source_frames, block, search, mv_precision, q, sse, work_pixel_ops, joules_micro, energy_provenance, the class's evidence (as §15.2), parent_receipt_hash, signer_pubkey, signer_id, sig and receipt_hash, read as §15.2 reads a decode receipt's.

16. WIF1 — one frame of integer inter-frame video

WIF1 is the frame form of WIV1 (§3): the payload of wai.video.int_motion.frame (keyframe 0) and wai.video.int_hyper.frame (keyframe 1), one frame an object. frame-stream says how frames make groups and streams, the manifest member that links them, and what a sink checks across them; this section pins one frame's bytes and its decode. A frame is an intra frame, which opens a group, or an inter frame, decoded against the frame before it in its group.

off	size	field	rule
0	4	magic WIF1
4	1	version	1
5	1	kind	0 intra, 1 inter
6	1	keyframe	0 or 1
7	1	mv_precision	inter: 1 (whole-pel) or 2 (half-pel); intra: 0
8	4	h	≥ 1
12	4	w	≥ 1
16	4	block	inter: ≥ 1; intra: 0
20	4	q	intra: 1..=65535; inter: 0
24	4	offset	i32; inter: 0
28	2	cdf_len	intra: 3..=4097; inter: 0
30	2	reserved	0

An intra frame continues with:

size	field	rule
cdf_len·4	the residual table	§2
4 + L	keyframe 0: the frame's residual stream	§2
4 + K	keyframe 1: u32 K, then a WIH1 payload (§4)	K is exactly the bytes that remain

and an inter frame with:

size	field	rule
nb·4	motion vectors	i16 dy, i16 dx, nb = ⌈h/block⌉·⌈w/block⌉, the block grid row-major
4 + L	the frame's residual stream	§2

Checks. §2's order, with these rules: the header's fields in the order kind, keyframe, mv_precision, h and w, block, q, offset, cdf_len (field_range), then the keyframe kind (keyframe_capability_mismatch: keyframe 0 is wai.video.int_motion.frame's, 1 is wai.video.int_hyper.frame's); then the caps of §7 on the frame h·w·3 and, for an inter frame, the motion-vector field 2·nb (too_large); then the sections in the order above, and nothing after them (trailing, or nested_length for a keyframe). A field one kind of frame does not use is fixed at 0, so each frame has one header. The header alone sizes every section, so a frame parses without the frames before it.

What a group's intra frame fixes. h, w, the residual table, q and offset are the group's: an inter frame reads its stream against its intra frame's table at its offset, steps by its q, and has its intra frame's h × w (geometry_mismatch, frame-stream §5). mv_precision and block are each inter frame's own.

Decode. Each frame has a residual stream of its own, decoded by E1 rans64_decode (t4097, lenient, canonical) and ending canonically after the frame's last symbol (§2): h·w·3 symbols, raster (y, x), then R, G, B.

• An intra frame with keyframe 0 is frame 0 of a WIV1 with keyframe 0 (§3): pred = 128, the sample clamp(pred + r·q, 0, 255).
• An intra frame with keyframe 1 is its WIH1's decode (§4), through the capability's parameter set, which must be h × w (geometry_mismatch).
• An inter frame is a frame f ≥ 1 of §3: pred reads the previous decoded frame of its group with the pixel's vector, whole-pel or half-pel, and the sample is clamp(pred + r·q, 0, 255) with its group's q.

The output is one RGB8 frame of h × w: the buffer of the frame's frame-state-equivalence digest (SPEC §7 Digests). The decode is §3's frame step, through the same stage operators (integer-operators §13.5), so a group whose intra frame carries a clip's table, q, offset and frame 0 (or keyframe), and whose inter frames carry the clip's vectors and each frame's residual symbols, decodes to the clip's frames, byte for byte (frame-stream §6). Since every frame's stream is its own, a frame's decode reads no byte of another object; what it reads of another frame is the decoded frame before it, which frame-stream §5 holds to be its predecessor's.

Reference implementation. int_payload::wire::{parse_wif1, write_wif1}; int_video::decode_unit_typed (the frame step, through int_video::ops::decode_unit); frame_stream::IntFrames, the stateful decoder; frame_stream::split_wiv1, which cuts a WIV1 clip into one group of WIF1 frames. Conformance: int-payload-conformance/frame-stream/.

17. WIC1 and WCF1 — integer conditional inter-frame video

wai.video.int_cond codes each frame conditionally: not as a residual against a prediction of its pixels, but as a latent coded given a temporal context, the previous frame's decoded feature, and decoded back into a feature and a frame with that context as a second input. Four integer networks carry the decode, each a WHM1 network (§9) lowered by whm1/1 (integer-operators §13.2), so every scale, every table, every feature and every frame is an integer function of the payload and the set, the encoder's on every decoder. Its payload is WIC1, a clip of WCF1 frames; wai.video.int_cond.frame, its frame form (frame-stream), carries one WCF1 frame an object. Both decode against one parameter set of one file, model.wcb, a WCB1 bundle (§17.3). Their registry class is NeuralIntegerExact, at decode-equivalence (determinism-tiers), and their outputs are frames, each a frame-state-equivalence digest's buffer (SPEC §7 Digests).

17.1 WCF1 — one frame

off	size	field	rule
0	4	magic WCF1
4	1	version	1
5	1	kind	0 intra, 1 inter
6	2	reserved	0
8	4	h	≥ 1: the frame's height
12	4	w	≥ 1: the frame's width
16	4	m	≥ 1: latent channels
20	4	sh	≥ 1: the latent's and the feature's height
24	4	sw	≥ 1: their width
28	4	cf	≥ 1: the reference feature's channels
32	4 + L	the latent stream	§2

Checks. §2's order: the header's fields kind, then h, w, m, sh, sw and cf (field_range); then the frame h·w·3, the scales and means 2·m·sh·sw and the reference feature cf·sh·sw, each at most 2^26 elements (too_large); then the stream, and nothing after it (trailing). The header alone sizes the frame, so it parses without the frames before it.

17.2 WIC1 — a clip

off	size	field	rule
0	4	magic WIC1
4	1	version	1
5	3	reserved	0
8	4	n_frames	1..=2^20
12	…	n_frames frames	each a u32 F, then F bytes: one WCF1 frame

Checks. §2's order: n_frames at least 1 (field_range) and at most 2^20 (too_large); then each frame in order: its length and its bytes present (stream_length), its bytes a WCF1 frame (§17.1's codes), frame 0 intra and every later frame inter (field_range); after frame 0, the clip n_frames·h·w·3 at most 2^30 bytes (too_large); every later frame with frame 0's header, h to cf (geometry_mismatch); and nothing after the last frame (trailing).

A clip is one group of the frame form (frame-stream §2): its split is its frames' bytes, copied, frame 0 the group's intra frame; its join writes a group's frames back into a clip, which must then parse as one (a group of no frame, an intra frame after the first or another header is refused with the parse's code). Each gives the other's input back, byte for byte, and since every frame's stream is its own, a group decodes to its clip's frames, frame for frame.

17.3 The parameter set: WCB1

One file, read from a set as model.wcb: the magic WCB1, then five sections, each a u32 length and that many bytes, in this order:

section	what it is
ctx	the temporal context network: the reference feature [cf, sh, sw] → the context [cc, sh, sw]; cc is its output channels
prior	the entropy context network: the context → [2·m, sh, sw], the scales σ (channels [0, m)) and the means μ ([m, 2·m))
dec	the conditional decoder: concat(ŷ, context), [m + cc, sh, sw] → the frame's feature [cf, sh, sw]
recon	the synthesis: the feature → [3, h, w]
the tables	a JSON object: afrac; scale_edges, N integers each fitting i64; cdfs, N arrays of integers each fitting i32; cdf_lengths, N non-negative integers; offsets, N integers each fitting i32. Other members are ignored

Its rules, each refused model_mismatch: each section inside the bundle, and nothing after the fifth; each network a WHM1 network (§9); the four sharing one fixed-point scale afrac in 1..=30, which is the tables' afrac; the tables read as §9 reads entropy.json (a JSON text, read strictly, whose numbers are integers as spelled); N from 1 to 65536; and each table, over its first n_b entries, a table of §2: 3 ≤ n_b ≤ 4097, n_b no more than its array's length, from 0 to 65536, strictly increasing. The networks' arithmetic is integer-operators §13.2, and the bundle's lowering is wcb1/1 (§13.4a there).

17.4 Decode

A decode checks in this order and refuses with the first that fails:

1. the parse (§17.1, or §17.2 for a clip);
2. the set's member model.wcb, present (set_incomplete) and a WCB1 bundle that keeps §17.3's rules (model_mismatch);
3. the four networks planned on the frame's header (§7, Planning the networks), each layer by §7's three rules (model_mismatch, too_large), all four before any shape is compared: ctx on [cf, sh, sw]; prior on [cc, sh, sw]; dec on [m + cc, sh, sw]; recon on [cf, sh, sw]. Then ctx keeps sh × sw, prior gives [2·m, sh, sw] and dec [cf, sh, sw] (each geometry_mismatch), and recon gives three channels (model_mismatch) of h × w (geometry_mismatch);
4. the four networks lowered and loaded under acc64 (model_mismatch).

Then each frame, in order. Its reference feature F is zero, [cf, sh, sw], for an intra frame, and the feature the frame before it in its group left for an inter frame (in a clip, the frame before it):

5. C = ctx(F), then prior(C), whose channels [0, m) are σ and [m, 2·m) are μ;
6. the latent ŷ [m, sh, sw], element by element in order (channel, then row, then column): its table is b, the ceil bucket of its σ over the scale edges (integer-operators E3, as §4 step 3 gives it); its symbol r is the next of the frame's stream against table b at its offset (§2); and it is r·2^afrac + μ, which must fit i64 (field_range). A symbol that runs the stream out is refused before its value is formed (stream_exhausted), and after the last symbol the stream ends canonically (stream_noncanonical);
7. F' = dec(concat(ŷ, C)), the latent's channels first;
8. recon(F'), then the RGB8 step (integer-operators §13.0): the frame, as RGB8 bytes, rows top to bottom, pixels left to right, R, G, B;
9. F' is the next frame's reference feature.

Each network runs through its lowering, whose check ranges refuse a value a layer's arithmetic could take out of i64 (field_range, §7 Exactness). The stage operators are E1 rans64_decode (t4097, lenient, canonical, one stream a frame), E2 dequantize and E3 bucket (ceil); concat, the zero feature and the split of prior's output are driver steps (integer-operators §13.5).

Memory and work. The header and the set declare the decode's output and work, as §7 says: the frames, n_frames·h·w·3 bytes; m·sh·sw symbols a frame; the networks' multiply-accumulates a frame, layer by layer, from their plan; and the reference feature, cf·sh·sw values of 8 bytes, kept from frame to frame.

17.5 The frame form

wai.video.int_cond.frame carries one WCF1 frame an object, in groups (frame-stream §2): a group's intra frame decodes against the zero feature, and each inter frame against the feature the frame before it left, which frame-stream §5 holds to be its predecessor's. An inter frame carries its group's intra frame's header, h to cf (geometry_mismatch); then it is decoded by §17.4 steps 2 to 9, through the set its pin selects. A sink keeps one feature for each stream: no pixel of an earlier frame is read.

17.6 Reference implementation and conformance

• int_payload::wire::{parse_wic1, write_wic1, parse_wcf1, write_wcf1}; int_cond_video::{CondModel, plan, load, decode_frame, decode_clip, split, join}; int_payload::decode_int for wai.video.int_cond; int_cond_video::CondFrames, the frame form's decoder under frame_stream::FrameStream; container::check_int_payload_framing, so the emit paths parse a payload of either string before they write it; wai_ops::lower::wcb1.
• The vector wai-rs/tests/vectors/int_cond/: a six-frame 64 × 64 clip and its set, whose networks tools/wai_int_cond_export.py trained on seeded procedural clips and quantised after training (per-output-channel 12-bit weights, activations at 2^12), then encoded with the integer networks themselves; its sidecar states each decoded frame's SHA-256, which tests/int_kernels_vectors.rs holds the clip to under every kernel policy.
• int-payload-conformance/int-cond/: the clip's frames, frame streams of the frame form (clean, two groups, a frame lost, another picture size, a stream that runs out), payload fixtures for every code a parse gives and for the decode's, set fixtures and join refusals, written by wai_int_payload_vectors gen and checked by wai_int_payload_vectors verify and tests/int_cond_video.rs; and checked again by int_payload_verify.py --int-cond, which shares no code with wai-rs and decodes the clip, every stream and every decode fixture with its own integer networks and rANS decoder.

17.7 Weights from a pretrained model (informative)

The committed vector's networks were trained for this topology. A pretrained conditional codec's own weights are lowered onto it by post-training quantisation, as wai.neural.int_hyper's are from the pretrained mbt2018_mean (tools/wai_sota_int_synth_export.py). DCVC-RT (wai.neural.dcvc_rt, SPEC §5) reads, as this topology does, the previous frame's decoded feature as its temporal context, with no motion estimation or compensation, so lowering its published weights takes no warp or motion-compensation operator. It takes the following; the second column says what this revision's forms hold of each.

what the lowering takes	in this revision
The checkpoint, pinned by its full SHA-256 and read as tensors only, so that the set the lowering writes is a function of known bytes	as for mbt2018_mean; the set is one file, pinned by its SHA-256 (SPEC §3.1)
Its modules in the four roles: the adaptor that turns the previous decoded feature into the temporal context is ctx; the path from that context to the latent's scales and means is prior; the decoder from the latent and the context to the feature is dec; the reconstruction from the feature to the frame is recon. The analysis side is the encoder's	the four sections of WCB1 (§17.3)
Its blocks: residual additions around a block, which a WHM1 chain cannot state; depthwise convolutions and sub-pixel resampling, which a chain states only as dense convolutions (a depthwise one with zero weights off its diagonal, its work multiplied by its channel count; a pixel shuffle as a transposed convolution with one weight of 1 an output)	each is a graph operator of wai-ops/1: conv with groups (integer-operators §5.2), add (§5.5), pixel_shuffle and pixel_unshuffle (§5.10), so wai-ops/1 gains no operator for them. Networks built of them are WHG1 graphs (§10 there) with a bounds certificate (§8 there), while WCB1's sections are WHM1 chains: such a set is a bundle of graphs
The entropy model's other inputs: a hyperprior, a second latent with a stream and tables of its own, whose decoded output joins the temporal context before the scales and means are formed; and spatial context, the latent decoded in passes over a partition of its positions, the scales and means formed again before each pass from the elements already decoded	each has an integer form here: a hyper-latent stream and its tables in WIH1 (§4, §9), passes in the coding order of WIM1 (§14), a 2 × 2 partition of positions in subpos_split and subpos_merge (integer-operators §5.10). WCF1 carries one stream decoded in one pass (§17.4), and a registered payload format is fixed (SPEC §8): a payload that carries them is a registration of its own
Its rate: the per-channel scales a rate index selects, at the latent, the feature and the reconstruction	one set a rate: a scale on a layer's output folded into that layer's requantisation multipliers, one on a layer's input into its weights. WCF1 names no rate, so a payload that names one is, again, a registration of its own
The quantisation: per-output-channel weights, activations at 2^afrac calibrated on the model's own activations, and every layer within §7 Exactness at acc64	WHM1 holds 16-bit weights (this vector uses 12 bits), and the four networks share one afrac (§17.3). A device's 32-bit subset (wai-ops/1-i32, integer-operators §9.1) runs a network at acc32 only where its certificate bounds every intermediate in 32 bits (§7.2, §8 there)
The tables: the model's scale table as scale_edges at 2^afrac, and one table an edge from its Gaussian at that scale	the tables section, as entropy.json holds mbt2018_mean's (§9): up to 65536 edges, each table of up to 4097 entries
The picture format: the samples the model codes	the decode ends in the RGB8 step, three channels at h × w (§17.4 step 8); a model that codes another sampling, such as 4:2:0, takes an output step of its own
The check: the integer decode held to the floating-point decode of the same streams, as tests/int_hyper_synth.rs holds int_hyper's within 1 dB	the export records each frame's PSNR against its source (expected.json)