open-standards/wai/int-payload-conformance

int-payload-conformance — the integer payloads

Vectors for the payload formats SPEC §5 pins to the WAI-native integer codecs (extensions/integer-payloads.md): WIV1 for wai.video.int_motion (keyframe 0) and wai.video.int_hyper (keyframe 1), WIH1 for wai.neural.int_hyper, WIA1 for wai.audio.int_codec and WIS1 for wai.splat.int_codec.

Provenance. Generated here: wai_int_payload_vectors gen converts the vectors under ../wai-rs/tests/vectors/ and changes them for the refusal fixtures; tests/vectors/README.md states each vector's provenance. One payload indexed here is not generated content: video_real (int_video) codes a third-party image (see there).

Code-equivalence. A conforming sink decodes every payload to the output digests its vector states, and refuses every fixture with the code it states, from the extension's closed list (§8).

• vectors.json — the index. For each payload: its capability, its path, its SHA-256, the JSON it is converted from, the parameter set it decodes against (each member's name in the set, and the committed file that holds it), and the sha256.json sidecar that states its decoded output's SHA-256. For each refusal fixture: its capability, its path, its code, and whether parsing alone refuses it (parse) or the decode does (decode). Paths are relative to wai/.
• The payloads sit beside their JSON sources under ../wai-rs/tests/vectors/: int_video/*.wiv1, int_hyper/tiny_bitstream.wih1, int_hyper/leaves_bitstream.wih1, int_audio/audio_bitstream.wia1 and int_splat/splat.wis1. leaves is a 128 × 128 generated picture coded with the set trained on generated pictures (int_hyper/leaves_set.json), decoded against that set. Their sidecars are the ones the JSON forms' decodes are checked against: a payload and its JSON source decode to the same bytes.
• refusals/*.bin — each a committed payload changed in one place (a byte, a field, a truncation, a word appended to a stream), so that it is refused with one code, or, for wiv1_end_state and wiv1_one_frame_end_state, a small clip written to end its stream in a state other than 2^31. The fixtures named *_order_* break two rules at once and pin which one a sink reports (the extension's §2 and §7), and so do the wiv1_geometry_flip_* fixtures: wiv1_geometry with one byte of its keyframe's last stream words changed, refused for the keyframe's size before that stream is read. The fixtures for bad_magic are the JSON sources themselves: a tool's JSON output is a converter input, never a payload.
• sets/ — a parameter-set member a fixture decodes against in place of a committed one: entropy_median_max.json is the tiny conditional codec's tables with one median at i64::MAX.

Files are compared as bytes; .gitattributes turns off line-ending conversion.

Generation. Every file here, and every payload under tests/vectors/, is written by one generator from the committed JSON alone, so it can be rebuilt and compared:

cd wai/wai-rs
cargo run --no-default-features --features video_int,audio_int,splat_int \
    --bin wai_int_payload_vectors -- gen ../int-payload-conformance

Verifiers.

• wai_int_payload_vectors verify ../int-payload-conformance (wai-rs) regenerates every file and requires it to equal the committed one byte for byte, decodes every payload through int_payload::decode_int to its sidecar's SHA-256, and requires every fixture to be refused with its code at its layer, and by int_payload::declared_cost with the same code unless the decode's refusal comes from a stream or a network run. tests/int_payload.rs runs the fixtures again in both build profiles, refuses every strict prefix of every payload, and runs a sweep of single-byte changes in which nothing may panic; tests/int_payload_preflight.rs checks that a header past the caps or past what the parameter set can fit is refused before anything sized from it is allocated, and that a payload whose streams cannot carry what its header declares is refused at its first exhausted symbol, having allocated only what it decoded.

• int_payload_verify.py, Python standard library only and sharing no code with wai-rs, written from the extension: it parses every payload with the extension's rules and check order, converts every JSON source independently and requires the committed bytes, decodes the WIV1 payloads with keyframe 0 in full (the rANS decoder with its bypass escape, its stop at the first exhausted symbol and its canonical end, whole-pel and half-pel motion compensation) to their sidecars' SHA-256, and checks every fixture: a parse fixture's code, and that a decode fixture parses (a WIV1 keyframe-0 one is decoded and must be refused with its code).

  python3 int-payload-conformance/int_payload_verify.py int-payload-conformance
  

Not decoded independently. The Python verifier does not decode WIH1, WIV1 with keyframe 1 (its keyframe is a WIH1), WIA1 or WIS1: their decodes run integer networks it does not implement. Their outputs are checked by wai-rs against the sidecars above, and also by the cross-architecture goldens in ../byte-exact-conformance/. For their decode fixtures the Python verifier checks that they parse; their codes are checked by wai-rs.

Frame streams

frame-stream/ holds the frame streams of extensions/frame-stream.md: the frame forms wai.video.int_motion.frame and wai.video.int_hyper.frame (WIF1, integer-payloads.md §16) and wai.neural.dcvc_rt.frame (WNF1, frame-stream §4), one frame an object.

• index.json — the index. Each stream: its capability, its mode (full: decoded; parse: checked to its links and geometry, for the neural frames, whose decode needs the sink-supplied networks), the parameter set it decodes against, the clips it was split from (split_from, with their sidecars), and its objects in order, each with its envelope, its content hash and what a sink does with it: ok (with the decoded frame's SHA-256 where the stream decodes it) or the code it is refused with. Then the payload fixtures, the neural clip with its frames and the clips and groups its split and join refuse, and the catalog cases.
• streams/<name>/NN.wai — the envelopes, each written once, where a stream first holds it. int_video, video_real and video_nk are those clips split into one group each; two_groups is two of them in one stream; dcvc is the synthetic neural clip's frames. Every other stream plants one fault of frame-stream §5: a frame lost (gap), a prev altered (prev_mismatch), an index skipped (index_mismatch), a join after a group's intra frame (joined), a link on an intra frame or none on an inter frame (intra_linked, inter_unlinked), a member missing or malformed (link_missing, link_malformed), another picture size (geometry, dcvc_geometry), another capability (capability_mismatch), a payload whose stream runs out (payload_refused), another pin (prior_changed), a neural frame lost (dcvc_gap).
• payloads/*.bin — WIF1 and WNF1 frames each changed in one place, refused at their parse or, for wif1_exhausted and wif1_noncanonical, their decode.
• dcvc/ — the synthetic whole-clip neural payload (clip.bin), its frames, and the clips (clip_*.bin) and groups (join_*/) the split and the join refuse. Its bitstreams are generated bytes, not an encode.
• catalog.json — MoQ catalog entries with and without wai_frame_stream (moq-streaming-format.md §4.3), and objects checked against them with their Object IDs.

Equivalence. A clean split's frames decode to its clip's frames, byte for byte, and to the SHA-256 the clip's sidecar states for each: the stream's frames are the clip's.

Generation and verifiers. wai_int_payload_vectors gen writes frame-stream/ with the rest of this directory, from the committed clips alone (wai::frame_stream::corpus::build), and verify regenerates it, decodes every stream through frame_stream::FrameStream, holds each clean split to its clip's whole decode, and checks every payload, split, join and catalog case (frame_stream::corpus::verify). int_payload_verify.py checks it too, and shares no code with wai-rs: it splits each clip again with its own rANS encoder and requires the committed frame payloads, decodes every keyframe-0 frame, makes frame-stream §5's checks over every stream with its own BLAKE3 (../claim-conformance/_blake3.py), and checks every payload, split, join and catalog case. It does not decode a WIH1 keyframe: the wai.video.int_hyper.frame streams it checks to their links and geometry, and wai-rs checks their decode.

python3 int-payload-conformance/int_payload_verify.py --frame-stream int-payload-conformance

NMSF tracks

frame-stream/nmsf/ holds the mapping of frame-stream §9 between a frame stream of wai.neural.dcvc_rt.frame or wai.neural.dcvc_fm.frame and a track of packaging nvc (draft-herz-moq-nmsf-01).

• index.json — the model bindings, and bindings that do not read (binding_cases); NMSF catalog entries, each with the WAI entry it maps to or the code that refuses it (nmsf_catalog); WAI entries, each with the NMSF entries and the pin it reads to, or the code (wai_catalog); rewrap streams, each NMSF object with its Object ID and the frame object it maps to, or the code (rewrap); unwrap streams, each frame object with the NMSF objects it maps to, or the code (unwrap); and frame objects whose number or capture_ms a sink refuses (links, frame-stream §5).
• objects/<stream>/NN.nmsf (NN.hyper.nmsf and NN.latent.nmsf in two-track mode) — NMSF objects: a 64 × 64 stream of two groups, the first frame of each an Intra frame, and the same objects changed in one place each, for every rule of frame-stream §9.4. Their bitstreams are generated bytes, not an encode.
• wai/<stream>/NN.wai — the frame objects. The single-track and two-track streams map to the same files: one frame object for both modes.

Round trip. Every rewrap stream's frame objects, admitted by a sink and unwrapped in order through the WAI entry its catalog case maps to, give back its NMSF objects byte for byte. wai_int_payload_vectors verify (frame_stream::nmsf::corpus::verify) and int_payload_verify.py, which reads every NMSF object and entry with its own parser and requires each committed frame object's payload and manifest, both check it, and both require every code of frame-stream §9.6 to be given by a case. --frame-stream runs it with the frame streams.

Conditional inter-frame video

int-cond/ holds wai.video.int_cond and its frame form wai.video.int_cond.frame (integer-payloads.md §17): the committed clip under ../wai-rs/tests/vectors/int_cond/ (clip.wic1, six 64 × 64 frames), its parameter set (model.wcb, networks trained and quantised by tools/wai_int_cond_export.py) and its sidecar (sha256.json, each decoded frame's SHA-256).

• index.json — the index: the set and its pin, the clip and its frames, the streams with what a sink does with each object (ok, with the frame's SHA-256, or the code it is refused with), the payload fixtures with their code and layer, the set fixtures and the join refusals.
• frames/NN.wcf1 — the clip split into its WCF1 frames, a copy of each.
• streams/<name>/NN.wai — frame objects pinned to the set: clip (the clip as one group), two_groups, and one fault each: gap (a frame lost), geometry (an inter frame of another picture size), payload_refused (a frame whose stream runs out).
• refusals/*.bin — WCF1 frames and WIC1 clips changed in one place each: a fixture for every code their parse gives, and for the decode's (stream_exhausted, stream_noncanonical, model_mismatch, geometry_mismatch, and field_range: a clip whose latent symbols, near ±2^31 and signed so that the conditional decoder's sums add up, carry a value out of a network's check range).
• sets/*.wcb — the set changed in one place each, against which the clip is refused (model_mismatch); no_set decodes with no set (set_incomplete).
• joins/<name>/ — groups the join refuses.

Equivalence. The clip decodes to its sidecar's SHA-256; its frames, as frame objects, decode to the clip's frames byte for byte.

Generation and verifiers. wai_int_payload_vectors gen writes int-cond/ from the committed vector alone (wai::int_cond_video::corpus::build), and verify regenerates it and checks every case (corpus::verify), as tests/int_cond_video.rs does. int_payload_verify.py checks it too and shares no code with wai-rs: it decodes the clip, every stream and every decode fixture with its own integer networks (WHM1 convolutions, transposed convolutions, ReLU and leaky ReLU, with whm1/1's check ranges) and its own rANS decoder, and requires every SHA-256 and every code.

python3 int-payload-conformance/int_payload_verify.py --int-cond int-payload-conformance