open-standards/wai/format-probe-conformance/README.md

format-probe-conformance — header probes

The corpus of extensions/format-probes.md §13 and of extensions/projection.md §7.

Feature equivalence. A conforming probe of a capability, given a vector's file, reaches the outcome vectors.json states for it: the parameters ({"accept": {...}}, the members each section of format-probes names) or the refusal ({"refuse": code}, a code of its §12). A conforming reader of the projection descriptor gives each binding of bindings its outcome (projection.md §5). Each fact of facts holds of its file's probe: what its encoder was asked for.

• ext/ — vectors written by encoders, from synthetic pictures (tools/yuv.py):
  • vvc/ H.266 Main 10 at 64×64 and at 100×58 (a conformance window);
  • mvhevc/ an MV-HEVC bitstream of two views, and hevc/ its base layer;
  • jxs/ JPEG XS codestreams: 4:2:2 at 10 bits, R, G and B under the reversible colour transform, and a lossless 4:4:4 one; jxs-video/ three frames in wai.video.jxs's framing;
  • avs3/ an AVS3 video stream of three pictures.
• gen/ — what wai_format_probe gen writes: LCEVC streams, MHAS streams, AVS3 audio frame headers, IEEE 1857.11 picture headers and projection descriptors from fixed values; AV2 length-delimited streams (av2/), AC-4 sync frames (ac4/), IA Sequences (iamf/) and IVAS payloads (ivas/), built from the syntax of format-probes §14 to §17 with known field values; and the negative controls, each a change to one vector, every code of §12 given.
• vectors.json — every vector's capability, file, SHA-256, origin and outcome; the bindings; the facts; the probes and codes of §2.4 and §12; and the fetched LCEVC streams with what their encoder was asked for.
• provenance.json — every external vector's encoder, input and command, its decodes (the output's size and SHA-256 under each decoder) and, for JPEG XS, whether the decoders agree.
• fetch.sh — fetches five LCEVC streams by digest (format-probes §13.3).

Provenance. ext/ is written by tools/make_external.sh on the build host, from synthetic pictures tools/yuv.py writes (a ramp, a checker and a moving bar per plane; the second view of the stereo pair is the first shifted four columns), with these encoders and decoders, each at the revision the script pins and refuses to run without:

key	role	licence
vvc_enc, vvc_dec	an open H.266 encoder and decoder	BSD-3-Clause-Clear
jxs	an open JPEG XS encoder and decoder	BSD-2-Clause-Patent
jxs2	a second, independent JPEG XS decoder (a crate, without default features)	MIT
mvhevc	the MV-HEVC reference encoder and decoder of the joint video team	BSD-3-Clause
avs3_enc	an open AVS3 video encoder	BSD-4-Clause
avs3_dec	an open AVS3 video decoder	BSD-3-Clause

The script names each repository and revision. None of their code or data is in this corpus: only the bitstreams they wrote from the synthetic pictures. gen/ is written by wai_format_probe gen from fixed values and from ext/; no permissively licensed encoder writes LCEVC, MPEG-H 3D Audio or AVS3 audio, so those vectors are the generator's, laid out as format-probes §6, §7 and §9 read them, and are not decodable streams. The fetched LCEVC streams are an open LCEVC decoder's unit-test streams (BSD-3-Clause-Clear), written by an LCEVC encoder from a short test sequence; their licence notice names their owner, so they are fetched by SHA-256 and not committed.

The AV2, AC-4, IAMF and IVAS vectors are the generator's alone: no encoder or decoder of those formats was run, and no third-party file is in the corpus. Each is laid out as format-probes §14 to §17 read it, from the published texts those sections name; past the fields the probes read they hold fixed filler bytes, so they are not decodable streams. Their codec strings include the texts' own examples (ac-4.02.01.03, ac-4.00.00.03, iamf.003.003.Opus).

Measured (provenance.json, on the arm64 build host): every H.266, MV-HEVC, H.265 and AVS3 vector decodes with its decoder, and the base layer of the MV-HEVC vector decodes to the same bytes alone as within the two-view stream. Every JPEG XS codestream decodes to identical bytes along the first decoder's C path and its SIMD path. The second decoder gives the same bytes on five of the six codestreams; it does not implement the MLS.12 profile of the lossless one (it refuses its Ppih), and on a copy whose Ppih and Plev are 0 its output differs, while the first decoder's output equals that vector's source picture, byte for byte. No x86 decode was measured.

Checks.

• wai_format_probe verify . (from ../wai-rs, feature format_probe) and wai-rs/tests/format_probe.rs: every vector's digest and outcome, every binding and fact, the generated files byte for byte and no other file under gen/, and the extension's tables; with --fetched <dir>, the five fetched streams, every one required.

• format_probe_verify.py, written from the two extensions and sharing no code with wai-rs: probes every vector, binding, fact and fetched stream itself, and requires every mutated copy of the corpus to fail, among them a stated AV2 profile and level, AC-4, IAMF and IVAS codec strings, and header bytes changed with their digests restated:

  bash fetch.sh /tmp/lcevc && python3 -I format_probe_verify.py . --fetched /tmp/lcevc