Each *-conformance/README.md now has a line that begins **Provenance.**: the generator that writes the corpus and its input, and for an input not generated here its source and licence. OER/2's corpus is settled and left as it is. byte-exact-conformance and quantum-ops-conformance get a README. wai-rs/tests/vectors/README.md gives a row per vector directory. Where the staged-delivery, staged-measurement and telemetry READMEs called their content "synthetic", that sentence is now their provenance line. Two vectors are recorded as not generated, from the provenance audit: int_video's video_real (a pan over a third-party picture) and neural-conformance's photo (a third-party picture, with a decoder trained on unrecorded images). Their lines say so and that they are to be replaced; the photo bullet no longer names the picture's source product. tests/corpus_provenance.rs requires the line in every corpus and a row for every vector directory; CI runs it with the carriage tests. CONTRIBUTING states the rule: corpus content is generated here, or recorded with a licence that permits redistribution, and a word like "synthetic" is not a provenance line. Co-Authored-By: Claude Opus 5.5 <[email protected]>
82 lines
5.2 KiB
Markdown
82 lines
5.2 KiB
Markdown
# int-payload-conformance — the integer payloads
|
|
|
|
Vectors for the payload formats SPEC §5 pins to the WAI-native integer codecs
|
|
([`extensions/integer-payloads.md`](../extensions/integer-payloads.md)): `WIV1` for
|
|
`wai.video.int_motion` (keyframe 0) and `wai.video.int_hyper` (keyframe 1), `WIH1`
|
|
for `wai.neural.int_hyper`, `WIA1` for `wai.audio.int_codec` and `WIS1` for
|
|
`wai.splat.int_codec`.
|
|
|
|
**Provenance.** Generated here: `wai_int_payload_vectors gen` converts the vectors under `../wai-rs/tests/vectors/` and changes them for the refusal fixtures; `tests/vectors/README.md` states each vector's provenance. One payload indexed here is not generated content: `video_real` (`int_video`) codes a third-party image (see there).
|
|
|
|
**Code-equivalence.** A conforming sink decodes every payload to the output digests
|
|
its vector states, and refuses every fixture with the code it states, from the
|
|
extension's closed list (§8).
|
|
|
|
- `vectors.json` — the index. For each payload: its capability, its path, its
|
|
SHA-256, the JSON it is converted from, the parameter set it decodes against (each
|
|
member's name in the set, and the committed file that holds it), and the
|
|
`sha256.json` sidecar that states its decoded output's SHA-256. For each refusal
|
|
fixture: its capability, its path, its code, and whether parsing alone refuses it
|
|
(`parse`) or the decode does (`decode`). Paths are relative to `wai/`.
|
|
- The payloads sit beside their JSON sources under
|
|
[`../wai-rs/tests/vectors/`](../wai-rs/tests/vectors/): `int_video/*.wiv1`,
|
|
`int_hyper/tiny_bitstream.wih1`, `int_audio/audio_bitstream.wia1` and
|
|
`int_splat/splat.wis1`. Their sidecars are the ones the JSON forms' decodes are
|
|
checked against: a payload and its JSON source decode to the same bytes.
|
|
- `refusals/*.bin` — each a committed payload changed in one place (a byte, a
|
|
field, a truncation, a word appended to a stream), so that it is refused with one
|
|
code, or, for `wiv1_end_state` and `wiv1_one_frame_end_state`, a small clip written
|
|
to end its stream in a state other than `2^31`. The fixtures named `*_order_*`
|
|
break two rules at once and pin which one a sink reports (the extension's §2 and
|
|
§7), and so do the `wiv1_geometry_flip_*` fixtures: `wiv1_geometry` with one byte
|
|
of its keyframe's last stream words changed, refused for the keyframe's size before
|
|
that stream is read. The fixtures for `bad_magic` are the JSON sources themselves: a tool's JSON
|
|
output is a converter input, never a payload.
|
|
- `sets/` — a parameter-set member a fixture decodes against in place of a committed
|
|
one: `entropy_median_max.json` is the tiny conditional codec's tables with one
|
|
median at `i64::MAX`.
|
|
|
|
Files are compared as bytes; `.gitattributes` turns off line-ending conversion.
|
|
|
|
**Generation.** Every file here, and every payload under `tests/vectors/`, is
|
|
written by one generator from the committed JSON alone, so it can be rebuilt and
|
|
compared:
|
|
|
|
cd wai/wai-rs
|
|
cargo run --no-default-features --features video_int,audio_int,splat_int \
|
|
--bin wai_int_payload_vectors -- gen ../int-payload-conformance
|
|
|
|
**Verifiers.**
|
|
|
|
- `wai_int_payload_vectors verify ../int-payload-conformance` (wai-rs) regenerates
|
|
every file and requires it to equal the committed one byte for byte, decodes every
|
|
payload through `int_payload::decode_int` to its sidecar's SHA-256, and requires
|
|
every fixture to be refused with its code at its layer, and by
|
|
`int_payload::declared_cost` with the same code unless the decode's refusal comes
|
|
from a stream or a network run. `tests/int_payload.rs` runs
|
|
the fixtures again in both build profiles, refuses every strict prefix of every
|
|
payload, and runs a sweep of single-byte changes in which nothing may panic;
|
|
`tests/int_payload_preflight.rs` checks that a header past the caps or past what the
|
|
parameter set can fit is refused before anything sized from it is allocated, and
|
|
that a payload whose streams cannot carry what its header declares is refused at
|
|
its first exhausted symbol, having allocated only what it decoded.
|
|
- `int_payload_verify.py`, Python standard library only and sharing no code with
|
|
wai-rs, written from the extension: it parses every payload with the extension's
|
|
rules and check order, converts every JSON source independently and requires the
|
|
committed bytes, decodes the `WIV1` payloads with keyframe 0 in full (the rANS
|
|
decoder with its bypass escape, its stop at the first exhausted symbol and its
|
|
canonical end, whole-pel and half-pel motion compensation) to their sidecars'
|
|
SHA-256, and checks every fixture: a `parse` fixture's code, and that a `decode`
|
|
fixture parses (a `WIV1` keyframe-0 one is decoded and must be refused with its
|
|
code).
|
|
|
|
python3 int-payload-conformance/int_payload_verify.py int-payload-conformance
|
|
|
|
**Not decoded independently.** The Python verifier does not decode `WIH1`, `WIV1`
|
|
with keyframe 1 (its keyframe is a `WIH1`), `WIA1` or `WIS1`: their decodes run
|
|
integer networks it does not implement. Their outputs are checked by wai-rs against
|
|
the sidecars above, and the `WIV1` and `WIH1` payloads also by the
|
|
cross-architecture goldens in
|
|
[`../byte-exact-conformance/`](../byte-exact-conformance/). For their `decode`
|
|
fixtures the Python verifier checks that they parse; their codes are checked by
|
|
wai-rs.
|