wai: the headline decode tests fail on a missing input, and CI supplies their inputs #107

Merged

dcharlot merged 3 commits from wai/land-06-ci-headline-tests into main 2026-09-30 19:51:45 -04:00

Conversation 0 Commits 3 Files changed 27 +1631 -147

2026-09-29

Transaction Science

Merge wai/land-05-prior-nnc into wai/land-06-ci-headline-tests ...

Brings in, through the refreshed previous link, main's OER differential fuzzer (54e89b56) and its audit of the OER extension against both verifiers (860ead8c, documentation only), so this link is checked against current main.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

bfb89415b1

• View at this point in history

2026-09-29 10:16:50 -04:00

2026-09-28

Transaction Science

ci(wai): run the JPEG AI cross-platform pinned-output test in the whole-decode job ...

The pinned-output test (tests/jpegai_cross_platform.rs) needs the same
inputs the whole-decode job already fetches and checks, but no job ran it.
The job now runs it with the other two tests, requires it by name, and
triggers on changes to it. determinism-tiers §6 and the CHANGELOG say so.

Verified locally with the job's own script and the pinned inputs: all three
required tests ran and passed, the pinned pixel hash was checked, and no test
printed a skip banner.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

7c74fd3979

• View at this point in history

2026-09-28 18:42:16 -04:00

Transaction Science

wai: the headline decode tests fail on a missing input, and CI supplies their inputs ...

The tests behind WAI's whole-decode claims read inputs from outside the
repository and returned early — reporting a pass — when those inputs were
absent. No workflow supplied them, so the claims were not checked
continuously.

- WAI_REQUIRE_SOTA_VECTORS=1 turns a missing input into a failure. A path
  variable naming a missing or incomplete input fails with or without the
  switch. Local runs without either still skip, and every skip prints a
  banner saying the claim was not verified.
- wai-sota-vectors.yml regenerates the int_hyper vectors from the pretrained
  mbt2018_mean checkpoint (pinned by SHA-256, with a pinned Python stack) and
  runs int_hyper_synth and int_video_sota_keyframe with the switch set. The
  regenerated vectors identify one run, not one machine: the encoder side is
  float.
- wai-jpegai-e2e.yml fetches the pinned JPEG AI inputs from the public models
  root, checks each against a SHA-256 manifest, and runs the whole-decode and
  reconstruction-binding tests with the switch set, requiring each test by
  name and the pinned pixel hash.
- Both jobs fail on any skip banner, and trigger on the kernels and float
  helpers the decode imports.
- The int_hyper claims are stated as the checks CI runs, with its quality
  measured on the pinned image; JPEG AI cross-machine identity is stated as a
  measurement, not a registry-class claim. determinism-tiers §5–§6 record the
  jobs.

Neither workflow has yet run through the forge.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

648f407f09

• View at this point in history

2026-09-28 18:40:59 -04:00