648f407f09
The tests behind WAI's whole-decode claims read inputs from outside the repository and returned early — reporting a pass — when those inputs were absent. No workflow supplied them, so the claims were not checked continuously. - WAI_REQUIRE_SOTA_VECTORS=1 turns a missing input into a failure. A path variable naming a missing or incomplete input fails with or without the switch. Local runs without either still skip, and every skip prints a banner saying the claim was not verified. - wai-sota-vectors.yml regenerates the int_hyper vectors from the pretrained mbt2018_mean checkpoint (pinned by SHA-256, with a pinned Python stack) and runs int_hyper_synth and int_video_sota_keyframe with the switch set. The regenerated vectors identify one run, not one machine: the encoder side is float. - wai-jpegai-e2e.yml fetches the pinned JPEG AI inputs from the public models root, checks each against a SHA-256 manifest, and runs the whole-decode and reconstruction-binding tests with the switch set, requiring each test by name and the pinned pixel hash. - Both jobs fail on any skip banner, and trigger on the kernels and float helpers the decode imports. - The int_hyper claims are stated as the checks CI runs, with its quality measured on the pinned image; JPEG AI cross-machine identity is stated as a measurement, not a registry-class claim. determinism-tiers §5–§6 record the jobs. Neither workflow has yet run through the forge. Co-Authored-By: Claude Opus 5.5 <[email protected]>
54 lines
2 KiB
Shell
Executable file
54 lines
2 KiB
Shell
Executable file
#!/usr/bin/env bash
|
|
# Fetch the JPEG-AI whole-decoder test inputs from WAI's content-versioned models
|
|
# root and check every byte against the SHA-256 manifest committed next to this
|
|
# script (tools/jpegai-e2e-v1.sha256).
|
|
#
|
|
# tools/wai_jpegai_fetch_e2e_inputs.sh [out-dir] # default /tmp/jpegai_e2e
|
|
#
|
|
# Then run the tests that read them (tests/jpegai_e2e_full.rs,
|
|
# tests/jpegai_reconstruction_binding.rs) with
|
|
# WAI_JPEGAI_CODESTREAM=<out-dir>/test256.bin
|
|
# WAI_JPEGAI_WEIGHTS_DIR=<out-dir>
|
|
# WAI_JPEGAI_NORM=<out-dir>/norm.json
|
|
#
|
|
# What is fetched: the eleven integer/PTQ weight files the decoder runs
|
|
# ({hs,hyp,synth}_{y,uv}.json, mcm{0..3}_y.json, icci.json), norm.json (normative
|
|
# tables plus the reference decoder's RGB output for the test picture) and
|
|
# test256.bin (the test codestream). They were extracted by
|
|
# tools/wai_jpegai_build_e2e_inputs.py from the JPEG-AI reference-software assets,
|
|
# which are not in this repository; the models root publishes the extraction under
|
|
# an immutable versioned path (jpegai/v1/). The manifest pins those exact bytes, so
|
|
# a changed or truncated file at the URL fails here instead of changing what the
|
|
# tests check.
|
|
#
|
|
# WAI_JPEGAI_INPUTS_URL overrides the base URL (a mirror holding the same files).
|
|
set -euo pipefail
|
|
|
|
here="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
|
manifest="$here/jpegai-e2e-v1.sha256"
|
|
base="${WAI_JPEGAI_INPUTS_URL:-https://wai.transaction.science/models/jpegai/v1}"
|
|
out="${1:-/tmp/jpegai_e2e}"
|
|
|
|
mkdir -p "$out"
|
|
out="$(cd "$out" && pwd)"
|
|
|
|
if command -v sha256sum >/dev/null 2>&1; then
|
|
sha() { sha256sum "$@"; }
|
|
else
|
|
sha() { shasum -a 256 "$@"; }
|
|
fi
|
|
|
|
while read -r want name; do
|
|
[ -n "$name" ] || continue
|
|
curl --proto '=https' --tlsv1.2 -fsSL --retry 3 -o "$out/$name.part" "$base/$name"
|
|
got="$(sha "$out/$name.part" | cut -d' ' -f1)"
|
|
if [ "$got" != "$want" ]; then
|
|
echo "error: $base/$name has sha256 $got, pinned $want" >&2
|
|
rm -f "$out/$name.part"
|
|
exit 1
|
|
fi
|
|
mv "$out/$name.part" "$out/$name"
|
|
echo "ok $want $name"
|
|
done < "$manifest"
|
|
|
|
echo "JPEG-AI e2e inputs verified in $out"
|