648f407f09
The tests behind WAI's whole-decode claims read inputs from outside the repository and returned early — reporting a pass — when those inputs were absent. No workflow supplied them, so the claims were not checked continuously. - WAI_REQUIRE_SOTA_VECTORS=1 turns a missing input into a failure. A path variable naming a missing or incomplete input fails with or without the switch. Local runs without either still skip, and every skip prints a banner saying the claim was not verified. - wai-sota-vectors.yml regenerates the int_hyper vectors from the pretrained mbt2018_mean checkpoint (pinned by SHA-256, with a pinned Python stack) and runs int_hyper_synth and int_video_sota_keyframe with the switch set. The regenerated vectors identify one run, not one machine: the encoder side is float. - wai-jpegai-e2e.yml fetches the pinned JPEG AI inputs from the public models root, checks each against a SHA-256 manifest, and runs the whole-decode and reconstruction-binding tests with the switch set, requiring each test by name and the pinned pixel hash. - Both jobs fail on any skip banner, and trigger on the kernels and float helpers the decode imports. - The int_hyper claims are stated as the checks CI runs, with its quality measured on the pinned image; JPEG AI cross-machine identity is stated as a measurement, not a registry-class claim. determinism-tiers §5–§6 record the jobs. Neither workflow has yet run through the forge. Co-Authored-By: Claude Opus 5.5 <[email protected]>
49 lines
2 KiB
Python
49 lines
2 KiB
Python
#!/usr/bin/env python3
|
|
"""Write the pinned source photograph the int_hyper integer-codec vectors (the
|
|
pretrained mbt2018_mean, exported to integers) are regenerated from.
|
|
|
|
python tools/wai_sota_source_image.py <out.png>
|
|
|
|
The image is a 256x256 crop of `skimage.data.astronaut()` — a photograph that
|
|
ships inside the scikit-image wheel and is documented there as public domain
|
|
(NASA; "No known copyright restrictions, released into the public domain"). It
|
|
is pinned by the SHA-256 of the cropped RGB bytes, so a different image can never
|
|
be substituted silently: a mismatch is an error, not a warning.
|
|
|
|
A real photograph, rather than a synthetic pattern, is what makes the quality
|
|
check meaningful: the integer decode is checked against the same model's float
|
|
decode on natural-image content, and must cost at most 1 dB there
|
|
(tests/int_hyper_synth.rs). That bar is relative, on this one image; it says
|
|
nothing about how the model compares with other codecs, or about other images.
|
|
The 42.6 dB development figure for this codec was taken on a different
|
|
photograph that is not in this repository.
|
|
"""
|
|
import hashlib
|
|
import sys
|
|
|
|
import numpy as np
|
|
from PIL import Image
|
|
from skimage import data
|
|
|
|
CROP = (slice(64, 320), slice(128, 384)) # rows, cols of the 512x512 original
|
|
SHA256 = "29cb3aee1a55f9585474d056b65edb0aaebbb80371ba200e85030d63665c19d6"
|
|
|
|
|
|
def main(out: str) -> int:
|
|
rgb = np.ascontiguousarray(data.astronaut()[CROP])
|
|
got = hashlib.sha256(rgb.tobytes()).hexdigest()
|
|
if rgb.shape != (256, 256, 3) or got != SHA256:
|
|
print(f"error: source crop is {rgb.shape} sha256 {got}; pinned (256, 256, 3) {SHA256}. "
|
|
"The scikit-image sample changed — pin the new hash only after reviewing the image.",
|
|
file=sys.stderr)
|
|
return 1
|
|
Image.fromarray(rgb).save(out)
|
|
print(f"wrote {out}: 256x256 RGB, sha256(rgb) {got}")
|
|
return 0
|
|
|
|
|
|
if __name__ == "__main__":
|
|
if len(sys.argv) != 2:
|
|
print(__doc__, file=sys.stderr)
|
|
raise SystemExit(2)
|
|
raise SystemExit(main(sys.argv[1]))
|