648f407f09
The tests behind WAI's whole-decode claims read inputs from outside the repository and returned early — reporting a pass — when those inputs were absent. No workflow supplied them, so the claims were not checked continuously. - WAI_REQUIRE_SOTA_VECTORS=1 turns a missing input into a failure. A path variable naming a missing or incomplete input fails with or without the switch. Local runs without either still skip, and every skip prints a banner saying the claim was not verified. - wai-sota-vectors.yml regenerates the int_hyper vectors from the pretrained mbt2018_mean checkpoint (pinned by SHA-256, with a pinned Python stack) and runs int_hyper_synth and int_video_sota_keyframe with the switch set. The regenerated vectors identify one run, not one machine: the encoder side is float. - wai-jpegai-e2e.yml fetches the pinned JPEG AI inputs from the public models root, checks each against a SHA-256 manifest, and runs the whole-decode and reconstruction-binding tests with the switch set, requiring each test by name and the pinned pixel hash. - Both jobs fail on any skip banner, and trigger on the kernels and float helpers the decode imports. - The int_hyper claims are stated as the checks CI runs, with its quality measured on the pinned image; JPEG AI cross-machine identity is stated as a measurement, not a registry-class claim. determinism-tiers §5–§6 record the jobs. Neither workflow has yet run through the forge. Co-Authored-By: Claude Opus 5.5 <[email protected]>
88 lines
3.9 KiB
Rust
88 lines
3.9 KiB
Rust
//! The LIVE JPEG AI decode: `jpegai::decode(codestream, model)` → RGB, with no
|
||
//! per-image schedule fixtures. Every per-image quantity is derived from the
|
||
//! bitstream (the residual mask `sigma_skip > 382`, the σ surfaces, indexes,
|
||
//! rvs); the model bundle supplies only checkpoint constants. Compared against
|
||
//! the reference decoder's PNG — the same bar as the staged `jpegai_e2e` test,
|
||
//! but through the dispatchable library entry point.
|
||
//!
|
||
//! This is the whole-decode test behind the dispatchable `wai.image.jpegai` ONNX
|
||
//! path, so a missing input follows the headline-claim rule (support/
|
||
//! require_vectors.rs): without its inputs it skips loudly, and with
|
||
//! `WAI_REQUIRE_SOTA_VECTORS=1` it fails. Its inputs — the float model bundle, the
|
||
//! traced nets under `model16/`, the extracted model constants and the reference
|
||
//! PNG — exist only in the local JPEG-AI asset directory
|
||
//! (`wai/jpegai-vm-assets`, exported from the ISO/IEC 6048-1 reference software;
|
||
//! not in the repository), and the test also needs the `neural` (ONNX Runtime) and
|
||
//! `codecs` features. No CI job runs it; it is verified only on a machine holding
|
||
//! those assets. `WAI_JPEGAI_ASSETS` points it at the directory from elsewhere; a
|
||
//! directory it names must exist and hold every input, or the test fails with or
|
||
//! without the switch. Unset or empty, the default below is used, and only that
|
||
//! default may be absent and skip.
|
||
#![cfg(all(feature = "neural", feature = "codecs"))]
|
||
|
||
#[path = "support/require_vectors.rs"]
|
||
mod require_vectors;
|
||
|
||
use require_vectors::InputDir;
|
||
use wai::neural::jpegai::{decode, JpegAiModel};
|
||
|
||
const TEST: &str = "jpegai_live::jpegai_live_decode_matches_reference";
|
||
const PRODUCE: &str = "the local JPEG-AI asset directory wai/jpegai-vm-assets (exported from the \
|
||
ISO/IEC 6048-1 reference software; not in the repository), or WAI_JPEGAI_ASSETS=<dir>";
|
||
|
||
/// `WAI_JPEGAI_ASSETS`, else `../jpegai-vm-assets` beside the crate. A variable that
|
||
/// names a directory that does not exist, or one missing an input, is a
|
||
/// misconfigured run and fails.
|
||
fn assets() -> InputDir {
|
||
require_vectors::env_dir(TEST, "WAI_JPEGAI_ASSETS", "../jpegai-vm-assets", PRODUCE)
|
||
}
|
||
|
||
#[test]
|
||
fn jpegai_live_decode_matches_reference() {
|
||
let a = assets();
|
||
// Needs the model bundle (incl. the extracted model-constant files) + the
|
||
// 256×256 traced nets + the reference PNG.
|
||
let missing: Vec<&str> = [
|
||
"e2e/test256.bin",
|
||
"e2e/test256_dec.png",
|
||
"jpegai_factorized_cdfs.json",
|
||
"jpegai_scaler_vec.json",
|
||
"jpegai_dequant_consts.json",
|
||
"model16/CCS_SGMM",
|
||
]
|
||
.into_iter()
|
||
.filter(|need| !a.join(need).exists())
|
||
.collect();
|
||
if !missing.is_empty() {
|
||
a.absent(
|
||
TEST,
|
||
&format!("{} not in {}", missing.join(", "), a.display()),
|
||
PRODUCE,
|
||
);
|
||
return;
|
||
}
|
||
|
||
let model = JpegAiModel::load(&a).expect("load jpeg-ai model bundle");
|
||
let codestream = std::fs::read(a.join("e2e/test256.bin")).unwrap();
|
||
let img = decode(&codestream, &model).expect("jpeg-ai decode");
|
||
assert_eq!((img.width, img.height), (256, 256));
|
||
assert_eq!(img.rgb.len(), 256 * 256 * 3);
|
||
|
||
let refimg = image::open(a.join("e2e/test256_dec.png")).unwrap().to_rgb8();
|
||
let mut mismatched = 0usize;
|
||
let mut maxdiff = 0i32;
|
||
for i in 0..256 * 256 {
|
||
let px = refimg.get_pixel((i % 256) as u32, (i / 256) as u32);
|
||
for c in 0..3 {
|
||
let d = (img.rgb[i * 3 + c] as i32 - px[c] as i32).abs();
|
||
if d > 0 {
|
||
mismatched += 1;
|
||
}
|
||
maxdiff = maxdiff.max(d);
|
||
}
|
||
}
|
||
let total = 3 * 256 * 256;
|
||
println!("live jpeg-ai decode: {mismatched}/{total} samples differ, max diff {maxdiff}");
|
||
assert!(maxdiff <= 1, "pixel deviation beyond rounding: {maxdiff}");
|
||
assert!(mismatched * 1000 <= total, "more than 0.1% mismatched: {mismatched}/{total}");
|
||
}
|