wai-rs: fast integer decode kernels that give the same integer for every output #108

Merged

dcharlot merged 6 commits from wai/land-07-int-decode-kernels into main 2026-09-30 19:52:11 -04:00

Conversation 0 Commits 6 Files changed 46 +6565 -465

2026-09-29

Transaction Science

Merge wai/land-06-ci-headline-tests into wai/land-07-int-decode-kernels ...

Brings in, through the refreshed previous link, main's OER differential fuzzer (54e89b56) and its audit of the OER extension against both verifiers (860ead8c, documentation only), so this link is checked against current main.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

63e5b4e875

• View at this point in history

2026-09-29 10:17:29 -04:00

Transaction Science

Merge wai/land-05-prior-nnc into wai/land-06-ci-headline-tests ...

Brings in, through the refreshed previous link, main's OER differential fuzzer (54e89b56) and its audit of the OER extension against both verifiers (860ead8c, documentation only), so this link is checked against current main.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

bfb89415b1

• View at this point in history

2026-09-29 10:16:50 -04:00

2026-09-28

Transaction Science

wai-rs: hold the fast integer kernels to the JPEG AI decode path's math-library guard ...

jpegai_synthesis and int_transformer now run their loops through
int_kernels, so int_kernels is on the JPEG AI exact decode path. Every other
module on that path denies clippy's disallowed_methods (wai-rs/clippy.toml)
so that no platform math-library call can enter it; int_kernels now does too.
It calls none of the listed methods, so this changes no code, only makes the
guard cover the whole path it describes.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

1c2af088fb

• View at this point in history

2026-09-28 18:44:11 -04:00

Transaction Science

wai-rs: fast integer decode kernels that give the same integer for every output ...

WAI's integer decoders ran scalar, single-threaded loops, and no throughput
figure existed for them. Integer arithmetic is exact, so the work can be
re-partitioned across lanes and threads without changing any result, as long
as each output element's arithmetic and overflow behaviour are unchanged.

- src/int_kernels.rs holds the fast convolution, transposed-convolution,
  matmul and elementwise kernels. Each gives the same integer as the scalar
  reference for every output element. Threading uses std:🧵:scope and
  is not used on wasm32.
- A fast kernel runs only where its own arithmetic cannot overflow and the
  reference can size its output; operands outside that bound, and geometries
  where the padded input outgrows the reference's proportion, are handed to
  the reference kernel. With overflow checks on, the fast path panics where
  the reference panics; with them off, it returns the same integers.
- Equivalence is tested, not argued: every committed integer-decoder vector
  is decoded under the reference and under each fast policy against its
  frozen hash (tests/int_kernels_vectors.rs), edge geometry runs through the
  public decoders (tests/int_kernels_geometry.rs), and wai-conformance runs
  both in dev and release profiles on every leg. The Kani harnesses are
  unchanged and still verify.
- wai_decode_bench times reference and fast decodes interleaved in one
  process, best of N. research/decode-throughput.md records the measured
  figures, the machine and the method, and which benchmarked vectors are
  trained models; the largest gain is on the int_hyper synthesis topology
  (synthetic weights).

Co-Authored-By: Claude Opus 5.5 <[email protected]>

f27109f4d8

• View at this point in history

2026-09-28 18:43:42 -04:00

Transaction Science

ci(wai): run the JPEG AI cross-platform pinned-output test in the whole-decode job ...

The pinned-output test (tests/jpegai_cross_platform.rs) needs the same
inputs the whole-decode job already fetches and checks, but no job ran it.
The job now runs it with the other two tests, requires it by name, and
triggers on changes to it. determinism-tiers §6 and the CHANGELOG say so.

Verified locally with the job's own script and the pinned inputs: all three
required tests ran and passed, the pinned pixel hash was checked, and no test
printed a skip banner.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

7c74fd3979

• View at this point in history

2026-09-28 18:42:16 -04:00

Transaction Science

wai: the headline decode tests fail on a missing input, and CI supplies their inputs ...

The tests behind WAI's whole-decode claims read inputs from outside the
repository and returned early — reporting a pass — when those inputs were
absent. No workflow supplied them, so the claims were not checked
continuously.

- WAI_REQUIRE_SOTA_VECTORS=1 turns a missing input into a failure. A path
  variable naming a missing or incomplete input fails with or without the
  switch. Local runs without either still skip, and every skip prints a
  banner saying the claim was not verified.
- wai-sota-vectors.yml regenerates the int_hyper vectors from the pretrained
  mbt2018_mean checkpoint (pinned by SHA-256, with a pinned Python stack) and
  runs int_hyper_synth and int_video_sota_keyframe with the switch set. The
  regenerated vectors identify one run, not one machine: the encoder side is
  float.
- wai-jpegai-e2e.yml fetches the pinned JPEG AI inputs from the public models
  root, checks each against a SHA-256 manifest, and runs the whole-decode and
  reconstruction-binding tests with the switch set, requiring each test by
  name and the pinned pixel hash.
- Both jobs fail on any skip banner, and trigger on the kernels and float
  helpers the decode imports.
- The int_hyper claims are stated as the checks CI runs, with its quality
  measured on the pinned image; JPEG AI cross-machine identity is stated as a
  measurement, not a registry-class claim. determinism-tiers §5–§6 record the
  jobs.

Neither workflow has yet run through the forge.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

648f407f09

• View at this point in history

2026-09-28 18:40:59 -04:00