Transaction-Science/open-standards
Watch
1
0
Fork
You've already forked open-standards
 0
Code Issues Pull requests Projects Releases 15 Packages Wiki Activity Actions

PERF1: int_hyper's integer decode on NEON, AVX2 and AVX-512 narrow kernels — the same bytes, re-measured (stacked on #453) #467

Closed
dcharlot wants to merge 0 commits from wai/int-decode-fast into main
pull from: wai/int-decode-fast
merge into: Transaction-Science:main
Conversation 0 Commits - Files changed -
dcharlot commented 2026-10-25 14:34:58 -04:00
Owner
Copy link

Stacked on #453 (DET1, branch wai/determinism-cost, head 4863036d). The forge tool opens against main only, so this diff also shows #453's seven commits; the history above them is linear (no merges). Land #453 first, or both together.

DET1 measured that wai.neural.int_hyper's integer decode gives the same bytes everywhere but cost far more time and energy than its float model on the same CPU. This PR makes the integer decode cheap without changing one output byte, and re-measures it with DET1's own procedure, tools and pictures, in DET1's record.

Results (DET1's procedure: 8 pictures, 7 rounds, 5 s idle brackets; medians, ranges in the record)

x86_64 host A (16-core SoC with AVX-512; package energy, powercap package-0, class OnChipCounter, scope the whole package, the round's marginal over its idle brackets; every round of every run read clean):

quality integer, the record's build (re-run) integer, this PR (AVX-512 VNNI) integer, this PR forced to AVX2 float, re-run beside them
q2 345.8 ms, 42.25 J 92.9 ms, 7.31 J 101.6 ms, 9.27 J 132.2 ms, 9.26 J
q6 728.0 ms, 92.14 J 144.8 ms, 12.43 J 166.3 ms, 16.46 J 210.7 ms, 14.03 J

On host A the integer decode now takes less time and less package energy than the float decode, at both qualities. (The record's own runs of 10-22: integer 347.3 ms/42.37 J and 729.6 ms/92.5 J, float 123.9 ms/9.07 J and 191.5 ms/13.73 J; the re-runs agree within a few tenths of a joule.)

x86_64 host B (14-core laptop CPU with AVX2 and AVX-VNNI, no AVX-512; package energy, OnChipCounter, package scope; the box is also a CI runner): integer only, the float stack was not set up there again.

quality integer, the record's build integer, this PR (AVX2 VNNI)
q2 714.0 ms, 43.84 J 170.2 ms, 11.56 J (2 of 7 rounds relative-only, so not "publishable" by the record's rule)
q6 1582.1 ms, 84.92 J 294.3 ms, 19.97 J

Build host (arm64 desktop SoC; the decoder's own CPU energy, per-task count ri_energy_nj, class OnChipCounter, scope this process's CPU only; load average 32–60 from other work throughout):

quality integer, the record's build integer, this PR (NEON narrow) float, re-run
q2 4.76 J 2.96 J 0.92 J
q6 11.01 J 6.43 J 1.28 J

Said plainly: on the build host the integer decode still costs more than the float decode, about five times at quality 6 and three times at quality 2, by the per-task count, which may also leave out the float library's matrix unit (record §5). The build host's times swing with its load by more than the kernels change them (the -after medians are above the -before ones with overlapping ranges), so only its per-task counts are compared.

Bytes: the 192 payloads decoded again on every platform with every kernel set it runs, 15 new runs: 0 of 2880 differ from the record (record total now 0 of 4608 over 24 runs). Platforms × sets: build host CPU (neon all cores, neon one core, wide), its GPU (Metal), wasm32-wasip2 there and on host A (wide, the only set on wasm), host A CPU (avx512-vnni, avx512, avx2-vnni, avx2, wide) and GPU (Vulkan), host B CPU (avx2-vnni, avx2, wide). Host B's GPU was not reached (its driver was not loaded after a reboot), so its GPU decode is the record's only. The GPU decodes run both graphs on the device (gpu=2 cpu=0 on every payload), so they are unchanged by the kernel sets; the runs confirm it.

Where the time went (profiled first)

  • arm64, one thread, q6 (sample): convolution 74 %, IGDN 20 % (+3 % integer square root), requantisation epilogue 2 %; everything else small.
  • x86_64 host A (perf): the same shape, but the 32×32→64 kernels were portable Rust there (no SIMD at the x86-64 baseline): one-thread q6 9.7 s, the big transposed convolution 4.3 s for 22.6 G multiply-accumulates.
  • Operand ranges, measured per layer: weights |w| ≤ 2047 (12-bit), activations from 7 463 to 316 606; the big layers' inputs fit 16 bits.
  • After the kernels, on 32 threads the critical path is serial: the interpreter's per-tensor range checks (wai-ops, one thread, no_std), freeing and first-touching 150 MB tensors, the rANS decode, re-parsing the set's models and entropy.json on every decode, and the harness's SHA-256 of each output.

What changed

  • Kernel sets (int_kernels::Isa; Policy::fast_isa; isas(); WAI_INT_ISA env): wide (the previous kernels, unchanged), neon, avx2, avx2-vnni, avx512 (F, BW, DQ), avx512-vnni. Detection at run time; a set the processor lacks is never executed (it becomes wide). wasm and other targets have no narrow set.
  • Narrow convolutions (int_kernels/narrow.rs, int_kernels/x86.rs): 16-bit operands; an input past 2¹⁵−1 split into two 15-bit limbs; 32-bit partial sums over blocks of kblk taps with kblk·(products per tap)·max|w|·max|operand| ≤ i32::MAX; each block sign-extended, scaled by its limb and added into 64-bit sums, all within Σ terms·max|w|·max|operand|·2^shift ≤ i64::MAX (the plan refuses otherwise and the wide kernels run). x86: two channels per 32-bit word, vpmaddwd/vpdpwssd; aarch64: smlal by weight lane. Tap blocks are outermost so a block's weights and input lines stay in L1 while the row is swept.
  • IGDN: x86 SIMD norms (vpmuldq exact from i32 operands, arithmetic shift — AVX2 restores the sign bits itself), NEON fuses shift and add at gf = 16 (ssra), AVX-512 forms roots and outputs in lanes; every root is seeded from the IEEE-754 square root and corrected in integers (the correction alone decides the result).
  • int_ops::Host: requantisation in i64 with checked steps (falls back to the interpreter's i128 on any overflow); rescale, clamp, relu, leaky on the workers; max|x| read in parallel and once per IGDN.
  • Tests: testkit::FAST and the decoder-level POLICIES now include every kernel set the processor runs (equivalence_policies()); new tests against an exact sum written out from the definition (random geometry, grouped/odd channels, every tail width), the block edges (|w|=|x|=2¹⁵−1, one tap more would overflow i32), the limb edges, the seeded root (including deliberately wrong seeds), the AVX-512 root/product lanes, the i64 requantisation, the element-wise operators, a declared-kernel-sets test, and tests/int_ops_conformance.rs (every graph vector of ops-conformance/vectors and graphs through the fast kernels under every policy and set, held to the stated integers or refusal).
  • CI: the kernel steps run int_ops_conformance; a new int-kernels-amd job on the linux-amd host runs the kernel suite in both profiles with every x86_64 set, AVX-512 included, and fails if the declared sets are not found.
  • Record and text: research/determinism-cost.md §4.6 (new) and brief, §3, §5; determinism-tiers §7.1 bullets; runs/ scripts and README; CHANGELOG; a pointer in decode-throughput.md. DET1's readings are byte-for-byte unchanged in the record (checked field by field); only the totals moved.

Decisions

  • Exactness by bounds, not by order. The wide kernels' rule was "same terms, same order, same accumulator". The narrow kernels regroup terms; they are exact because no value at any step leaves its type, proved from the actual operand maxima before the kernel runs. The int_kernels docs now state both rules.
  • The interpreter (wai-ops) is untouched. Its serial range checks are now the largest single cost on 32 threads; speeding them needs a change to the reference crate, left out.
  • Float seed for the integer root. Safe for any seed (corrected and, on AVX-512, verified in integers); the wide set keeps the integer routine.
  • Labels/tags added to the same record: -before (the record's build re-run), -after, -after-avx2, -again (float re-run); decode tags <platform>-isa-<set>.

Checks

F=neural_int,video_int,audio_int,world_int,splat_int,generate,score,staged,stage_measure,legacy_int,i32_forms; T= the 21 test targets of the CI kernel steps plus int_ops_conformance. Host A = x86_64 with AVX-512 (every x86_64 set), build host = arm64 (NEON).

check where command result
kernel suite, overflow checks on host A, all 5 x86_64 sets (WAI_INT_ISAS_EXPECT asserted) cargo test --locked --no-default-features --features $F --lib $T lib 490 passed; every target ok
kernel suite, release host A same with --release all ok
kernel suite, overflow checks on and release build host (wide, neon) same, both profiles lib 490 passed; every target ok
ops corpus through the fast kernels both hosts, every set; and wasm32-wasip2 (wasmtime 50) --test int_ops_conformance 1 passed (every graph vector of vectors/, graphs/)
ops corpus, CPU build host wai-ops: cargo test --locked --release; wai-verify vectors verify . all ok; "ops-conformance: every file verifies"
ops corpus, Wasm build host wai-ops tests --target wasm32-wasip2 under wasmtime; wai-verify.wasm vectors verify all ok; every file verifies
ops corpus, Metal build host wai-ops-gpu: cargo test --locked --release (WAI_OPS_GPU_EXPECT=gpu-metal-0) adapters as declared; corpus mismatches=0; mutants killed 18 of 18
device vs interpreter with the fast kernels behind it build host, Metal wai-rs --test ops_gpu int_hyper, video int_hyper, audio, splat, int_synth, mlicpp, world: graphs_differ=0
DET1's 1,728 decodes and more 3 hosts wai_det_cost decode of all 192 payloads per platform × set 15 new runs, 2880 decodes, 0 differ from the record
DET1's verifier (its CI step) local python3 -I tools/wai_determinism_cost_verify.py record.json determinism-cost.md determinism-tiers.md ok: 32 timed runs, 582 quoted figures match
det_cost_bin, int_hyper_synth host A cargo test --release --features neural_int,video_int,legacy_int --test det_cost_bin --test int_hyper_synth ok
neural tests build host --features ffi,neural,video_int --test neural_cost_limit --test neural_native_decode --test ffi_neural_pin ok (3, 38, 4)
decode symbols host A --test jpegai_decode_symbols no forbidden math-library function
byte-exact goldens both hosts byte-exact-conformance: cargo test --locked --release ok (112 x86_64, 111 arm64)
wai-verify native build + tests both hosts cargo build/test --release --features cli ok (the native build was broken by the first layout; fixed in 986edfea)
Kani compiles the change build host wai-verify: cargo kani --lib --harness int_transform::kani_layernorm_is_total --harness int_transform::kani_softmax_singleton_is_certain 2 verified
wai-web wasm32 and native builds build host cargo build --locked --release [--target wasm32-unknown-unknown] ok
clippy, no new notes both hosts cargo clippy --locked --all-targets --features $F (+ neural_int,ops_gpu on arm64), notes in touched files against the base none in touched files
repo checks local check-tests-in-ci, workflow-concurrency, spec-refs, wai-section-refs, no-vanity-metrics, public-text self-test, comparative-prose, orphan-sources; YAML parse all exit 0
name check local check-public-names.sh on the 36 changed files; commit messages against the patterns clean

Falsifications (committed tree, one mutation each, then restored)

mutation (one change) kernel set caught by result
F1 AVX-512 masked tail drops one lane avx512* narrow_convolutions_give_the_exact_sums; int_ops_conformance; and the real decodes: a mutated wai_det_cost on the 48 q2/q6 payloads exit 101, 101; 48 of 48 decodes differ from the record
F2 a 32-bit block one tap longer than its bound all narrow narrow_blocks_flush_before_any_partial_sum_leaves_i32 101 (x86_64), 101 (arm64)
F3 NEON flush reads the low lanes for the high half neon narrow_convolutions_give_the_exact_sums 101
F4 NEON fused IGDN shift made rounding in one lane (vsraq→vrsraq) neon first run: not caught (exit 0; a norm a unit off seldom moves its root). Added the_simd_norms_are_the_scalar_norms (777dd239) 101
F5 AVX2 IGDN loses the sign bits of negative terms avx2* igdn_equals_reference_in_every_mode (signed narrow case) 101
F6 i64 half-even requantisation rounds up at the half all the_i64_requantisation_is_the_interpreters 101
F7 the seeded root without its upward correction narrow first run: not caught (exit 0; a correct square root never needs it). Test now seeds wrong seeds (c061fbb0) 101 (x86_64), 101 (arm64)
F8 AVX-512 IGDN output shifted by frac + 1 avx512* the_avx512_roots_and_products_are_the_scalar_ones 101
F9 the high limb weighted 2^14 all narrow narrow_convolutions_give_the_exact_sums 101 (x86_64); arm64: the first loop run passed because its snapshot did not carry the mutation (see below), the rerun 101

A tooling note for the lead (F9 on arm64): the remote-build wrapper snapshots the worktree with a copied index. An edit that keeps a file's size, made in the same second as the git checkout that restored it, keeps the index's stat match, so git add -A skips it and the snapshot is the unmutated tree. The loop's F9 edit (15 → 14) was such an edit; F2's edit of the same file before it had changed the size. Rerun with a gap, the mutation reached the build and was caught. Same-size mutations need a pause (or git update-index --really-refresh) before the wrapper runs.

Not done / open

  • The build host's integer decode still costs more than its float decode (above). Further NEON work (the tight smlal loop is already 16 per tap) or a GPU energy figure on that host are the levers; the GPU rail did not read live in DET1's brackets.
  • Host B: no float re-run, no GPU run this time.
  • Serial costs left on the critical path: the interpreter's range checks, tensor allocation and freeing (page faults), rANS, and re-parsing the set's models and entropy tables per decode (int_payload caches only the lowered graphs, by design).
  • The AVX-512 kernels run in CI only on the linux-amd host job added here; the other legs cover NEON and AVX2/AVX-VNNI.

🤖 Generated with Claude Code

Closed: Landed on main through merge train E (#499, merge bf39ec9a); verified by content: this PR's head is in main

**Stacked on #453 (DET1, branch `wai/determinism-cost`, head 4863036d).** The forge tool opens against `main` only, so this diff also shows #453's seven commits; the history above them is linear (no merges). Land #453 first, or both together. DET1 measured that `wai.neural.int_hyper`'s integer decode gives the same bytes everywhere but cost far more time and energy than its float model on the same CPU. This PR makes the integer decode cheap without changing one output byte, and re-measures it with DET1's own procedure, tools and pictures, in DET1's record. ## Results (DET1's procedure: 8 pictures, 7 rounds, 5 s idle brackets; medians, ranges in the record) **x86_64 host A** (16-core SoC with AVX-512; package energy, powercap `package-0`, class OnChipCounter, scope the whole package, the round's marginal over its idle brackets; every round of every run read clean): | quality | integer, the record's build (re-run) | integer, this PR (AVX-512 VNNI) | integer, this PR forced to AVX2 | float, re-run beside them | |---|---|---|---|---| | q2 | 345.8 ms, 42.25 J | **92.9 ms, 7.31 J** | 101.6 ms, 9.27 J | 132.2 ms, 9.26 J | | q6 | 728.0 ms, 92.14 J | **144.8 ms, 12.43 J** | 166.3 ms, 16.46 J | 210.7 ms, 14.03 J | On host A the integer decode now takes less time and less package energy than the float decode, at both qualities. (The record's own runs of 10-22: integer 347.3 ms/42.37 J and 729.6 ms/92.5 J, float 123.9 ms/9.07 J and 191.5 ms/13.73 J; the re-runs agree within a few tenths of a joule.) **x86_64 host B** (14-core laptop CPU with AVX2 and AVX-VNNI, no AVX-512; package energy, OnChipCounter, package scope; the box is also a CI runner): integer only, the float stack was not set up there again. | quality | integer, the record's build | integer, this PR (AVX2 VNNI) | |---|---|---| | q2 | 714.0 ms, 43.84 J | 170.2 ms, 11.56 J (2 of 7 rounds relative-only, so not "publishable" by the record's rule) | | q6 | 1582.1 ms, 84.92 J | 294.3 ms, 19.97 J | **Build host** (arm64 desktop SoC; the decoder's own CPU energy, per-task count `ri_energy_nj`, class OnChipCounter, scope this process's CPU only; load average 32–60 from other work throughout): | quality | integer, the record's build | integer, this PR (NEON narrow) | float, re-run | |---|---|---|---| | q2 | 4.76 J | 2.96 J | 0.92 J | | q6 | 11.01 J | 6.43 J | 1.28 J | **Said plainly: on the build host the integer decode still costs more than the float decode, about five times at quality 6 and three times at quality 2, by the per-task count**, which may also leave out the float library's matrix unit (record §5). The build host's times swing with its load by more than the kernels change them (the `-after` medians are above the `-before` ones with overlapping ranges), so only its per-task counts are compared. **Bytes:** the 192 payloads decoded again on every platform with every kernel set it runs, 15 new runs: 0 of 2880 differ from the record (record total now 0 of 4608 over 24 runs). Platforms × sets: build host CPU (neon all cores, neon one core, wide), its GPU (Metal), wasm32-wasip2 there and on host A (wide, the only set on wasm), host A CPU (avx512-vnni, avx512, avx2-vnni, avx2, wide) and GPU (Vulkan), host B CPU (avx2-vnni, avx2, wide). Host B's GPU was not reached (its driver was not loaded after a reboot), so its GPU decode is the record's only. The GPU decodes run both graphs on the device (`gpu=2 cpu=0` on every payload), so they are unchanged by the kernel sets; the runs confirm it. ## Where the time went (profiled first) - **arm64, one thread, q6** (`sample`): convolution 74 %, IGDN 20 % (+3 % integer square root), requantisation epilogue 2 %; everything else small. - **x86_64 host A** (`perf`): the same shape, but the 32×32→64 kernels were *portable* Rust there (no SIMD at the x86-64 baseline): one-thread q6 9.7 s, the big transposed convolution 4.3 s for 22.6 G multiply-accumulates. - Operand ranges, measured per layer: weights |w| ≤ 2047 (12-bit), activations from 7 463 to 316 606; the big layers' inputs fit 16 bits. - After the kernels, on 32 threads the critical path is serial: the interpreter's per-tensor range checks (`wai-ops`, one thread, no_std), freeing and first-touching 150 MB tensors, the rANS decode, re-parsing the set's models and `entropy.json` on every decode, and the harness's SHA-256 of each output. ## What changed - **Kernel sets** (`int_kernels::Isa`; `Policy::fast_isa`; `isas()`; `WAI_INT_ISA` env): `wide` (the previous kernels, unchanged), `neon`, `avx2`, `avx2-vnni`, `avx512` (F, BW, DQ), `avx512-vnni`. Detection at run time; a set the processor lacks is never executed (it becomes `wide`). wasm and other targets have no narrow set. - **Narrow convolutions** (`int_kernels/narrow.rs`, `int_kernels/x86.rs`): 16-bit operands; an input past 2¹⁵−1 split into two 15-bit limbs; 32-bit partial sums over blocks of `kblk` taps with `kblk·(products per tap)·max|w|·max|operand| ≤ i32::MAX`; each block sign-extended, scaled by its limb and added into 64-bit sums, all within `Σ terms·max|w|·max|operand|·2^shift ≤ i64::MAX` (the plan refuses otherwise and the wide kernels run). x86: two channels per 32-bit word, `vpmaddwd`/`vpdpwssd`; aarch64: `smlal` by weight lane. Tap blocks are outermost so a block's weights and input lines stay in L1 while the row is swept. - **IGDN**: x86 SIMD norms (`vpmuldq` exact from i32 operands, arithmetic shift — AVX2 restores the sign bits itself), NEON fuses shift and add at gf = 16 (`ssra`), AVX-512 forms roots and outputs in lanes; every root is seeded from the IEEE-754 square root and corrected in integers (the correction alone decides the result). - **`int_ops::Host`**: requantisation in `i64` with checked steps (falls back to the interpreter's `i128` on any overflow); `rescale`, `clamp`, `relu`, `leaky` on the workers; `max|x|` read in parallel and once per IGDN. - **Tests**: `testkit::FAST` and the decoder-level `POLICIES` now include every kernel set the processor runs (`equivalence_policies()`); new tests against an exact sum written out from the definition (random geometry, grouped/odd channels, every tail width), the block edges (|w|=|x|=2¹⁵−1, one tap more would overflow i32), the limb edges, the seeded root (including deliberately wrong seeds), the AVX-512 root/product lanes, the i64 requantisation, the element-wise operators, a declared-kernel-sets test, and `tests/int_ops_conformance.rs` (every graph vector of `ops-conformance/vectors` and `graphs` through the fast kernels under every policy and set, held to the stated integers or refusal). - **CI**: the kernel steps run `int_ops_conformance`; a new `int-kernels-amd` job on the `linux-amd` host runs the kernel suite in both profiles with every x86_64 set, AVX-512 included, and fails if the declared sets are not found. - **Record and text**: `research/determinism-cost.md` §4.6 (new) and brief, §3, §5; `determinism-tiers` §7.1 bullets; `runs/` scripts and README; CHANGELOG; a pointer in `decode-throughput.md`. DET1's readings are byte-for-byte unchanged in the record (checked field by field); only the totals moved. ## Decisions - **Exactness by bounds, not by order.** The wide kernels' rule was "same terms, same order, same accumulator". The narrow kernels regroup terms; they are exact because no value at any step leaves its type, proved from the actual operand maxima before the kernel runs. The `int_kernels` docs now state both rules. - **The interpreter (`wai-ops`) is untouched.** Its serial range checks are now the largest single cost on 32 threads; speeding them needs a change to the reference crate, left out. - **Float seed for the integer root.** Safe for any seed (corrected and, on AVX-512, verified in integers); the wide set keeps the integer routine. - **Labels/tags** added to the same record: `-before` (the record's build re-run), `-after`, `-after-avx2`, `-again` (float re-run); decode tags `<platform>-isa-<set>`. ## Checks `F=neural_int,video_int,audio_int,world_int,splat_int,generate,score,staged,stage_measure,legacy_int,i32_forms`; `T=` the 21 test targets of the CI kernel steps plus `int_ops_conformance`. Host A = x86_64 with AVX-512 (every x86_64 set), build host = arm64 (NEON). | check | where | command | result | |---|---|---|---| | kernel suite, overflow checks on | host A, all 5 x86_64 sets (`WAI_INT_ISAS_EXPECT` asserted) | `cargo test --locked --no-default-features --features $F --lib $T` | lib 490 passed; every target ok | | kernel suite, release | host A | same with `--release` | all ok | | kernel suite, overflow checks on and release | build host (wide, neon) | same, both profiles | lib 490 passed; every target ok | | ops corpus through the fast kernels | both hosts, every set; and wasm32-wasip2 (wasmtime 50) | `--test int_ops_conformance` | 1 passed (every graph vector of `vectors/`, `graphs/`) | | ops corpus, CPU | build host | `wai-ops: cargo test --locked --release`; `wai-verify vectors verify .` | all ok; "ops-conformance: every file verifies" | | ops corpus, Wasm | build host | `wai-ops` tests `--target wasm32-wasip2` under wasmtime; `wai-verify.wasm vectors verify` | all ok; every file verifies | | ops corpus, Metal | build host | `wai-ops-gpu: cargo test --locked --release` (`WAI_OPS_GPU_EXPECT=gpu-metal-0`) | adapters as declared; corpus mismatches=0; mutants killed 18 of 18 | | device vs interpreter with the fast kernels behind it | build host, Metal | `wai-rs --test ops_gpu` | int_hyper, video int_hyper, audio, splat, int_synth, mlicpp, world: graphs_differ=0 | | DET1's 1,728 decodes and more | 3 hosts | `wai_det_cost decode` of all 192 payloads per platform × set | 15 new runs, 2880 decodes, 0 differ from the record | | DET1's verifier (its CI step) | local | `python3 -I tools/wai_determinism_cost_verify.py record.json determinism-cost.md determinism-tiers.md` | ok: 32 timed runs, 582 quoted figures match | | `det_cost_bin`, `int_hyper_synth` | host A | `cargo test --release --features neural_int,video_int,legacy_int --test det_cost_bin --test int_hyper_synth` | ok | | neural tests | build host | `--features ffi,neural,video_int --test neural_cost_limit --test neural_native_decode --test ffi_neural_pin` | ok (3, 38, 4) | | decode symbols | host A | `--test jpegai_decode_symbols` | no forbidden math-library function | | byte-exact goldens | both hosts | `byte-exact-conformance: cargo test --locked --release` | ok (112 x86_64, 111 arm64) | | wai-verify native build + tests | both hosts | `cargo build/test --release --features cli` | ok (the native build was broken by the first layout; fixed in 986edfea) | | Kani compiles the change | build host | `wai-verify: cargo kani --lib --harness int_transform::kani_layernorm_is_total --harness int_transform::kani_softmax_singleton_is_certain` | 2 verified | | wai-web wasm32 and native builds | build host | `cargo build --locked --release [--target wasm32-unknown-unknown]` | ok | | clippy, no new notes | both hosts | `cargo clippy --locked --all-targets --features $F` (+ `neural_int,ops_gpu` on arm64), notes in touched files against the base | none in touched files | | repo checks | local | check-tests-in-ci, workflow-concurrency, spec-refs, wai-section-refs, no-vanity-metrics, public-text self-test, comparative-prose, orphan-sources; YAML parse | all exit 0 | | name check | local | `check-public-names.sh` on the 36 changed files; commit messages against the patterns | clean | ## Falsifications (committed tree, one mutation each, then restored) | mutation (one change) | kernel set | caught by | result | |---|---|---|---| | F1 AVX-512 masked tail drops one lane | avx512* | `narrow_convolutions_give_the_exact_sums`; `int_ops_conformance`; and the real decodes: a mutated `wai_det_cost` on the 48 q2/q6 payloads | exit 101, 101; **48 of 48 decodes differ** from the record | | F2 a 32-bit block one tap longer than its bound | all narrow | `narrow_blocks_flush_before_any_partial_sum_leaves_i32` | 101 (x86_64), 101 (arm64) | | F3 NEON flush reads the low lanes for the high half | neon | `narrow_convolutions_give_the_exact_sums` | 101 | | F4 NEON fused IGDN shift made rounding in one lane (`vsraq`→`vrsraq`) | neon | first run: **not caught** (exit 0; a norm a unit off seldom moves its root). Added `the_simd_norms_are_the_scalar_norms` (777dd239) | 101 | | F5 AVX2 IGDN loses the sign bits of negative terms | avx2* | `igdn_equals_reference_in_every_mode` (signed narrow case) | 101 | | F6 i64 half-even requantisation rounds up at the half | all | `the_i64_requantisation_is_the_interpreters` | 101 | | F7 the seeded root without its upward correction | narrow | first run: **not caught** (exit 0; a correct square root never needs it). Test now seeds wrong seeds (c061fbb0) | 101 (x86_64), 101 (arm64) | | F8 AVX-512 IGDN output shifted by frac + 1 | avx512* | `the_avx512_roots_and_products_are_the_scalar_ones` | 101 | | F9 the high limb weighted 2^14 | all narrow | `narrow_convolutions_give_the_exact_sums` | 101 (x86_64); arm64: the first loop run passed because its snapshot did not carry the mutation (see below), the rerun 101 | **A tooling note for the lead (F9 on arm64):** the remote-build wrapper snapshots the worktree with a copied index. An edit that keeps a file's size, made in the same second as the `git checkout` that restored it, keeps the index's stat match, so `git add -A` skips it and the snapshot is the unmutated tree. The loop's F9 edit (15 → 14) was such an edit; F2's edit of the same file before it had changed the size. Rerun with a gap, the mutation reached the build and was caught. Same-size mutations need a pause (or `git update-index --really-refresh`) before the wrapper runs. ## Not done / open - The build host's integer decode still costs more than its float decode (above). Further NEON work (the tight `smlal` loop is already 16 per tap) or a GPU energy figure on that host are the levers; the GPU rail did not read live in DET1's brackets. - Host B: no float re-run, no GPU run this time. - Serial costs left on the critical path: the interpreter's range checks, tensor allocation and freeing (page faults), rANS, and re-parsing the set's models and entropy tables per decode (`int_payload` caches only the lowered graphs, by design). - The AVX-512 kernels run in CI only on the `linux-amd` host job added here; the other legs cover NEON and AVX2/AVX-VNNI. 🤖 Generated with [Claude Code](https://claude.com/claude-code) --- **Closed:** Landed on main through merge train E (#499, merge bf39ec9a); verified by content: this PR's head is in main 
wai_determinism_cost.py selects the pinned reference set's self-test
pictures, exports mbt2018_mean at all eight published qualities to integer
sets (each checkpoint pinned by SHA-256), codes every picture with both
codecs, decodes the float streams on any device, and computes distortion,
BD-rate and the record's summary. wai_det_energy.py times a decoder child
in rounds and reads its counters around them (per-task CPU energy and the
GPU rail on macOS, package energy on Linux), with idle brackets.
wai_det_cost (wai-rs bin, neural_int; ops_gpu for a GPU) decodes WIH1
payloads through int_payload::decode_int, and serves timed rounds.
wai_determinism_cost_verify.py recomputes the record's summary from its
readings and checks every figure the text quotes from it.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
The per-host scripts that took the readings (build host, two x86_64
hosts) go beside the record, with how each platform is tagged. The
summary gains the integer decodes' total across platforms and the PSNR
change of float decodes whose entropy decode diverged, in both the tool
and its independent verifier; per-decode energy is stated in joules.
wai_det_energy reads each accelerator client's GPU time per registry
node. The conformance workflow's independent-verifiers job checks the
record, the texts quoting it, and the decoder bin.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
research/determinism-cost.md and its record (record.json): mbt2018_mean at
all eight published qualities, integer export against the model's own
float codec, on the pinned reference set's 24 self-test pictures. BD-rate
by PSNR and MS-SSIM per picture; each codec's decodes repeated across nine
integer and seven float platforms and compared byte for byte, the float
ones placed in or after the entropy decode; decode time and energy with
counter, scope, idle brackets, liveness and ratio over seven rounds.

determinism-tiers §7.1 quotes the record, each figure tied to its place in
it by a marker the verifier checks; §7.2 says what WAI claims and does not,
and makes normative that a record figure is never stated beyond what the
record names, that no receipt takes its energy from it, and that a cost
figure in the spec names its measurement. SPEC §5's int_hyper row points
to §7. CHANGELOG entry.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
energy-measurement §3.1's liveness rule (two advances in a row, five or
more, four in five, over 300 ms) for the busy bracket and both idle
brackets, recomputed from the bracket facts the record keeps, as the tool
that writes the record applies it.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
Co-Authored-By: Claude Opus 5.5 <[email protected]>
Two-point Gauss-Legendre on each knot interval's share of the common
range is exact for the cubic the interpolant carries there, so the
verifier's BD-rates agree with the tool's analytic integral to float
precision, and a one-unit change in one picture's SSE no longer falls
inside the tolerance.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
det_cost_bin: a refusal line must carry its code exactly
Some checks failed
wai-pack / wai-pack · independent verifier (Python) (pull_request) Has been cancelled
Details
wai-pack / wai-pack · the floor through a media prober (pull_request) Has been cancelled
Details
wai-pack / wai-pack · dependency licences (pull_request) Has been cancelled
Details
wai-ops-gpu integer operators on GPUs / does this change touch it (pull_request) Has been cancelled
Details
wai-ops-gpu integer operators on GPUs / builds · no adapter is not a pass (linux) (pull_request) Has been cancelled
Details
wai-ops-gpu integer operators on GPUs / Metal GPU and browser WebGPU (macos) (pull_request) Has been cancelled
Details
WAI energy receipts / does this change touch it (pull_request) Has been cancelled
Details
WAI energy receipts / energy receipts · linux (pull_request) Has been cancelled
Details
WAI energy receipts / energy receipts · linux-arm64 (pull_request) Has been cancelled
Details
WAI energy receipts / energy receipts · macos (pull_request) Has been cancelled
Details
WAI conformance / does this change touch it (pull_request) Has been cancelled
Details
WAI conformance / cargo-class corpora · linux (pull_request) Has been cancelled
Details
WAI conformance / cargo-class corpora · linux-arm64 (pull_request) Has been cancelled
Details
WAI conformance / cargo-class corpora · macos (pull_request) Has been cancelled
Details
WAI conformance / quantum core and corpus under wasm32-wasip2 (wasmtime) (pull_request) Has been cancelled
Details
WAI conformance / v2 vocabulary · manifests validate as stated (pull_request) Has been cancelled
Details
WAI conformance / independent verifiers · claims, JWP, key discovery and integer payloads (Python) (pull_request) Has been cancelled
Details
WAI conformance / delivery telemetry · CMCD and CMSD against an independent library (pull_request) Has been cancelled
Details
WAI conformance / cross-arch byte-equality · linux (pull_request) Has been cancelled
Details
WAI conformance / cross-arch byte-equality · linux-arm64 (pull_request) Has been cancelled
Details
WAI conformance / cross-arch byte-equality · macos (pull_request) Has been cancelled
Details
WAI conformance / codec wrappers · fuzz smoke, text decode limits, C entry points, provenance, the semantic path (pull_request) Has been cancelled
Details
WAI conformance / C2PA manifest emission · signed, read back, validated (pull_request) Has been cancelled
Details
standards workspaces build + test / which workspaces this change touches (pull_request) Has been cancelled
Details
standards workspaces build + test / macOS energy backends (pull_request) Has been cancelled
Details
repository checks / lockfiles, tests in CI, workflow hygiene, spec and section references, public text (pull_request) Has been cancelled
Details
joule-moq-web browser client / joule-moq-web · does this change touch it (pull_request) Has been cancelled
Details
joule-moq-web browser client / joule-moq-web · browser end-to-end (macos) (pull_request) Has been cancelled
Details
joule-moq-room room service / joule-moq-room · does this change touch it (pull_request) Has been cancelled
Details
joule-moq-room room service / joule-moq-room, every target (pull_request) Has been cancelled
Details
4863036d8a 
The test compared a refusal line by prefix, so a decoder reporting
`bad_magicx` for `bad_magic` passed; found by falsifying it.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
The integer decoders' convolutions spent their time in 32x32->64-bit
multiply-accumulates, portable Rust on x86_64. A kernel set (Isa) now
selects narrow kernels where the processor runs them: 16-bit operands
(an input past 2^15 - 1 split into two 15-bit limbs), 32-bit partial sums
over blocks of taps sized so no partial sum leaves i32, then 64-bit sums
(vpmaddwd / vpdpwssd on x86_64 with two channels per word, smlal on
aarch64). The inverse GDN's norms run in 64-bit SIMD lanes on x86_64
(vpmuldq, an exact product, then the floor shift), the NEON ones fuse the
shift and the add at gf = 16, and each integer root is seeded from the
IEEE-754 square root and corrected in integers. The wide kernels stay, as
Isa::Wide, and run wherever a narrow plan does not apply.

int_ops lends the interpreter rescale, clamp, relu and leaky on the
workers, and requantises in i64 where every value fits (else in the
interpreter's i128). max|x| is read in parallel, once per inverse GDN.

The equivalence tests run every kernel set the processor runs: random
geometry against the exact sums written out, the block edges, the limb
edges, the seeded root, the i64 requantisation and the element-wise
operators against the interpreter.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
Each decoder-level equivalence test now runs int_kernels::equivalence_policies:
the reference, the fast kernels on every tiling, and each kernel set the
processor runs. tests/int_ops_conformance.rs decodes every graph vector of
ops-conformance/ (vectors/ and graphs/) through the interpreter with the
fast kernels behind it, under each of those policies, and holds every
decode to the integers or refusal the vector states.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
Co-Authored-By: Claude Opus 5.5 <[email protected]>
The narrow kernels now take a block of at most 128 taps outermost and sweep
the whole row under it, so one block's weights and the input lines its
taps read stay in the first-level cache. On AVX-512 (now F, BW and DQ) the
inverse GDN's roots are seeded from the IEEE-754 square root, corrected
and held to s^2 <= n < (s+1)^2 in lanes, and (x * s) >> frac is vpmullq,
exact for |x| < 2^31; a step that fails the check is left to the scalar
form. The weights' largest magnitude is read in parallel too.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
The kernel steps of wai-conformance run tests/int_ops_conformance.rs on
each leg. A new job on the linux-amd host runs the same tests in both
profiles there, whose processor runs every x86_64 kernel set, AVX-512
among them, which no other leg's does; the host declares its sets, and
the_kernel_sets_found_are_the_declared_ones fails where the sets found
differ, so a missing set is a failure and not a skip.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
research/determinism-cost.md §4.6, in the same record and by the same
procedure: the 192 payloads decoded again on every platform with every
kernel set it runs (15 new runs, 0 decodes differing from the record's),
and the record's time and energy runs repeated in one session per host
beside the record's own build and the float decoder. On x86_64 host A the
integer decode on the narrow kernels took 144.8 ms and 12.43 J of marginal
package energy at quality 6 against 210.7 ms and 14.03 J for the float
decode beside it (the record's build: 728.0 ms, 92.14 J). On the build
host its own CPU energy fell from 11.01 J to 6.43 J at quality 6 and is
still above the float decode's 1.28 J by that count. Host B's integer
decode was measured before and after, without a float decode beside it.
The scripts that took the readings are in runs/; decode-throughput.md
notes the new sets; the changelog records the change.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
A correctly rounded square root seeds every u64 within one of its root, so
the correction steps that undo a wrong seed ran in no test; removing the
upward one passed them all. The root is now formed from any seed by
isqrt_from, and the test seeds it up to a thousand below and above the
root, and with the largest seed.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
Falsifying the NEON fused-shift kernel (one lane's ssra made a rounding
shift) passed every test: a norm a unit off seldom moves its integer root,
so the output tests could not see it. the_simd_norms_are_the_scalar_norms
holds each SIMD form of the norms (NEON at any gf and fused at 16, AVX2
with non-negative and signed gamma, AVX-512) to the scalar sum before any
root is taken. The requantisation and element-wise tests drew magnitudes
past Rng::signed's 2^62, which overflowed with overflow checks on; they
now draw within it and list the ends of i64 instead.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
Co-Authored-By: Claude Opus 5.5 <[email protected]>
wai-rs: the narrow kernels' files are named by path, and the model checker runs the wide kernels
Some checks failed
wai formal verification (integer core) / kani (integer-core) (pull_request) Has been cancelled
Details
WAI energy receipts / does this change touch it (pull_request) Has been cancelled
Details
WAI energy receipts / energy receipts · linux (pull_request) Has been cancelled
Details
WAI energy receipts / energy receipts · linux-arm64 (pull_request) Has been cancelled
Details
WAI energy receipts / energy receipts · macos (pull_request) Has been cancelled
Details
WAI conformance / does this change touch it (pull_request) Has been cancelled
Details
WAI conformance / cargo-class corpora · linux (pull_request) Has been cancelled
Details
WAI conformance / cargo-class corpora · linux-arm64 (pull_request) Has been cancelled
Details
WAI conformance / cargo-class corpora · macos (pull_request) Has been cancelled
Details
WAI conformance / fast integer kernels, every x86_64 kernel set (AVX-512 silicon) (pull_request) Has been cancelled
Details
WAI conformance / quantum core and corpus under wasm32-wasip2 (wasmtime) (pull_request) Has been cancelled
Details
WAI conformance / v2 vocabulary · manifests validate as stated (pull_request) Has been cancelled
Details
WAI conformance / independent verifiers · claims, JWP, key discovery and integer payloads (Python) (pull_request) Has been cancelled
Details
WAI conformance / delivery telemetry · CMCD and CMSD against an independent library (pull_request) Has been cancelled
Details
WAI conformance / cross-arch byte-equality · linux (pull_request) Has been cancelled
Details
WAI conformance / cross-arch byte-equality · linux-arm64 (pull_request) Has been cancelled
Details
WAI conformance / cross-arch byte-equality · macos (pull_request) Has been cancelled
Details
WAI conformance / codec wrappers · fuzz smoke, text decode limits, C entry points, provenance, the semantic path (pull_request) Has been cancelled
Details
WAI conformance / C2PA manifest emission · signed, read back, validated (pull_request) Has been cancelled
Details
wai cross-architecture byte-equality / does this change touch it (pull_request) Has been cancelled
Details
wai cross-architecture byte-equality / linux-aarch64 (pull_request) Has been cancelled
Details
wai cross-architecture byte-equality / linux-x86_64 (pull_request) Has been cancelled
Details
wai cross-architecture byte-equality / macos-arm64 (reference) (pull_request) Has been cancelled
Details
standards workspaces build + test / which workspaces this change touches (pull_request) Has been cancelled
Details
standards workspaces build + test / macOS energy backends (pull_request) Has been cancelled
Details
repository checks / lockfiles, tests in CI, workflow hygiene, spec and section references, public text (pull_request) Has been cancelled
Details
joule-moq-web browser client / joule-moq-web · does this change touch it (pull_request) Has been cancelled
Details
joule-moq-web browser client / joule-moq-web · browser end-to-end (macos) (pull_request) Has been cancelled
Details
joule-moq-room room service / joule-moq-room · does this change touch it (pull_request) Has been cancelled
Details
joule-moq-room room service / joule-moq-room, every target (pull_request) Has been cancelled
Details
986edfea33 
wai-verify and byte-exact-conformance include int_kernels.rs by `path`,
and a file included that way resolves its own `mod narrow;` beside it, so
wai-verify's native build failed (file not found for module `narrow`).
Both submodules now carry their `path`, relative to int_kernels.rs, which
holds in either build. Under `cfg(kani)` the narrow sets are left out:
no processor-feature probe, no SIMD, the wide kernels and the integer
root, as the IGDN harness checked them before.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
dcharlot closed this pull request 2026-10-26 20:57:53 -04:00
Some checks failed
wai-web wasm32 / does this change touch it (pull_request) Has been cancelled
Details
wai-web wasm32 / wai-web wasm32 build · corpus, menu and writers (pull_request) Has been cancelled
Details
wai-web reach builds / wai-web reach · does this change touch it (pull_request) Has been cancelled
Details
wai-web reach builds / wai-web reach builds · the package of record, every output as recorded (pull_request) Has been cancelled
Details
wai-web reach builds / wai-web reach builds · this tree's wai-web, every rung on its engine line (pull_request) Has been cancelled
Details
WAI mbt2018_mean integer decode (regenerated vectors) / does this change touch it (pull_request) Has been cancelled
Details
WAI mbt2018_mean integer decode (regenerated vectors) / regenerate mbt2018_mean integer vectors → Rust decode, required (pull_request) Has been cancelled
Details
wai-quantum studio and desktop / does this change touch it (pull_request) Has been cancelled
Details
wai-quantum studio and desktop / studio · browser (pull_request) Has been cancelled
Details
wai-quantum studio and desktop / desktop app · macos (pull_request) Has been cancelled
Details
wai-pack / wai-pack · does this change touch it (pull_request) Has been cancelled
Details
wai-pack / wai-pack · linux (pull_request) Has been cancelled
Details
wai-pack / wai-pack · linux-arm64 (pull_request) Has been cancelled
Details
wai-pack / wai-pack · macos (pull_request) Has been cancelled
Details
wai-pack / wai-pack · independent verifier (Python) (pull_request) Has been cancelled
Details
wai-pack / wai-pack · the floor through a media prober (pull_request) Has been cancelled
Details
wai-pack / wai-pack · dependency licences (pull_request) Has been cancelled
Details
wai-ops-gpu integer operators on GPUs / does this change touch it (pull_request) Has been cancelled
Details
wai-ops-gpu integer operators on GPUs / builds · no adapter is not a pass (linux) (pull_request) Has been cancelled
Details
wai-ops-gpu integer operators on GPUs / Metal GPU and browser WebGPU (macos) (pull_request) Has been cancelled
Details
WAI neural sink gates / does this change touch it (pull_request) Has been cancelled
Details
WAI neural sink gates / model-free refusals of the neural sink (pull_request) Has been cancelled
Details
WAI JPEG-AI whole decode (pinned inputs, required) / does this change touch it (pull_request) Has been cancelled
Details
WAI JPEG-AI whole decode (pinned inputs, required) / fetch pinned JPEG-AI inputs → whole decode + pixel hash, required (pull_request) Has been cancelled
Details
wai formal verification (integer core) / does this change touch it (pull_request) Has been cancelled
Details
wai formal verification (integer core) / every harness in one shard (pull_request) Has been cancelled
Details
wai formal verification (integer core) / kani (arith) (pull_request) Has been cancelled
Details
wai formal verification (integer core) / kani (isqrt512) (pull_request) Has been cancelled
Details
wai formal verification (integer core) / kani (plan) (pull_request) Has been cancelled
Details
wai formal verification (integer core) / kani (sound) (pull_request) Has been cancelled
Details
wai formal verification (integer core) / kani (stage) (pull_request) Has been cancelled
Details
wai formal verification (integer core) / kani (rans64-t2) (pull_request) Has been cancelled
Details
wai formal verification (integer core) / kani (stage-long) (pull_request) Has been cancelled
Details
wai formal verification (integer core) / kani (integer-core) (pull_request) Has been cancelled
Details
WAI energy receipts / does this change touch it (pull_request) Has been cancelled
Details
WAI energy receipts / energy receipts · linux (pull_request) Has been cancelled
Details
WAI energy receipts / energy receipts · linux-arm64 (pull_request) Has been cancelled
Details
WAI energy receipts / energy receipts · macos (pull_request) Has been cancelled
Details
WAI conformance / does this change touch it (pull_request) Has been cancelled
Details
WAI conformance / cargo-class corpora · linux (pull_request) Has been cancelled
Details
WAI conformance / cargo-class corpora · linux-arm64 (pull_request) Has been cancelled
Details
WAI conformance / cargo-class corpora · macos (pull_request) Has been cancelled
Details
WAI conformance / fast integer kernels, every x86_64 kernel set (AVX-512 silicon) (pull_request) Has been cancelled
Details
WAI conformance / quantum core and corpus under wasm32-wasip2 (wasmtime) (pull_request) Has been cancelled
Details
WAI conformance / v2 vocabulary · manifests validate as stated (pull_request) Has been cancelled
Details
WAI conformance / independent verifiers · claims, JWP, key discovery and integer payloads (Python) (pull_request) Has been cancelled
Details
WAI conformance / delivery telemetry · CMCD and CMSD against an independent library (pull_request) Has been cancelled
Details
WAI conformance / cross-arch byte-equality · linux (pull_request) Has been cancelled
Details
WAI conformance / cross-arch byte-equality · linux-arm64 (pull_request) Has been cancelled
Details
WAI conformance / cross-arch byte-equality · macos (pull_request) Has been cancelled
Details
WAI conformance / codec wrappers · fuzz smoke, text decode limits, C entry points, provenance, the semantic path (pull_request) Has been cancelled
Details
WAI conformance / C2PA manifest emission · signed, read back, validated (pull_request) Has been cancelled
Details
wai cross-architecture byte-equality / does this change touch it (pull_request) Has been cancelled
Details
wai cross-architecture byte-equality / linux-aarch64 (pull_request) Has been cancelled
Details
wai cross-architecture byte-equality / linux-x86_64 (pull_request) Has been cancelled
Details
wai cross-architecture byte-equality / macos-arm64 (reference) (pull_request) Has been cancelled
Details
standards workspaces build + test / which workspaces this change touches (pull_request) Has been cancelled
Details
standards workspaces build + test / macOS energy backends (pull_request) Has been cancelled
Details
repository checks / lockfiles, tests in CI, workflow hygiene, spec and section references, public text (pull_request) Has been cancelled
Details
joule-moq-web browser client / joule-moq-web · does this change touch it (pull_request) Has been cancelled
Details
joule-moq-web browser client / joule-moq-web · browser end-to-end (macos) (pull_request) Has been cancelled
Details
joule-moq-room room service / joule-moq-room · does this change touch it (pull_request) Has been cancelled
Details
joule-moq-room room service / joule-moq-room, every target (pull_request) Has been cancelled
Details

Pull request closed

Please reopen this pull request to perform a merge.
Sign in to join this conversation.
Reviewers
No reviewers
Labels
Clear labels
No items
No labels
Milestone
Clear milestone
No items
No milestone
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
Transaction-Science/open-standards!467
No description provided.