WAI across fabrics: each step on the fabric that does it with the least measured energy (packaging §11.6) #438

Closed
dcharlot wants to merge 13 commits from wai/cross-fabric into wai/pack-efficiency
Owner

Each step of a WAI pack or verify can now run on the fabric that does it with the least measured energy: the CPU, a GPU, a hardware media engine, or (where it is sound) a neural engine. Every step is defined in integers, so where it runs changes no byte of a package and no verdict. Only a metered package's measurement claim says where the steps ran and what each fabric drew. For the owner's review. Do not merge before that review. Stacked on #426 (wai/pack-efficiency), which is also unmerged; this PR's base is that branch.

The research behind each choice, with its sources, is in docs/research/wai-cross-fabric.md.

What changed

  1. Steps and fabrics (packaging §11.6).
    • The computations of a pack and a verify are named steps: source-decode, source-read, segment, base-search, base-code, refine-code, sse, object-hash, digest, floor-read, floor-decode, index, io.
    • A packager runs a step only on a backend that has given the reference's output on qualifying inputs. Any call that backend cannot take runs on the CPU, with the same output. So §11.2 holds across fabrics, and §11.2 now says so.
    • A verifier computes a check on another fabric only where the package states the value the check compares with, such as an object's content hash. It decides any mismatch again on the reference, so its verdict cannot change with the fabric.
    • No step is computed by a hardware encoder: no vendor fixes such an encoder's bytes. A floor encoded on one is a §7.5 floor, naming the encoder, its version, and the device and driver.
    • A floor states no decoded-output digest, so a hardware decode never decides a floor verdict.
  2. The fabric record (jwp-receipts §2.4.8, energy-measurement §6.3).
    • A measurement claim may carry fabrics: each step with every fabric that ran it and its backend, and the figure divided into one share per fabric.
    • Each share is attributed task (the operating system's account of the work's own energy) or rail (the fabric's whole rail, the work its sole user, at least 99% of its busy time).
    • A shared rail is never divided by busy time. The repo's own measurement (jouleclaw-energy, process_gpu) found that this overcharges a workload 3 to 15 times.
    • fabrics is the one claim member that is absent rather than null, so every existing claim and package keeps its bytes.
    • The record lives in the claim, not the receipt. In the receipt it would make LISTING differ by machine and void §11.2.
    • New reasons: measurement_fabric_unknown, measurement_fabric_order, measurement_fabric_unmetered, measurement_fabric_unused, measurement_fabric_sum.
  3. wai-pack-gpu, a new crate with its own lockfile, as wai-ops-gpu has.
    • It provides the motion search, SSE and BLAKE3's chunk function as WGSL compute kernels through wgpu 30.0.1 (Metal, Vulkan, DX12).
    • The kernels use 32-bit words with every value bounded, so WGSL's defined integer arithmetic gives one result on every conformant device.
    • The motion search takes each block's minimum by a reduction over a total order (cost, then |dy|+|dx|, then index), so the result does not depend on which thread visits which candidate.
  4. wai-pack: fabric dispatch (--fabric cpu|gpu|auto:<costs.json>, wai-pack fabric list|calibrate).
    • Each call goes to the qualified backend with the least measured µJ per call plus nJ per unit, much as an energy-aware scheduler places work. The CPU wins a tie, takes any call nothing has measured, and takes any call another fabric does not take.
    • Feature gpu: base-search, sse and object-hash run on a GPU.
    • Feature fabric-meter (macOS): --meter fabric meters a pack per fabric without privileges and writes the claim's fabric record. The CPU figure is the kernel's per-task ri_energy_nj. The GPU figure is the IOReport GPU Energy rail, read at its republish, taken only when the process held at least 99% of every client's GPU time; otherwise there is no GPU figure and the pack is refused pack-meter.
    • Without either feature, wai-pack builds and behaves as before.
  5. wai-pack floor-decode decodes a package's floors with libdav1d (CPU) or VideoToolbox (media engine) through ffmpeg, and prints each piece's digest. It is informative, for comparing decoders.
    • The media engine returns AV1 frames as P010. They are read back exactly, or the frame is refused.
    • ffmpeg's own 16-to-8-bit conversion rounds and dithers, so it is never used.
  6. wai-rs hooks.
    • int_video_enc::encode_with(…, &mut dyn MotionSearch): only the search moves; every other step of the encode is unchanged.
    • stage_index::check_object_hashed.

Conformance

  • Claim vectors: appended after every existing one, which a new test pins by a digest computed from the committed corpus before the change. They cover one accept, four step-2 shape refusals (null among them), and the step-4 reasons. One of those refuses a share divided by GPU time.
  • Package mutations: metered-fabrics (accepted) and metered-fabrics-unknown-step (measurement-claim-invalid). Both verifiers give both cases their reason.
  • Tests:
    • wai-pack-gpu: kernels against references transcribed from the text; mutants (edited kernel expressions, each caught); many threads on one device, every call completing on the device.
    • wai-pack tests/fabric.rs:
      • placement, the cost table and the claim's record;
      • with gpu, every golden packed on the GPU and byte for byte the committed package;
      • every golden and every package and staged mutation given the CPU's verdict with the hashes on the GPU;
      • an edited kernel refused by qualification and, forced in, moving no verdict.
  • CI: the macOS leg runs the GPU steps with an absent adapter as a failure.

Falsification

Each new test was falsified once on the build host, by xfab-falsify.sh in rb's checkout. The script made one exact edit to the guarded code, required the test to fail, restored the file through git, and required it to pass. All 20 cases failed when broken and passed when restored.

Test Edit
wai-rs encode_with_runs_only_the_search_elsewhere a search's vector count unchecked; the search handed the source instead of the reconstruction
wai-pack every_mutation_names_its_reason check_object_hashed ignoring the hash handed in
wai-rs the_vectors_before_the_fabric_record_are_unchanged the fabric vectors inserted before the session's
wai-rs the_pinned_corpus_reproduces_and_passes rule 6 without the sum; time-share admitted; a null record read as absent
claim_verify.py over the corpus rule 6 without the sum; a step on an unmetered fabric admitted
pack_verify.py and wai-pack conformance check over pack-conformance a fabric record's steps not held to §11.6
wai-pack floor_decode unit test P010 low bits not checked
wai-pack tests/fabric.rs (CPU) the reference placing a call on the GPU; a fabric chosen without a CPU measurement; a share on a fabric no step ran on dropped
wai-pack-gpu tests/concurrent.rs the map's callback not waited for
wai-pack tests/fabric.rs (GPU) the shipped kernel's tie rule flipped; a diagonal half-pel rounding off by one; an object-hash refusal not decided again on the CPU; the GPU's hashes never used

The kernel tests are also held by tests/mutants.rs: 12 edited kernel expressions, each caught on every run. On its first run the suite found that the BLAKE3 kernel's last-word mask was dead code, because the host already pads every message with zeros. The mask was removed and its mutant replaced.

The tightened fabric test was not falsifiable at first: dropping a share on a fabric no step ran on was also caught by the claim's own sum rule, one layer down. It now requires the packer's own reason, and is falsified.

Measured on big-buck-bunny

Big-buck-bunny (596 s, 14,315 frames; the 256×144 integer proxy and the ladder's four 10-bit AV1 rungs) on the build host, shared and loaded. Every run was metered per fabric (method in the research doc, §9):

  • CPU: the kernel's account of the process's own energy, and of any VideoToolbox service the run started.
  • GPU: the rail, only where the run held at least 99% of all GPU time.
  • Media engine: no unprivileged counter on macOS 27, so a media-engine figure is its CPU side only, a lower bound.

Digests, files and instruction counts are exact. Times and energies are indicative (load average 14–50 for the packs, 25–90 for the decodes).

Exact.

  • Phase 0's av1-360 policy packed with the search on the GPU gives pack digest 7454acc6…1118, phase 0's, with every file the same. This held in three runs, and with --fabric auto too.
  • Run B of #426 (four floor rungs, the ladder's CMAF) packed on the GPU gives pack digest 3912bd7c… and receipt e0256cd9…, #426's own. Run C (floor-only) gives f8871444…, #426's own.
  • The media engine and dav1d agree on every frame of all four floors (SHA-256 per frame), and on every one of the 600 pieces through wai-pack floor-decode.
  • The proxy decoded on the media engine is phase 0's proxy.y4m, byte for byte.

Pack and verify, joules per fabric (the GPU otherwise idle):

Run Wall CPU GPU Total Peak CPU baseline
av1-360 pack, search on the GPU 56 s 573 J 1,284 J 1,857 J 0.95 GB 20,299 J, 3,689 s, 3.04 GB
av1-360 pack, --fabric auto 56 s 571 J 1,283 J 1,854 J 0.76 GB (same)
run B pack, search on the GPU, --jobs 12 71 s 525 J 1,239 J 1,763 J 1.62 GB 19,541 J, 980 s (--jobs 6), 0.17 GB
run C pack (floor-only), hashes on the GPU 2.2 s 4.4 J 2.3 J 6.7 J 0.65 GB 3.7 J, 1.5 s, 0.04 GB
wai-pack verify av1-360, CPU / hashes on the GPU 32 / 31 s 118 / 120 J 0 / – 118 / ≥120 J 0.58 / 2.04 GB 31–33 s
wai-pack verify run B, CPU / hashes on the GPU 34 / 32 s 119 / 122 J 0 / 0.2 J 119 / 122 J 1.55 / 3.44 GB 35 s, 121 J
  • CPU instructions per av1-360 pack: 98.3 × 10¹² → 2.05 × 10¹².
  • The CPU time left after the search moved (116 s of phase 0's 3,635 s) shows the search was at least 96.8% of the CPU-only pack.
  • pack_verify.py stays on the CPU by design. Its baselines (2,070 s serial and 206 s at --jobs 12 on A) are unchanged, and it accepts the GPU-made packages, which are byte-identical to the CPU's.

The measured cost table (wai-pack fabric calibrate):

  • search: CPU 20.2 nJ per candidate-sample; GPU 11.3 mJ per call + 1.47 nJ.
  • SSE: CPU 0.85 nJ per byte; GPU 3.1 mJ per call + 0.76 nJ.
  • BLAKE3: CPU 7 µJ per call + 1.33 nJ per byte; GPU 1.7 mJ per call + 1.43 nJ.

So auto puts the search on the GPU and keeps SSE and the hashes on the CPU.

Decoding, CPU (dav1d) against the media engine (the engine's own draw unmetered):

dav1d media engine (CPU side)
1080p floor 23.1 s, 198.7 J 22.0 s, 17.4 J
720p floor 12.4 s, 111.0 J 17.3 s, 11.3 J
540p floor 7.7 s, 73.3 J 14.1 s, 8.5 J
360p floor 5.1 s, 53.8 J 11.3 s, 15.9 J
proxy source (1080p decoded, scaled to 256×144) 22.7 s, 304.1 J 25.6 s, 350.5 J

The proxy gains nothing because its Lanczos scale stays on the CPU.

A metered pack, sealed. The av1-360 pack with --meter fabric, run while the GPU was otherwise idle:

  • It sealed a ModelBased figure of 2,248.4 J. Its claim's fabric record gives the CPU 475.4 J (task) and the GPU 1,773.0 J (rail).
  • The record's steps are base-search, sse and object-hash on the GPU and the rest on the CPU. No search call fell back to the CPU.
  • wai-pack verify accepts it, and pack_verify.py holds 3622 of 3622 checks.

A refusal, as designed. When another process held part of the GPU's time, the metered packs were refused pack-meter rather than sealed with a guessed GPU share. One read: "the GPU was shared: this process had 45.2 s of its 65.0 s, under 99%".

Judged unsound, and not built

  • Any step on the neural engine: it computes in FP16, exact only to 2048, and its results are not fixed across generations.
  • A hardware encoder for any step: no vendor fixes its bytes. The floors are supplied, and the packer refuses to encode one. Of the fleet's x86_64 hosts, the Windows box's discrete GPU has no AV1 engine; jetson-hub's GPU was not probed, because its SSH needs an interactive check this work did not do.
  • The platform's hardware motion estimation in place of §5.5's search: its metric, window and determinism are undocumented.
  • Dividing a shared GPU rail by GPU time, which the repo has measured overcharging a workload 3 to 15 times.
  • Media-engine and neural-engine energy on macOS 27: no counter is live without a private entitlement. A step run there makes a metered span refuse rather than guess.
  • A floor verdict from a hardware decode: floors state no digest, and hardware can conceal an error.
  • ffmpeg's P010-to-8-bit conversion in any comparison: it rounds and dithers, and was measured differing from dav1d's own.
  • GPU BLAKE3 and SSE for WAI's object sizes: sound and built, but the measured placement keeps them on the CPU.

🤖 Generated with Claude Code


Closed: Landed in main through #465 (the packaging train): its content was squashed onto main as 836d0255d with #426, #438 and #452 together, then the review fixes on top; #465 merged as dc0d5ca5. Of the files this PR changed, all but the ones the stacked PRs share or the review changed are byte-identical on main.

Each step of a WAI pack or verify can now run on the fabric that does it with the least measured energy: the CPU, a GPU, a hardware media engine, or (where it is sound) a neural engine. Every step is defined in integers, so where it runs changes no byte of a package and no verdict. Only a metered package's measurement claim says where the steps ran and what each fabric drew. **For the owner's review. Do not merge before that review.** Stacked on #426 (`wai/pack-efficiency`), which is also unmerged; this PR's base is that branch. The research behind each choice, with its sources, is in `docs/research/wai-cross-fabric.md`. ## What changed 1. **Steps and fabrics** (packaging §11.6). - The computations of a pack and a verify are named steps: `source-decode`, `source-read`, `segment`, `base-search`, `base-code`, `refine-code`, `sse`, `object-hash`, `digest`, `floor-read`, `floor-decode`, `index`, `io`. - A packager runs a step only on a backend that has given the reference's output on qualifying inputs. Any call that backend cannot take runs on the CPU, with the same output. So §11.2 holds across fabrics, and §11.2 now says so. - A verifier computes a check on another fabric only where the package states the value the check compares with, such as an object's content hash. It decides any mismatch again on the reference, so its verdict cannot change with the fabric. - No step is computed by a hardware encoder: no vendor fixes such an encoder's bytes. A floor encoded on one is a §7.5 floor, naming the encoder, its version, and the device and driver. - A floor states no decoded-output digest, so a hardware decode never decides a floor verdict. 2. **The fabric record** (jwp-receipts §2.4.8, energy-measurement §6.3). - A measurement claim may carry `fabrics`: each step with every fabric that ran it and its backend, and the figure divided into one share per fabric. - Each share is attributed `task` (the operating system's account of the work's own energy) or `rail` (the fabric's whole rail, the work its sole user, at least 99% of its busy time). - A shared rail is never divided by busy time. The repo's own measurement (`jouleclaw-energy`, `process_gpu`) found that this overcharges a workload 3 to 15 times. - `fabrics` is the one claim member that is absent rather than null, so every existing claim and package keeps its bytes. - The record lives in the claim, not the receipt. In the receipt it would make `LISTING` differ by machine and void §11.2. - New reasons: `measurement_fabric_unknown`, `measurement_fabric_order`, `measurement_fabric_unmetered`, `measurement_fabric_unused`, `measurement_fabric_sum`. 3. **wai-pack-gpu**, a new crate with its own lockfile, as wai-ops-gpu has. - It provides the motion search, SSE and BLAKE3's chunk function as WGSL compute kernels through wgpu 30.0.1 (Metal, Vulkan, DX12). - The kernels use 32-bit words with every value bounded, so WGSL's defined integer arithmetic gives one result on every conformant device. - The motion search takes each block's minimum by a reduction over a total order (cost, then |dy|+|dx|, then index), so the result does not depend on which thread visits which candidate. 4. **wai-pack: fabric dispatch** (`--fabric cpu|gpu|auto:<costs.json>`, `wai-pack fabric list|calibrate`). - Each call goes to the qualified backend with the least measured µJ per call plus nJ per unit, much as an energy-aware scheduler places work. The CPU wins a tie, takes any call nothing has measured, and takes any call another fabric does not take. - Feature `gpu`: `base-search`, `sse` and `object-hash` run on a GPU. - Feature `fabric-meter` (macOS): `--meter fabric` meters a pack per fabric without privileges and writes the claim's fabric record. The CPU figure is the kernel's per-task `ri_energy_nj`. The GPU figure is the IOReport `GPU Energy` rail, read at its republish, taken only when the process held at least 99% of every client's GPU time; otherwise there is no GPU figure and the pack is refused `pack-meter`. - Without either feature, wai-pack builds and behaves as before. 5. **`wai-pack floor-decode`** decodes a package's floors with libdav1d (CPU) or VideoToolbox (media engine) through ffmpeg, and prints each piece's digest. It is informative, for comparing decoders. - The media engine returns AV1 frames as P010. They are read back exactly, or the frame is refused. - ffmpeg's own 16-to-8-bit conversion rounds and dithers, so it is never used. 6. **wai-rs hooks.** - `int_video_enc::encode_with(…, &mut dyn MotionSearch)`: only the search moves; every other step of the encode is unchanged. - `stage_index::check_object_hashed`. ## Conformance - **Claim vectors:** appended after every existing one, which a new test pins by a digest computed from the committed corpus before the change. They cover one accept, four step-2 shape refusals (`null` among them), and the step-4 reasons. One of those refuses a share divided by GPU time. - **Package mutations:** `metered-fabrics` (accepted) and `metered-fabrics-unknown-step` (`measurement-claim-invalid`). Both verifiers give both cases their reason. - **Tests:** - wai-pack-gpu: kernels against references transcribed from the text; mutants (edited kernel expressions, each caught); many threads on one device, every call completing on the device. - wai-pack `tests/fabric.rs`: - placement, the cost table and the claim's record; - with `gpu`, every golden packed on the GPU and byte for byte the committed package; - every golden and every package and staged mutation given the CPU's verdict with the hashes on the GPU; - an edited kernel refused by qualification and, forced in, moving no verdict. - **CI:** the macOS leg runs the GPU steps with an absent adapter as a failure. ## Falsification Each new test was falsified once on the build host, by `xfab-falsify.sh` in rb's checkout. The script made one exact edit to the guarded code, required the test to fail, restored the file through git, and required it to pass. All 20 cases failed when broken and passed when restored. | Test | Edit | |---|---| | wai-rs `encode_with_runs_only_the_search_elsewhere` | a search's vector count unchecked; the search handed the source instead of the reconstruction | | wai-pack `every_mutation_names_its_reason` | `check_object_hashed` ignoring the hash handed in | | wai-rs `the_vectors_before_the_fabric_record_are_unchanged` | the fabric vectors inserted before the session's | | wai-rs `the_pinned_corpus_reproduces_and_passes` | rule 6 without the sum; `time-share` admitted; a `null` record read as absent | | `claim_verify.py` over the corpus | rule 6 without the sum; a step on an unmetered fabric admitted | | `pack_verify.py` and `wai-pack conformance check` over pack-conformance | a fabric record's steps not held to §11.6 | | wai-pack `floor_decode` unit test | P010 low bits not checked | | wai-pack `tests/fabric.rs` (CPU) | the reference placing a call on the GPU; a fabric chosen without a CPU measurement; a share on a fabric no step ran on dropped | | wai-pack-gpu `tests/concurrent.rs` | the map's callback not waited for | | wai-pack `tests/fabric.rs` (GPU) | the shipped kernel's tie rule flipped; a diagonal half-pel rounding off by one; an object-hash refusal not decided again on the CPU; the GPU's hashes never used | The kernel tests are also held by `tests/mutants.rs`: 12 edited kernel expressions, each caught on every run. On its first run the suite found that the BLAKE3 kernel's last-word mask was dead code, because the host already pads every message with zeros. The mask was removed and its mutant replaced. The tightened fabric test was not falsifiable at first: dropping a share on a fabric no step ran on was also caught by the claim's own sum rule, one layer down. It now requires the packer's own reason, and is falsified. ## Measured on big-buck-bunny Big-buck-bunny (596 s, 14,315 frames; the 256×144 integer proxy and the ladder's four 10-bit AV1 rungs) on the build host, shared and loaded. Every run was metered per fabric (method in the research doc, §9): - **CPU:** the kernel's account of the process's own energy, and of any VideoToolbox service the run started. - **GPU:** the rail, only where the run held at least 99% of all GPU time. - **Media engine:** no unprivileged counter on macOS 27, so a media-engine figure is its CPU side only, a lower bound. Digests, files and instruction counts are exact. Times and energies are indicative (load average 14–50 for the packs, 25–90 for the decodes). **Exact.** - Phase 0's av1-360 policy packed with the search on the GPU gives pack digest `7454acc6…1118`, phase 0's, with every file the same. This held in three runs, and with `--fabric auto` too. - Run B of #426 (four floor rungs, the ladder's CMAF) packed on the GPU gives pack digest `3912bd7c…` and receipt `e0256cd9…`, #426's own. Run C (floor-only) gives `f8871444…`, #426's own. - The media engine and dav1d agree on every frame of all four floors (SHA-256 per frame), and on every one of the 600 pieces through `wai-pack floor-decode`. - The proxy decoded on the media engine is phase 0's `proxy.y4m`, byte for byte. **Pack and verify, joules per fabric** (the GPU otherwise idle): | Run | Wall | CPU | GPU | Total | Peak | CPU baseline | |---|---:|---:|---:|---:|---:|---| | av1-360 pack, search on the GPU | 56 s | 573 J | 1,284 J | **1,857 J** | 0.95 GB | 20,299 J, 3,689 s, 3.04 GB | | av1-360 pack, `--fabric auto` | 56 s | 571 J | 1,283 J | **1,854 J** | 0.76 GB | (same) | | run B pack, search on the GPU, `--jobs 12` | 71 s | 525 J | 1,239 J | **1,763 J** | 1.62 GB | 19,541 J, 980 s (`--jobs 6`), 0.17 GB | | run C pack (floor-only), hashes on the GPU | 2.2 s | 4.4 J | 2.3 J | **6.7 J** | 0.65 GB | 3.7 J, 1.5 s, 0.04 GB | | `wai-pack verify` av1-360, CPU / hashes on the GPU | 32 / 31 s | 118 / 120 J | 0 / – | 118 / ≥120 J | 0.58 / 2.04 GB | 31–33 s | | `wai-pack verify` run B, CPU / hashes on the GPU | 34 / 32 s | 119 / 122 J | 0 / 0.2 J | 119 / 122 J | 1.55 / 3.44 GB | 35 s, 121 J | - CPU instructions per av1-360 pack: 98.3 × 10¹² → 2.05 × 10¹². - The CPU time left after the search moved (116 s of phase 0's 3,635 s) shows the search was at least 96.8% of the CPU-only pack. - `pack_verify.py` stays on the CPU by design. Its baselines (2,070 s serial and 206 s at `--jobs 12` on A) are unchanged, and it accepts the GPU-made packages, which are byte-identical to the CPU's. **The measured cost table** (`wai-pack fabric calibrate`): - search: CPU 20.2 nJ per candidate-sample; GPU 11.3 mJ per call + 1.47 nJ. - SSE: CPU 0.85 nJ per byte; GPU 3.1 mJ per call + 0.76 nJ. - BLAKE3: CPU 7 µJ per call + 1.33 nJ per byte; GPU 1.7 mJ per call + 1.43 nJ. So `auto` puts the search on the GPU and keeps SSE and the hashes on the CPU. **Decoding, CPU (dav1d) against the media engine** (the engine's own draw unmetered): | | dav1d | media engine (CPU side) | |---|---|---| | 1080p floor | 23.1 s, 198.7 J | 22.0 s, 17.4 J | | 720p floor | 12.4 s, 111.0 J | 17.3 s, 11.3 J | | 540p floor | 7.7 s, 73.3 J | 14.1 s, 8.5 J | | 360p floor | 5.1 s, 53.8 J | 11.3 s, 15.9 J | | proxy source (1080p decoded, scaled to 256×144) | 22.7 s, 304.1 J | 25.6 s, 350.5 J | The proxy gains nothing because its Lanczos scale stays on the CPU. **A metered pack, sealed.** The av1-360 pack with `--meter fabric`, run while the GPU was otherwise idle: - It sealed a `ModelBased` figure of 2,248.4 J. Its claim's fabric record gives the CPU 475.4 J (`task`) and the GPU 1,773.0 J (`rail`). - The record's steps are `base-search`, `sse` and `object-hash` on the GPU and the rest on the CPU. No search call fell back to the CPU. - `wai-pack verify` accepts it, and `pack_verify.py` holds 3622 of 3622 checks. **A refusal, as designed.** When another process held part of the GPU's time, the metered packs were refused `pack-meter` rather than sealed with a guessed GPU share. One read: "the GPU was shared: this process had 45.2 s of its 65.0 s, under 99%". ## Judged unsound, and not built - **Any step on the neural engine:** it computes in FP16, exact only to 2048, and its results are not fixed across generations. - **A hardware encoder for any step:** no vendor fixes its bytes. The floors are supplied, and the packer refuses to encode one. Of the fleet's x86_64 hosts, the Windows box's discrete GPU has no AV1 engine; jetson-hub's GPU was not probed, because its SSH needs an interactive check this work did not do. - **The platform's hardware motion estimation in place of §5.5's search:** its metric, window and determinism are undocumented. - **Dividing a shared GPU rail by GPU time**, which the repo has measured overcharging a workload 3 to 15 times. - **Media-engine and neural-engine energy on macOS 27:** no counter is live without a private entitlement. A step run there makes a metered span refuse rather than guess. - **A floor verdict from a hardware decode:** floors state no digest, and hardware can conceal an error. - **ffmpeg's P010-to-8-bit conversion in any comparison:** it rounds and dithers, and was measured differing from dav1d's own. - **GPU BLAKE3 and SSE for WAI's object sizes:** sound and built, but the measured placement keeps them on the CPU. 🤖 Generated with [Claude Code](https://claude.com/claude-code) --- **Closed:** Landed in main through #465 (the packaging train): its content was squashed onto main as 836d0255d with #426, #438 and #452 together, then the review fixes on top; #465 merged as dc0d5ca5. Of the files this PR changed, all but the ones the stacked PRs share or the review changed are byte-identical on main.
int_video_enc::encode_with takes the motion search as a MotionSearch, whose rule is the
encoder's (every candidate's cost, the least by cost, then by |dy| + |dx|, then the
earlier vector): every other step of the encode stays here, so a search that keeps the
rule gives encode's payload byte for byte. CpuSearch is the reference, and encode is
encode_with over it; a search that gives another number of vectors than the frame's
blocks is refused (field_range).

stage_index::check_object_hashed is check_object with the content hash handed in, so a
verifier can hash a component's objects in one batch on another fabric; check_object is
it over the reference hash.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
A measurement claim may carry `fabrics` (energy-measurement §6.3): the steps of the work
the figure covers, each with every fabric that ran it (cpu, gpu, media, neural) and the
backend that did, and the figure divided into one share per fabric, each attributed as
`task` (the operating system's account of the work's own energy) or `rail` (the
fabric's whole rail, the work its sole user, at least 99% of its busy time). No
attribution divides a shared rail by its users' busy time, which does not apportion
energy.

`fabrics` is the one claim member that is absent rather than null (§2.4), so every
claim sealed before it keeps its bytes and link hash. Rule 6 refuses, in order: a
fabric or attribution outside §6.3 (measurement_fabric_unknown); no step or no share,
or either out of order or repeated (measurement_fabric_order); a step on a fabric with
no share (measurement_fabric_unmetered); a share on a fabric no step ran on
(measurement_fabric_unused); shares that do not sum to the figure
(measurement_fabric_sum).

wai-rs: MeasurementClaim::fabrics, its parse, rules and serialisation, and
meter_report::emit_with_fabrics. claim_verify.py: the same rules, written from the
text. claim-conformance: 14 vectors appended after the 567 before them, which a new
test pins by digest (computed from the committed corpus before the change); one
accept, four step-2 shape refusals (null among them), the step-4 reasons, and a share
attributed by GPU time refused.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
A crate of its own, with its own lockfile, as wai-ops-gpu is: WGSL compute kernels
through wgpu 30.0.1 (Metal, Vulkan, DX12), in 32-bit words with every intermediate
bounded inside its type, so a device that executes WGSL's integer arithmetic gives the
CPU's result for every input.

- motion.wgsl: one workgroup per block, each thread the least of the candidates it
  visits by (cost, |dy| + |dx|, index), the workgroup reducing them to the least: the
  order is total, so the result is the encoder's whichever thread meets which
  candidate first. Half-pel prediction with integer rounding and clamped coordinates,
  as integer-payloads §3 defines it.
- sse.wgsl: partial sums of at most 4,096 bytes a thread (under 2^32), added on the
  host in 64 bits.
- blake3.wgsl: BLAKE3's chunk function, one thread per chunk of one message; a message
  of several chunks has its parent nodes merged on the host by the reference crate's
  subtree functions, in the tree the specification fixes.

A size a kernel does not take (a block whose cost can pass 2^32 - 1, a range past 127,
a binding past the device's) is refused before anything runs, so the caller computes
it on the CPU. A software rasteriser is not opened: it runs on the CPU.

tests/kernels.rs holds each kernel to a reference transcribed from the extension text
(the BLAKE3 crate for BLAKE3), on frames of noise, near ties, flat fields (every
candidate ties), alternating columns (two vectors tie at one length) and hard edges,
at sizes that are not whole blocks and both precisions; tests/mutants.rs edits one
expression of a kernel at a time (a tie rule, a rounding addend, a clamp bound, a
reduction bound, a square, a rotation, a permutation entry, the root flag, the chunk
counter, a block length) and requires the cases to catch each, on every adapter. With
no adapter the tests print SKIPPED, unless WAI_PACK_GPU_REQUIRE is set.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
fabric.rs names the steps of a pack and a verify (source-read, segment, base-search,
base-code, refine-code, sse, object-hash, digest, floor-read, index, io, ...) and places
each call of one on a fabric, the way an energy-aware scheduler places work: a backend
is a candidate only once it has given the CPU reference's output on qualifying inputs
(seeded frame pairs of every kind the search meets, seeded SSE and BLAKE3 inputs), and
among candidates the least `uJ per call + nJ per unit * units` of a measured cost table
wins, the CPU on a tie or where nothing was measured. The CPU is the reference and the
fallback: a call another fabric does not take runs on the CPU, with the same output.
--fabric cpu (the default), gpu, or auto:<costs.json>.

With feature `gpu`, base-search, sse and object-hash run on a GPU through wai-pack-gpu.
The packer's coder, its floors' object hashes, and the verifier's object hashes go
through the placement; the verifier hashes a component's objects in one batch and
decides any object-hash refusal again on the CPU, so its verdict is the CPU's.

With feature `fabric-meter` (macOS), --meter fabric meters the pack per fabric, without
privileges: the CPU's energy as the kernel accounts it to the process (task), and the
GPU rail (IOReport "GPU Energy"), read on its republish at each end, where the process
held at least 99% of every client's GPU time (rail); a shared GPU gives no figure, and
the pack is refused pack-meter, saying why. The claim carries the fabric record
(jwp-receipts §2.4.8). `wai-pack fabric list` shows each step's qualified backends,
`fabric calibrate` measures the cost table.

tests/fabric.rs: placement, the cost table, and the claim's record on every build; with
`gpu`, on every GPU, the search against the reference (qualifying frames, seeded noise
at every block size and range, every fixture's frame pairs, whole encodes), every golden
packed on the GPU byte for byte the committed package, every golden and package and
staged mutation given the CPU's verdict with the hashes on the GPU, and an edited kernel
refused by qualification and, forced in, moving no verdict. pack_verify.py holds a
package's fabric record to the steps of §11.6.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
`wai-pack floor-decode <package> --decoder dav1d|videotoolbox` decodes every floor
rung's CMAF files (init.mp4 and the fragments in the floor playlist's order) through
ffmpeg, with libdav1d on the CPU (film grain exported, not applied) or VideoToolbox on
the media engine (its frames required to be the hardware's: a decoder that fell back to
software fails), and prints each piece's SHA-256 of its 4:2:0 pictures at the stream's
bit depth. Informative, for comparing decoders and fabrics (packaging §11.6 item 4):
nothing refuses a floor for it.

The media engine gives AV1 frames as P010 whatever the stream's bit depth. They are read
back exactly: a 10-bit sample is the word's high ten bits, an 8-bit one is recovered only
from a shift or a bit replication, and a word with low bits set, or a 10-bit value no
8-bit value widens to, fails the picture. ffmpeg's own 16-to-8-bit conversion rounds and
dithers, so it is never used. A rung whose sequence header carries film grain is not
decoded on the media engine, which applies grain no rule fixes.

av1: SequenceHeader::film_grain_params_present reads the bit after color_config();
SequenceHeader::parse does not, so what a packager accepts is unchanged.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
Two package mutations of the metered golden (§9.4): `metered-fabrics`, whose measurement
claim carries a fabric record of two steps on the CPU and the figure as its one share,
which both verifiers accept; and `metered-fabrics-unknown-step`, a claim that keeps every
rule of jwp-receipts §2.4.8 but whose record names a step packaging §11.6 does not
define, refused measurement-claim-invalid by wai-pack and pack_verify.py alike. Every
other file of the corpus regenerates unchanged.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
The computations of a pack and a verify are named steps, each of which may run on a
CPU, a GPU, a media engine or a neural engine. Every step is a function of its inputs,
defined in integers (and floor-decode by the AV1 decoding process), and none names a
fabric:

1. a packager runs a step only on a backend that has given the reference's output on
   qualifying inputs, never on one that computes otherwise than the step's definition,
   and leaves a call a backend does not take to the CPU; so §11.2 holds across fabrics
   (amended to say so), and only a metered package's claim says where steps ran;
2. a verifier computes a check on another fabric only where the package states the
   value it compares with, and decides a mismatch again on the reference: a verdict that
   changes with the fabric is the verifier's fault;
3. no step is computed by a hardware encoder, whose output no rule fixes; a floor encoded
   on one is a §7.5 floor naming its encoder, version, device and driver;
4. a floor states no decoded-output digest, so no floor verdict comes from a hardware
   decode; wider hardware formats are read back exactly or refused, never rounded;
5. a metered package's claim may carry the fabric record (jwp-receipts §2.4.8), its steps
   §11.6's terms (§11.3 step 7); an unmetered package carries none.

Terms gain Step and Fabric; §11.4 says what the reference packager places where.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
The macOS leg, whose host has a GPU, runs wai-pack-gpu's clippy, its kernels against
their references and its mutants, and wai-pack's fabric differential with the `gpu`
feature, with WAI_PACK_GPU_REQUIRE set so that an absent adapter fails rather than
skips; each step requires its report lines. The paths include wai/wai-pack-gpu. The
test-count floors gain --test=fabric and raise --lib to the target's count.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
With several threads polling one device, a map's callback can run inside another
thread's poll, after this thread's wait for its submission has returned. The read then
found no result and handed the call back to the CPU: correct, since the CPU computes the
same vectors, but it happened to 15 of 24,930 motion-search calls of a 12-piece pack of
big-buck-bunny. The read now waits for the callback, polling without blocking until a
deadline, and fails only past it.

tests/concurrent.rs: 16 threads share one device, 9,600 calls of the search and of
BLAKE3, every one required to complete on the device with the reference's result. With
the old wait the test fails.

Also clippy's clamp and as_chunks forms.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
The fabric test held a split with a GPU share and no GPU step to pack-measurement-claim,
which the claim's own sum rule would also give were the share silently dropped; it now
requires the packer's reason, so the refusal in fabric_record is what the test holds.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
The changelog's v0.4.0 entry for packaging §11.6, the measurement claim's fabric record,
wai-pack-gpu, wai-pack's fabric dispatch, meter and floor-decode, and their conformance;
the claim corpus's README describes the fabric record's vectors.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
docs/research: WAI across fabrics, the research and the measurements
Some checks failed
WAI energy receipts / energy receipts · linux (pull_request) Has been cancelled
WAI energy receipts / energy receipts · linux-arm64 (pull_request) Has been cancelled
WAI energy receipts / energy receipts · macos (pull_request) Has been cancelled
WAI HTTP delivery / WAI HTTP · does this change touch it (pull_request) Has been cancelled
WAI HTTP delivery / HTTP corpus · falsify mutations 0/3 (pull_request) Has been cancelled
WAI HTTP delivery / HTTP corpus · falsify mutations 1/3 (pull_request) Has been cancelled
WAI HTTP delivery / HTTP corpus · falsify mutations 2/3 (pull_request) Has been cancelled
WAI HTTP delivery / HTTP corpus · falsify patches 0/3 (pull_request) Has been cancelled
WAI HTTP delivery / HTTP corpus · falsify patches 1/3 (pull_request) Has been cancelled
WAI HTTP delivery / HTTP corpus · falsify patches 2/3 (pull_request) Has been cancelled
WAI HTTP delivery / HTTP corpus · every patch and mutation in exactly one shard (pull_request) Has been cancelled
WAI HTTP delivery / HTTP corpus · verified both ways, and the differential (pull_request) Has been cancelled
WAI HTTP delivery / cache rules · controls and labelled caches (pull_request) Has been cancelled
WAI neural sink gates / does this change touch it (pull_request) Has been cancelled
WAI neural sink gates / model-free refusals of the neural sink (pull_request) Has been cancelled
wai-ops-gpu integer operators on GPUs / does this change touch it (pull_request) Has been cancelled
wai-ops-gpu integer operators on GPUs / builds · no adapter is not a pass (linux) (pull_request) Has been cancelled
wai-ops-gpu integer operators on GPUs / Metal GPU and browser WebGPU (macos) (pull_request) Has been cancelled
wai-pack / wai-pack · does this change touch it (pull_request) Has been cancelled
wai-pack / wai-pack · linux (pull_request) Has been cancelled
wai-pack / wai-pack · linux-arm64 (pull_request) Has been cancelled
wai-pack / wai-pack · macos (pull_request) Has been cancelled
wai-pack / wai-pack · independent verifier (Python) (pull_request) Has been cancelled
wai-pack / wai-pack · the floor through a media prober (pull_request) Has been cancelled
wai-pack / wai-pack · dependency licences (pull_request) Has been cancelled
wai-quantum studio and desktop / does this change touch it (pull_request) Has been cancelled
wai-quantum studio and desktop / studio · browser (pull_request) Has been cancelled
wai-quantum studio and desktop / desktop app · macos (pull_request) Has been cancelled
wai-web wasm32 / does this change touch it (pull_request) Has been cancelled
wai-web wasm32 / wai-web wasm32 build · corpus, menu and writers (pull_request) Has been cancelled
30f3ece5b7
Which fabric can run each step of a WAI pack and verify, on the fleet as probed, through
which API, with the primary sources: VideoToolbox and the media engines (AV1 decode from
M3; no AV1 encode on any Apple chip through the 2026 launches), WGSL, Metal and CUDA
integer semantics, the neural engine (FP16: no exact step), NVENC/NVDEC, QSV, VCN and
Vulkan Video; AV1 §7.2's exactness and film grain; hardware encoders' irreproducibility;
and energy without root on macOS 27 (only IOReport's GPU rail is live; the kernel's
per-task CPU energy; per-client GPU time; the sole-user rule), Linux and Windows. The
design choice each finding decided, what was judged unsound, and the big-buck-bunny
measurements: packages byte-identical on the GPU, the media engine's pictures dav1d's,
and joules per fabric against the CPU baselines.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
docs/research: a metered pack sealed with its fabric record
Some checks failed
WAI conformance / cross-arch byte-equality · linux-arm64 (pull_request) Successful in 34s
WAI conformance / quantum core and corpus under wasm32-wasip2 (wasmtime) (pull_request) Successful in 2m22s
WAI conformance / C2PA manifest emission · signed, read back, validated (pull_request) Successful in 2m18s
WAI conformance / cross-arch byte-equality · linux (pull_request) Successful in 27s
WAI energy receipts / energy receipts · linux (pull_request) Successful in 10m3s
WAI neural sink gates / model-free refusals of the neural sink (pull_request) Successful in 1m21s
WAI energy receipts / energy receipts · linux-arm64 (pull_request) Successful in 11m50s
WAI HTTP delivery / HTTP corpus · every patch and mutation in exactly one shard (pull_request) Successful in 5s
wai-ops-gpu integer operators on GPUs / builds · no adapter is not a pass (linux) (pull_request) Successful in 1m52s
WAI conformance / cargo-class corpora · linux-arm64 (pull_request) Successful in 23m42s
WAI HTTP delivery / HTTP corpus · verified both ways, and the differential (pull_request) Successful in 2m12s
WAI conformance / cargo-class corpora · linux (pull_request) Successful in 19m5s
wai-pack / wai-pack · linux (pull_request) Successful in 1m33s
WAI HTTP delivery / cache rules · controls and labelled caches (pull_request) Successful in 1m20s
wai-pack / wai-pack · linux-arm64 (pull_request) Successful in 1m37s
wai-pack / wai-pack · the floor through a media prober (pull_request) Failing after 17s
wai-pack / wai-pack · dependency licences (pull_request) Successful in 1m23s
wai-quantum studio and desktop / studio · browser (pull_request) Successful in 2m40s
wai-web wasm32 / wai-web wasm32 build · corpus, menu and writers (pull_request) Successful in 4m9s
wai-pack / wai-pack · independent verifier (Python) (pull_request) Successful in 7m15s
WAI conformance / independent verifiers · claims, JWP, key discovery and integer payloads (Python) (pull_request) Successful in 35m1s
joule-moq-web browser client / joule-moq-web · browser end-to-end (macos) (pull_request) Successful in 6m1s
standards workspaces build + test / macOS energy backends (pull_request) Successful in 9m18s
WAI conformance / cargo-class corpora · macos (pull_request) Successful in 13m39s
WAI conformance / cross-arch byte-equality · macos (pull_request) Successful in 19s
WAI conformance / codec wrappers · fuzz smoke, text decode limits, C entry points, provenance, the semantic path (pull_request) Successful in 55s
WAI energy receipts / energy receipts · macos (pull_request) Successful in 5m17s
wai-pack / wai-pack · macos (pull_request) Successful in 2m50s
wai-quantum studio and desktop / desktop app · macos (pull_request) Successful in 2m28s
wai-ops-gpu integer operators on GPUs / Metal GPU and browser WebGPU (macos) (pull_request) Successful in 2m35s
a56f22cddf
The av1-360 pack with --fabric gpu --meter fabric, run when the GPU was otherwise idle:
a ModelBased figure of 2,248.4 J divided into the CPU's 475.4 J (task) and the GPU's
1,773.0 J (rail), the claim's steps the search, SSE and hashes on the GPU and the rest on
the CPU, accepted by wai-pack verify and by pack_verify.py; and the same pack of run B,
refused pack-meter a minute later when the GPU was shared.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
dcharlot closed this pull request 2026-10-26 08:45:37 -04:00
Some checks failed
joule-moq-web browser client / joule-moq-web · does this change touch it (pull_request) Successful in 6s
joule-moq-room room service / joule-moq-room · does this change touch it (pull_request) Successful in 11s
wai-web reach builds / wai-web reach · does this change touch it (pull_request) Successful in 5s
wai-web reach builds / wai-web reach builds · the package of record, every output as recorded (pull_request) Successful in 2m45s
standards workspaces build + test / which workspaces this change touches (pull_request) Successful in 5s
WAI conformance / does this change touch it (pull_request) Successful in 5s
WAI energy receipts / does this change touch it (pull_request) Successful in 7s
WAI HTTP delivery / WAI HTTP · does this change touch it (pull_request) Successful in 11s
repository checks / lockfiles, tests in CI, workflow hygiene, spec and section references, public text (pull_request) Successful in 41s
WAI neural sink gates / does this change touch it (pull_request) Successful in 10s
WAI HTTP delivery / HTTP corpus · falsify mutations 0/3 (pull_request) Has been skipped
WAI HTTP delivery / HTTP corpus · falsify mutations 1/3 (pull_request) Has been skipped
WAI HTTP delivery / HTTP corpus · falsify mutations 2/3 (pull_request) Has been skipped
WAI HTTP delivery / HTTP corpus · falsify patches 0/3 (pull_request) Has been skipped
WAI HTTP delivery / HTTP corpus · falsify patches 1/3 (pull_request) Has been skipped
WAI HTTP delivery / HTTP corpus · falsify patches 2/3 (pull_request) Has been skipped
wai-pack / wai-pack · does this change touch it (pull_request) Successful in 6s
wai-ops-gpu integer operators on GPUs / does this change touch it (pull_request) Successful in 6s
wai-quantum studio and desktop / does this change touch it (pull_request) Successful in 7s
wai-web wasm32 / does this change touch it (pull_request) Successful in 5s
wai-web reach builds / wai-web reach builds · this tree's wai-web, every rung on its engine line (pull_request) Successful in 4m40s
joule-moq-room room service / joule-moq-room, every target (pull_request) Successful in 1m20s
WAI conformance / v2 vocabulary · manifests validate as stated (pull_request) Successful in 33s
standards workspaces build + test / joule-code-rs (joule-code/joule-code-rs, joule-code-rs, linux-arm64, true) (pull_request) Successful in 8m9s
WAI conformance / delivery telemetry · CMCD and CMSD against an independent library (pull_request) Successful in 13s
WAI conformance / cross-arch byte-equality · linux-arm64 (pull_request) Successful in 34s
WAI conformance / quantum core and corpus under wasm32-wasip2 (wasmtime) (pull_request) Successful in 2m22s
WAI conformance / C2PA manifest emission · signed, read back, validated (pull_request) Successful in 2m18s
WAI conformance / cross-arch byte-equality · linux (pull_request) Successful in 27s
WAI energy receipts / energy receipts · linux (pull_request) Successful in 10m3s
WAI neural sink gates / model-free refusals of the neural sink (pull_request) Successful in 1m21s
WAI energy receipts / energy receipts · linux-arm64 (pull_request) Successful in 11m50s
WAI HTTP delivery / HTTP corpus · every patch and mutation in exactly one shard (pull_request) Successful in 5s
wai-ops-gpu integer operators on GPUs / builds · no adapter is not a pass (linux) (pull_request) Successful in 1m52s
WAI conformance / cargo-class corpora · linux-arm64 (pull_request) Successful in 23m42s
WAI HTTP delivery / HTTP corpus · verified both ways, and the differential (pull_request) Successful in 2m12s
WAI conformance / cargo-class corpora · linux (pull_request) Successful in 19m5s
wai-pack / wai-pack · linux (pull_request) Successful in 1m33s
WAI HTTP delivery / cache rules · controls and labelled caches (pull_request) Successful in 1m20s
wai-pack / wai-pack · linux-arm64 (pull_request) Successful in 1m37s
wai-pack / wai-pack · the floor through a media prober (pull_request) Failing after 17s
wai-pack / wai-pack · dependency licences (pull_request) Successful in 1m23s
wai-quantum studio and desktop / studio · browser (pull_request) Successful in 2m40s
wai-web wasm32 / wai-web wasm32 build · corpus, menu and writers (pull_request) Successful in 4m9s
wai-pack / wai-pack · independent verifier (Python) (pull_request) Successful in 7m15s
WAI conformance / independent verifiers · claims, JWP, key discovery and integer payloads (Python) (pull_request) Successful in 35m1s
joule-moq-web browser client / joule-moq-web · browser end-to-end (macos) (pull_request) Successful in 6m1s
standards workspaces build + test / macOS energy backends (pull_request) Successful in 9m18s
WAI conformance / cargo-class corpora · macos (pull_request) Successful in 13m39s
WAI conformance / cross-arch byte-equality · macos (pull_request) Successful in 19s
WAI conformance / codec wrappers · fuzz smoke, text decode limits, C entry points, provenance, the semantic path (pull_request) Successful in 55s
WAI energy receipts / energy receipts · macos (pull_request) Successful in 5m17s
wai-pack / wai-pack · macos (pull_request) Successful in 2m50s
wai-quantum studio and desktop / desktop app · macos (pull_request) Successful in 2m28s
wai-ops-gpu integer operators on GPUs / Metal GPU and browser WebGPU (macos) (pull_request) Successful in 2m35s

Pull request closed

Sign in to join this conversation.
No reviewers
No labels
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
Transaction-Science/open-standards!438
No description provided.