open-standards/wai/research/cross-fabric.md

WAI across fabrics: research for the packer and the verifiers

Status. Research note, informative, not normative. It records what decided the change that lets each step of a WAI pack or verify run on the fabric that does it with the least measured energy (the CPU, a GPU, a hardware media engine, or a neural processor): packaging §11.6, jwp-receipts §2.4.8 and energy-measurement §6.3. Nothing here changes a byte of a package or a verdict.

Method. Every statement about an interface or a specification was checked against that document on 2026-10-22. Items marked [measured] were run for this note (the setup is in §9); items marked [local] were read from the platform's SDK headers on the machine that ran them. Exact counts and digests are exact. Times and energies are indicative: each table in §9 is one run, on a shared host whose load is given, and none is a sample of anything.

1. The hardware the measurements ran on

Hardware is named as the rest of the repository names it: by architecture, PCI vendor and device id, and operating-system version.

• arm64 desktops (macOS 27.0.1), each with an integrated GPU and a media engine that decodes AV1 in hardware and encodes no AV1. The measurements of §9 ran on one of them, shared with other work. Every one reports the platform's current GPU interface.
• A Windows x86_64 host with an integrated GPU of PCI vendor 0x8086, device 0x9a49, and a discrete GPU of PCI vendor 0x10de, device 0x1f15. Its vendor's published support matrix gives that discrete GPU's architecture neither AV1 decode nor AV1 encode; the integrated GPU decodes AV1, with a film-grain path its vendor does not state as bit-exact (§6.1).
• Linux x86_64 hosts with GPUs of PCI vendor 0x10de, device 0x28e1, of vendor 0x8086, device 0xa7a0, and of vendor 0x1002, device 0x1586, as determinism-tiers §5 records them. Their media engines were not probed for this note.

2. The steps, and the fabrics that can run each

Step	CPU	GPU	Media engine	Neural processor	Placed on
source-decode (a coded source to frames, before a pack)	dav1d		AV1 decode where the engine has it		either, on equal pictures (§6)
source-read, segment	yes				CPU (sequential, cheap)
base-search (the integer motion search)	reference	exact integer kernel	none: no engine runs §5.5's search, and the platform's hardware motion estimation is not it (§3)	unsound (FP16, §5)	GPU where it qualifies
base-code, refine-code (residuals, rANS)	reference		none: no engine codes WIV1 or WIR1		CPU (rANS is sequential)
sse	reference	exact (partial sums under 2^32)		unsound	by measured cost
object-hash (BLAKE3)	reference	exact (chunks on the device, parents on the host)			by measured cost
digest (SHA-256)	yes	no: one message is one chain (FIPS 180-4 §6.2.2)			CPU
floor-read (AV1 and CMAF structure)	yes				CPU
floor-decode	dav1d		AV1 decode, as above		informative only (§6.3)
index, io	yes				CPU

What this decides: the GPU takes the one step that dominates a pack's work, the motion search (at least 96.8% of the CPU-only pack's CPU time in the run of §9.2), plus SSE and BLAKE3 where the measured cost says so, which on the host of §9 it does not (§9.3). No hardware encoder computes any WAI step. The media engine decodes AV1 sources and floors. The neural processor computes nothing exact.

3. Media engines

• AV1 decode, and no AV1 encode, on the arm64 desktops. Every published specification of their media engines lists AV1 decode and no AV1 encode, and the platform's compression headers name no AV1 encoder [local]. ffmpeg's encoders for the platform's framework cover H.264, HEVC and ProRes only.
• What the decode needs. A decode session takes the sequence header OBU from an av1C record; a low-overhead OBU stream (-f obu) gives it none up front, so it does not decode there, while the ladder's CMAF (with av1C) does [measured]. The reference tool therefore decodes floors from the package's CMAF files.
• What it gives back [measured] (ffmpeg 9.0.2, macOS 27.0.1):
  • The ladder of §9 is AV1 Main profile, 10-bit (each rung's av1C). The engine's hardware frames are P010; a download as 8-bit NV12 is refused.
  • Its raw P010 frames equal, frame by frame, dav1d's planar 10-bit output repacked to P010. But ffmpeg's P010-to-8-bit conversion of them does not equal dav1d's own 10-to-8-bit conversion: that conversion rounds and dithers, so a comparison stays at the stream's bit depth or converts exactly.
  • Earlier comparisons hashed -f framemd5 outputs with stderr piped into the hash; the "difference" they showed was ffmpeg's error text. Every comparison in §9 hashes the output file, not the console.
• Hardware motion estimation. The platform offers a motion-estimation session returning 4×4 or 16×16 vectors, and documents neither the engine, its cost metric, its search window nor its determinism. It cannot compute §5.5's normative search.
• Other platforms. One vendor's published support matrix gives AV1 decode from one GPU generation and AV1 encode from a later one, and states that its decoder runs independently of the compute engine. Another's media driver decodes AV1 from its twelfth graphics generation, in its full-feature build. A third's kernel driver decodes AV1 on its third video-core generation and encodes from its fourth. Vulkan defines VK_KHR_video_decode_av1 from Vulkan 1.3.277 and VK_KHR_video_encode_av1 from 1.3.302.

4. GPU compute

• WGSL integer semantics (WGSL, W3C Candidate Recommendation Draft): i32 operations "use two's complement arithmetic and have defined overflow behavior"; >> on a signed value inserts copies of the sign bit (§8.10); an integer division by zero evaluates to e1 (§8.8); an out-of-bounds access is a dynamic error the implementation does not perform as written (§6.5.7). So a WGSL kernel that keeps every value inside its type and every index inside its buffer has one result on every conformant device. The WAI kernels do: a block's cost is at most block² · 765, refused on the host where that passes 2^32 − 1. What a translator does with the cases WGSL defines and its target languages do not is determinism-tiers §5's subject.
• The backend languages. MSL leaves signed integer overflow undefined and its atomics relaxed, so only commutative integer reductions are order-free there. The WAI kernels use no atomics: each block's minimum is a tree reduction over a total order, so it does not depend on which thread visits which candidate.
• BLAKE3 on GPUs. The specification splits input into 1 KiB chunks hashed independently. Published GPU ports measured slower than one CPU core on a discrete GPU, or than eight CPU threads on an integrated one, and none reported energy. Measured here in §9.3.

5. Neural processors

The neural processor of the arm64 desktops computes in FP16, which the platform's machine-learning framework can permit but not require, and its documentation states that its capabilities differ by hardware and software generation. FP16 represents integers exactly only up to 2048, and one 8-bit squared difference is 65025. No WAI step can run on it. The fabric term is npu, the term determinism-tiers uses for wai-ops/1-npu.

6. Determinism and conformance

6.1 AV1 decoding is exact, film grain aside

AV1 Bitstream & Decoding Process Specification, Version 1.0.0 with Errata 1, §7.2: "When film_grain_params_present is equal to 0, decoders shall produce output frames that are identical in all respects and have the same output order as those produced by the decoding process specified herein." With film grain, a conformant decoder may instead match only the pre-grain frames of §7.18.2 and add grain with no "perceptually significant differences". dav1d states that it passes the AV1 conformance streams with grain on. One GPU vendor's media driver states that its film-grain path does not guarantee a bit match, and AV1 decode defects have been fixed in more than one vendor's driver.

Tested [measured]: the media engine decoded each rung of the ladder of §9, and its pictures were compared with dav1d's (§9.1). The ladder carries no film grain: dav1d gives the same frames with grain applied and with it exported.

6.2 Hardware encoders are not reproducible

No vendor states that its hardware encoder is deterministic. One vendor's encoder guide documents split-frame encoding that turns on with the number of engines and the frame height, and another's low-power AV1 rate control runs in firmware. Software encoders are reproducible only under pinned conditions.

6.3 What it means for WAI

• Receipts. A package's bytes do not depend on where its steps ran, because every step is integer and the GPU kernels compute the same integers: the receipt, the index and every object are the CPU's (§9.1). Where the steps ran is recorded only in a metered package's measurement claim, which §11.2 already exempts, so an unmetered package records nothing machine-specific.
• §11.2 on non-deterministic hardware. No WAI step may be computed by a hardware encoder. A floor a packager is given is an input; a floor encoded on a hardware encoder is one §7.5 covers, and §11.2 does not hold its bytes.
• Verifiers' independence. A check whose expected value the package states (an object's content hash) may run anywhere: a match shows the fabric computed it, and a mismatch is computed again on the CPU before it is reported. A floor states no decoded-output digest, so a hardware decode never decides a floor verdict, and a hardware decoder may conceal an error that dav1d reports. The Python verifier stays on the CPU and shares no code with wai-rs; the GPU path is wai-pack's.

7. Measuring energy per fabric without privileges

7.1 macOS 27

• IOReport is a private interface (the SDK ships a stub and no header [local]). On macOS 27 its Energy Model CPU and neural-processor channels stay visible and stop updating for a reader without a private entitlement. [measured] on the host of §9: of 645 Energy Model channels, only GPU Energy (nJ) moved over a 6 s span during a hardware decode, a dav1d decode and idle; the CPU, per-core, neural-processor, video-encoder, memory and display channels read zero, and there is no decoder energy channel. So the media engine and the neural processor have no unprivileged counter there; a privileged sampler reads the neural processor, and nothing in this change runs privileged.
• CPU, per process: proc_pid_rusage(RUSAGE_INFO_V6) gives the kernel's account of the task's own CPU energy (ri_energy_nj) and the energy it billed to the task and serviced for others (ri_billed_energy, ri_serviced_energy), readable for one's own process or a child. A task's task share is ri_energy_nj + ri_billed_energy − ri_serviced_energy, the definition the sink-metering profiles use, so energy a system service spent for the process is in it. Other load on the host enters it only through contention.
• GPU, per process: the accelerator's user clients in the I/O registry carry each creator's pid and its accumulated GPU time (ns), readable by any process [measured]. The repository's own measurement found that dividing the shared rail by GPU time charged a decode 3 to 15 times its own GPU energy, and that a client disappears with its process. So the rail is attributed to a process only when it was the rail's sole user: the clients are sampled every 200 ms through the span, each client's GPU time differenced between samples, and the span refused when a client vanishes, a difference is negative, or the process held under 99% of the GPU time the samples saw. The rail is then sealed whole, its idle and static draw over the span included, never divided by busy time. The rail holds a reading for hundreds of milliseconds and then catches up in one step, so it is read on its next republish at each end, inside the sampled window.

7.2 Linux and Windows (not run here)

• RAPL's energy_uj became root-only in Linux 5.10, and perf's RAPL events are system-wide only, so reading CPU energy there takes an owner's act (cap_perfmon, or a group-readable energy_uj).
• NVML's total-energy query (mJ) is not marked as needing privilege; the in-kernel hwmon power and energy attributes of two other GPU drivers are world-readable.
• Windows' Energy Meter Interface is readable unelevated.

8. The design choices, and what decided them

Choice	Decided by
The GPU computes the motion search, SSE and BLAKE3 chunks, in WGSL integer kernels through wgpu, not in one platform's shading language	WGSL's defined integer arithmetic (§4) gives one result on every conformant device; the repository's GPU precedent (wai-ops-gpu) is wgpu. Exactness is claimed only on the adapters a run showed (wai-pack-gpu's crate documentation lists them), with qualification at open as the runtime gate
A backend runs a step only after it gives the reference's output on qualifying inputs; any call it cannot take runs on the CPU	§11.2 must hold on every fabric; a device or driver can be wrong (§6.1's driver defects)
Placement by a measured cost table (µJ per call + nJ per unit), the CPU on a tie or where unmeasured	[measured] the GPU's fixed cost per call (1.7 mJ for a hash, 3.1 mJ for an SSE) exceeds what a pack's calls save, while the search saves 14× per unit (§9.3); placing by fabric alone made the floor-only pack dearer (§9.2)
The fabric record lives in the measurement claim, not the receipt	the receipt and LISTING would otherwise differ by machine, voiding §11.2 for every package; the claim is already §11.2's exemption
fabrics is absent rather than null	every existing claim, and every package that carries one, keeps its bytes
Attributions task and rail only; a rail share is the whole rail, sealed only for its sole user	the repository's measurement of a 3–15× overcharge from dividing by busy time (§7.1)
The figure is an OnChipCounter reading with a declared uncertainty over its span	both readings are counters (energy-measurement §2, §2.9); a counter reading is not a model
Hardware AV1 decode for sources and for informative floor decodes; never a floor verdict	AV1 §7.2 (exact without grain); floors state no digest; hardware may conceal errors
Hardware frames read from P010 exactly, never through ffmpeg's 16-to-8-bit conversion	[measured]: that conversion differs from dav1d's (§3)
No hardware encoder computes a WAI step	§6.2
No neural-processor backend	§5

9. Measurements

Setup. A 596 s, 24 fps open-licence film, 14,315 frames: a 256×144 integer proxy and the ladder's four 10-bit AV1 rungs (1080, 720, 540 and 360 lines). One arm64 desktop (macOS 27.0.1), shared with other work; each run's load average is in its report. wai-pack at this change, release, features gpu,fabric-meter; ffmpeg 9.0.2 with dav1d 1.5.4. Each run was metered by an external sampler that read the kernel's account of the child's CPU energy before reaping it (and of every decode service the run started: hardware decodes run out of process, and a 2,000-frame decode billed 1.20 J to ffmpeg and 1.09 J to its service), and the IOReport GPU Energy rail and every client's GPU time every 200 ms, with idle brackets before and after.

Class and scope of every figure below. CPU figures are OnChipCounter readings of the process's own task energy. GPU figures are OnChipCounter readings of the whole GPU rail over the run, given only where the run held at least 99% of all GPU time; otherwise there is no GPU figure. The media engine has no counter, so a media-engine run's figure is its CPU side alone, a lower bound. Each table is one run. None was repeated, so none is a median, and none carries a measured spread; they are indicative.

9.1 Exact results

• Packages are the same bytes on the GPU. A 360-line rung's policy packed with --fabric gpu: the CPU pack's digest (7454acc6…1118), every file the same (diff -r), in three runs, and with --fabric auto:<costs> too. A four-rung pack over the ladder's own CMAF, the base coded once: pack digest 3912bd7c…b9b7f and receipt e0256cd9…, the CPU's. A floor-only pack: f8871444…e032, the CPU's.
• The media engine decodes what dav1d decodes. Each of the four rungs, all 14,315 frames: the SHA-256 of every frame in P010 equal between the engine's hardware frames and dav1d's output. And wai-pack floor-decode of the four-rung package: the same digest from both decoders for each of its 600 pieces, in the stream's own 10-bit form, the P010 words read back exactly.
• A source decoded by the media engine is the same source. The proxy decoded from the 1080-line rung on the engine, then by the same scaler: SHA-256 8d236f5b…b56d, the CPU decode's proxy byte for byte.
• Placement. In a 12-piece pack, 24,915 of 24,930 motion-search calls ran on the GPU before the map-wait fix. The other 15 were handed back to the CPU, with the same vectors.
• Instructions. The CPU-only 360-line pack retired 98.3 × 10¹² CPU instructions; the GPU pack retired 2.05 × 10¹².

9.2 Pack and verify, joules per fabric (one run each)

The GPU's other clients held under 1% of its time in every pack of this run.

Run	Wall	CPU (task)	GPU (rail)	Total	Peak memory	CPU-only baseline (task)
360-line pack, search on the GPU	56 s	573 J	1,284 J	1,857 J	0.95 GB	20,299 J, 3,689 s, 3.04 GB
360-line pack, auto (search on the GPU; SSE and hashes on the CPU)	56 s	571 J	1,283 J	1,854 J	0.76 GB	as above
four-rung CMAF pack, search on the GPU, --jobs 12	71 s	525 J	1,239 J	1,763 J	1.62 GB	19,541 J, 980 s at --jobs 6, 0.17 GB
floor-only pack, hashes on the GPU	2.2 s	4.4 J	2.3 J	6.7 J	0.65 GB	3.7 J, 1.5 s, 0.04 GB
wai-pack verify, 360-line, CPU	32 s	118 J	0	118 J	0.58 GB	31–33 s
wai-pack verify, 360-line, hashes on the GPU	31 s	120 J	none (GPU shared)	≥ 120 J	2.04 GB
wai-pack verify, four-rung, CPU	34 s	119 J	0	119 J	1.55 GB	35 s, 121 J
wai-pack verify, four-rung, hashes on the GPU	32 s	122 J	0.2 J	122 J	3.44 GB

What the table shows:

• Moving the motion search to the GPU cut the 360-line pack's energy 9 to 11 times over the two runs made: 1,857 J here and 2,248 J in the sealed run of §9.5, against a 20,299 J baseline. The baseline is CPU task energy alone, and the GPU runs add the whole rail. The CPU work left (residuals, rANS, digests, I/O: 116 s of CPU against the baseline's 3,635 s) shows that the search was at least 96.8% of the baseline's CPU time.
• Moving the object hashes or SSE gained nothing: the floor-only pack, whose only movable step is hashing, cost more on the GPU.
• wai-pack verify is dominated by the integer base's decode, which is sequential rANS, so it stays on the CPU.

9.3 The measured cost table (wai-pack fabric calibrate, one run)

Each backend, two call sizes, 6 s each, fitted to µJ per call + nJ per unit:

Step	CPU	GPU	Placement
base-search (unit: candidate-sample)	0 + 20.2 nJ	11,276 µJ + 1.47 nJ	GPU from about 600,000 units, a 32×24 frame at the policy's range; every frame of the film is 32 million
sse (unit: byte)	0 + 0.85 nJ	3,136 µJ + 0.76 nJ	CPU below about 35 MB a call; a piece of the film is 5 MB
object-hash (unit: byte)	7 µJ + 1.33 nJ	1,699 µJ + 1.43 nJ	CPU always: the GPU's BLAKE3 is no cheaper per byte either

The GPU's per-call cost includes its idle and static draw over the call, since the rail is the whole GPU's.

9.4 Decoding: CPU against the media engine (one run)

The decode service was counted; the GPU was shared in this run, and no decode used it.

Decode of the whole floor	dav1d: wall, CPU	media engine: wall, CPU (ffmpeg and its service), engine
1080 lines	23.1 s, 198.7 J	22.0 s, 17.4 J, + unmetered
720 lines	12.4 s, 111.0 J	17.3 s, 11.3 J, + unmetered
540 lines	7.7 s, 73.3 J	14.1 s, 8.5 J, + unmetered
360 lines	5.1 s, 53.8 J	11.3 s, 15.9 J, + unmetered
proxy source (1080 lines decoded and scaled to 256×144)	22.7 s, 304.1 J	25.6 s, 350.5 J, + unmetered

What the table shows:

• A floor decode on the media engine cost the CPU 8 to 11 times less at 540 to 1080 lines, and 3.4 times less at 360, where the per-frame download dominates.
• The engine's own draw is not visible to an unprivileged reader on macOS 27 (§7.1), so these are lower bounds.
• For the proxy source the engine saved nothing: the Lanczos scale and the P010 repack on the CPU cost more than the decode saved. Placing source-decode on the media engine is worth it only where the scaling also moves, and moving it would change the source's bytes unless the scaler is exact.

9.5 A metered pack, sealed (one run)

• Sealed. The 360-line pack with --fabric gpu --meter fabric, run when the GPU's other clients had held under 0.5% of its time over the preceding 10 s, sealed 2,248.4 J with the fabric record in its measurement claim: the CPU's share 475.4 J (task) and the GPU's 1,773.0 J (rail, the pack its sole user); the steps base-search, sse and object-hash on the GPU and every other step on the CPU, with no search call handed back to the CPU.
• Which meter. That run used the meter's first revision, which sealed the figure as ModelBased, took the GPU's time at the two ends only, and multiplied the rail by the pack's share of GPU time (at least 99%). The meter now seals OnChipCounter with the whole rail and samples GPU time through the span (§7.1). The run has not been repeated under it, so its figures are the first revision's.
• Verified. wai-pack verify accepts the package, and pack_verify.py holds every check of it (3622 of 3622), the claim among them.
• Refused. The same metered four-rung pack, a minute later, found the GPU shared (39.9 s of its 50.0 s) and was refused pack-meter, as was the calibration then.

9.6 Load, and what is indicative

• The host's one-minute load average was 14 to 50 over the runs of §9.2 and 25 to 90 over §9.4.
• The CPU figures are each process's own, by the kernel's attribution, so other load changes them only through contention.
• The GPU figures of §9.2 are from a run that held the GPU alone. In the run of §9.4 another process held about a third of the GPU's time; the GPU packs then had no GPU figure, and the metered packs were refused pack-meter ("the GPU was shared: this process had 45.2 s of its 65.0 s, under 99%").
• Instruction counts and digests are exact.

10. Judged unsound

• Any WAI step on the neural processor (§5).
• A hardware encoder for any WAI step: floors are supplied, the packer refuses to encode one (pack-floor-encode-unsupported), and no hardware encoder's bytes are reproducible (§6.2).
• The platform's hardware motion estimation in place of §5.5's search (§3).
• Dividing a shared GPU rail by GPU time (§7.1).
• Media-engine and neural-processor energy on macOS 27 without privileges: no live counter (§7.1); a step run there makes a metered span refuse rather than guess.
• A floor verdict from a hardware decoder (§6.3).
• ffmpeg's P010-to-8-bit conversion in a comparison (§3).