wai-quantum 0.6.2: the HIP matrix-instruction kernel and the Metal tensor-operation kernel #280

Merged

dcharlot merged 2 commits from wai/quantum-wmma into main 2026-10-17 04:01:04 -04:00

Conversation 0 Commits 2 Files changed 21 +1974 -49

dcharlot commented 2026-10-16 10:07:50 -04:00

Owner

Copy link

Two matrix paths, released together as wai-quantum 0.6.2 (additive; every pinned digest unchanged):

1. The HIP member's matrix-instruction kernel: every product by gfx11's bf16 WMMA instruction. Building it surfaced a hardware finding: this instruction is not exact once a negative term enters a sum.
2. The Metal member's tensor-operation kernel: every product by the shading language's 4.0 tensor operation, the path to the GPU's matrix hardware. All three Metal kernels are now measured on an M5 as well as the M3.

1. The HIP matrix-instruction kernel

wmma_limbs.hip is a second HIP kernel for the gate, for the gfx11 architectures. Every product is formed by the wavefront-of-32 16 × 16 × 16 bf16 matrix instruction, accumulated in f32, in the instruction's own operand layout. One wavefront multiplies sixteen pairs' limbs by the gate's limb weights in seven 16-column steps.

What the instruction does with negative terms

This instruction's f32 accumulation is not exact once a negative term enters a sum. I measured it on gfx1151 with the instruction alone, in a standalone program:

Dot product	Result	Exact?
1·1 + (−1)·0	0.99999994 (1 − 2^-24)	no
1·1 + (−1)·1	−5.96e-08 (−2^-24)	no
128 − 64	63.9999924	no
−8988 + 8120 − 868	−1736.00024	no
255² + 255²	130050	yes
1·1 + 1·0	1	yes

The CUDA kernel's signed top limb therefore fails the probe on this GPU. One coefficient comes out off by one, because the kernel truncates 0.99999994 to 0.

The fix is to make every operand non-negative:

• Each value is taken offset-binary, x + 2^31, so all four 8-bit limbs are unsigned.
• The instruction then forms (e + 2^31)·(a + 2^31) exactly.
• The kernel removes the offset in 64-bit integers: e·a = (e+2^31)(a+2^31) − 2^31(e+a) − 2^62, modulo 2^64.

After the fix, every case that had failed is exact. The non-integer results that IREE's HIP build gave for this GPU (0.6.0) are consistent with this behaviour, but I didn't check which instruction that build chose.

API (additive, 0.6.2)

• hip::WMMA_SOURCE, WMMA_IMPLEMENTATION, wmma_implementation_id{,_of}, WMMA_MUTANTS, wmma_mutate, WMMA_THREADS_PER_ITEM, DeviceIdentity::has_wmma.
• HipMember::wmma(): the same device and member id, under its own implementation id (hip-wmma-limbs8/…). It is probed before it is returned.
• HipError::Unsupported (new) for an architecture outside gfx11.
• The member records how many threads its kernel takes per item and launches that many.

Mutants (8)

	Mutant	Caught by
W1	rounding constant 2^28	probe
W2	shift by 29	probe
W3	nine-bit limbs, which bf16 does not hold	probe
W4	the result's rows read in another instruction's layout	probe
W5	control test inverted	certificate
W6	the sign of row 0's last term	certificate
W7	range guard never fires	range
W8	the offset's 2^62 removed at half its value	probe

Measured

On framework-128gb's integrated GPU (gfx1151, silicon):

• hip_wmma passes 9/9: 66020 probe pairs agree, all four pinned certificates reproduce, circuits up to 22 qubits match native, and 8 of 8 mutants are killed.
• hip_member (9/9, 6 of 6 mutants) and hip_sv (3/3, pin b7d129d0…) pass unchanged on the same tree.

On the build host, which has no HIP library, hip_wmma and hip_member each pass 9/9 over no device. Clippy is clean, and I checked the new workflow step's greps against the real report.

CI

• wai-quantum-hip, linux-amd job: a new step that requires 9 passed, the member line, 4 hip-wmma-limbs8 certificates and 8 of 8 mutants.
• Absent job: runs the new suite over no device.

2. The Metal tensor-operation kernel

tensor_ops.metal (metal-tensor-ops-limbs8/…, MetalMember::tensor_ops()) forms every product with the shading language's 4.0 tensor operation (matmul2d: bf16 inputs, f32 accumulation). The platform maps that operation to the GPU's matrix hardware where it has it.

• Scratch: the operation reads and writes device memory, so each threadgroup owns a scratch slice: 64 × 32 limbs, 32 × 128 weights, and 64 × 128 f32 coefficients. A 64-pair tile is one multiply, in four 32-column runs.
• Operands are offset-binary, as in the HIP kernel, so the result never depends on how an accumulator treats a negative term.
• Requirements: GPU family 1007 or later and operating system 26 or later; otherwise MetalError::Unsupported.
• Eight mutants (N1–N8): they include nine-bit limbs, every run reading the first 32 weight columns, and a wrong offset correction.

Measured (silicon)

GPU family	Kernel	Result
1009 (M3 Ultra build host, same as the macos runner)	tensor op	9/9, 66020 probe pairs, 4 pins, 22 qubits, 8 of 8 mutants
1010 (M5 Max)	integer kernel	9/9, 4 pins, 6 of 6 mutants
1010 (M5 Max)	SIMD-group matrix kernel	9/9, 4 pins, 7 of 7 mutants
1010 (M5 Max)	tensor op	9/9, 4 pins, 8 of 8 mutants

The M5 run used test binaries built on the build host and run once, niced, on the dev Mac, the fleet's only M5. Memory stayed at 56% free throughout.

With signed limbs instead, a throwaway diagnostic shows the tensor op agrees on all 66020 pairs on both generations. So Apple's path is exact with negative terms; AMD's instruction is not. The platform chooses where the operation runs, and that placement isn't observed here.

On Linux all three Metal suites pass over no device, and clippy is clean on macOS and Linux.

CI

wai-quantum-metal gains a tensor-operation step on the macos runner (family 1009): 9 passed, the member line, 4 certificates and 8 of 8 mutants. Its absent leg now runs all three suites. Family 1010 has no CI runner, so it was measured by hand.

All repo checks are clean: prose, vanity metrics, tests-in-CI, concurrency, lockfiles, orphans and public names (including the .hip file).

🤖 Generated with Claude Code

Two matrix paths, released together as wai-quantum 0.6.2 (additive; every pinned digest unchanged): 1. **The HIP member's matrix-instruction kernel**: every product by gfx11's bf16 WMMA instruction. Building it surfaced a hardware finding: this instruction is not exact once a negative term enters a sum. 2. **The Metal member's tensor-operation kernel**: every product by the shading language's 4.0 tensor operation, the path to the GPU's matrix hardware. All three Metal kernels are now measured on an M5 as well as the M3. # 1. The HIP matrix-instruction kernel `wmma_limbs.hip` is a second HIP kernel for the gate, for the gfx11 architectures. Every product is formed by the wavefront-of-32 16 × 16 × 16 bf16 matrix instruction, accumulated in f32, in the instruction's own operand layout. One wavefront multiplies sixteen pairs' limbs by the gate's limb weights in seven 16-column steps. ## What the instruction does with negative terms This instruction's f32 accumulation is **not exact once a negative term enters a sum**. I measured it on gfx1151 with the instruction alone, in a standalone program: | Dot product | Result | Exact? | |---|---|---| | `1·1 + (−1)·0` | `0.99999994` (`1 − 2^-24`) | no | | `1·1 + (−1)·1` | `−5.96e-08` (`−2^-24`) | no | | `128 − 64` | `63.9999924` | no | | `−8988 + 8120 − 868` | `−1736.00024` | no | | `255² + 255²` | `130050` | yes | | `1·1 + 1·0` | `1` | yes | The CUDA kernel's signed top limb therefore fails the probe on this GPU. One coefficient comes out off by one, because the kernel truncates `0.99999994` to 0. The fix is to make every operand non-negative: - Each value is taken offset-binary, `x + 2^31`, so all four 8-bit limbs are unsigned. - The instruction then forms `(e + 2^31)·(a + 2^31)` exactly. - The kernel removes the offset in 64-bit integers: `e·a = (e+2^31)(a+2^31) − 2^31(e+a) − 2^62`, modulo `2^64`. After the fix, every case that had failed is exact. The non-integer results that IREE's HIP build gave for this GPU (0.6.0) are consistent with this behaviour, but I didn't check which instruction that build chose. ## API (additive, 0.6.2) - `hip::WMMA_SOURCE`, `WMMA_IMPLEMENTATION`, `wmma_implementation_id{,_of}`, `WMMA_MUTANTS`, `wmma_mutate`, `WMMA_THREADS_PER_ITEM`, `DeviceIdentity::has_wmma`. - `HipMember::wmma()`: the same device and member id, under its own implementation id (`hip-wmma-limbs8/…`). It is probed before it is returned. - `HipError::Unsupported` (new) for an architecture outside gfx11. - The member records how many threads its kernel takes per item and launches that many. ## Mutants (8) | | Mutant | Caught by | |---|---|---| | W1 | rounding constant `2^28` | probe | | W2 | shift by 29 | probe | | W3 | nine-bit limbs, which bf16 does not hold | probe | | W4 | the result's rows read in another instruction's layout | probe | | W5 | control test inverted | certificate | | W6 | the sign of row 0's last term | certificate | | W7 | range guard never fires | range | | W8 | the offset's `2^62` removed at half its value | probe | ## Measured On framework-128gb's integrated GPU (gfx1151, silicon): - `hip_wmma` passes 9/9: 66020 probe pairs agree, all four pinned certificates reproduce, circuits up to 22 qubits match native, and 8 of 8 mutants are killed. - `hip_member` (9/9, 6 of 6 mutants) and `hip_sv` (3/3, pin `b7d129d0…`) pass unchanged on the same tree. On the build host, which has no HIP library, `hip_wmma` and `hip_member` each pass 9/9 over no device. Clippy is clean, and I checked the new workflow step's greps against the real report. ## CI - `wai-quantum-hip`, linux-amd job: a new step that requires 9 passed, the member line, 4 `hip-wmma-limbs8` certificates and 8 of 8 mutants. - Absent job: runs the new suite over no device. # 2. The Metal tensor-operation kernel `tensor_ops.metal` (`metal-tensor-ops-limbs8/…`, `MetalMember::tensor_ops()`) forms every product with the shading language's 4.0 tensor operation (`matmul2d`: bf16 inputs, f32 accumulation). The platform maps that operation to the GPU's matrix hardware where it has it. - **Scratch:** the operation reads and writes device memory, so each threadgroup owns a scratch slice: 64 × 32 limbs, 32 × 128 weights, and 64 × 128 f32 coefficients. A 64-pair tile is one multiply, in four 32-column runs. - **Operands** are offset-binary, as in the HIP kernel, so the result never depends on how an accumulator treats a negative term. - **Requirements:** GPU family 1007 or later and operating system 26 or later; otherwise `MetalError::Unsupported`. - **Eight mutants (N1–N8):** they include nine-bit limbs, every run reading the first 32 weight columns, and a wrong offset correction. ## Measured (silicon) | GPU family | Kernel | Result | |---|---|---| | 1009 (M3 Ultra build host, same as the `macos` runner) | tensor op | 9/9, 66020 probe pairs, 4 pins, 22 qubits, 8 of 8 mutants | | 1010 (M5 Max) | integer kernel | 9/9, 4 pins, 6 of 6 mutants | | 1010 (M5 Max) | SIMD-group matrix kernel | 9/9, 4 pins, 7 of 7 mutants | | 1010 (M5 Max) | tensor op | 9/9, 4 pins, 8 of 8 mutants | The M5 run used test binaries built on the build host and run once, niced, on the dev Mac, the fleet's only M5. Memory stayed at 56% free throughout. With signed limbs instead, a throwaway diagnostic shows the tensor op agrees on all 66020 pairs on **both** generations. So Apple's path is exact with negative terms; AMD's instruction is not. The platform chooses where the operation runs, and that placement isn't observed here. On Linux all three Metal suites pass over no device, and clippy is clean on macOS and Linux. ## CI `wai-quantum-metal` gains a tensor-operation step on the `macos` runner (family 1009): 9 passed, the member line, 4 certificates and 8 of 8 mutants. Its absent leg now runs all three suites. Family 1010 has no CI runner, so it was measured by hand. All repo checks are clean: prose, vanity metrics, tests-in-CI, concurrency, lockfiles, orphans and public names (including the `.hip` file). 🤖 Generated with [Claude Code](https://claude.com/claude-code)

dcharlot added 1 commit 2026-10-16 10:07:51 -04:00

wai-quantum 0.6.2: the HIP member's matrix-instruction kernel ... 30b8f23268

A second HIP kernel for the gate (wmma_limbs.hip, HipMember::wmma), for
the gfx11 architectures: every product formed by the wavefront-of-32
16x16x16 bf16 matrix instruction, accumulated in f32, in the
instruction's own operand layout; one wavefront takes sixteen pairs in
seven 16-column steps.

The instruction's accumulation is not exact once a negative term enters
a sum. Measured on gfx1151, on the instruction alone: 1*1 + (-1)*0
returns 1 - 2^-24, 1 - 1 returns -2^-24, 128 - 64 returns 64 - 2^-17;
non-negative sums are exact. The CUDA kernel's signed top limb fails the
probe there. So the limbs are offset-binary (x + 2^31, all unsigned) and
the kernel removes the offset in 64-bit integers:
e*a = (e+2^31)(a+2^31) - 2^31(e+a) - 2^62 mod 2^64.

Own implementation id (hip-wmma-limbs8/...), HipError::Unsupported for
other architectures, eight mutants (nine-bit limbs, the result rows in
another layout, a wrong offset correction among them).

Measured on gfx1151 (silicon): hip_wmma 9/9, 66020 probe pairs, all four
pins, 22 qubits, 8 of 8 mutants; hip_member 9/9 and hip_sv 3/3
unchanged. With no HIP, both suites pass over no device. CI: a step of
wai-quantum-hip and its absent leg.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

dcharlot changed title from wai-quantum 0.6.2: the HIP member's matrix-instruction kernel to wai-quantum 0.6.2: the HIP matrix-instruction kernel and the Metal tensor-operation kernel 2026-10-16 10:19:52 -04:00

dcharlot added 1 commit 2026-10-16 10:19:54 -04:00

wai-quantum: the Metal member's tensor-operation kernel; M5 measured ... fb08994eee

tensor_ops.metal (metal-tensor-ops-limbs8/..., MetalMember::tensor_ops):
every product formed by the shading language's 4.0 tensor operation
(matmul2d, bf16 accumulated in f32), which the platform maps to the GPU's
matrix hardware where it has it. Each threadgroup owns a scratch slice in
device memory (64x32 limbs, 32x128 weights, 64x128 f32 coefficients); a
64-pair tile is one multiply in four 32-column runs. Offset-binary limbs,
as for the HIP kernel. Family 1007+ under OS 26+, else Unsupported. Eight
mutants. Build records each kernel's language version and scratch.

Measured, silicon:
- family 1009 (M3 Ultra build host / macos runner): 9/9, 66020 probe
  pairs, four pins, 22 qubits, 8 of 8 mutants;
- family 1010 (the only M5 in the fleet, the dev Mac; test binaries
  built on the build host, run once niced): all three Metal kernels 9/9,
  four pins each, 6/6, 7/7 and 8/8 mutants.
With signed limbs instead, the tensor op agrees on all 66020 pairs on
both generations. Off macOS all three suites pass over no device. CI:
a step of wai-quantum-metal and its absent leg.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

dcharlot merged commit 2938bffc40 into main 2026-10-17 04:01:04 -04:00

dcharlot referenced this pull request from a commit 2026-10-17 04:01:05 -04:00

Merge pull request 'wai-quantum 0.6.2: the HIP matrix-instruction kernel and the Metal tensor-operation kernel' (#280) from wai/quantum-wmma into main

dcharlot referenced this pull request from a commit 2026-10-18 07:19:04 -04:00

Merge remote-tracking branch 'origin/main' into wai/d2c-m7a

Sign in to join this conversation.

Reviewers

No reviewers

Labels

Clear labels

No items

No labels

Milestone

Clear milestone

No items

No milestone

Projects

Clear projects

No items

No project

Assignees

Clear assignees

No assignees

1 participant

Notifications

Subscribe

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference

Transaction-Science/open-standards!280

No description provided.