ci: wai-quantum's byte-pinning tests on nine more instruction sets #248

Merged

dcharlot merged 3 commits from wai/quantum-processors-ci into main 2026-10-14 11:50:38 -04:00

Conversation 0 Commits 3 Files changed 1 +141

dcharlot commented 2026-10-14 01:54:53 -04:00

Owner

Copy link

What

A workflow, wai-quantum on every processor, that runs wai-quantum's byte-pinning tests on nine instruction sets no runner has:

Target	How it runs	Result before this PR (by hand on the hardware host)
s390x, sparc64, powerpc64 (big-endian)	QEMU user mode	23/23 bit_identity, 2/2 fixtures, 104 library tests
powerpc64le, riscv64gc	QEMU user mode	same
armv7 (32-bit)	QEMU user mode, 4 CPUs	same
aarch64 Linux	QEMU user mode	same
loongarch64	QEMU user mode, cross's development image	same
i686 (32-bit)	directly on the x86_64 host	same

The suites:

• bit_identity: 23 digests across every numeric layer, taken natively and confirmed under wasip2.
• frame_reference: the noisy-Clifford fixtures written by an independent tool.
• The library modules that pin exact bytes: the core and its corpora, the fabric and its certificates, the strict JSON reader, receipts, energy labels and the bill of materials.

These are emulator results. They prove what each target's binary computes under QEMU, not what that silicon does. The summary line of each leg says which it is.

How it runs

Each leg runs on the container runner, and its job container is that target's cross image: linker, sysroot and QEMU, with the runner preset in its environment. Every image is pinned by digest.

• The job installs the pinned Rust (from rust-toolchain.toml) into the runner's persistent cache, then runs cargo test directly with a target directory of its own there. A build script compiled in one image fails in another with an older glibc, hence one directory per target.
• The source is the forge's archive of the exact commit. Its embedded commit id is checked before anything is built (the images' git is too old to fetch a bare commit).
• Each container is capped at 6 GiB: an uncapped emulated run of the whole library once took the 15 GiB host down.
• A host-wide lock runs one leg at a time, across every run of this workflow. Only the byte-pinning modules run under emulation, never the whole library.

The first version of this PR ran cross on the hardware runner. That runner's jobs run as a user without Docker, by design, so cross could not start its containers. Granting Docker there would give its jobs root on the host, so the legs moved to the container runner instead. s390x, armv7, i686 and loongarch64 passed in this form by hand before the push.

armv7: an emulator limit, found and bounded

With all 20 host CPUs, dense_state_vector was killed by the container's memory limit. The kernel's record shows qemu-arm at 6.2 GB resident. A 32-bit guest can address at most 4 GiB, so the excess is the emulator's own memory.

QEMU 5.1's 32-bit ARM guest grows that memory with the guest's thread count. The test spawns scoped threads per gate, one per CPU. The same QEMU version runs the aarch64 guest with 20 threads without growth.

Pinned to four CPUs, the container peaked under 1 GiB and all 23 digests passed. Four threads still take the threaded paths.

loongarch64

cross 0.2.5 has no LoongArch image. That leg uses cross's development image, pinned by digest, which carries the linker and qemu-loongarch64.

🤖 Generated with Claude Code

## What A workflow, `wai-quantum on every processor`, that runs wai-quantum's byte-pinning tests on nine instruction sets no runner has: | Target | How it runs | Result before this PR (by hand on the hardware host) | |---|---|---| | s390x, sparc64, powerpc64 (big-endian) | QEMU user mode | 23/23 bit_identity, 2/2 fixtures, 104 library tests | | powerpc64le, riscv64gc | QEMU user mode | same | | armv7 (32-bit) | QEMU user mode, 4 CPUs | same | | aarch64 Linux | QEMU user mode | same | | loongarch64 | QEMU user mode, cross's development image | same | | i686 (32-bit) | directly on the x86_64 host | same | The suites: - `bit_identity`: 23 digests across every numeric layer, taken natively and confirmed under wasip2. - `frame_reference`: the noisy-Clifford fixtures written by an independent tool. - The library modules that pin exact bytes: the core and its corpora, the fabric and its certificates, the strict JSON reader, receipts, energy labels and the bill of materials. These are emulator results. They prove what each target's binary computes under QEMU, not what that silicon does. The summary line of each leg says which it is. ## How it runs Each leg runs on the container runner, and its job container is that target's cross image: linker, sysroot and QEMU, with the runner preset in its environment. Every image is pinned by digest. - The job installs the pinned Rust (from `rust-toolchain.toml`) into the runner's persistent cache, then runs `cargo test` directly with a target directory of its own there. A build script compiled in one image fails in another with an older glibc, hence one directory per target. - The source is the forge's archive of the exact commit. Its embedded commit id is checked before anything is built (the images' git is too old to fetch a bare commit). - Each container is capped at 6 GiB: an uncapped emulated run of the whole library once took the 15 GiB host down. - A host-wide lock runs one leg at a time, across every run of this workflow. Only the byte-pinning modules run under emulation, never the whole library. The first version of this PR ran `cross` on the hardware runner. That runner's jobs run as a user without Docker, by design, so `cross` could not start its containers. Granting Docker there would give its jobs root on the host, so the legs moved to the container runner instead. s390x, armv7, i686 and loongarch64 passed in this form by hand before the push. ## armv7: an emulator limit, found and bounded With all 20 host CPUs, `dense_state_vector` was killed by the container's memory limit. The kernel's record shows `qemu-arm` at 6.2 GB resident. A 32-bit guest can address at most 4 GiB, so the excess is the emulator's own memory. QEMU 5.1's 32-bit ARM guest grows that memory with the guest's thread count. The test spawns scoped threads per gate, one per CPU. The same QEMU version runs the aarch64 guest with 20 threads without growth. Pinned to four CPUs, the container peaked under 1 GiB and all 23 digests passed. Four threads still take the threaded paths. ## loongarch64 cross 0.2.5 has no LoongArch image. That leg uses cross's development image, pinned by digest, which carries the linker and `qemu-loongarch64`. 🤖 Generated with [Claude Code](https://claude.com/claude-code)

dcharlot added 1 commit 2026-10-14 01:54:54 -04:00

ci: wai-quantum's byte-pinning tests on nine more instruction sets ... 415c914e4e

A new workflow runs bit_identity (23 digests), the frame_reference fixtures
and the byte-pinning library modules for s390x, sparc64, powerpc64 and
powerpc64le, riscv64, armv7, i686, aarch64 and loongarch64 Linux, under
user-mode QEMU on the hardware host (i686 runs on the x86_64 host directly).
Every leg passed by hand before this commit.

Each leg runs one at a time, in a container capped at 6 GiB, with a target
directory of its own. armv7's container is pinned to four CPUs: QEMU 5.1's
32-bit ARM guest grows the emulator's own memory with the guest's thread
count and passed 6 GiB with twenty. loongarch64 uses cross's development
image, pinned by digest.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

dcharlot added 1 commit 2026-10-14 02:14:40 -04:00

ci: run each processor leg in its target's image on the container runner ... 6b1708549c

The hardware runner's jobs run as a user without Docker, by design, so
cross could not start its containers there. Each leg's job container is now
that target's cross image, pinned by digest and capped at 6 GiB (armv7 also
pinned to four CPUs). The job installs the pinned Rust into the runner's
persistent cache, runs cargo directly with the image's linker and QEMU
runner, and holds a host-wide lock so legs never overlap. The source comes
as the forge's archive of the exact commit, its embedded commit id checked
(the images' git is too old to fetch a bare commit).

s390x, armv7, i686 and loongarch64 passed this way by hand before the push.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

dcharlot added 1 commit 2026-10-14 08:56:27 -04:00

ci: decompress the source archive to a file before reading its commit id ... 57240f7ac0

git get-tar-commit-id stops reading at the archive's header, so gzip, writing
into the pipe, died of a broken pipe (exit 141), and pipefail failed every
leg's fetch step before any test ran.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

dcharlot merged commit fea5aed2d0 into main 2026-10-14 11:50:38 -04:00

dcharlot referenced this pull request from a commit 2026-10-14 11:50:38 -04:00

Merge pull request 'ci: wai-quantum's byte-pinning tests on nine more instruction sets' (#248) from wai/quantum-processors-ci into main

Sign in to join this conversation.

Reviewers

No reviewers

Labels

Clear labels

No items

No labels

Milestone

Clear milestone

No items

No milestone

Projects

Clear projects

No items

No project

Assignees

Clear assignees

No assignees

1 participant

Notifications

Subscribe

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference

Transaction-Science/open-standards!248

No description provided.