ci: gate jobs get their own pool (superseded — landed directly on main as 7907de6a0) #667
2026-11-06
ci: re-run, after removing the one linux host that could not emulate aarch64
All checks were successful
standards workspaces build + test / openpay (openpay, openpay, linux-arm64) (pull_request) Successful in 9m34s
WAI conformance / delivery telemetry · CMCD and CMSD against an independent library (pull_request) Successful in 19s
WAI conformance / cross-arch byte-equality · linux-arm64 (pull_request) Successful in 47s
WAI conformance / C2PA manifest emission · signed, read back, validated (pull_request) Successful in 2m38s
WAI conformance / each feature of the independent verifiers' steps compiles alone (pull_request) Successful in 19m59s
WAI conformance / cargo-class corpora · linux-arm64 (pull_request) Successful in 23m13s
WAI energy receipts / energy receipts · linux-arm64 (pull_request) Successful in 29m42s
WAI HTTP delivery / HTTP corpus · falsify mutations 0/3 (pull_request) Successful in 40m33s
WAI HTTP delivery / HTTP corpus · falsify mutations 1/3 (pull_request) Successful in 40m0s
WAI HTTP delivery / HTTP corpus · falsify mutations 2/3 (pull_request) Successful in 40m10s
WAI HTTP delivery / cache rules · controls and labelled caches (pull_request) Successful in 2m14s
WAI HTTP delivery / HTTP corpus · verified both ways, and the differential (pull_request) Successful in 3m1s
wai-ops-npu integer operators on neural processors / builds · no device is not a pass (linux-arm64) (pull_request) Successful in 28s
wai-ops integer operators / linux-aarch64 (pull_request) Successful in 1m48s
WAI HTTP delivery / HTTP corpus · falsify patches 0/3 (pull_request) Successful in 26m39s
wai-pack / wai-pack · linux-arm64 (pull_request) Successful in 2m52s
wai-pack / wai-pack · the floor through a media prober (pull_request) Successful in 46s
wai-pack / wai-pack · dependency licences (pull_request) Successful in 24s
wai-quantum npu-cores member / member · npu-cores absent (linux-arm64) (pull_request) Successful in 38s
wai-quantum HIP member / member · HIP absent (linux-arm64) (pull_request) Successful in 1m28s
wai-quantum pulse and dynamics / the same bits · linux-arm64 (pull_request) Successful in 1m59s
wai-quantum tensor-engine member / member · tensor engine absent (linux-arm64) (pull_request) Successful in 27s
wai-quantum extension points / the registry and the example · linux-arm64 (pull_request) Successful in 3m42s
wai-quantum variational and algorithm library / parameters, gradients, optimisers, VQE, QAOA and builders · linux-arm64, release (pull_request) Successful in 2m38s
wai-pack / wai-pack · independent verifier (Python) (pull_request) Successful in 9m43s
wai-sink-android native sink library / the JNI layer and the platform-free half on the host JVM · linux-arm64 (pull_request) Successful in 3m27s
WAI HTTP delivery / HTTP corpus · falsify patches 1/3 (pull_request) Successful in 24m24s
wai-vector vector graphics / linux-aarch64 (pull_request) Successful in 4m11s
WAI HTTP delivery / HTTP corpus · falsify patches 2/3 (pull_request) Successful in 26m12s
WAI conformance / independent verifiers · claims, JWP, key discovery and integer payloads (Python) (pull_request) Successful in 1h38m28s
No change to the tree. The previous run of this branch settled at 264 green and 3 red, and all three were traced to a single host rather than to anything here. `framework-32gb-0000` was added to the `linux` label earlier today. It is the only host on that label with **no QEMU binfmt handler** -- Ubuntu 26.04 dropped `qemu-user-static`, and the box never had it -- so an emulated aarch64 job landing there hands its ELF to execvp, glibc falls back to /bin/sh and the test dies with "exited abnormally", exit 2. Confirmed by runner uuid: both failing legs (`sve 2048-bit` and `matrix extension`) ran on `7049b355…` = fw32-0000-linux, while the sve 128-bit, sve 512-bit and both riscv64 legs passed on hosts that do have the handler. Job placement is a lottery, so the symptom looked like flakiness. That runner is now stopped and disabled. Every remaining `linux` host carries `qemu-user-static 1:8.2.2`, which is the exact version these workflows record as provenance (`qemu-user 8.2.2 (the host's binfmt handler)`). Installing QEMU on that host instead was rejected deliberately: 26.04 offers only 10.2.1, and running it under a label whose workflows publish "8.2.2" would make the report state a version that did not run. A false provenance string is worse than less capacity. The third failure, `macOS energy backends`, was a job whose log stops mid-compile with no error -- an interrupted run, not a verdict. It re-runs with everything else. Forgejo 16.0.6 has no usable per-job re-run (the web route returns 404 and there is no API route), so a push is the only way to re-run a job. Co-Authored-By: Claude Opus 5 <[email protected]>
ci: re-run, after the sccache fault that failed this branch's macOS legs
All checks were successful
WAI HTTP delivery / cache rules · controls and labelled caches (pull_request) Successful in 13m19s
WAI conformance / v2 vocabulary · manifests validate as stated (pull_request) Successful in 42s
WAI HTTP delivery / HTTP corpus · verified both ways, and the differential (pull_request) Successful in 14m26s
WAI conformance / delivery telemetry · CMCD and CMSD against an independent library (pull_request) Successful in 18s
WAI conformance / cross-arch byte-equality · linux-arm64 (pull_request) Successful in 1m4s
joule-moq-room room service / joule-moq-room, every target (pull_request) Successful in 9m54s
wai cross-architecture byte-equality / linux-aarch64 (pull_request) Successful in 1m35s
wai-pack / wai-pack · linux-arm64 (pull_request) Successful in 4m31s
WAI conformance / C2PA manifest emission · signed, read back, validated (pull_request) Successful in 12m27s
wai-pack / wai-pack · the floor through a media prober (pull_request) Successful in 40s
WAI energy receipts / energy receipts · linux-arm64 (pull_request) Successful in 25m3s
WAI conformance / each feature of the independent verifiers' steps compiles alone (pull_request) Successful in 20m3s
wai-pack / wai-pack · dependency licences (pull_request) Successful in 2m25s
wai-quantum HIP member / member · HIP absent (linux-arm64) (pull_request) Successful in 1m27s
wai-quantum npu-cores member / member · npu-cores absent (linux-arm64) (pull_request) Successful in 51s
wai-pack / wai-pack · independent verifier (Python) (pull_request) Successful in 11m31s
wai-ops integer operators / linux-aarch64 (pull_request) Successful in 1m55s
wai-quantum tensor-engine member / member · tensor engine absent (linux-arm64) (pull_request) Successful in 55s
wai-quantum pulse and dynamics / the same bits · linux-arm64 (pull_request) Successful in 2m22s
wai-quantum variational and algorithm library / parameters, gradients, optimisers, VQE, QAOA and builders · linux-arm64, release (pull_request) Successful in 2m33s
wai-vector vector graphics / linux-aarch64 (pull_request) Successful in 4m34s
wai-sink-android native sink library / the JNI layer and the platform-free half on the host JVM · linux-arm64 (pull_request) Successful in 4m54s
wai-quantum extension points / the registry and the example · linux-arm64 (pull_request) Successful in 8m51s
WAI conformance / cargo-class corpora · linux-arm64 (pull_request) Successful in 39m47s
WAI conformance / independent verifiers · claims, JWP, key discovery and integer payloads (Python) (pull_request) Successful in 1h42m33s
wai-quantum CUDA member / member · CUDA (jetson-hub) (pull_request) Successful in 1m44s
wai-quantum OpenCL member / member · OpenCL GPU and CPU (jetson-hub) (pull_request) Successful in 1m0s
wai-quantum service / service energy · linux packages (pull_request) Successful in 1m56s
wai-quantum portable tensor program / tensor program · GPUs (jetson-hub) (pull_request) Successful in 30s
wai-quantum WebGPU member / member · GPU, Vulkan (jetson-hub) (pull_request) Successful in 21m1s
No change to the tree. The previous run of this branch was 244 green and 23 red,
and all 23 were one infrastructure fault rather than anything here: every failing
job inspected (13 of 13) died with
sccache: error: failed to execute compile
sccache: caused by: error reading compile response from server
`tx-mbp-m4max-128` was the only one of the three `macos` runners routing compiles
through sccache -- a `rustc-wrapper = "sccache"` line in its `~/.cargo/config.toml`,
where `tx-dev-mac` has no sccache at all and `tx-mbp-m5max-48` has the binary and no
wrapper. So a macOS job passed or failed according to which runner claimed it. Its
server had wedged, and `--show-stats` reported "Mismatch of client/server versions?",
which is what sccache prints whenever it cannot reach the server.
The wrapper is now disabled on that host, so the three macOS runners compile
identically. This commit exists only because the forge has no rerun endpoint: a tag or
branch push is the only way to re-run a job.
The gate change itself was already proven by that run -- all **62** gate contexts were
green on the new `gate` label, and every `linux`, `linux-arm64` and wasm leg passed.
Co-Authored-By: Claude Opus 5 <[email protected]>
ci: gate jobs get their own pool, so a path question stops blocking real work
All checks were successful
wai-vector vector graphics / independent checker (Python) (pull_request) Successful in 3m1s
WAI HTTP delivery / HTTP corpus · falsify patches 1/3 (pull_request) Successful in 22m10s
wai-quantum signal processor / processor and member · signal processor, emulated (pull_request) Successful in 18m42s
WAI HTTP delivery / HTTP corpus · falsify mutations 1/3 (pull_request) Successful in 32m19s
WAI HTTP delivery / HTTP corpus · verified both ways, and the differential (pull_request) Successful in 3m0s
WAI HTTP delivery / cache rules · controls and labelled caches (pull_request) Successful in 1m27s
wai-ops-npu integer operators on neural processors / builds · no device is not a pass (linux-arm64) (pull_request) Successful in 21s
wai-quantum docs, course and locales / every command and snippet in the docs · linux (pull_request) Successful in 25m33s
wai-web wasm32 / wai-web wasm32 build · corpus, menu and writers (pull_request) Successful in 8m37s
wai-ops integer operators / linux-aarch64 (pull_request) Successful in 1m4s
wai-quantum studio and desktop / desktop app · linux (pull_request) Successful in 15m40s
wai-pack / wai-pack · linux-arm64 (pull_request) Successful in 3m6s
wai formal verification (integer core) / kani (arith) (pull_request) Successful in 35m38s
wai-pack / wai-pack · the floor through a media prober (pull_request) Successful in 26s
WAI HTTP delivery / HTTP corpus · falsify patches 2/3 (pull_request) Successful in 17m21s
WAI HTTP delivery / HTTP corpus · falsify mutations 0/3 (pull_request) Successful in 40m40s
wai-quantum HIP member / member · HIP absent (linux-arm64) (pull_request) Successful in 47s
wai-quantum npu-cores member / member · npu-cores absent (linux-arm64) (pull_request) Successful in 42s
wai-pack / wai-pack · dependency licences (pull_request) Successful in 2m16s
WAI Studio / Studio in WebKit · linux (pull_request) Successful in 15m47s
wai-quantum tensor-engine member / member · tensor engine absent (linux-arm64) (pull_request) Successful in 38s
wai-quantum pulse and dynamics / the same bits · linux-arm64 (pull_request) Successful in 1m19s
wai-pack / wai-pack · independent verifier (Python) (pull_request) Successful in 6m39s
wai-sink-android native sink library / the JNI layer and the platform-free half on the host JVM · linux-arm64 (pull_request) Successful in 2m27s
wai-quantum variational and algorithm library / parameters, gradients, optimisers, VQE, QAOA and builders · linux-arm64, release (pull_request) Successful in 2m52s
WAI HTTP delivery / HTTP corpus · falsify mutations 2/3 (pull_request) Successful in 40m19s
wai-vector vector graphics / linux-aarch64 (pull_request) Successful in 2m26s
wai-quantum extension points / the registry and the example · linux-arm64 (pull_request) Successful in 4m7s
WAI energy receipts / energy receipts · linux-arm64 (pull_request) Successful in 22m42s
WAI conformance / independent verifiers · claims, JWP, key discovery and integer payloads (Python) (pull_request) Successful in 1h15m1s
Measured on the forge rather than guessed: of 918 jobs waiting in this
repository, **293 (32%) were gate jobs** -- "does this change touch it" -- and
every one of them was queued on `linux`, the most contended label in the fleet
(555 of the 918). A gate is `actions/checkout` plus
`python3 scripts/pr-touches-paths.py`: seconds of work. They were sitting behind
multi-hour Kani proofs on the same five-to-fourteen slots, and because every
other job in a workflow `needs:` the gate, nothing in that workflow could start
until one was picked up.
The mechanism built to save runner time had become the thing filling the runner
queue.
This routes them to a dedicated `gate` label. Nothing else changes: the diff is
**62 files, one line each**, `runs-on: linux` -> `runs-on: gate`, with no edit to
any step, condition or output.
Why a gate job can run anywhere, which is what makes the pool cheap: it clones
and matches paths, and compiles nothing. So the label is served by hosts that
cannot serve `linux` at all --
thor2-gate arm64 (NVIDIA Thor, 14 cores/122 GB), capacity 8
jetson-hub-gate x86_64, capacity 4
-- which adds 12 slots of gate throughput without taking a single slot away
from real work. `gate:docker://catthehacker/ubuntu:act-22.04` resolves to each
host's native architecture, so the two are interchangeable. Two providers on
purpose: with one, a host going down would stall all 62 workflows at their gate.
Two things were verified before the sweep, because a blind substitution would
have been wrong:
* The gate `name:` is followed immediately by `runs-on: linux` in 62 of 63
cases. The 63rd, `standards-build.yml:174`, is a *step* called "which
workspaces this change touches", not a job -- it has no `runs-on` and is
untouched.
* After the change all 77 workflows still parse, and every `runs-on: gate` in
the repository belongs to a job whose name is a gate. No other job moved.
What this does not fix, stated plainly: the remaining 625 waiting jobs are real
work, and 21 of the 22 open pull requests here were opened on one day. Taking a
third of the queue out of the way makes the real jobs reachable; it does not make
the fleet big enough for that rate.
Co-Authored-By: Claude Opus 5 <[email protected]>