WAI breaking changes and the conformance runner / WAI breaking and conform · their tests, and the independent verifiers through wai conform · linux (push) Successful in 8m44s
The service's Linux job has been red on main since f032d85d3, a different test each time. The
line it prints, "unshare: unshare failed: Operation not permitted", is not the failure: it is
the probe of the_service_runs_its_jobs_as_pid_1, run with its standard error inherited. The job
container (catthehacker/ubuntu:act-22.04 under Docker's default seccomp profile, no
CAP_SYS_ADMIN) is refused a user namespace, and the test then passed without running, and said
so only in a line the harness swallows. What failed:
- e694cb096 (run 28659): tests/contract.rs's contract test, "Connection reset by peer" on its
first health check, 9 ms after the binary started. Each test picked a port by binding
127.0.0.1:0 and closing it, and handed it to its service. A child another test forks in that
moment (the pid 1 probe, a service) inherits the listener until its exec, so the port is held
open by that copy: the test's first connection reaches it and is reset, or the next one is
refused once it closes, before the service has bound the port. Reproduced on the Framework
in catthehacker/ubuntu:act-24.04: 4 runs in 80 of the test binary failed so (reset at
contract.rs:23, refused at :18), the ports of the two tests distinct, the service alive and
listening 300 ms later. Now each service binds 127.0.0.1:0 itself and the test reads the port
from its log, as tests/depth.rs does: 0 failures in 200 runs in the same container, 0 in 20
natively.
- f032d85d3, 009e44c2d and e7c7b209b (runs 28505, 28618, 28732): tests/depth.rs's
a_client_s_quick_calls_pass_two_of_its_own_long_checks. The loaded runner refused the README's
threshold alone (J031, "the README's threshold is taken"), projecting from a pilot of the
call's own shots that it would take 28 s, past the 25 s an error-correction call may take; or
took it alone and then refused one of the two sent at once (J031, "threshold 1 #0 at 1.10 s").
Reproduced on the Framework with the test and its service held to two CPUs beside six busy
loops: the README's threshold refused alone, or taken in 21 to 22 s and the second of the
burst refused, projected at 25 to 27 s. The test now sizes its long check to the host: a
threshold refused, or taking more than 14 s alone, has its shots scaled to the 12 s its time
(or the refusal's projection) gives, at least 1000, and is sent again, at most three times; a
long check past the 10 s a request waits for a turn, with room under the 25 s for two at once.
Under that load main's test failed four runs in four (refused alone twice, refused in the
burst twice), a version that scaled refusals alone failed two in four, and the test as it is
now passed seven in seven: taken alone in 21.7 to 23.5 s and scaled to 11.3 to 12.7 s, or
refused and scaled to 10.1 and 8.0 s.
- 655b366aa (run 28688): "No space left on device" in the runner's checkout, before any test.
Not the code's.
The pid 1 test now passes without running only on the kernel's refusal of the namespaces
(EPERM, ENOSPC, EINVAL from unshare(2), read with LC_ALL=C), and says so on standard error past
the harness's capture ("SKIPPED the_service_runs_its_jobs_as_pid_1: ..."); the workflow prints
it as a warning, and its image step, which required the test's "ok", now also fails on the skip.
Any other failure of the probe fails the test. Running it on CI needs a runner setting, not a
weaker test: the job container given --security-opt seccomp=unconfined (or CAP_SYS_ADMIN), or a
host-executor label. The service now runs under unshare --kill-child: without it, killing
unshare left the service running as pid 1 of its own namespace (one left by an earlier run has
been running on the Framework for 31 hours, pid 33931 there and 1 in its namespace).
Gates, on the Framework: cargo test --locked --release passes, every test the workflow requires
by name among them (and tests/depth.rs's 42 again with its last change), and clippy --release
--all-targets -D warnings; the pid 1 test runs there natively and passes.
Planted faults the tests still catch: the deadline not held (jobs.rs: contract.rs fails on the
run past it), workers started from /proc/<std::process::id()>/exe, 1
in the namespace (the pid 1 test fails: J032, the reader could not start), no unshare on PATH
and an unshare that fails another way (both fail the pid 1 test, neither skips), and a turn lent
to a client that never goes back for the client's own waiting call (offload.rs Others::waiting:
the depth test fails, a critique refused J032 after 10 s, natively and under the load).
Co-Authored-By: Claude Opus 5.5 <[email protected]>
22cb26cae9