[superseded by #181] energy tests: assert what the probe reports, not what the host's scheduler allows #178
2026-10-07
energy tests: the Linux-gated call sites a macOS run cannot see
All checks were successful
energy meters on real hardware / RAPL and NVML on jetson-hub (pull_request) Successful in 32s
standards workspaces build + test / jouleclaw-rs (pull_request) Successful in 4m18s
standards workspaces build + test / aac-rs (pull_request) Successful in 2m12s
standards workspaces build + test / joule-code-rs (pull_request) Successful in 1m36s
standards workspaces build + test / joulecontract-rs (pull_request) Successful in 50s
standards workspaces build + test / map-rs (pull_request) Successful in 55s
standards workspaces build + test / mesh-rs (pull_request) Successful in 45s
standards workspaces build + test / openpay (pull_request) Successful in 4m52s
jouleclaw public surface must stay documented / Public-surface audit (jouleclaw-rs) (pull_request) Successful in 1m2s
standards workspaces build + test / arl-rs (pull_request) Successful in 1m13s
standards workspaces build + test / eoc-rs (pull_request) Successful in 1m58s
standards workspaces build + test / jcp-rs (pull_request) Successful in 1m45s
standards workspaces build + test / joule-ui-rs (pull_request) Successful in 3m26s
standards workspaces build + test / joulehook-rs (pull_request) Successful in 1m11s
standards workspaces build + test / proof-rs (pull_request) Successful in 1m17s
standards workspaces build + test / sandbox-rs (pull_request) Successful in 2m17s
standards workspaces build + test / sense-rs (pull_request) Successful in 1m9s
standards workspaces build + test / smart-byte-rs (pull_request) Successful in 3m40s
standards workspaces build + test / jouledesk-rs (pull_request) Successful in 3m45s
standards workspaces build + test / macOS energy backends (pull_request) Successful in 10m23s
CI's jetson-hub job failed to compile `sandbox-meter`'s `tests/rapl_hardware.rs`, which is `#![cfg(target_os = "linux")]` — invisible to every run I had done. Two sites, found by compiling the Linux paths rather than by waiting for another CI round: - `tests/rapl_hardware.rs` hands `RaplPackages::live_in` a tick built on the test's own `oracle::burn`. It now returns a `TickOutcome` carrying the CPU the oracle measured and a declared spin, with a comment saying that the spin count is this test's assertion that a real spin ran and not a reading taken from the probe — the oracle stays independent of what it checks. - `probe::counter_liveness_tick`, the public tick handed to `counter_liveness` for a real counter, is `cfg(any(macos, linux))` but reached on the Linux side through `MsrRaplMeter::host`. It now returns the outcome `counter_tick` already produces. `cargo check --target aarch64-unknown-linux-gnu --all-targets` compiles the gated code and found the second one; it is a weak oracle (it does not run anything) but it is exactly the right shape for this class of error, which is a type mismatch in code the host platform never compiles. `CARGO_FEATURE_NO_NEON=1` is needed because blake3's build script otherwise wants Linux C headers. `sandbox-meter`, `sandbox-exec-microvm`, `proof-cascade`, `eoc-meter` and `jouleclaw-energy` all check clean for that target. The whole sandbox workspace cannot: `zstd-sys` needs a real cross compiler. Instead I checked directly that nothing outside those crates references the changed entry points (`counter_liveness`, `counter_liveness_tick`, the three `live_in`s, or `TickOutcome`), so the per-crate checks cover every caller. macOS unchanged: sandbox-meter 124 + 19 passed. Co-Authored-By: Claude Opus 5 <[email protected]>
energy tests: the same load-sensitive tick floor in proof-cascade and sandbox-meter
Some checks failed
energy meters on real hardware / RAPL and NVML on jetson-hub (pull_request) Failing after 12s
jouleclaw public surface must stay documented / Public-surface audit (jouleclaw-rs) (pull_request) Has been cancelled
standards workspaces build + test / jouleclaw-rs (pull_request) Has been cancelled
standards workspaces build + test / aac-rs (pull_request) Has been cancelled
standards workspaces build + test / arl-rs (pull_request) Has been cancelled
standards workspaces build + test / eoc-rs (pull_request) Has been cancelled
standards workspaces build + test / jcp-rs (pull_request) Has been cancelled
standards workspaces build + test / joule-code-rs (pull_request) Has been cancelled
standards workspaces build + test / joule-ui-rs (pull_request) Has been cancelled
standards workspaces build + test / joulecontract-rs (pull_request) Has been cancelled
standards workspaces build + test / jouledesk-rs (pull_request) Has been cancelled
standards workspaces build + test / joulehook-rs (pull_request) Has been cancelled
standards workspaces build + test / map-rs (pull_request) Has been cancelled
standards workspaces build + test / mesh-rs (pull_request) Has been cancelled
standards workspaces build + test / openpay (pull_request) Has been cancelled
standards workspaces build + test / proof-rs (pull_request) Has been cancelled
standards workspaces build + test / sandbox-rs (pull_request) Has been cancelled
standards workspaces build + test / sense-rs (pull_request) Has been cancelled
standards workspaces build + test / smart-byte-rs (pull_request) Has been cancelled
standards workspaces build + test / macOS energy backends (pull_request) Has been cancelled
The first commit fixed two tests. CI then failed #177 on a *third* copy of the same code, in `proof-cascade`, with the same assertion and nearly the same comment beside it: c >= TICK_CPU / 2, "... The tick burns thread CPU time, so host load does not shorten it; only its {TICK_WALL:?} wall-clock cap can ..." The sentence contains its own refutation: the wall cap is exactly how load shortens a tick. Rather than wait for CI to find the rest one PR at a time, I swept for the pattern and found a fourth copy in `sandbox-meter`. All four are now fixed the same way. **proof-cascade.** `spin_tick` returns a `TickOutcome` (CPU seen burned, turns of the spin loop) and the probe collects one per tick; `ProbeRun` gains `interval_wall` beside `interval_burn`. Three assertions were load-sensitive — `assert_sampling`'s per-interval floor, an aggregate floor over a whole scripted call, and a per-interval floor in the live-rail test — and all three now require that each tick's loop ran, plus that each interval reached the tick of CPU or spent the wall cap. `burn_thread_cpu` also now reports the CPU it measured rather than discarding it, which it was computing anyway. **sandbox-meter.** Same shape: `spin_for` returns a `TickOutcome`, the probe records `interval_wall` and the per-tick outcomes, and the one load-sensitive assertion (`PROBE_TICK / 2` per interval) becomes the spin-count rule plus the either-condition bound. This crate already had `assert_tick` using the either-condition rule for the tick itself, so the fix brings the interval check in line with the crate's own convention — and strengthens both, since the OR rule alone admits a tick that slept through its wall cap. `TickOutcome` becomes part of `sandbox-meter`'s public surface, because it already appears in one: `RaplPackages::live_in` and `counter_liveness` take a tick that returns it, so `sandbox-exec-microvm` has to be able to name it and build one. Its four fake-tree call sites now report a turn, with a comment saying why that is honest rather than a figure chosen to clear a threshold. Three doc comments that asserted "a tick burns a full tick however loaded the machine is" are corrected in place. That claim is what the CI failures falsified, and leaving it next to a fixed assertion would just reintroduce the mistake. Falsified in both crates: a tick that sleeps instead of spinning, and a tick skipped entirely, are caught. One sleep falsification initially looked like a pass and was a compile error — checked, not assumed. In proof-cascade a tick cut short of both stopping conditions is caught too. Whole workspaces: proof-rs 139 passed, sandbox-rs 324 passed (and from the first commit, eoc-rs 717, jouleclaw-energy 57). The two clippy warnings in proof-cascade are in untouched code — the same pair that predates the change in eoc-meter, these files being copies of one another. Co-Authored-By: Claude Opus 5 <[email protected]>
energy tests: assert what the probe reports, not what the host's scheduler allows
Some checks failed
energy meters on real hardware / RAPL and NVML on jetson-hub (pull_request) Successful in 33s
jouleclaw public surface must stay documented / Public-surface audit (jouleclaw-rs) (pull_request) Has been cancelled
standards workspaces build + test / jouleclaw-rs (pull_request) Has been cancelled
standards workspaces build + test / aac-rs (pull_request) Has been cancelled
standards workspaces build + test / arl-rs (pull_request) Has been cancelled
standards workspaces build + test / eoc-rs (pull_request) Has been cancelled
standards workspaces build + test / jcp-rs (pull_request) Has been cancelled
standards workspaces build + test / joule-code-rs (pull_request) Has been cancelled
standards workspaces build + test / joule-ui-rs (pull_request) Has been cancelled
standards workspaces build + test / joulecontract-rs (pull_request) Has been cancelled
standards workspaces build + test / jouledesk-rs (pull_request) Has been cancelled
standards workspaces build + test / joulehook-rs (pull_request) Has been cancelled
standards workspaces build + test / map-rs (pull_request) Has been cancelled
standards workspaces build + test / mesh-rs (pull_request) Has been cancelled
standards workspaces build + test / openpay (pull_request) Has been cancelled
standards workspaces build + test / proof-rs (pull_request) Has been cancelled
standards workspaces build + test / sandbox-rs (pull_request) Has been cancelled
standards workspaces build + test / sense-rs (pull_request) Has been cancelled
standards workspaces build + test / smart-byte-rs (pull_request) Has been cancelled
standards workspaces build + test / macOS energy backends (pull_request) Has been cancelled
Two tests on the macOS runner failed on load rather than on behaviour. The runner is the dev Mac, so "load" means other sessions' work, and both tests asserted a threshold on something the scheduler controls. **eoc-meter — `the_probe_burns_its_tick_of_thread_cpu_between_samples`.** It required each inter-sample gap to carry at least half the 25 ms tick of thread CPU, on the reasoning that the tick spins on the *thread's* clock and so "host load cannot fail it". That reasoning was wrong, and the test's own failure message said why: a tick also stops at `PROBE_TICK_WALL_CAP`, so a host giving this thread a sliver of a core caps the tick with little CPU to show. That is how it failed in CI. No CPU threshold can replace it, because the two outcomes that matter are not separable by time: a spin the host starved into the cap and a tick that simply *slept* both show a long wall span against little CPU. I first tried a per-turn CPU floor, reasoning that a turn is fixed work so starvation lowers turns per second and not the cost of a turn. Measuring it refuted that — a turn that sleeps 1 ms costs 13.3-15.8 µs of thread CPU against a real turn's 2.13-2.92 µs, because a blocking body's context switch is charged to the thread. Even the direction of the threshold was wrong, so the assertion came back out rather than staying in as a number that reads like a guarantee while discriminating nothing. The information has to come from the thing under test. `spin_for_cpu` now returns a `TickOutcome` — the CPU it saw itself burn, and how many turns of the spin loop it ran — and `probe_with` collects one per tick. Per tick the test requires that the loop ran at all (a tick that never span did not burn CPU, at any load) and that it stopped for one of its two stated reasons rather than early; the gaps measured by the tests' own clocks still bound it from above and keep CPU under wall. `SampleCall` gains absolute wall readings beside its CPU pair so the inter-sample gap is derivable in both clocks from one source. Falsified: a tick that sleeps instead of spinning, and a tick skipped altogether, are both caught now — the sleeping one was **not** caught by the weaker intermediate version I tried, which is how I found that version was wrong. Stated in the test's doc comment: this does not establish that the spin body is CPU-bound rather than blocking, because nothing the probe reports separates those. **jouleclaw-energy — `the_process_counter_meters_this_process_and_nothing_else`.** It required a child process to burn ≥ 0.2 J in a 300 ms wall window. The child is an ordinary-priority shell loop competing for a core, so a busy host starves it below the line — 0.147 J at load average 149. How much work a *competing* process gets done in a fixed wall window is the scheduler talking, not this counter. The test's claim is attribution, so that is what it now asserts: the child's burn against this process's window. The old pair gave a factor of two (0.2 J against 0.1 J); attribution either holds by orders of magnitude or does not hold at all, and in the starved run above it held by a factor of about 1 070. The absolute floor stays an order of magnitude lower, only to keep the check from being vacuous if the child never ran. Verified under the load that broke them: both pass three times each at load average ~190. eoc-rs workspace 717 passed; jouleclaw-energy 57 passed. The two clippy warnings in eoc-meter predate this change (checked against the stashed baseline). Co-Authored-By: Claude Opus 5 <[email protected]>