ci(standards-build): run all five IOReport meters on macOS, and fail a step whose test never ran (merge LAST) #102

Merged

dcharlot merged 5 commits from ci/macos-energy-widen into main 2026-10-01 23:37:35 -04:00

Conversation 0 Commits 5 Files changed 2 +125 -35

dcharlot commented 2026-09-27 03:32:12 -04:00

Owner

Copy link

Merge LAST, after #110, #114, #115, #116 and #117. Until those land, this PR's own macOS job runs main's old meter tests and goes red: they neither print the guard line nor pass on this host. That red is expected, and it is exactly what this job exists to catch.

What changed from the first version of this PR

The first version (07fb8360, now replaced) rested on a wrong premise. It added proof-cascade, eoc-meter and jouleclaw-energy to the macOS job and left sandbox-meter out, with a comment saying sandbox-meter failed where jcp-energy passed because they read IOReport differently. They don't. Both read the same CPU rail, and on macOS 27 an unprivileged reader sees that rail republished only in rare batches (the M3 Ultra held for ≥22 min, then jumped ~1.3 kJ per die in one sample).

• jcp-energy's old test asserted that the summed rails advanced. The live GPU rail satisfied that whenever something drew on screen, so it passed by day and failed at night.
• sandbox-meter's test asserted the CPU class itself, so it failed.
• The three steps the first version added passed for the same vacuous reason as jcp-energy's.

The job's header was wrong the same way. It called a 2026-09-19 zero "healed" by 09-21, and cited cumulative_advances_after_cpu_burn as proof that the job catches the real symptom. That test was GPU-satisfied, and #110 deletes it.

What it does now

• Runs the lib tests of all five IOReport meters on the macOS runner: jcp-energy, proof-cascade, eoc-meter, jouleclaw-energy (--features ioreport, because it is double-gated) and sandbox-meter.
• Fails a step whose frozen-rail test never ran. Each step runs with --nocapture and fails unless the output contains ioreport liveness:, which only that test prints. A filter, a missing feature or a cfg can no longer turn zero tests into a green step. Without --nocapture, a passing test's eprintln! is captured, which is why that line never appeared in the old job's log.
• With the five backend PRs merged, the job is deterministic:
  • green while every meter declines a CPU rail that does not keep moving;
  • red if any meter selects one;
  • on a host whose rail is live, red unless CPU work meters non-zero.
• Load-robust. Those tests were verified 10/10 under 2×-core artificial load on the runner machine itself.

The guard, observed on this PR's own run (#765, on main's meter code)

The jcp-energy step ran main's tests: 8 passed, including ioreport::tests::cumulative_advances_after_cpu_burn ... ok, which the GPU rail satisfied at the time. The step then exited 1, because no frozen-rail test printed ioreport liveness:. The old job would have reported that run green. Once #110 lands, the same step runs the new tests, which print the line.

One file: .github/workflows/standards-build.yml.

Later commits on this branch

• 8ba3b72a: the five macOS steps run each meter's lib and integration tests; --lib is dropped. Every crate now has a tests/ file checking that its liveness probe runs exactly once per process. That check only holds in a fresh process, and --lib would have skipped it.
• 47a2ac77: the job's comment describes the final rule. Every summed rail must keep advancing in at least 5 intervals, in 4 of every 5, over at least 300 ms. One lump, even one published in two steps, is not a live rail.
• ae4c2dd8: joule-code-remediate's formal gate now runs against a real z3 in CI.
  • joule-code-rs joins the z3: true matrix entries, and the joule-code conformance gate installs z3 too.
  • Both set JOULE_FIX_EXPECT_Z3=1, so a missing solver fails the tests instead of quietly taking the no-solver branch.
  • It pairs with #114, which makes those tests assert in both branches.

Merge order: #110, #114, #115, #116 and #117, then #119 (specs), then the stacked #120–#124, then this PR last.

Feature-gated NVML tests (c694a4c3)

Three features are enabled by no workspace member, so cargo test --workspace never compiled the NVML backends:

• jcp-energy nvml;
• eoc-meter cuda;
• jouleclaw-energy nvml.

A matrix field feature_tests now runs those three crates once more with the feature on. #129's Implausible rule is among the code this now covers. The container has no GPU and Docker masks powercap, so the hardware tests take their "unavailable, and the oracle agrees" branch, and the unit tests run in full. This PR still merges last, after #129.

🤖 Generated with Claude Code

> **Merge LAST, after #110, #114, #115, #116 and #117.** Until those land, this PR's own macOS job runs main's old meter tests and goes red: they neither print the guard line nor pass on this host. That red is expected, and it is exactly what this job exists to catch. ### What changed from the first version of this PR The first version (`07fb8360`, now replaced) rested on a wrong premise. It added proof-cascade, eoc-meter and jouleclaw-energy to the macOS job and left sandbox-meter out, with a comment saying sandbox-meter failed where jcp-energy passed *because they read IOReport differently*. **They don't.** Both read the same CPU rail, and on macOS 27 an unprivileged reader sees that rail republished only in rare batches (the M3 Ultra held for ≥22 min, then jumped ~1.3 kJ per die in one sample). - **jcp-energy's old test** asserted that the *summed* rails advanced. The live GPU rail satisfied that whenever something drew on screen, so it passed by day and failed at night. - **sandbox-meter's test** asserted the CPU class itself, so it failed. - **The three steps the first version added** passed for the same vacuous reason as jcp-energy's. The job's header was wrong the same way. It called a 2026-09-19 zero "healed" by 09-21, and cited `cumulative_advances_after_cpu_burn` as proof that the job catches the real symptom. That test was GPU-satisfied, and #110 deletes it. ### What it does now - **Runs the lib tests of all five IOReport meters on the macOS runner:** jcp-energy, proof-cascade, eoc-meter, jouleclaw-energy (`--features ioreport`, because it is double-gated) and sandbox-meter. - **Fails a step whose frozen-rail test never ran.** Each step runs with `--nocapture` and fails unless the output contains `ioreport liveness:`, which only that test prints. A filter, a missing feature or a cfg can no longer turn zero tests into a green step. Without `--nocapture`, a passing test's `eprintln!` is captured, which is why that line never appeared in the old job's log. - **With the five backend PRs merged, the job is deterministic:** - green while every meter declines a CPU rail that does not keep moving; - red if any meter selects one; - on a host whose rail is live, red unless CPU work meters non-zero. - **Load-robust.** Those tests were verified 10/10 under 2×-core artificial load on the runner machine itself. ### The guard, observed on this PR's own run (#765, on main's meter code) The jcp-energy step ran main's tests: `8 passed`, including `ioreport::tests::cumulative_advances_after_cpu_burn ... ok`, which the GPU rail satisfied at the time. The step then exited 1, because no frozen-rail test printed `ioreport liveness:`. **The old job would have reported that run green.** Once #110 lands, the same step runs the new tests, which print the line. One file: `.github/workflows/standards-build.yml`. ### Later commits on this branch - **`8ba3b72a`:** the five macOS steps run each meter's lib **and integration** tests; `--lib` is dropped. Every crate now has a `tests/` file checking that its liveness probe runs exactly once per process. That check only holds in a fresh process, and `--lib` would have skipped it. - **`47a2ac77`:** the job's comment describes the final rule. Every summed rail must keep advancing in at least 5 intervals, in 4 of every 5, over at least 300 ms. One lump, even one published in two steps, is not a live rail. - **`ae4c2dd8`:** joule-code-remediate's formal gate now runs against a real z3 in CI. - joule-code-rs joins the `z3: true` matrix entries, and the joule-code conformance gate installs z3 too. - Both set `JOULE_FIX_EXPECT_Z3=1`, so a missing solver fails the tests instead of quietly taking the no-solver branch. - It pairs with #114, which makes those tests assert in both branches. **Merge order:** #110, #114, #115, #116 and #117, then #119 (specs), then the stacked #120–#124, then **this PR last**. ### Feature-gated NVML tests (c694a4c3) Three features are enabled by no workspace member, so `cargo test --workspace` never compiled the NVML backends: - jcp-energy `nvml`; - eoc-meter `cuda`; - jouleclaw-energy `nvml`. A matrix field `feature_tests` now runs those three crates once more with the feature on. #129's `Implausible` rule is among the code this now covers. The container has no GPU and Docker masks powercap, so the hardware tests take their "unavailable, and the oracle agrees" branch, and the unit tests run in full. This PR still merges last, after #129. 🤖 Generated with [Claude Code](https://claude.com/claude-code)

dcharlot added 1 commit 2026-09-27 03:32:12 -04:00

ci(standards-build): the other IOReport meters were unexercised too ... 07fb8360f8

`jcp-energy` was never the only one. Six crates in this repo each carry their own
Apple-Silicon IOReport meter, and every workspace holding one is pinned to a
linux runner — so none of them had ever run on the platform whose private API
they call. #100 closed one of six.

This adds the three that pass: `proof-cascade`'s meter, `eoc-meter`, and
`jouleclaw-energy`. Measured before adding, not after.

`jouleclaw-energy` is the interesting one: it is gated
`#[cfg(all(feature = "ioreport", target_os = "macos"))]`, so a macOS runner alone
reaches none of it. Without the feature the crate reports **23** tests, with it
**29** — six tests hinge on the flag as well as the platform. Both directions
measured rather than assumed, which is the only way that difference is visible.

## sandbox-meter is deliberately absent, and why

It fails on macOS today, and not environmentally. Its `CPU Energy` channel
returns the **identical raw value on every successive read** across a 0.4 s CPU
burn — verified directly: three reads, one value, `6581800904` each time, while
`GPU Energy` advanced normally in the same snapshots. So `soc_cpu_uj` is
structurally 0 while `PowerMeter::is_hardware()` still reports true, and the
crate's own assertion `a busy window costs SoC energy` catches it.

The cause is a real implementation difference, not machine state. Both crates
were run back-to-back three times on the same host at the same moment:
`jcp-energy` passed 3/3, `sandbox-meter` failed 3/3. `jcp-energy` reads through
`IOReportCreateSamplesDelta`; `sandbox-meter` reads the raw cumulative field
from `IOReportCreateSamples`, which on this OS does not advance for that channel.
Channel discovery is not the problem — the name matches, the `mJ` unit label is
recognised and the conversion is correct. Only the value is frozen.

Two earlier hypotheses were wrong and are worth recording so nobody re-runs
them: the channel is **not** missing or renamed (`"CPU Energy"` and
`"GPU Energy"` are both present in the enumerated channel list), and `to_uj`
does **not** reject the unit (it returns `Some` for both `mJ` and `nJ`).

Adding it here would make this job permanently red, and repairing a meter is not
a CI change — it is a correctness fix to shipped energy measurement, on a claim
(`is_hardware`) that a receipt rests on. It belongs in its own change with its
own falsification, not smuggled into a workflow commit.

Verified: all four steps run green locally with the exact commands the job uses.

Co-Authored-By: Claude Opus 5 <[email protected]>

dcharlot referenced this pull request 2026-09-29 10:37:22 -04:00

jcp-energy: IOReport only when its CPU rail keeps advancing, never on a batch refresh #110

dcharlot referenced this pull request from a commit 2026-09-30 08:04:48 -04:00

jcp-energy: the liveness rule pairs consecutive samples, and tests pin each step

dcharlot referenced this pull request from a commit 2026-09-30 09:06:52 -04:00

jcp-energy: the probe burns thread CPU, and its sampler answers to the oracle

dcharlot referenced this pull request 2026-09-30 14:31:46 -04:00

sandbox-meter: a CPU counter that does not keep moving is never a measurement, and no consumer relabels an estimate #114

dcharlot referenced this pull request 2026-09-30 14:31:48 -04:00

proof-cascade: never meter CPU work through an IOReport CPU rail that does not keep moving #115

dcharlot referenced this pull request 2026-09-30 14:31:50 -04:00

eoc-meter: never build the IOReport counter on a CPU rail that does not keep moving #116

dcharlot referenced this pull request 2026-09-30 14:31:52 -04:00

jouleclaw-energy: IOReport only when its CPU rail keeps moving; consumers never zero-fill or relabel #117

dcharlot force-pushed ci/macos-energy-widen from 07fb8360f8 to cfbaf43f70 2026-09-30 14:32:34 -04:00 Compare

dcharlot changed title from ci(standards-build): the other IOReport meters were unexercised too to ci(standards-build): run all five IOReport meters on macOS, and fail a step whose test never ran (merge LAST) 2026-09-30 14:32:46 -04:00

dcharlot referenced this pull request from a commit 2026-09-30 17:59:01 -04:00

jcp-energy: the production probe runs under test, and no pin reads wall time

dcharlot added 1 commit 2026-09-30 18:00:08 -04:00

ci(standards-build): run each IOReport meter's integration tests on macOS too ... 8ba3b72a04

Each of the five meter crates now has a tests/ file that checks its CPU-rail
liveness probe runs exactly once per process: jcp-energy
(ioreport_probed_once), sandbox-meter (ioreport_probe_once_per_process),
proof-cascade (ioreport_once, ioreport_unopenable), eoc-meter
(liveness_probe_once_per_process) and jouleclaw-energy. Those checks only
hold in a fresh process, so they are integration tests. `cargo test --lib`
skipped them, so a per-thread or re-probing cache would have passed this job.
Dropping `--lib` runs them too, and the `ioreport liveness:` guard is
unchanged.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

dcharlot added 1 commit 2026-09-30 18:58:53 -04:00

ci(standards-build): describe the liveness rule the meters now use ... 47a2ac7706

The job's comment said a meter uses IOReport once the CPU rail advances in
two separate intervals. The five meters have since tightened that rule. A
refresh can be published in steps: one was caught as +61.8 kJ, then +3.4 J
64 ms later. So two advancing intervals can still be a single refresh.
Now the rail has to advance in at least 5 intervals, in at least 4 of every
5, over at least 300 ms. Comment only.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

dcharlot added 1 commit 2026-09-30 20:12:10 -04:00

ci: run joule-code-remediate's formal gate against a real z3 ... ae4c2dd80e

joule-code-remediate's formal gate shells out to z3, and neither job that
tests it installed a solver: joule-code-rs in standards-build, and the
joule-code conformance gate. Its z3 tests used to return early without a
solver, so CI passed without ever running the real-solver path.
sandbox-meter/cpu-rail-liveness (#114) makes those tests assert in both
branches, and adds JOULE_FIX_EXPECT_Z3=1 to turn a missing solver into a
failure.

joule-code-rs now joins the `z3: true` matrix entries, as
joulecontract-rs already does on the same runner kind. The conformance
gate installs z3 the same way. Both set JOULE_FIX_EXPECT_Z3=1 where z3
was installed. Elsewhere the variable is empty, and only joule-code-
remediate reads it.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

dcharlot referenced this pull request from a commit 2026-09-30 23:07:57 -04:00

op-vault: test that a token is independent of the PAN, not free of "4242"

dcharlot referenced this pull request 2026-09-30 23:08:06 -04:00

op-vault: test that a token is independent of the PAN, not free of "4242" #126

dcharlot added 1 commit 2026-10-01 02:13:18 -04:00

ci: build and run the energy meters' NVML-gated tests ... c694a4c380

jcp-energy `nvml`, eoc-meter `cuda` and jouleclaw-energy `nvml` are
features no workspace member enables, so `cargo test --workspace` never
compiled the NVML backends, nor ran the tests behind them. That includes
#129's plausibility rule, which marks a counter that runs past its board
Implausible. A matrix field `feature_tests` now runs those crates once
more with the feature on.

In the container there is no GPU and powercap is masked, so the hardware
tests take their "unavailable, and the oracle agrees" branch. The unit
tests behind the features run in full.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

dcharlot force-pushed ci/macos-energy-widen from c694a4c380 to 4e23637dba 2026-10-01 18:40:41 -04:00 Compare

dcharlot merged commit 117bad22c3 into main 2026-10-01 23:37:35 -04:00

dcharlot referenced this pull request from a commit 2026-10-01 23:37:37 -04:00

Merge pull request 'ci(standards-build): run all five IOReport meters on macOS, and fail a step whose test never ran (merge LAST)' (#102) from ci/macos-energy-widen into main

Sign in to join this conversation.

Reviewers

No reviewers

Labels

Clear labels

No items

No labels

Milestone

Clear milestone

No items

No milestone

Projects

Clear projects

No items

No project

Assignees

Clear assignees

No assignees

1 participant

Notifications

Subscribe

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference

Transaction-Science/open-standards!102

No description provided.