Transaction-Science/open-standards
Watch
1
0
Fork
You've already forked open-standards
 0
Code Issues Pull requests Projects Releases 17 Packages Wiki Activity Actions 160

ci(standards-build): run all five IOReport meters on macOS, and fail a step whose test never ran (merge LAST) #102

Merged
dcharlot merged 5 commits from ci/macos-energy-widen into main 2026-10-01 23:37:35 -04:00
Conversation 0 Commits 5 Files changed 2 +125 -35
2 changed files with 125 additions and 35 deletions
Download patch file Download diff file Expand all files Collapse all files

14
.github/workflows/joule-code-conformance.yml vendored
View file

@ -53,8 +53,22 @@ jobs:
joule-code/joule-code-rs/target
key: joule-code-${{ runner.os }}-${{ hashFiles('joule-code/joule-code-rs/Cargo.lock') }}
# joule-code-remediate's formal gate shells out to `z3`, and the runner
# image has no solver. Its tests assert in both branches. With a solver
# installed here, JOULE_FIX_EXPECT_Z3=1 makes its absence a failure, so
# this gate runs the real-solver branch rather than the fail-closed one.
- name: Install z3
shell: bash
run: |
SUDO=""; [ "$(id -u)" -ne 0 ] && SUDO="sudo"
$SUDO apt-get update -qq
$SUDO apt-get install -y -qq z3
z3 --version
- name: Reference library tests
working-directory: joule-code/joule-code-rs
env:
JOULE_FIX_EXPECT_Z3: '1'
run: cargo test --workspace
# joule-moq's QUIC transport and its example compile only with `quic`. Named

146
.github/workflows/standards-build.yml vendored
View file

@ -50,11 +50,11 @@ jobs:
fail-fast: false
matrix:
include:
- { name: jcp-rs, dir: jcp/jcp-rs, runner: linux, conformance: jcp-conformance }
- { name: jcp-rs, dir: jcp/jcp-rs, runner: linux, conformance: jcp-conformance, feature_tests: "-p jcp-energy --features nvml" }
- { name: sandbox-rs, dir: sandbox/sandbox-rs, runner: linux, conformance: sandbox-conformance }
- { name: smart-byte-rs, dir: smart-byte/smart-byte-rs, runner: linux }
- { name: openpay, dir: openpay, runner: linux-arm64 }
- { name: joule-code-rs, dir: joule-code/joule-code-rs, runner: linux-arm64 }
- { name: joule-code-rs, dir: joule-code/joule-code-rs, runner: linux-arm64, z3: true }
- { name: map-rs, dir: map/map-rs, runner: linux-arm64, conformance: map-conformance }
- { name: aac-rs, dir: aac/aac-rs, runner: linux-arm64, conformance: aac-conformance }
- { name: joulecontract-rs, dir: joulecontract/joulecontract-rs, runner: linux-arm64, z3: true }
@ -63,7 +63,7 @@ jobs:
- { name: sense-rs, dir: sense/sense-rs, runner: macos }
- { name: proof-rs, dir: proof/proof-rs, runner: linux, z3: true, conformance: proof-conformance }
- { name: joulehook-rs, dir: joulehook/joulehook-rs, runner: macos }
- { name: eoc-rs, dir: eoc/eoc-rs, runner: linux }
- { name: eoc-rs, dir: eoc/eoc-rs, runner: linux, feature_tests: "-p eoc-meter --features cuda" }
- { name: mesh-rs, dir: mesh/mesh-rs, runner: linux-arm64 }
- { name: joule-ui-rs, dir: joule-ui/joule-ui-rs, runner: macos }
# jouleclaw-rs is the one entry whose runner IS a correctness choice.
@ -82,11 +82,11 @@ jobs:
# failing for no apparent reason, check `df -h /mnt/lima-colima-txarm`
# on the txarm VM before suspecting the code: a full cache volume
# fails whatever happens to be building, not whatever changed.
- { name: jouleclaw-rs, dir: jouleclaw/jouleclaw-rs, runner: linux-arm64, rustflags: "-C target-feature=+fp16", cache_suffix: "-nodebug" }
- { name: jouleclaw-rs, dir: jouleclaw/jouleclaw-rs, runner: linux-arm64, rustflags: "-C target-feature=+fp16", cache_suffix: "-nodebug", feature_tests: "-p jouleclaw-energy --features rapl,nvml" }
steps:
- uses: actions/checkout@v5
# Two workspaces shell out to `z3`, and the runner image has no solver.
# Three workspaces shell out to `z3`, and the runner image has no solver.
#
# This step used to be load-bearing for correctness: without a solver the
# ladder escalated past its real rungs into a model STAND-IN that reported
@ -101,7 +101,12 @@ jobs:
# stay covered either way — that regression test uses a goal z3 cannot
# parse, so it asserts the same thing on a machine that has one.
#
# Both z3 jobs are pinned to Linux runners deliberately: the macOS runner is
# joule-code-remediate's formal gate is the third. Its tests assert in both
# branches, so they pass without a solver, and JOULE_FIX_EXPECT_Z3=1 below
# makes a missing z3 fail them where this step installed one. Otherwise CI
# would only ever run the no-solver branch.
#
# The z3 jobs are pinned to Linux runners deliberately: the macOS runner is
# a host executor with no apt. `sudo` may or may not exist depending on
# whether the container runs as root, so fall back to a bare invocation.
- name: Install z3
@ -167,8 +172,23 @@ jobs:
working-directory: ${{ matrix.dir }}
env:
RUSTFLAGS: ${{ matrix.rustflags }}
# Where z3 was installed above, its absence is a failure, not a skip.
JOULE_FIX_EXPECT_Z3: ${{ matrix.z3 && '1' || '' }}
run: cargo test --workspace
# The NVML backends sit behind features no workspace member enables
# (jcp-energy `nvml`, eoc-meter `cuda`, jouleclaw-energy `nvml`), so the
# step above never compiles them, let alone runs their tests. Here they
# build and their tests run. The container has no GPU and Docker masks
# powercap, so the hardware tests take their "unavailable, and the
# oracle agrees" branch; the rule's unit tests run in full.
- name: Feature-gated energy tests
if: matrix.feature_tests
working-directory: ${{ matrix.dir }}
env:
RUSTFLAGS: ${{ matrix.rustflags }}
run: cargo test ${{ matrix.feature_tests }}
# Run each standard's conformance suite.
#
# These suites live in a binary's `main()`, not in `#[test]`, so
@ -198,33 +218,42 @@ jobs:
RUSTFLAGS: ${{ matrix.rustflags }}
run: cargo run -q -p ${{ matrix.conformance }} --bin ${{ matrix.conformance }}
# ── The macOS energy backend is exercised NOWHERE otherwise ───────────────
# ── The macOS energy backends are exercised NOWHERE otherwise ─────────────
#
# `jcp-energy`'s IOReport meter — the Apple Silicon rung of the energy
# ladder — is behind `#[cfg(target_os = "macos")]`, and `jcp-rs` is pinned to
# the linux runner above. A platform-gated test plus a single-platform runner
# assignment is zero coverage, and it is indistinguishable from passing: the
# code compiles out, the suite reports green, and nothing has been measured.
# Same family as a feature-gated test with no `--features`, with `target_os`
# as the gate.
# Five crates carry an Apple Silicon IOReport energy meter: jcp-energy,
# proof-cascade, eoc-meter, jouleclaw-energy and sandbox-meter. Each is
# behind `#[cfg(target_os = "macos")]` (jouleclaw-energy also behind its
# `ioreport` feature), and every workspace holding one is pinned to a linux
# runner above. A platform-gated test on a single-platform runner is zero
# coverage, and it is indistinguishable from passing.
#
# This is not hypothetical. On 2026-09-19 the counter read 0 µJ across a full
# run on the dev Mac — `IoReport::open()` succeeding while `read_uj()`
# returned nothing — and by 2026-09-21 it had healed with no code change in
# between. Both observations came from a human running the suite by hand;
# CI saw neither. A meter that silently reads zero and later recovers is
# exactly the failure a human spot-check is worst at catching.
# What this job checks, and why it used to flake. On macOS 27 an unprivileged
# reader does not see the IOReport "Energy Model" CPU rails (`CPU Energy`,
# `DIE_<n>_CPU Energy`) move under load. They are republished only in rare
# lumps: on the M3 Ultra both dies held for over 22 minutes, then jumped
# about 1.3 kJ each in a single 100 ms sample. On the M5 Max the GPU rail
# stays live. The old jcp-energy test asserted that the *summed* rails
# advanced during a CPU burn, and the GPU rail satisfied that whenever
# something drew on screen. It passed by day and failed at night. The
# 2026-09-19 zero that "healed" by 09-21 was this effect, not a recovery.
#
# Deliberately narrow: `-p jcp-energy` only. `jcp-rs` stays on linux, so this
# ADDS the platform that was missing rather than trading one for the other,
# and it costs one small crate rather than a second full workspace run.
# Every meter now uses IOReport only when the CPU rail itself keeps
# advancing while the probe burns CPU: in at least 5 sample intervals, at
# least 4 in every 5, over at least 300 ms. A lump is published in steps (one
# was caught as +61.8 kJ, then +3.4 J 64 ms later), so two advancing
# intervals are still just a lump. Otherwise the meter falls back to its
# estimator rung. Its tests assert against an independent oracle that reads
# the CPU rail directly, so this job is deterministic:
# - green while each meter declines a rail that does not keep moving;
# - red if any meter selects one;
# - on a host whose rail is live, red unless CPU work meters non-zero.
#
# Verified to catch the real symptom: forcing `read_uj()` to return `Some(0)`
# makes `cumulative_advances_after_cpu_burn` fail with
# `energy must advance: before=0 after=0` — byte-identical to what was
# observed on 2026-09-19.
# Each step fails unless its test printed the `ioreport liveness:` line. That
# line is the only proof the platform-gated test ran and asserted a branch,
# rather than a filter, a feature or a cfg leaving zero tests to pass. It goes
# to stderr and is captured on success, hence `--nocapture` and `2>&1`.
macos-energy-backend:
name: macOS energy backend (jcp-energy)
name: macOS energy backends
runs-on: macos
steps:
- uses: actions/checkout@v4
@ -237,20 +266,67 @@ jobs:
curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh -s -- -y --profile minimal
echo "$HOME/.cargo/bin" >> "$GITHUB_PATH"
- name: Cache cargo registry + target
- name: Cache cargo registry + targets
uses: actions/cache@v4
with:
path: |
~/.cargo/registry
~/.cargo/git
jcp/jcp-rs/target
key: standards-jcp-energy-macos-${{ hashFiles('jcp/jcp-rs/Cargo.lock', 'rust-toolchain.toml') }}
proof/proof-rs/target
eoc/eoc-rs/target
jouleclaw/jouleclaw-rs/target
sandbox/sandbox-rs/target
key: standards-macos-energy-${{ hashFiles('jcp/jcp-rs/Cargo.lock', 'proof/proof-rs/Cargo.lock', 'eoc/eoc-rs/Cargo.lock', 'jouleclaw/jouleclaw-rs/Cargo.lock', 'sandbox/sandbox-rs/Cargo.lock', 'rust-toolchain.toml') }}
restore-keys: |
standards-jcp-energy-macos-
standards-macos-energy-
# `--lib` keeps this to the crate's own tests, which is where every
# `cfg(target_os = "macos")` block in this crate lives.
- name: Energy backend tests (the cfg(macos) blocks)
# Each crate's lib tests plus its integration tests. Each crate has a
# tests/ file that checks the liveness probe runs exactly once per process,
# and that check only holds in a fresh process. `--lib` would skip it.
- name: jcp-energy
shell: bash
working-directory: jcp/jcp-rs
run: cargo test --lib -p jcp-energy
run: |
set -o pipefail
log=$(mktemp)
cargo test -p jcp-energy -- --nocapture 2>&1 | tee "$log"
grep -q 'ioreport liveness:' "$log"
- name: proof-cascade
shell: bash
working-directory: proof/proof-rs
run: |
set -o pipefail
log=$(mktemp)
cargo test -p proof-cascade -- --nocapture 2>&1 | tee "$log"
grep -q 'ioreport liveness:' "$log"
- name: eoc-meter
shell: bash
working-directory: eoc/eoc-rs
run: |
set -o pipefail
log=$(mktemp)
cargo test -p eoc-meter -- --nocapture 2>&1 | tee "$log"
grep -q 'ioreport liveness:' "$log"
# Double-gated: `cfg(all(feature = "ioreport", target_os = "macos"))`.
# A macOS runner alone reaches none of it.
- name: jouleclaw-energy (feature AND platform gated)
shell: bash
working-directory: jouleclaw/jouleclaw-rs
run: |
set -o pipefail
log=$(mktemp)
cargo test -p jouleclaw-energy --features ioreport -- --nocapture 2>&1 | tee "$log"
grep -q 'ioreport liveness:' "$log"
- name: sandbox-meter
shell: bash
working-directory: sandbox/sandbox-rs
run: |
set -o pipefail
log=$(mktemp)
cargo test -p sandbox-meter -- --nocapture 2>&1 | tee "$log"
grep -q 'ioreport liveness:' "$log"