eoc-meter: RAPL reads each package's energy-status MSR where powercap lists none #627

Merged
dcharlot merged 2 commits from eoc-meter/rapl-msr into main 2026-11-05 04:53:00 -05:00
Owner

Some kernels ship without the powercap RAPL driver (Ubuntu without linux-modules-extra, as on some rented bare-metal servers). On those hosts /sys/class/powercap lists no package even though the counters are there. This ports the MSR fallback from #618 (jouleclaw-energy) into this meter, with no dependency added: the standards do not depend on one another.

Where powercap lists no package, each package's MSR_PKG_ENERGY_STATUS is read on that package's first CPU through /dev/cpu/<n>/msr:

  • Registers: 0x606 and 0x611 on Intel family 6; 0xC0010299 and 0xC001029B on AMD Zen and Hygon.
  • Count: the 32-bit count is converted at the package's own unit (bits 12:8 of the power-unit MSR) and wraps at 2^32 counts.
  • Zero unit: a unit that reads zero means no counter.
  • Unchanged: powercap is still preferred, and the liveness rule is the same: every package readable and live, or no figure.

eoc-meter specifics

  • Probe: records where its packages are read (the powercap directory, or / for MSR packages), and the counter reads them there. Zones are picked by name, so a listed but unreadable package zone stays with powercap.
  • Docs and errors: the CounterStatus::Unavailable and rapl_status docs say why an unprivileged MSR host is Unavailable (no CAP_SYS_RAWIO), and read errors name the MSR when that is the source.
  • tests/rapl_hardware.rs: its oracle read powercap only, so as root on an MSR host it would have disagreed with the meter. It now has its own MSR reader and brackets the meter's figure with MSR reads.
  • Unit tests: 6 new MSR tests on fake device files.

Verified

  • Linux x86_64: cargo test -p eoc-meter (default and cuda) passes; clippy is clean; cargo check --workspace --all-targets is clean.
  • Pre-existing failures, not from this change:
    • Clippy with --features cuda fails on 3 type-complexity lints in tests/nvml_hardware.rs. CI doesn't lint that configuration.
    • cargo test --workspace failed one eoc binary test, a_hit_is_charged_an_estimate_calibrated_on_this_host, under workspace load ("rapl: CounterHeld"). That host has a powercap zone, so the new code doesn't run there, and alone the test passes 3 of 3 on this branch and on main.
  • macOS (CI step): cargo test -p eoc-meter passes with the ioreport liveness: grep; clippy is clean.
  • Two bare-metal servers with no powercap zone, as root:
    • Xeon E-2278G: Live, 12/12 intervals; 8.56 J over a 300 ms burn (28.5 W), inside the oracle's MSR bracket; detect() picks rapl.
    • Two Xeon Silver 4215R sockets: both Live 12/12; 23.07 J (76.9 W).
  • Same servers, unprivileged: Unavailable, the oracle agrees, and detect() falls back.

Second commit: die scope

Linux scopes Intel's RAPL MSRs per die and AMD's and Hygon's per package. arch/x86/events/rapl.c says "AMD/HYGON platforms have a per-PKG package energy counter" and "CLX-AP is multi-die and its RAPL MSRs are die-scope". The powercap driver names a multi-die Intel package's zones package-<p>-die-<d>, one top-level zone per die.

  • MSR fallback: reading each package on its first CPU counted one die of a multi-die Intel package, an under-count. A domain is now (physical_package_id, die_id) on Intel and the package on AMD and Hygon. Each domain is read on its own first CPU and they are summed. Single-die parts are unchanged.
  • Powercap names: the rule now also accepts package-<n>-die-<m>, so a multi-die host's die zones count as packages. It accepts nothing looser.
  • Hardware test oracle: scopes MSR domains the same way, using its own code.

New unit tests: a two-die Intel package is two domains, summed; an AMD package whose CPUs list several die_ids is one domain; a two-die powercap tree sums both die zones; and the name rule's accepted names and near misses. Tests and clippy pass as before on Linux and macOS.

🤖 Generated with Claude Code

Some kernels ship without the powercap RAPL driver (Ubuntu without `linux-modules-extra`, as on some rented bare-metal servers). On those hosts `/sys/class/powercap` lists no package even though the counters are there. This ports the MSR fallback from #618 (`jouleclaw-energy`) into this meter, with no dependency added: the standards do not depend on one another. Where powercap lists no package, each package's `MSR_PKG_ENERGY_STATUS` is read on that package's first CPU through `/dev/cpu/<n>/msr`: - **Registers:** 0x606 and 0x611 on Intel family 6; 0xC0010299 and 0xC001029B on AMD Zen and Hygon. - **Count:** the 32-bit count is converted at the package's own unit (bits 12:8 of the power-unit MSR) and wraps at 2^32 counts. - **Zero unit:** a unit that reads zero means no counter. - **Unchanged:** powercap is still preferred, and the liveness rule is the same: every package readable and live, or no figure. ## eoc-meter specifics - **Probe:** records where its packages are read (the powercap directory, or `/` for MSR packages), and the counter reads them there. Zones are picked by name, so a listed but unreadable package zone stays with powercap. - **Docs and errors:** the `CounterStatus::Unavailable` and `rapl_status` docs say why an unprivileged MSR host is Unavailable (no `CAP_SYS_RAWIO`), and read errors name the MSR when that is the source. - **`tests/rapl_hardware.rs`:** its oracle read powercap only, so as root on an MSR host it would have disagreed with the meter. It now has its own MSR reader and brackets the meter's figure with MSR reads. - **Unit tests:** 6 new MSR tests on fake device files. ## Verified - **Linux x86_64:** `cargo test -p eoc-meter` (default and `cuda`) passes; clippy is clean; `cargo check --workspace --all-targets` is clean. - **Pre-existing failures, not from this change:** - Clippy with `--features cuda` fails on 3 type-complexity lints in `tests/nvml_hardware.rs`. CI doesn't lint that configuration. - `cargo test --workspace` failed one `eoc` binary test, `a_hit_is_charged_an_estimate_calibrated_on_this_host`, under workspace load ("rapl: CounterHeld"). That host has a powercap zone, so the new code doesn't run there, and alone the test passes 3 of 3 on this branch and on `main`. - **macOS (CI step):** `cargo test -p eoc-meter` passes with the `ioreport liveness:` grep; clippy is clean. - **Two bare-metal servers with no powercap zone, as root:** - Xeon E-2278G: Live, 12/12 intervals; 8.56 J over a 300 ms burn (28.5 W), inside the oracle's MSR bracket; `detect()` picks `rapl`. - Two Xeon Silver 4215R sockets: both Live 12/12; 23.07 J (76.9 W). - **Same servers, unprivileged:** Unavailable, the oracle agrees, and `detect()` falls back. ## Second commit: die scope Linux scopes Intel's RAPL MSRs per die and AMD's and Hygon's per package. `arch/x86/events/rapl.c` says "AMD/HYGON platforms have a per-PKG package energy counter" and "CLX-AP is multi-die and its RAPL MSRs are die-scope". The powercap driver names a multi-die Intel package's zones `package-<p>-die-<d>`, one top-level zone per die. - **MSR fallback:** reading each package on its first CPU counted one die of a multi-die Intel package, an under-count. A domain is now (`physical_package_id`, `die_id`) on Intel and the package on AMD and Hygon. Each domain is read on its own first CPU and they are summed. Single-die parts are unchanged. - **Powercap names:** the rule now also accepts `package-<n>-die-<m>`, so a multi-die host's die zones count as packages. It accepts nothing looser. - **Hardware test oracle:** scopes MSR domains the same way, using its own code. New unit tests: a two-die Intel package is two domains, summed; an AMD package whose CPUs list several `die_id`s is one domain; a two-die powercap tree sums both die zones; and the name rule's accepted names and near misses. Tests and clippy pass as before on Linux and macOS. 🤖 Generated with [Claude Code](https://claude.com/claude-code)
eoc-meter: RAPL reads each package's energy-status MSR where powercap lists none
All checks were successful
standards workspaces build + test / which workspaces this change touches (pull_request) Successful in 10s
standards workspaces build + test / eoc-rs (eoc/eoc-rs, -p eoc-meter --features cuda, eoc-rs, linux) (pull_request) Successful in 2m9s
repository checks / lockfiles, advisories, tests in CI, workflow hygiene, spec and section references, public text (pull_request) Successful in 1m33s
energy meters on real hardware / sink decode table · per-task energy and the GPU rail on macOS (pull_request) Successful in 2m58s
energy meters on real hardware / RAPL and NVML on jetson-hub (pull_request) Successful in 36s
energy meters on real hardware / sink decode table · RAPL on jetson-hub (NVML surveyed) (pull_request) Successful in 2m26s
standards workspaces build + test / macOS energy backends (pull_request) Successful in 10m28s
0b08e3d161
Some kernels ship without the powercap RAPL driver (Ubuntu without
linux-modules-extra, as on some rented bare-metal servers), so
/sys/class/powercap lists no package while the counters are there, and the
RAPL rung was unavailable on hosts that can measure. Where powercap lists no
package, the meter now reads the same counters directly: each package's
MSR_PKG_ENERGY_STATUS (0x611, Intel family 6; 0xC001029B, AMD Zen and Hygon)
through /dev/cpu/<first cpu of the package>/msr, the 32-bit count converted at
the package's own unit (bits 12:8 of the power-unit MSR), so it wraps at 2^32
counts in microjoules. A power-unit MSR that reads zero is no counter (KVM
guests read zeros). Powercap is still preferred, and the liveness rule is
unchanged: every package readable and live, or no figure. The probe records
where its packages are read (the powercap directory, or the filesystem root
for MSR packages), and the counter reads them there. CounterStatus and
rapl_status say why an MSR host without CAP_SYS_RAWIO is Unavailable. Ported
from jouleclaw-energy's RAPL MSR fallback, adapted to eoc's package and probe
types; no new dependency.

The hardware test's oracle reads the MSRs itself where powercap lists no
package (vendor and family, each package's first CPU, the registers at their
offsets), so its verdict and its package bracket cover such a host.

On two bare-metal servers with no powercap zone, as root: a Xeon E-2278G
live, 8.561 J over a 300 ms one-thread burn (28.5 W), inside the oracle's MSR
bracket; two Xeon Silver 4215R sockets, both live, 23.071 J (76.9 W), inside
it. As an unprivileged user (no CAP_SYS_RAWIO) the status is Unavailable, the
oracle agrees, and detect() falls through.

Tests: fake device files (a register's bytes at its offset) cover a two-socket
host read on each package's first CPU, the 32-bit wrap, AMD's registers, a zero
unit, an unknown CPU, a missing device, powercap preferred, and the production
probe over MSR packages (live, and frozen when one package holds). Unit tests
pass on Linux x86_64 (default and cuda) and on macOS (the CI step, its
`ioreport liveness:` line printed); clippy is clean on macOS and on Linux in
the default build.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
eoc-meter: RAPL MSR domains are per die on Intel, per package on AMD; powercap die zones count
All checks were successful
standards workspaces build + test / which workspaces this change touches (pull_request) Successful in 6s
repository checks / lockfiles, advisories, tests in CI, workflow hygiene, spec and section references, public text (pull_request) Successful in 2m35s
energy meters on real hardware / sink decode table · per-task energy and the GPU rail on macOS (pull_request) Successful in 2m46s
standards workspaces build + test / eoc-rs (eoc/eoc-rs, -p eoc-meter --features cuda, eoc-rs, linux) (pull_request) Successful in 2m43s
energy meters on real hardware / RAPL and NVML on jetson-hub (pull_request) Successful in 36s
standards workspaces build + test / macOS energy backends (pull_request) Successful in 6m57s
energy meters on real hardware / sink decode table · RAPL on jetson-hub (NVML surveyed) (pull_request) Successful in 2m22s
335f7f1fa7
The kernel scopes Intel's RAPL energy counters per die. Its perf PMU treats
the package counter as package-scope only on AMD and Hygon ("AMD/HYGON
platforms have a per-PKG package energy counter", rapl_pkg_pmu_is_pkg_scope()
in arch/x86/events/rapl.c), and on a multi-die Intel package (Cascade Lake-AP)
"its RAPL MSRs are die-scope"; its powercap driver then lists one zone per die,
named package-<p>-die-<d> (intel_rapl_common.c). Both paths here under-counted
such a package, the unsafe direction at a ceiling:

- The MSR fallback grouped CPUs by physical_package_id alone, so it read one
  die of a multi-die Intel package. On Intel a counter domain is now
  (physical_package_id, die_id), die_id 0 where the file is absent; on AMD
  and Hygon it stays the package. Each domain is read on its lowest-numbered
  CPU and every domain is summed, all or nothing as before (zero unit, wrap,
  powercap preferred, status text unchanged). A die of a multi-die package
  has its own rail name, package-<p>-die-<d> (MSR), in the liveness rails and
  in errors.
- The powercap name rule accepted only package-<n>, so such a host's die
  zones were no package at all. It now also accepts package-<p>-die-<d>, and
  nothing looser (not psys, dram, package-, package-0-die-,
  package-0-core-0), so every die zone is summed and watched.

The hardware test's oracle follows both rules: its zone check accepts die
zones, and its MSR reader groups Intel CPUs by package and die.

Tests (fake files): an Intel package of two dies is two counters, each read
on its own first CPU, both summed, with distinct rails, and a die whose device
is missing leaves no figure; an AMD package whose CPUs report different
die_ids is one counter; an Intel host with no die_id files, or all on die 0,
is read per package as before; the name rule over die names and near misses;
a fake two-die powercap tree sums both zones and is frozen when one die holds.
Unit tests pass on Linux x86_64 (default and cuda) and on macOS (the CI step,
its `ioreport liveness:` line printed); clippy is clean on macOS and on Linux
in the default build.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
Sign in to join this conversation.
No reviewers
No labels
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
Transaction-Science/open-standards!627
No description provided.