jcp-energy: RAPL reads each package's energy-status MSR where powercap lists none #625

Merged

dcharlot merged 2 commits from jcp-energy/rapl-msr into main 2026-11-05 04:52:42 -05:00

Conversation 0 Commits 2 Files changed 2 +945 -98

dcharlot commented 2026-11-04 21:02:16 -05:00

Owner

Copy link

Some kernels ship without the powercap RAPL driver (Ubuntu without linux-modules-extra, as on some rented bare-metal servers). On those hosts /sys/class/powercap lists no package even though the counters are there. This ports the MSR fallback from #618 (jouleclaw-energy) into this meter, with no dependency added: the standards do not depend on one another.

Where powercap lists no package, each package's MSR_PKG_ENERGY_STATUS is read on that package's first CPU through /dev/cpu/<n>/msr:

• Registers: 0x606 and 0x611 on Intel family 6; 0xC0010299 and 0xC001029B on AMD Zen and Hygon.
• Count: the 32-bit count is converted at the package's own unit (bits 12:8 of the power-unit MSR) and wraps at 2^32 counts.
• Zero unit: a unit that reads zero means no counter.
• Unchanged: powercap is still preferred, and the liveness rule is the same: every package readable and live, or no figure.

jcp-energy specifics

• Readers: Rapl now holds powercap zones or MSR packages (RaplPackage). A package zone that is listed but can't be read never falls through to the MSRs.
• rapl_access(): checks the MSRs where powercap has no package zone. Permission denied on any package gives NeedsPrivilege; before, such a host was reported as Absent. No msr module, an unknown CPU or a zero unit gives NoInterface. The survey note names the MSR path and what it needs.
• tests/rapl_hardware.rs: gains an MSR oracle that is independent of the crate. The existing CI grep for package-only inner still holds.
• Unit tests on fake device files: two-socket host, 32-bit wrap, AMD registers, zero unit, unknown CPU, missing device, unreadable counter, no topology, powercap preferred (including a listed but unreadable zone), the production probe (Live, and Frozen when one package holds), and access answers with injected permission errors.

Verified

• Linux x86_64: cargo test -p jcp-energy (default and nvml) and cargo test -p jcp-gateway pass.
• macOS (CI steps): jcp-energy (with the ioreport liveness: grep) and jcp-gateway (with the upstream tree: grep) pass. Clippy on jcp-energy and jcp-gateway is clean.
• Linux clippy: reports 3 lints that are already on main, in untouched survey code (lib.rs ~6266, ~6306, ~6415). CI lints this crate on macOS only.
• Two bare-metal servers with no powercap zone, as root:
  • Xeon E-2278G: Live, 12/12 intervals; 8.85 J over a 300 ms burn (about 29.5 W), HwShunt, Whole, inside the oracle's bracket.
  • Two Xeon Silver 4215R sockets: both Live 12/12; 23.1 J over 300 ms (about 77 W).
• Same servers, unprivileged: RAPL is Unavailable, access is NeedsPrivilege, and the meter falls to the default-power estimate.

Second commit: die scope

Linux scopes Intel's RAPL MSRs per die and AMD's and Hygon's per package. arch/x86/events/rapl.c says "AMD/HYGON platforms have a per-PKG package energy counter" and "CLX-AP is multi-die and its RAPL MSRs are die-scope". The powercap driver names a multi-die Intel package's zones package-<p>-die-<d>, one top-level zone per die.

• MSR fallback: reading each package on its first CPU counted one die of a multi-die Intel package, an under-count. A domain is now (physical_package_id, die_id) on Intel and the package on AMD and Hygon. Each domain is read on its own first CPU and they are summed. Single-die parts are unchanged.
• Powercap names: the rule now also accepts package-<n>-die-<m>, so a multi-die host's die zones count as packages. It accepts nothing looser.
• Hardware test oracle: scopes MSR domains the same way, using its own code.

New unit tests: a two-die Intel package is two domains, summed; an AMD package whose CPUs list several die_ids is one domain; a two-die powercap tree sums both die zones; and the name rule's accepted names and near misses. Tests and clippy pass as before on Linux and macOS.

🤖 Generated with Claude Code

Some kernels ship without the powercap RAPL driver (Ubuntu without `linux-modules-extra`, as on some rented bare-metal servers). On those hosts `/sys/class/powercap` lists no package even though the counters are there. This ports the MSR fallback from #618 (`jouleclaw-energy`) into this meter, with no dependency added: the standards do not depend on one another. Where powercap lists no package, each package's `MSR_PKG_ENERGY_STATUS` is read on that package's first CPU through `/dev/cpu/<n>/msr`: - **Registers:** 0x606 and 0x611 on Intel family 6; 0xC0010299 and 0xC001029B on AMD Zen and Hygon. - **Count:** the 32-bit count is converted at the package's own unit (bits 12:8 of the power-unit MSR) and wraps at 2^32 counts. - **Zero unit:** a unit that reads zero means no counter. - **Unchanged:** powercap is still preferred, and the liveness rule is the same: every package readable and live, or no figure. ## jcp-energy specifics - **Readers:** `Rapl` now holds powercap zones or MSR packages (`RaplPackage`). A package zone that is listed but can't be read never falls through to the MSRs. - **`rapl_access()`:** checks the MSRs where powercap has no package zone. Permission denied on any package gives `NeedsPrivilege`; before, such a host was reported as `Absent`. No `msr` module, an unknown CPU or a zero unit gives `NoInterface`. The survey note names the MSR path and what it needs. - **`tests/rapl_hardware.rs`:** gains an MSR oracle that is independent of the crate. The existing CI grep for `package-only inner` still holds. - **Unit tests on fake device files:** two-socket host, 32-bit wrap, AMD registers, zero unit, unknown CPU, missing device, unreadable counter, no topology, powercap preferred (including a listed but unreadable zone), the production probe (Live, and Frozen when one package holds), and access answers with injected permission errors. ## Verified - **Linux x86_64:** `cargo test -p jcp-energy` (default and `nvml`) and `cargo test -p jcp-gateway` pass. - **macOS (CI steps):** `jcp-energy` (with the `ioreport liveness:` grep) and `jcp-gateway` (with the `upstream tree:` grep) pass. Clippy on `jcp-energy` and `jcp-gateway` is clean. - **Linux clippy:** reports 3 lints that are already on `main`, in untouched survey code (lib.rs ~6266, ~6306, ~6415). CI lints this crate on macOS only. - **Two bare-metal servers with no powercap zone, as root:** - Xeon E-2278G: Live, 12/12 intervals; 8.85 J over a 300 ms burn (about 29.5 W), HwShunt, Whole, inside the oracle's bracket. - Two Xeon Silver 4215R sockets: both Live 12/12; 23.1 J over 300 ms (about 77 W). - **Same servers, unprivileged:** RAPL is Unavailable, access is `NeedsPrivilege`, and the meter falls to the default-power estimate. ## Second commit: die scope Linux scopes Intel's RAPL MSRs per die and AMD's and Hygon's per package. `arch/x86/events/rapl.c` says "AMD/HYGON platforms have a per-PKG package energy counter" and "CLX-AP is multi-die and its RAPL MSRs are die-scope". The powercap driver names a multi-die Intel package's zones `package-<p>-die-<d>`, one top-level zone per die. - **MSR fallback:** reading each package on its first CPU counted one die of a multi-die Intel package, an under-count. A domain is now (`physical_package_id`, `die_id`) on Intel and the package on AMD and Hygon. Each domain is read on its own first CPU and they are summed. Single-die parts are unchanged. - **Powercap names:** the rule now also accepts `package-<n>-die-<m>`, so a multi-die host's die zones count as packages. It accepts nothing looser. - **Hardware test oracle:** scopes MSR domains the same way, using its own code. New unit tests: a two-die Intel package is two domains, summed; an AMD package whose CPUs list several `die_id`s is one domain; a two-die powercap tree sums both die zones; and the name rule's accepted names and near misses. Tests and clippy pass as before on Linux and macOS. 🤖 Generated with [Claude Code](https://claude.com/claude-code)

dcharlot added 1 commit 2026-11-04 21:02:16 -05:00

jcp-energy: RAPL reads each package's energy-status MSR where powercap lists none ... 5b178b0d78

Some kernels ship without the powercap RAPL driver (Ubuntu without
linux-modules-extra, as on some rented bare-metal servers), so
/sys/class/powercap lists no package while the counters are there, and the
RAPL rung was Unavailable on hosts that can measure. Where powercap lists no
package zone, the reader now takes the same counters directly: each package's
MSR_PKG_ENERGY_STATUS (0x611, Intel family 6; 0xC001029B, AMD Zen and Hygon)
through /dev/cpu/<first cpu of the package>/msr, the 32-bit count converted at
the package's own unit (bits 12:8 of the power-unit MSR), so it wraps at 2^32
counts in microjoules. A power-unit MSR that reads zero is no counter (KVM
guests read zeros). Powercap is still preferred, a listed package zone this
process cannot read is no reason to read the MSRs instead, and the liveness
rule is unchanged: every package readable and live, or no figure.

rapl_access now asks the MSRs where powercap exposes no package zone, so a
host whose MSR device files need CAP_SYS_RAWIO reports NeedsPrivilege rather
than Absent; no msr module, an unknown CPU or a zero unit is NoInterface. The
hardware test's oracle reads the MSRs itself on such a host, so the status,
the gate and the package-only bracket are checked there too.

On two bare-metal servers with no powercap zone, as root: a Xeon E-2278G
live, 8.85 J over a 300 ms one-thread burn, inside the oracle's bracket; two
Xeon Silver 4215R sockets, package 1 read on CPU 8, live, 23.1 J over the same
burn. As an unprivileged user (no CAP_SYS_RAWIO) RAPL is Unavailable, its
access NeedsPrivilege, and the meter is the default-power estimate.

Tests: fake device files (a register's bytes at its offset) cover a two-socket
host read on each package's first CPU, the 32-bit wrap, AMD's registers, a zero
unit, an unknown CPU, a missing device, a counter that does not read, powercap
preferred, the production probe over MSR packages, and the access answer with
permission errors injected. Passing on Linux x86_64 (default and nvml) and on
macOS (the CI step, with its ioreport grep; jcp-gateway; clippy clean for
jcp-energy and jcp-gateway).

Co-Authored-By: Claude Opus 5.5 <[email protected]>

dcharlot added 1 commit 2026-11-04 21:21:20 -05:00

jcp-energy: RAPL MSR domains are per die on Intel, per package on AMD; die zones are package zones ... 9333cbd5ee

The MSR fallback grouped CPUs by physical_package_id alone, but Linux scopes
Intel's RAPL counters per die: arch/x86/events/rapl.c gives only AMD and Hygon
a per-package package energy counter, and a multi-die Intel part (Cascade
Lake-AP) has die-scope RAPL MSRs. Reading each package's first CPU there
counts one die, an under-count, the unsafe direction at an energy ceiling.
A counter domain is now, on Intel family 6, the pair (physical_package_id,
die_id), die 0 where the kernel lists no die_id; on AMD and Hygon it stays the
package. Each domain is read on its lowest-numbered CPU and all are summed,
under the same rules: every domain or none, a zero unit is no counter, the
wrap at 2^32 counts, powercap preferred. rapl_access asks the same domains. A
die domain's rail is package-<n>-die-<m> msr where the package has more than
one die, package-<n> msr where it has one.

The powercap side had the matching gap. On such a part intel_rapl_common.c
names each top-level zone "package-%d-die-%d" (the MSRs not package-scope and
more than one die per package), one intel-rapl:<n> zone per die, and the
package-name rule accepted only package-<n>: the host listed no package. The
rule now also accepts package-<digits>-die-<digits>, and nothing looser, so
every die zone is summed. The hardware test's oracle follows both rules.

Tests: an Intel package with two dies is two domains, each read on its own
first CPU and summed, with a one-die package beside it keeping its name; a die
whose device file is missing leaves no reader; access over the dies; an AMD
package whose CPUs list different die_ids is one domain; an Intel host with no
die_id is read per package; the name rule (package-0-die-1 yes; package-,
package-0-die-, package-0-core-0, package-0-die-1-die-2 no); a fake two-die
powercap tree summing both zones. Passing on Linux x86_64 (default, nvml,
jcp-gateway) and on macOS (the CI steps with their greps; clippy clean).

Co-Authored-By: Claude Opus 5.5 <[email protected]>

dcharlot merged commit e54e0e25c3 into main 2026-11-05 04:52:40 -05:00

dcharlot referenced this pull request from a commit 2026-11-05 04:52:44 -05:00

Merge pull request 'jcp-energy: RAPL reads each package's energy-status MSR where powercap lists none' (#625) from jcp-energy/rapl-msr into main

Sign in to join this conversation.

Reviewers

No reviewers

Labels

Clear labels

No items

No labels

Milestone

Clear milestone

No items

No milestone

Projects

Clear projects

No items

No project

Assignees

Clear assignees

No assignees

1 participant

Notifications

Subscribe

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference

Transaction-Science/open-standards!625

No description provided.