sandbox-meter: RAPL reads each domain's energy-status MSR where powercap lists none; die zones are packages #629

Merged

dcharlot merged 2 commits from sandbox-meter/rapl-msr into main 2026-11-05 04:53:09 -05:00

Conversation 0 Commits 2 Files changed 7 +1319 -126

dcharlot commented 2026-11-04 21:21:04 -05:00

Owner

Copy link

Some kernels ship without the powercap RAPL driver (Ubuntu without linux-modules-extra, as on some rented bare-metal servers). On those hosts /sys/class/powercap lists no package even though the counters are there. This ports the MSR fallback from #618 (jouleclaw-energy) into sandbox-meter, with no dependency added: the standards do not depend on one another.

What changes

• MSR fallback: where powercap lists no package zone, the probe reads the energy-status MSRs instead and reports itself as rapl-msr.
  • Registers: 0x606 and 0x611 on Intel family 6; 0xC0010299 and 0xC001029B on AMD Zen and Hygon.
  • Count: converted at each domain's own unit and wraps at 2^32 counts.
  • Zero unit: a unit that reads zero means no counter.
  • Unchanged: powercap is still preferred, and the liveness rule is the same: every domain readable and live, or no figure.
• Domains follow the kernel: a domain is a die of a package on Intel (die_id), because Intel's RAPL MSRs are die-scope on multi-die parts. On AMD and Hygon, whose package energy counter is per package, a domain is the package. Each domain is read through the first of its CPUs whose MSRs read.
• Offline CPUs: a CPU with no readable topology refuses the whole MSR probe, so a domain is never left out unseen. This follows the sandbox standard's own MSR rules (sandbox-exec-microvm's MsrRaplMeter).
• Powercap die zones: a zone named package-<p>-die-<d> (the kernel's name for each die's zone of a multi-die Intel package) is a package domain, and every die zone is summed. The rule accepts nothing looser.
• Declined probes say why: without CAP_SYS_RAWIO, the message names /dev/cpu/0/msr, "Permission denied", and what reading an MSR needs. joule-meter prints which RAPL interface it reads, or why it declined.
• API: RaplPackages gains live_at (a fake host's powercap tree plus filesystem root) and source(). live_in stays powercap-only.
• Tests: the new hardware test tests/rapl_msr_hardware.rs, and tests/rapl_hardware.rs's oracle now accepts die zones.

Verified

• Linux x86_64 (a powercap host): cargo test -p sandbox-meter passes, including rapl_hardware 4/4 and rapl_msr_hardware 2/2; the MSR path is correctly not used there. cargo check --workspace --all-targets is clean, and sandbox-exec-microvm and sandbox-tee pass. Clippy is clean apart from 3 type-complexity lints that are already on main, in untouched tests/nvml_hardware.rs.
• macOS (CI step): cargo test -p sandbox-meter passes, with both greps (ioreport liveness:, tree meter:). Clippy on sandbox-meter and sandbox-proof is clean.
• Two bare-metal servers with no powercap zone, as root:
  • Xeon E-2278G: package-0-die-0 Live, 12/12 intervals. A 300 ms burn read 8.85 J, inside the oracle's bracket. joule-meter reads 2.04 W idle and 122.25 W on all cores.
  • Two Xeon Silver 4215R sockets: both domains Live; 23.07 J (76.9 W), inside the bracket. joule-meter reads 26.75 W idle and 207.09 W on all cores.
• Same servers, unprivileged: Unavailable, with the permission reason, and the meter falls back to the estimator.

Not in this PR

• sandbox-tee's example (examples/attested_efficiency.rs) maps only rapl-powercap, so on an MSR host it attests Estimator. That fails safe.
• sandbox-proof (src/lib.rs) reads /sys/class/powercap itself and has no MSR path.

🤖 Generated with Claude Code

Some kernels ship without the powercap RAPL driver (Ubuntu without `linux-modules-extra`, as on some rented bare-metal servers). On those hosts `/sys/class/powercap` lists no package even though the counters are there. This ports the MSR fallback from #618 (`jouleclaw-energy`) into `sandbox-meter`, with no dependency added: the standards do not depend on one another. ## What changes - **MSR fallback:** where powercap lists no package zone, the probe reads the energy-status MSRs instead and reports itself as `rapl-msr`. - **Registers:** 0x606 and 0x611 on Intel family 6; 0xC0010299 and 0xC001029B on AMD Zen and Hygon. - **Count:** converted at each domain's own unit and wraps at 2^32 counts. - **Zero unit:** a unit that reads zero means no counter. - **Unchanged:** powercap is still preferred, and the liveness rule is the same: every domain readable and live, or no figure. - **Domains follow the kernel:** a domain is a die of a package on Intel (`die_id`), because Intel's RAPL MSRs are die-scope on multi-die parts. On AMD and Hygon, whose package energy counter is per package, a domain is the package. Each domain is read through the first of its CPUs whose MSRs read. - **Offline CPUs:** a CPU with no readable topology refuses the whole MSR probe, so a domain is never left out unseen. This follows the sandbox standard's own MSR rules (`sandbox-exec-microvm`'s `MsrRaplMeter`). - **Powercap die zones:** a zone named `package-<p>-die-<d>` (the kernel's name for each die's zone of a multi-die Intel package) is a package domain, and every die zone is summed. The rule accepts nothing looser. - **Declined probes say why:** without `CAP_SYS_RAWIO`, the message names `/dev/cpu/0/msr`, "Permission denied", and what reading an MSR needs. `joule-meter` prints which RAPL interface it reads, or why it declined. - **API:** `RaplPackages` gains `live_at` (a fake host's powercap tree plus filesystem root) and `source()`. `live_in` stays powercap-only. - **Tests:** the new hardware test `tests/rapl_msr_hardware.rs`, and `tests/rapl_hardware.rs`'s oracle now accepts die zones. ## Verified - **Linux x86_64 (a powercap host):** `cargo test -p sandbox-meter` passes, including `rapl_hardware` 4/4 and `rapl_msr_hardware` 2/2; the MSR path is correctly not used there. `cargo check --workspace --all-targets` is clean, and `sandbox-exec-microvm` and `sandbox-tee` pass. Clippy is clean apart from 3 type-complexity lints that are already on `main`, in untouched `tests/nvml_hardware.rs`. - **macOS (CI step):** `cargo test -p sandbox-meter` passes, with both greps (`ioreport liveness:`, `tree meter:`). Clippy on `sandbox-meter` and `sandbox-proof` is clean. - **Two bare-metal servers with no powercap zone, as root:** - Xeon E-2278G: `package-0-die-0` Live, 12/12 intervals. A 300 ms burn read 8.85 J, inside the oracle's bracket. `joule-meter` reads 2.04 W idle and 122.25 W on all cores. - Two Xeon Silver 4215R sockets: both domains Live; 23.07 J (76.9 W), inside the bracket. `joule-meter` reads 26.75 W idle and 207.09 W on all cores. - **Same servers, unprivileged:** Unavailable, with the permission reason, and the meter falls back to the estimator. ## Not in this PR - **`sandbox-tee`'s example** (`examples/attested_efficiency.rs`) maps only `rapl-powercap`, so on an MSR host it attests `Estimator`. That fails safe. - **`sandbox-proof`** (`src/lib.rs`) reads `/sys/class/powercap` itself and has no MSR path. 🤖 Generated with [Claude Code](https://claude.com/claude-code)

dcharlot added 2 commits 2026-11-04 21:21:05 -05:00

sandbox-meter: RAPL reads each package's energy-status MSR where powercap lists none ... 6a80a0cf43

Some kernels ship without the powercap RAPL driver (Ubuntu without
linux-modules-extra, as on some rented bare-metal servers), so
/sys/class/powercap lists no package while the counters are there, and the
meter estimated hosts that can measure. Where powercap lists no package zone,
the RAPL probe now reads the same counters directly, as `rapl-msr`: each
domain's MSR_PKG_ENERGY_STATUS (0x611, Intel family 6; 0xC001029B, AMD Zen and
Hygon) through /dev/cpu/<n>/msr, the 32-bit count converted at the domain's
own unit (bits 12:8 of the power-unit MSR, 0x606 or 0xC0010299), so it wraps
at 2^32 counts in microjoules. A domain is a package on AMD and Hygon and a die
of a package on Intel, as sandbox-exec-microvm's MSR meter has it, read through
the first of its CPUs whose MSRs read. Every domain or none: a CPU with no
readable topology (offline), a domain whose MSRs read through none of its
CPUs, or a zero unit (KVM guests read zeros) opens nothing. Powercap is still
preferred, and a tree that lists a package zone, or a zone that may be one, is
read through powercap or not at all. The liveness rule is unchanged: every
domain readable and live, or no figure.

A declined probe now says why its counters did not open: the probe run records
the reason, and RaplPackages::host() / RaplProbe::given append it to the gate's
text. Without CAP_SYS_RAWIO: "no RAPL package zone in /sys/class/powercap; and
the energy-status MSRs: package-0-die-0: the RAPL MSRs read through none of
its CPUs (16 tried; first /dev/cpu/0/msr: Permission denied (os error 13));
reading an MSR needs the msr module, CAP_SYS_RAWIO and read access to
/dev/cpu/<n>/msr". joule-meter prints the RAPL interface it reads, or that
reason. RaplPackages gains live_at (a fake host: powercap tree plus filesystem
root) and source(); live_in stays powercap-only.

On two bare-metal servers with no powercap zone (msr module loaded), as root:
a one-socket Xeon E-2278G live (package-0-die-0, 12 of 12 intervals), a 300 ms
one-thread burn 8.85 J inside the oracle's bracket (8.849 to 8.877 J),
joule-meter 2.04 W idle and 122.3 W on all cores; two Xeon Silver 4215R
sockets live (package-0-die-0 and package-1-die-0), the burn 23.07 J (76.9 W)
inside the bracket, joule-meter 26.8 W idle and 207.1 W on all cores. As an
unprivileged user the probe declines with the reason above and the meter
estimates.

Tests: fake roots (a register's bytes at its offset in a fake device file)
cover a two-socket Intel host, the 32-bit wrap, AMD's registers and per-package
scope, an Intel package of two dies, a domain read through its second CPU, a
zero unit, an unknown CPU, a missing device, an offline CPU, a missing die_id,
no CPU, powercap preferred (and an unreadable powercap package not falling
through), the public counter and the liveness run over MSR domains, and the
declined probe's reason. A new hardware test, tests/rapl_msr_hardware.rs, checks
the MSR path against an oracle of its own that reads the MSRs; the powercap
hardware test hands such a host to it. Unit and hardware tests pass on Linux
x86_64 (powercap host) and on both servers above as root and unprivileged; the
macOS CI step passes (both greps), and clippy is clean on macOS (sandbox-meter,
sandbox-proof) and on Linux apart from three existing type_complexity errors in
tests/nvml_hardware.rs.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

sandbox-meter: a powercap zone named package-<p>-die-<d> is a package's domain ... 3e5a9cec26

On a multi-die Intel package the kernel scopes RAPL per die and lists one
top-level intel-rapl:<n> zone per die, named "package-<p>-die-<d>"
(intel_rapl_common's "package-%d-die-%d", used when the MSRs are not package
scope and a package has more than one die). The probe accepted only
"package-<n>", so such a host listed no package zone: estimated, or, with the
MSR fallback, read through the MSRs as root only. A zone's name now names a
package's domain when it is "package-<digits>" or
"package-<digits>-die-<digits>", and nothing looser (not psys, dram,
"package-", "package-0-die-", "package-0-core-0", "package-0-die-1-die-2"), so
every die's zone is summed into the CPU figure under the same rules: every
domain readable and live, or no figure. The hardware tests' oracles (the
powercap one, and the MSR one's check for a powercap package) apply the same
name rule.

Tests: the name rule over both forms and the near misses; a fake two-die
powercap tree (two die zones, psys, each die's core subzone) holding exactly
the two die zones, metering a span as their rise alone, and frozen when one
die holds. Pass on Linux x86_64 (cargo test -p sandbox-meter; clippy clean
apart from the existing type_complexity errors in tests/nvml_hardware.rs) and
in the macOS CI step (both greps; clippy on sandbox-meter and sandbox-proof
clean).

Co-Authored-By: Claude Opus 5.5 <[email protected]>

dcharlot merged commit 076e6e731e into main 2026-11-05 04:53:09 -05:00

dcharlot referenced this pull request from a commit 2026-11-05 04:53:10 -05:00

Merge pull request 'sandbox-meter: RAPL reads each domain's energy-status MSR where powercap lists none; die zones are packages' (#629) from sandbox-meter/rapl-msr into main

dcharlot referenced this pull request 2026-11-05 10:23:43 -05:00

sandbox: the proof and microvm oracles read RAPL MSRs where powercap lists none; sandbox-tee attests rapl-msr as RAPL #651

Sign in to join this conversation.

Reviewers

No reviewers

Labels

Clear labels

No items

No labels

Milestone

Clear milestone

No items

No milestone

Projects

Clear projects

No items

No project

Assignees

Clear assignees

No assignees

1 participant

Notifications

Subscribe

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference

Transaction-Science/open-standards!629

No description provided.