diff --git a/proof/README.md b/proof/README.md index b1c87f3b3..bd18b8d38 100644 --- a/proof/README.md +++ b/proof/README.md @@ -40,7 +40,7 @@ proof/ - **`proof-cascade`** — the escalation router (every joule accounted, `wasted_uj` disclosed, and a goal's figure graded by the weakest of the attempts it sums), the content-addressed proof cache (the energy flywheel, durable as line-delimited JSON), batch energy accounting against a frontier-only baseline with the per-tier Amdahl breakdown, and a **real hardware energy meter** (`meter` — the `HwShunt` backend): it runs a real proof on every spare core and returns its measured microjoules from the first live source — on macOS **this process's own CPU energy counter** (`proc_pid_rusage`'s `ri_energy_nj`, public API, no privilege), read before the workers start and after they are joined, so other processes are excluded by construction and no idle baseline is subtracted; else IOReport's Energy Model, read directly through its C API on Apple Silicon and bracketed against idle; on Linux RAPL's package counters, bracketed against idle — so the headline `joules per verified proof` can be a real reading, not only a model. A vendor or kernel counter's reading is graded `HwShunt` only while the counter is live for the domain it claims ([`SPEC.md`](SPEC.md) §5), and the meter checks that before it meters anything, by one rule for every source (the counter keeps advancing in at least 5 sample intervals, at least 4 of every 5, over at least 300 ms while the probe burns CPU): the process counter itself, every IOReport CPU energy rail the figure sums (`CPU Energy`, or each die's `DIE__CPU Energy`), every RAPL package counter the figure sums (`psys` is neither summed nor probed). Each figure carries its domain: the process counter's is this process's CPU energy only — not other processes, child processes, GPU, ANE or DRAM. Returns nothing where no source is live — the caller keeps its honestly-tagged estimate rather than inventing a figure. - **`proof-receipt`** — the `proof/1` JCR-1 receipt: canonical det-CBOR + COSE_Sign1/EdDSA, vendored byte-compatible with the family's cross-language verifiers; refuses to seal an unproved goal. - **`proof-conformance`** — C1–C5 + a negative pack, all checks real, published vectors in `conformance/v1/`. -- **`proof-cli`** — the `proof` reference demo: a proof workload reporting joules-per-proof against a frontier-only baseline — like for like, charged only for the goals the cascade proved, with the all-goals figure and its goal count beside it — the Amdahl breakdown, the cache flywheel, and a sealed + verified receipt. `proof --measure` adds a **real hardware reading** of a non-trivial T0 proof (`HwShunt`) where the meter produces one, and prints which source metered it, its domain, and why each other source was not used; that needs a counter the meter has seen to be live. On the Macs measured so far (macOS 27.0.1, an M5 Max and an M3 Ultra) IOReport's CPU energy rail is frozen for an unprivileged reader, but this process's CPU energy counter is live, so `--measure` meters with it; the receipt still seals the cascade's own `Estimator` figure for the first goal. On Linux, RAPL's `energy_uj` is root-only on current kernels. +- **`proof-cli`** — the `proof` reference demo: a proof workload reporting joules-per-proof against a frontier-only baseline — like for like, charged only for the goals the cascade proved, with the all-goals figure and its goal count beside it — the Amdahl breakdown, the cache flywheel, and a sealed + verified receipt. `proof --measure` adds a **real hardware reading** of a non-trivial T0 proof (`HwShunt`) where the meter produces one, and prints which source metered it, its domain, and why each other source was not used; that needs a counter the meter has seen to be live. On the Macs measured so far (macOS 27.0.1, an M5 Max and an M3 Ultra) IOReport's CPU energy rail is frozen for an unprivileged reader, but this process's CPU energy counter is live, so `--measure` meters with it; the receipt still seals the cascade's own `Estimator` figure for the first goal. On Linux, RAPL's `energy_uj` is root-only on current kernels; where powercap lists no package (a kernel without the `intel_rapl_msr` driver), the meter reads the energy-status MSRs instead — each die on Intel, each package on AMD and Hygon, as Linux scopes them — which needs `CAP_SYS_RAWIO`. `cargo test --workspace` green; `cargo clippy --workspace --all-targets` is warning-free for Linux, and on macOS reports two lints in proof-cascade's diff --git a/proof/proof-rs/crates/proof-cascade/src/meter.rs b/proof/proof-rs/crates/proof-cascade/src/meter.rs index 8ea8349c7..d7db61a4b 100644 --- a/proof/proof-rs/crates/proof-cascade/src/meter.rs +++ b/proof/proof-rs/crates/proof-cascade/src/meter.rs @@ -15,7 +15,9 @@ //! is seen to have left its core (below). //! 2. **IOReport's Energy Model** (Apple Silicon, read directly via its C API — //! no `macmon`, no sudo): SoC power bracketed busy against idle. -//! 3. **RAPL** `energy_uj` (Linux): package power bracketed busy against idle. +//! 3. **RAPL** (Linux): package power bracketed busy against idle, from each +//! package's powercap `energy_uj`, or, on a kernel without the powercap +//! driver, its energy-status MSR. //! //! Returns `None` where no source is live — the caller keeps its //! honestly-tagged estimate rather than inventing a number. No @@ -119,7 +121,8 @@ //! **On Linux, RAPL counts as reachable only while every package counter is //! live** ([`rapl_status`](crate::meter::rapl_status)). The same probe loop and //! the same rule run over the `energy_uj` counters of the RAPL zones named -//! `package-`, while the probe burns CPU; RAPL is used only when EVERY +//! `package-` (or, one per die of a multi-die Intel package, +//! `package--die-`), while the probe burns CPU; RAPL is used only when EVERY //! package counter keeps advancing — in at least five separate sample //! intervals, in at least 80% of the intervals observed, over at least 300 ms //! — because the figure sums all of them. A counter that holds, steps in a @@ -129,6 +132,27 @@ //! its liveness is not probed. `energy_uj` is root-only on current kernels, //! so an unprivileged run usually reads no zone: `Unavailable`, and `None`. //! +//! **Without the powercap driver.** Some kernels ship without `intel_rapl_msr` +//! (Ubuntu without `linux-modules-extra`, as on some bare-metal servers), so +//! powercap lists no package while the counters themselves are there. Where +//! powercap lists no package, the meter reads the same counters directly: +//! each RAPL domain's energy-status MSR (`MSR_PKG_ENERGY_STATUS`, 0x611, on +//! Intel family 6; 0xC001029B on AMD Zen and Hygon), through the `msr` +//! driver's device file of the domain's lowest-numbered CPU +//! (`/dev/cpu//msr`, which needs the `msr` module and `CAP_SYS_RAWIO`, so +//! it too reads only as root). On Intel a domain is a die: Linux treats the +//! package counter as package-scope only on AMD and Hygon (its RAPL PMU's +//! `rapl_pkg_pmu_is_pkg_scope()`), and its powercap driver names a die-scoped +//! zone `package-

-die-` where a package holds more than one die, so a +//! multi-die Intel package is read on each die's first CPU and its dies +//! summed. On AMD and Hygon a domain is the package. The 32-bit +//! count is converted at the package's own energy-status unit (bits 12:8 of the +//! power-unit MSR, 0x606 or 0xC0010299), so it wraps at 2^32 counts in +//! microjoules, and resolves one count (about 61 µJ at the common unit of +//! 2^-14 J). A power-unit MSR that reads zero is no counter: some hypervisors +//! let a guest read these MSRs and return zeros. The same probe and the same +//! rule then decide whether RAPL is used, and the bracket is the same. +//! //! The chains are short on purpose. Each source's production status is one //! expression — the generic probe loop (`liveness::probe_watching`) on the //! real parts (the reader, the thread-CPU tick and the clocks) and the pure @@ -163,7 +187,9 @@ pub enum MeterSource { ProcessEnergy, /// IOReport's Energy Model (Apple Silicon): SoC power, busy over idle. IoReport, - /// RAPL's package counters (Linux): package power, busy over idle. + /// RAPL's package counters (Linux): package power, busy over idle. Each + /// package's powercap zone, or its energy-status MSR where powercap lists + /// no package. Rapl, } @@ -184,7 +210,10 @@ impl MeterSource { (not other processes, not child processes, not GPU, ANE or DRAM energy)" } MeterSource::IoReport => "SoC energy (CPU, GPU and other Energy Model domains) over the idle baseline", - MeterSource::Rapl => "CPU package energy (every RAPL package zone) over the idle baseline", + MeterSource::Rapl => { + "CPU package energy (every RAPL package: its powercap zone, or its energy-status MSR) over the idle \ + baseline" + } } } } @@ -530,14 +559,16 @@ fn admits(status: IoReportStatus) -> bool { pub enum CounterStatus { /// The counter could not be read: no such interface on this platform, no /// permission to read it (RAPL's `energy_uj` is root-only on current - /// kernels), or never two consecutive reads within the probe's five + /// kernels, and its energy-status MSRs need the `msr` module and + /// `CAP_SYS_RAWIO`), or never two consecutive reads within the probe's five /// budgets. Never used. (The process counter is never `Absent`: it is one /// counter, read or not. It is `Unavailable` too where the measuring /// thread's own energy count does not read: each read of a window is /// checked against that count, so without it no window could be read.) Unavailable, /// The interface was read, but holds no counter for the domain a figure - /// would claim (RAPL: zones, but no `package-` zone). Never used. + /// would claim (RAPL: zones, but no `package-` zone, and no package + /// energy-status MSR that reads). Never used. Absent, /// The counter did not keep advancing while the probe burned CPU — for /// RAPL, at least one package counter did not — it held, or moved in a @@ -578,8 +609,9 @@ fn admits_counter(status: CounterStatus) -> bool { const RAPL_LIVENESS_BUDGET: std::time::Duration = std::time::Duration::from_millis(1000); /// The RAPL liveness probe on its parts: `zones` reads every top-level -/// powercap zone (sysfs, on Linux), and the tick and clocks are as for -/// IOReport. It opens when a read finds any zone, and hands the rule each +/// powercap zone (sysfs, on Linux), or each package's energy-status MSR where +/// powercap lists no package ([`rapl::zones_from`]), and the tick and clocks +/// are as for IOReport. It opens when a read finds any zone, and hands the rule each /// read's package counters ([`rapl::read_of`]), every one of which must keep /// advancing ([`rapl::PACKAGES`]). A read that finds no zone is a failed read. /// Generic, so the tests run this very composition on scripted zones on every @@ -641,9 +673,11 @@ fn rapl_host_status() -> CounterStatus { } /// This host's RAPL status, probed once per process and cached: whether every -/// RAPL package counter (`intel-rapl:` zones named `package-`) kept -/// advancing while the probe burned CPU. On Linux, [`meter_source`] (and -/// through it [`measure_marginal`]) and `rapl_power_over` use RAPL only when +/// RAPL package counter (`intel-rapl:` zones named `package-` or +/// `package--die-`, or each RAPL domain's energy-status MSR where +/// powercap lists none) kept advancing while +/// the probe burned CPU. On Linux, [`meter_source`] (and through it +/// [`measure_marginal`]) and `rapl_power_over` use RAPL only when /// this is [`CounterStatus::Live`]. The first call in a process pays the probe (about /// `RAPL_LIVENESS_BUDGET` of burning CPU and re-reading sysfs on the calling /// thread when the counters are readable; nothing when they are not). Always @@ -2926,19 +2960,34 @@ mod macos_process { } // --------------------------------------------------------------------------- -// Linux RAPL: the energy between two reads of the powercap zones, and what -// the liveness rule is handed from one read, kept apart from the sysfs reads -// so they are pure, tested functions on every platform. +// Linux RAPL: one read of the package counters (powercap's zones, or each +// package's energy-status MSR where powercap lists none), the energy between +// two reads, and what the liveness rule is handed from one read. Every reader +// takes its root as a path, so the tests run them on fake trees on every Unix. // --------------------------------------------------------------------------- #[cfg_attr(not(target_os = "linux"), allow(dead_code))] mod rapl { use super::liveness::{Listed, Read, Sample, Watched}; + use std::path::{Path, PathBuf}; + + /// Where the Linux kernel lists its powercap zones, RAPL's among them. + pub(super) const POWERCAP_ROOT: &str = "/sys/class/powercap"; + + /// Intel's power-unit MSR (family 6). + pub(super) const MSR_RAPL_POWER_UNIT: u64 = 0x606; + /// Intel's package energy-status MSR (family 6). + pub(super) const MSR_PKG_ENERGY_STATUS: u64 = 0x611; + /// AMD Zen's (and Hygon's) power-unit MSR. + pub(super) const MSR_AMD_RAPL_POWER_UNIT: u64 = 0xC001_0299; + /// AMD Zen's (and Hygon's) package energy-status MSR. + pub(super) const MSR_AMD_PKG_ENERGY_STATUS: u64 = 0xC001_029B; /// One powercap zone (`/sys/class/powercap/intel-rapl:`), read once. #[derive(Debug, Clone, Copy, PartialEq, Eq)] pub(super) struct Zone { pub(super) index: u32, - /// Whether the zone is a package (its `name` is `package-`), not a + /// Whether the zone is a package (its `name` is `package-`, or + /// `package--die-` for one die of a multi-die package), not a /// platform (`psys`) or other top-level zone. Only package zones are /// summed, and only their liveness is probed: `psys` covers the /// package too, so summing it beside one counts that energy twice. @@ -2951,9 +3000,21 @@ mod rapl { } /// Whether a top-level zone's `name` file names a package: `package-`, - /// `` one or more digits. + /// or one die of a multi-die Intel package, `package--die-` — the + /// kernel's powercap driver names each top-level zone so (one + /// `intel-rapl:` zone per die) where the MSRs are die-scope and a + /// package holds more than one die (intel_rapl_common.c), and every die + /// zone is part of the package's energy. `` and `` one or more + /// digits; nothing looser (not `package-0-die-`, not `package-0-core-0`). pub(super) fn is_package_name(name: &str) -> bool { - name.trim().strip_prefix("package-").is_some_and(|n| !n.is_empty() && n.bytes().all(|b| b.is_ascii_digit())) + let digits = |n: &str| !n.is_empty() && n.bytes().all(|b| b.is_ascii_digit()); + let Some(rest) = name.trim().strip_prefix("package-") else { + return false; + }; + match rest.split_once("-die-") { + Some((package, die)) => digits(package) && digits(die), + None => digits(rest), + } } /// The package zones of one read, in zone order: the zones whose energy a @@ -2963,7 +3024,9 @@ mod rapl { } /// The name a package zone's counter has in the sample the liveness rule - /// is handed: its sysfs zone, `intel-rapl:`. + /// is handed: its sysfs zone, `intel-rapl:`. An MSR domain goes by + /// the same name at its domain index, so each die of a package is a rail + /// of its own. pub(super) fn rail_name(index: u32) -> String { format!("intel-rapl:{index}") } @@ -3007,6 +3070,241 @@ mod rapl { max.checked_sub(e0)?.checked_add(e1) } + /// The file at `path`, read and parsed as an unsigned integer, or `None`. + fn read_u64(path: &Path) -> Option { + std::fs::read_to_string(path).ok()?.trim().parse::().ok() + } + + /// Every readable top-level RAPL zone under the powercap directory `root` + /// (`intel-rapl:0`..`7`): whether it is a package (its `name`), its + /// `energy_uj` counter, and the `max_energy_range_uj` it wraps at, when + /// that is readable. A zone whose `energy_uj` does not read is left out. + pub(super) fn powercap_zones_under(root: &Path) -> Vec { + (0..8u32) + .filter_map(|index| { + let base = root.join(format!("intel-rapl:{index}")); + Some(Zone { + index, + package: std::fs::read_to_string(base.join("name")).is_ok_and(|n| is_package_name(&n)), + energy_uj: read_u64(&base.join("energy_uj"))?, + max_energy_range_uj: read_u64(&base.join("max_energy_range_uj")), + }) + }) + .collect() + } + + /// Whether powercap lists a package under `root`: a top-level zone whose + /// `name` is a package's ([`is_package_name`]: `package-`, or a die's + /// `package--die-`), readable counter or not (the `name` file is + /// world-readable; `energy_uj` is root-only). + pub(super) fn powercap_lists_package(root: &Path) -> bool { + (0..8u32).any(|index| { + std::fs::read_to_string(root.join(format!("intel-rapl:{index}/name"))).is_ok_and(|n| is_package_name(&n)) + }) + } + + /// The scope of a CPU's RAPL energy-status MSRs: the domain one counter + /// covers. Linux's own rule: its RAPL PMU treats the package counter as + /// package-scope only on AMD and Hygon (`rapl_pkg_pmu_is_pkg_scope()`, + /// arch/x86/events/rapl.c, whose comments read "AMD/HYGON platforms have a + /// per-PKG package energy counter" and "CLX-AP is multi-die and its RAPL + /// MSRs are die-scope"), and its powercap driver names a zone + /// `package-

-die-` where the MSRs are not package-scope and a + /// package holds more than one die (intel_rapl_common.c). + #[derive(Debug, Clone, Copy, PartialEq, Eq)] + pub(super) enum MsrScope { + /// Intel: one counter per die (`topology/die_id` within + /// `topology/physical_package_id`), so a multi-die package has one + /// counter on each die. + Die, + /// AMD Zen and Hygon: one counter per package. + Package, + } + + /// One RAPL domain read from its energy-status MSR, where the kernel lists + /// no powercap zone for it: a die on Intel, a package on AMD and Hygon + /// ([`MsrScope`]). + #[derive(Debug, Clone, PartialEq, Eq)] + pub(super) struct MsrDomain { + /// The domain's place in (package, die) order, from 0: its zone + /// index, and so its rail `intel-rapl:` — distinct for each + /// die of a package, as the kernel's powercap zones are. + pub(super) index: u32, + /// Its `physical_package_id`. + pub(super) package: u32, + /// Its `die_id` within the package on Intel (0 where the kernel lists + /// none); 0 on AMD and Hygon, whose counter is the package's. + pub(super) die: u32, + /// The `msr` driver's device file of the domain's lowest-numbered CPU, + /// `/dev/cpu//msr`. + pub(super) device: PathBuf, + /// The energy-status MSR: [`MSR_PKG_ENERGY_STATUS`] or + /// [`MSR_AMD_PKG_ENERGY_STATUS`]. + pub(super) register: u64, + /// The energy-status unit, 2^-`unit_shift` J per count: bits 12:8 of + /// the power-unit MSR. Never zero. + pub(super) unit_shift: u32, + } + + impl MsrDomain { + /// The domain's energy, in microjoules, at a raw read of its + /// energy-status MSR: the low 32 bits are the count. + pub(super) fn uj(&self, raw: u64) -> u64 { + ((u128::from(raw & 0xFFFF_FFFF) * 1_000_000) >> self.unit_shift) as u64 + } + + /// The microjoule figure's wrap: 2^32 counts. + pub(super) fn range_uj(&self) -> u64 { + (((1u128 << 32) * 1_000_000) >> self.unit_shift) as u64 + } + } + + /// The 64-bit MSR `register` read through the `msr` driver's device file + /// `device`, where the register number is the file offset. `None` when the + /// file cannot be opened (no `msr` module, no `CAP_SYS_RAWIO`, no read + /// access) or the register cannot be read (the CPU does not implement it). + #[cfg(unix)] + pub(super) fn read_msr(device: &Path, register: u64) -> Option { + use std::os::unix::fs::FileExt; + let file = std::fs::File::open(device).ok()?; + let mut raw = [0u8; 8]; + file.read_exact_at(&mut raw, register).ok()?; + Some(u64::from_le_bytes(raw)) + } + + #[cfg(not(unix))] + pub(super) fn read_msr(_device: &Path, _register: u64) -> Option { + None + } + + /// The CPU's vendor and family from `/proc/cpuinfo` text: the first CPU's + /// `vendor_id` and `cpu family`. + fn cpu_vendor_family(cpuinfo: &str) -> Option<(&str, u32)> { + let field = |key: &str| { + cpuinfo.lines().find_map(|line| { + let (k, v) = line.split_once(':')?; + (k.trim() == key).then(|| v.trim()) + }) + }; + Some((field("vendor_id")?, field("cpu family")?.parse().ok()?)) + } + + /// The power-unit and energy-status MSRs for a CPU's vendor and family, + /// and their scope, or `None` where this meter knows none: Intel family 6 + /// (per die), AMD Zen (family 0x17 and later) and Hygon (per package). + fn msr_registers(vendor: &str, family: u32) -> Option<(u64, u64, MsrScope)> { + match vendor { + "GenuineIntel" if family == 6 => Some((MSR_RAPL_POWER_UNIT, MSR_PKG_ENERGY_STATUS, MsrScope::Die)), + "AuthenticAMD" | "HygonGenuine" if family >= 0x17 => { + Some((MSR_AMD_RAPL_POWER_UNIT, MSR_AMD_PKG_ENERGY_STATUS, MsrScope::Package)) + } + _ => None, + } + } + + /// Each RAPL domain's lowest-numbered CPU, keyed (package, die), from + /// `/sys/devices/system/cpu/cpu/topology/`: `physical_package_id`, + /// and, where the scope is [`MsrScope::Die`], `die_id` (0 where the file + /// is absent: a kernel that lists no die has one per package). Where the + /// scope is [`MsrScope::Package`] the die is 0 whatever `die_id` says. + fn first_cpu_per_domain(root: &Path, scope: MsrScope) -> std::collections::BTreeMap<(u32, u32), u32> { + let mut first = std::collections::BTreeMap::new(); + let Ok(entries) = std::fs::read_dir(root.join("sys/devices/system/cpu")) else { + return first; + }; + for entry in entries.flatten() { + let Some(cpu) = entry.file_name().to_str().and_then(|n| n.strip_prefix("cpu")?.parse::().ok()) else { + continue; + }; + let topology = entry.path().join("topology"); + let Some(package) = read_u64(&topology.join("physical_package_id")).and_then(|p| u32::try_from(p).ok()) + else { + continue; + }; + let die = match scope { + MsrScope::Die => match read_u64(&topology.join("die_id")) { + None => 0, + Some(d) => match u32::try_from(d) { + Ok(d) => d, + Err(_) => continue, + }, + }, + MsrScope::Package => 0, + }; + let lowest = first.entry((package, die)).or_insert(cpu); + *lowest = (*lowest).min(cpu); + } + first + } + + /// Every RAPL domain's energy-status MSR under the filesystem root `root` + /// — each die on Intel, each package on AMD and Hygon ([`MsrScope`]) — in + /// (package, die) order: read through the device file of the domain's + /// lowest-numbered CPU (`/dev/cpu//msr`), at the unit its + /// power-unit MSR reads. Empty when the CPU is not one whose MSRs this + /// meter knows, when no CPU topology is listed, or when any domain's + /// power-unit MSR cannot be read or reads a zero unit — a figure that left + /// a die out would miss its energy, and a zero unit is a hypervisor's + /// stand-in, not a counter. + pub(super) fn msr_domains_under(root: &Path) -> Vec { + let Ok(cpuinfo) = std::fs::read_to_string(root.join("proc/cpuinfo")) else { + return Vec::new(); + }; + let Some((unit_register, register, scope)) = cpu_vendor_family(&cpuinfo).and_then(|(v, f)| msr_registers(v, f)) + else { + return Vec::new(); + }; + let mut domains = Vec::new(); + for (index, ((package, die), cpu)) in (0u32..).zip(first_cpu_per_domain(root, scope)) { + let device = root.join(format!("dev/cpu/{cpu}/msr")); + let Some(unit) = read_msr(&device, unit_register) else { + return Vec::new(); + }; + let unit_shift = ((unit >> 8) & 0x1F) as u32; + if unit_shift == 0 { + return Vec::new(); + } + domains.push(MsrDomain { index, package, die, device, register, unit_shift }); + } + domains + } + + /// One read of every MSR domain, as package zones: each domain's count at + /// its unit, wrapping at 2^32 counts ([`MsrDomain::range_uj`]), so the + /// figure sums every die of every package. Empty when there is no domain, + /// or when any one cannot be read: a read that left a die out would miss + /// its energy while claiming the whole CPU. + pub(super) fn msr_zones(packages: &[MsrDomain]) -> Vec { + packages + .iter() + .map(|p| { + Some(Zone { + index: p.index, + package: true, + energy_uj: p.uj(read_msr(&p.device, p.register)?), + max_energy_range_uj: Some(p.range_uj()), + }) + }) + .collect::>>() + .unwrap_or_default() + } + + /// One read of this host's RAPL zones: powercap's readable top-level zones + /// under `powercap` where it lists a package (preferred). Where it lists + /// none, each RAPL domain's energy-status MSR (`msr` lists the domains, and + /// is called only then); and where no MSR reads either, powercap's other + /// zones, so a host that lists only `psys` still reads as having no + /// package. + pub(super) fn zones_from(powercap: &Path, msr: impl FnOnce() -> Vec) -> Vec { + if !powercap_lists_package(powercap) { + let packages = msr_zones(&msr()); + if !packages.is_empty() { + return packages; + } + } + powercap_zones_under(powercap) + } + /// The energy between two reads of the same zones: the sum of per-zone /// deltas, each taken modulo that zone's own `max_energy_range_uj`. Each /// zone wraps at its own range, so wrapping the SUM of the counters turns @@ -3071,14 +3369,33 @@ mod rapl { assert_eq!(zone_delta_uj(1_000, 10, Some(0)), None, "a zero range is unreadable"); } - /// A package zone is named `package-` (the sysfs file ends in a - /// newline); `psys`, `dram`, `core` and malformed names are not. + /// A package zone is named `package-`, or `package--die-` + /// for one die of a multi-die package (the sysfs file ends in a + /// newline); `psys`, `dram`, `core`, a core or die-less suffix, and + /// malformed names are not. #[test] fn only_a_package_name_is_a_package() { - for name in ["package-0", "package-1\n", "package-12"] { + for name in ["package-0", "package-1\n", "package-12", "package-0-die-0", "package-1-die-12\n"] { assert!(is_package_name(name), "{name:?}"); } - for name in ["psys", "dram", "core", "uncore", "package-", "package-x", "package-0a", "Package-0", ""] { + for name in [ + "psys", + "dram", + "core", + "uncore", + "package-", + "package-x", + "package-0a", + "Package-0", + "", + "package-0-die-", + "package-0-die", + "package--die-0", + "package-0-die-x", + "package-0-die-1-die-2", + "package-0-core-0", + "package-0-Die-1", + ] { assert!(!is_package_name(name), "{name:?}"); } assert!(is_package_rail("intel-rapl:0") && is_package_rail("intel-rapl:12")); @@ -3108,6 +3425,375 @@ mod rapl { assert_eq!(delta_uj(&one, &[zone(1, 2_000, Some(RANGE))]), None, "a different zone"); assert_eq!(zone_delta_uj(RANGE + 5, 3, Some(RANGE)), None, "first read above the range"); } + + /// The MSR path, over fake device files: a regular file whose bytes at + /// a register's offset are that register is read exactly as the `msr` + /// driver's device file is. + #[cfg(unix)] + mod msr { + use super::super::*; + use super::RANGE; + use std::cell::Cell; + use std::os::unix::fs::FileExt; + use std::path::{Path, PathBuf}; + + const INTEL: &str = "processor\t: 0\nvendor_id\t: GenuineIntel\ncpu family\t: 6\nmodel\t\t: 85\n"; + const AMD: &str = "processor\t: 0\nvendor_id\t: AuthenticAMD\ncpu family\t: 26\nmodel\t\t: 112\n"; + /// A power-unit MSR whose energy-status unit (bits 12:8) is 2^-14 J, + /// the common Intel value (0xA0E03). + const UNIT_14: u64 = 0xA_0E03; + /// 2^-16 J, as on AMD Zen. + const UNIT_16: u64 = 0x10 << 8; + /// 2^32 counts of 2^-14 J, in µJ. + const MSR_RANGE_14: u64 = 262_144_000_000; + + /// A fake filesystem root for one test, removed when dropped: its + /// `/proc/cpuinfo`, each CPU's `physical_package_id`, the CPUs' + /// `dev/cpu//msr` files, and a powercap directory. + struct MsrRoot(PathBuf); + + impl MsrRoot { + /// `package_of_cpu[n]` is CPU `n`'s package. + fn new(tag: &str, cpuinfo: &str, package_of_cpu: &[u32]) -> Self { + let topology: Vec<(u32, Option)> = package_of_cpu.iter().map(|&p| (p, None)).collect(); + Self::with_dies(tag, cpuinfo, &topology) + } + + /// `topology[n]` is CPU `n`'s package and, when given, its + /// `die_id` (no `die_id` file otherwise). + fn with_dies(tag: &str, cpuinfo: &str, topology: &[(u32, Option)]) -> Self { + let root = std::env::temp_dir().join(format!("proof-cascade-msr-{tag}-{}", std::process::id())); + let _ = std::fs::remove_dir_all(&root); + std::fs::create_dir_all(root.join("proc")).expect("create proc"); + std::fs::write(root.join("proc/cpuinfo"), cpuinfo).expect("write cpuinfo"); + for (cpu, (package, die)) in topology.iter().enumerate() { + let dir = root.join(format!("sys/devices/system/cpu/cpu{cpu}/topology")); + std::fs::create_dir_all(&dir).expect("create a CPU's topology"); + std::fs::write(dir.join("physical_package_id"), format!("{package}\n")).expect("write its package"); + if let Some(die) = die { + std::fs::write(dir.join("die_id"), format!("{die}\n")).expect("write its die"); + } + } + Self(root) + } + + /// Set CPU `cpu`'s MSR `register` to `value`. + fn set(&self, cpu: u32, register: u64, value: u64) -> &Self { + let dir = self.0.join(format!("dev/cpu/{cpu}")); + std::fs::create_dir_all(&dir).expect("create dev/cpu/"); + let file = std::fs::OpenOptions::new() + .create(true) + .truncate(false) + .write(true) + .open(dir.join("msr")) + .expect("open the fake msr file"); + file.write_all_at(&value.to_le_bytes(), register).expect("write the register"); + self + } + + /// Top-level powercap zone `intel-rapl:` named `name`, + /// with `energy_uj` when given (root-only on a real host: + /// `None` leaves the file out), wrapping at [`RANGE`]. + fn zone(&self, index: u32, name: &str, energy_uj: Option) -> &Self { + let dir = self.powercap().join(format!("intel-rapl:{index}")); + std::fs::create_dir_all(&dir).expect("create the zone"); + std::fs::write(dir.join("name"), format!("{name}\n")).expect("write name"); + std::fs::write(dir.join("max_energy_range_uj"), format!("{RANGE}\n")).expect("write range"); + match energy_uj { + Some(e) => std::fs::write(dir.join("energy_uj"), format!("{e}\n")).expect("write energy_uj"), + None => { + let _ = std::fs::remove_file(dir.join("energy_uj")); + } + } + self + } + + fn root(&self) -> &Path { + &self.0 + } + + fn powercap(&self) -> PathBuf { + self.0.join("sys/class/powercap") + } + } + + impl Drop for MsrRoot { + fn drop(&mut self) { + let _ = std::fs::remove_dir_all(&self.0); + } + } + + fn listed(zones: &[Zone]) -> Vec<(u32, bool, u64, Option)> { + zones.iter().map(|z| (z.index, z.package, z.energy_uj, z.max_energy_range_uj)).collect() + } + + /// A two-socket Intel host (CPUs 0–1 on package 0, 2–3 on + /// package 1) with no powercap driver: each package read on its + /// first CPU, at its unit, and the figure their summed deltas. + #[test] + fn a_two_socket_host_without_powercap_reads_each_packages_msr() { + let host = MsrRoot::new("two-socket", INTEL, &[0, 0, 1, 1]); + host.set(0, MSR_RAPL_POWER_UNIT, UNIT_14).set(0, MSR_PKG_ENERGY_STATUS, 1_000); + host.set(2, MSR_RAPL_POWER_UNIT, UNIT_14).set(2, MSR_PKG_ENERGY_STATUS, 2_000); + let packages = msr_domains_under(host.root()); + let devices: Vec<_> = packages.iter().map(|p| (p.index, p.device.clone())).collect(); + assert_eq!( + devices, + vec![(0, host.root().join("dev/cpu/0/msr")), (1, host.root().join("dev/cpu/2/msr"))], + "package 1 is read on its first CPU, 2" + ); + assert!(packages.iter().all(|p| (p.register, p.unit_shift) == (MSR_PKG_ENERGY_STATUS, 14)), "{packages:?}"); + + // 1,000 and 2,000 counts of 61.035 µJ, each floored. + let before = zones_from(&host.powercap(), || packages.clone()); + assert_eq!(listed(&before), vec![(0, true, 61_035, Some(MSR_RANGE_14)), (1, true, 122_070, Some(MSR_RANGE_14))]); + let names: Vec = read_of(&before).sample.into_iter().map(|(n, _)| n).collect(); + assert_eq!(names, ["intel-rapl:0", "intel-rapl:1"], "the rule watches each package"); + host.set(0, MSR_PKG_ENERGY_STATUS, 3_000).set(2, MSR_PKG_ENERGY_STATUS, 2_100); + let after = zones_from(&host.powercap(), || packages.clone()); + assert_eq!(delta_uj(&before, &after), Some((183_105 - 61_035) + (128_173 - 122_070))); + } + + fn domains(list: &[MsrDomain]) -> Vec<(u32, u32, u32, PathBuf)> { + list.iter().map(|d| (d.index, d.package, d.die, d.device.clone())).collect() + } + + /// Intel scopes its RAPL MSRs per die: two packages of two dies + /// each are four domains, each read on its own die's first CPU, in + /// (package, die) order, each a rail of its own, and the figure + /// sums all four. Read per package, only each package's first die + /// would count — an under-count. + #[test] + fn an_intel_package_with_two_dies_is_read_on_each_die() { + let topology = [ + (0, Some(0)), + (0, Some(0)), + (0, Some(1)), + (0, Some(1)), + (1, Some(0)), + (1, Some(0)), + (1, Some(1)), + (1, Some(1)), + ]; + let host = MsrRoot::with_dies("two-dies", INTEL, &topology); + for (cpu, count) in [(0, 1_000), (2, 2_000), (4, 3_000), (6, 4_000)] { + host.set(cpu, MSR_RAPL_POWER_UNIT, UNIT_14).set(cpu, MSR_PKG_ENERGY_STATUS, count); + } + let found = msr_domains_under(host.root()); + let dev = |cpu: u32| host.root().join(format!("dev/cpu/{cpu}/msr")); + assert_eq!(domains(&found), vec![(0, 0, 0, dev(0)), (1, 0, 1, dev(2)), (2, 1, 0, dev(4)), (3, 1, 1, dev(6))]); + + let before = zones_from(&host.powercap(), || found.clone()); + assert_eq!( + listed(&before), + vec![ + (0, true, 61_035, Some(MSR_RANGE_14)), + (1, true, 122_070, Some(MSR_RANGE_14)), + (2, true, 183_105, Some(MSR_RANGE_14)), + (3, true, 244_140, Some(MSR_RANGE_14)), + ] + ); + let rails: Vec = read_of(&before).sample.into_iter().map(|(n, _)| n).collect(); + assert_eq!(rails, ["intel-rapl:0", "intel-rapl:1", "intel-rapl:2", "intel-rapl:3"], "each die is a rail"); + for (cpu, count) in [(0, 1_100), (2, 2_100), (4, 3_100), (6, 4_100)] { + host.set(cpu, MSR_PKG_ENERGY_STATUS, count); + } + let after = zones_from(&host.powercap(), || found.clone()); + // 100 counts of 61.035 µJ on each die, each floored. + let each: Vec = before.iter().zip(&after).map(|(b, a)| a.energy_uj - b.energy_uj).collect(); + assert_eq!(each, [6_103, 6_103, 6_103, 6_104]); + assert_eq!(delta_uj(&before, &after), Some(6_103 * 3 + 6_104), "every die is summed"); + } + + /// AMD and Hygon scope the package counter per package: CPUs + /// whose `die_id` differs within one package are still one domain, + /// read on the package's first CPU. + #[test] + fn an_amd_package_whose_cpus_carry_two_die_ids_is_one_domain() { + let host = MsrRoot::with_dies("amd-dies", AMD, &[(0, Some(0)), (0, Some(0)), (0, Some(1)), (0, Some(1))]); + host.set(0, MSR_AMD_RAPL_POWER_UNIT, UNIT_16).set(0, MSR_AMD_PKG_ENERGY_STATUS, 65_536); + let found = msr_domains_under(host.root()); + assert_eq!(domains(&found), vec![(0, 0, 0, host.root().join("dev/cpu/0/msr"))]); + assert_eq!(listed(&msr_zones(&found)), vec![(0, true, 1_000_000, Some(65_536_000_000))]); + } + + /// An Intel host whose kernel lists no `die_id` has one die per + /// package: read per package, as before. + #[test] + fn an_intel_host_with_no_die_id_files_is_read_per_package() { + let host = MsrRoot::new("no-die-id", INTEL, &[0, 0, 1, 1]); + for cpu in [0, 2] { + host.set(cpu, MSR_RAPL_POWER_UNIT, UNIT_14).set(cpu, MSR_PKG_ENERGY_STATUS, 10); + } + let found = msr_domains_under(host.root()); + let dev = |cpu: u32| host.root().join(format!("dev/cpu/{cpu}/msr")); + assert_eq!(domains(&found), vec![(0, 0, 0, dev(0)), (1, 1, 0, dev(2))]); + } + + /// A multi-die Intel package under powercap: the kernel lists one + /// top-level zone per die, named `package-

-die-`. Each is a + /// package zone, so powercap lists a package (the MSRs are never + /// listed), and both are summed. + #[test] + fn a_two_die_powercap_tree_sums_both_zones() { + let host = MsrRoot::new("two-die-powercap", INTEL, &[0]); + host.zone(0, "package-0-die-0", Some(1_000)).zone(1, "package-0-die-1", Some(5_000)); + host.zone(2, "psys", Some(90_000)); + assert!(powercap_lists_package(&host.powercap())); + let called = Cell::new(false); + let listing = || { + called.set(true); + Vec::new() + }; + let before = zones_from(&host.powercap(), listing); + assert!(!called.get(), "powercap lists the dies: the MSRs are not listed"); + assert_eq!( + listed(&before), + vec![(0, true, 1_000, Some(RANGE)), (1, true, 5_000, Some(RANGE)), (2, false, 90_000, Some(RANGE))] + ); + let rails: Vec = read_of(&before).sample.into_iter().map(|(n, _)| n).collect(); + assert_eq!(rails, ["intel-rapl:0", "intel-rapl:1"], "both dies are watched; psys is not"); + host.zone(0, "package-0-die-0", Some(4_000)).zone(1, "package-0-die-1", Some(6_500)); + host.zone(2, "psys", Some(99_000)); + let after = zones_from(&host.powercap(), Vec::new); + assert_eq!(delta_uj(&packages(&before), &packages(&after)), Some(3_000 + 1_500), "both dies, not psys"); + } + + /// The 32-bit count wraps at 2^32 counts, so the microjoule figure + /// wraps at that range, losing at most a microjoule; bits above the + /// count are not energy. + #[test] + fn the_count_wraps_at_2_to_the_32_counts() { + let host = MsrRoot::new("wrap", INTEL, &[0]); + host.set(0, MSR_RAPL_POWER_UNIT, UNIT_14).set(0, MSR_PKG_ENERGY_STATUS, 0xFFFF_FFF0); + let packages = msr_domains_under(host.root()); + let before = msr_zones(&packages); + host.set(0, MSR_PKG_ENERGY_STATUS, 0xABCD_0000_0000_0010); + let after = msr_zones(&packages); + assert!(after[0].energy_uj < before[0].energy_uj, "the count wrapped"); + // 32 counts of 61.035 µJ is 1,953.125 µJ. + assert_eq!(delta_uj(&before, &after), Some(1_953)); + } + + /// AMD Zen reads its own power-unit and energy-status MSRs. + #[test] + fn amd_zen_reads_its_own_msrs() { + let host = MsrRoot::new("amd", AMD, &[0, 0]); + host.set(0, MSR_AMD_RAPL_POWER_UNIT, UNIT_16).set(0, MSR_AMD_PKG_ENERGY_STATUS, 65_536); + let packages = msr_domains_under(host.root()); + assert_eq!(packages.len(), 1); + assert_eq!((packages[0].register, packages[0].unit_shift), (MSR_AMD_PKG_ENERGY_STATUS, 16)); + let zones = msr_zones(&packages); + assert_eq!(zones[0].energy_uj, 1_000_000, "65,536 counts of 2^-16 J is one joule"); + assert_eq!(zones[0].max_energy_range_uj, Some(65_536_000_000)); + } + + /// No MSR counter where the power-unit MSR reads a zero unit (a + /// hypervisor's stand-in, seen under KVM), where the CPU is not one + /// whose MSRs are known, or where a package's device file is + /// missing; and a package that stops reading leaves no zone at all, + /// not the others. + #[test] + fn a_zero_unit_an_unknown_cpu_or_a_missing_device_is_no_counter() { + let zero = MsrRoot::new("zero-unit", INTEL, &[0]); + zero.set(0, MSR_RAPL_POWER_UNIT, 0).set(0, MSR_PKG_ENERGY_STATUS, 0); + assert!(msr_domains_under(zero.root()).is_empty(), "a zero unit"); + + let arm = MsrRoot::new("arm", "processor\t: 0\nCPU implementer\t: 0x41\n", &[0]); + assert!(msr_domains_under(arm.root()).is_empty(), "no vendor_id"); + + let netburst = MsrRoot::new("family-15", "vendor_id\t: GenuineIntel\ncpu family\t: 15\n", &[0]); + netburst.set(0, MSR_RAPL_POWER_UNIT, UNIT_14); + assert!(msr_domains_under(netburst.root()).is_empty(), "not family 6"); + + let missing = MsrRoot::new("missing-device", INTEL, &[0, 1]); + missing.set(0, MSR_RAPL_POWER_UNIT, UNIT_14); + assert!(msr_domains_under(missing.root()).is_empty(), "package 1 has no device file"); + + let lost = MsrRoot::new("lost-device", INTEL, &[0, 1]); + for cpu in [0, 1] { + lost.set(cpu, MSR_RAPL_POWER_UNIT, UNIT_14).set(cpu, MSR_PKG_ENERGY_STATUS, 10); + } + let packages = msr_domains_under(lost.root()); + assert_eq!(msr_zones(&packages).len(), 2); + std::fs::remove_file(lost.root().join("dev/cpu/1/msr")).expect("remove package 1's device"); + assert!(msr_zones(&packages).is_empty(), "package 1 no longer reads"); + assert!(zones_from(&lost.powercap(), || packages.clone()).is_empty(), "no zone at all: Unavailable"); + } + + /// Powercap zones are preferred; the MSRs are listed and read only + /// where powercap lists no package. A host that lists only `psys` + /// reads its MSRs, and, with none, still reads as having no + /// package; a listed package whose counter does not read (root-only) + /// leaves powercap in charge, MSRs unread. + #[test] + fn powercap_is_preferred_and_the_msrs_are_the_fallback() { + let host = MsrRoot::new("fallback", INTEL, &[0]); + host.set(0, MSR_RAPL_POWER_UNIT, UNIT_14).set(0, MSR_PKG_ENERGY_STATUS, 5_000); + let packages = msr_domains_under(host.root()); + let powercap = host.powercap(); + + let msr = zones_from(&powercap, || packages.clone()); + assert_eq!(listed(&msr), vec![(0, true, 305_175, Some(MSR_RANGE_14))], "no powercap tree: the MSR"); + + host.zone(1, "psys", Some(9_000)); + assert_eq!(zones_from(&powercap, || packages.clone()), msr, "a psys-only tree: still the MSR"); + assert_eq!(listed(&zones_from(&powercap, Vec::new)), vec![(1, false, 9_000, Some(RANGE))], "and with no MSR, psys"); + + let called = Cell::new(false); + let listing = || { + called.set(true); + packages.clone() + }; + host.zone(0, "package-0", Some(7_000)); + assert_eq!(listed(&zones_from(&powercap, listing)), vec![(0, true, 7_000, Some(RANGE)), (1, false, 9_000, Some(RANGE))]); + assert!(!called.get(), "powercap lists a package: the MSRs are not listed"); + + host.zone(0, "package-0", None); + let unreadable = zones_from(&powercap, || { + called.set(true); + packages.clone() + }); + assert_eq!(listed(&unreadable), vec![(1, false, 9_000, Some(RANGE))]); + assert!(!called.get(), "a listed package whose counter does not read is still powercap's"); + } + + /// The production composition `rapl_host_status` runs + /// ([`crate::meter::rapl_probe_with`]), over MSR packages read the + /// way `rapl_zones` reads the host: live when every package's count + /// keeps advancing, frozen when one holds. + #[test] + fn the_production_probe_judges_msr_packages_by_the_same_rule() { + use crate::meter::{counter_status_of, rapl_probe_with, CounterStatus, RAPL_LIVENESS_BUDGET}; + use std::time::Duration; + let run = |tag: &str, moving: &[u32]| { + let host = MsrRoot::new(tag, INTEL, &[0, 1]); + for cpu in [0, 1] { + host.set(cpu, MSR_RAPL_POWER_UNIT, UNIT_14).set(cpu, MSR_PKG_ENERGY_STATUS, 1_000_000); + } + let packages = msr_domains_under(host.root()); + let powercap = host.powercap(); + let (clock, count) = (Cell::new(Duration::ZERO), Cell::new(1_000_000u64)); + let probed = rapl_probe_with( + || zones_from(&powercap, || packages.clone()), + || { + count.set(count.get() + 18_432); // 45 W over a 25 ms tick, in 61.035 µJ counts + for &cpu in moving { + host.set(cpu, MSR_PKG_ENERGY_STATUS, count.get()); + } + clock.set(clock.get() + Duration::from_millis(25)); + crate::meter::probe_clock::TickOutcome { cpu: Some(Duration::from_millis(25)), spins: 1 } + }, + || clock.get(), + || Some(clock.get()), + RAPL_LIVENESS_BUDGET, + ); + counter_status_of(&probed) + }; + assert_eq!(run("probe-both", &[0, 1]), CounterStatus::Live); + assert_eq!(run("probe-one", &[0]), CounterStatus::Frozen, "package 1 held its count"); + } + } } } @@ -5107,7 +5793,9 @@ mod macos_ioreport { // 949dd0104c49, "powercap: restrict energy meter to root access", in 5.10 and // backported to stable: the CVE-2020-8694 / PLATYPUS mitigation). An // unprivileged process therefore usually reads no zone, and the meter declines -// unless it runs as root or an administrator has granted read access. +// unless it runs as root or an administrator has granted read access. Where +// powercap lists no package, each package's energy-status MSR is read instead +// (`rapl::zones_from`), which needs `CAP_SYS_RAWIO`: also root. // --------------------------------------------------------------------------- #[cfg(target_os = "linux")] fn idle_power() -> Option { @@ -5141,24 +5829,21 @@ fn rapl_power_over(d: std::time::Duration) -> Option { let dt = t.elapsed().as_secs_f64(); Some(rapl::delta_uj(&z0, &z1)? as f64 / 1e6 / dt) } -/// Every readable top-level RAPL zone (`intel-rapl:0`..`7`): whether it is a -/// package (its `name`), its `energy_uj` counter, and the -/// `max_energy_range_uj` it wraps at, when that is readable. +/// One read of this host's RAPL zones ([`rapl::zones_from`]): every readable +/// top-level powercap zone where powercap lists a package, else each RAPL +/// domain's energy-status MSR (each die on Intel, each package on AMD). #[cfg(target_os = "linux")] fn rapl_zones() -> Vec { - let text = |path: String| std::fs::read_to_string(path).ok(); - let read = |path: String| text(path)?.trim().parse::().ok(); - (0..8u32) - .filter_map(|index| { - let base = format!("/sys/class/powercap/intel-rapl:{index}"); - Some(rapl::Zone { - index, - package: text(format!("{base}/name")).is_some_and(|n| rapl::is_package_name(&n)), - energy_uj: read(format!("{base}/energy_uj"))?, - max_energy_range_uj: read(format!("{base}/max_energy_range_uj")), - }) - }) - .collect() + rapl::zones_from(std::path::Path::new(rapl::POWERCAP_ROOT), || host_msr_domains().to_vec()) +} + +/// This host's RAPL domains' energy-status MSRs, listed once per process: the +/// CPU topology and each domain's power unit do not change while it runs. +/// Listed only where powercap lists no package. +#[cfg(target_os = "linux")] +fn host_msr_domains() -> &'static [rapl::MsrDomain] { + static DOMAINS: std::sync::OnceLock> = std::sync::OnceLock::new(); + DOMAINS.get_or_init(|| rapl::msr_domains_under(std::path::Path::new("/"))) } #[cfg(not(any(target_os = "macos", target_os = "linux")))] diff --git a/proof/proof-rs/crates/proof-cascade/tests/rapl_hardware.rs b/proof/proof-rs/crates/proof-cascade/tests/rapl_hardware.rs index dd2c06df0..31ed470ff 100644 --- a/proof/proof-rs/crates/proof-cascade/tests/rapl_hardware.rs +++ b/proof/proof-rs/crates/proof-cascade/tests/rapl_hardware.rs @@ -15,6 +15,18 @@ //! over the calls, fit inside the oracle's package-only bracket around the //! whole measurement (idle and busy windows), which a figure that added //! psys or a package's core sub-zone overruns. +//! +//! A package zone is one named `package-`, or, on a multi-die Intel +//! package, each die's `package--die-`. Where powercap lists no package +//! (a kernel without the powercap driver), the meter reads each RAPL domain's +//! energy-status MSR instead, and so does the oracle, with its own reader: +//! each die on Intel and each package on AMD and Hygon, as Linux scopes them, +//! read on its lowest-numbered CPU's `/dev/cpu//msr` at the vendor's +//! registers, the 32-bit count at the domain's unit. Those domains join the +//! oracle's zones as `msr-package:

` or `msr-package:

-die-`, named +//! `package-

` or `package-

-die-`, and every test above runs over +//! them unchanged; without `CAP_SYS_RAWIO` they do not read, and the status +//! must say so. #![cfg(target_os = "linux")] @@ -79,14 +91,93 @@ mod oracle { thread_cpu() - started } - /// One entry of the powercap directory. + /// One entry of the powercap directory, or one package's energy-status + /// MSR where powercap lists no package. #[derive(Debug, Clone)] pub struct Zone { - /// The directory entry: `intel-rapl:0`, `intel-rapl:0:0`, `intel-rapl-mmio:0`, ... + /// The directory entry: `intel-rapl:0`, `intel-rapl:0:0`, `intel-rapl-mmio:0`, ...; + /// `msr-package:` for an MSR package. pub dir: String, pub name: Option, pub energy_uj: Option, pub max_uj: Option, + /// How to read an MSR package; `None` for a powercap entry. + pub msr: Option, + } + + /// One package's energy-status MSR, as the oracle reads it. + #[derive(Debug, Clone)] + pub struct Msr { + /// `/dev/cpu//msr`. + pub device: PathBuf, + /// The energy-status register: 0x611 (Intel), 0xC001029B (AMD, Hygon). + pub register: u64, + /// The unit: 2^-shift J per count, bits 12:8 of the power-unit MSR. + pub shift: u32, + } + + /// The 8 bytes at offset `register` of the `msr` device file `device`. + fn msr_read(device: &Path, register: u64) -> Option { + use std::os::unix::fs::FileExt; + let mut b = [0u8; 8]; + std::fs::File::open(device).ok()?.read_exact_at(&mut b, register).ok()?; + Some(u64::from_le_bytes(b)) + } + + /// The microjoules in the 32-bit count of `raw`, at 2^-shift J a count. + fn msr_uj(raw: u64, shift: u32) -> u64 { + (u128::from(raw & 0xFFFF_FFFF) * 1_000_000 / (1u128 << shift)) as u64 + } + + /// This host's RAPL domains' energy-status MSRs, as zones: each die on + /// Intel family 6 (0x606 / 0x611; `die_id` within `physical_package_id`, + /// 0 where the kernel lists none), `msr-package:

-die-` named + /// `package-

-die-`; each package on AMD family 0x17 and later and + /// Hygon (0xC0010299 / 0xC001029B), `msr-package:

` named + /// `package-

`. Each read on its lowest-numbered CPU, wrapping at 2^32 + /// counts. Empty unless every domain's power unit reads, and none is zero. + fn msr_packages() -> Vec { + let Some(cpuinfo) = text(PathBuf::from("/proc/cpuinfo")) else { return Vec::new() }; + let value = |key: &str| { + cpuinfo.lines().find_map(|l| l.split_once(':').filter(|(k, _)| k.trim() == key).map(|(_, v)| v.trim().to_string())) + }; + let (Some(vendor), Some(family)) = (value("vendor_id"), value("cpu family").and_then(|f| f.parse::().ok())) else { + return Vec::new(); + }; + let (unit_register, register, per_die) = match (vendor.as_str(), family) { + ("GenuineIntel", 6) => (0x606, 0x611, true), + ("AuthenticAMD" | "HygonGenuine", f) if f >= 0x17 => (0xC001_0299, 0xC001_029B, false), + _ => return Vec::new(), + }; + let mut first = std::collections::BTreeMap::<(u64, Option), u64>::new(); + for e in std::fs::read_dir("/sys/devices/system/cpu").into_iter().flatten().flatten() { + let name = e.file_name().to_string_lossy().into_owned(); + let Some(cpu) = name.strip_prefix("cpu").and_then(|n| n.parse::().ok()) else { continue }; + let Some(package) = number(e.path().join("topology/physical_package_id")) else { continue }; + let die = per_die.then(|| number(e.path().join("topology/die_id")).unwrap_or(0)); + first.entry((package, die)).and_modify(|c| *c = (*c).min(cpu)).or_insert(cpu); + } + let mut zones = Vec::new(); + for ((package, die), cpu) in first { + let device = PathBuf::from(format!("/dev/cpu/{cpu}/msr")); + let Some(unit) = msr_read(&device, unit_register) else { return Vec::new() }; + let shift = ((unit >> 8) & 0x1F) as u32; + if shift == 0 { + return Vec::new(); + } + let (dir, name) = match die { + Some(d) => (format!("msr-package:{package}-die-{d}"), format!("package-{package}-die-{d}")), + None => (format!("msr-package:{package}"), format!("package-{package}")), + }; + zones.push(Zone { + dir, + name: Some(name), + energy_uj: msr_read(&device, register).map(|raw| msr_uj(raw, shift)), + max_uj: Some(((1u128 << 32) * 1_000_000 / (1u128 << shift)) as u64), + msr: Some(Msr { device, register, shift }), + }); + } + zones } fn text(p: PathBuf) -> Option { @@ -97,18 +188,30 @@ mod oracle { text(p)?.parse().ok() } - /// Every entry under `root`, in name order. + /// Every entry under `root`, in name order; and, where none of them is a + /// package, this host's package energy-status MSRs that read. pub fn zones(root: &Path) -> Vec { - let Ok(entries) = std::fs::read_dir(root) else { return Vec::new() }; - let mut zones: Vec = entries - .flatten() - .map(|e| { - let dir = e.file_name().to_string_lossy().into_owned(); - let p = e.path(); - Zone { name: text(p.join("name")), energy_uj: number(p.join("energy_uj")), max_uj: number(p.join("max_energy_range_uj")), dir } - }) - .collect(); + let mut zones: Vec = match std::fs::read_dir(root) { + Ok(entries) => entries + .flatten() + .map(|e| { + let dir = e.file_name().to_string_lossy().into_owned(); + let p = e.path(); + Zone { + name: text(p.join("name")), + energy_uj: number(p.join("energy_uj")), + max_uj: number(p.join("max_energy_range_uj")), + msr: None, + dir, + } + }) + .collect(), + Err(_) => Vec::new(), + }; zones.sort_by(|a, b| a.dir.cmp(&b.dir)); + if !zones.iter().any(is_package) { + zones.extend(msr_packages()); + } zones } @@ -121,9 +224,17 @@ mod oracle { z.dir.strip_prefix("intel-rapl:").is_some_and(digits) } - /// A package zone: top-level, named `package-`. + /// A package zone: top-level, named `package-`, or `package--die-` + /// (one die of a multi-die package); or an MSR domain. pub fn is_package(z: &Zone) -> bool { - top_level(z) && z.name.as_deref().and_then(|n| n.strip_prefix("package-")).is_some_and(digits) + let named = |n: &str| match n.strip_prefix("package-") { + Some(rest) => match rest.split_once("-die-") { + Some((p, d)) => digits(p) && digits(d), + None => digits(rest), + }, + None => false, + }; + z.msr.is_some() || (top_level(z) && z.name.as_deref().is_some_and(named)) } /// The platform zone. @@ -145,9 +256,15 @@ mod oracle { } } - /// `energy_uj` of each of `dirs` under `root`, in order; `None` if any fails. + /// `energy_uj` of each of `dirs` under `root` (an MSR package's count, at + /// its unit), in order; `None` if any fails. pub fn read(root: &Path, dirs: &[&Zone]) -> Option> { - dirs.iter().map(|z| number(root.join(&z.dir).join("energy_uj"))).collect() + dirs.iter() + .map(|z| match &z.msr { + Some(m) => msr_read(&m.device, m.register).map(|raw| msr_uj(raw, m.shift)), + None => number(root.join(&z.dir).join("energy_uj")), + }) + .collect() } /// The summed energy of `which` of `set` between two reads of `set`. @@ -271,7 +388,7 @@ mod oracle { let _ = std::fs::remove_dir_all(&root); std::fs::create_dir_all(&root).expect("create the masked view"); let masked = root.join("masked-devices-virtual-powercap"); - for z in zones(Path::new(POWERCAP)) { + for z in zones(Path::new(POWERCAP)).iter().filter(|z| z.msr.is_none()) { std::os::unix::fs::symlink(masked.join(&z.dir), root.join(&z.dir)).expect("symlink"); } MaskedView(root) diff --git a/proof/proof-rs/crates/proof-cli/src/main.rs b/proof/proof-rs/crates/proof-cli/src/main.rs index fcf469202..b7164881c 100644 --- a/proof/proof-rs/crates/proof-cli/src/main.rs +++ b/proof/proof-rs/crates/proof-cli/src/main.rs @@ -319,8 +319,14 @@ fn ioreport_reason() -> &'static str { fn rapl_reason() -> &'static str { use proof_cascade::meter::{rapl_status, CounterStatus}; match rapl_status() { - CounterStatus::Unavailable => "no readable RAPL counter at /sys/class/powercap/intel-rapl:*/energy_uj", - CounterStatus::Absent => "RAPL zones are readable, but none of them is a package zone", + CounterStatus::Unavailable => { + "no readable RAPL counter at /sys/class/powercap/intel-rapl:*/energy_uj (root-only on current kernels), \ + nor, where powercap lists no package, a readable package energy-status MSR at /dev/cpu//msr (that \ + needs the msr module and CAP_SYS_RAWIO)" + } + CounterStatus::Absent => { + "RAPL zones are readable, but none of them is a package zone, and no package energy-status MSR reads" + } CounterStatus::Frozen => { "a RAPL package counter is readable but did not keep advancing while this process burned CPU, \ so it cannot meter CPU work"