From c637862902179afd544731b2edc0feedca109fff Mon Sep 17 00:00:00 2001 From: Transaction Science Date: Tue, 29 Sep 2026 10:48:58 -0400 Subject: [PATCH 01/22] eoc-meter: never construct the IOReport counter on a frozen CPU rail MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Every energy backend in this family owes one contract: measured when the platform exposes it, estimated when it doesn't, and the provenance always says which; a reading never silently upgrades its own provenance. eoc-meter's IOReport counter broke it on current macOS, and eoc sealed the result into its receipts as measured. Ground truth, read from the channels on both machines (macOS 27.0.1), not inferred from test outcomes: - M5 Max: of 364 Energy Model channels, only `GPU Energy` moved across a 3 s single-thread CPU burn (+851,622,003 nJ). `CPU Energy` held at 541,893,638 mJ (absolute) and no CPU sub-rail moved. - M3 Ultra: of 645 channels, none moved in 3 s; over 12 s only `GPU Energy` did (+14,809,813 nJ). `DIE_0_CPU Energy` / `DIE_1_CPU Energy` held at 9,494,336 / 10,212,656 mJ. - The CPU rails are not frozen for good; they are refreshed now and then. A 25-minute watch, one sample a second, saw the M5 Max's `CPU Energy` hold at 541,893,638 mJ throughout (PR #110 records 73,176,481 mJ for an M5 Max earlier today; this one has been up since 00:23). The M3 Ultra's rails held from at least 10:47 to 11:09, then both dies stepped within one sample, +1,344,869 and +1,391,637 mJ, and held again. The defect, as eoc showed it on main (860ead8c): - `MacosIoReportCounter::try_detect()` returned the counter whenever a subscription opened, so `detect()` chose it on every Apple Silicon host. Its reading sums the Energy Model rails, so on the M5 Max it advanced only through the GPU rail. - `eoc query "totally novel question"` on the M5 Max printed `cost : 50136686 µJ (measured)`: the neural stage's 50 J estimate plus a 136,686 µJ GPU-only delta, which `attach_cascade_cost` tags Measured because it is positive. The receipt seals that tag. Four runs of four did this (+136,686, +533,511, +272,220 and +231,665 µJ). - `eoc meter` printed `counter: ioreport` and `idle : 307124 µJ over 1.0 s` on the M5 Max, `idle : 10279 µJ over 1.0 s` on the M3 Ultra: GPU energy, presented as the machine's idle draw. - The test meant to guard this, `ioreport_advances_after_cpu_burn`, asserted only that the summed total advanced across a 200 ms burn. The GPU rail satisfied it on the M5 Max; on the M3 Ultra it failed (`energy must advance: before=0 after=0`); where IOReport can't open it returned early and passed. The fix. `try_detect()` now returns the counter only when `ioreport_status()` is `Live`, and `Err(NoCounter)` otherwise, so `detect()` falls through. The probe is jcp-energy's (PR #110, dfec5d44), copied in to keep the crate self-contained: burn CPU on the probing thread, re-sample after every ~25 ms of it for up to a second, and decide on the raw integers of `CPU Energy` or `DIE__CPU Energy`, never on a summed total, because the live GPU rail is what corrupts the total. Probed once per process and cached. `IoReportStatus` (`Unavailable`, `CpuRailAbsent`, `CpuRailFrozen`, `Live`) and `ioreport_status()` are public so a caller can say why a Mac is not metered by IOReport. `Err(NoCounter)` is reused rather than adding an `eoc_core::Error` variant: the reason lives in the status, and the error enum is another crate's public API. Two deliberate differences from the reference, both measured: - The probe samples its own subscription to the Energy Model group, not every channel. Sampling every channel took ~781 ms per call on the M5 Max and ~144 ms on the M3 Ultra, the group ~6 ms. On every channel a one-second probe took 2 samples and spent 3% of its window spinning on the M5 Max (12% on the M3 Ultra); on the group it takes 31 samples and spins 77% (75%). "Poll every ~25 ms while this thread spins" holds only on the group. Both subscriptions read the same absolute CPU values. The counter's own subscription is unchanged, and the test oracle reads that one. - A rail counts as live only when one CPU channel steps at two separate samples, not one. A live rail sampled every ~30 ms under load steps many times a second; a refresh steps once. On a single step, a refresh landing in the probe's second would select the counter for the life of the process, and every later refresh (~1.3 kJ per die, above) would land whole in whichever caller's interval straddled it, tagged Measured. A rail too slow to step twice in a second is declined: estimated, never over-claimed. What runs instead, end to end. `detect()` now falls through to `MacosPowermetricsCounter`, every read of which is `Err(NoCounter)`, or to `StubCounter`, which reads 0. Neither can produce a positive delta, so `attach_cascade_cost` keeps each stage's own source and no consumer here (eoc-cascade, eoc-local, eoc-streaming) can relabel the fallback as a reading; eoc-cascade is unchanged. `eoc query` now prints `cost : 50000000 µJ (estimated)` on both machines. `eoc meter` printed a failed read as `0 µJ over 1.0 s (0.000 W)`; it now prints the IOReport status and `not metered (counter read failed: no joule counter available)`. Tests. The aggregate test is replaced by three that assert against an independent oracle. The oracle samples the counter's all-channel subscription, reads the CPU rails' ABSOLUTE values (no deltas), matches names with `ends_with("CPU Energy")` rather than the implementation's group filter and die-prefix stripping, and burns CPU on its thread for three times the probe's budget between one read before and one after. Every branch asserts. The oracle is not blind: the same absolute reads show `GPU Energy` moving on the M5 Max. - `the_ioreport_counter_is_never_selected_on_a_frozen_cpu_rail`: the property. Logs `ioreport liveness: status=.. oracle=.. selected=..`. - `the_liveness_probe_never_reports_a_frozen_rail_as_live`: the dangerous direction, on a fresh uncached probe, which must also agree with the oracle on what exists. - `a_selected_ioreport_counter_meters_cpu_work`: when the counter is available its CPU rail is live and a 200 ms burn meters non-zero; when it is not, the status is not `Live`. - `a_cpu_rail_that_steps_once_is_not_live` and `a_cpu_rail_that_keeps_stepping_is_live`: the step rule, fed the M3 Ultra's observed refresh. A refresh cannot be scheduled, so no hardware test can catch a single-step rule on demand. - eoc: `a_counter_that_did_not_read_is_never_printed_as_a_figure`. Observed on both machines: `status=CpuRailFrozen oracle=CpuRailMoved(false) selected=false` and `probe=Some(Frozen)`, 9/9 passing; on the M3 Ultra through the `DIE_0_`/`DIE_1_` rails. Falsified, each reverted with git checkout: - F1, a probe that always says Live: the frozen-rail test fails on both machines ("IOReport selected (status=Live) although its CPU rail did not move under 3s of CPU burn ..."), and so does the probe test. The third test fails too on the M3 Ultra (before=0 after=0) but passes on the M5 Max, where the GPU rail moves the summed counter: the defect itself. - F2, main's ungated `try_detect()`: the frozen-rail test fails on both ("IOReport selected (status=CpuRailFrozen) ..."), and so does the third. - F3, liveness decided on every Energy Model rail, as a total would: fails on the M5 Max, where the GPU rail steps inside the probe window; passes on the M3 Ultra, where no rail stepped twice in that second. - F4, main's `eoc meter` rendering: the CLI test fails on "0 µJ over 1.0 s (0.000 W)". - F5, the reference's single-step rule: `a_cpu_rail_that_steps_once_is_not_live` fails ("one refresh counted as a live rail"). Every crate that depends on eoc-meter or eoc-cascade passes: eoc, eoc-agent, eoc-carbon, eoc-cascade, eoc-distributed, eoc-local, eoc-observability, eoc-route-learned, eoc-spec-decode, eoc-streaming, and eoc-wasm (no tests). Clippy adds nothing on eoc-meter or eoc; eoc-meter's two existing warnings remain. The probe costs about a second of one thread's CPU, once per process, on Apple Silicon; `eoc query` on the M5 Max still got faster (1.11-1.24 s, against 1.39-1.43 s on main, which paid for two all-channel reads per query). Not done here, deliberately: - `counts_once` admits CPU sub-rails beside `CPU Energy` (`PACC_0`..`PACC_5`, `MCPM*`, `PCPM*` on the M5 Max; `DIE_n_PACC*_CPM*`, `DIE_n_EACC_CPM*` on the M3 Ultra): a double count if those rails go live. - The counter's own reads sample every channel (~781 ms a read on the M5 Max) although its sum uses only the Energy Model group. - The powermetrics rung's docs say it degrades to `StubCounter`. It never does; every read errors. - With a live counter, `attach_cascade_cost` tags a stage's estimate plus the measured overhead as one Measured total. - `JouleCost::zero()` is Measured, so a cache hit claims a measured zero; the streaming meter tags a zero delta from any non-stub counter Measured; `sci_from_eval_report` tags every report Measured. - A rail that stops tracking after selection: the probe runs once a process. - Reporting the live GPU rail as an explicitly partial reading, and the grade IOReport readings carry: separate decisions. Co-Authored-By: Claude Opus 5.5 --- eoc/eoc-rs/crates/eoc-meter/src/lib.rs | 559 ++++++++++++++++++++++++- eoc/eoc-rs/crates/eoc/src/main.rs | 62 ++- 2 files changed, 594 insertions(+), 27 deletions(-) diff --git a/eoc/eoc-rs/crates/eoc-meter/src/lib.rs b/eoc/eoc-rs/crates/eoc-meter/src/lib.rs index 7a18d9414..5112a313f 100644 --- a/eoc/eoc-rs/crates/eoc-meter/src/lib.rs +++ b/eoc/eoc-rs/crates/eoc-meter/src/lib.rs @@ -10,6 +10,14 @@ //! //! - Linux RAPL is read via `/sys/class/powercap/intel-rapl:0/energy_uj`. //! No external crate required. +//! - Apple Silicon IOReport Energy Model counters are read through Apple's +//! IOReport C API (no sudo, no external binary) — **only when the CPU +//! energy rail is live** ([`ioreport_status`]). A subscription can open +//! while every CPU rail holds its value, refreshed only now and then; the +//! summed rails then advance through the GPU rail, a stale refresh, or not +//! at all, and would pass that off as a reading of CPU work. When the CPU +//! rail is not live the counter is declined and [`detect`] falls through to +//! the next rung. //! - macOS `powermetrics` is invoked via `Command::new("powermetrics")`. //! Requires root in practice — the wrapper degrades to `StubCounter` //! when the process can't fork it. @@ -18,7 +26,7 @@ // `deny` not `forbid`: the whole crate is safe except the audited Apple-Silicon // IOReport FFI module (`macos_ioreport`), which carries a scoped -// `#[allow(unsafe_code)]`. Real on-die energy reading is unreachable without the +// `#[allow(unsafe_code)]`. The on-die energy counters are unreachable without the // `unsafe extern "C"` IOReport calls; everything else stays unsafe-free. #![deny(unsafe_code)] @@ -53,18 +61,31 @@ impl JouleCounter for StubCounter { } } -// Apple Silicon IOReport energy counter — a real, no-sudo, no-external-binary -// reading of the SoC Energy Model channels (the same counters powermetrics / -// Activity Monitor use) through Apple's private IOReport C API. Replaces the -// always-erroring `powermetrics` stub so eoc reports Measured energy on Apple -// Silicon instead of degrading to Estimated. Ported from jouleclaw-energy's -// validated `apple_ioreport` backend, adapted to eoc's `JouleCounter`. +// Apple Silicon IOReport energy counter — a no-sudo, no-external-binary read of +// the SoC Energy Model channels through Apple's private IOReport C API. Ported +// from jouleclaw-energy's `apple_ioreport` backend, adapted to eoc's +// `JouleCounter`. +// +// An open subscription is not evidence that the counters are live. On macOS +// 27.0.1, for an unprivileged reader, the CPU rails (`CPU Energy`, `PCPU`, +// `MCPU*`, `PACC_*`, `DIE__CPU Energy`) hold their value under full CPU load +// on both an M5 Max and an M3 Ultra, and are refreshed only now and then (an M3 +// Ultra's held for at least 22 minutes, then stepped by ~1.3 kJ per die in one +// sample), while `GPU Energy` still moves (steadily on the M5 Max, +// intermittently on the M3 Ultra). Summed, the rails report GPU-only energy, +// zero, or a refresh of past CPU energy, and consumers such as eoc-cascade, +// eoc-local and eoc-streaming tag what a hardware counter returns as Measured. +// So the counter is constructed only when `ioreport_status()` is `Live` — the +// CPU rail itself kept stepping while the probing thread burned CPU — and +// otherwise `try_detect()` returns `Err(NoCounter)`, leaving the caller's own +// estimate. #[cfg(target_os = "macos")] #[allow(non_camel_case_types, unsafe_code)] mod macos_ioreport { use super::*; use std::os::raw::{c_int, c_long, c_void}; use std::sync::Mutex; + use std::time::{Duration, Instant}; type CFTypeRef = *const c_void; type CFStringRef = *const c_void; @@ -76,6 +97,13 @@ mod macos_ioreport { unsafe extern "C" { fn IOReportCopyAllChannels(a: u64, b: u64) -> CFMutableDictionaryRef; + fn IOReportCopyChannelsInGroup( + group: CFStringRef, + subgroup: CFStringRef, + a: u64, + b: u64, + c: u64, + ) -> CFMutableDictionaryRef; fn IOReportCreateSubscription( allowed: *const c_void, desired_channels: CFMutableDictionaryRef, @@ -143,6 +171,9 @@ mod macos_ioreport { } /// Apple Silicon IOReport energy counter (SoC-total, Measured-grade). + /// + /// Constructed only on a host whose CPU energy rail is live; see + /// [`crate::ioreport_status`]. pub struct MacosIoReportCounter { state: Mutex, } @@ -152,9 +183,18 @@ mod macos_ioreport { unsafe impl Sync for MacosIoReportCounter {} impl MacosIoReportCounter { - /// Open an IOReport subscription; `Err(NoCounter)` if IOReport can't open - /// (non-Apple-Silicon, Rosetta, or a future macOS that changes the API). + /// Open an IOReport subscription on a host whose CPU energy rail is + /// live. `Err(NoCounter)` if IOReport can't open (non-Apple-Silicon, + /// Rosetta, or a future macOS that changes the API), **and** if it opens + /// but its CPU energy rail is absent, or does not keep advancing under + /// load — then the summed rails would report GPU-only energy, zero, or a + /// stale refresh as the energy of CPU work. [`crate::ioreport_status`] + /// says which. pub fn try_detect() -> Result { + // Opening a subscription is not enough: the CPU rail must be live. + if crate::ioreport_status() != crate::IoReportStatus::Live { + return Err(Error::NoCounter); + } match Self::open() { Some(state) => Ok(Self { state: Mutex::new(state) }), None => Err(Error::NoCounter), @@ -187,6 +227,212 @@ mod macos_ioreport { Some(State { sub, channels: subscribed, prior_sample: None, cumulative_uj: 0 }) } } + + /// A subscription to the Energy Model group alone, for the liveness + /// probe. The CPU rails are the same counters the all-channel + /// subscription above reads, but sampling only this group took ~6 ms on + /// an M5 Max and an M3 Ultra, against ~780 ms and ~145 ms for every + /// channel — slow enough that a probe sampling all channels would spend + /// nearly its whole budget inside IOReport instead of burning CPU. + fn open_energy_model() -> Option { + unsafe { + let group = CFStringCreateWithCString( + std::ptr::null_mut(), + c"Energy Model".as_ptr() as *const _, + K_CFSTRING_ENCODING_UTF8, + ); + if group.is_null() { + return None; + } + let desired = IOReportCopyChannelsInGroup(group, std::ptr::null(), 0, 0, 0); + CFRelease(group); + if desired.is_null() { + return None; + } + let mut subscribed: CFMutableDictionaryRef = std::ptr::null_mut(); + let sub = IOReportCreateSubscription( + std::ptr::null(), + desired, + &mut subscribed as *mut _, + 0, + std::ptr::null(), + ); + CFRelease(desired as CFTypeRef); + if sub.is_null() { + return None; + } + if subscribed.is_null() { + CFRelease(sub); + return None; + } + Some(State { sub, channels: subscribed, prior_sample: None, cumulative_uj: 0 }) + } + } + } + + /// Whether this host's CPU energy rail is live, frozen, or not present. + #[derive(Debug, Clone, Copy, PartialEq, Eq)] + enum CpuRail { + /// A CPU energy channel stepped at [`LIVE_AFTER_STEPS`] separate samples + /// while this thread burned CPU. + Live, + /// The rail exists but held its value across the whole burn, or stepped + /// only once — a refresh, not a rail that tracks the work. + Frozen, + /// No `CPU Energy` channel exists in the Energy Model group (or no + /// sample could be taken to look for one). + Absent, + } + + /// How many separate samples one CPU energy channel must step at before the + /// rail counts as live. A live rail, sampled every ~30 ms while this thread + /// burns CPU, steps many times a second. On macOS 27.0.1 the CPU rails are + /// instead refreshed now and then: both of an M3 Ultra's held for at least + /// 22 minutes, then stepped by ~1.3 kJ each within one sample. One step inside + /// the probe's window would otherwise select the counter for the life of the + /// process, and every later refresh would land, whole, in some caller's + /// interval. + const LIVE_AFTER_STEPS: u32 = 2; + + /// Per CPU energy channel, the number of samples at which its raw cumulative + /// delta (since the probe's first sample) rose. + #[derive(Debug, Default)] + struct RailSteps { + prev: Vec, + steps: Vec, + } + + impl RailSteps { + /// Fold in the next sample's per-channel deltas, in channel order. + fn observe(&mut self, deltas: &[i64]) { + if deltas.len() != self.steps.len() { + // First sample, or the channel set changed: count from here. + self.prev = vec![0; deltas.len()]; + self.steps = vec![0; deltas.len()]; + } + for (i, &d) in deltas.iter().enumerate() { + if d > self.prev[i] { + self.steps[i] += 1; + } + self.prev[i] = d; + } + } + + fn live(&self) -> bool { + self.steps.iter().any(|&n| n >= LIVE_AFTER_STEPS) + } + } + + impl State { + /// Burn CPU on this thread for up to `budget`, re-sampling after every + /// ~25 ms of spinning, and report whether the CPU energy rail **itself** + /// keeps advancing: one channel must step at [`LIVE_AFTER_STEPS`] separate + /// samples. On a subscription to the Energy Model group alone (see + /// `open_energy_model`) a sample costs a few milliseconds, so the thread + /// spends most of the budget spinning; on an all-channel subscription it + /// would spend most of it waiting on IOReport. + /// + /// The summed total is not a liveness signal. Measured on an M5 Max and an + /// M3 Ultra, both on macOS 27.0.1: every CPU rail in the Energy Model group + /// (`CPU Energy`, `PCPU`, `MCPU*`, `PACC_*`, `DIE_n_CPU Energy`) held the + /// same cumulative value under full CPU load, while `GPU Energy` still + /// moved. A total that moves therefore proves only that the GPU did + /// something. Liveness is decided on the CPU rail alone. Takes its own + /// samples, so no counter's cumulative reading is touched. + fn probe_cpu_rail(&self, budget: Duration) -> CpuRail { + unsafe { + let s0 = IOReportCreateSamples(self.sub, self.channels, std::ptr::null()); + if s0.is_null() { + return CpuRail::Absent; + } + let started = Instant::now(); + let mut acc: u64 = 0; + let mut saw_rail = false; + let mut steps = RailSteps::default(); + while !steps.live() && started.elapsed() < budget { + let tick = Instant::now(); + while tick.elapsed() < Duration::from_millis(25) { + acc = acc.wrapping_add(1); + } + std::hint::black_box(acc); + let s1 = IOReportCreateSamples(self.sub, self.channels, std::ptr::null()); + if s1.is_null() { + continue; + } + let delta = IOReportCreateSamplesDelta(s0, s1, std::ptr::null()); + CFRelease(s1); + if delta.is_null() { + continue; + } + let deltas = cpu_rail_deltas(delta); + CFRelease(delta); + if !deltas.is_empty() { + saw_rail = true; + steps.observe(&deltas); + } + } + CFRelease(s0); + match (saw_rail, steps.live()) { + (_, true) => CpuRail::Live, + (true, false) => CpuRail::Frozen, + (false, false) => CpuRail::Absent, + } + } + } + } + + /// Probe the CPU rail on an Energy Model subscription. Backs the cached + /// [`crate::ioreport_status`]; the probe's subscription is dropped after. + pub(crate) fn probe_status(budget: Duration) -> crate::IoReportStatus { + match MacosIoReportCounter::open_energy_model() { + Some(state) => match state.probe_cpu_rail(budget) { + CpuRail::Live => crate::IoReportStatus::Live, + CpuRail::Frozen => crate::IoReportStatus::CpuRailFrozen, + CpuRail::Absent => crate::IoReportStatus::CpuRailAbsent, + }, + // No Energy Model subscription. Tell "no IOReport" from "no such + // group" by opening the subscription the counter itself would use. + None => match MacosIoReportCounter::open() { + Some(_) => crate::IoReportStatus::CpuRailAbsent, + None => crate::IoReportStatus::Unavailable, + }, + } + } + + /// In a delta sample: the raw integer delta of each CPU energy channel + /// (`CPU Energy`, or `DIE__CPU Energy` on multi-die parts), in channel + /// order; empty when there is none. Raw integers, before unit conversion, so + /// a small advance in a fine unit cannot truncate to zero. + fn cpu_rail_deltas(delta: CFDictionaryRef) -> Vec { + let mut deltas = Vec::new(); + unsafe { + let key = CFStringCreateWithCString( + std::ptr::null_mut(), + c"IOReportChannels".as_ptr() as *const _, + K_CFSTRING_ENCODING_UTF8, + ); + if key.is_null() { + return deltas; + } + let arr = CFDictionaryGetValue(delta, key as *const _) as CFArrayRef; + CFRelease(key as CFTypeRef); + if arr.is_null() { + return deltas; + } + for i in 0..CFArrayGetCount(arr) { + let item = CFArrayGetValueAtIndex(arr, i); + if item.is_null() || !cfstring_eq_ic(IOReportChannelGetGroup(item), "energy model") { + continue; + } + let Some(name) = cfstring_to_string(IOReportChannelGetChannelName(item)) else { + continue; + }; + if strip_die_prefix(&name).eq_ignore_ascii_case("CPU Energy") { + deltas.push(IOReportSimpleGetIntegerValue(item, 0) as i64); + } + } + } + deltas } impl JouleCounter for MacosIoReportCounter { @@ -344,22 +590,239 @@ mod macos_ioreport { assert!(!counts_once("GPU0_0")); } + /// A rail that steps once and holds is a refresh, not a rail tracking + /// the work — even a large step, even on every die at once. The deltas are + /// the refresh observed on an M3 Ultra (`DIE_0_`/`DIE_1_CPU Energy`, mJ). #[test] - fn ioreport_advances_after_cpu_burn() { - let Ok(c) = MacosIoReportCounter::try_detect() else { - return; // not Apple Silicon + fn a_cpu_rail_that_steps_once_is_not_live() { + let mut s = RailSteps::default(); + for d in [[0, 0], [0, 0], [1_344_869, 1_391_637], [1_344_869, 1_391_637]] { + s.observe(&d); + } + assert!(!s.live(), "one refresh counted as a live rail: {s:?}"); + // Two dies stepping once each, at different samples, is still one step each. + let mut s = RailSteps::default(); + for d in [[5, 0], [5, 7], [5, 7]] { + s.observe(&d); + } + assert!(!s.live(), "two single-step channels counted as a live rail: {s:?}"); + // No CPU rail at all. + let mut s = RailSteps::default(); + s.observe(&[]); + assert!(!s.live()); + } + + /// A rail that keeps stepping while the thread burns is live. + #[test] + fn a_cpu_rail_that_keeps_stepping_is_live() { + let mut s = RailSteps::default(); + for d in [[0], [3], [3], [9]] { + s.observe(&d); + } + assert!(s.live(), "a rail that stepped at two samples was not live: {s:?}"); + let mut s = RailSteps::default(); + for d in [[0, 4], [0, 11]] { + s.observe(&d); + } + assert!(s.live(), "one die stepping at every sample was not live: {s:?}"); + } + + /// What the independent oracle saw on this host. + #[derive(Debug, PartialEq)] + enum Oracle { + NoIoReport, + NoCpuRail, + CpuRailMoved(bool), + } + + /// An oracle that shares nothing with the liveness probe but the raw FFI: + /// it samples the subscription the COUNTER uses (every channel, not the + /// probe's Energy Model group), reads each CPU rail's ABSOLUTE cumulative + /// value (no delta samples), matches names with `ends_with("CPU Energy")` + /// rather than the die-prefix stripping and group filter the + /// implementation uses, and burns CPU on this thread for the whole + /// `budget` between one read before and one after — an all-channel + /// sample can take most of a second, so it is not polled mid-burn. + fn oracle_cpu_rail(budget: Duration) -> Oracle { + let Some(st) = MacosIoReportCounter::open() else { + return Oracle::NoIoReport; }; - let _ = c.read_microjoules(); - let before = c.read_microjoules().unwrap(); - let start = std::time::Instant::now(); + let read = || -> Option> { + unsafe { + let smp = IOReportCreateSamples(st.sub, st.channels, std::ptr::null()); + if smp.is_null() { + return None; + } + let key = CFStringCreateWithCString( + std::ptr::null_mut(), + c"IOReportChannels".as_ptr() as *const _, + K_CFSTRING_ENCODING_UTF8, + ); + let mut vals = Vec::new(); + if !key.is_null() { + let arr = CFDictionaryGetValue(smp, key as *const _) as CFArrayRef; + CFRelease(key as CFTypeRef); + if !arr.is_null() { + for i in 0..CFArrayGetCount(arr) { + let it = CFArrayGetValueAtIndex(arr, i); + if it.is_null() { + continue; + } + let name = cfstring_to_string(IOReportChannelGetChannelName(it)) + .unwrap_or_default(); + if name.ends_with("CPU Energy") { + vals.push(IOReportSimpleGetIntegerValue(it, 0) as i64); + } + } + } + } + CFRelease(smp); + Some(vals) + } + }; + let Some(v0) = read() else { + return Oracle::NoIoReport; + }; + if v0.is_empty() { + return Oracle::NoCpuRail; + } + let started = Instant::now(); let mut acc: u64 = 0; - while start.elapsed() < std::time::Duration::from_millis(200) { + while started.elapsed() < budget { acc = acc.wrapping_add(1); } std::hint::black_box(acc); - let _ = c.read_microjoules(); - let after = c.read_microjoules().unwrap(); - assert!(after > before, "energy must advance: before={before} after={after}"); + let v1 = read().expect("oracle: IOReportCreateSamples failed after the burn"); + Oracle::CpuRailMoved(v1 != v0) + } + + /// The property that matters: the IOReport counter is **never** + /// constructed, and never picked by [`crate::detect`], when its CPU rail + /// is frozen — then it would report GPU-only energy, or zero, as the + /// energy of CPU work, and consumers would tag it Measured. Asserts the + /// meter's decision against what the rail actually did; every branch + /// asserts, so there is no way to pass without checking anything. + /// + /// The oracle gets three times the probe's budget, so a rail too slow for + /// the probe but moving within the oracle's window is the only case where + /// declining is accepted — and that fails safe: estimated, never + /// over-claimed. + #[test] + fn the_ioreport_counter_is_never_selected_on_a_frozen_cpu_rail() { + let status = crate::ioreport_status(); + let constructible = MacosIoReportCounter::try_detect().is_ok(); + let selected = crate::detect().name() == "ioreport"; + let oracle = oracle_cpu_rail(crate::CPU_RAIL_LIVENESS_BUDGET * 3); + // Recorded so a CI log says which branch below was actually asserted. + eprintln!("ioreport liveness: status={status:?} oracle={oracle:?} selected={selected}"); + match oracle { + Oracle::NoIoReport => { + assert!(!selected, "IOReport selected (status={status:?}) but no subscription opens"); + assert_eq!(status, IoReportStatus::Unavailable); + } + Oracle::NoCpuRail => { + assert!(!selected, "IOReport selected (status={status:?}) but it has no CPU rail"); + assert_eq!(status, IoReportStatus::CpuRailAbsent); + } + Oracle::CpuRailMoved(false) => { + assert!( + !selected, + "IOReport selected (status={status:?}) although its CPU rail did not move \ + under {:?} of CPU burn — it would report GPU-only energy, or zero, as the \ + energy of CPU work", + crate::CPU_RAIL_LIVENESS_BUDGET * 3 + ); + assert_eq!(status, IoReportStatus::CpuRailFrozen); + } + Oracle::CpuRailMoved(true) => { + // Live rail: the probe had a third of the oracle's time. Live must + // be selected; a probe that missed a slow rail declines (safe side). + assert!( + matches!(status, IoReportStatus::Live | IoReportStatus::CpuRailFrozen), + "the oracle saw a CPU rail move but the status is {status:?}" + ); + assert_eq!(selected, status == IoReportStatus::Live); + } + } + // On macOS IOReport is the first rung, so detect() picks it exactly when + // try_detect() constructs it, and that is exactly when the status is Live. + assert_eq!(selected, constructible, "detect() and try_detect() disagree"); + assert_eq!(constructible, status == IoReportStatus::Live); + } + + /// The dangerous direction on its own, against a fresh (uncached) probe on + /// the same subscription the status uses: it never reports Live for a + /// rail the oracle saw hold, and on every host it agrees with the oracle + /// about what exists. + #[test] + fn the_liveness_probe_never_reports_a_frozen_rail_as_live() { + let probe = MacosIoReportCounter::open_energy_model() + .map(|st| st.probe_cpu_rail(crate::CPU_RAIL_LIVENESS_BUDGET)); + let oracle = oracle_cpu_rail(crate::CPU_RAIL_LIVENESS_BUDGET * 3); + eprintln!("ioreport probe: probe={probe:?} oracle={oracle:?}"); + if probe == Some(CpuRail::Live) { + assert_eq!( + oracle, + Oracle::CpuRailMoved(true), + "probe said Live, oracle saw the CPU rail hold its value" + ); + } + match oracle { + Oracle::NoIoReport => assert_eq!(probe, None), + Oracle::NoCpuRail => assert!( + matches!(probe, None | Some(CpuRail::Absent)), + "the oracle found no CPU rail but the probe reported {probe:?}" + ), + Oracle::CpuRailMoved(false) => assert_eq!( + probe, + Some(CpuRail::Frozen), + "the CPU rail held under the oracle's burn, so the probe must report Frozen" + ), + Oracle::CpuRailMoved(true) => assert!( + matches!(probe, Some(CpuRail::Live | CpuRail::Frozen)), + "the oracle saw a CPU rail move but the probe reported {probe:?}" + ), + } + } + + /// When the counter IS available, a CPU burn meters as non-zero energy + /// through a rail that moves; when it is not, the status says why and is + /// never `Live`. + #[test] + fn a_selected_ioreport_counter_meters_cpu_work() { + match MacosIoReportCounter::try_detect() { + Ok(c) => { + assert_eq!( + crate::ioreport_status(), + IoReportStatus::Live, + "try_detect() constructed the counter although the status is not Live" + ); + let rail = MacosIoReportCounter::open_energy_model() + .map(|st| st.probe_cpu_rail(crate::CPU_RAIL_LIVENESS_BUDGET)); + assert_eq!(rail, Some(CpuRail::Live), "counter available but its CPU rail is {rail:?}"); + let _ = c.read_microjoules(); // first read only sets the baseline sample + let before = c.read_microjoules().unwrap(); + let t = Instant::now(); + let mut acc: u64 = 0; + while t.elapsed() < Duration::from_millis(200) { + acc = acc.wrapping_add(1); + } + std::hint::black_box(acc); + let after = c.read_microjoules().unwrap(); + assert!( + after > before, + "a live CPU rail metered 200 ms of CPU work as zero: before={before} after={after}" + ); + } + Err(e) => { + assert!(matches!(e, Error::NoCounter), "declined with {e:?}, expected NoCounter"); + assert_ne!( + crate::ioreport_status(), + IoReportStatus::Live, + "status Live but try_detect() declined" + ); + } + } } } } @@ -367,6 +830,53 @@ mod macos_ioreport { #[cfg(target_os = "macos")] pub use macos_ioreport::MacosIoReportCounter; +/// Why the IOReport counter was or was not made available on this host. +#[derive(Debug, Clone, Copy, PartialEq, Eq)] +pub enum IoReportStatus { + /// No IOReport subscription could be opened (not macOS, not Apple + /// Silicon, or Rosetta). + Unavailable, + /// IOReport opened, but no `CPU Energy` rail was found in its Energy Model + /// group (or that group could not be subscribed) — an unfamiliar channel + /// naming. Never used. + CpuRailAbsent, + /// The CPU energy rail exists but did not keep advancing while CPU was + /// burned: it held, or stepped only once (a refresh). The platform is not + /// exposing live CPU energy to this process, so using the counter would + /// report GPU-only energy, zero, or a stale refresh as the energy of CPU + /// work. Observed on macOS 27.0.1 on both an M5 Max and an M3 Ultra. + CpuRailFrozen, + /// The CPU energy rail keeps advancing under load; the counter is available. + Live, +} + +/// How long the liveness probe burns CPU before concluding the CPU rail is +/// frozen: a live rail gets a second of single-thread CPU work to count, +/// sampled after every ~25 ms of it, and must step at two separate samples. +/// A rail too slow for that fails safe — declined, so callers estimate; never +/// over-claimed. Paid once per process; see [`ioreport_status`]. +#[cfg(target_os = "macos")] +pub const CPU_RAIL_LIVENESS_BUDGET: std::time::Duration = std::time::Duration::from_millis(1000); + +/// This host's IOReport status, probed once per process and cached. +/// +/// `MacosIoReportCounter::try_detect` (and so [`detect`]) makes the IOReport +/// counter available only when this is [`IoReportStatus::Live`]. The first +/// call on macOS burns CPU on the calling thread for up to +/// `CPU_RAIL_LIVENESS_BUDGET`. Exposed so a caller or CLI can say *why* an +/// Apple Silicon host is not metered by IOReport. +pub fn ioreport_status() -> IoReportStatus { + #[cfg(target_os = "macos")] + { + static STATUS: std::sync::OnceLock = std::sync::OnceLock::new(); + *STATUS.get_or_init(|| macos_ioreport::probe_status(CPU_RAIL_LIVENESS_BUDGET)) + } + #[cfg(not(target_os = "macos"))] + { + IoReportStatus::Unavailable + } +} + #[cfg(target_os = "linux")] mod linux_rapl { use super::*; @@ -496,8 +1006,9 @@ pub use nvml::NvmlCounter; /// Runtime-detect the best available counter on this host. /// /// Preference order: NVML (if compiled with `cuda` and present) → RAPL on -/// Linux → IOReport on macOS (real Energy-Model counters, no sudo) → -/// powermetrics on macOS → `StubCounter`. +/// Linux → IOReport on macOS (Energy-Model counters, no sudo; only when its +/// CPU energy rail is live — see [`ioreport_status`]) → powermetrics on +/// macOS → `StubCounter`. pub fn detect() -> Box { #[cfg(feature = "cuda")] if let Ok(c) = NvmlCounter::try_detect() { @@ -537,4 +1048,10 @@ mod tests { let _ = c.read_microjoules(); assert!(!c.name().is_empty()); } + + #[cfg(not(target_os = "macos"))] + #[test] + fn ioreport_is_unavailable_off_macos() { + assert_eq!(ioreport_status(), IoReportStatus::Unavailable); + } } diff --git a/eoc/eoc-rs/crates/eoc/src/main.rs b/eoc/eoc-rs/crates/eoc/src/main.rs index 0df01d7c6..4498e891d 100644 --- a/eoc/eoc-rs/crates/eoc/src/main.rs +++ b/eoc/eoc-rs/crates/eoc/src/main.rs @@ -5,7 +5,8 @@ //! - `eoc query ""` — run a prompt through the cascade and print //! the stage that resolved it plus the attributed joule cost. //! - `eoc meter` — detect available hardware energy counters and report -//! the micro-joules consumed during one second of idle. +//! the micro-joules consumed during one second of idle, or why nothing +//! was metered. #![forbid(unsafe_code)] @@ -40,7 +41,8 @@ enum Cmd { /// The prompt to resolve. prompt: String, }, - /// Detect the joule counter and report idle micro-joules per second. + /// Detect the joule counter and report idle micro-joules per second, or + /// why nothing was metered. Meter, } @@ -90,10 +92,58 @@ async fn run_query(prompt: String) -> Result<()> { async fn run_meter() -> Result<()> { let counter = detect(); println!("counter: {}", counter.name()); - let start = counter.read_microjoules().unwrap_or(0); + #[cfg(target_os = "macos")] + println!("ioreport: {}", ioreport_line(eoc_meter::ioreport_status())); + let start = counter.read_microjoules(); tokio::time::sleep(Duration::from_secs(1)).await; - let end = counter.read_microjoules().unwrap_or(0); - let delta = end.saturating_sub(start); - println!("idle : {} µJ over 1.0 s ({:.3} W)", delta, (delta as f64) / 1_000_000.0); + let end = counter.read_microjoules(); + println!("idle : {}", idle_line(counter.name(), start, end)); Ok(()) } + +/// Why the IOReport counter is, or is not, the one in use on this Mac. +#[cfg(target_os = "macos")] +fn ioreport_line(status: eoc_meter::IoReportStatus) -> &'static str { + use eoc_meter::IoReportStatus; + match status { + IoReportStatus::Live => "live (its CPU energy rail keeps advancing under load)", + IoReportStatus::CpuRailFrozen => { + "declined: its CPU energy rail did not keep advancing under load, so \ + its readings would be GPU-only energy, zero, or a stale refresh" + } + IoReportStatus::CpuRailAbsent => "declined: no CPU energy rail in its Energy Model group", + IoReportStatus::Unavailable => "unavailable: no IOReport subscription could be opened", + } +} + +/// The idle figure, printed only when a hardware counter actually read. +/// A failed read or the stub is reported as not metered — never as 0 W. +fn idle_line(counter: &str, start: eoc_core::Result, end: eoc_core::Result) -> String { + if counter == "stub" { + return "not metered (stub counter: no hardware energy counter)".to_string(); + } + match (start, end) { + (Ok(s), Ok(e)) => { + let delta = e.saturating_sub(s); + format!("{delta} µJ over 1.0 s ({:.3} W)", (delta as f64) / 1_000_000.0) + } + (Err(err), _) | (_, Err(err)) => format!("not metered (counter read failed: {err})"), + } +} + +#[cfg(test)] +mod tests { + use super::idle_line; + use eoc_core::Error; + + #[test] + fn a_counter_that_did_not_read_is_never_printed_as_a_figure() { + let failed = idle_line("powermetrics", Err(Error::NoCounter), Err(Error::NoCounter)); + assert_eq!(failed, "not metered (counter read failed: no joule counter available)"); + let half = idle_line("rapl", Ok(1_000), Err(Error::Io("gone".into()))); + assert!(half.starts_with("not metered"), "{half}"); + let stub = idle_line("stub", Ok(0), Ok(0)); + assert!(stub.starts_with("not metered"), "{stub}"); + assert_eq!(idle_line("rapl", Ok(1_000), Ok(501_000)), "500000 µJ over 1.0 s (0.500 W)"); + } +} -- 2.54.0 (Apple Git-157) From 3b827e2a527e690df1e1aa1a26279c4cee6989c1 Mon Sep 17 00:00:00 2001 From: Transaction Science Date: Wed, 30 Sep 2026 08:04:39 -0400 Subject: [PATCH 02/22] eoc-meter: CPU-rail liveness pairs consecutive samples, and every step of it is tested MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit c6378629 gated the IOReport counter on a CPU-rail liveness probe, but no one had checked it against the family rule, and the part of the probe where this family's defect first appeared was never tested. Auditing it against the rule found these gaps: - The probe loop stopped at the one-second budget however few intervals it had seen. The rule is to keep sampling until the rail is Live, or until the budget has elapsed AND at least two intervals were seen, capped at five budgets. The loop ran inline in the FFI function, so its pairing, termination and failed-sample handling could not be tested at all. - A failed first sample, or a probe that got no interval, was reported as CpuRailAbsent ("no CPU rail"). The rule reports Unavailable: nothing was learned. - The oracle read the rail twice, before and after a burn on its own thread, and called one change "moved". So a single batch refresh inside its window counted as a moving rail. It opened its subscription through the implementation's MacosIoReportCounter::open() and converted names with the implementation's cfstring_to_string. So c6378629's doc claim that it "shares nothing with the liveness probe but the raw FFI" was false, and so was the message's "the same absolute reads show GPU Energy moving": the oracle never read GPU Energy. - Nothing pinned the once-per-process cache, the probe's cadence on the Energy Model group, the verdict-to-status mapping, the Live-only gate, or what a declined host meters. - open_energy_model() leaked the subscribed-channel dictionary when IOReportCreateSubscription returned null. The counter's own open() did too, before this branch. What holds now: - The rule is pure (mod rail_liveness). It is compiled on every platform under test, so the Linux CI leg runs it too. The FFI only takes a sample and lists each Energy Model channel as (name, absolute raw integer). RailWatch::observe keys each CPU rail (the name after stripping DIE__ is "CPU Energy", case-insensitive) by (channel name, ordinal among same-named channels in that sample). It compares each usable sample with the previous usable one, never with the first. So a rail advances at most once per interval, and dies are never summed. The M3 Ultra lists 264 of its 645 Energy Model names twice. Two same-named rails stepping together are one advance each, not two advances of one rail. - watch(sample, tick, now, budget) is the probe loop. It is generic over the sampler, the tick and the clock. The real probe passes the IOReport sampler, 25 ms of CPU spin on the calling thread, and Instant. Live means the same rail advanced in >= 2 intervals. The loop keeps sampling until Live, or until the budget has elapsed with >= 2 intervals, capped at five budgets. No interval at all is no verdict, which maps to Unavailable. - status_of() and admits_counter() are pure. try_detect() admits the counter only on Live. - PROBE_RUNS counts the real probe behind ioreport_status(). - The oracle opens its own all-channel subscription with raw calls and converts names with its own helper. It reads absolute values for ends_with("CPU Energy") and ends_with("GPU Energy") over and over for 3 s, while a helper thread burns CPU for the whole window. It says the rail moved only if one rail changed at >= 2 reads, and it must get >= 3 reads before it may say Frozen. - The powermetrics docs said the rung degrades to StubCounter. It never does: every read errors. The docs now say so, and a test pins it. Tests added: - Scripted samples and a scripted clock: - (a) the M5 Max refresh 480,083,160 -> 541,893,638 is Frozen: exactly 40 intervals and 41 samples, while GPU Energy moves at every sample; - (b) the M3 Ultra's two dies stepping +1,344,869 / +1,391,637 in one sample is Frozen; - (c) same-named rails are counted apart; - (d) a rail stepping every sample is Live after exactly 2 intervals and 3 samples; - (e) failed samples sample to the cap (201 samples) and give no verdict, which is Unavailable; - (f) past the budget the probe stops at once with two intervals, and keeps sampling with fewer; - (g) one step on each die, in different intervals, is Frozen. The verdict-to-status mapping, the Live-only gate and the name rule each have a test. - Hardware tests: - the_liveness_probe_runs_once_per_process: exactly one run after repeated ioreport_status(), detect() and try_detect() calls; - the fresh-probe test requires >= 10 intervals, and the probe thread on CPU for >= 50% of the probe, whenever the verdict is not Live; - on a declined host, detect()'s fallback measures 100 ms of real CPU work as no figure (both reads error, or it is the stub). - In the eoc binary, work_on_an_unmetered_host_keeps_its_estimate runs the CLI's own cascade around a backend that burns 100 ms. On this host it must come back as the backend's 50,000,000 µJ estimate, tagged estimated. (async-trait becomes a dev-dependency of eoc for that backend.) `eoc meter` now says Unavailable can also mean "yielded no samples". Observed on the M5 Max, 3 runs of 3, macOS 27.0.1: ioreport probe: probed=Some(Some(Frozen)) cached=CpuRailFrozen took=1.11s on-cpu=95-98% intervals=30-31 ioreport liveness: status=CpuRailFrozen oracle=CpuRailMoved(false) selected=false (oracle: 9-12 reads, CPU rail changes [0], GPU rail 8-11) eoc-meter 19 passed, eoc 2 passed Observed on the M3 Ultra (10.10.10.2), same binaries by sha256: intervals=32, on-cpu=100%, CPU rail changes [0, 0] on DIE_0/DIE_1, GPU rail changed at 17 and 20 of 21 oracle reads. eoc-meter 19 passed, eoc 2 passed. On both machines, `eoc query` prints `cost : 50000000 µJ (estimated)`, and `eoc meter` prints the declined status and "not metered". Falsified, each restored with git checkout (clean status every time). Each of these fails at least one test: - comparing every sample with the first, in RailWatch and in the loop; - one advance counting as Live; - advances summed across dies; - rails keyed by name alone; - GPU Energy counted as a CPU rail; - no DIE_ stripping; - c6378629's stop-at-budget loop; - no five-budget cap; - None mapped to Live or to CpuRailAbsent; - the gate admitting CpuRailAbsent; - Absent reported for rails that did not advance; - a coarse 1e9 threshold; - no OnceLock (the probe ran 14 times); - an all-channel probe (2 intervals in 1.86 s); - a probe that sleeps (26% on CPU); - a powermetrics fallback reading Ok(0) or a time-based count; - main's ungated try_detect(); - a probe that always says Live. One mutation survives: the counter's own read returning Ok(0). On a frozen host the counter is never constructed, so only a Live host runs that path. Leak check: 150 probes plus 150 all-channel opens under leaks --atExit gave "0 leaks for 0 total leaked bytes". A control with CFRelease(sample) removed gave 100,080 leaks. The whole eoc workspace passes: 594 passed, 0 failed, 3 ignored. That covers every dependant c6378629 ran. Clippy adds nothing: the two eoc-meter warnings are on lines that predate this branch. Not done here: counts_once still admits CPU sub-rails; the counter's reads still sample every channel; and a rail that freezes after selection is not re-probed. Co-Authored-By: Claude Opus 5.5 --- eoc/eoc-rs/Cargo.lock | 1 + eoc/eoc-rs/crates/eoc-meter/src/lib.rs | 1192 +++++++++++++++++------- eoc/eoc-rs/crates/eoc/Cargo.toml | 3 + eoc/eoc-rs/crates/eoc/src/main.rs | 73 +- 4 files changed, 904 insertions(+), 365 deletions(-) diff --git a/eoc/eoc-rs/Cargo.lock b/eoc/eoc-rs/Cargo.lock index 5232af78d..02cf854f8 100644 --- a/eoc/eoc-rs/Cargo.lock +++ b/eoc/eoc-rs/Cargo.lock @@ -716,6 +716,7 @@ name = "eoc" version = "0.1.0" dependencies = [ "anyhow", + "async-trait", "clap", "eoc-cache", "eoc-cascade", diff --git a/eoc/eoc-rs/crates/eoc-meter/src/lib.rs b/eoc/eoc-rs/crates/eoc-meter/src/lib.rs index 5112a313f..1397e9a68 100644 --- a/eoc/eoc-rs/crates/eoc-meter/src/lib.rs +++ b/eoc/eoc-rs/crates/eoc-meter/src/lib.rs @@ -18,9 +18,9 @@ //! at all, and would pass that off as a reading of CPU work. When the CPU //! rail is not live the counter is declined and [`detect`] falls through to //! the next rung. -//! - macOS `powermetrics` is invoked via `Command::new("powermetrics")`. -//! Requires root in practice — the wrapper degrades to `StubCounter` -//! when the process can't fork it. +//! - macOS `powermetrics`: detected when the binary is on the PATH, but the +//! reference wrapper does not parse it, so every read returns +//! `Err(NoCounter)`: no figure, and callers keep their own estimate. //! - NVIDIA NVML is gated behind the `cuda` feature. Off by default. //! - WASM and unknown targets fall through to `StubCounter`. @@ -61,6 +61,399 @@ impl JouleCounter for StubCounter { } } +// The CPU-rail liveness rule, kept apart from the IOReport FFI so every part of +// it — the name rule, the pairing of samples, the per-rail count, when to stop +// sampling, and the verdict's mapping to a status — runs under test with +// scripted samples and a scripted clock. The FFI only takes a sample and lists +// its channels as (name, absolute raw integer); everything that decides lives +// here. Compiled for tests on every platform, so a Linux CI leg runs it too. +#[cfg(any(target_os = "macos", test))] +mod rail_liveness { + use std::time::Duration; + + /// In how many separate sample intervals the same CPU rail must advance + /// before it counts as live. + /// + /// One is not enough. On macOS 27.0.1 an unprivileged reader's CPU rails + /// are not permanently frozen; they are refreshed in rare batches. An M3 + /// Ultra's two dies held for at least 22 minutes, then stepped +1,344,869 + /// and +1,391,637 mJ within one 100 ms sample, then held again; an M5 Max's + /// `CPU Energy` did not change in 13,614 samples over 25 minutes, yet had + /// moved from 480,083,160 to 541,893,638 mJ earlier that hour. A probe + /// whose window straddles a refresh sees one step. Called live, it would + /// select the counter for the life of the process, and every later refresh + /// would land, whole, in whichever caller's interval straddled it. A live + /// rail sampled every ~30 ms under load steps in nearly every interval. + pub(crate) const LIVE_RAIL_ADVANCES: u32 = 2; + + /// However slow sampling is, the probe stops after this many budgets. + pub(crate) const PROBE_BUDGET_CAP: u32 = 5; + + /// What the liveness probe saw of the CPU energy rail. + #[derive(Debug, Clone, Copy, PartialEq, Eq)] + pub(crate) enum CpuRail { + /// The same CPU energy rail advanced in at least + /// [`LIVE_RAIL_ADVANCES`] separate sample intervals. + Live, + /// A CPU energy rail exists, but none advanced in that many intervals: + /// it held, or stepped once (a batch refresh). + Frozen, + /// Samples were taken, and no `CPU Energy` channel was in them. + Absent, + } + + /// `DIE__CPU Energy` → `CPU Energy`; any other name unchanged. + pub(crate) fn strip_die_prefix(name: &str) -> &str { + if let Some(rest) = name.strip_prefix("DIE_") { + let after = rest.trim_start_matches(|c: char| c.is_ascii_digit()); + if let Some(r) = after.strip_prefix('_') { + return r; + } + } + name + } + + /// A CPU energy aggregate: `CPU Energy`, optionally `DIE__`-prefixed, + /// compared case-insensitively. Sub-rails (`PCPU`, `MCPU*`, `PACC_*`, + /// `DIE_n_PACC*_CPM*`) are not the aggregate and are not watched. + pub(crate) fn is_cpu_energy_rail(name: &str) -> bool { + strip_die_prefix(name).eq_ignore_ascii_case("CPU Energy") + } + + /// One CPU rail's identity within a sample: its channel name and its + /// ordinal among the channels of that name in the same sample. An M3 + /// Ultra's Energy Model group lists 264 of its 645 channel names twice + /// (once per die, with no `DIE_` prefix); the CPU aggregates are uniquely + /// named today, but two same-named rails stepping together in one refresh + /// must count as one advance each, never as two advances of one rail. + pub(crate) type RailKey = (String, u32); + + /// The CPU rails in one sample, keyed by [`RailKey`], in sample order. + fn cpu_rails(sample: &[(String, i64)]) -> Vec<(RailKey, i64)> { + let mut rails: Vec<(RailKey, i64)> = Vec::new(); + for (name, raw) in sample { + if !is_cpu_energy_rail(name) { + continue; + } + let ordinal = rails.iter().filter(|((n, _), _)| n == name).count() as u32; + rails.push(((name.clone(), ordinal), *raw)); + } + rails + } + + /// The liveness rule over a sequence of samples. + /// + /// Each usable sample is compared with the PREVIOUS usable one — never + /// with the first — on each CPU rail's absolute raw integer, before unit + /// conversion (a small advance in a fine unit cannot truncate to zero). A + /// rail advances in an interval when its value rose over that interval; so + /// a rail advances at most once per interval, and each rail is counted on + /// its own: two dies stepping once each never add up to a live rail. The + /// summed total is never consulted, because the GPU rail, which is live on + /// an M5 Max, is exactly what corrupts it. + #[derive(Debug, Default)] + pub(crate) struct RailWatch { + /// The CPU rails of the last usable sample. + prev: Option>, + /// Intervals observed: pairs of consecutive usable samples. + pub(crate) intervals: u32, + /// Every CPU rail seen in an interval, with the number of intervals in + /// which it advanced. + pub(crate) rails: Vec<(RailKey, u32)>, + } + + impl RailWatch { + /// Fold in one usable sample: every channel in it, as (name, absolute + /// raw integer). A sample that could not be taken is simply not + /// observed, so the next usable one pairs with the last usable one. + pub(crate) fn observe(&mut self, sample: &[(String, i64)]) { + let cur = cpu_rails(sample); + if let Some(prev) = &self.prev { + self.intervals += 1; + for (key, now) in &cur { + let advanced = prev + .iter() + .find(|(k, _)| k == key) + .is_some_and(|(_, before)| now > before); + match self.rails.iter_mut().find(|(k, _)| k == key) { + Some((_, n)) => *n += u32::from(advanced), + None => self.rails.push((key.clone(), u32::from(advanced))), + } + } + } + self.prev = Some(cur); + } + + /// Has one CPU rail advanced in enough separate intervals to be live? + pub(crate) fn is_live(&self) -> bool { + self.rails.iter().any(|(_, n)| *n >= LIVE_RAIL_ADVANCES) + } + + /// Keep sampling? Until the rail proves live; or until `budget` has + /// elapsed **and** enough intervals were observed for a live rail to + /// have advanced [`LIVE_RAIL_ADVANCES`] times (slow or failed samples + /// must not make a live rail look frozen); and never past + /// [`PROBE_BUDGET_CAP`] budgets. + pub(crate) fn wants_more(&self, elapsed: Duration, budget: Duration) -> bool { + !self.is_live() + && elapsed < budget * PROBE_BUDGET_CAP + && (elapsed < budget || self.intervals < LIVE_RAIL_ADVANCES) + } + + /// The verdict, or `None` when not one interval could be observed. + pub(crate) fn verdict(&self) -> Option { + if self.intervals == 0 { + None + } else if self.is_live() { + Some(CpuRail::Live) + } else if self.rails.is_empty() { + Some(CpuRail::Absent) + } else { + Some(CpuRail::Frozen) + } + } + } + + /// The probe loop, generic over how a sample is taken, what happens between + /// samples, and the clock, so it runs under test exactly as it runs on + /// hardware. Takes a sample, then ticks and samples again for as long as + /// [`RailWatch::wants_more`] says. `sample` returns `None` when no sample + /// could be taken. The real probe passes an IOReport sampler, ~25 ms of CPU + /// spin on the calling thread, and [`std::time::Instant`]. + pub(crate) fn watch( + mut sample: impl FnMut() -> Option>, + mut tick: impl FnMut(), + now: impl Fn() -> Duration, + budget: Duration, + ) -> RailWatch { + let started = now(); + let mut w = RailWatch::default(); + if let Some(s) = sample() { + w.observe(&s); + } + while w.wants_more(now().saturating_sub(started), budget) { + tick(); + if let Some(s) = sample() { + w.observe(&s); + } + } + w + } + + /// The status a probe verdict gives this host. No verdict — not one + /// interval could be observed — is `Unavailable`: nothing was learned about + /// the rail, and only `Live` makes the counter available. + pub(crate) fn status_of(verdict: Option) -> crate::IoReportStatus { + match verdict { + None => crate::IoReportStatus::Unavailable, + Some(CpuRail::Absent) => crate::IoReportStatus::CpuRailAbsent, + Some(CpuRail::Frozen) => crate::IoReportStatus::CpuRailFrozen, + Some(CpuRail::Live) => crate::IoReportStatus::Live, + } + } + + /// Whether a host with this status may use the IOReport counter: only when + /// its CPU rail is live. Absent, frozen or unavailable all decline, so + /// callers estimate rather than seal a reading that did not track the work. + pub(crate) fn admits_counter(status: crate::IoReportStatus) -> bool { + status == crate::IoReportStatus::Live + } + + #[cfg(test)] + mod tests { + use super::*; + use std::cell::Cell; + + const TICK: Duration = Duration::from_millis(25); + const BUDGET: Duration = Duration::from_millis(1000); + + /// Run [`watch`] over scripted samples — `gen(i)` is the i-th sample, + /// `None` for one that could not be taken — with a clock that only the + /// tick moves, by `step`. Returns the watch and how many samples it took. + fn scripted(step: Duration, mut generate: impl FnMut(usize) -> Option>) -> (RailWatch, usize) { + let clock = Cell::new(Duration::ZERO); + let mut taken = 0usize; + let w = watch( + || { + assert!(taken < 100_000, "the probe never stopped sampling"); + let s = generate(taken); + taken += 1; + s.map(|v| v.into_iter().map(|(n, raw)| (n.to_string(), raw)).collect()) + }, + || clock.set(clock.get() + step), + || clock.get(), + BUDGET, + ); + (w, taken) + } + + fn key(name: &str, ordinal: u32) -> RailKey { + (name.to_string(), ordinal) + } + + #[test] + fn only_the_cpu_energy_aggregate_is_a_cpu_rail() { + for name in ["CPU Energy", "cpu energy", "DIE_0_CPU Energy", "DIE_1_CPU Energy", "DIE_12_CPU Energy"] { + assert!(is_cpu_energy_rail(name), "{name}"); + } + for name in ["GPU Energy", "DIE_0_GPU Energy", "ANE Energy", "PCPU", "MCPU0", "PACC_0", "DIE_0_PACC0_CPU0", "ECPUDTL00", "CPU Energy X", "DIE__CPU"] { + assert!(!is_cpu_energy_rail(name), "{name}"); + } + } + + /// (a) The M5 Max refresh: `CPU Energy` held at 480,083,160, stepped + /// once to 541,893,638 and held, while `GPU Energy` moved at every + /// sample. One advance in 40 intervals: Frozen. A rule that compared + /// every sample with the first would see an advance in every interval + /// after the step and call it live. + #[test] + fn a_the_m5_max_refresh_is_frozen() { + let (w, taken) = scripted(TICK, |i| { + Some(vec![ + ("CPU Energy", if i < 2 { 480_083_160 } else { 541_893_638 }), + ("GPU Energy", 1_000 + 7 * i as i64), + ]) + }); + assert_eq!(w.verdict(), Some(CpuRail::Frozen), "one refresh counted as a live rail: {w:?}"); + assert_eq!(w.rails, vec![(key("CPU Energy", 0), 1)], "the refresh is one advance of one rail"); + // The budget is 40 ticks; with two intervals long since seen, it stops there. + assert_eq!((w.intervals, taken), (40, 41)); + } + + /// (b) The M3 Ultra refresh: both dies step once, together, in one + /// sample, amid unprefixed per-die channels that repeat a name. + #[test] + fn b_the_m3_ultra_two_die_refresh_in_one_sample_is_frozen() { + let (w, _) = scripted(TICK, |i| { + let stepped = i >= 5; + Some(vec![ + ("DIE_0_CPU Energy", 9_494_336 + if stepped { 1_344_869 } else { 0 }), + ("ECPUDTL00", 10 * i as i64), + ("DIE_1_CPU Energy", 10_212_656 + if stepped { 1_391_637 } else { 0 }), + ("ECPUDTL00", 11 * i as i64), + ("GPU Energy", 3 * i as i64), + ]) + }); + assert_eq!(w.verdict(), Some(CpuRail::Frozen), "one refresh of both dies counted as live: {w:?}"); + assert_eq!(w.rails, vec![(key("DIE_0_CPU Energy", 0), 1), (key("DIE_1_CPU Energy", 0), 1)]); + } + + /// (c) Two channels with the same name are two rails: stepping together + /// once is one advance each — in the same interval or in different ones — + /// never two advances of one rail. And either of them can prove live. + #[test] + fn c_two_same_named_cpu_rails_are_counted_apart() { + let (w, _) = scripted(TICK, |i| { + let j = if i >= 3 { 1_300_000 } else { 0 }; + Some(vec![("CPU Energy", 100 + j), ("CPU Energy", 200 + j)]) + }); + assert_eq!(w.verdict(), Some(CpuRail::Frozen), "one refresh of two same-named rails counted as live: {w:?}"); + assert_eq!(w.rails, vec![(key("CPU Energy", 0), 1), (key("CPU Energy", 1), 1)]); + + let (w, _) = scripted(TICK, |i| { + Some(vec![("CPU Energy", 100 + if i >= 3 { 5 } else { 0 }), ("CPU Energy", 200 + if i >= 7 { 5 } else { 0 })]) + }); + assert_eq!(w.verdict(), Some(CpuRail::Frozen), "two same-named rails stepping once each counted as live: {w:?}"); + + // The first of two same-named rails is live; the second holds. + let (w, taken) = scripted(TICK, |i| Some(vec![("CPU Energy", 100 + i as i64), ("CPU Energy", 200)])); + assert_eq!(w.verdict(), Some(CpuRail::Live), "{w:?}"); + assert_eq!((w.intervals, taken), (2, 3)); + } + + /// (d) A rail stepping at every sample is live after exactly two + /// intervals, and sampling stops right there. + #[test] + fn d_a_rail_stepping_every_sample_is_live_after_two_intervals() { + let (w, taken) = scripted(TICK, |i| Some(vec![("GPU Energy", 5), ("CPU Energy", 1_000 + 13 * i as i64)])); + assert_eq!(w.verdict(), Some(CpuRail::Live)); + assert_eq!((w.intervals, taken), (2, 3), "a live rail must stop the probe at its second advance"); + assert_eq!(w.rails, vec![(key("CPU Energy", 0), 2)]); + // Two advances need not be in consecutive intervals. + let (w, taken) = scripted(TICK, |i| Some(vec![("CPU Energy", [0, 0, 0, 4, 4, 4, 4, 4, 4, 9][i.min(9)])])); + assert_eq!(w.verdict(), Some(CpuRail::Live)); + assert_eq!((w.intervals, taken), (9, 10)); + } + + /// (e) Samples that cannot be taken keep the probe sampling to its cap of + /// five budgets, and end with no verdict: Unavailable. + #[test] + fn e_failed_samples_sample_to_the_cap_and_give_no_verdict() { + let (w, taken) = scripted(TICK, |_| None); + assert_eq!(w.verdict(), None); + assert_eq!(status_of(w.verdict()), crate::IoReportStatus::Unavailable); + assert_eq!(taken, 5 * 40 + 1, "the probe must keep sampling to five budgets"); + // One usable sample is still no interval. + let (w, taken) = scripted(TICK, |i| (i == 0).then(|| vec![("CPU Energy", 7)])); + assert_eq!((w.verdict(), w.intervals, taken), (None, 0, 201)); + } + + /// (f) Past the budget, the probe stops at once if it has seen two + /// intervals; with fewer it keeps sampling until it has two, or hits the cap. + #[test] + fn f_the_budget_stops_the_probe_only_once_two_intervals_are_seen() { + // Every tick jumps past the budget: it still takes three samples. + let (w, taken) = scripted(BUDGET + Duration::from_millis(1), |_| Some(vec![("CPU Energy", 42)])); + assert_eq!(w.verdict(), Some(CpuRail::Frozen)); + assert_eq!((w.intervals, taken), (2, 3)); + // One interval, then failures: past the budget it keeps sampling, to the cap. + let (w, taken) = scripted(TICK, |i| (i < 2).then(|| vec![("CPU Energy", 42)])); + assert_eq!((w.verdict(), w.intervals, taken), (Some(CpuRail::Frozen), 1, 201)); + // The second interval arrives at 2.5 s, past the budget: it stops right there. + let (w, taken) = scripted(TICK, |i| (i < 2 || i == 100).then(|| vec![("CPU Energy", 42)])); + assert_eq!((w.verdict(), w.intervals, taken), (Some(CpuRail::Frozen), 2, 101)); + // Within the budget, two intervals are not enough to stop. + let (w, taken) = scripted(Duration::from_millis(100), |_| Some(vec![("CPU Energy", 42)])); + assert_eq!((w.intervals, taken), (10, 11)); + } + + /// (g) DIE_0 steps once and DIE_1 steps once, in different intervals: + /// advances are counted per rail, never summed across dies. + #[test] + fn g_one_step_on_each_die_in_different_intervals_is_frozen() { + let (w, _) = scripted(TICK, |i| { + Some(vec![ + ("DIE_0_CPU Energy", if i >= 3 { 9_500_000 } else { 9_494_336 }), + ("DIE_1_CPU Energy", if i >= 6 { 10_300_000 } else { 10_212_656 }), + ]) + }); + assert_eq!(w.verdict(), Some(CpuRail::Frozen), "one step per die counted as a live rail: {w:?}"); + assert_eq!(w.rails, vec![(key("DIE_0_CPU Energy", 0), 1), (key("DIE_1_CPU Energy", 0), 1)]); + } + + /// No CPU rail in any sample is Absent; a rail that goes down (a reset) + /// has not advanced. + #[test] + fn no_cpu_rail_is_absent_and_a_reset_is_not_an_advance() { + let (w, _) = scripted(TICK, |i| Some(vec![("GPU Energy", i as i64), ("PCPU", i as i64)])); + assert_eq!(w.verdict(), Some(CpuRail::Absent)); + let (w, _) = scripted(TICK, |i| Some(vec![("CPU Energy", 1_000 - i as i64)])); + assert_eq!(w.verdict(), Some(CpuRail::Frozen)); + assert_eq!(w.rails, vec![(key("CPU Energy", 0), 0)]); + } + + /// Every verdict maps to its status; only a live rail is `Live`. + #[test] + fn the_verdict_maps_to_the_status() { + use crate::IoReportStatus as S; + assert_eq!(status_of(None), S::Unavailable); + assert_eq!(status_of(Some(CpuRail::Absent)), S::CpuRailAbsent); + assert_eq!(status_of(Some(CpuRail::Frozen)), S::CpuRailFrozen); + assert_eq!(status_of(Some(CpuRail::Live)), S::Live); + } + + /// Only a live CPU rail admits the counter. + #[test] + fn only_a_live_rail_admits_the_counter() { + use crate::IoReportStatus as S; + assert!(admits_counter(S::Live)); + for declined in [S::Unavailable, S::CpuRailAbsent, S::CpuRailFrozen] { + assert!(!admits_counter(declined), "{declined:?} admitted the counter"); + } + } + } +} + // Apple Silicon IOReport energy counter — a no-sudo, no-external-binary read of // the SoC Energy Model channels through Apple's private IOReport C API. Ported // from jouleclaw-energy's `apple_ioreport` backend, adapted to eoc's @@ -76,15 +469,17 @@ impl JouleCounter for StubCounter { // zero, or a refresh of past CPU energy, and consumers such as eoc-cascade, // eoc-local and eoc-streaming tag what a hardware counter returns as Measured. // So the counter is constructed only when `ioreport_status()` is `Live` — the -// CPU rail itself kept stepping while the probing thread burned CPU — and -// otherwise `try_detect()` returns `Err(NoCounter)`, leaving the caller's own -// estimate. +// same CPU rail advanced in two separate sample intervals while the probing +// thread burned CPU — and otherwise `try_detect()` returns `Err(NoCounter)`, +// leaving the caller's own estimate. #[cfg(target_os = "macos")] #[allow(non_camel_case_types, unsafe_code)] mod macos_ioreport { use super::*; + use crate::rail_liveness::{self, RailWatch}; use std::os::raw::{c_int, c_long, c_void}; use std::sync::Mutex; + use std::sync::atomic::{AtomicUsize, Ordering}; use std::time::{Duration, Instant}; type CFTypeRef = *const c_void; @@ -182,6 +577,32 @@ mod macos_ioreport { unsafe impl Send for MacosIoReportCounter {} unsafe impl Sync for MacosIoReportCounter {} + /// Subscribe to `desired` (consumed). `None`, with everything released, + /// if IOReport returns no subscription or no subscribed-channel dictionary. + unsafe fn subscribe(desired: CFMutableDictionaryRef) -> Option { + unsafe { + let mut subscribed: CFMutableDictionaryRef = std::ptr::null_mut(); + let sub = IOReportCreateSubscription( + std::ptr::null(), + desired, + &mut subscribed as *mut _, + 0, + std::ptr::null(), + ); + CFRelease(desired as CFTypeRef); + if sub.is_null() || subscribed.is_null() { + if !sub.is_null() { + CFRelease(sub); + } + if !subscribed.is_null() { + CFRelease(subscribed as CFTypeRef); + } + return None; + } + Some(State { sub, channels: subscribed, prior_sample: None, cumulative_uj: 0 }) + } + } + impl MacosIoReportCounter { /// Open an IOReport subscription on a host whose CPU energy rail is /// live. `Err(NoCounter)` if IOReport can't open (non-Apple-Silicon, @@ -192,7 +613,7 @@ mod macos_ioreport { /// says which. pub fn try_detect() -> Result { // Opening a subscription is not enough: the CPU rail must be live. - if crate::ioreport_status() != crate::IoReportStatus::Live { + if !rail_liveness::admits_counter(crate::ioreport_status()) { return Err(Error::NoCounter); } match Self::open() { @@ -201,39 +622,25 @@ mod macos_ioreport { } } + /// The counter's own subscription: every channel. fn open() -> Option { unsafe { let all = IOReportCopyAllChannels(0, 0); if all.is_null() { return None; } - let mut subscribed: CFMutableDictionaryRef = std::ptr::null_mut(); - let sub = IOReportCreateSubscription( - std::ptr::null(), - all, - &mut subscribed as *mut _, - 0, - std::ptr::null(), - ); - if sub.is_null() { - CFRelease(all as CFTypeRef); - return None; - } - CFRelease(all as CFTypeRef); - if subscribed.is_null() { - CFRelease(sub); - return None; - } - Some(State { sub, channels: subscribed, prior_sample: None, cumulative_uj: 0 }) + subscribe(all) } } /// A subscription to the Energy Model group alone, for the liveness /// probe. The CPU rails are the same counters the all-channel - /// subscription above reads, but sampling only this group took ~6 ms on - /// an M5 Max and an M3 Ultra, against ~780 ms and ~145 ms for every - /// channel — slow enough that a probe sampling all channels would spend - /// nearly its whole budget inside IOReport instead of burning CPU. + /// subscription above reads, with the same absolute values, but one + /// sample of this group takes ~6-12 ms on an M5 Max and an M3 Ultra, + /// against 0.13-0.8 s and ~0.14 s for every channel — slow enough that a + /// probe sampling all channels would spend nearly its whole budget + /// inside IOReport instead of burning CPU. On an M5 Max a probe on + /// every channel saw 2-3 intervals in ~2 s; on this group, 25-31 in one. fn open_energy_model() -> Option { unsafe { let group = CFStringCreateWithCString( @@ -249,162 +656,32 @@ mod macos_ioreport { if desired.is_null() { return None; } - let mut subscribed: CFMutableDictionaryRef = std::ptr::null_mut(); - let sub = IOReportCreateSubscription( - std::ptr::null(), - desired, - &mut subscribed as *mut _, - 0, - std::ptr::null(), - ); - CFRelease(desired as CFTypeRef); - if sub.is_null() { - return None; - } - if subscribed.is_null() { - CFRelease(sub); - return None; - } - Some(State { sub, channels: subscribed, prior_sample: None, cumulative_uj: 0 }) + subscribe(desired) } } } - /// Whether this host's CPU energy rail is live, frozen, or not present. - #[derive(Debug, Clone, Copy, PartialEq, Eq)] - enum CpuRail { - /// A CPU energy channel stepped at [`LIVE_AFTER_STEPS`] separate samples - /// while this thread burned CPU. - Live, - /// The rail exists but held its value across the whole burn, or stepped - /// only once — a refresh, not a rail that tracks the work. - Frozen, - /// No `CPU Energy` channel exists in the Energy Model group (or no - /// sample could be taken to look for one). - Absent, - } - - /// How many separate samples one CPU energy channel must step at before the - /// rail counts as live. A live rail, sampled every ~30 ms while this thread - /// burns CPU, steps many times a second. On macOS 27.0.1 the CPU rails are - /// instead refreshed now and then: both of an M3 Ultra's held for at least - /// 22 minutes, then stepped by ~1.3 kJ each within one sample. One step inside - /// the probe's window would otherwise select the counter for the life of the - /// process, and every later refresh would land, whole, in some caller's - /// interval. - const LIVE_AFTER_STEPS: u32 = 2; - - /// Per CPU energy channel, the number of samples at which its raw cumulative - /// delta (since the probe's first sample) rose. - #[derive(Debug, Default)] - struct RailSteps { - prev: Vec, - steps: Vec, - } - - impl RailSteps { - /// Fold in the next sample's per-channel deltas, in channel order. - fn observe(&mut self, deltas: &[i64]) { - if deltas.len() != self.steps.len() { - // First sample, or the channel set changed: count from here. - self.prev = vec![0; deltas.len()]; - self.steps = vec![0; deltas.len()]; - } - for (i, &d) in deltas.iter().enumerate() { - if d > self.prev[i] { - self.steps[i] += 1; - } - self.prev[i] = d; - } - } - - fn live(&self) -> bool { - self.steps.iter().any(|&n| n >= LIVE_AFTER_STEPS) - } - } - impl State { - /// Burn CPU on this thread for up to `budget`, re-sampling after every - /// ~25 ms of spinning, and report whether the CPU energy rail **itself** - /// keeps advancing: one channel must step at [`LIVE_AFTER_STEPS`] separate - /// samples. On a subscription to the Energy Model group alone (see - /// `open_energy_model`) a sample costs a few milliseconds, so the thread - /// spends most of the budget spinning; on an all-channel subscription it - /// would spend most of it waiting on IOReport. - /// - /// The summed total is not a liveness signal. Measured on an M5 Max and an - /// M3 Ultra, both on macOS 27.0.1: every CPU rail in the Energy Model group - /// (`CPU Energy`, `PCPU`, `MCPU*`, `PACC_*`, `DIE_n_CPU Energy`) held the - /// same cumulative value under full CPU load, while `GPU Energy` still - /// moved. A total that moves therefore proves only that the GPU did - /// something. Liveness is decided on the CPU rail alone. Takes its own - /// samples, so no counter's cumulative reading is touched. - fn probe_cpu_rail(&self, budget: Duration) -> CpuRail { + /// One sample of this subscription, as every Energy Model channel's + /// (name, absolute raw integer), in sample order. `None` when IOReport + /// returns no sample, or a sample with no channel list. This only + /// extracts; the liveness rule lives in [`rail_liveness`]. + fn energy_model_sample(&self) -> Option> { unsafe { - let s0 = IOReportCreateSamples(self.sub, self.channels, std::ptr::null()); - if s0.is_null() { - return CpuRail::Absent; - } - let started = Instant::now(); - let mut acc: u64 = 0; - let mut saw_rail = false; - let mut steps = RailSteps::default(); - while !steps.live() && started.elapsed() < budget { - let tick = Instant::now(); - while tick.elapsed() < Duration::from_millis(25) { - acc = acc.wrapping_add(1); - } - std::hint::black_box(acc); - let s1 = IOReportCreateSamples(self.sub, self.channels, std::ptr::null()); - if s1.is_null() { - continue; - } - let delta = IOReportCreateSamplesDelta(s0, s1, std::ptr::null()); - CFRelease(s1); - if delta.is_null() { - continue; - } - let deltas = cpu_rail_deltas(delta); - CFRelease(delta); - if !deltas.is_empty() { - saw_rail = true; - steps.observe(&deltas); - } - } - CFRelease(s0); - match (saw_rail, steps.live()) { - (_, true) => CpuRail::Live, - (true, false) => CpuRail::Frozen, - (false, false) => CpuRail::Absent, + let smp = IOReportCreateSamples(self.sub, self.channels, std::ptr::null()); + if smp.is_null() { + return None; } + let channels = energy_model_channels(smp); + CFRelease(smp); + channels } } } - /// Probe the CPU rail on an Energy Model subscription. Backs the cached - /// [`crate::ioreport_status`]; the probe's subscription is dropped after. - pub(crate) fn probe_status(budget: Duration) -> crate::IoReportStatus { - match MacosIoReportCounter::open_energy_model() { - Some(state) => match state.probe_cpu_rail(budget) { - CpuRail::Live => crate::IoReportStatus::Live, - CpuRail::Frozen => crate::IoReportStatus::CpuRailFrozen, - CpuRail::Absent => crate::IoReportStatus::CpuRailAbsent, - }, - // No Energy Model subscription. Tell "no IOReport" from "no such - // group" by opening the subscription the counter itself would use. - None => match MacosIoReportCounter::open() { - Some(_) => crate::IoReportStatus::CpuRailAbsent, - None => crate::IoReportStatus::Unavailable, - }, - } - } - - /// In a delta sample: the raw integer delta of each CPU energy channel - /// (`CPU Energy`, or `DIE__CPU Energy` on multi-die parts), in channel - /// order; empty when there is none. Raw integers, before unit conversion, so - /// a small advance in a fine unit cannot truncate to zero. - fn cpu_rail_deltas(delta: CFDictionaryRef) -> Vec { - let mut deltas = Vec::new(); + /// In a sample (absolute, not a delta): each Energy Model channel's name and + /// raw integer value. `None` when the sample carries no channel list. + fn energy_model_channels(sample: CFDictionaryRef) -> Option> { unsafe { let key = CFStringCreateWithCString( std::ptr::null_mut(), @@ -412,13 +689,14 @@ mod macos_ioreport { K_CFSTRING_ENCODING_UTF8, ); if key.is_null() { - return deltas; + return None; } - let arr = CFDictionaryGetValue(delta, key as *const _) as CFArrayRef; + let arr = CFDictionaryGetValue(sample, key as *const _) as CFArrayRef; CFRelease(key as CFTypeRef); if arr.is_null() { - return deltas; + return None; } + let mut out = Vec::new(); for i in 0..CFArrayGetCount(arr) { let item = CFArrayGetValueAtIndex(arr, i); if item.is_null() || !cfstring_eq_ic(IOReportChannelGetGroup(item), "energy model") { @@ -427,12 +705,54 @@ mod macos_ioreport { let Some(name) = cfstring_to_string(IOReportChannelGetChannelName(item)) else { continue; }; - if strip_die_prefix(&name).eq_ignore_ascii_case("CPU Energy") { - deltas.push(IOReportSimpleGetIntegerValue(item, 0) as i64); - } + out.push((name, IOReportSimpleGetIntegerValue(item, 0) as i64)); } + Some(out) + } + } + + /// CPU burned on the probing thread before each liveness re-sample. + const PROBE_TICK: Duration = Duration::from_millis(25); + + /// Spin this thread for `d`. + fn burn(d: Duration) { + let started = Instant::now(); + let mut acc: u64 = 0; + while started.elapsed() < d { + acc = acc.wrapping_add(1); + } + std::hint::black_box(acc); + } + + /// How many times [`probe_status`] — the probe behind the cached + /// [`crate::ioreport_status`] — has run in this process. Read by a test + /// that pins the cache: it must be exactly one. + pub(crate) static PROBE_RUNS: AtomicUsize = AtomicUsize::new(0); + + /// One liveness probe on a fresh Energy Model subscription: sample, burn + /// [`PROBE_TICK`] of CPU on this thread, sample again, by the rule in + /// [`rail_liveness::watch`]. `None` when the subscription does not open. The + /// probe takes its own samples, so no counter's cumulative reading is + /// touched. + pub(crate) fn probe(budget: Duration) -> Option { + let state = MacosIoReportCounter::open_energy_model()?; + let t0 = Instant::now(); + Some(rail_liveness::watch(|| state.energy_model_sample(), || burn(PROBE_TICK), || t0.elapsed(), budget)) + } + + /// This host's status from one probe. Backs the cached + /// [`crate::ioreport_status`]; the probe's subscription is dropped after. + pub(crate) fn probe_status(budget: Duration) -> crate::IoReportStatus { + PROBE_RUNS.fetch_add(1, Ordering::SeqCst); + match probe(budget) { + Some(watch) => rail_liveness::status_of(watch.verdict()), + // No Energy Model subscription. Tell "no IOReport" from "no such + // group" by opening the subscription the counter itself would use. + None => match MacosIoReportCounter::open() { + Some(_) => crate::IoReportStatus::CpuRailAbsent, + None => crate::IoReportStatus::Unavailable, + }, } - deltas } impl JouleCounter for MacosIoReportCounter { @@ -524,18 +844,8 @@ mod macos_ioreport { uj as u64 } - fn strip_die_prefix(name: &str) -> &str { - if let Some(rest) = name.strip_prefix("DIE_") { - let after = rest.trim_start_matches(|c: char| c.is_ascii_digit()); - if let Some(r) = after.strip_prefix('_') { - return r; - } - } - name - } - fn counts_once(name: &str) -> bool { - let base = strip_die_prefix(name); + let base = rail_liveness::strip_die_prefix(name); if base.eq_ignore_ascii_case("CPU Energy") || base.eq_ignore_ascii_case("GPU Energy") || base.eq_ignore_ascii_case("ANE Energy") @@ -574,6 +884,19 @@ mod macos_ioreport { #[cfg(test)] mod tests { use super::*; + use crate::rail_liveness::CpuRail; + use std::sync::atomic::AtomicBool; + + unsafe extern "C" { + fn clock_gettime_nsec_np(clock_id: u32) -> u64; + } + /// `CLOCK_THREAD_CPUTIME_ID` in macOS `<_time.h>`. + const CLOCK_THREAD_CPUTIME_ID: u32 = 16; + + /// CPU time this thread has consumed. + fn thread_cpu_time() -> Duration { + Duration::from_nanos(unsafe { clock_gettime_nsec_np(CLOCK_THREAD_CPUTIME_ID) }) + } #[test] fn energy_to_uj_scales_by_unit() { @@ -590,44 +913,9 @@ mod macos_ioreport { assert!(!counts_once("GPU0_0")); } - /// A rail that steps once and holds is a refresh, not a rail tracking - /// the work — even a large step, even on every die at once. The deltas are - /// the refresh observed on an M3 Ultra (`DIE_0_`/`DIE_1_CPU Energy`, mJ). - #[test] - fn a_cpu_rail_that_steps_once_is_not_live() { - let mut s = RailSteps::default(); - for d in [[0, 0], [0, 0], [1_344_869, 1_391_637], [1_344_869, 1_391_637]] { - s.observe(&d); - } - assert!(!s.live(), "one refresh counted as a live rail: {s:?}"); - // Two dies stepping once each, at different samples, is still one step each. - let mut s = RailSteps::default(); - for d in [[5, 0], [5, 7], [5, 7]] { - s.observe(&d); - } - assert!(!s.live(), "two single-step channels counted as a live rail: {s:?}"); - // No CPU rail at all. - let mut s = RailSteps::default(); - s.observe(&[]); - assert!(!s.live()); - } + // ── An independent oracle ──────────────────────────────────────────── - /// A rail that keeps stepping while the thread burns is live. - #[test] - fn a_cpu_rail_that_keeps_stepping_is_live() { - let mut s = RailSteps::default(); - for d in [[0], [3], [3], [9]] { - s.observe(&d); - } - assert!(s.live(), "a rail that stepped at two samples was not live: {s:?}"); - let mut s = RailSteps::default(); - for d in [[0, 4], [0, 11]] { - s.observe(&d); - } - assert!(s.live(), "one die stepping at every sample was not live: {s:?}"); - } - - /// What the independent oracle saw on this host. + /// What the oracle saw on this host. #[derive(Debug, PartialEq)] enum Oracle { NoIoReport, @@ -635,87 +923,189 @@ mod macos_ioreport { CpuRailMoved(bool), } - /// An oracle that shares nothing with the liveness probe but the raw FFI: - /// it samples the subscription the COUNTER uses (every channel, not the - /// probe's Energy Model group), reads each CPU rail's ABSOLUTE cumulative - /// value (no delta samples), matches names with `ends_with("CPU Energy")` - /// rather than the die-prefix stripping and group filter the - /// implementation uses, and burns CPU on this thread for the whole - /// `budget` between one read before and one after — an all-channel - /// sample can take most of a second, so it is not polled mid-burn. - fn oracle_cpu_rail(budget: Duration) -> Oracle { - let Some(st) = MacosIoReportCounter::open() else { - return Oracle::NoIoReport; + #[derive(Debug)] + struct OracleRun { + verdict: Oracle, + reads: u32, + cpu_changes: Vec, + gpu_changes: Vec, + } + + impl OracleRun { + fn bare(verdict: Oracle) -> Self { + Self { verdict, reads: 0, cpu_changes: Vec::new(), gpu_changes: Vec::new() } + } + } + + /// The oracle's own CFString conversion, so a broken implementation + /// helper cannot make the oracle agree with it. + fn oracle_name(s: CFStringRef) -> String { + if s.is_null() { + return String::new(); + } + let mut buf = [0u8; 512]; + let ok = unsafe { + CFStringGetCString(s, buf.as_mut_ptr() as *mut i8, buf.len() as c_long, K_CFSTRING_ENCODING_UTF8) }; - let read = || -> Option> { - unsafe { - let smp = IOReportCreateSamples(st.sub, st.channels, std::ptr::null()); - if smp.is_null() { - return None; - } - let key = CFStringCreateWithCString( - std::ptr::null_mut(), - c"IOReportChannels".as_ptr() as *const _, - K_CFSTRING_ENCODING_UTF8, - ); - let mut vals = Vec::new(); - if !key.is_null() { - let arr = CFDictionaryGetValue(smp, key as *const _) as CFArrayRef; - CFRelease(key as CFTypeRef); - if !arr.is_null() { - for i in 0..CFArrayGetCount(arr) { - let it = CFArrayGetValueAtIndex(arr, i); - if it.is_null() { - continue; - } - let name = cfstring_to_string(IOReportChannelGetChannelName(it)) - .unwrap_or_default(); - if name.ends_with("CPU Energy") { - vals.push(IOReportSimpleGetIntegerValue(it, 0) as i64); - } + if !ok { + return String::new(); + } + let end = buf.iter().position(|&b| b == 0).unwrap_or(buf.len()); + String::from_utf8_lossy(&buf[..end]).into_owned() + } + + /// One read of the oracle's subscription: the ABSOLUTE raw value of + /// every channel, in any group, whose name ends with `CPU Energy`, and + /// of every one ending with `GPU Energy`, in sample order. + fn oracle_read(sub: IOReportSubscriptionRef, chans: CFMutableDictionaryRef) -> Option<(Vec, Vec)> { + unsafe { + let smp = IOReportCreateSamples(sub, chans, std::ptr::null()); + if smp.is_null() { + return None; + } + let key = CFStringCreateWithCString( + std::ptr::null_mut(), + c"IOReportChannels".as_ptr() as *const _, + K_CFSTRING_ENCODING_UTF8, + ); + let (mut cpu, mut gpu) = (Vec::new(), Vec::new()); + if !key.is_null() { + let arr = CFDictionaryGetValue(smp, key as *const _) as CFArrayRef; + CFRelease(key as CFTypeRef); + if !arr.is_null() { + for i in 0..CFArrayGetCount(arr) { + let it = CFArrayGetValueAtIndex(arr, i); + if it.is_null() { + continue; + } + let name = oracle_name(IOReportChannelGetChannelName(it)); + if name.ends_with("CPU Energy") { + cpu.push(IOReportSimpleGetIntegerValue(it, 0) as i64); + } else if name.ends_with("GPU Energy") { + gpu.push(IOReportSimpleGetIntegerValue(it, 0) as i64); } } } - CFRelease(smp); - Some(vals) } - }; - let Some(v0) = read() else { - return Oracle::NoIoReport; - }; - if v0.is_empty() { - return Oracle::NoCpuRail; + CFRelease(smp); + Some((cpu, gpu)) } - let started = Instant::now(); - let mut acc: u64 = 0; - while started.elapsed() < budget { - acc = acc.wrapping_add(1); - } - std::hint::black_box(acc); - let v1 = read().expect("oracle: IOReportCreateSamples failed after the burn"); - Oracle::CpuRailMoved(v1 != v0) } + /// An oracle that shares nothing with the liveness probe but the raw FFI + /// declarations. It opens its own subscription over every channel (not + /// the probe's Energy Model group) with its own calls, converts names + /// with its own helper, reads ABSOLUTE values (no delta samples), + /// matches names with `ends_with("CPU Energy")` in any group (not the + /// implementation's group filter and die-prefix stripping), and keeps a + /// HELPER thread burning CPU for the whole `window`, so the window is + /// under load even while an all-channel sample blocks this thread. + /// + /// It reads over and over for the whole window, and counts per rail at + /// how many reads the value changed since the read before. + /// `CpuRailMoved(true)` only if one CPU rail changed at two or more + /// reads: a live rail changes at nearly every read, a batch-refreshed one + /// at most once in a window this short, and one change is not a live + /// reading. Its own literal threshold, not the implementation's constant. + fn oracle_cpu_rail(window: Duration) -> OracleRun { + let (sub, chans) = unsafe { + let all = IOReportCopyAllChannels(0, 0); + if all.is_null() { + return OracleRun::bare(Oracle::NoIoReport); + } + let mut chans: CFMutableDictionaryRef = std::ptr::null_mut(); + let sub = IOReportCreateSubscription(std::ptr::null(), all, &mut chans, 0, std::ptr::null()); + CFRelease(all as CFTypeRef); + if sub.is_null() || chans.is_null() { + if !sub.is_null() { + CFRelease(sub); + } + if !chans.is_null() { + CFRelease(chans as CFTypeRef); + } + return OracleRun::bare(Oracle::NoIoReport); + } + (sub, chans) + }; + let stop = AtomicBool::new(false); + let run = std::thread::scope(|s| { + s.spawn(|| { + let mut acc: u64 = 0; + while !stop.load(Ordering::Relaxed) { + acc = acc.wrapping_add(1); + } + std::hint::black_box(acc); + }); + let started = Instant::now(); + let run = 'watch: { + let Some((mut cpu_prev, mut gpu_prev)) = oracle_read(sub, chans) else { + break 'watch OracleRun::bare(Oracle::NoIoReport); + }; + if cpu_prev.is_empty() { + break 'watch OracleRun::bare(Oracle::NoCpuRail); + } + let mut run = OracleRun { + verdict: Oracle::CpuRailMoved(false), + reads: 1, + cpu_changes: vec![0; cpu_prev.len()], + gpu_changes: vec![0; gpu_prev.len()], + }; + while started.elapsed() < window { + let Some((cpu, gpu)) = oracle_read(sub, chans) else { + continue; + }; + run.reads += 1; + for (n, (now, before)) in run.cpu_changes.iter_mut().zip(cpu.iter().zip(&cpu_prev)) { + *n += u32::from(now != before); + } + for (n, (now, before)) in run.gpu_changes.iter_mut().zip(gpu.iter().zip(&gpu_prev)) { + *n += u32::from(now != before); + } + (cpu_prev, gpu_prev) = (cpu, gpu); + } + run.verdict = Oracle::CpuRailMoved(run.cpu_changes.iter().any(|&n| n >= 2)); + run + }; + stop.store(true, Ordering::Relaxed); + run + }); + unsafe { + CFRelease(sub); + CFRelease(chans as CFTypeRef); + } + run + } + + // ── Against the hardware ───────────────────────────────────────────── + /// The property that matters: the IOReport counter is **never** /// constructed, and never picked by [`crate::detect`], when its CPU rail - /// is frozen — then it would report GPU-only energy, or zero, as the - /// energy of CPU work, and consumers would tag it Measured. Asserts the - /// meter's decision against what the rail actually did; every branch - /// asserts, so there is no way to pass without checking anything. + /// is not live — then it would report GPU-only energy, zero, or a batch + /// refresh as the energy of CPU work, and consumers would tag it + /// Measured. Asserts the meter's decision against what the rail actually + /// did; every branch asserts, so there is no way to pass without + /// checking anything. /// /// The oracle gets three times the probe's budget, so a rail too slow for /// the probe but moving within the oracle's window is the only case where /// declining is accepted — and that fails safe: estimated, never - /// over-claimed. + /// over-claimed. A single batch refresh in both windows does not rescue a + /// probe that called it live: the oracle counts it as one change. #[test] fn the_ioreport_counter_is_never_selected_on_a_frozen_cpu_rail() { let status = crate::ioreport_status(); let constructible = MacosIoReportCounter::try_detect().is_ok(); let selected = crate::detect().name() == "ioreport"; - let oracle = oracle_cpu_rail(crate::CPU_RAIL_LIVENESS_BUDGET * 3); - // Recorded so a CI log says which branch below was actually asserted. - eprintln!("ioreport liveness: status={status:?} oracle={oracle:?} selected={selected}"); - match oracle { + let window = crate::CPU_RAIL_LIVENESS_BUDGET * 3; + let run = oracle_cpu_rail(window); + // One line, so a CI log says which branch below was actually asserted. + eprintln!( + "ioreport liveness: status={status:?} oracle={:?} selected={selected} \ + (oracle: {} reads over {window:?} with a helper thread burning CPU; \ + changes per CPU rail {:?}, per GPU rail {:?})", + run.verdict, run.reads, run.cpu_changes, run.gpu_changes + ); + match run.verdict { Oracle::NoIoReport => { assert!(!selected, "IOReport selected (status={status:?}) but no subscription opens"); assert_eq!(status, IoReportStatus::Unavailable); @@ -726,22 +1116,31 @@ mod macos_ioreport { } Oracle::CpuRailMoved(false) => { assert!( + run.reads >= 3, + "the oracle read only {} times in {window:?}: too few to see a rail change twice", + run.reads + ); + assert!( !selected, - "IOReport selected (status={status:?}) although its CPU rail did not move \ - under {:?} of CPU burn — it would report GPU-only energy, or zero, as the \ - energy of CPU work", - crate::CPU_RAIL_LIVENESS_BUDGET * 3 + "IOReport selected (status={status:?}) although its CPU rail did not keep moving \ + under {window:?} of CPU burn — it would report GPU-only energy, zero, or a batch \ + refresh as the energy of CPU work" + ); + assert_eq!( + status, + IoReportStatus::CpuRailFrozen, + "the oracle did not see the CPU rail keep moving under {window:?} of CPU burn" ); - assert_eq!(status, IoReportStatus::CpuRailFrozen); } Oracle::CpuRailMoved(true) => { - // Live rail: the probe had a third of the oracle's time. Live must - // be selected; a probe that missed a slow rail declines (safe side). + // A live rail. The probe had a third of the oracle's time: if it + // saw two advances the counter must be selected; if not, + // declining is the safe side. assert!( matches!(status, IoReportStatus::Live | IoReportStatus::CpuRailFrozen), - "the oracle saw a CPU rail move but the status is {status:?}" + "the oracle saw a CPU rail keep moving but the status is {status:?}" ); - assert_eq!(selected, status == IoReportStatus::Live); + assert_eq!(selected, status == IoReportStatus::Live, "status={status:?}"); } } // On macOS IOReport is the first rung, so detect() picks it exactly when @@ -750,44 +1149,105 @@ mod macos_ioreport { assert_eq!(constructible, status == IoReportStatus::Live); } - /// The dangerous direction on its own, against a fresh (uncached) probe on - /// the same subscription the status uses: it never reports Live for a - /// rail the oracle saw hold, and on every host it agrees with the oracle - /// about what exists. + /// The dangerous direction on its own, against a fresh (uncached) probe + /// on its own Energy Model subscription beside the cached status: it + /// never reports Live for a rail the oracle did not see keep moving, and + /// on every host both agree with the oracle about what exists. + /// + /// Also the probe's cadence: a probe that does not end Live samples for + /// at least its whole budget, and on the Energy Model group, re-sampled + /// after every ~25 ms of burn, that is ~30 intervals a second. At least + /// ten are required; a probe on the all-channel subscription saw 2-3 in + /// ~2 s on an M5 Max. + /// And it burns CPU on this thread between samples, rather than waiting: + /// at least half the probe's wall time is this thread's CPU time. #[test] fn the_liveness_probe_never_reports_a_frozen_rail_as_live() { - let probe = MacosIoReportCounter::open_energy_model() - .map(|st| st.probe_cpu_rail(crate::CPU_RAIL_LIVENESS_BUDGET)); - let oracle = oracle_cpu_rail(crate::CPU_RAIL_LIVENESS_BUDGET * 3); - eprintln!("ioreport probe: probe={probe:?} oracle={oracle:?}"); - if probe == Some(CpuRail::Live) { - assert_eq!( - oracle, - Oracle::CpuRailMoved(true), - "probe said Live, oracle saw the CPU rail hold its value" + let t = Instant::now(); + let cpu = thread_cpu_time(); + let watch = probe(crate::CPU_RAIL_LIVENESS_BUDGET); + let spun = thread_cpu_time().saturating_sub(cpu); + let took = t.elapsed(); + let spun_pct = 100.0 * spun.as_secs_f64() / took.as_secs_f64(); + let probed = watch.as_ref().map(RailWatch::verdict); + let cached = crate::ioreport_status(); + let run = oracle_cpu_rail(crate::CPU_RAIL_LIVENESS_BUDGET * 3); + let intervals = watch.as_ref().map_or(0, |w| w.intervals); + eprintln!( + "ioreport probe: probed={probed:?} cached={cached:?} took={took:?} on-cpu={spun_pct:.0}% \ + intervals={intervals} advances per CPU rail {:?}; oracle={:?} ({} reads, changes per CPU \ + rail {:?}, per GPU rail {:?})", + watch.as_ref().map(|w| w.rails.clone()).unwrap_or_default(), + run.verdict, + run.reads, + run.cpu_changes, + run.gpu_changes + ); + if let Some(verdict) = probed + && verdict != Some(CpuRail::Live) + { + assert!( + intervals >= 10, + "the probe observed only {intervals} sample intervals in {took:?} (verdict {verdict:?}): \ + it must sample the Energy Model group after every ~25 ms of burn" + ); + assert!( + spun_pct >= 50.0, + "the probe's thread was on CPU for only {spun_pct:.0}% of {took:?}: it must burn CPU \ + between samples, not wait" ); } - match oracle { - Oracle::NoIoReport => assert_eq!(probe, None), + if probed == Some(Some(CpuRail::Live)) || cached == IoReportStatus::Live { + assert_eq!( + run.verdict, + Oracle::CpuRailMoved(true), + "the probe said Live (probed={probed:?}, cached={cached:?}), the oracle did not see \ + the CPU rail keep moving" + ); + } + match run.verdict { + Oracle::NoIoReport => { + assert!(matches!(probed, None | Some(None)), "the oracle could not open IOReport, the probe did"); + assert_eq!(cached, IoReportStatus::Unavailable); + } Oracle::NoCpuRail => assert!( - matches!(probe, None | Some(CpuRail::Absent)), - "the oracle found no CPU rail but the probe reported {probe:?}" + matches!(probed, None | Some(Some(CpuRail::Absent))) && cached == IoReportStatus::CpuRailAbsent, + "the oracle found no CPU rail, the probe reported {probed:?} (cached={cached:?})" ), - Oracle::CpuRailMoved(false) => assert_eq!( - probe, - Some(CpuRail::Frozen), - "the CPU rail held under the oracle's burn, so the probe must report Frozen" + Oracle::CpuRailMoved(false) => assert!( + run.reads >= 3 && probed == Some(Some(CpuRail::Frozen)) && cached == IoReportStatus::CpuRailFrozen, + "the CPU rail did not keep moving under the oracle's burn ({} reads, three needed to see \ + two changes), so the probe must say Frozen: probed={probed:?}, cached={cached:?}", + run.reads ), Oracle::CpuRailMoved(true) => assert!( - matches!(probe, Some(CpuRail::Live | CpuRail::Frozen)), - "the oracle saw a CPU rail move but the probe reported {probe:?}" + matches!(probed, Some(Some(CpuRail::Live | CpuRail::Frozen))), + "the oracle saw a CPU rail keep moving but the probe reported {probed:?}" ), } } + /// The probe runs once per process: however often the status is asked + /// for and the counter is detected, the real probe behind the cache ran + /// exactly once. Counted, not timed: a re-probe that ends Live is fast. + #[test] + fn the_liveness_probe_runs_once_per_process() { + let first = crate::ioreport_status(); + for _ in 0..3 { + assert_eq!(crate::ioreport_status(), first, "the cached status changed between calls"); + } + let _ = crate::detect(); + let _ = MacosIoReportCounter::try_detect(); + let runs = PROBE_RUNS.load(Ordering::SeqCst); + assert_eq!(runs, 1, "the liveness probe behind ioreport_status() ran {runs} times in this process"); + } + /// When the counter IS available, a CPU burn meters as non-zero energy - /// through a rail that moves; when it is not, the status says why and is - /// never `Live`. + /// through a rail that moves. When it is not, the status is never `Live`, + /// and what [`crate::detect`] falls back to meters real work as no figure + /// at all — its reads fail, or it is the stub, whose zero every consumer + /// treats as no reading — so each caller keeps its own estimate: never a + /// hardware-tagged number, and never a zero passed off as a reading. #[test] fn a_selected_ioreport_counter_meters_cpu_work() { match MacosIoReportCounter::try_detect() { @@ -797,17 +1257,11 @@ mod macos_ioreport { IoReportStatus::Live, "try_detect() constructed the counter although the status is not Live" ); - let rail = MacosIoReportCounter::open_energy_model() - .map(|st| st.probe_cpu_rail(crate::CPU_RAIL_LIVENESS_BUDGET)); + let rail = probe(crate::CPU_RAIL_LIVENESS_BUDGET).and_then(|w| w.verdict()); assert_eq!(rail, Some(CpuRail::Live), "counter available but its CPU rail is {rail:?}"); let _ = c.read_microjoules(); // first read only sets the baseline sample let before = c.read_microjoules().unwrap(); - let t = Instant::now(); - let mut acc: u64 = 0; - while t.elapsed() < Duration::from_millis(200) { - acc = acc.wrapping_add(1); - } - std::hint::black_box(acc); + burn(Duration::from_millis(200)); let after = c.read_microjoules().unwrap(); assert!( after > before, @@ -815,12 +1269,29 @@ mod macos_ioreport { ); } Err(e) => { + let status = crate::ioreport_status(); assert!(matches!(e, Error::NoCounter), "declined with {e:?}, expected NoCounter"); - assert_ne!( - crate::ioreport_status(), - IoReportStatus::Live, - "status Live but try_detect() declined" + assert_ne!(status, IoReportStatus::Live, "status Live but try_detect() declined"); + let fallback = crate::detect(); + let before = fallback.read_microjoules(); + burn(Duration::from_millis(100)); + let after = fallback.read_microjoules(); + eprintln!( + "ioreport declined ({status:?}): fallback={} read {before:?} then {after:?} across \ + 100 ms of CPU work", + fallback.name() ); + assert_ne!(fallback.name(), "ioreport", "detect() fell back to the declined counter"); + if fallback.name() == "stub" { + assert!(matches!((&before, &after), (Ok(0), Ok(0))), "the stub read {before:?}, {after:?}"); + } else { + assert!( + before.is_err() && after.is_err(), + "the fallback counter {} returned a figure ({before:?}, {after:?}) though no \ + hardware counter measured the work", + fallback.name() + ); + } } } } @@ -834,26 +1305,31 @@ pub use macos_ioreport::MacosIoReportCounter; #[derive(Debug, Clone, Copy, PartialEq, Eq)] pub enum IoReportStatus { /// No IOReport subscription could be opened (not macOS, not Apple - /// Silicon, or Rosetta). + /// Silicon, or Rosetta), or one opened but never yielded two usable + /// samples, so nothing could be learned about the CPU rail. Unavailable, /// IOReport opened, but no `CPU Energy` rail was found in its Energy Model /// group (or that group could not be subscribed) — an unfamiliar channel - /// naming. Never used. + /// naming. The counter is not used. CpuRailAbsent, /// The CPU energy rail exists but did not keep advancing while CPU was - /// burned: it held, or stepped only once (a refresh). The platform is not - /// exposing live CPU energy to this process, so using the counter would - /// report GPU-only energy, zero, or a stale refresh as the energy of CPU - /// work. Observed on macOS 27.0.1 on both an M5 Max and an M3 Ultra. + /// burned: it held, or stepped in only one sample interval (a refresh). + /// The platform is not exposing live CPU energy to this process, so using + /// the counter would report GPU-only energy, zero, or a stale refresh as + /// the energy of CPU work. Observed on macOS 27.0.1 on both an M5 Max and + /// an M3 Ultra. CpuRailFrozen, - /// The CPU energy rail keeps advancing under load; the counter is available. + /// The same CPU energy rail advanced in two separate sample intervals under + /// load; the counter is available. Live, } /// How long the liveness probe burns CPU before concluding the CPU rail is -/// frozen: a live rail gets a second of single-thread CPU work to count, -/// sampled after every ~25 ms of it, and must step at two separate samples. -/// A rail too slow for that fails safe — declined, so callers estimate; never +/// not live: a live rail gets a second of single-thread CPU work, sampled +/// after every ~25 ms of it, to advance in two separate sample intervals. If +/// sampling is so slow that fewer than two intervals fit in the budget, the +/// probe keeps going until two have, and never past five budgets. A rail too +/// slow for that fails safe — declined, so callers estimate; never /// over-claimed. Paid once per process; see [`ioreport_status`]. #[cfg(target_os = "macos")] pub const CPU_RAIL_LIVENESS_BUDGET: std::time::Duration = std::time::Duration::from_millis(1000); @@ -862,9 +1338,10 @@ pub const CPU_RAIL_LIVENESS_BUDGET: std::time::Duration = std::time::Duration::f /// /// `MacosIoReportCounter::try_detect` (and so [`detect`]) makes the IOReport /// counter available only when this is [`IoReportStatus::Live`]. The first -/// call on macOS burns CPU on the calling thread for up to -/// `CPU_RAIL_LIVENESS_BUDGET`. Exposed so a caller or CLI can say *why* an -/// Apple Silicon host is not metered by IOReport. +/// call on macOS burns CPU on the calling thread for about +/// `CPU_RAIL_LIVENESS_BUDGET` (less when the rail proves live, at most five +/// budgets). Exposed so a caller or CLI can say *why* an Apple Silicon host is +/// not metered by IOReport. pub fn ioreport_status() -> IoReportStatus { #[cfg(target_os = "macos")] { @@ -877,6 +1354,7 @@ pub fn ioreport_status() -> IoReportStatus { } } + #[cfg(target_os = "linux")] mod linux_rapl { use super::*; @@ -921,19 +1399,17 @@ pub use linux_rapl::LinuxRaplCounter; mod macos_powermetrics { use super::*; - /// Wraps `powermetrics(1)` to estimate cumulative energy. macOS does - /// not expose a cumulative joule counter at user level; this wrapper - /// reports the integral of the most recent `--samplers cpu_power` - /// reading. In practice it requires root, so it usually degrades to - /// `StubCounter` at detection time. + /// Stands in for `powermetrics(1)`. It is detected whenever the binary + /// is on the PATH, but it does not run or parse it (that needs root): + /// every read returns `Err(NoCounter)`, so callers keep their own + /// estimate. It never degrades to `StubCounter`. pub struct MacosPowermetricsCounter; impl MacosPowermetricsCounter { /// Attempt to detect `powermetrics` availability. pub fn try_detect() -> Result { - // We don't require root here — `detect()` will fall back to - // StubCounter if reads fail. We do require the binary to be - // present on the PATH. + // Only the binary's presence on the PATH is checked; reads + // always fail (see `read_microjoules`). let ok = std::process::Command::new("which") .arg("powermetrics") .output() diff --git a/eoc/eoc-rs/crates/eoc/Cargo.toml b/eoc/eoc-rs/crates/eoc/Cargo.toml index b5d8e1f48..a2933a5ea 100644 --- a/eoc/eoc-rs/crates/eoc/Cargo.toml +++ b/eoc/eoc-rs/crates/eoc/Cargo.toml @@ -23,3 +23,6 @@ eoc-cascade.workspace = true clap.workspace = true anyhow.workspace = true tokio = { version = "1", features = ["rt-multi-thread", "macros", "sync", "time"] } + +[dev-dependencies] +async-trait.workspace = true diff --git a/eoc/eoc-rs/crates/eoc/src/main.rs b/eoc/eoc-rs/crates/eoc/src/main.rs index 4498e891d..926015646 100644 --- a/eoc/eoc-rs/crates/eoc/src/main.rs +++ b/eoc/eoc-rs/crates/eoc/src/main.rs @@ -21,7 +21,7 @@ use eoc_core::Query; use eoc_graph::{GraphStage, Triple}; use eoc_kv::{KvBackend, KvStage, MemoryKvBackend}; use eoc_meter::detect; -use eoc_neural::{EchoBackend, NeuralStage}; +use eoc_neural::{EchoBackend, NeuralBackend, NeuralStage}; #[derive(Parser, Debug)] #[command( @@ -56,6 +56,11 @@ async fn main() -> Result<()> { } fn build_cascade() -> Cascade { + build_cascade_with(Box::new(EchoBackend::new().with_prefix("[neural fallback] "))) +} + +/// The CLI's cascade around a given neural backend, metered by [`detect`]. +fn build_cascade_with(neural: Box) -> Cascade { let cache = Arc::new(LruCache::new(1024)); let kv_backend = Box::new(MemoryKvBackend::new()); @@ -71,9 +76,7 @@ fn build_cascade() -> Cascade { Triple::new("Mars", "fourth planet from", "the Sun"), ]); - let neural = Arc::new(NeuralStage::new(Box::new( - EchoBackend::new().with_prefix("[neural fallback] "), - ))); + let neural = Arc::new(NeuralStage::new(neural)); Cascade::new(cache, kv, graph, neural).with_meter(Arc::from(detect())) } @@ -112,7 +115,7 @@ fn ioreport_line(status: eoc_meter::IoReportStatus) -> &'static str { its readings would be GPU-only energy, zero, or a stale refresh" } IoReportStatus::CpuRailAbsent => "declined: no CPU energy rail in its Energy Model group", - IoReportStatus::Unavailable => "unavailable: no IOReport subscription could be opened", + IoReportStatus::Unavailable => "unavailable: IOReport could not be opened, or yielded no samples", } } @@ -133,8 +136,64 @@ fn idle_line(counter: &str, start: eoc_core::Result, end: eoc_core::Result< #[cfg(test)] mod tests { - use super::idle_line; - use eoc_core::Error; + use super::{build_cascade_with, idle_line}; + use async_trait::async_trait; + use eoc_core::{Error, JouleCost, JouleSource, Query, Response, Stage}; + use eoc_neural::NeuralBackend; + use std::time::{Duration, Instant}; + + /// A neural backend that does ~100 ms of real CPU work and reports its own + /// estimate for it, as a real backend without a meter would. + struct BurningBackend; + + const BURN_ESTIMATE_UJ: u64 = 50_000_000; + + #[async_trait] + impl NeuralBackend for BurningBackend { + async fn infer(&self, q: &Query) -> Response { + let started = Instant::now(); + let mut acc: u64 = 0; + while started.elapsed() < Duration::from_millis(100) { + acc = acc.wrapping_add(1); + } + std::hint::black_box(acc); + Response::new(q.id, "burned".to_string(), Stage::Neural, JouleCost::estimated(BURN_ESTIMATE_UJ)) + } + } + + /// The whole CLI path on this host, around ~100 ms of real CPU work. Where + /// the meter [`super::detect`] picks is no hardware reading — the stub, or + /// the powermetrics stand-in, which is what a Mac whose IOReport CPU rail is + /// not live falls back to — the cost is the backend's own estimate: the + /// same non-zero figure, tagged estimated. Never a zero for the work, and + /// never tagged measured. + #[tokio::test] + async fn work_on_an_unmetered_host_keeps_its_estimate() { + let cascade = build_cascade_with(Box::new(BurningBackend)); + let meter = cascade.meter().name(); + #[cfg(target_os = "macos")] + assert_eq!( + meter == "ioreport", + eoc_meter::ioreport_status() == eoc_meter::IoReportStatus::Live, + "the CLI's meter is {meter} while IOReport is {:?}", + eoc_meter::ioreport_status() + ); + let r = cascade.resolve(Query::new("a question nothing has cached")).await; + eprintln!("eoc cost on this host: meter={meter} cost={}", r.joule_cost); + assert_eq!(r.stage, Stage::Neural); + if matches!(meter, "stub" | "powermetrics") { + assert_eq!(r.joule_cost, JouleCost::estimated(BURN_ESTIMATE_UJ), "meter={meter}"); + } else { + // A hardware counter: the estimate plus what it measured, never less. + assert!(r.joule_cost.microjoules >= BURN_ESTIMATE_UJ, "{}", r.joule_cost); + assert_eq!( + r.joule_cost.source == JouleSource::Measured, + r.joule_cost.microjoules > BURN_ESTIMATE_UJ, + "{}", + r.joule_cost + ); + } + } #[test] fn a_counter_that_did_not_read_is_never_printed_as_a_figure() { -- 2.54.0 (Apple Git-157) From 467acb385f47b9e756cc188f7662bdf982a20dd1 Mon Sep 17 00:00:00 2001 From: Transaction Science Date: Wed, 30 Sep 2026 08:05:00 -0400 Subject: [PATCH 03/22] eoc-streaming: a stream's energy is measured only when the counter advanced MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit StreamMeter::start marked a stream measured as soon as one counter read succeeded, as long as the counter was not named "stub". So any other counter that read but did not move had its zero sealed as JouleSource::Measured. That includes a frozen rail, a placeholder, or a counter with no resolution over the span. The flag was also set before any checkpoint, so snapshot() reported a measured 0 µJ for a stream that had not been metered at all. c6378629 listed this as deferred. A second defect sat in the same function. When the start read failed, the baseline was 0, so the first successful checkpoint reported the counter's whole since-boot total as the stream's energy. What holds now. The baseline is an Option. A checkpoint computes a delta only when there is a baseline and a reading now. The account is `measured` only when the counter advanced over the span (now > start). Without a baseline or a reading, the account stays as it was. StreamAccount's fields and their types are unchanged. The `measured` doc now says what the flag means. Tests: a non-stub counter that reads a constant is not a measurement (before and after a checkpoint). A counter that advances by 450 µJ is JouleCost::measured(450). A failed start read followed by a reading of 9,000,000 µJ measures nothing. Falsified, each restored with git checkout: - the old `name() != "stub"` rule fails two tests; - `>=` in place of `>` fails three; - a 0 baseline after a failed start fails with `left: (9000000, true)`; - marking the stream measured at start fails with "measured before any checkpoint". eoc-streaming passes: 20 unit tests plus its integration tests. The whole eoc workspace passes: 594 passed, 0 failed, 3 ignored. Not done: an unmetered stream still reports `0 µJ (estimated)`, because StreamAccount has no way to say "not metered". Saying so honestly would change a public type. Co-Authored-By: Claude Opus 5.5 --- .../crates/eoc-streaming/src/account.rs | 100 ++++++++++++++---- 1 file changed, 81 insertions(+), 19 deletions(-) diff --git a/eoc/eoc-rs/crates/eoc-streaming/src/account.rs b/eoc/eoc-rs/crates/eoc-streaming/src/account.rs index 0d5011105..fea7019e4 100644 --- a/eoc/eoc-rs/crates/eoc-streaming/src/account.rs +++ b/eoc/eoc-rs/crates/eoc-streaming/src/account.rs @@ -21,7 +21,10 @@ pub struct StreamAccount { pub tokens: u64, /// Cumulative micro-joule delta attributed to this stream. pub microjoules: u64, - /// Whether the joule reading is from a hardware counter. + /// Whether `microjoules` is a hardware counter's reading of this span: + /// the counter read at the start and at the latest checkpoint, and + /// advanced between them. A zero delta, a failed read or the stub leaves + /// it `false`: a counter that did not move has not measured the work. pub measured: bool, } @@ -48,8 +51,9 @@ impl StreamAccount { #[derive(Clone)] pub struct StreamMeter { counter: Arc, - start_uj: u64, - measured: bool, + /// The counter's reading when the stream started; `None` when that read + /// failed, and then no span can be measured. + start_uj: Option, state: Arc>, } @@ -58,27 +62,23 @@ impl std::fmt::Debug for StreamMeter { f.debug_struct("StreamMeter") .field("counter", &self.counter.name()) .field("start_uj", &self.start_uj) - .field("measured", &self.measured) .finish() } } impl StreamMeter { /// Start metering a stream. Reads the counter once; if the read - /// fails the meter degrades to `Estimated` and continues. + /// fails the meter degrades to `Estimated` and continues. Nothing is + /// measured until a checkpoint sees the counter advance. pub fn start(counter: Arc) -> Self { - let (start_uj, measured) = match counter.read_microjoules() { - Ok(uj) => (uj, counter.name() != "stub"), - Err(_) => (0, false), - }; + let start_uj = counter.read_microjoules().ok(); Self { counter, start_uj, - measured, state: Arc::new(Mutex::new(StreamAccount { tokens: 0, microjoules: 0, - measured, + measured: false, })), } } @@ -92,17 +92,21 @@ impl StreamMeter { /// Refresh the joule delta from the counter. Cheap and safe to /// call at every checkpoint; the cost is one counter read. + /// + /// The delta is a measurement only if the counter advanced since the + /// start. A counter that reads but does not move — a frozen rail, a + /// placeholder — has not measured the work, and its zero is never + /// sealed as one. pub fn checkpoint(&self) -> StreamResult { - let now_uj = match self.counter.read_microjoules() { - Ok(uj) => uj, - Err(_) => { - // No measurement available — leave the existing delta in place. - let g = self.state.lock().map_err(poisoned)?; - return Ok(*g); - } + let (Some(start_uj), Ok(now_uj)) = (self.start_uj, self.counter.read_microjoules()) else { + // No baseline, or no reading now: nothing to measure. Leave the + // account as it was — never a counter's since-boot total. + let g = self.state.lock().map_err(poisoned)?; + return Ok(*g); }; let mut g = self.state.lock().map_err(poisoned)?; - g.microjoules = now_uj.saturating_sub(self.start_uj); + g.microjoules = now_uj.saturating_sub(start_uj); + g.measured = now_uj > start_uj; Ok(*g) } @@ -128,6 +132,64 @@ mod tests { use super::*; use eoc_meter::StubCounter; + /// A counter that reads the scripted values in turn (then the last one + /// again); `None` is a failed read. + struct Scripted { + reads: Mutex>>, + } + + impl Scripted { + fn new(reads: &[Option]) -> Arc { + let mut reads = reads.to_vec(); + reads.reverse(); + Arc::new(Self { reads: Mutex::new(reads) }) + } + } + + impl JouleCounter for Scripted { + fn read_microjoules(&self) -> eoc_core::Result { + let mut r = self.reads.lock().unwrap(); + let next = if r.len() > 1 { r.pop().unwrap() } else { r[0] }; + next.ok_or(eoc_core::Error::NoCounter) + } + fn name(&self) -> &'static str { + "scripted" + } + } + + /// A non-stub counter that reads but does not move has not measured the + /// stream: its zero is estimated, never sealed as a measurement. + #[test] + fn a_counter_that_did_not_advance_is_not_a_measurement() { + let meter = StreamMeter::start(Scripted::new(&[Some(7_000)])); + assert!(!meter.snapshot().unwrap().measured, "measured before any checkpoint"); + meter.on_token(3).unwrap(); + let acct = meter.finish().unwrap(); + assert_eq!((acct.microjoules, acct.measured), (0, false)); + assert_eq!(acct.joule_cost().source, JouleSource::Estimated); + } + + /// A counter that advanced over the span is a measurement of it. + #[test] + fn a_counter_that_advanced_is_a_measurement() { + let meter = StreamMeter::start(Scripted::new(&[Some(7_000), Some(7_000), Some(7_450)])); + let mid = meter.checkpoint().unwrap(); + assert_eq!((mid.microjoules, mid.measured), (0, false), "no advance yet"); + let acct = meter.finish().unwrap(); + assert_eq!((acct.microjoules, acct.measured), (450, true)); + assert_eq!(acct.joule_cost(), JouleCost::measured(450)); + } + + /// Without a starting reading there is no span: a later reading is the + /// counter's since-boot total, not this stream's energy. + #[test] + fn a_failed_start_measures_nothing() { + let meter = StreamMeter::start(Scripted::new(&[None, Some(9_000_000)])); + let acct = meter.finish().unwrap(); + assert_eq!((acct.microjoules, acct.measured), (0, false)); + assert_eq!(acct.joule_cost().source, JouleSource::Estimated); + } + #[test] fn stub_counter_estimated() { let meter = StreamMeter::start(Arc::new(StubCounter)); -- 2.54.0 (Apple Git-157) From bc6d497ef17883289a45923e4a2e306a815f3f84 Mon Sep 17 00:00:00 2001 From: Transaction Science Date: Wed, 30 Sep 2026 08:54:27 -0400 Subject: [PATCH 04/22] eoc-meter: the liveness tests hold on a loaded host, and pin what the probe does MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The round-3 review of 3b827e2a found the hardware liveness tests flaky on the M5 Max that also runs the macOS CI. With three test processes at once and a load average of 97-172, 8 of 12 runs failed on "the probe's thread was on CPU for only 33% ... 50%", and the oracle once completed 2 reads in its fixed 3 s window. Both assertions measured wall time, which host load stretches. A sleeping probe (26-27% on CPU) sat close to a loaded burning one (33-57%). The probe now burns thread CPU, not wall time. Each tick spins until this thread's CPU clock (clock_gettime(CLOCK_THREAD_CPUTIME_ID)) has advanced 25 ms. A descheduled thread's CPU clock stands still, so load stretches a tick in wall time but never shortens its CPU. A wall cap of 500 ms bounds a tick on a host that gives the thread under 5% of a core. The spin is a pure function, spin_for_cpu, tested with scripted clocks, and the real tick is still injected into the generic loop. The probe records each sample call: its wall duration, and the thread's CPU clock before and after. The tests read that record instead of timing the probe from outside, and no hardware test asserts a wall-clock ratio. - the_probe_burns_its_tick_of_thread_cpu_between_samples: the CPU consumed between one sample call and the next must lie between 12.5 and 50 ms, against a literal 25 ms in the test. Only the 500 ms wall cap could make a tick fall short, and the message says so. The upper bound catches a slower cadence: round 3's surviving PROBE_TICK = 100 ms now fails. The recorded tick CPU is also checked against the test's own reading of the thread clock over the whole probe. - the_probe_samples_only_the_energy_model_group: the median wall duration of the probe's own sample calls must be under 50 ms. The Energy Model group takes ~6-12 ms and every channel 130-800 ms. This replaces the ">= 10 intervals" and ">= 50% on CPU" assertions. - the_probe_sampler_reads_each_cpu_rail_as_the_oracle_does: for three samples, the oracle reads each CPU rail just before and just after the sample the probe's sampler takes. A rail that held must read EQUAL, and a rail that moved must read between the two oracle reads. The sampler must also list exactly the oracle's CPU rails. This catches a sampler that returns a constant 0 or another rail's value. The oracle now reads until its window has passed AND it has 3 reads, the fewest in which a rail can change twice, capped at five windows. Fewer than 3 reads is a test failure, never a Frozen verdict. A failed first read no longer reports "no IOReport". Round-3 survivors: - F21: the counter's own read path now has a test on the hosts we have. the_counter_advances_when_a_rail_it_sums_advanced builds the counter past the gate on purpose. It takes a baseline, then reads GPU Energy (which the counter sums) independently in absolute values, with the oracle's own unit table, until it has risen by 1 µJ or 3 s pass, then reads the counter again. If the GPU rose, the counter must have advanced. If not, the counter is printed, not asserted, and the line says so. - probe_status mapped "no Energy Model subscription" inline, and a mutation to Live survived. It now goes through the pure status_from_probe(). Following the family rule, a group that cannot be subscribed is Unavailable, never CpuRailAbsent: CpuRailAbsent now means the group was sampled and held no CPU rail. The IoReportStatus docs and the `eoc meter` line for Unavailable say so. No variant changed. The pure rules each have their own test now: a decrease is not an advance; a rail absent from the previous sample only sets its baseline; no subscription or no sample is Unavailable; no verdict is Unavailable. The once-per-process pin releases six threads together, and each asks for the status, try_detect() and detect(). The process-wide PROBE_RUNS must read exactly 1. The test that prints "ioreport liveness: status=" keeps its name and line. On this M5 Max, unloaded, 28 passed: per-tick thread CPU 25.01-25.03 ms, median sample 11.8 ms, CPU Energy read identically by the sampler and the oracle, GPU Energy +10.6 mJ in 257 ms with the counter at 0 -> 53,238 µJ. Co-Authored-By: Claude Opus 5.5 --- eoc/eoc-rs/crates/eoc-meter/src/lib.rs | 988 +++++++++++++++++++------ eoc/eoc-rs/crates/eoc/src/main.rs | 4 +- 2 files changed, 762 insertions(+), 230 deletions(-) diff --git a/eoc/eoc-rs/crates/eoc-meter/src/lib.rs b/eoc/eoc-rs/crates/eoc-meter/src/lib.rs index 1397e9a68..0fc65cee4 100644 --- a/eoc/eoc-rs/crates/eoc-meter/src/lib.rs +++ b/eoc/eoc-rs/crates/eoc-meter/src/lib.rs @@ -240,6 +240,35 @@ mod rail_liveness { w } + /// Spin on the calling thread until `thread_cpu`, this thread's CPU clock, + /// has advanced by `want`, or `wall` has advanced by `wall_cap`, whichever + /// comes first. Returns the thread CPU consumed. + /// + /// The target is CPU time, not wall time, so every tick burns the same CPU + /// however loaded the host is: while the thread is descheduled its CPU + /// clock stands still, and the tick simply takes longer. Only the wall cap + /// can cut a tick short, on a host that gives this thread less than + /// `want / wall_cap` of a core (5% for the probe's 25 ms in 500 ms). + pub(crate) fn spin_for_cpu( + thread_cpu: impl Fn() -> Duration, + wall: impl Fn() -> Duration, + want: Duration, + wall_cap: Duration, + ) -> Duration { + let (cpu0, wall0) = (thread_cpu(), wall()); + let mut acc: u64 = 0; + loop { + let used = thread_cpu().saturating_sub(cpu0); + if used >= want || wall().saturating_sub(wall0) >= wall_cap { + std::hint::black_box(acc); + return used; + } + for _ in 0..256 { + acc = std::hint::black_box(acc.wrapping_add(1)); + } + } + } + /// The status a probe verdict gives this host. No verdict — not one /// interval could be observed — is `Unavailable`: nothing was learned about /// the rail, and only `Live` makes the counter available. @@ -252,6 +281,19 @@ mod rail_liveness { } } + /// The status one probe gives this host. `probe` is `None` when the Energy + /// Model group subscription could not be opened, and otherwise the probe's + /// verdict, itself `None` when not one interval could be observed. Neither + /// taught anything about the CPU rail, so both are `Unavailable`: never + /// `Live`, and never `CpuRailAbsent`, which says samples were taken and no + /// CPU rail was in them. + pub(crate) fn status_from_probe(probe: Option>) -> crate::IoReportStatus { + match probe { + None => crate::IoReportStatus::Unavailable, + Some(verdict) => status_of(verdict), + } + } + /// Whether a host with this status may use the IOReport counter: only when /// its CPU rail is live. Absent, frozen or unavailable all decline, so /// callers estimate rather than seal a reading that did not track the work. @@ -421,14 +463,53 @@ mod rail_liveness { assert_eq!(w.rails, vec![(key("DIE_0_CPU Energy", 0), 1), (key("DIE_1_CPU Energy", 0), 1)]); } - /// No CPU rail in any sample is Absent; a rail that goes down (a reset) - /// has not advanced. + /// Samples with no CPU rail in them, however much else moves: Absent. #[test] - fn no_cpu_rail_is_absent_and_a_reset_is_not_an_advance() { + fn no_cpu_rail_is_absent() { let (w, _) = scripted(TICK, |i| Some(vec![("GPU Energy", i as i64), ("PCPU", i as i64)])); assert_eq!(w.verdict(), Some(CpuRail::Absent)); + assert!(w.rails.is_empty(), "{w:?}"); + } + + /// A rail that goes DOWN between consecutive samples — a reset or a + /// wrap — has not advanced. A rail falling at every sample is Frozen + /// with no advance at all; and one real step followed by a reset is + /// one advance, not the two a live rail needs. + #[test] + fn a_decrease_is_not_an_advance() { let (w, _) = scripted(TICK, |i| Some(vec![("CPU Energy", 1_000 - i as i64)])); - assert_eq!(w.verdict(), Some(CpuRail::Frozen)); + assert_eq!(w.verdict(), Some(CpuRail::Frozen), "a falling rail counted as live: {w:?}"); + assert_eq!(w.rails, vec![(key("CPU Energy", 0), 0)], "a decrease counted as an advance"); + + let (w, _) = scripted(TICK, |i| Some(vec![("CPU Energy", [500, 500, 500, 900, 900, 900, 10][i.min(6)])])); + assert_eq!(w.verdict(), Some(CpuRail::Frozen), "one step and one reset counted as live: {w:?}"); + assert_eq!(w.rails, vec![(key("CPU Energy", 0), 1)], "the reset counted as an advance"); + } + + /// A rail absent from the previous sample only sets its baseline in + /// this one; it has not advanced. A rail that joins late, at any value, + /// and holds is Frozen with no advance; and a rail that drops out of + /// every other sample is never "new, so advanced" when it comes back. + #[test] + fn a_rail_new_in_a_sample_only_sets_its_baseline() { + let (w, _) = scripted(TICK, |i| { + let mut s = vec![("DIE_0_CPU Energy", 100)]; + if i >= 3 { + s.push(("DIE_1_CPU Energy", 5_000_000)); + } + Some(s) + }); + assert_eq!(w.verdict(), Some(CpuRail::Frozen), "{w:?}"); + assert_eq!( + w.rails, + vec![(key("DIE_0_CPU Energy", 0), 0), (key("DIE_1_CPU Energy", 0), 0)], + "a rail's first appearance counted as an advance" + ); + + let (w, _) = scripted(TICK, |i| { + Some(if i % 2 == 0 { vec![("CPU Energy", 7)] } else { vec![("GPU Energy", i as i64)] }) + }); + assert_eq!(w.verdict(), Some(CpuRail::Frozen), "a rail's reappearances counted as advances: {w:?}"); assert_eq!(w.rails, vec![(key("CPU Energy", 0), 0)]); } @@ -442,6 +523,74 @@ mod rail_liveness { assert_eq!(status_of(Some(CpuRail::Live)), S::Live); } + /// No verdict (not one interval observed) is `Unavailable`: never + /// `Live`, never `CpuRailAbsent`. + #[test] + fn no_verdict_is_unavailable() { + let status = status_of(None); + assert_eq!(status, crate::IoReportStatus::Unavailable); + assert!(!admits_counter(status), "no verdict admitted the counter"); + } + + /// When the Energy Model group subscription cannot be opened there is + /// no probe; when it opens but never yields two usable samples there is + /// no verdict. Both are `Unavailable` — nothing was learned about the + /// CPU rail — never `Live`, and never `CpuRailAbsent`. + #[test] + fn no_energy_model_subscription_is_unavailable() { + use crate::IoReportStatus as S; + assert_eq!(status_from_probe(None), S::Unavailable, "no group subscription"); + let (never, _) = scripted(TICK, |_| None); + assert_eq!(status_from_probe(Some(never.verdict())), S::Unavailable, "a subscription with no sample"); + let (once, _) = scripted(TICK, |i| (i == 0).then(|| vec![("CPU Energy", 7)])); + assert_eq!(status_from_probe(Some(once.verdict())), S::Unavailable, "a subscription with one sample"); + // A probe that did sample still maps through its verdict. + assert_eq!(status_from_probe(Some(Some(CpuRail::Absent))), S::CpuRailAbsent); + assert_eq!(status_from_probe(Some(Some(CpuRail::Frozen))), S::CpuRailFrozen); + assert_eq!(status_from_probe(Some(Some(CpuRail::Live))), S::Live); + } + + /// A clock pair for [`spin_for_cpu`] that only the spin moves: every + /// read of the CPU clock is one step, which advances this thread's CPU + /// clock by `cpu_step` and the wall clock by `wall_step`. + fn spin_scripted(cpu_step: Duration, wall_step: Duration, want: Duration, cap: Duration) -> (Duration, u32) { + let steps = Cell::new(0u32); + let used = spin_for_cpu( + || { + assert!(steps.get() < 1_000_000, "the tick never stopped"); + steps.set(steps.get() + 1); + cpu_step * steps.get() + }, + || wall_step * steps.get(), + want, + cap, + ); + (used, steps.get()) + } + + /// The tick spins until this thread's CPU clock has advanced by the + /// tick, not the wall clock: a thread that gets a quarter of a core + /// still burns the whole 25 ms (in 100 ms of wall time). Only the wall + /// cap ends a tick early: a thread that never runs, or gets under 5% of + /// a core, stops at 500 ms of wall time. + #[test] + fn the_tick_spins_until_the_thread_cpu_clock_advanced() { + let ms = Duration::from_millis; + let (want, cap) = (ms(25), ms(500)); + // A quarter of a core: 25 ms of CPU, 100 ms of wall time. + let (used, steps) = spin_scripted(ms(1), ms(4), want, cap); + assert_eq!((used, steps), (ms(25), 26), "the tick must burn 25 ms of thread CPU, however slowly it gets it"); + // A whole core. + let (used, _) = spin_scripted(ms(1), ms(1), want, cap); + assert_eq!(used, ms(25)); + // Never scheduled: the wall cap ends it with no CPU burned. + let (used, steps) = spin_scripted(Duration::ZERO, ms(10), want, cap); + assert_eq!((used, steps), (Duration::ZERO, 51), "the wall cap must end a tick that gets no CPU"); + // 2.5% of a core: the wall cap, at 13 ms of CPU. + let (used, steps) = spin_scripted(ms(1), ms(40), want, cap); + assert_eq!((used, steps), (ms(13), 14)); + } + /// Only a live CPU rail admits the counter. #[test] fn only_a_live_rail_admits_the_counter() { @@ -477,7 +626,7 @@ mod rail_liveness { mod macos_ioreport { use super::*; use crate::rail_liveness::{self, RailWatch}; - use std::os::raw::{c_int, c_long, c_void}; + use std::os::raw::{c_int, c_long, c_uint, c_void}; use std::sync::Mutex; use std::sync::atomic::{AtomicUsize, Ordering}; use std::time::{Duration, Instant}; @@ -535,6 +684,28 @@ mod macos_ioreport { buffer_size: c_long, encoding: u32, ) -> bool; + fn clock_gettime(clock_id: c_uint, tp: *mut Timespec) -> c_int; + } + + /// `struct timespec` on 64-bit macOS. + #[repr(C)] + struct Timespec { + tv_sec: c_long, + tv_nsec: c_long, + } + + /// `CLOCK_THREAD_CPUTIME_ID` in macOS ``. + const CLOCK_THREAD_CPUTIME_ID: c_uint = 16; + + /// CPU time the calling thread has consumed. Zero if the clock cannot be + /// read, which leaves a probe tick spinning to its wall cap: slower, but + /// still burning CPU. + fn thread_cpu_time() -> Duration { + let mut ts = Timespec { tv_sec: 0, tv_nsec: 0 }; + if unsafe { clock_gettime(CLOCK_THREAD_CPUTIME_ID, &mut ts) } != 0 { + return Duration::ZERO; + } + Duration::new(ts.tv_sec.max(0) as u64, ts.tv_nsec.clamp(0, 999_999_999) as u32) } #[link(name = "IOReport", kind = "dylib")] @@ -711,48 +882,78 @@ mod macos_ioreport { } } - /// CPU burned on the probing thread before each liveness re-sample. + /// The CPU the probing thread burns between two liveness samples: this + /// much of its own CPU time, measured on its CPU clock, so host load + /// stretches a tick's wall time but never shortens its CPU. const PROBE_TICK: Duration = Duration::from_millis(25); - /// Spin this thread for `d`. - fn burn(d: Duration) { + /// However little CPU the probing thread gets, one tick ends after this + /// much wall time. + const PROBE_TICK_WALL_CAP: Duration = Duration::from_millis(500); + + /// One tick of the probe: spin until this thread's CPU clock has advanced + /// [`PROBE_TICK`], or [`PROBE_TICK_WALL_CAP`] of wall time has passed. + fn probe_tick() { let started = Instant::now(); - let mut acc: u64 = 0; - while started.elapsed() < d { - acc = acc.wrapping_add(1); - } - std::hint::black_box(acc); + rail_liveness::spin_for_cpu(thread_cpu_time, || started.elapsed(), PROBE_TICK, PROBE_TICK_WALL_CAP); } /// How many times [`probe_status`] — the probe behind the cached - /// [`crate::ioreport_status`] — has run in this process. Read by a test - /// that pins the cache: it must be exactly one. + /// [`crate::ioreport_status`] — has run in this process, on any thread. + /// Read by a test that pins the cache: it must be exactly one. pub(crate) static PROBE_RUNS: AtomicUsize = AtomicUsize::new(0); + /// One sample call the probe made: its wall-clock duration, and this + /// thread's CPU clock just before and just after it. Kept so a test can pin + /// what the probe itself did — what each sample cost, and the CPU burned + /// between samples — without timing the probe from outside. + #[derive(Debug, Clone, Copy)] + #[cfg_attr(not(test), allow(dead_code))] // read only by the tests + pub(crate) struct SampleCall { + pub(crate) wall: Duration, + pub(crate) cpu_before: Duration, + pub(crate) cpu_after: Duration, + } + + /// What one liveness probe saw, and every sample call it made, in order. + #[derive(Debug)] + #[cfg_attr(not(test), allow(dead_code))] // `calls` is read only by the tests + pub(crate) struct ProbeRun { + pub(crate) watch: RailWatch, + pub(crate) calls: Vec, + } + /// One liveness probe on a fresh Energy Model subscription: sample, burn - /// [`PROBE_TICK`] of CPU on this thread, sample again, by the rule in + /// [`PROBE_TICK`] of this thread's CPU, sample again, by the rule in /// [`rail_liveness::watch`]. `None` when the subscription does not open. The /// probe takes its own samples, so no counter's cumulative reading is /// touched. - pub(crate) fn probe(budget: Duration) -> Option { + pub(crate) fn probe(budget: Duration) -> Option { let state = MacosIoReportCounter::open_energy_model()?; + let mut calls = Vec::new(); let t0 = Instant::now(); - Some(rail_liveness::watch(|| state.energy_model_sample(), || burn(PROBE_TICK), || t0.elapsed(), budget)) + let watch = rail_liveness::watch( + || { + let cpu_before = thread_cpu_time(); + let started = Instant::now(); + let sample = state.energy_model_sample(); + calls.push(SampleCall { wall: started.elapsed(), cpu_before, cpu_after: thread_cpu_time() }); + sample + }, + probe_tick, + || t0.elapsed(), + budget, + ); + Some(ProbeRun { watch, calls }) } - /// This host's status from one probe. Backs the cached - /// [`crate::ioreport_status`]; the probe's subscription is dropped after. + /// This host's status from one probe, through the pure + /// [`rail_liveness::status_from_probe`]: a group subscription that does not + /// open is `Unavailable`. Backs the cached [`crate::ioreport_status`]; the + /// probe's subscription is dropped after. pub(crate) fn probe_status(budget: Duration) -> crate::IoReportStatus { PROBE_RUNS.fetch_add(1, Ordering::SeqCst); - match probe(budget) { - Some(watch) => rail_liveness::status_of(watch.verdict()), - // No Energy Model subscription. Tell "no IOReport" from "no such - // group" by opening the subscription the counter itself would use. - None => match MacosIoReportCounter::open() { - Some(_) => crate::IoReportStatus::CpuRailAbsent, - None => crate::IoReportStatus::Unavailable, - }, - } + rail_liveness::status_from_probe(probe(budget).map(|run| run.watch.verdict())) } impl JouleCounter for MacosIoReportCounter { @@ -890,12 +1091,36 @@ mod macos_ioreport { unsafe extern "C" { fn clock_gettime_nsec_np(clock_id: u32) -> u64; } - /// `CLOCK_THREAD_CPUTIME_ID` in macOS `<_time.h>`. - const CLOCK_THREAD_CPUTIME_ID: u32 = 16; - /// CPU time this thread has consumed. - fn thread_cpu_time() -> Duration { - Duration::from_nanos(unsafe { clock_gettime_nsec_np(CLOCK_THREAD_CPUTIME_ID) }) + /// `CLOCK_THREAD_CPUTIME_ID` in macOS ``: the tests' own copy. + const OWN_CLOCK_THREAD_CPUTIME_ID: u32 = 16; + + /// This thread's CPU time, read by the tests themselves + /// (`clock_gettime_nsec_np`), not through the implementation's clock. + fn own_thread_cpu() -> Duration { + Duration::from_nanos(unsafe { clock_gettime_nsec_np(OWN_CLOCK_THREAD_CPUTIME_ID) }) + } + + /// The tick the probe is specified to burn, stated here rather than + /// read from the implementation's constant. + const TICK: Duration = Duration::from_millis(25); + + /// Real CPU work for the tests: spin this thread for `d` of wall time. + fn spin(d: Duration) { + let started = Instant::now(); + let mut acc: u64 = 0; + while started.elapsed() < d { + acc = std::hint::black_box(acc.wrapping_add(1)); + } + } + + /// The median of a non-empty list (the mean of the middle two when even). + fn median(v: &[Duration]) -> Duration { + assert!(!v.is_empty(), "the median of nothing"); + let mut v = v.to_vec(); + v.sort(); + let m = v.len() / 2; + if v.len().is_multiple_of(2) { (v[m - 1] + v[m]) / 2 } else { v[m] } } #[test] @@ -915,28 +1140,6 @@ mod macos_ioreport { // ── An independent oracle ──────────────────────────────────────────── - /// What the oracle saw on this host. - #[derive(Debug, PartialEq)] - enum Oracle { - NoIoReport, - NoCpuRail, - CpuRailMoved(bool), - } - - #[derive(Debug)] - struct OracleRun { - verdict: Oracle, - reads: u32, - cpu_changes: Vec, - gpu_changes: Vec, - } - - impl OracleRun { - fn bare(verdict: Oracle) -> Self { - Self { verdict, reads: 0, cpu_changes: Vec::new(), gpu_changes: Vec::new() } - } - } - /// The oracle's own CFString conversion, so a broken implementation /// helper cannot make the oracle agree with it. fn oracle_name(s: CFStringRef) -> String { @@ -954,126 +1157,236 @@ mod macos_ioreport { String::from_utf8_lossy(&buf[..end]).into_owned() } - /// One read of the oracle's subscription: the ABSOLUTE raw value of - /// every channel, in any group, whose name ends with `CPU Energy`, and - /// of every one ending with `GPU Energy`, in sample order. - fn oracle_read(sub: IOReportSubscriptionRef, chans: CFMutableDictionaryRef) -> Option<(Vec, Vec)> { - unsafe { - let smp = IOReportCreateSamples(sub, chans, std::ptr::null()); - if smp.is_null() { - return None; + /// One channel as the oracle read it: its group, name and unit label, + /// and its ABSOLUTE raw integer (no delta samples). + #[derive(Debug, Clone)] + struct OracleChannel { + group: String, + name: String, + unit: String, + raw: i64, + } + + /// The oracle's own subscription, over every channel (not the probe's + /// Energy Model group), opened with its own calls. It shares nothing + /// with the implementation but the raw FFI declarations. + struct OracleSub { + sub: IOReportSubscriptionRef, + chans: CFMutableDictionaryRef, + } + + impl OracleSub { + fn open() -> Option { + unsafe { + let all = IOReportCopyAllChannels(0, 0); + if all.is_null() { + return None; + } + let mut chans: CFMutableDictionaryRef = std::ptr::null_mut(); + let sub = IOReportCreateSubscription(std::ptr::null(), all, &mut chans, 0, std::ptr::null()); + CFRelease(all as CFTypeRef); + if sub.is_null() || chans.is_null() { + if !sub.is_null() { + CFRelease(sub); + } + if !chans.is_null() { + CFRelease(chans as CFTypeRef); + } + return None; + } + Some(Self { sub, chans }) } - let key = CFStringCreateWithCString( - std::ptr::null_mut(), - c"IOReportChannels".as_ptr() as *const _, - K_CFSTRING_ENCODING_UTF8, - ); - let (mut cpu, mut gpu) = (Vec::new(), Vec::new()); - if !key.is_null() { - let arr = CFDictionaryGetValue(smp, key as *const _) as CFArrayRef; - CFRelease(key as CFTypeRef); + } + + /// One read: every channel, in any group, whose name ends with `CPU + /// Energy` or `GPU Energy`, in sample order. `None` when IOReport + /// returns no sample, or one with no channel list. + fn read(&self) -> Option> { + unsafe { + let smp = IOReportCreateSamples(self.sub, self.chans, std::ptr::null()); + if smp.is_null() { + return None; + } + let key = CFStringCreateWithCString( + std::ptr::null_mut(), + c"IOReportChannels".as_ptr() as *const _, + K_CFSTRING_ENCODING_UTF8, + ); + let arr = if key.is_null() { + std::ptr::null() + } else { + let arr = CFDictionaryGetValue(smp, key as *const _) as CFArrayRef; + CFRelease(key as CFTypeRef); + arr + }; + let mut out = None; if !arr.is_null() { + let mut chans = Vec::new(); for i in 0..CFArrayGetCount(arr) { let it = CFArrayGetValueAtIndex(arr, i); if it.is_null() { continue; } let name = oracle_name(IOReportChannelGetChannelName(it)); - if name.ends_with("CPU Energy") { - cpu.push(IOReportSimpleGetIntegerValue(it, 0) as i64); - } else if name.ends_with("GPU Energy") { - gpu.push(IOReportSimpleGetIntegerValue(it, 0) as i64); + if name.ends_with("CPU Energy") || name.ends_with("GPU Energy") { + chans.push(OracleChannel { + group: oracle_name(IOReportChannelGetGroup(it)), + unit: oracle_name(IOReportChannelGetUnitLabel(it)), + raw: IOReportSimpleGetIntegerValue(it, 0) as i64, + name, + }); } } + out = Some(chans); } + CFRelease(smp); + out } - CFRelease(smp); - Some((cpu, gpu)) } } - /// An oracle that shares nothing with the liveness probe but the raw FFI - /// declarations. It opens its own subscription over every channel (not - /// the probe's Energy Model group) with its own calls, converts names - /// with its own helper, reads ABSOLUTE values (no delta samples), - /// matches names with `ends_with("CPU Energy")` in any group (not the - /// implementation's group filter and die-prefix stripping), and keeps a - /// HELPER thread burning CPU for the whole `window`, so the window is - /// under load even while an all-channel sample blocks this thread. + impl Drop for OracleSub { + fn drop(&mut self) { + unsafe { + CFRelease(self.sub); + CFRelease(self.chans as CFTypeRef); + } + } + } + + /// A rise of `raw` in a rail's unit, in whole µJ, by the oracle's own + /// unit table. An unknown unit counts only whole millions of raw units, + /// a µJ in the finest unit (pJ). + fn oracle_uj(raw: i64, unit: &str) -> u64 { + let raw = raw.max(0) as u64; + match unit { + "pJ" => raw / 1_000_000, + "nJ" => raw / 1_000, + "uJ" | "µJ" => raw, + "mJ" => raw.saturating_mul(1_000), + "J" => raw.saturating_mul(1_000_000), + _ => raw / 1_000_000, + } + } + + /// What the oracle saw on this host. + #[derive(Debug, PartialEq)] + enum Oracle { + NoIoReport, + NoCpuRail, + CpuRailMoved(bool), + } + + #[derive(Debug)] + struct OracleRun { + verdict: Oracle, + reads: u32, + took: Duration, + cpu_changes: Vec, + gpu_changes: Vec, + } + + /// The fewest reads in which one rail can change twice, the oracle's + /// bar for a moving rail. The oracle reads past its window until it has + /// this many. + const ORACLE_MIN_READS: u32 = 3; + + /// However slow its reads, the oracle stops after this many windows. + const ORACLE_WINDOW_CAP: u32 = 5; + + impl OracleRun { + /// The oracle's evidence is only as good as its reads. Fewer than + /// three cannot show a rail changing twice, so they say nothing about + /// whether the CPU rail is frozen: that is a test FAILURE, never a + /// Frozen verdict. With no subscription there are no reads to have. + fn assert_evidence(&self) { + if self.verdict == Oracle::NoIoReport { + return; + } + assert!( + self.reads >= ORACLE_MIN_READS, + "the oracle completed only {} reads in {:?}, though it keeps reading past its window until \ + it has {ORACLE_MIN_READS}, for up to {ORACLE_WINDOW_CAP} windows: too few to see any rail \ + change twice, so there is no evidence about the CPU rail either way", + self.reads, + self.took + ); + } + } + + /// The oracle's view of the CPU rail, over at least `window` while a + /// HELPER thread burns CPU the whole time (so the window is under load + /// even while an all-channel read blocks this thread). /// - /// It reads over and over for the whole window, and counts per rail at - /// how many reads the value changed since the read before. + /// It reads its own all-channel subscription over and over, matches + /// names with `ends_with("CPU Energy")` in any group (not the + /// implementation's group filter and die-prefix stripping), and counts + /// per rail at how many reads the value changed since the read before. /// `CpuRailMoved(true)` only if one CPU rail changed at two or more /// reads: a live rail changes at nearly every read, a batch-refreshed one /// at most once in a window this short, and one change is not a live /// reading. Its own literal threshold, not the implementation's constant. + /// It reads until the window has passed AND it has + /// [`ORACLE_MIN_READS`] reads, for at most [`ORACLE_WINDOW_CAP`] + /// windows, so slow reads on a loaded host lengthen the window rather + /// than thin the evidence. fn oracle_cpu_rail(window: Duration) -> OracleRun { - let (sub, chans) = unsafe { - let all = IOReportCopyAllChannels(0, 0); - if all.is_null() { - return OracleRun::bare(Oracle::NoIoReport); - } - let mut chans: CFMutableDictionaryRef = std::ptr::null_mut(); - let sub = IOReportCreateSubscription(std::ptr::null(), all, &mut chans, 0, std::ptr::null()); - CFRelease(all as CFTypeRef); - if sub.is_null() || chans.is_null() { - if !sub.is_null() { - CFRelease(sub); - } - if !chans.is_null() { - CFRelease(chans as CFTypeRef); - } - return OracleRun::bare(Oracle::NoIoReport); - } - (sub, chans) + let Some(sub) = OracleSub::open() else { + return OracleRun { + verdict: Oracle::NoIoReport, + reads: 0, + took: Duration::ZERO, + cpu_changes: Vec::new(), + gpu_changes: Vec::new(), + }; }; let stop = AtomicBool::new(false); - let run = std::thread::scope(|s| { + std::thread::scope(|s| { s.spawn(|| { let mut acc: u64 = 0; while !stop.load(Ordering::Relaxed) { - acc = acc.wrapping_add(1); + acc = std::hint::black_box(acc.wrapping_add(1)); } - std::hint::black_box(acc); }); let started = Instant::now(); - let run = 'watch: { - let Some((mut cpu_prev, mut gpu_prev)) = oracle_read(sub, chans) else { - break 'watch OracleRun::bare(Oracle::NoIoReport); + let (mut reads, mut saw_cpu_rail) = (0u32, false); + let (mut cpu_changes, mut gpu_changes) = (Vec::new(), Vec::new()); + let mut prev: Option<(Vec, Vec)> = None; + let names = |v: &[OracleChannel]| v.iter().map(|c| c.name.clone()).collect::>(); + while (started.elapsed() < window || reads < ORACLE_MIN_READS) + && started.elapsed() < window * ORACLE_WINDOW_CAP + { + let Some(chans) = sub.read() else { + continue; }; - if cpu_prev.is_empty() { - break 'watch OracleRun::bare(Oracle::NoCpuRail); - } - let mut run = OracleRun { - verdict: Oracle::CpuRailMoved(false), - reads: 1, - cpu_changes: vec![0; cpu_prev.len()], - gpu_changes: vec![0; gpu_prev.len()], - }; - while started.elapsed() < window { - let Some((cpu, gpu)) = oracle_read(sub, chans) else { - continue; - }; - run.reads += 1; - for (n, (now, before)) in run.cpu_changes.iter_mut().zip(cpu.iter().zip(&cpu_prev)) { - *n += u32::from(now != before); + reads += 1; + let (cpu, gpu): (Vec<_>, Vec<_>) = chans.into_iter().partition(|c| c.name.ends_with("CPU Energy")); + saw_cpu_rail |= !cpu.is_empty(); + match &prev { + Some((pc, pg)) if names(pc) == names(&cpu) && names(pg) == names(&gpu) => { + for (n, (now, before)) in cpu_changes.iter_mut().zip(cpu.iter().zip(pc)) { + *n += u32::from(now.raw != before.raw); + } + for (n, (now, before)) in gpu_changes.iter_mut().zip(gpu.iter().zip(pg)) { + *n += u32::from(now.raw != before.raw); + } } - for (n, (now, before)) in run.gpu_changes.iter_mut().zip(gpu.iter().zip(&gpu_prev)) { - *n += u32::from(now != before); + // The first read, or rails that differ from the last read's: a new baseline. + _ => { + cpu_changes = vec![0; cpu.len()]; + gpu_changes = vec![0; gpu.len()]; } - (cpu_prev, gpu_prev) = (cpu, gpu); } - run.verdict = Oracle::CpuRailMoved(run.cpu_changes.iter().any(|&n| n >= 2)); - run - }; + prev = Some((cpu, gpu)); + } stop.store(true, Ordering::Relaxed); - run - }); - unsafe { - CFRelease(sub); - CFRelease(chans as CFTypeRef); - } - run + let verdict = if saw_cpu_rail { + Oracle::CpuRailMoved(cpu_changes.iter().any(|&n| n >= 2)) + } else { + Oracle::NoCpuRail + }; + OracleRun { verdict, reads, took: started.elapsed(), cpu_changes, gpu_changes } + }) } // ── Against the hardware ───────────────────────────────────────────── @@ -1084,27 +1397,27 @@ mod macos_ioreport { /// refresh as the energy of CPU work, and consumers would tag it /// Measured. Asserts the meter's decision against what the rail actually /// did; every branch asserts, so there is no way to pass without - /// checking anything. + /// checking anything, and an oracle with too few reads is a failure. /// - /// The oracle gets three times the probe's budget, so a rail too slow for - /// the probe but moving within the oracle's window is the only case where - /// declining is accepted — and that fails safe: estimated, never - /// over-claimed. A single batch refresh in both windows does not rescue a - /// probe that called it live: the oracle counts it as one change. + /// The oracle gets at least three times the probe's budget, so a rail too + /// slow for the probe but moving within the oracle's window is the only + /// case where declining is accepted — and that fails safe: estimated, + /// never over-claimed. A single batch refresh in both windows does not + /// rescue a probe that called it live: the oracle counts it as one change. #[test] fn the_ioreport_counter_is_never_selected_on_a_frozen_cpu_rail() { let status = crate::ioreport_status(); let constructible = MacosIoReportCounter::try_detect().is_ok(); let selected = crate::detect().name() == "ioreport"; - let window = crate::CPU_RAIL_LIVENESS_BUDGET * 3; - let run = oracle_cpu_rail(window); + let run = oracle_cpu_rail(crate::CPU_RAIL_LIVENESS_BUDGET * 3); // One line, so a CI log says which branch below was actually asserted. eprintln!( "ioreport liveness: status={status:?} oracle={:?} selected={selected} \ - (oracle: {} reads over {window:?} with a helper thread burning CPU; \ + (oracle: {} reads over {:?} with a helper thread burning CPU; \ changes per CPU rail {:?}, per GPU rail {:?})", - run.verdict, run.reads, run.cpu_changes, run.gpu_changes + run.verdict, run.reads, run.took, run.cpu_changes, run.gpu_changes ); + run.assert_evidence(); match run.verdict { Oracle::NoIoReport => { assert!(!selected, "IOReport selected (status={status:?}) but no subscription opens"); @@ -1112,24 +1425,24 @@ mod macos_ioreport { } Oracle::NoCpuRail => { assert!(!selected, "IOReport selected (status={status:?}) but it has no CPU rail"); - assert_eq!(status, IoReportStatus::CpuRailAbsent); + assert!( + matches!(status, IoReportStatus::CpuRailAbsent | IoReportStatus::Unavailable), + "the oracle found no CPU rail, the status is {status:?}" + ); } Oracle::CpuRailMoved(false) => { - assert!( - run.reads >= 3, - "the oracle read only {} times in {window:?}: too few to see a rail change twice", - run.reads - ); assert!( !selected, "IOReport selected (status={status:?}) although its CPU rail did not keep moving \ - under {window:?} of CPU burn — it would report GPU-only energy, zero, or a batch \ - refresh as the energy of CPU work" + under {:?} of CPU burn — it would report GPU-only energy, zero, or a batch \ + refresh as the energy of CPU work", + run.took ); assert_eq!( status, IoReportStatus::CpuRailFrozen, - "the oracle did not see the CPU rail keep moving under {window:?} of CPU burn" + "the oracle did not see the CPU rail keep moving under {:?} of CPU burn", + run.took ); } Oracle::CpuRailMoved(true) => { @@ -1153,72 +1466,50 @@ mod macos_ioreport { /// on its own Energy Model subscription beside the cached status: it /// never reports Live for a rail the oracle did not see keep moving, and /// on every host both agree with the oracle about what exists. - /// - /// Also the probe's cadence: a probe that does not end Live samples for - /// at least its whole budget, and on the Energy Model group, re-sampled - /// after every ~25 ms of burn, that is ~30 intervals a second. At least - /// ten are required; a probe on the all-channel subscription saw 2-3 in - /// ~2 s on an M5 Max. - /// And it burns CPU on this thread between samples, rather than waiting: - /// at least half the probe's wall time is this thread's CPU time. #[test] fn the_liveness_probe_never_reports_a_frozen_rail_as_live() { - let t = Instant::now(); - let cpu = thread_cpu_time(); - let watch = probe(crate::CPU_RAIL_LIVENESS_BUDGET); - let spun = thread_cpu_time().saturating_sub(cpu); - let took = t.elapsed(); - let spun_pct = 100.0 * spun.as_secs_f64() / took.as_secs_f64(); - let probed = watch.as_ref().map(RailWatch::verdict); + let run = probe(crate::CPU_RAIL_LIVENESS_BUDGET); + let probed = run.as_ref().map(|r| r.watch.verdict()); let cached = crate::ioreport_status(); - let run = oracle_cpu_rail(crate::CPU_RAIL_LIVENESS_BUDGET * 3); - let intervals = watch.as_ref().map_or(0, |w| w.intervals); + let oracle = oracle_cpu_rail(crate::CPU_RAIL_LIVENESS_BUDGET * 3); + let walls: Vec = run.as_ref().map(|r| r.calls.iter().map(|c| c.wall).collect()).unwrap_or_default(); eprintln!( - "ioreport probe: probed={probed:?} cached={cached:?} took={took:?} on-cpu={spun_pct:.0}% \ - intervals={intervals} advances per CPU rail {:?}; oracle={:?} ({} reads, changes per CPU \ - rail {:?}, per GPU rail {:?})", - watch.as_ref().map(|w| w.rails.clone()).unwrap_or_default(), - run.verdict, - run.reads, - run.cpu_changes, - run.gpu_changes + "ioreport probe: probed={probed:?} cached={cached:?} intervals={} sample calls={} \ + median sample cost={:?} advances per CPU rail {:?}; oracle={:?} ({} reads over {:?}, changes \ + per CPU rail {:?}, per GPU rail {:?})", + run.as_ref().map_or(0, |r| r.watch.intervals), + walls.len(), + (!walls.is_empty()).then(|| median(&walls)), + run.as_ref().map(|r| r.watch.rails.clone()).unwrap_or_default(), + oracle.verdict, + oracle.reads, + oracle.took, + oracle.cpu_changes, + oracle.gpu_changes ); - if let Some(verdict) = probed - && verdict != Some(CpuRail::Live) - { - assert!( - intervals >= 10, - "the probe observed only {intervals} sample intervals in {took:?} (verdict {verdict:?}): \ - it must sample the Energy Model group after every ~25 ms of burn" - ); - assert!( - spun_pct >= 50.0, - "the probe's thread was on CPU for only {spun_pct:.0}% of {took:?}: it must burn CPU \ - between samples, not wait" - ); - } + oracle.assert_evidence(); if probed == Some(Some(CpuRail::Live)) || cached == IoReportStatus::Live { assert_eq!( - run.verdict, + oracle.verdict, Oracle::CpuRailMoved(true), "the probe said Live (probed={probed:?}, cached={cached:?}), the oracle did not see \ the CPU rail keep moving" ); } - match run.verdict { + match oracle.verdict { Oracle::NoIoReport => { assert!(matches!(probed, None | Some(None)), "the oracle could not open IOReport, the probe did"); assert_eq!(cached, IoReportStatus::Unavailable); } Oracle::NoCpuRail => assert!( - matches!(probed, None | Some(Some(CpuRail::Absent))) && cached == IoReportStatus::CpuRailAbsent, + matches!(probed, None | Some(None) | Some(Some(CpuRail::Absent))) + && matches!(cached, IoReportStatus::CpuRailAbsent | IoReportStatus::Unavailable), "the oracle found no CPU rail, the probe reported {probed:?} (cached={cached:?})" ), Oracle::CpuRailMoved(false) => assert!( - run.reads >= 3 && probed == Some(Some(CpuRail::Frozen)) && cached == IoReportStatus::CpuRailFrozen, - "the CPU rail did not keep moving under the oracle's burn ({} reads, three needed to see \ - two changes), so the probe must say Frozen: probed={probed:?}, cached={cached:?}", - run.reads + probed == Some(Some(CpuRail::Frozen)) && cached == IoReportStatus::CpuRailFrozen, + "the CPU rail did not keep moving under the oracle's burn, so the probe must say Frozen: \ + probed={probed:?}, cached={cached:?}" ), Oracle::CpuRailMoved(true) => assert!( matches!(probed, Some(Some(CpuRail::Live | CpuRail::Frozen))), @@ -1227,18 +1518,256 @@ mod macos_ioreport { } } - /// The probe runs once per process: however often the status is asked - /// for and the counter is detected, the real probe behind the cache ran - /// exactly once. Counted, not timed: a re-probe that ends Live is fast. + /// The probe burns CPU between samples, and exactly its tick of it, + /// pinned per tick on this thread's CPU clock so host load cannot fail + /// it: the CPU the thread consumed between the end of one sample call + /// and the start of the next must be at least half the 25 ms tick (a + /// probe that sleeps, or does nothing, between samples burns ~0) and at + /// most twice it (a slower cadence, e.g. a 100 ms tick, burns more). A + /// tick spins until the thread's CPU clock has advanced 25 ms, so a + /// loaded host stretches it in wall time only. And the tick CPU the + /// probe recorded is checked against the test's own reading of the + /// thread's CPU clock over the whole probe. + #[test] + fn the_probe_burns_its_tick_of_thread_cpu_between_samples() { + let own0 = own_thread_cpu(); + let started = Instant::now(); + let Some(run) = probe(crate::CPU_RAIL_LIVENESS_BUDGET) else { + let status = crate::ioreport_status(); + eprintln!("ioreport probe tick: no Energy Model subscription opens here (status={status:?}): no probe to pin"); + assert_eq!(status, IoReportStatus::Unavailable); + return; + }; + let own = own_thread_cpu().saturating_sub(own0); + let took = started.elapsed(); + let ticks: Vec = + run.calls.windows(2).map(|w| w[1].cpu_before.saturating_sub(w[0].cpu_after)).collect(); + let recorded: Duration = ticks.iter().sum(); + eprintln!( + "ioreport probe tick: {} ticks of this thread's CPU between {} sample calls: min {:?}, median {:?}, \ + max {:?} (specified 25 ms); the test's own reading of the thread's CPU clock over the whole \ + probe: {own:?} in {took:?} of wall time", + ticks.len(), + run.calls.len(), + ticks.iter().min(), + (!ticks.is_empty()).then(|| median(&ticks)), + ticks.iter().max() + ); + assert!(ticks.len() >= 2, "the probe made only {} sample calls", run.calls.len()); + for (i, t) in ticks.iter().enumerate() { + assert!( + *t >= TICK / 2, + "tick {i} burned only {t:?} of this thread's CPU (every tick: {ticks:?}): the probe must burn \ + CPU between samples, not wait or skip. A tick spins until the thread's CPU clock has \ + advanced 25 ms, so host load cannot shorten it; only its 500 ms wall cap can, on a host \ + that gives this thread under 5% of a core" + ); + assert!( + *t <= TICK * 2, + "tick {i} burned {t:?} of this thread's CPU (every tick: {ticks:?}): the tick is 25 ms of CPU" + ); + } + assert!( + own >= recorded, + "the probe recorded {recorded:?} of tick CPU, but this thread's CPU clock advanced only {own:?} \ + over the whole probe" + ); + } + + /// The probe samples the Energy Model group alone: the median wall + /// duration of its own sample calls, as the probe recorded them, is under + /// 50 ms. One sample of the group takes ~6-12 ms on an M5 Max and an M3 + /// Ultra; one of every channel 130-800 ms, which would leave the probe + /// sampling instead of burning. The median, not the mean or the maximum, + /// so a few samples delayed by a loaded host cannot fail it. + #[test] + fn the_probe_samples_only_the_energy_model_group() { + let Some(run) = probe(crate::CPU_RAIL_LIVENESS_BUDGET) else { + let status = crate::ioreport_status(); + eprintln!("ioreport probe sample cost: no Energy Model subscription opens here (status={status:?})"); + assert_eq!(status, IoReportStatus::Unavailable); + return; + }; + let walls: Vec = run.calls.iter().map(|c| c.wall).collect(); + let cost = median(&walls); + eprintln!( + "ioreport probe sample cost: median {cost:?} over {} sample calls (min {:?}, max {:?}); \ + the limit is 50 ms (the Energy Model group ~6-12 ms, every channel 130-800 ms)", + walls.len(), + walls.iter().min(), + walls.iter().max() + ); + assert!( + cost < Duration::from_millis(50), + "the probe's median sample call took {cost:?} (every call: {walls:?}): it must sample its own \ + subscription to the Energy Model group, not every channel" + ); + } + + /// The probe's sampler reads each CPU rail's true value. Around each of + /// three samples from the sampler the probe uses, on the probe's own + /// Energy Model subscription, the oracle reads the same rail names + /// (Energy Model group, absolute) just before and just after. A frozen + /// rail is constant, so where the oracle saw a rail hold across its two + /// reads the sampler's value must EQUAL it; where the rail moved, the + /// sampler's value must lie between the oracle's reads. The sampler must + /// also list exactly the oracle's CPU rails. A sampler that returned a + /// constant 0, a sum of rails, or another rail's value fails here. + #[test] + fn the_probe_sampler_reads_each_cpu_rail_as_the_oracle_does() { + let Some(state) = MacosIoReportCounter::open_energy_model() else { + let status = crate::ioreport_status(); + eprintln!("ioreport sampler: no Energy Model subscription opens here (status={status:?})"); + assert_eq!(status, IoReportStatus::Unavailable); + return; + }; + let oracle = OracleSub::open().expect("the probe's Energy Model subscription opened, the oracle's did not"); + let oracle_rails = || -> Vec<(String, i64)> { + oracle + .read() + .expect("an oracle read") + .into_iter() + .filter(|c| c.group == "Energy Model" && c.name.ends_with("CPU Energy")) + .map(|c| (c.name, c.raw)) + .collect() + }; + let names = |v: &[(String, i64)]| v.iter().map(|(n, _)| n.clone()).collect::>(); + let mut seen = Vec::new(); + for _ in 0..3 { + let before = oracle_rails(); + let sampled: Vec<(String, i64)> = state + .energy_model_sample() + .expect("the probe's sampler returned no sample") + .into_iter() + .filter(|(n, _)| n.ends_with("CPU Energy")) + .collect(); + let after = oracle_rails(); + assert_eq!(names(&before), names(&after), "the oracle's CPU rails changed between two reads"); + assert_eq!( + names(&sampled), + names(&before), + "the probe's sampler does not list the CPU rails the oracle reads" + ); + for ((name, got), ((_, lo), (_, hi))) in sampled.iter().zip(before.iter().zip(&after)) { + if lo == hi { + assert_eq!( + got, lo, + "{name} held at {lo} across the oracle's reads either side of the sample, and the probe's \ + sampler read {got}" + ); + } else { + assert!( + (*lo.min(hi)..=*lo.max(hi)).contains(got), + "{name} moved {lo} -> {hi} across the oracle's reads either side of the sample, and the \ + probe's sampler read {got}, outside them" + ); + } + seen.push(format!("{name}: oracle {lo}..{hi}, sampler {got}")); + } + } + eprintln!("ioreport sampler: {} CPU-rail values checked against the oracle: {seen:?}", seen.len()); + } + + /// The counter's own read path, which a declined host never runs: built + /// past the liveness gate on purpose, it must sum what its rails did. A + /// reader returning a constant (the counter open, and reading 0 µJ over + /// a whole run) can be told from a true zero only by a rail that moved, + /// and the counter sums `GPU Energy`, which moves on an M5 Max. So: take + /// the counter's baseline sample; read `GPU Energy` in the Energy Model + /// group independently (the oracle's own subscription, absolute values) + /// until it has risen by at least 1 µJ by the oracle's own unit table, or + /// 3 s pass; then read the counter again. The counter's window contains + /// the oracle's and the rail is cumulative, so if GPU Energy rose the + /// counter must have advanced. If it did not rise, nothing can tell a + /// constant from a true zero: the counter is printed, NOT asserted, and + /// the printed line says so. + #[test] + fn the_counter_advances_when_a_rail_it_sums_advanced() { + let Some(state) = MacosIoReportCounter::open() else { + let selected = crate::detect().name() == "ioreport"; + eprintln!("ioreport counter: no IOReport subscription opens here; selected={selected}"); + assert!(!selected, "detect() selected an IOReport counter that does not open"); + return; + }; + let counter = MacosIoReportCounter { state: Mutex::new(state) }; + let oracle = OracleSub::open().expect("the counter's all-channel subscription opened, the oracle's did not"); + let gpu = || { + oracle.read().map(|chans| { + chans + .into_iter() + .filter(|c| c.group == "Energy Model" && c.name.ends_with("GPU Energy")) + .collect::>() + }) + }; + let base = counter.read_microjoules().expect("the counter's baseline read"); + let g0 = gpu().expect("an independent read of GPU Energy"); + let started = Instant::now(); + let (mut rose_raw, mut rose_uj, mut unit) = (0i64, 0u64, String::new()); + while !g0.is_empty() && rose_uj == 0 && started.elapsed() < Duration::from_secs(3) { + let Some(g) = gpu() else { + continue; + }; + for (now, before) in g.iter().zip(&g0).filter(|(now, before)| now.name == before.name) { + let uj = oracle_uj(now.raw - before.raw, &now.unit); + if uj >= rose_uj { + (rose_raw, rose_uj, unit) = (now.raw - before.raw, uj, now.unit.clone()); + } + } + } + let window = started.elapsed(); + let after = counter.read_microjoules().expect("the counter's second read"); + eprintln!( + "ioreport counter: GPU Energy (independent, absolute; {} rail(s)) rose {rose_raw} {unit} = {rose_uj} µJ \ + over {window:?}; the counter, over a window containing it, read {base} -> {after} µJ", + g0.len() + ); + if rose_uj > 0 { + assert!( + after > base, + "GPU Energy rose {rose_uj} µJ inside the counter's window, but the counter read {base} -> \ + {after} µJ: it did not sum a rail it counts" + ); + } else { + eprintln!( + "ioreport counter: GPU Energy did not rise inside the counter's window, so a constant counter \ + cannot be told from a true zero: the counter is NOT asserted on this run" + ); + } + } + + /// The probe runs once per PROCESS, however many threads ask at once. + /// Six threads, released together, each ask for the status and + /// construct the backend (`try_detect()` and `detect()`). The real probe + /// behind the cache, counted by a process-wide static, not a + /// thread-local, ran exactly once, and every thread got the same answer. + /// Counted, not timed: a re-probe that ends Live is fast. #[test] fn the_liveness_probe_runs_once_per_process() { - let first = crate::ioreport_status(); - for _ in 0..3 { - assert_eq!(crate::ioreport_status(), first, "the cached status changed between calls"); - } - let _ = crate::detect(); - let _ = MacosIoReportCounter::try_detect(); + const THREADS: usize = 6; + let gate = std::sync::Barrier::new(THREADS); + let seen: Vec<(IoReportStatus, bool, &'static str)> = std::thread::scope(|s| { + let handles: Vec<_> = (0..THREADS) + .map(|_| { + s.spawn(|| { + gate.wait(); + let status = crate::ioreport_status(); + let built = MacosIoReportCounter::try_detect().is_ok(); + let picked = crate::detect().name(); + (status, built, picked) + }) + }) + .collect(); + handles.into_iter().map(|h| h.join().expect("a thread asking for the status panicked")).collect() + }); let runs = PROBE_RUNS.load(Ordering::SeqCst); + eprintln!("ioreport cache: {THREADS} threads asked at once and got {seen:?}; real probes in this process: {runs}"); + let first = seen[0].0; + for (status, built, picked) in &seen { + assert_eq!(*status, first, "the threads got different statuses: {seen:?}"); + assert_eq!(*built, *status == IoReportStatus::Live, "try_detect() disagrees with the status: {seen:?}"); + assert_eq!(*picked == "ioreport", *built, "detect() disagrees with try_detect(): {seen:?}"); + } + assert_eq!(crate::ioreport_status(), first, "the cached status changed"); assert_eq!(runs, 1, "the liveness probe behind ioreport_status() ran {runs} times in this process"); } @@ -1257,11 +1786,11 @@ mod macos_ioreport { IoReportStatus::Live, "try_detect() constructed the counter although the status is not Live" ); - let rail = probe(crate::CPU_RAIL_LIVENESS_BUDGET).and_then(|w| w.verdict()); + let rail = probe(crate::CPU_RAIL_LIVENESS_BUDGET).and_then(|r| r.watch.verdict()); assert_eq!(rail, Some(CpuRail::Live), "counter available but its CPU rail is {rail:?}"); let _ = c.read_microjoules(); // first read only sets the baseline sample let before = c.read_microjoules().unwrap(); - burn(Duration::from_millis(200)); + spin(Duration::from_millis(200)); let after = c.read_microjoules().unwrap(); assert!( after > before, @@ -1274,7 +1803,7 @@ mod macos_ioreport { assert_ne!(status, IoReportStatus::Live, "status Live but try_detect() declined"); let fallback = crate::detect(); let before = fallback.read_microjoules(); - burn(Duration::from_millis(100)); + spin(Duration::from_millis(100)); let after = fallback.read_microjoules(); eprintln!( "ioreport declined ({status:?}): fallback={} read {before:?} then {after:?} across \ @@ -1304,13 +1833,13 @@ pub use macos_ioreport::MacosIoReportCounter; /// Why the IOReport counter was or was not made available on this host. #[derive(Debug, Clone, Copy, PartialEq, Eq)] pub enum IoReportStatus { - /// No IOReport subscription could be opened (not macOS, not Apple - /// Silicon, or Rosetta), or one opened but never yielded two usable - /// samples, so nothing could be learned about the CPU rail. + /// No subscription to IOReport's Energy Model group could be opened (not + /// macOS, not Apple Silicon, Rosetta, or no such group), or one opened but + /// never yielded two usable samples, so nothing could be learned about the + /// CPU rail. Unavailable, - /// IOReport opened, but no `CPU Energy` rail was found in its Energy Model - /// group (or that group could not be subscribed) — an unfamiliar channel - /// naming. The counter is not used. + /// IOReport's Energy Model group was sampled, and no `CPU Energy` rail was + /// in it — an unfamiliar channel naming. The counter is not used. CpuRailAbsent, /// The CPU energy rail exists but did not keep advancing while CPU was /// burned: it held, or stepped in only one sample interval (a refresh). @@ -1326,10 +1855,11 @@ pub enum IoReportStatus { /// How long the liveness probe burns CPU before concluding the CPU rail is /// not live: a live rail gets a second of single-thread CPU work, sampled -/// after every ~25 ms of it, to advance in two separate sample intervals. If -/// sampling is so slow that fewer than two intervals fit in the budget, the -/// probe keeps going until two have, and never past five budgets. A rail too -/// slow for that fails safe — declined, so callers estimate; never +/// after every 25 ms of the probing thread's own CPU time (a tick that cannot +/// get it ends after 500 ms of wall time), to advance in two separate sample +/// intervals. If sampling is so slow that fewer than two intervals fit in the +/// budget, the probe keeps going until two have, and never past five budgets. +/// A rail too slow for that fails safe — declined, so callers estimate; never /// over-claimed. Paid once per process; see [`ioreport_status`]. #[cfg(target_os = "macos")] pub const CPU_RAIL_LIVENESS_BUDGET: std::time::Duration = std::time::Duration::from_millis(1000); diff --git a/eoc/eoc-rs/crates/eoc/src/main.rs b/eoc/eoc-rs/crates/eoc/src/main.rs index 926015646..0f3966308 100644 --- a/eoc/eoc-rs/crates/eoc/src/main.rs +++ b/eoc/eoc-rs/crates/eoc/src/main.rs @@ -115,7 +115,9 @@ fn ioreport_line(status: eoc_meter::IoReportStatus) -> &'static str { its readings would be GPU-only energy, zero, or a stale refresh" } IoReportStatus::CpuRailAbsent => "declined: no CPU energy rail in its Energy Model group", - IoReportStatus::Unavailable => "unavailable: IOReport could not be opened, or yielded no samples", + IoReportStatus::Unavailable => { + "unavailable: its Energy Model group could not be opened, or yielded no usable samples" + } } } -- 2.54.0 (Apple Git-157) From 7aa424188c7f36420b180cfb757df71b8ff9f02a Mon Sep 17 00:00:00 2001 From: Transaction Science Date: Wed, 30 Sep 2026 08:54:54 -0400 Subject: [PATCH 05/22] eoc-meter: the IOReport counter counts CPU energy once counts_once, the filter on what the counter sums, admitted CPU rails whose names do not say "CPU" beside the `CPU Energy` aggregate they add up to. Round 3 carried this as latent. It is now confirmed from one absolute sample of the Energy Model group on each chip: - M5 Max: `CPU Energy` = `MCPU0` + `MCPU1` + `PCPU` (1,863,990,848 vs 1,863,990,847 mJ). `PCPU` = `PACC_0..5` + `PCPU0..5_SRAM` + `PCPM` + `PCPM_SRAM`, and `MCPUn` = its cores + their SRAM + `MCPMn` + `MCPMn_SRAM`. counts_once admitted `PACC_*`, `MCPM*`, `PCPM` and their `_SRAM` rails. - M3 Ultra: `DIE_n_CPU Energy` = its `EACC_CPU` + `PACC0_CPU` + `PACC1_CPU`, each the sum of its cores, their SRAM and its `_CPM` + `_CPM_SRAM` rails (all to within 1-8 mJ). counts_once admitted `DIE_n_EACC_CPM`, `DIE_n_PACC*_CPM` and their `_SRAM` rails. On a host whose CPU rail is live, the counter would have summed that CPU energy twice and sealed the result as measured. Neither chip is live on macOS 27.0.1, so no reading so far was affected. counts_once now also drops per-cluster accumulators (`PACC*`, `EACC*`, `MACC*`) and CPU power managers (`*CPM*`). The aggregates, and the domains that are no aggregate's summand (DRAM, AMCC, DCS, display, ISP, AVE, MSR, FAB, PCIe), are still summed. counts_once_drops_subrails now pins this on the real channel names of both chips. Not changed: neither chip has an `ANE Energy` aggregate, and counts_once drops `ANE0` (and `ANE0_0`/`ANE0_1`) as summands, so ANE energy is never summed. Counting it needs a rule that looks at the whole sample, and that is left for later. Co-Authored-By: Claude Opus 5.5 --- eoc/eoc-rs/crates/eoc-meter/src/lib.rs | 55 ++++++++++++++++++++++---- 1 file changed, 48 insertions(+), 7 deletions(-) diff --git a/eoc/eoc-rs/crates/eoc-meter/src/lib.rs b/eoc/eoc-rs/crates/eoc-meter/src/lib.rs index 0fc65cee4..f65faeca9 100644 --- a/eoc/eoc-rs/crates/eoc-meter/src/lib.rs +++ b/eoc/eoc-rs/crates/eoc-meter/src/lib.rs @@ -1045,6 +1045,18 @@ mod macos_ioreport { uj as u64 } + /// Whether the counter sums this rail, so that energy is counted once. The + /// aggregates `CPU Energy`, `GPU Energy` and `ANE Energy` (optionally + /// `DIE__`-prefixed) are summed; their summands are not. A summand is any + /// other rail whose name says CPU, GPU or ANE, and also the CPU summands + /// whose names do not: per-cluster accumulators (`PACC*`, `EACC*`, `MACC*`) + /// and CPU power managers (`*CPM*`, with their `_SRAM` rails). In one + /// absolute sample on an M5 Max, `CPU Energy` = `MCPU0` + `MCPU1` + `PCPU`, + /// and `PCPU` = `PACC_0..5` + `PCPU0..5_SRAM` + `PCPM` + `PCPM_SRAM`; on an + /// M3 Ultra, each `DIE__CPU Energy` = its `EACC_CPU` + `PACC0_CPU` + + /// `PACC1_CPU`, each the sum of its cores, their SRAM and its `_CPM` rails + /// (all to within 1-8 mJ). Summing `PACC_*` or `*_CPM` beside `CPU Energy` + /// would count that CPU energy twice. fn counts_once(name: &str) -> bool { let base = rail_liveness::strip_die_prefix(name); if base.eq_ignore_ascii_case("CPU Energy") @@ -1053,11 +1065,12 @@ mod macos_ioreport { { return true; } - let l = name.to_ascii_lowercase(); - if l.contains("cpu") || l.contains("gpu") || l.contains("ane") { + let u = base.to_ascii_uppercase(); + if u.contains("CPU") || u.contains("GPU") || u.contains("ANE") { return false; } - true + let cpu_summand = ["PACC", "EACC", "MACC"].iter().any(|p| u.starts_with(p)) || u.contains("CPM"); + !cpu_summand } fn cfstring_to_string(s: CFStringRef) -> Option { @@ -1130,12 +1143,40 @@ mod macos_ioreport { assert_eq!(energy_to_uj(2, "J"), 2_000_000); } + /// The counter sums each aggregate once and none of its summands, on + /// the real Energy Model channel names of both chips. #[test] fn counts_once_drops_subrails() { - assert!(counts_once("CPU Energy")); - assert!(counts_once("DIE_0_CPU Energy")); - assert!(!counts_once("DIE_0_PACC0_CPU0")); - assert!(!counts_once("GPU0_0")); + // Aggregates, and domains that are no aggregate's summand. + for name in [ + "CPU Energy", "DIE_0_CPU Energy", "DIE_1_CPU Energy", "GPU Energy", "ANE Energy", "AFR0", "ISP0", + "AVE0", "MSR0", "AMCC0", "DCS0", "DRAM0", "DISP0", "DISPEXT0", "FAB0", "PCIe Port 0 Energy", + "apciec0 Energy", "ISP0_1", "DRAM0_0", "AMCC0_1", "DISPEXT0_1", + ] { + assert!(counts_once(name), "{name} is not summed"); + } + // An M5 Max, from one absolute sample: `CPU Energy` = `MCPU0` + + // `MCPU1` + `PCPU`; `MCPUn` = its cores + their SRAM + `MCPMn` + + // `MCPMn_SRAM`; `PCPU` = `PACC_0..5` + `PCPU0..5_SRAM` + `PCPM` + + // `PCPM_SRAM`. + for name in [ + "MCPU0", "MCPU1", "PCPU", "MCPU0_3", "MCPU1_5_SRAM", "PCPU2_SRAM", "PACC_0", "PACC_5", "MCPM0", + "MCPM1", "PCPM", "MCPM0_SRAM", "MCPM1_SRAM", "PCPM_SRAM", "MCPU0DTL0e", + ] { + assert!(!counts_once(name), "{name}, a summand of CPU Energy on an M5 Max, is summed beside it"); + } + // An M3 Ultra: `DIE_n_CPU Energy` = its `EACC_CPU` + `PACC0_CPU` + + // `PACC1_CPU`, each = its cores + their SRAM + its `_CPM` + `_CPM_SRAM`. + for name in [ + "DIE_0_EACC_CPU", "DIE_1_PACC0_CPU", "DIE_0_PACC1_CPU", "DIE_0_PACC1_CPU3", "DIE_1_EACC_CPU2_SRAM", + "DIE_0_EACC_CPM", "DIE_1_PACC0_CPM", "DIE_0_PACC1_CPM_SRAM", "DIE_1_EACC_CPM_SRAM", "ECPUDTL00", + ] { + assert!(!counts_once(name), "{name}, a summand of DIE_n_CPU Energy on an M3 Ultra, is summed beside it"); + } + // The GPU's own rails, beside the `GPU Energy` aggregate. + for name in ["GPU0", "GPU0_0", "GPU CS0_0", "GPU SRAM0_0", "GPU CS SRAM0_0"] { + assert!(!counts_once(name), "{name} is summed beside GPU Energy"); + } } // ── An independent oracle ──────────────────────────────────────────── -- 2.54.0 (Apple Git-157) From 0e18d6fe9bb1545abdb3a4d25a0f7219dcd301bc Mon Sep 17 00:00:00 2001 From: Transaction Science Date: Wed, 30 Sep 2026 09:14:20 -0400 Subject: [PATCH 06/22] eoc-meter: a tick that spins on the wrong clock fails its test, not hangs it In the_tick_spins_until_the_thread_cpu_clock_advanced, the scripted wall clock advanced only when the spin read the CPU clock. So a spin_for_cpu that measured its target on the wall clock never read the CPU clock, the scripted time never moved, and the test hung until killed (900 s in the mutation run) instead of failing. Each scripted clock now advances on its own reads. The correct spin reads each clock once per turn, so the expected values are unchanged. A spin that targets wall time now stops at 28 ms of wall time and fails the assertion that it burned 25 ms of CPU. Co-Authored-By: Claude Opus 5.5 --- eoc/eoc-rs/crates/eoc-meter/src/lib.rs | 25 ++++++++++++++++--------- 1 file changed, 16 insertions(+), 9 deletions(-) diff --git a/eoc/eoc-rs/crates/eoc-meter/src/lib.rs b/eoc/eoc-rs/crates/eoc-meter/src/lib.rs index f65faeca9..e930d673b 100644 --- a/eoc/eoc-rs/crates/eoc-meter/src/lib.rs +++ b/eoc/eoc-rs/crates/eoc-meter/src/lib.rs @@ -550,22 +550,29 @@ mod rail_liveness { assert_eq!(status_from_probe(Some(Some(CpuRail::Live))), S::Live); } - /// A clock pair for [`spin_for_cpu`] that only the spin moves: every - /// read of the CPU clock is one step, which advances this thread's CPU - /// clock by `cpu_step` and the wall clock by `wall_step`. + /// A clock pair for [`spin_for_cpu`] that only the spin moves: each read + /// of the CPU clock advances it by `cpu_step`, and each read of the wall + /// clock advances it by `wall_step`. The spin reads each once per turn, + /// so a turn is `cpu_step` of CPU in `wall_step` of wall time; and a + /// spin that ignores one clock still ends on the other. Returns the CPU + /// the spin reported and how many times it read the CPU clock. fn spin_scripted(cpu_step: Duration, wall_step: Duration, want: Duration, cap: Duration) -> (Duration, u32) { - let steps = Cell::new(0u32); + let (cpu_reads, wall_reads) = (Cell::new(0u32), Cell::new(0u32)); let used = spin_for_cpu( || { - assert!(steps.get() < 1_000_000, "the tick never stopped"); - steps.set(steps.get() + 1); - cpu_step * steps.get() + assert!(cpu_reads.get() < 1_000_000, "the tick never stopped"); + cpu_reads.set(cpu_reads.get() + 1); + cpu_step * cpu_reads.get() + }, + || { + assert!(wall_reads.get() < 1_000_000, "the tick never stopped"); + wall_reads.set(wall_reads.get() + 1); + wall_step * wall_reads.get() }, - || wall_step * steps.get(), want, cap, ); - (used, steps.get()) + (used, cpu_reads.get()) } /// The tick spins until this thread's CPU clock has advanced by the -- 2.54.0 (Apple Git-157) From 33f6d78c2c47d0068d1cbdfd809beeda757ade88 Mon Sep 17 00:00:00 2001 From: Transaction Science Date: Wed, 30 Sep 2026 16:07:38 -0400 Subject: [PATCH 07/22] eoc-meter: the tests drive the production liveness chain itself, and hold under load The round-5 review of 0e18d6fe found eight gaps. Four were on the dangerous side: - The group-only pin (median wall time per sample call < 50 ms) failed spuriously on the loaded M5 Max that runs macOS CI: 2 of 23 load runs. - The once-per-process count sat in the cache wrapper, so a constructor that probed uncached through probe() passed. An instrumented run made 21 real probes while the test printed 1. - Mapping "no Energy Model subscription" to Live or CpuRailAbsent inline in probe_status, around the pure mapping, survived on both chips. - try_detect()'s gate rewritten inline to decline only CpuRailFrozen survived on both chips. - A stale sampler survived, because only the frozen CPU rails were bracketed. - A probe_tick that spun 25 ms of wall time passed unloaded. - the_counter_advances_when_a_rail_it_sums_advanced asserted nothing when GPU Energy did not rise. - CPU_RAIL_LIVENESS_BUDGET's doc called a second of wall time "a second of single-thread CPU work". What holds now. The production chain is thin, branch-free, and the one the tests run. host_status() adds one to a process-wide AtomicUsize, then returns status_of(probe_with(real opener, real tick, thread CPU clock, wall clock, budget).verdict()). ioreport_status() caches it in a OnceLock and is its only production caller. probe_with is the only place that composes the opener, the sampler, the tick and the clocks. An opener that fails gives a run with no verdict, and the pure status_of maps no verdict to Unavailable. try_detect() is try_detect_given(ioreport_status()). try_detect_given and detect() both gate through the pure admits_counter(), which admits Live only. The real tick is one call of the pure spin_for_cpu with this thread's CPU clock and the wall clock injected. Fault injection sits inside the real leaves. In test builds, a script installed on a thread stands in for IOReport in open_energy_model() and State::group_sample(), and nowhere above them. The tests call the real host_status(), uncached, under these scripts: - the opener fails: Unavailable, with no sample taken and no tick run - a single batch jump: CpuRailFrozen - two advances: Live, at the third sample - one step on each of two dies: CpuRailFrozen - a constant rail: CpuRailFrozen, and every production tick burned 25 ms of thread CPU by the test's own clock A status installed on a thread likewise stands in for the cached one, so the real try_detect() and detect() run under all four statuses. The pins. No pin bounds wall time from above, so host load cannot fail one. - Group-only, by structure. Every sample the production probe took has exactly as many channels as the Energy Model group in the oracle's all-channel read, and every one is in the group: 364 of 11,886 channels on the M5 Max, 645 of 14,345 on the M3 Ultra. Also, the median thread CPU of a sample call must be under 50 ms. It measured 6-9 ms; every channel cost 110-120 ms in round 5. - The clock, against the tests' own CLOCK_THREAD_CPUTIME_ID literal (16). The implementation's clock keeps pace while this thread burns. It does not move across a 100 ms sleep, or while another thread burns 100 ms. Each production tick is at least 25 ms of thread CPU or 500 ms of wall time, and never more CPU than wall. This is checked once unloaded, and once beside 2 x ncpu busy threads, where at least one tick must have shared its core. - Freshness. The production probe runs, uncached and outside host_status(), between two oracle reads of the Energy Model group's CPU and GPU energy rails. The run records the (key, raw) values it handed the rule, GPU Energy included. They must carry the oracle's keys, EQUAL a rail that held, and lie within a rail that moved. When the oracle saw GPU Energy change, the record must show it changing too. - Once per process. tests/liveness_probe_once_per_process.rs runs in its own process. It releases 8 threads at a cold cache through status, construction and selection, in three orders. Two counts must each be exactly 1: runs of host_status(), and opens of the probe's sampler. The second count catches a probe run outside the cache that never touched host_status(). - Declines before anything opens. Given Unavailable, CpuRailAbsent or CpuRailFrozen, try_detect_given returns Err(NoCounter), and detect_with, try_detect() and detect() decline too. None of them attempts to open the counter's subscription, by a per-thread count inside its opener. Given Live, the attempt is counted. This test runs on every macOS host, whether or not IOReport opens there. - The counter's value. The counter is built past the gate on purpose, and its delta over a window of at least 3 s must lie between two rises of the oracle's sum of the same summed rails: the rise inside the window, and the rise around it. The oracle rounds rail by rail, as the counter does. 0 == 0 is asserted too; no branch skips. The budget doc now says the second is wall time. A loaded host gives the probe fewer intervals, which fails safe. Evidence. eoc-meter: 37 lib tests + the once-per-process test pass. - M5 Max, unloaded: per-tick thread CPU 25.02-25.03 ms. Median thread CPU per sample call 6.5 ms. `ioreport liveness: status=CpuRailFrozen oracle=CpuRailMoved(false) selected=false`. The counter read 136,442 uJ against a bracket of 126,920..162,549 uJ; GPU Energy was the only rail that moved. - M5 Max, 5 runs with a helper process sampling every IOReport channel in a tight loop: 5/5 ok, 37 passed each. - M5 Max, 5 runs beside 1 x ncpu busy spinners (load average 23-39): 5/5 ok, 37 passed each. The probe saw 4-11 intervals. - M3 Ultra (10.10.10.2, macOS 27.0.1), 1 run: 37 passed, and the once-per-process test passed with 1 run and 1 open. Both DIE_n_CPU Energy rails were read equal to the oracle. 12 of 17 production ticks shared their core with 64 busy threads, each still burning 25.00-25.01 ms of CPU. - The whole eoc workspace: 613 passed, 0 failed, 3 ignored. The ignored three are pre-existing ignored doctests. - clippy --no-deps on eoc-meter, all targets: the only two warnings are from 1bae1ea1 (a duplicated #[link] attribute, and a b"...\0" literal). The aarch64-linux cross type-check and clippy are clean. Mutations. Each was a temporary edit, then git checkout; the tree was clean after every one. All 24 were caught (test that failed in brackets): - compare with the first sample [9 pure rule tests + host_status] - decrease counts as an advance [a_decrease_is_not_an_advance] - no subscription -> Live, and -> CpuRailAbsent, inside host_status [host_status_is_the_rule_on_the_production_composition] - single advance -> Live, inside host_status and inside the production sampler closure [the same] - no-op tick; tick spinning 25 ms of wall time; tick timed on the wall clock [the production-tick and probe-tick tests] - per-thread cache [once-per-process: "ran 8 times"] - try_detect probing uncached, through host_status and through production_probe [once-per-process: 9 runs / "sampler was opened 9 times"; only_a_live_status_constructs_or_selects_the_counter] - try_detect's gate inlined to decline only CpuRailFrozen, and try_detect_given likewise [a_declined_status_declines_before_anything_opens] - try_detect_given opening before it declines [the same: "left: 4, right: 0"] - sampler constant 0, in the FFI extraction and in the production closure [sampler/freshness tests, host_status] - stale sampler, in the production closure and cached per thread below the script hook [freshness: "every one of the 39 samples ... read [126274575439834]: its sampler is not sampling"] - thread clock = CLOCK_MONOTONIC and = CLOCK_PROCESS_CPUTIME_ID [the_probe_thread_clock_is_this_threads_cpu_time, and others] - counts_once re-admitting PACC_* [counts_once_drops_subrails] - the counter's sum skipping GPU Energy, or counting every rail twice [the_counter_advances_when_a_rail_it_sums_advanced] Not done here, deliberately: - A sampler stale BELOW the script hook is caught only through GPU Energy moving during the 1 s probe. On a host whose GPU rail holds, the freshness line says "NOT asserted" for it. Then a stale value is also the true value, and the held-rail equality still asserts. Staleness at or above the hook is caught by the scripted tests on any host. - The crate was not rustfmt-clean before this branch, and it is not reformatted here, so the diff stays reviewable. - Linux: a cross type-check and clippy only, not a run. Co-Authored-By: Claude Opus 5.5 --- eoc/eoc-rs/crates/eoc-meter/src/lib.rs | 1474 ++++++++++++++--- .../tests/liveness_probe_once_per_process.rs | 93 ++ 2 files changed, 1291 insertions(+), 276 deletions(-) create mode 100644 eoc/eoc-rs/crates/eoc-meter/tests/liveness_probe_once_per_process.rs diff --git a/eoc/eoc-rs/crates/eoc-meter/src/lib.rs b/eoc/eoc-rs/crates/eoc-meter/src/lib.rs index e930d673b..835c5a020 100644 --- a/eoc/eoc-rs/crates/eoc-meter/src/lib.rs +++ b/eoc/eoc-rs/crates/eoc-meter/src/lib.rs @@ -120,6 +120,15 @@ mod rail_liveness { strip_die_prefix(name).eq_ignore_ascii_case("CPU Energy") } + /// A GPU energy aggregate: `GPU Energy`, optionally `DIE__`-prefixed, + /// compared case-insensitively. Never watched. A probe run records it + /// beside the CPU rails because it is the one Energy Model rail that moves + /// on hosts whose CPU rails hold (the M5 Max and the M3 Ultra on macOS + /// 27.0.1), so a sampler that stopped sampling shows there. + pub(crate) fn is_gpu_energy_rail(name: &str) -> bool { + strip_die_prefix(name).eq_ignore_ascii_case("GPU Energy") + } + /// One CPU rail's identity within a sample: its channel name and its /// ordinal among the channels of that name in the same sample. An M3 /// Ultra's Energy Model group lists 264 of its 645 channel names twice @@ -130,9 +139,16 @@ mod rail_liveness { /// The CPU rails in one sample, keyed by [`RailKey`], in sample order. fn cpu_rails(sample: &[(String, i64)]) -> Vec<(RailKey, i64)> { + keyed(sample, is_cpu_energy_rail) + } + + /// The channels of one sample whose names pass `keep`, keyed by + /// [`RailKey`] (the ordinal counts same-named channels among those kept), + /// in sample order. + fn keyed(sample: &[(String, i64)], keep: impl Fn(&str) -> bool) -> Vec<(RailKey, i64)> { let mut rails: Vec<(RailKey, i64)> = Vec::new(); for (name, raw) in sample { - if !is_cpu_energy_rail(name) { + if !keep(name) { continue; } let ordinal = rails.iter().filter(|((n, _), _)| n == name).count() as u32; @@ -218,8 +234,8 @@ mod rail_liveness { /// samples, and the clock, so it runs under test exactly as it runs on /// hardware. Takes a sample, then ticks and samples again for as long as /// [`RailWatch::wants_more`] says. `sample` returns `None` when no sample - /// could be taken. The real probe passes an IOReport sampler, ~25 ms of CPU - /// spin on the calling thread, and [`std::time::Instant`]. + /// could be taken. [`probe_with`] runs it with the sampler, tick and wall + /// clock it is given, and records each sample call. pub(crate) fn watch( mut sample: impl FnMut() -> Option>, mut tick: impl FnMut(), @@ -269,9 +285,110 @@ mod rail_liveness { } } - /// The status a probe verdict gives this host. No verdict — not one - /// interval could be observed — is `Unavailable`: nothing was learned about - /// the rail, and only `Live` makes the counter available. + /// One sample, as a sampler returns it: the shape of the IOReport sample it + /// read, and that sample's Energy Model channels. + #[derive(Debug, Clone, PartialEq, Eq)] + pub(crate) struct Sample { + /// Channels in the IOReport sample, in any group. + pub(crate) channels: usize, + /// Whether every one of those channels is in the Energy Model group. + pub(crate) group_only: bool, + /// Each Energy Model channel's (name, absolute raw integer), in sample + /// order: what the probe hands the rule. + pub(crate) values: Vec<(String, i64)>, + } + + /// One call the probe made to its sampler, as the probe itself recorded it, + /// so a test can pin what the probe did without timing it from outside. + #[derive(Debug, Clone)] + #[cfg_attr(not(test), allow(dead_code))] // read only by the tests + pub(crate) struct SampleCall { + /// Wall time of the call. Recorded, never asserted: on a shared host + /// another process's all-channel sample can hold IOReport for hundreds + /// of milliseconds, and this call waits for it. + pub(crate) wall: Duration, + /// The probing thread's CPU clock just before and just after the call. + pub(crate) cpu_before: Duration, + pub(crate) cpu_after: Duration, + /// For a sample that was taken: how many channels it held, and whether + /// every one is in the Energy Model group. `None` for no sample. + pub(crate) shape: Option<(usize, bool)>, + /// For a sample that was taken: every CPU rail and GPU Energy aggregate + /// in what was handed to the rule, as ([`RailKey`], raw), in sample + /// order. `None` for no sample. + pub(crate) handed: Option>, + } + + /// What one liveness probe did and saw. + #[derive(Debug)] + #[cfg_attr(not(test), allow(dead_code))] // all but `watch` read only by the tests + pub(crate) struct ProbeRun { + /// Whether the opener returned a sampler. + pub(crate) opened: bool, + /// Ticks run between samples. + pub(crate) ticks: u32, + /// Every sample call, in order. + pub(crate) calls: Vec, + /// The rule, folded over every sample that was taken. + pub(crate) watch: RailWatch, + } + + impl ProbeRun { + /// The rule's verdict: `None` when not one interval was observed, and + /// so also when the opener returned no sampler. + pub(crate) fn verdict(&self) -> Option { + self.watch.verdict() + } + } + + /// The liveness probe, generic over its real-world inputs so that tests + /// drive the very composition the production status runs: `open` makes the + /// sampler (`None`: no subscription, so no sample and no tick), `tick` runs + /// between samples, and `thread_cpu` and `wall` are the clocks. Samples by + /// the rule in [`watch`] for `budget`, and records every sample call. + pub(crate) fn probe_with( + open: impl FnOnce() -> Option, + mut tick: impl FnMut(), + thread_cpu: impl Fn() -> Duration, + wall: impl Fn() -> Duration, + budget: Duration, + ) -> ProbeRun + where + S: FnMut() -> Option, + { + let Some(mut sampler) = open() else { + return ProbeRun { opened: false, ticks: 0, calls: Vec::new(), watch: RailWatch::default() }; + }; + let (mut calls, mut ticks) = (Vec::new(), 0u32); + let watch = watch( + || { + let (cpu_before, started) = (thread_cpu(), wall()); + let sample = sampler(); + let (ended, cpu_after) = (wall(), thread_cpu()); + calls.push(SampleCall { + wall: ended.saturating_sub(started), + cpu_before, + cpu_after, + shape: sample.as_ref().map(|s| (s.channels, s.group_only)), + handed: sample.as_ref().map(|s| keyed(&s.values, |n| is_cpu_energy_rail(n) || is_gpu_energy_rail(n))), + }); + sample.map(|s| s.values) + }, + || { + ticks += 1; + tick(); + }, + &wall, + budget, + ); + ProbeRun { opened: true, ticks, calls, watch } + } + + /// The status a probe verdict gives this host. No verdict — no + /// subscription, or not one interval observed — is `Unavailable`: nothing + /// was learned about the rail, and only `Live` makes the counter available. + /// Never `CpuRailAbsent`, which says samples were taken and no CPU rail was + /// in them. pub(crate) fn status_of(verdict: Option) -> crate::IoReportStatus { match verdict { None => crate::IoReportStatus::Unavailable, @@ -281,22 +398,10 @@ mod rail_liveness { } } - /// The status one probe gives this host. `probe` is `None` when the Energy - /// Model group subscription could not be opened, and otherwise the probe's - /// verdict, itself `None` when not one interval could be observed. Neither - /// taught anything about the CPU rail, so both are `Unavailable`: never - /// `Live`, and never `CpuRailAbsent`, which says samples were taken and no - /// CPU rail was in them. - pub(crate) fn status_from_probe(probe: Option>) -> crate::IoReportStatus { - match probe { - None => crate::IoReportStatus::Unavailable, - Some(verdict) => status_of(verdict), - } - } - /// Whether a host with this status may use the IOReport counter: only when /// its CPU rail is live. Absent, frozen or unavailable all decline, so /// callers estimate rather than seal a reading that did not track the work. + /// The one gate every constructor and selection path passes through. pub(crate) fn admits_counter(status: crate::IoReportStatus) -> bool { status == crate::IoReportStatus::Live } @@ -532,22 +637,144 @@ mod rail_liveness { assert!(!admits_counter(status), "no verdict admitted the counter"); } - /// When the Energy Model group subscription cannot be opened there is - /// no probe; when it opens but never yields two usable samples there is - /// no verdict. Both are `Unavailable` — nothing was learned about the - /// CPU rail — never `Live`, and never `CpuRailAbsent`. + /// A sample the way the FFI returns one of the Energy Model group: + /// every channel in the group. + fn group_sample(values: &[(&str, i64)]) -> Sample { + Sample { + channels: values.len(), + group_only: true, + values: values.iter().map(|(n, raw)| (n.to_string(), *raw)).collect(), + } + } + + /// What a fake run of [`probe_with`] did: the run, and how many times + /// the fake opener, sampler and tick were called. + struct Fake { + run: ProbeRun, + opens: u32, + samples: usize, + ticks: u32, + } + + /// [`probe_with`] as the production status composes it (an opener, a + /// tick, a thread CPU clock, a wall clock, the budget), with fakes for + /// every real-world input: the opener opens or not; `generate(i)` is + /// the i-th sample; the wall clock moves only on a tick, by `step`, and + /// the thread CPU clock by 25 ms per tick and 1 ms per sample call. + fn fake(opens: bool, step: Duration, mut generate: impl FnMut(usize) -> Option) -> Fake { + let (wall, cpu) = (Cell::new(Duration::ZERO), Cell::new(Duration::ZERO)); + let (opened, ticked) = (Cell::new(0u32), Cell::new(0u32)); + let mut taken = 0usize; + let sampler = || { + assert!(taken < 100_000, "the probe never stopped sampling"); + let s = generate(taken); + taken += 1; + cpu.set(cpu.get() + Duration::from_millis(1)); + s + }; + let run = probe_with( + || { + opened.set(opened.get() + 1); + opens.then_some(sampler) + }, + || { + ticked.set(ticked.get() + 1); + wall.set(wall.get() + step); + cpu.set(cpu.get() + TICK); + }, + || cpu.get(), + || wall.get(), + BUDGET, + ); + Fake { run, opens: opened.get(), samples: taken, ticks: ticked.get() } + } + + /// No subscription: the opener is asked once, returns no sampler, and + /// the probe takes no sample and runs no tick. No verdict: + /// `Unavailable` — never `Live`, and never `CpuRailAbsent`. A + /// subscription that never yields two usable samples is `Unavailable` + /// too: nothing was learned about the CPU rail. #[test] fn no_energy_model_subscription_is_unavailable() { use crate::IoReportStatus as S; - assert_eq!(status_from_probe(None), S::Unavailable, "no group subscription"); - let (never, _) = scripted(TICK, |_| None); - assert_eq!(status_from_probe(Some(never.verdict())), S::Unavailable, "a subscription with no sample"); - let (once, _) = scripted(TICK, |i| (i == 0).then(|| vec![("CPU Energy", 7)])); - assert_eq!(status_from_probe(Some(once.verdict())), S::Unavailable, "a subscription with one sample"); - // A probe that did sample still maps through its verdict. - assert_eq!(status_from_probe(Some(Some(CpuRail::Absent))), S::CpuRailAbsent); - assert_eq!(status_from_probe(Some(Some(CpuRail::Frozen))), S::CpuRailFrozen); - assert_eq!(status_from_probe(Some(Some(CpuRail::Live))), S::Live); + let f = fake(false, TICK, |_| panic!("sampled without a subscription")); + assert_eq!((f.opens, f.samples, f.ticks), (1, 0, 0), "no subscription: no sample, no tick"); + assert!(!f.run.opened && f.run.calls.is_empty() && f.run.ticks == 0, "{:?}", f.run); + assert_eq!(f.run.verdict(), None); + assert_eq!(status_of(f.run.verdict()), S::Unavailable, "no group subscription"); + let never = fake(true, TICK, |_| None); + assert_eq!(status_of(never.run.verdict()), S::Unavailable, "a subscription with no sample"); + assert!(never.run.calls.iter().all(|c| c.shape.is_none() && c.handed.is_none()), "a failed sample recorded as taken"); + let once = fake(true, TICK, |i| (i == 0).then(|| group_sample(&[("CPU Energy", 7)]))); + assert_eq!(status_of(once.run.verdict()), S::Unavailable, "a subscription with one sample"); + } + + /// The M5 Max refresh through the whole composition: `CPU Energy` + /// steps once, `GPU Energy` moves at every sample. Frozen. Every tick + /// runs between two samples, and the run records, for every sample, its + /// shape, the thread CPU clock it was given, and exactly the CPU and GPU + /// rails it handed the rule. + #[test] + fn probe_with_a_single_batch_jump_is_frozen() { + let sample = |i: usize| { + group_sample(&[ + ("CPU Energy", if i < 3 { 480_083_160 } else { 541_893_638 }), + ("PCPU", 7 * i as i64), + ("GPU Energy", 1_000 + 7 * i as i64), + ]) + }; + let f = fake(true, TICK, |i| Some(sample(i))); + assert_eq!(f.run.verdict(), Some(CpuRail::Frozen), "one refresh counted as a live rail: {:?}", f.run.watch); + assert_eq!(status_of(f.run.verdict()), crate::IoReportStatus::CpuRailFrozen); + assert_eq!((f.samples, f.ticks, f.run.ticks as usize, f.run.calls.len()), (41, 40, 40, 41)); + for (i, call) in f.run.calls.iter().enumerate() { + assert_eq!(call.shape, Some((3, true)), "sample {i}"); + let want = vec![ + (key("CPU Energy", 0), if i < 3 { 480_083_160 } else { 541_893_638 }), + (key("GPU Energy", 0), 1_000 + 7 * i as i64), + ]; + assert_eq!(call.handed.as_ref(), Some(&want), "sample {i}: the record is not what the rule was handed"); + // The injected clocks: the fake thread clock moves 1 ms in a + // sample call, the fake wall clock not at all. + assert_eq!(call.cpu_after - call.cpu_before, Duration::from_millis(1), "sample {i}"); + assert_eq!(call.wall, Duration::ZERO, "sample {i}"); + } + // Between one call's end and the next one's start, the fake thread + // clock moved by exactly the tick. + for w in f.run.calls.windows(2) { + assert_eq!(w[1].cpu_before - w[0].cpu_after, TICK); + } + } + + /// Two advances of the same CPU rail, in two intervals: Live, and the + /// probe stops right there. + #[test] + fn probe_with_two_advances_is_live() { + let f = fake(true, TICK, |i| Some(group_sample(&[("GPU Energy", 5), ("CPU Energy", 1_000 + 13 * i as i64)]))); + assert_eq!(f.run.verdict(), Some(CpuRail::Live)); + assert_eq!(status_of(f.run.verdict()), crate::IoReportStatus::Live); + assert_eq!((f.samples, f.ticks), (3, 2), "a live rail must stop the probe at its second advance"); + } + + /// Two dies, each stepping once, in different intervals: Frozen, and + /// each die's own value is what the rule was handed. A transform that + /// gave each CPU rail the sum of the CPU rails would step both "rails" + /// in both intervals and call them live. + #[test] + fn probe_with_one_step_on_each_die_in_different_intervals_is_frozen() { + let die = |i: usize| { + [ + ("DIE_0_CPU Energy", if i >= 3 { 9_500_000 } else { 9_494_336 }), + ("DIE_1_CPU Energy", if i >= 6 { 10_300_000 } else { 10_212_656 }), + ] + }; + let f = fake(true, TICK, |i| Some(group_sample(&die(i)))); + assert_eq!(f.run.verdict(), Some(CpuRail::Frozen), "one step per die counted as a live rail: {:?}", f.run.watch); + assert_eq!(f.run.watch.rails, vec![(key("DIE_0_CPU Energy", 0), 1), (key("DIE_1_CPU Energy", 0), 1)]); + for (i, call) in f.run.calls.iter().enumerate() { + let [(n0, v0), (n1, v1)] = die(i); + assert_eq!(call.handed, Some(vec![(key(n0, 0), v0), (key(n1, 0), v1)]), "sample {i}"); + } } /// A clock pair for [`spin_for_cpu`] that only the spin moves: each read @@ -632,10 +859,10 @@ mod rail_liveness { #[allow(non_camel_case_types, unsafe_code)] mod macos_ioreport { use super::*; - use crate::rail_liveness::{self, RailWatch}; + use crate::rail_liveness::{self, ProbeRun, Sample}; use std::os::raw::{c_int, c_long, c_uint, c_void}; - use std::sync::Mutex; use std::sync::atomic::{AtomicUsize, Ordering}; + use std::sync::{Mutex, OnceLock}; use std::time::{Duration, Instant}; type CFTypeRef = *const c_void; @@ -715,6 +942,12 @@ mod macos_ioreport { Duration::new(ts.tv_sec.max(0) as u64, ts.tv_nsec.clamp(0, 999_999_999) as u32) } + /// Wall time on the monotonic clock, from a fixed origin in this process. + fn wall_now() -> Duration { + static ORIGIN: OnceLock = OnceLock::new(); + ORIGIN.get_or_init(Instant::now).elapsed() + } + #[link(name = "IOReport", kind = "dylib")] #[link(name = "IOKit", kind = "framework")] #[link(name = "CoreFoundation", kind = "framework")] @@ -790,8 +1023,15 @@ mod macos_ioreport { /// stale refresh as the energy of CPU work. [`crate::ioreport_status`] /// says which. pub fn try_detect() -> Result { - // Opening a subscription is not enough: the CPU rail must be live. - if !rail_liveness::admits_counter(crate::ioreport_status()) { + Self::try_detect_given(crate::ioreport_status()) + } + + /// The constructor behind [`Self::try_detect`], given this host's + /// status: every status the pure gate does not admit declines before + /// anything opens. Opening a subscription is not enough; the CPU rail + /// must be live. + pub(crate) fn try_detect_given(status: crate::IoReportStatus) -> Result { + if !rail_liveness::admits_counter(status) { return Err(Error::NoCounter); } match Self::open() { @@ -802,6 +1042,10 @@ mod macos_ioreport { /// The counter's own subscription: every channel. fn open() -> Option { + // Test builds only: count the attempt on this thread, so a test can + // pin that a declined status never gets this far. + #[cfg(test)] + script::note_counter_open(); unsafe { let all = IOReportCopyAllChannels(0, 0); if all.is_null() { @@ -820,6 +1064,12 @@ mod macos_ioreport { /// inside IOReport instead of burning CPU. On an M5 Max a probe on /// every channel saw 2-3 intervals in ~2 s; on this group, 25-31 in one. fn open_energy_model() -> Option { + // Test builds only: a script installed on this thread stands in for + // IOReport here and in `State::group_sample`, and nowhere else. + #[cfg(test)] + if let Some(opens) = script::opens() { + return opens.then(State::detached); + } unsafe { let group = CFStringCreateWithCString( std::ptr::null_mut(), @@ -840,26 +1090,47 @@ mod macos_ioreport { } impl State { - /// One sample of this subscription, as every Energy Model channel's - /// (name, absolute raw integer), in sample order. `None` when IOReport - /// returns no sample, or a sample with no channel list. This only - /// extracts; the liveness rule lives in [`rail_liveness`]. - fn energy_model_sample(&self) -> Option> { + /// A state that holds no subscription (its drop releases nothing), for + /// a script to sample in place of IOReport. + #[cfg(test)] + fn detached() -> State { + State { + sub: std::ptr::null(), + channels: std::ptr::null_mut(), + prior_sample: None, + cumulative_uj: 0, + } + } + + /// One sample of this subscription: how many channels it held, whether + /// every one is in the Energy Model group, and each Energy Model + /// channel's (name, absolute raw integer), in sample order. `None` when + /// IOReport returns no sample, or a sample with no channel list. This + /// only extracts; the liveness rule lives in [`rail_liveness`]. + fn group_sample(&self) -> Option { + // Test builds only: a script installed on this thread stands in for + // IOReport here and in `open_energy_model`, and nowhere else. + #[cfg(test)] + if let Some(sample) = script::next() { + return sample; + } unsafe { let smp = IOReportCreateSamples(self.sub, self.channels, std::ptr::null()); if smp.is_null() { return None; } - let channels = energy_model_channels(smp); + let sample = energy_model_channels(smp); CFRelease(smp); - channels + sample } } } - /// In a sample (absolute, not a delta): each Energy Model channel's name and - /// raw integer value. `None` when the sample carries no channel list. - fn energy_model_channels(sample: CFDictionaryRef) -> Option> { + /// In a sample (absolute, not a delta): its channel count, whether every + /// channel is in the Energy Model group (by `IOReportChannelGetGroup`), and + /// each Energy Model channel's name and raw integer value. `None` when the + /// sample carries no channel list. + fn energy_model_channels(sample: CFDictionaryRef) -> Option { unsafe { let key = CFStringCreateWithCString( std::ptr::null_mut(), @@ -874,18 +1145,20 @@ mod macos_ioreport { if arr.is_null() { return None; } - let mut out = Vec::new(); - for i in 0..CFArrayGetCount(arr) { + let count = CFArrayGetCount(arr); + let (mut group_only, mut values) = (true, Vec::new()); + for i in 0..count { let item = CFArrayGetValueAtIndex(arr, i); if item.is_null() || !cfstring_eq_ic(IOReportChannelGetGroup(item), "energy model") { + group_only = false; continue; } let Some(name) = cfstring_to_string(IOReportChannelGetChannelName(item)) else { continue; }; - out.push((name, IOReportSimpleGetIntegerValue(item, 0) as i64)); + values.push((name, IOReportSimpleGetIntegerValue(item, 0) as i64)); } - Some(out) + Some(Sample { channels: count.max(0) as usize, group_only, values }) } } @@ -901,66 +1174,141 @@ mod macos_ioreport { /// One tick of the probe: spin until this thread's CPU clock has advanced /// [`PROBE_TICK`], or [`PROBE_TICK_WALL_CAP`] of wall time has passed. fn probe_tick() { - let started = Instant::now(); - rail_liveness::spin_for_cpu(thread_cpu_time, || started.elapsed(), PROBE_TICK, PROBE_TICK_WALL_CAP); + rail_liveness::spin_for_cpu(thread_cpu_time, wall_now, PROBE_TICK, PROBE_TICK_WALL_CAP); } - /// How many times [`probe_status`] — the probe behind the cached + /// How many times [`host_status`] — the probe behind the cached /// [`crate::ioreport_status`] — has run in this process, on any thread. - /// Read by a test that pins the cache: it must be exactly one. - pub(crate) static PROBE_RUNS: AtomicUsize = AtomicUsize::new(0); + pub(crate) static HOST_STATUS_RUNS: AtomicUsize = AtomicUsize::new(0); - /// One sample call the probe made: its wall-clock duration, and this - /// thread's CPU clock just before and just after it. Kept so a test can pin - /// what the probe itself did — what each sample cost, and the CPU burned - /// between samples — without timing the probe from outside. - #[derive(Debug, Clone, Copy)] - #[cfg_attr(not(test), allow(dead_code))] // read only by the tests - pub(crate) struct SampleCall { - pub(crate) wall: Duration, - pub(crate) cpu_before: Duration, - pub(crate) cpu_after: Duration, - } + /// How many times this process has opened the liveness probe's sampler, + /// on any thread: every real probe opens one, whoever calls it. + pub(crate) static SAMPLER_OPENS: AtomicUsize = AtomicUsize::new(0); - /// What one liveness probe saw, and every sample call it made, in order. - #[derive(Debug)] - #[cfg_attr(not(test), allow(dead_code))] // `calls` is read only by the tests - pub(crate) struct ProbeRun { - pub(crate) watch: RailWatch, - pub(crate) calls: Vec, - } - - /// One liveness probe on a fresh Energy Model subscription: sample, burn - /// [`PROBE_TICK`] of this thread's CPU, sample again, by the rule in - /// [`rail_liveness::watch`]. `None` when the subscription does not open. The - /// probe takes its own samples, so no counter's cumulative reading is - /// touched. - pub(crate) fn probe(budget: Duration) -> Option { + /// The real opener: a fresh subscription to the Energy Model group, and the + /// sampler that reads it. The probe takes its own samples, so no counter's + /// cumulative reading is touched; the subscription is dropped with the + /// sampler, when the probe ends. + fn open_group_sampler() -> Option Option> { + SAMPLER_OPENS.fetch_add(1, Ordering::SeqCst); let state = MacosIoReportCounter::open_energy_model()?; - let mut calls = Vec::new(); - let t0 = Instant::now(); - let watch = rail_liveness::watch( - || { - let cpu_before = thread_cpu_time(); - let started = Instant::now(); - let sample = state.energy_model_sample(); - calls.push(SampleCall { wall: started.elapsed(), cpu_before, cpu_after: thread_cpu_time() }); - sample - }, - probe_tick, - || t0.elapsed(), - budget, - ); - Some(ProbeRun { watch, calls }) + Some(move || state.group_sample()) } - /// This host's status from one probe, through the pure - /// [`rail_liveness::status_from_probe`]: a group subscription that does not - /// open is `Unavailable`. Backs the cached [`crate::ioreport_status`]; the - /// probe's subscription is dropped after. - pub(crate) fn probe_status(budget: Duration) -> crate::IoReportStatus { - PROBE_RUNS.fetch_add(1, Ordering::SeqCst); - rail_liveness::status_from_probe(probe(budget).map(|run| run.watch.verdict())) + /// The production probe: [`rail_liveness::probe_with`] on the real opener, + /// the real tick, this thread's CPU clock and the monotonic wall clock, for + /// [`crate::CPU_RAIL_LIVENESS_BUDGET`]. The only production caller of + /// `probe_with`, called in production only by [`host_status`]; the + /// hardware tests run it too, uncounted and uncached, to pin what the + /// production composition hands the rule. + pub(crate) fn production_probe() -> ProbeRun { + rail_liveness::probe_with(open_group_sampler, probe_tick, thread_cpu_time, wall_now, crate::CPU_RAIL_LIVENESS_BUDGET) + } + + /// This host's status: the pure mapping of the production probe's verdict. + /// Counts its own runs first; cached, once per process, by + /// [`crate::ioreport_status`], its only production caller. + pub(crate) fn host_status() -> crate::IoReportStatus { + HOST_STATUS_RUNS.fetch_add(1, Ordering::SeqCst); + rail_liveness::status_of(production_probe().verdict()) + } + + /// Test support: a scripted sample stream that stands in for IOReport in + /// the probe's two FFI leaves — opening the Energy Model group + /// subscription, and taking one sample of it — on the thread that installs + /// it. Everything above those leaves runs as in production: the counted + /// [`host_status`], [`production_probe`] with the real tick and clocks, the + /// real opener's sampler closure, the rule and the status mapping. So a + /// test can run the production status itself with no subscription, a + /// batch refresh, or a live rail, none of which these hosts produce on + /// demand. Likewise a status installed on a thread stands in for the + /// cached one in [`crate::ioreport_status`], so the real constructor and + /// selection run under every status; and each thread counts its attempts + /// to open the counter's own all-channel subscription. + #[cfg(test)] + pub(crate) mod script { + use super::Sample; + use std::cell::RefCell; + + /// What the leaves return while a script is installed. + pub(crate) struct Script { + /// Whether the group subscription opens. + pub(crate) opens: bool, + /// The i-th sample; `None` for one that cannot be taken. + pub(crate) samples: Box Option>, + } + + thread_local! { + static SCRIPT: RefCell> = const { RefCell::new(None) }; + static STATUS: std::cell::Cell> = const { std::cell::Cell::new(None) }; + static COUNTER_OPENS: std::cell::Cell = const { std::cell::Cell::new(0) }; + } + + /// Called by the counter's own opener on every attempt to open its + /// all-channel subscription. + pub(crate) fn note_counter_open() { + COUNTER_OPENS.with(|c| c.set(c.get() + 1)); + } + + /// How many times this thread has tried to open the counter's + /// all-channel subscription. Per thread, so tests running beside it + /// that open one do not move it. + pub(crate) fn counter_opens() -> usize { + COUNTER_OPENS.with(|c| c.get()) + } + + /// `Some(status)` while a test has installed a status for + /// [`crate::ioreport_status`] to return on this thread. + pub(crate) fn status() -> Option { + STATUS.with(|s| s.get()) + } + + /// Run `f` on this thread with [`crate::ioreport_status`] returning + /// `status`, so the real `try_detect()` and `detect()` can be run + /// under every status. Removed even if `f` panics. + pub(crate) fn with_status(status: crate::IoReportStatus, f: impl FnOnce() -> R) -> R { + struct Remove; + impl Drop for Remove { + fn drop(&mut self) { + STATUS.with(|s| s.set(None)); + } + } + STATUS.with(|s| s.set(Some(status))); + let _remove = Remove; + f() + } + + /// `Some(whether the subscription opens)` while a script is installed. + pub(crate) fn opens() -> Option { + SCRIPT.with(|s| s.borrow().as_ref().map(|(script, _)| script.opens)) + } + + /// `Some(the next scripted sample)` while a script is installed. + pub(crate) fn next() -> Option> { + SCRIPT.with(|s| { + s.borrow_mut().as_mut().map(|(script, served)| { + *served += 1; + (script.samples)(*served - 1) + }) + }) + } + + /// Run `f` on this thread with `script` installed; returns what `f` + /// returned and how many samples the script served. The script is + /// removed even if `f` panics. + pub(crate) fn with(script: Script, f: impl FnOnce() -> R) -> (R, usize) { + struct Remove; + impl Drop for Remove { + fn drop(&mut self) { + SCRIPT.with(|s| s.borrow_mut().take()); + } + } + SCRIPT.with(|s| *s.borrow_mut() = Some((script, 0))); + let _remove = Remove; + let out = f(); + let served = SCRIPT.with(|s| s.borrow().as_ref().map_or(0, |(_, served)| *served)); + (out, served) + } } impl JouleCounter for MacosIoReportCounter { @@ -1104,15 +1452,19 @@ mod macos_ioreport { #[cfg(test)] mod tests { + use super::script::{self, Script}; use super::*; - use crate::rail_liveness::CpuRail; + use crate::rail_liveness::{CpuRail, RailKey, SampleCall}; + use std::cell::RefCell; + use std::rc::Rc; use std::sync::atomic::AtomicBool; unsafe extern "C" { fn clock_gettime_nsec_np(clock_id: u32) -> u64; } - /// `CLOCK_THREAD_CPUTIME_ID` in macOS ``: the tests' own copy. + /// `CLOCK_THREAD_CPUTIME_ID` in macOS ``: the tests' own copy, + /// declared and read independently of the implementation's. const OWN_CLOCK_THREAD_CPUTIME_ID: u32 = 16; /// This thread's CPU time, read by the tests themselves @@ -1125,6 +1477,16 @@ mod macos_ioreport { /// read from the implementation's constant. const TICK: Duration = Duration::from_millis(25); + /// The wall time after which a tick is specified to give up, stated + /// here rather than read from the implementation's constant. + const TICK_WALL_CAP: Duration = Duration::from_millis(500); + + /// How far two readings of different clocks can disagree by + /// granularity alone: both step at the 24 MHz timebase (~42 ns), so a + /// tick that ran without a break can read a few tens of ns more CPU + /// than wall time. Real CPU never exceeds wall by more. + const CLOCK_STEP: Duration = Duration::from_micros(1); + /// Real CPU work for the tests: spin this thread for `d` of wall time. fn spin(d: Duration) { let started = Instant::now(); @@ -1224,15 +1586,39 @@ mod macos_ioreport { } impl OracleSub { + /// A subscription to every channel. fn open() -> Option { + Self::subscribe(unsafe { IOReportCopyAllChannels(0, 0) }) + } + + /// A subscription to the Energy Model group alone, by the oracle's + /// own calls: reads of ~7 ms instead of the 0.1-0.8 s of every + /// channel, for brackets that must be tight. + fn open_group() -> Option { unsafe { - let all = IOReportCopyAllChannels(0, 0); - if all.is_null() { + let group = CFStringCreateWithCString( + std::ptr::null_mut(), + c"Energy Model".as_ptr() as *const _, + K_CFSTRING_ENCODING_UTF8, + ); + if group.is_null() { + return None; + } + let desired = IOReportCopyChannelsInGroup(group, std::ptr::null(), 0, 0, 0); + CFRelease(group); + Self::subscribe(desired) + } + } + + /// Subscribe to `desired` (consumed; null gives `None`). + fn subscribe(desired: CFMutableDictionaryRef) -> Option { + unsafe { + if desired.is_null() { return None; } let mut chans: CFMutableDictionaryRef = std::ptr::null_mut(); - let sub = IOReportCreateSubscription(std::ptr::null(), all, &mut chans, 0, std::ptr::null()); - CFRelease(all as CFTypeRef); + let sub = IOReportCreateSubscription(std::ptr::null(), desired, &mut chans, 0, std::ptr::null()); + CFRelease(desired as CFTypeRef); if sub.is_null() || chans.is_null() { if !sub.is_null() { CFRelease(sub); @@ -1246,10 +1632,11 @@ mod macos_ioreport { } } - /// One read: every channel, in any group, whose name ends with `CPU - /// Energy` or `GPU Energy`, in sample order. `None` when IOReport - /// returns no sample, or one with no channel list. - fn read(&self) -> Option> { + /// One read: how many channels the sample held, in any group, and + /// every channel `keep(group, name)` accepts, in sample order. + /// `None` when IOReport returns no sample, or one with no channel + /// list. + fn read_where(&self, keep: impl Fn(&str, &str) -> bool) -> Option<(usize, Vec)> { unsafe { let smp = IOReportCreateSamples(self.sub, self.chans, std::ptr::null()); if smp.is_null() { @@ -1269,28 +1656,53 @@ mod macos_ioreport { }; let mut out = None; if !arr.is_null() { + let count = CFArrayGetCount(arr); let mut chans = Vec::new(); - for i in 0..CFArrayGetCount(arr) { + for i in 0..count { let it = CFArrayGetValueAtIndex(arr, i); if it.is_null() { continue; } + let group = oracle_name(IOReportChannelGetGroup(it)); let name = oracle_name(IOReportChannelGetChannelName(it)); - if name.ends_with("CPU Energy") || name.ends_with("GPU Energy") { + if keep(&group, &name) { chans.push(OracleChannel { - group: oracle_name(IOReportChannelGetGroup(it)), unit: oracle_name(IOReportChannelGetUnitLabel(it)), raw: IOReportSimpleGetIntegerValue(it, 0) as i64, + group, name, }); } } - out = Some(chans); + out = Some((count.max(0) as usize, chans)); } CFRelease(smp); out } } + + /// One read: every channel, in any group, whose name ends with `CPU + /// Energy` or `GPU Energy`, in sample order. + fn read(&self) -> Option> { + self.read_where(|_, name| name.ends_with("CPU Energy") || name.ends_with("GPU Energy")).map(|(_, c)| c) + } + + /// One read's shape: (every channel, the Energy Model group's channels). + fn shape(&self) -> Option<(usize, usize)> { + self.read_where(|group, _| group == "Energy Model").map(|(all, group)| (all, group.len())) + } + + /// One read of the Energy Model group's CPU and GPU energy rails (the + /// oracle's own name rule), keyed by (name, ordinal among the same + /// name), in sample order. + fn energy_rails(&self) -> Vec<(RailKey, i64)> { + let (_, chans) = self + .read_where(|group, name| { + group == "Energy Model" && (name.ends_with("CPU Energy") || name.ends_with("GPU Energy")) + }) + .expect("an oracle read"); + oracle_keyed(chans.into_iter().map(|c| (c.name, c.raw))) + } } impl Drop for OracleSub { @@ -1302,19 +1714,15 @@ mod macos_ioreport { } } - /// A rise of `raw` in a rail's unit, in whole µJ, by the oracle's own - /// unit table. An unknown unit counts only whole millions of raw units, - /// a µJ in the finest unit (pJ). - fn oracle_uj(raw: i64, unit: &str) -> u64 { - let raw = raw.max(0) as u64; - match unit { - "pJ" => raw / 1_000_000, - "nJ" => raw / 1_000, - "uJ" | "µJ" => raw, - "mJ" => raw.saturating_mul(1_000), - "J" => raw.saturating_mul(1_000_000), - _ => raw / 1_000_000, + /// (name, value) pairs keyed by (name, ordinal among the same name), in + /// order: the oracle's own keying. + fn oracle_keyed(items: impl IntoIterator) -> Vec<(RailKey, V)> { + let mut out: Vec<(RailKey, V)> = Vec::new(); + for (name, v) in items { + let ordinal = out.iter().filter(|((n, _), _)| *n == name).count() as u32; + out.push(((name, ordinal), v)); } + out } /// What the oracle saw on this host. @@ -1437,6 +1845,334 @@ mod macos_ioreport { }) } + // ── The production status, on scripted IOReport leaves ─────────────── + + /// A sample the way the FFI returns one of the Energy Model group: + /// every channel in the group. + fn group_sample(values: &[(&str, i64)]) -> Sample { + Sample { + channels: values.len(), + group_only: true, + values: values.iter().map(|(n, raw)| (n.to_string(), *raw)).collect(), + } + } + + /// One production tick as the tests' own clocks saw it. + #[derive(Debug, Clone, Copy)] + struct TickSeen { + cpu: Duration, + wall: Duration, + } + + /// What a script saw of the production ticks: when it last returned a + /// sample (wall, then this thread's CPU), and every tick since the first. + #[derive(Default)] + struct TickMarks { + returned: Option<(Instant, Duration)>, + ticks: Vec, + } + + /// The production status — the counted, uncached [`host_status`] — + /// with a script standing in for IOReport's two FFI leaves: the group + /// subscription opens or not, and `sample(i)` is its i-th sample. + /// Returns the status, how many samples the script served, and every + /// production tick as the tests' own clocks saw it: from the moment the + /// script returned one sample to the moment it was asked for the next, + /// with the CPU reading nested inside the wall reading. + fn scripted_host_status( + opens: bool, + sample: impl Fn(usize) -> Option + 'static, + ) -> (IoReportStatus, usize, Vec) { + let seen: Rc> = Rc::default(); + let marks = Rc::clone(&seen); + let script = Script { + opens, + samples: Box::new(move |i| { + let (cpu, wall) = (own_thread_cpu(), Instant::now()); + let mut m = marks.borrow_mut(); + if let Some((wall0, cpu0)) = m.returned { + m.ticks.push(TickSeen { cpu: cpu.saturating_sub(cpu0), wall: wall.saturating_duration_since(wall0) }); + } + assert!(i < 10_000, "the probe took {i} samples in one budget: its tick does not burn CPU"); + let s = sample(i); + m.returned = Some((Instant::now(), own_thread_cpu())); + s + }), + }; + let (status, served) = script::with(script, host_status); + let ticks = seen.borrow().ticks.clone(); + (status, served, ticks) + } + + /// The production status itself: [`host_status`] — the real opener's + /// sampler closure, [`production_probe`] with the real tick and clocks, + /// the rule and the mapping — with a script standing in for IOReport's + /// two FFI leaves. No subscription is `Unavailable`, with no sample + /// taken; a single batch refresh is `CpuRailFrozen`; two advances are + /// `Live`, with no sample past the third; one step on each of two dies, + /// in different intervals, is `CpuRailFrozen`. These hosts produce none + /// of the first three on demand, so no hardware test can reach them. + /// Every scripted step falls in the first two intervals, which the + /// probe always observes however loaded the host: it takes at least + /// three samples, and its first tick ends within the budget. + #[test] + fn host_status_is_the_rule_on_the_production_composition() { + let (status, served, ticks) = scripted_host_status(false, |_| panic!("sampled without a subscription")); + assert_eq!( + (status, served, ticks.len()), + (IoReportStatus::Unavailable, 0, 0), + "no group subscription must be Unavailable, with no sample and no tick" + ); + + let refresh = + |i: usize| [("CPU Energy", if i < 1 { 480_083_160 } else { 541_893_638 }), ("GPU Energy", 1_000 + 7 * i as i64)]; + let started = Instant::now(); + let (status, served, _) = scripted_host_status(true, move |i| Some(group_sample(&refresh(i)))); + let took = started.elapsed(); + assert_eq!(status, IoReportStatus::CpuRailFrozen, "a single batch refresh ({served} samples) must be frozen"); + // A frozen verdict takes the whole budget, 1 s of wall time: the probe + // samples until it has passed. Load can only lengthen this. + assert!(took >= Duration::from_secs(1), "the production probe gave a frozen verdict after {took:?}"); + + let live = |i: usize| [("GPU Energy", 5), ("CPU Energy", 1_000 + 13 * i as i64)]; + let (status, served, _) = scripted_host_status(true, move |i| Some(group_sample(&live(i)))); + assert_eq!((status, served), (IoReportStatus::Live, 3), "two advances must be live at the third sample"); + + let dies = |i: usize| { + [ + ("DIE_0_CPU Energy", if i >= 1 { 9_500_000 } else { 9_494_336 }), + ("DIE_1_CPU Energy", if i >= 2 { 10_300_000 } else { 10_212_656 }), + ] + }; + let (status, served, _) = scripted_host_status(true, move |i| Some(group_sample(&dies(i)))); + assert_eq!(status, IoReportStatus::CpuRailFrozen, "one step on each die ({served} samples) must be frozen"); + } + + /// Each production tick, in the production composition, burns 25 ms of + /// this thread's CPU by the tests' own clock, or ran into its 500 ms + /// wall cap; and none shows more CPU than wall time. Between one + /// scripted sample and the next the production code runs its tick and + /// only bookkeeping besides, so the script's own clock readings + /// bracket each tick. + #[test] + fn each_production_tick_burns_its_cpu_by_the_tests_own_clock() { + let (status, served, ticks) = + scripted_host_status(true, |i| Some(group_sample(&[("CPU Energy", 42), ("GPU Energy", i as i64)]))); + let cpus: Vec = ticks.iter().map(|t| t.cpu).collect(); + let walls: Vec = ticks.iter().map(|t| t.wall).collect(); + eprintln!( + "ioreport production ticks (the tests' own clocks): {} ticks between {served} scripted samples; \ + thread CPU min {:?} median {:?} max {:?}; wall min {:?} max {:?}", + ticks.len(), + cpus.iter().min(), + (!cpus.is_empty()).then(|| median(&cpus)), + cpus.iter().max(), + walls.iter().min(), + walls.iter().max() + ); + assert_eq!(status, IoReportStatus::CpuRailFrozen); + assert!(ticks.len() >= 2 && ticks.len() == served - 1, "{} ticks for {served} samples", ticks.len()); + for (i, t) in ticks.iter().enumerate() { + assert!( + t.cpu >= TICK || t.wall >= TICK_WALL_CAP, + "production tick {i} burned {:?} of this thread's CPU in {:?} of wall time (every tick: {ticks:?}): \ + a tick burns 25 ms of the thread's CPU, or gives up after 500 ms of wall time", + t.cpu, + t.wall + ); + assert!( + t.cpu <= t.wall + CLOCK_STEP, + "production tick {i}: {:?} of this thread's CPU in {:?} of wall time", + t.cpu, + t.wall + ); + } + } + + /// The production tick burns thread CPU, not wall time. With every core + /// oversubscribed by busy threads, a tick that spun for 25 ms of WALL + /// time would get only part of that as CPU; the production tick keeps + /// spinning until this thread's own CPU clock has advanced 25 ms (or its + /// 500 ms wall cap). Pinned per tick, by the tests' own clocks, in the + /// production composition, over a frozen verdict's whole budget, which + /// always holds two ticks at least; and the contention must have been + /// real: at least one tick shared its core. The probe starts only once + /// every busy thread is running. + #[test] + fn the_production_tick_burns_cpu_not_wall_time_on_a_shared_core() { + let spinners = 2 * std::thread::available_parallelism().map_or(8, |n| n.get()); + let (stop, running) = (AtomicBool::new(false), AtomicUsize::new(0)); + let (status, served, ticks) = std::thread::scope(|s| { + // Stops the spinners however the scope's body ends. + struct Stop<'a>(&'a AtomicBool); + impl Drop for Stop<'_> { + fn drop(&mut self) { + self.0.store(true, Ordering::Relaxed); + } + } + let _stop = Stop(&stop); + for _ in 0..spinners { + s.spawn(|| { + running.fetch_add(1, Ordering::SeqCst); + let mut acc: u64 = 0; + while !stop.load(Ordering::Relaxed) { + acc = std::hint::black_box(acc.wrapping_add(1)); + } + }); + } + while running.load(Ordering::SeqCst) < spinners { + std::thread::yield_now(); + } + scripted_host_status(true, |i| Some(group_sample(&[("CPU Energy", 42), ("GPU Energy", i as i64)]))) + }); + let shared = ticks.iter().filter(|t| t.wall > t.cpu + Duration::from_millis(5)).count(); + eprintln!( + "ioreport production ticks on a shared core ({spinners} busy threads): status={status:?} after {served} \ + samples; {shared} of {} ticks shared their core; every tick (cpu, wall): {:?}", + ticks.len(), + ticks.iter().map(|t| (t.cpu, t.wall)).collect::>() + ); + assert_eq!(status, IoReportStatus::CpuRailFrozen); + assert!(ticks.len() >= 2 && ticks.len() == served - 1, "{} ticks for {served} samples", ticks.len()); + for (i, t) in ticks.iter().enumerate() { + assert!( + t.cpu >= TICK || t.wall >= TICK_WALL_CAP, + "production tick {i} burned {:?} of this thread's CPU in {:?} of wall time on a shared core: \ + the tick must spin until the thread's CPU clock has advanced 25 ms, not for 25 ms of wall time", + t.cpu, + t.wall + ); + assert!( + t.cpu <= t.wall + CLOCK_STEP, + "production tick {i}: {:?} of this thread's CPU in {:?} of wall time", + t.cpu, + t.wall + ); + } + assert!( + shared >= 1, + "no tick shared its core with the {spinners} busy threads ({ticks:?}), so this run could not tell a \ + tick on the thread's CPU clock from one on the wall clock" + ); + } + + /// The implementation's thread clock is this thread's CPU time, checked + /// against the tests' own `CLOCK_THREAD_CPUTIME_ID`: it keeps pace with + /// the tests' clock while this thread burns CPU; it does not advance + /// across a 100 ms sleep (so it is no wall clock); and it does not + /// advance while another thread burns 100 ms of CPU and this one waits + /// (so it is not the process's CPU clock). + #[test] + fn the_probe_thread_clock_is_this_threads_cpu_time() { + let ms = Duration::from_millis; + let (own0, p0) = (own_thread_cpu(), thread_cpu_time()); + while own_thread_cpu() - own0 < ms(20) { + std::hint::black_box(0u64); + } + let (p1, own1) = (thread_cpu_time(), own_thread_cpu()); + let (burned, own) = (p1.saturating_sub(p0), own1 - own0); + + let p0 = thread_cpu_time(); + std::thread::sleep(ms(100)); + let slept = thread_cpu_time().saturating_sub(p0); + + let p0 = thread_cpu_time(); + let helper = std::thread::scope(|s| { + s.spawn(|| { + let c0 = own_thread_cpu(); + let mut acc: u64 = 0; + while own_thread_cpu() - c0 < ms(100) { + acc = std::hint::black_box(acc.wrapping_add(1)); + } + own_thread_cpu() - c0 + }) + .join() + .expect("the helper thread panicked") + }); + let waited = thread_cpu_time().saturating_sub(p0); + eprintln!( + "ioreport thread clock: {burned:?} while this thread burned {own:?} (tests' own clock); {slept:?} across \ + a 100 ms sleep; {waited:?} while a helper burned {helper:?} and this thread waited" + ); + assert!( + burned <= own && own - burned < ms(1), + "the implementation's thread clock advanced {burned:?} while the tests' own thread clock advanced {own:?}" + ); + assert!( + slept < ms(5), + "the implementation's thread clock advanced {slept:?} across a 100 ms sleep: it reads wall time, not \ + this thread's CPU" + ); + assert!(helper >= ms(100), "the helper burned only {helper:?}"); + assert!( + waited < ms(5), + "the implementation's thread clock advanced {waited:?} while another thread burned {helper:?} of CPU and \ + this one waited: it reads the process's CPU, not this thread's" + ); + } + + /// A declined status declines before anything opens: given + /// `Unavailable`, `CpuRailAbsent` or `CpuRailFrozen`, the constructor + /// returns `Err(NoCounter)` without trying to open the counter's + /// subscription, and so does selection, whatever IOReport does on this + /// host. Given `Live`, the constructor does try to open it — the count + /// is live. + #[test] + fn a_declined_status_declines_before_anything_opens() { + use IoReportStatus as S; + for status in [S::Unavailable, S::CpuRailAbsent, S::CpuRailFrozen] { + let before = script::counter_opens(); + let given = MacosIoReportCounter::try_detect_given(status); + assert!(matches!(given, Err(Error::NoCounter)), "try_detect_given({status:?}) did not decline with NoCounter"); + let picked = crate::detect_with(|| status).name(); + assert_ne!(picked, "ioreport", "detect_with({status:?}) selected the IOReport counter"); + let (built, picked) = + script::with_status(status, || (MacosIoReportCounter::try_detect().is_ok(), crate::detect().name())); + assert!(!built && picked != "ioreport", "try_detect()/detect() under {status:?}: built={built} picked={picked}"); + assert_eq!( + script::counter_opens(), + before, + "under {status:?} the constructor or selection tried to open the counter's subscription before declining" + ); + } + let before = script::counter_opens(); + let _ = MacosIoReportCounter::try_detect_given(S::Live); + assert_eq!(script::counter_opens(), before + 1, "given Live, the constructor did not try to open its subscription"); + } + + /// Every constructor and selection path gates on the pure + /// `admits_counter`: given each declined status, `try_detect_given` + /// declines, and `detect_with` falls through to a counter that is not + /// IOReport; given `Live`, both build the IOReport counter wherever the + /// all-channel subscription opens. And the real `try_detect()` and + /// `detect()`, run with each status installed for `ioreport_status()` + /// on this thread, do the same; with none installed they agree with + /// the cached status. + #[test] + fn only_a_live_status_constructs_or_selects_the_counter() { + use IoReportStatus as S; + let opens = MacosIoReportCounter::open().is_some(); + for status in [S::Unavailable, S::CpuRailAbsent, S::CpuRailFrozen, S::Live] { + let admitted = status == S::Live && opens; + let given = MacosIoReportCounter::try_detect_given(status); + assert_eq!(given.is_ok(), admitted, "try_detect_given({status:?}), the subscription opening={opens}"); + if let Err(e) = given { + assert!(matches!(e, Error::NoCounter), "try_detect_given({status:?}) declined with {e:?}"); + } + let picked = crate::detect_with(|| status).name(); + assert_eq!(picked == "ioreport", admitted, "detect_with({status:?}) selected {picked}"); + let (built, picked) = script::with_status(status, || { + assert_eq!(crate::ioreport_status(), status, "the installed status"); + (MacosIoReportCounter::try_detect().is_ok(), crate::detect().name()) + }); + assert_eq!(built, admitted, "try_detect() with the status {status:?}, the subscription opening={opens}"); + assert_eq!(picked == "ioreport", admitted, "detect() with the status {status:?} selected {picked}"); + } + let status = crate::ioreport_status(); + assert_eq!(MacosIoReportCounter::try_detect().is_ok(), status == S::Live && opens, "status={status:?}"); + assert_eq!(crate::detect().name() == "ioreport", status == S::Live && opens, "status={status:?}"); + } + // ── Against the hardware ───────────────────────────────────────────── /// The property that matters: the IOReport counter is **never** @@ -1510,25 +2246,24 @@ mod macos_ioreport { assert_eq!(constructible, status == IoReportStatus::Live); } - /// The dangerous direction on its own, against a fresh (uncached) probe - /// on its own Energy Model subscription beside the cached status: it - /// never reports Live for a rail the oracle did not see keep moving, and - /// on every host both agree with the oracle about what exists. + /// The dangerous direction on its own, against the production + /// composition run fresh (uncounted, uncached) beside the cached status: + /// it never reports Live for a rail the oracle did not see keep moving, + /// and on every host both agree with the oracle about what exists. #[test] fn the_liveness_probe_never_reports_a_frozen_rail_as_live() { - let run = probe(crate::CPU_RAIL_LIVENESS_BUDGET); - let probed = run.as_ref().map(|r| r.watch.verdict()); + let run = production_probe(); + let probed = run.opened.then(|| run.verdict()); let cached = crate::ioreport_status(); let oracle = oracle_cpu_rail(crate::CPU_RAIL_LIVENESS_BUDGET * 3); - let walls: Vec = run.as_ref().map(|r| r.calls.iter().map(|c| c.wall).collect()).unwrap_or_default(); eprintln!( - "ioreport probe: probed={probed:?} cached={cached:?} intervals={} sample calls={} \ - median sample cost={:?} advances per CPU rail {:?}; oracle={:?} ({} reads over {:?}, changes \ - per CPU rail {:?}, per GPU rail {:?})", - run.as_ref().map_or(0, |r| r.watch.intervals), - walls.len(), - (!walls.is_empty()).then(|| median(&walls)), - run.as_ref().map(|r| r.watch.rails.clone()).unwrap_or_default(), + "ioreport probe: probed={probed:?} cached={cached:?} intervals={} sample calls={} ticks={} \ + advances per CPU rail {:?}; oracle={:?} ({} reads over {:?}, changes per CPU rail {:?}, per GPU \ + rail {:?})", + run.watch.intervals, + run.calls.len(), + run.ticks, + run.watch.rails, oracle.verdict, oracle.reads, oracle.took, @@ -1566,34 +2301,50 @@ mod macos_ioreport { } } - /// The probe burns CPU between samples, and exactly its tick of it, - /// pinned per tick on this thread's CPU clock so host load cannot fail - /// it: the CPU the thread consumed between the end of one sample call - /// and the start of the next must be at least half the 25 ms tick (a - /// probe that sleeps, or does nothing, between samples burns ~0) and at - /// most twice it (a slower cadence, e.g. a 100 ms tick, burns more). A - /// tick spins until the thread's CPU clock has advanced 25 ms, so a - /// loaded host stretches it in wall time only. And the tick CPU the - /// probe recorded is checked against the test's own reading of the - /// thread's CPU clock over the whole probe. + /// The production probe burns CPU between samples, and exactly its tick + /// of it, pinned per tick on the probe's own records of this thread's + /// CPU clock (identified as such by the thread-clock test) so host load + /// cannot fail it: the CPU the thread consumed between the end of one + /// sample call and the start of the next must be at least half the + /// 25 ms tick (a probe that sleeps, or does nothing, between samples + /// burns ~0) and at most twice it (a slower cadence, e.g. a 100 ms tick, + /// burns more). And the tick CPU the probe recorded is checked against + /// the tests' own reading of the thread's CPU clock over the whole probe, + /// and every thread-clock reading the probe recorded must lie between the + /// tests' own readings of this thread's CPU clock just before and just + /// after it: the clock the production composition hands the probe is + /// this thread's CPU clock, not the wall clock or the process's. #[test] fn the_probe_burns_its_tick_of_thread_cpu_between_samples() { let own0 = own_thread_cpu(); let started = Instant::now(); - let Some(run) = probe(crate::CPU_RAIL_LIVENESS_BUDGET) else { + let run = production_probe(); + let own1 = own_thread_cpu(); + let own = own1.saturating_sub(own0); + let took = started.elapsed(); + for (i, c) in run.calls.iter().enumerate() { + assert!( + own0 <= c.cpu_before && c.cpu_before <= c.cpu_after && c.cpu_after <= own1, + "sample call {i} recorded the probe's thread clock at {:?}..{:?}, outside this thread's own CPU \ + clock readings {own0:?}..{own1:?} around the probe: the production probe is not timing this \ + thread's CPU", + c.cpu_before, + c.cpu_after + ); + } + if !run.opened { let status = crate::ioreport_status(); eprintln!("ioreport probe tick: no Energy Model subscription opens here (status={status:?}): no probe to pin"); assert_eq!(status, IoReportStatus::Unavailable); + assert!(run.calls.is_empty() && run.ticks == 0, "{run:?}"); return; - }; - let own = own_thread_cpu().saturating_sub(own0); - let took = started.elapsed(); + } let ticks: Vec = run.calls.windows(2).map(|w| w[1].cpu_before.saturating_sub(w[0].cpu_after)).collect(); let recorded: Duration = ticks.iter().sum(); eprintln!( "ioreport probe tick: {} ticks of this thread's CPU between {} sample calls: min {:?}, median {:?}, \ - max {:?} (specified 25 ms); the test's own reading of the thread's CPU clock over the whole \ + max {:?} (specified 25 ms); the tests' own reading of the thread's CPU clock over the whole \ probe: {own:?} in {took:?} of wall time", ticks.len(), run.calls.len(), @@ -1602,6 +2353,7 @@ mod macos_ioreport { ticks.iter().max() ); assert!(ticks.len() >= 2, "the probe made only {} sample calls", run.calls.len()); + assert_eq!(run.ticks as usize, ticks.len(), "every tick runs between two sample calls"); for (i, t) in ticks.iter().enumerate() { assert!( *t >= TICK / 2, @@ -1622,33 +2374,64 @@ mod macos_ioreport { ); } - /// The probe samples the Energy Model group alone: the median wall - /// duration of its own sample calls, as the probe recorded them, is under - /// 50 ms. One sample of the group takes ~6-12 ms on an M5 Max and an M3 - /// Ultra; one of every channel 130-800 ms, which would leave the probe - /// sampling instead of burning. The median, not the mean or the maximum, - /// so a few samples delayed by a loaded host cannot fail it. + /// The production probe samples the Energy Model group and nothing else, + /// pinned by structure, not time: every sample it took held exactly the + /// Energy Model group's channels — as many as the oracle's all-channel + /// read counts in that group, far fewer than every channel — and every + /// one of them is in the group. And, second, what a sample costs the + /// probing thread in CPU: the group takes ~6-9 ms of it on an M5 Max and + /// an M3 Ultra, every channel ~80-120 ms. No wall-clock bound: another + /// process's all-channel read can hold IOReport, and a group sample + /// then waits in wall time without costing this thread CPU. #[test] fn the_probe_samples_only_the_energy_model_group() { - let Some(run) = probe(crate::CPU_RAIL_LIVENESS_BUDGET) else { - let status = crate::ioreport_status(); - eprintln!("ioreport probe sample cost: no Energy Model subscription opens here (status={status:?})"); - assert_eq!(status, IoReportStatus::Unavailable); + let Some(oracle) = OracleSub::open() else { + let run = production_probe(); + eprintln!("ioreport probe structure: no IOReport subscription opens here; probe opened={}", run.opened); + assert!(!run.opened && run.calls.is_empty(), "the oracle cannot open IOReport, the probe did: {run:?}"); + assert_eq!(crate::ioreport_status(), IoReportStatus::Unavailable); return; }; + let (all, group) = oracle.shape().expect("an oracle read"); + let run = production_probe(); + let taken: Vec<&SampleCall> = run.calls.iter().filter(|c| c.shape.is_some()).collect(); + let mut shapes: Vec<(usize, bool)> = taken.iter().filter_map(|c| c.shape).collect(); + shapes.dedup(); + let cpus: Vec = run.calls.iter().map(|c| c.cpu_after.saturating_sub(c.cpu_before)).collect(); let walls: Vec = run.calls.iter().map(|c| c.wall).collect(); - let cost = median(&walls); eprintln!( - "ioreport probe sample cost: median {cost:?} over {} sample calls (min {:?}, max {:?}); \ - the limit is 50 ms (the Energy Model group ~6-12 ms, every channel 130-800 ms)", - walls.len(), - walls.iter().min(), + "ioreport probe structure: {} of {} sample calls took a sample, shaped (channels, all Energy Model) \ + {shapes:?}; the Energy Model group is {group} of the {all} channels in the oracle's all-channel \ + read; thread CPU per sample call: median {:?} (limit 50 ms), max {:?}; wall per call (not \ + asserted): median {:?}, max {:?}", + taken.len(), + run.calls.len(), + (!cpus.is_empty()).then(|| median(&cpus)), + cpus.iter().max(), + (!walls.is_empty()).then(|| median(&walls)), walls.iter().max() ); + assert!(run.opened, "the oracle's all-channel subscription opened, the probe's group subscription did not"); + assert!(taken.len() >= 2, "the probe took only {} samples", taken.len()); + assert!( + group > 0 && group * 10 <= all, + "the oracle counts {group} Energy Model channels of {all}: the group is not far below every channel" + ); + for (i, c) in taken.iter().enumerate() { + assert_eq!( + c.shape, + Some((group, true)), + "sample {i} held {:?} (channels, all in the Energy Model group), the group is {group} of {all} \ + channels: the probe must sample its own subscription to the Energy Model group, not every \ + channel", + c.shape + ); + } + let cost = median(&cpus); assert!( cost < Duration::from_millis(50), - "the probe's median sample call took {cost:?} (every call: {walls:?}): it must sample its own \ - subscription to the Energy Model group, not every channel" + "the probe's sample calls took a median {cost:?} of this thread's CPU (every call: {cpus:?}): the \ + Energy Model group costs ~6-9 ms, every channel ~80-120 ms" ); } @@ -1684,8 +2467,9 @@ mod macos_ioreport { for _ in 0..3 { let before = oracle_rails(); let sampled: Vec<(String, i64)> = state - .energy_model_sample() + .group_sample() .expect("the probe's sampler returned no sample") + .values .into_iter() .filter(|(n, _)| n.ends_with("CPU Energy")) .collect(); @@ -1716,107 +2500,198 @@ mod macos_ioreport { eprintln!("ioreport sampler: {} CPU-rail values checked against the oracle: {seen:?}", seen.len()); } - /// The counter's own read path, which a declined host never runs: built - /// past the liveness gate on purpose, it must sum what its rails did. A - /// reader returning a constant (the counter open, and reading 0 µJ over - /// a whole run) can be told from a true zero only by a rail that moved, - /// and the counter sums `GPU Energy`, which moves on an M5 Max. So: take - /// the counter's baseline sample; read `GPU Energy` in the Energy Model - /// group independently (the oracle's own subscription, absolute values) - /// until it has risen by at least 1 µJ by the oracle's own unit table, or - /// 3 s pass; then read the counter again. The counter's window contains - /// the oracle's and the rail is cumulative, so if GPU Energy rose the - /// counter must have advanced. If it did not rise, nothing can tell a - /// constant from a true zero: the counter is printed, NOT asserted, and - /// the printed line says so. + /// What the production composition hands the rule is what the rails + /// hold. The oracle reads the Energy Model group's CPU and GPU energy + /// rails, the production probe runs (uncounted, uncached), and the + /// oracle reads again. In every sample the probe took: its rails are + /// exactly the oracle's; a CPU rail the oracle saw hold must EQUAL the + /// oracle's value, and one that moved must lie within the oracle's two + /// reads; so must GPU Energy, which moves on these hosts, so a sampler + /// replaying a sample from before the bracket fails. And where the + /// oracle saw GPU Energy change across the bracket, the probe's samples + /// must show it changing too, so a sampler replaying its own first + /// sample fails; where the oracle saw it hold, that is not asserted, and + /// the line says so. #[test] - fn the_counter_advances_when_a_rail_it_sums_advanced() { - let Some(state) = MacosIoReportCounter::open() else { - let selected = crate::detect().name() == "ioreport"; - eprintln!("ioreport counter: no IOReport subscription opens here; selected={selected}"); - assert!(!selected, "detect() selected an IOReport counter that does not open"); + fn the_production_probe_hands_the_rule_what_the_oracle_reads() { + let Some(oracle) = OracleSub::open() else { + let run = production_probe(); + eprintln!("ioreport freshness: no IOReport subscription opens here; probe opened={}", run.opened); + assert!(!run.opened && run.calls.is_empty(), "the oracle cannot open IOReport, the probe did: {run:?}"); return; }; - let counter = MacosIoReportCounter { state: Mutex::new(state) }; - let oracle = OracleSub::open().expect("the counter's all-channel subscription opened, the oracle's did not"); - let gpu = || { - oracle.read().map(|chans| { - chans - .into_iter() - .filter(|c| c.group == "Energy Model" && c.name.ends_with("GPU Energy")) - .collect::>() - }) + let before = oracle.energy_rails(); + let run = production_probe(); + let after = oracle.energy_rails(); + let keys = |v: &[(RailKey, i64)]| { + let mut k: Vec = v.iter().map(|(k, _)| k.clone()).collect(); + k.sort(); + k }; - let base = counter.read_microjoules().expect("the counter's baseline read"); - let g0 = gpu().expect("an independent read of GPU Energy"); - let started = Instant::now(); - let (mut rose_raw, mut rose_uj, mut unit) = (0i64, 0u64, String::new()); - while !g0.is_empty() && rose_uj == 0 && started.elapsed() < Duration::from_secs(3) { - let Some(g) = gpu() else { - continue; - }; - for (now, before) in g.iter().zip(&g0).filter(|(now, before)| now.name == before.name) { - let uj = oracle_uj(now.raw - before.raw, &now.unit); - if uj >= rose_uj { - (rose_raw, rose_uj, unit) = (now.raw - before.raw, uj, now.unit.clone()); + assert_eq!(keys(&before), keys(&after), "the oracle's rails changed between its two reads"); + let bracket = |key: &RailKey| -> (i64, i64) { + let find = |v: &[(RailKey, i64)]| v.iter().find(|(k, _)| k == key).map(|(_, raw)| *raw); + (find(&before).expect("a rail the oracle read"), find(&after).expect("a rail the oracle read")) + }; + assert!(run.opened, "the oracle's all-channel subscription opened, the probe's group subscription did not"); + let handed: Vec<&Vec<(RailKey, i64)>> = run.calls.iter().filter_map(|c| c.handed.as_ref()).collect(); + assert!(handed.len() >= 2, "the probe took only {} samples", handed.len()); + for (i, sample) in handed.iter().enumerate() { + assert_eq!( + keys(sample), + keys(&before), + "sample {i}: the production probe handed the rule these CPU and GPU rails, the oracle reads others" + ); + for (key, got) in sample.iter() { + let (lo, hi) = bracket(key); + if lo == hi { + assert_eq!( + *got, lo, + "sample {i}: {key:?} held at {lo} across the oracle's reads either side of the production \ + probe, and the probe handed the rule {got}" + ); + } else { + assert!( + (lo.min(hi)..=lo.max(hi)).contains(got), + "sample {i}: {key:?} moved {lo} -> {hi} across the oracle's reads either side of the \ + production probe, and the probe handed the rule {got}, outside them" + ); } } } - let window = started.elapsed(); - let after = counter.read_microjoules().expect("the counter's second read"); + let mut lines = Vec::new(); + for (key, lo) in before.iter().filter(|((name, _), _)| name.ends_with("GPU Energy")) { + let hi = bracket(key).1; + let mut values: Vec = + handed.iter().filter_map(|s| s.iter().find(|(k, _)| k == key).map(|(_, v)| *v)).collect(); + values.dedup(); + if *lo != hi { + assert!( + values.len() >= 2, + "{key:?} moved {lo} -> {hi} across the oracle's reads, but every one of the {} samples the \ + production probe took read {values:?}: its sampler is not sampling", + handed.len() + ); + lines.push(format!("{}: moved {lo} -> {hi}, the probe read {} distinct values within it", key.0, values.len())); + } else { + lines.push(format!("{}: held at {lo} across the bracket, so freshness is NOT asserted on it", key.0)); + } + } + let cpu: Vec = before + .iter() + .filter(|((name, _), _)| name.ends_with("CPU Energy")) + .map(|(key, lo)| format!("{}: {lo}..{}", key.0, bracket(key).1)) + .collect(); eprintln!( - "ioreport counter: GPU Energy (independent, absolute; {} rail(s)) rose {rose_raw} {unit} = {rose_uj} µJ \ - over {window:?}; the counter, over a window containing it, read {base} -> {after} µJ", - g0.len() + "ioreport freshness: {} samples from the production probe, each handing the rule the oracle's rails; \ + CPU rails (oracle before..after) {cpu:?}; {lines:?}", + handed.len() ); - if rose_uj > 0 { - assert!( - after > base, - "GPU Energy rose {rose_uj} µJ inside the counter's window, but the counter read {base} -> \ - {after} µJ: it did not sum a rail it counts" - ); - } else { - eprintln!( - "ioreport counter: GPU Energy did not rise inside the counter's window, so a constant counter \ - cannot be told from a true zero: the counter is NOT asserted on this run" - ); + assert!(!lines.is_empty(), "the oracle read no GPU Energy rail in the Energy Model group"); + } + + /// A rise of `raw` in `unit`, in whole µJ, rounded down, by the oracle's + /// own table: `None` for a unit it has no exact conversion for. A fall + /// is no rise. + fn exact_uj(raw: i64, unit: &str) -> Option { + let raw = raw.max(0) as u64; + match unit { + "pJ" => Some(raw / 1_000_000), + "nJ" => Some(raw / 1_000), + "uJ" | "µJ" => Some(raw), + "mJ" => Some(raw.saturating_mul(1_000)), + "J" => Some(raw.saturating_mul(1_000_000)), + _ => None, } } - /// The probe runs once per PROCESS, however many threads ask at once. - /// Six threads, released together, each ask for the status and - /// construct the backend (`try_detect()` and `detect()`). The real probe - /// behind the cache, counted by a process-wide static, not a - /// thread-local, ran exactly once, and every thread got the same answer. - /// Counted, not timed: a re-probe that ends Live is fast. + /// Every Energy Model channel the counter sums (by [`counts_once`]), as + /// the oracle reads it: ((name, ordinal), (unit, absolute raw)). + fn summed_rails(oracle: &OracleSub) -> Vec<(RailKey, (String, i64))> { + let (_, chans) = oracle.read_where(|group, _| group == "Energy Model").expect("an oracle read"); + oracle_keyed(chans.into_iter().filter(|c| counts_once(&c.name)).map(|c| (c.name, (c.unit, c.raw)))) + } + + /// What the summed rails rose from one oracle read to another, rail by + /// rail, rounded down to whole µJ per rail as the counter rounds. + fn rose_uj(from: &[(RailKey, (String, i64))], to: &[(RailKey, (String, i64))]) -> u64 { + assert_eq!( + from.iter().map(|(k, _)| k).collect::>(), + to.iter().map(|(k, _)| k).collect::>(), + "the summed rails changed between two oracle reads" + ); + from.iter() + .zip(to) + .map(|((key, (unit, a)), (_, (_, b)))| { + exact_uj(b - a, unit).unwrap_or_else(|| panic!("{key:?} is in {unit:?}, which the oracle cannot convert")) + }) + .sum() + } + + /// The counter's own read path, which a declined host never runs: built + /// past the liveness gate on purpose (given `Live`), it must report + /// exactly what its rails did. The oracle reads every rail the counter + /// sums (its own subscription to the Energy Model group, absolute + /// values, ~7 ms a read) just before and just after the counter's + /// baseline read, and again just before and just after its second read, + /// holding the window between them open for at least 3 s and until the + /// summed rails rose (GPU Energy moves on these hosts), for at most 5 s. + /// The rails are cumulative, so the counter's delta must lie between + /// their rise from the second oracle read to the third (inside the + /// counter's window) and from the first to the fourth (around it) — + /// rail by rail, rounded down as the counter rounds. Nothing moved: + /// 0 == 0, still asserted. A counter that left a moving rail out lands + /// below the bracket. The bracket is wider than the window by the + /// counter's own two reads (every channel: 0.1 s alone, over 1 s while + /// other processes hold IOReport), so one that summed a moving rail + /// twice lands above it only while those two reads together take less + /// than the window. #[test] - fn the_liveness_probe_runs_once_per_process() { - const THREADS: usize = 6; - let gate = std::sync::Barrier::new(THREADS); - let seen: Vec<(IoReportStatus, bool, &'static str)> = std::thread::scope(|s| { - let handles: Vec<_> = (0..THREADS) - .map(|_| { - s.spawn(|| { - gate.wait(); - let status = crate::ioreport_status(); - let built = MacosIoReportCounter::try_detect().is_ok(); - let picked = crate::detect().name(); - (status, built, picked) - }) - }) - .collect(); - handles.into_iter().map(|h| h.join().expect("a thread asking for the status panicked")).collect() - }); - let runs = PROBE_RUNS.load(Ordering::SeqCst); - eprintln!("ioreport cache: {THREADS} threads asked at once and got {seen:?}; real probes in this process: {runs}"); - let first = seen[0].0; - for (status, built, picked) in &seen { - assert_eq!(*status, first, "the threads got different statuses: {seen:?}"); - assert_eq!(*built, *status == IoReportStatus::Live, "try_detect() disagrees with the status: {seen:?}"); - assert_eq!(*picked == "ioreport", *built, "detect() disagrees with try_detect(): {seen:?}"); + fn the_counter_advances_when_a_rail_it_sums_advanced() { + let counter = match MacosIoReportCounter::try_detect_given(IoReportStatus::Live) { + Ok(counter) => counter, + Err(e) => { + let selected = crate::detect().name() == "ioreport"; + eprintln!("ioreport counter: no IOReport subscription opens here ({e:?}); selected={selected}"); + assert!(matches!(e, Error::NoCounter), "declined with {e:?}"); + assert!(OracleSub::open().is_none(), "the counter's all-channel subscription did not open, the oracle's did"); + assert!(!selected, "detect() selected an IOReport counter that does not open"); + return; + } + }; + let oracle = OracleSub::open_group().expect("the counter's subscription opened, the oracle's group one did not"); + let o1 = summed_rails(&oracle); + let base = counter.read_microjoules().expect("the counter's baseline read"); + let o2 = summed_rails(&oracle); + let started = Instant::now(); + let mut o3 = summed_rails(&oracle); + while started.elapsed() < Duration::from_secs(3) + || (rose_uj(&o2, &o3) == 0 && started.elapsed() < Duration::from_secs(5)) + { + std::thread::sleep(Duration::from_millis(20)); + o3 = summed_rails(&oracle); } - assert_eq!(crate::ioreport_status(), first, "the cached status changed"); - assert_eq!(runs, 1, "the liveness probe behind ioreport_status() ran {runs} times in this process"); + let after = counter.read_microjoules().expect("the counter's second read"); + let o4 = summed_rails(&oracle); + let (inner, outer) = (rose_uj(&o2, &o3), rose_uj(&o1, &o4)); + let delta = after.saturating_sub(base); + let moved: Vec = o1 + .iter() + .zip(&o4) + .filter(|((_, (_, a)), (_, (_, b)))| a != b) + .map(|((key, (unit, a)), (_, (_, b)))| format!("{} +{} {unit}", key.0, b - a)) + .collect(); + eprintln!( + "ioreport counter: {} rails summed; across the bracket they rose {inner} µJ inside the counter's \ + window and {outer} µJ around it (moved: {moved:?}); the counter read {base} -> {after} µJ = {delta} µJ", + o1.len() + ); + assert!(after >= base, "the counter went backwards: {base} -> {after} µJ"); + assert!( + (inner..=outer).contains(&delta), + "the counter read {delta} µJ over its window, but the rails it sums rose {inner} µJ inside that window \ + and {outer} µJ around it (moved: {moved:?}): it did not sum what its rails did" + ); } /// When the counter IS available, a CPU burn meters as non-zero energy @@ -1834,7 +2709,7 @@ mod macos_ioreport { IoReportStatus::Live, "try_detect() constructed the counter although the status is not Live" ); - let rail = probe(crate::CPU_RAIL_LIVENESS_BUDGET).and_then(|r| r.watch.verdict()); + let rail = production_probe().verdict(); assert_eq!(rail, Some(CpuRail::Live), "counter available but its CPU rail is {rail:?}"); let _ = c.read_microjoules(); // first read only sets the baseline sample let before = c.read_microjoules().unwrap(); @@ -1901,14 +2776,18 @@ pub enum IoReportStatus { Live, } -/// How long the liveness probe burns CPU before concluding the CPU rail is -/// not live: a live rail gets a second of single-thread CPU work, sampled -/// after every 25 ms of the probing thread's own CPU time (a tick that cannot -/// get it ends after 500 ms of wall time), to advance in two separate sample -/// intervals. If sampling is so slow that fewer than two intervals fit in the -/// budget, the probe keeps going until two have, and never past five budgets. -/// A rail too slow for that fails safe — declined, so callers estimate; never -/// over-claimed. Paid once per process; see [`ioreport_status`]. +/// How long, in WALL time, the liveness probe keeps sampling before it +/// concludes the CPU rail is not live. It spends that second burning CPU on +/// the calling thread in ticks of 25 ms of that thread's own CPU time, +/// sampling the rail after each (a tick that cannot get its CPU ends after +/// 500 ms of wall time). A loaded host gives the probing thread less of a core +/// in the same second, so the probe sees fewer intervals (on an M5 Max under +/// 1 × ncpu busy spinners, often half as many or fewer), and a live rail has +/// fewer chances to show its two advances. That fails safe: declined, so +/// callers estimate; never over-claimed. If sampling is so slow that fewer +/// than two intervals fit in the budget, the probe keeps going until two +/// have, and never past five budgets. Paid once per process; see +/// [`ioreport_status`]. #[cfg(target_os = "macos")] pub const CPU_RAIL_LIVENESS_BUDGET: std::time::Duration = std::time::Duration::from_millis(1000); @@ -1917,14 +2796,20 @@ pub const CPU_RAIL_LIVENESS_BUDGET: std::time::Duration = std::time::Duration::f /// `MacosIoReportCounter::try_detect` (and so [`detect`]) makes the IOReport /// counter available only when this is [`IoReportStatus::Live`]. The first /// call on macOS burns CPU on the calling thread for about -/// `CPU_RAIL_LIVENESS_BUDGET` (less when the rail proves live, at most five -/// budgets). Exposed so a caller or CLI can say *why* an Apple Silicon host is -/// not metered by IOReport. +/// `CPU_RAIL_LIVENESS_BUDGET` of wall time (less when the rail proves live, at +/// most five budgets). Exposed so a caller or CLI can say *why* an Apple +/// Silicon host is not metered by IOReport. pub fn ioreport_status() -> IoReportStatus { #[cfg(target_os = "macos")] { + // Test builds only: a status a test installed on this thread stands in + // for the cached one. + #[cfg(test)] + if let Some(status) = macos_ioreport::script::status() { + return status; + } static STATUS: std::sync::OnceLock = std::sync::OnceLock::new(); - *STATUS.get_or_init(|| macos_ioreport::probe_status(CPU_RAIL_LIVENESS_BUDGET)) + *STATUS.get_or_init(macos_ioreport::host_status) } #[cfg(not(target_os = "macos"))] { @@ -1932,6 +2817,36 @@ pub fn ioreport_status() -> IoReportStatus { } } +/// Test support, not API: how many times the liveness probe behind +/// [`ioreport_status`] has run in this process (always 0 off macOS). The +/// cache must keep it at 1 however many threads ask, construct or select. +#[doc(hidden)] +pub fn liveness_probe_runs() -> usize { + #[cfg(target_os = "macos")] + { + macos_ioreport::HOST_STATUS_RUNS.load(std::sync::atomic::Ordering::SeqCst) + } + #[cfg(not(target_os = "macos"))] + { + 0 + } +} + +/// Test support, not API: how many times this process has opened the +/// liveness probe's Energy Model sampler, whoever called it (always 0 off +/// macOS). A probe run outside the cache opens one too. +#[doc(hidden)] +pub fn liveness_sampler_opens() -> usize { + #[cfg(target_os = "macos")] + { + macos_ioreport::SAMPLER_OPENS.load(std::sync::atomic::Ordering::SeqCst) + } + #[cfg(not(target_os = "macos"))] + { + 0 + } +} + #[cfg(target_os = "linux")] mod linux_rapl { @@ -2064,6 +2979,13 @@ pub use nvml::NvmlCounter; /// CPU energy rail is live — see [`ioreport_status`]) → powermetrics on /// macOS → `StubCounter`. pub fn detect() -> Box { + detect_with(ioreport_status) +} + +/// The selection behind [`detect`], given where the IOReport status comes +/// from (asked only when the ladder reaches the IOReport rung). +#[cfg_attr(not(target_os = "macos"), allow(unused_variables))] +pub(crate) fn detect_with(ioreport: impl FnOnce() -> IoReportStatus) -> Box { #[cfg(feature = "cuda")] if let Ok(c) = NvmlCounter::try_detect() { return Box::new(c); @@ -2073,7 +2995,7 @@ pub fn detect() -> Box { return Box::new(c); } #[cfg(target_os = "macos")] - if let Ok(c) = MacosIoReportCounter::try_detect() { + if let Ok(c) = MacosIoReportCounter::try_detect_given(ioreport()) { return Box::new(c); } #[cfg(target_os = "macos")] diff --git a/eoc/eoc-rs/crates/eoc-meter/tests/liveness_probe_once_per_process.rs b/eoc/eoc-rs/crates/eoc-meter/tests/liveness_probe_once_per_process.rs new file mode 100644 index 000000000..7c4f537b5 --- /dev/null +++ b/eoc/eoc-rs/crates/eoc-meter/tests/liveness_probe_once_per_process.rs @@ -0,0 +1,93 @@ +//! The liveness probe behind `eoc_meter::ioreport_status()` runs once per +//! PROCESS, however many threads ask for the status, construct the IOReport +//! counter or select a counter, and in whatever order. +//! +//! This file is a test binary of its own, so it runs in a fresh process, and +//! its single test is the first use of the cache: nothing in the process has +//! probed before it starts (asserted). Two process-wide counts are pinned: the +//! runs of the probe behind the cache, and the opens of the probe's Energy +//! Model sampler, which every real probe makes whoever calls it — so a +//! constructor or selection path that probed outside the cache would show as a +//! second open even if it bypassed the cached function entirely. + +use std::sync::Barrier; + +use eoc_meter::IoReportStatus; + +/// Threads released together at the cold cache. +const THREADS: usize = 8; + +/// What one thread got: the status, whether the IOReport counter was +/// constructed, and the counter `detect()` selected. +type Seen = (IoReportStatus, bool, &'static str); + +#[cfg(target_os = "macos")] +fn constructed() -> bool { + eoc_meter::MacosIoReportCounter::try_detect().is_ok() +} + +#[cfg(not(target_os = "macos"))] +fn constructed() -> bool { + false +} + +/// Status, construction and selection, each thread starting on a different +/// one of the three paths into the cache. +fn ask(thread: usize) -> Seen { + match thread % 3 { + 0 => { + let status = eoc_meter::ioreport_status(); + (status, constructed(), eoc_meter::detect().name()) + } + 1 => { + let built = constructed(); + let picked = eoc_meter::detect().name(); + (eoc_meter::ioreport_status(), built, picked) + } + _ => { + let picked = eoc_meter::detect().name(); + let built = constructed(); + (eoc_meter::ioreport_status(), built, picked) + } + } +} + +#[test] +fn the_liveness_probe_runs_once_per_process() { + assert_eq!( + (eoc_meter::liveness_probe_runs(), eoc_meter::liveness_sampler_opens()), + (0, 0), + "something in this process probed before the test's first use of the cache" + ); + let gate = Barrier::new(THREADS); + let seen: Vec = std::thread::scope(|s| { + let handles: Vec<_> = (0..THREADS) + .map(|thread| { + let gate = &gate; + s.spawn(move || { + gate.wait(); + ask(thread) + }) + }) + .collect(); + handles.into_iter().map(|h| h.join().expect("a thread asking for the status panicked")).collect() + }); + let (runs, opens) = (eoc_meter::liveness_probe_runs(), eoc_meter::liveness_sampler_opens()); + eprintln!( + "ioreport cache: {THREADS} threads at a cold cache asked, constructed and selected and got {seen:?}; \ + probe runs in this process: {runs}, sampler opens: {opens}" + ); + let first = seen[0].0; + for (status, built, picked) in &seen { + assert_eq!(*status, first, "the threads got different statuses: {seen:?}"); + assert_eq!(*built, *status == IoReportStatus::Live, "construction disagrees with the status: {seen:?}"); + assert_eq!(*picked == "ioreport", *built, "selection disagrees with construction: {seen:?}"); + } + assert_eq!(eoc_meter::ioreport_status(), first, "the cached status changed"); + let once = usize::from(cfg!(target_os = "macos")); + assert_eq!(runs, once, "the liveness probe behind ioreport_status() ran {runs} times in this process"); + assert_eq!(opens, once, "the liveness probe's sampler was opened {opens} times in this process: a probe ran outside the cache"); + if !cfg!(target_os = "macos") { + assert_eq!(first, IoReportStatus::Unavailable); + } +} -- 2.54.0 (Apple Git-157) From 65ff45ba7584c615665f5230965d926dd01e46be Mon Sep 17 00:00:00 2001 From: Transaction Science Date: Wed, 30 Sep 2026 18:43:51 -0400 Subject: [PATCH 08/22] eoc-meter: a rail is live only if it keeps advancing, never on a two-step refresh The jouleclaw-energy verifier found the defect on 2026-09-30, on the M5 Max (macOS 27.0.1). The liveness rule said Live when the same CPU rail advanced in two separate sample intervals (LIVE_RAIL_ADVANCES = 2). That rule assumed a batch refresh lands as one step. It does not always. A watcher sampling the Energy Model group alone every ~16 ms saw `CPU Energy` step 2,643,843,378 -> 2,705,607,863 (+61,764,485), then 2,705,607,863 -> 2,705,611,298 (+3,435) 64 ms later. MCPU0, MCPU1 and PCPU each stepped twice too. The probe samples every ~31 ms, so a probe window that holds such a refresh sees two advancing intervals and says Live, and the status is cached for the process. That happened: two real probes said Live while the oracle saw no sustained movement, and both coincided with a refresh. Refreshes come ~30 minutes apart, so this is rare, but it is exactly the failure the rule exists to prevent. On this M5 Max `CPU Energy` has held at 2,705,611,298, that refresh's second step, in every run for this commit that printed it. The rule now asks for a sustained advance. A rail is live only if the same rail (same key: name + ordinal) meets three bars after some interval: - it advanced in at least MIN_LIVE_ADVANCES = 5 separate intervals; - those were at least 80% of every interval observed so far (advancing * 5 >= intervals * 4); - the samples observed so far span at least MIN_LIVE_WINDOW = 300 ms of wall time, measured from the first usable sample. It is evaluated after each interval, and Live is declared the first time all three hold. A burst of steps fails the window while it is young, and fails the share once it is not. The rule needs each sample's wall time, so RailWatch::observe now takes it: watch() reads its injected clock as each sample returns, and tests script that clock. The probe keeps sampling until the rail is Live, or until the budget (1 s) has elapsed and at least 5 intervals were observed, and never past 5 budgets. Otherwise the verdict is Frozen, Absent or Unavailable, as before. Nothing else changed: per-rail keys, consecutive-sample pairing, a decrease or a first appearance is not an advance, one advance per rail per interval, the group-only subscription, the thread-CPU tick, the pure status mapping, the OnceLock cache, the scripted leaves, try_detect_given / admits_counter, counts_once, and the "ioreport liveness: status=" line. The cost: a live rail is declared after 300 ms at the least, not ~60 ms. A frozen verdict still takes the ~1 s budget. The test oracle had the same weakness. It said CpuRailMoved(true) when one rail changed at two reads, so a refresh that coincided with a false Live vouched for it. Now it is CpuRailMoved(true) only if one rail changed at least 5 times, at no fewer than 80% of the read pairs it compared. It keeps reading past its window until it has compared 5 pairs (6 reads), so a live rail could meet the bar, and it stops at 5 windows. Too few pairs is still a test failure, never a verdict. The per-read tally is now a pure OracleTally, so the bar runs on scripted reads too. The docs that described the old two-step rule are updated: the constants, CpuRail, RailWatch, the macos_ioreport module comment, IoReportStatus::{CpuRailFrozen, Live}, CPU_RAIL_LIVENESS_BUDGET and ioreport_status. Tests. The rule tests use a scripted 31 ms interval unless noted. (*) marks the ones that must fail under the old ">= 2 advances" rule. That was checked by temporarily restoring the old rule: 13 tests failed, every (*) test among them. The F9 test failed on its two-step script: "a refresh in two steps (5 samples) must be frozen", left: Live. - (*) h_the_two_step_refresh_is_frozen: the observed +61,764,485 / +3,435 refresh is Frozen at every position tried. The steps land in intervals (3,4), (1,2), (1,3), (3,5), (20,21) and (31,33), beside a moving GPU rail and sub-rails that step with it. - (*) i_a_refresh_in_three_to_five_steps_at_the_start_is_frozen: 3, 4 or 5 steps within 150 ms at the start (30 ms a sample) are Frozen. At five steps every interval so far has advanced; only the window stops it. - (*) j_the_m3_ultra_two_steps_on_each_die_are_frozen: DIE_0 and DIE_1 each step twice within 64 ms, together or one interval apart, at the start and mid-probe, amid same-named per-die channels. Frozen. - d_a_rail_advancing_every_interval_is_live_at_five_advances_and_300_ms: not Live at 279 ms; Live at the 10th interval (310 ms), and the probe stops there. At 60 ms a sample it is Live at the 5th interval (exactly 300 ms); at 59 ms, only at the 6th. - k_a_live_rail_missing_one_interval_in_five_is_live_two_is_not_yet: missing 1 interval in every 5 is Live at 8 of 10. Missing 2 of the first 5 (100 ms a sample) is not Live at 3 of 5 or at 7 of 9; it is Live at 8 of 10. Missing 2 in every 5 is Frozen. - l_a_live_rail_on_a_loaded_host_is_live_after_five_intervals: at 100 ms intervals, Live after exactly 5 intervals (500 ms). - f_the_budget_stops_the_probe_only_once_five_intervals_are_seen: with fewer than 5 intervals the probe keeps sampling past the budget, and the 5-budget cap stops it (at 2 s ticks: 3 intervals, all advancing, Frozen). - sustained_needs_five_advances_80_percent_and_300_ms: each bar at its edge. - probe_with_the_two_step_refresh_is_frozen and probe_with_a_sustained_advance_is_live: the same rule through probe_with's fakes. The sustained case is Live at 12 intervals, 300 ms. - F9, the real host_status() on scripted IOReport leaves. The old "two advances -> Live at the third sample" script is now a sustained advance: Live after 6..=14 samples and >= 300 ms of real wall time. Also added: (*) the two-step refresh through host_status() gives CpuRailFrozen. - the_oracle_does_not_count_a_refresh_as_movement: the two-step refresh, a single step and a five-step burst are no movement; 5 of 5, 16 of 20 and every read are movement; 15 of 20 and 4 of 4 are not. - Minimums follow the rule. A frozen verdict holds at least 5 ticks, and the budget comments say five intervals. Mutations. Each was a temporary edit, then `git checkout`; the tree was clean after every one. All four were caught (tests that failed): - MIN_LIVE_ADVANCES = 2: f_..., l_..., sustained_... - 80% share check removed: a_decrease_is_not_an_advance (a rail rising and falling in turn), i_..., k_..., sustained_... - 300 ms window check removed: c_..., d_..., i_..., k_..., probe_with_a_sustained_advance_is_live, sustained_..., host_status_is_the_rule_on_the_production_composition - oracle back to ">= 2 changes": the_oracle_does_not_count_a_refresh_as_movement Evidence. - M5 Max, unloaded: eoc-meter has 45 lib tests (37 before) and the once-per-process test, all passing. `ioreport liveness: status=CpuRailFrozen oracle=CpuRailMoved(false) selected=false (oracle: 7 reads over 3.062004833s with a helper thread burning CPU; changes per CPU rail [0], per GPU rail [5], of 6 compared pairs)` - M5 Max, 3 runs of all eoc-meter tests beside a helper process sampling every IOReport channel in a tight loop: 3/3 ok, 45 + 1 passed each. The oracle read past its 3 s window to reach 5 compared pairs: `ioreport liveness: status=CpuRailFrozen oracle=CpuRailMoved(false) selected=false (oracle: 6 reads over 3.664727125s with a helper thread burning CPU; changes per CPU rail [0], per GPU rail [4], of 5 compared pairs)`, then 11 reads over 3.54 s, then 9 reads over 3.57 s. The fresh probe said Frozen every time (12-14 intervals over ~1 s). - M3 Ultra (10.10.10.2, macOS 27.0.1), 1 run of the lib and once-per-process test binaries: 45 + 1 passed. `ioreport liveness: status=CpuRailFrozen oracle=CpuRailMoved(false) selected=false (oracle: 18 reads over 3.101584s with a helper thread burning CPU; changes per CPU rail [0, 0], per GPU rail [17], of 17 compared pairs)` - The whole eoc workspace: 621 passed, 0 failed, 3 ignored. The ignored three are the pre-existing ignored doctests. - clippy --no-deps on eoc-meter, all targets: no new warnings. The only two are the pre-existing ones (a duplicated #[link] attribute, and a b"...\0" literal). The aarch64-linux cross clippy could not run here: blake3's C build finds no Linux sysroot. Co-Authored-By: Claude Opus 5.5 --- eoc/eoc-rs/crates/eoc-meter/src/lib.rs | 676 +++++++++++++++++++------ 1 file changed, 532 insertions(+), 144 deletions(-) diff --git a/eoc/eoc-rs/crates/eoc-meter/src/lib.rs b/eoc/eoc-rs/crates/eoc-meter/src/lib.rs index 835c5a020..cd41e6259 100644 --- a/eoc/eoc-rs/crates/eoc-meter/src/lib.rs +++ b/eoc/eoc-rs/crates/eoc-meter/src/lib.rs @@ -72,31 +72,57 @@ mod rail_liveness { use std::time::Duration; /// In how many separate sample intervals the same CPU rail must advance - /// before it counts as live. + /// before it can count as live; see [`sustained`] for the whole rule. /// - /// One is not enough. On macOS 27.0.1 an unprivileged reader's CPU rails - /// are not permanently frozen; they are refreshed in rare batches. An M3 - /// Ultra's two dies held for at least 22 minutes, then stepped +1,344,869 - /// and +1,391,637 mJ within one 100 ms sample, then held again; an M5 Max's - /// `CPU Energy` did not change in 13,614 samples over 25 minutes, yet had - /// moved from 480,083,160 to 541,893,638 mJ earlier that hour. A probe - /// whose window straddles a refresh sees one step. Called live, it would - /// select the counter for the life of the process, and every later refresh - /// would land, whole, in whichever caller's interval straddled it. A live - /// rail sampled every ~30 ms under load steps in nearly every interval. - pub(crate) const LIVE_RAIL_ADVANCES: u32 = 2; + /// One is not enough, and neither is two. On macOS 27.0.1 an unprivileged + /// reader's CPU rails are not permanently frozen; they are refreshed in + /// rare batches. An M3 Ultra's two dies held for at least 22 minutes, then + /// stepped +1,344,869 and +1,391,637 mJ within one 100 ms sample, then held + /// again; an M5 Max's `CPU Energy` did not change in 13,614 samples over 25 + /// minutes, yet had moved from 480,083,160 to 541,893,638 mJ earlier that + /// hour. And a refresh is not always one step: a watcher sampling the M5 + /// Max's Energy Model group every ~16 ms saw `CPU Energy` step + /// 2,643,843,378 -> 2,705,607,863 (+61,764,485), then 2,705,607,863 -> + /// 2,705,611,298 (+3,435) 64 ms later (`MCPU0`, `MCPU1` and `PCPU` each + /// stepped twice too). The probe samples every ~31 ms, so a probe whose + /// window holds such a refresh sees two advancing intervals. Called live, + /// it would select the counter for the life of the process, and every + /// later refresh would land, whole, in whichever caller's interval + /// straddled it. A live rail sampled every ~30 ms under load steps in + /// nearly every interval, for as long as it is watched. + pub(crate) const MIN_LIVE_ADVANCES: u32 = 5; + + /// How much wall time, from the first sample, the probe must have watched + /// a rail advance before it can count as live. A refresh published in + /// steps lands within ~64 ms; five or more intervals at the probe's + /// ~31 ms can all fall inside a burst that short, and would then be every + /// interval observed. Over 300 ms of sustained advance they cannot. + pub(crate) const MIN_LIVE_WINDOW: Duration = Duration::from_millis(300); /// However slow sampling is, the probe stops after this many budgets. pub(crate) const PROBE_BUDGET_CAP: u32 = 5; + /// The liveness rule for one rail, evaluated after each interval: it + /// advanced in at least [`MIN_LIVE_ADVANCES`] separate intervals, those + /// were at least 80% of every interval observed so far, and the samples + /// observed so far span at least [`MIN_LIVE_WINDOW`] of wall time from the + /// first. A burst of steps — a refresh — fails the window while it is + /// young and the share once it is not; a live rail passes all three. + pub(crate) fn sustained(advances: u32, intervals: u32, window: Duration) -> bool { + advances >= MIN_LIVE_ADVANCES + && u64::from(advances) * 5 >= u64::from(intervals) * 4 + && window >= MIN_LIVE_WINDOW + } + /// What the liveness probe saw of the CPU energy rail. #[derive(Debug, Clone, Copy, PartialEq, Eq)] pub(crate) enum CpuRail { - /// The same CPU energy rail advanced in at least - /// [`LIVE_RAIL_ADVANCES`] separate sample intervals. + /// The same CPU energy rail kept advancing, by [`sustained`]: in at + /// least [`MIN_LIVE_ADVANCES`] separate intervals, at least 80% of + /// those observed, over at least [`MIN_LIVE_WINDOW`]. Live, - /// A CPU energy rail exists, but none advanced in that many intervals: - /// it held, or stepped once (a batch refresh). + /// A CPU energy rail exists, but none kept advancing: it held, or + /// stepped only in a burst (a batch refresh, in one step or several). Frozen, /// Samples were taken, and no `CPU Energy` channel was in them. Absent, @@ -164,26 +190,38 @@ mod rail_liveness { /// conversion (a small advance in a fine unit cannot truncate to zero). A /// rail advances in an interval when its value rose over that interval; so /// a rail advances at most once per interval, and each rail is counted on - /// its own: two dies stepping once each never add up to a live rail. The + /// its own: two dies stepping twice each never add up to a live rail. The /// summed total is never consulted, because the GPU rail, which is live on - /// an M5 Max, is exactly what corrupts it. + /// an M5 Max, is exactly what corrupts it. After each interval every rail + /// is put to [`sustained`], over the intervals observed so far and the wall + /// time since the first usable sample; the first time one passes, the watch + /// says Live, and stays Live. #[derive(Debug, Default)] pub(crate) struct RailWatch { /// The CPU rails of the last usable sample. prev: Option>, + /// The wall-clock reading of the first usable sample. + first: Option, /// Intervals observed: pairs of consecutive usable samples. pub(crate) intervals: u32, + /// Wall time from the first usable sample to the last. + pub(crate) window: Duration, /// Every CPU rail seen in an interval, with the number of intervals in /// which it advanced. pub(crate) rails: Vec<(RailKey, u32)>, + /// Whether one rail has passed [`sustained`] after some interval. + live: bool, } impl RailWatch { /// Fold in one usable sample: every channel in it, as (name, absolute - /// raw integer). A sample that could not be taken is simply not - /// observed, so the next usable one pairs with the last usable one. - pub(crate) fn observe(&mut self, sample: &[(String, i64)]) { + /// raw integer), and `at`, the wall-clock reading when the sample was + /// in hand. A sample that could not be taken is simply not observed, + /// so the next usable one pairs with the last usable one. + pub(crate) fn observe(&mut self, sample: &[(String, i64)], at: Duration) { let cur = cpu_rails(sample); + let first = *self.first.get_or_insert(at); + self.window = at.saturating_sub(first); if let Some(prev) = &self.prev { self.intervals += 1; for (key, now) in &cur { @@ -196,24 +234,27 @@ mod rail_liveness { None => self.rails.push((key.clone(), u32::from(advanced))), } } + let (intervals, window) = (self.intervals, self.window); + self.live |= self.rails.iter().any(|(_, n)| sustained(*n, intervals, window)); } self.prev = Some(cur); } - /// Has one CPU rail advanced in enough separate intervals to be live? + /// Has one CPU rail kept advancing, by [`sustained`], after some + /// interval observed so far? pub(crate) fn is_live(&self) -> bool { - self.rails.iter().any(|(_, n)| *n >= LIVE_RAIL_ADVANCES) + self.live } /// Keep sampling? Until the rail proves live; or until `budget` has - /// elapsed **and** enough intervals were observed for a live rail to - /// have advanced [`LIVE_RAIL_ADVANCES`] times (slow or failed samples - /// must not make a live rail look frozen); and never past - /// [`PROBE_BUDGET_CAP`] budgets. + /// elapsed **and** at least [`MIN_LIVE_ADVANCES`] intervals were + /// observed, enough for a live rail to have shown its advances (slow or + /// failed samples must not make a live rail look frozen); and never + /// past [`PROBE_BUDGET_CAP`] budgets. pub(crate) fn wants_more(&self, elapsed: Duration, budget: Duration) -> bool { !self.is_live() && elapsed < budget * PROBE_BUDGET_CAP - && (elapsed < budget || self.intervals < LIVE_RAIL_ADVANCES) + && (elapsed < budget || self.intervals < MIN_LIVE_ADVANCES) } /// The verdict, or `None` when not one interval could be observed. @@ -234,8 +275,9 @@ mod rail_liveness { /// samples, and the clock, so it runs under test exactly as it runs on /// hardware. Takes a sample, then ticks and samples again for as long as /// [`RailWatch::wants_more`] says. `sample` returns `None` when no sample - /// could be taken. [`probe_with`] runs it with the sampler, tick and wall - /// clock it is given, and records each sample call. + /// could be taken; each one taken is handed to the rule with the wall + /// clock `now` read as it returns. [`probe_with`] runs it with the + /// sampler, tick and wall clock it is given, and records each sample call. pub(crate) fn watch( mut sample: impl FnMut() -> Option>, mut tick: impl FnMut(), @@ -245,12 +287,12 @@ mod rail_liveness { let started = now(); let mut w = RailWatch::default(); if let Some(s) = sample() { - w.observe(&s); + w.observe(&s, now()); } while w.wants_more(now().saturating_sub(started), budget) { tick(); if let Some(s) = sample() { - w.observe(&s); + w.observe(&s, now()); } } w @@ -414,6 +456,41 @@ mod rail_liveness { const TICK: Duration = Duration::from_millis(25); const BUDGET: Duration = Duration::from_millis(1000); + /// One probe interval on an unloaded M5 Max: a 25 ms tick and a ~6 ms + /// Energy Model sample. + const INTERVAL: Duration = Duration::from_millis(31); + + /// The observed two-step refresh of the M5 Max's `CPU Energy`, as a + /// probe sampling every interval sees it when the steps land in + /// intervals `first` and `second` (`first` < `second`): held, stepped + /// +61,764,485, stepped +3,435, held. + fn two_step_refresh(i: usize, first: usize, second: usize) -> i64 { + let a = 2_643_843_378; + if i < first { + a + } else if i < second { + a + 61_764_485 + } else { + a + 61_764_485 + 3_435 + } + } + + /// Fold samples `0..n` straight into a [`RailWatch`], the i-th stamped + /// `i * step` on the wall clock. Returns the index of the sample after + /// which the watch first said Live, if any, and the watch. + fn first_live(step: Duration, n: usize, generate: impl Fn(usize) -> Vec<(&'static str, i64)>) -> (Option, RailWatch) { + let mut w = RailWatch::default(); + let mut live_at = None; + for i in 0..n { + let s: Vec<(String, i64)> = generate(i).into_iter().map(|(n, raw)| (n.to_string(), raw)).collect(); + w.observe(&s, step * i as u32); + if w.is_live() && live_at.is_none() { + live_at = Some(i); + } + } + (live_at, w) + } + /// Run [`watch`] over scripted samples — `gen(i)` is the i-th sample, /// `None` for one that could not be taken — with a clock that only the /// tick moves, by `step`. Returns the watch and how many samples it took. @@ -463,7 +540,7 @@ mod rail_liveness { }); assert_eq!(w.verdict(), Some(CpuRail::Frozen), "one refresh counted as a live rail: {w:?}"); assert_eq!(w.rails, vec![(key("CPU Energy", 0), 1)], "the refresh is one advance of one rail"); - // The budget is 40 ticks; with two intervals long since seen, it stops there. + // The budget is 40 ticks; with five intervals long since seen, it stops there. assert_eq!((w.intervals, taken), (40, 41)); } @@ -502,24 +579,169 @@ mod rail_liveness { }); assert_eq!(w.verdict(), Some(CpuRail::Frozen), "two same-named rails stepping once each counted as live: {w:?}"); - // The first of two same-named rails is live; the second holds. + // The first of two same-named rails is live; the second holds. At + // 25 ms a sample, the 300 ms window is what it waits for: 12 intervals. let (w, taken) = scripted(TICK, |i| Some(vec![("CPU Energy", 100 + i as i64), ("CPU Energy", 200)])); assert_eq!(w.verdict(), Some(CpuRail::Live), "{w:?}"); - assert_eq!((w.intervals, taken), (2, 3)); + assert_eq!((w.intervals, taken), (12, 13)); + assert_eq!(w.rails, vec![(key("CPU Energy", 0), 12), (key("CPU Energy", 1), 0)]); } - /// (d) A rail stepping at every sample is live after exactly two - /// intervals, and sampling stops right there. + /// (d) A rail advancing in every interval, sampled every 31 ms, is not + /// Live at its fifth advance (155 ms), nor at its ninth (279 ms): only + /// at the tenth, once the window reaches 300 ms (310 ms). Sampling stops + /// right there. Five advances and the window are both needed, and the + /// window is inclusive: at 60 ms a sample, Live at the fifth interval + /// (exactly 300 ms); at 59 ms, not until the sixth (295 ms is short). #[test] - fn d_a_rail_stepping_every_sample_is_live_after_two_intervals() { - let (w, taken) = scripted(TICK, |i| Some(vec![("GPU Energy", 5), ("CPU Energy", 1_000 + 13 * i as i64)])); + fn d_a_rail_advancing_every_interval_is_live_at_five_advances_and_300_ms() { + let live = |i: usize| vec![("GPU Energy", 5), ("CPU Energy", 1_000 + 13 * i as i64)]; + let (at, w) = first_live(INTERVAL, 40, live); + assert_eq!(at, Some(10), "a rail advancing every 31 ms must be Live at the 10th interval (310 ms), not before: {w:?}"); + let (at, w) = first_live(INTERVAL, 10, live); + assert_eq!((at, w.intervals, w.window), (None, 9, Duration::from_millis(279)), "Live before 300 ms: {w:?}"); + assert_eq!(w.rails, vec![(key("CPU Energy", 0), 9)]); + + let (w, taken) = scripted(INTERVAL, |i| Some(live(i))); assert_eq!(w.verdict(), Some(CpuRail::Live)); - assert_eq!((w.intervals, taken), (2, 3), "a live rail must stop the probe at its second advance"); - assert_eq!(w.rails, vec![(key("CPU Energy", 0), 2)]); - // Two advances need not be in consecutive intervals. - let (w, taken) = scripted(TICK, |i| Some(vec![("CPU Energy", [0, 0, 0, 4, 4, 4, 4, 4, 4, 9][i.min(9)])])); - assert_eq!(w.verdict(), Some(CpuRail::Live)); - assert_eq!((w.intervals, taken), (9, 10)); + assert_eq!((w.intervals, taken), (10, 11), "a live rail must stop the probe as soon as the rule holds"); + assert_eq!((w.rails.clone(), w.window), (vec![(key("CPU Energy", 0), 10)], Duration::from_millis(310))); + + assert_eq!(first_live(Duration::from_millis(60), 20, live).0, Some(5), "300 ms is enough"); + assert_eq!(first_live(Duration::from_millis(59), 20, live).0, Some(6), "295 ms is not"); + } + + /// The rule's three bars, at their edges: five advances, 80% of the + /// intervals observed, 300 ms of window. + #[test] + fn sustained_needs_five_advances_80_percent_and_300_ms() { + let ms = Duration::from_millis; + assert!(sustained(5, 5, ms(300))); + assert!(!sustained(4, 4, ms(1000)), "four advances"); + assert!(!sustained(5, 5, ms(299)), "299 ms"); + assert!(sustained(5, 6, ms(300)), "5 of 6 is 83%"); + assert!(!sustained(5, 7, ms(1000)), "5 of 7 is 71%"); + assert!(sustained(8, 10, ms(300)), "8 of 10 is 80%"); + assert!(!sustained(7, 9, ms(1000)), "7 of 9 is 78%"); + assert!(sustained(800, 1000, ms(5000)) && !sustained(799, 1000, ms(5000))); + assert!(!sustained(2, 2, ms(1000)), "two advances: the old rule's Live"); + } + + /// (h) The observed macOS 27 refresh published in two steps, +61,764,485 + /// then +3,435 (64 ms apart, so in consecutive or next-but-one 31 ms + /// intervals), wherever it falls in the probe, beside a moving GPU rail + /// and sub-rails that step with it: two advances, Frozen. The old rule + /// (two advancing intervals) called every one of these Live. + #[test] + fn h_the_two_step_refresh_is_frozen() { + for (first, second) in [(3, 4), (1, 2), (1, 3), (3, 5), (20, 21), (31, 33)] { + let sample = |i: usize| { + let cpu = two_step_refresh(i, first, second); + vec![ + ("CPU Energy", cpu), + ("MCPU0", cpu / 4), + ("MCPU1", cpu / 4), + ("PCPU", cpu / 2), + ("GPU Energy", 1_000 + 7 * i as i64), + ] + }; + let (w, taken) = scripted(INTERVAL, |i| Some(sample(i))); + assert_eq!( + w.verdict(), + Some(CpuRail::Frozen), + "the two-step refresh in intervals {first} and {second} counted as a live rail: {w:?}" + ); + assert_eq!(w.rails, vec![(key("CPU Energy", 0), 2)], "steps in intervals {first} and {second}"); + assert_eq!((w.intervals, taken), (33, 34), "the budget is 33 intervals of 31 ms"); + assert_eq!(first_live(INTERVAL, 200, sample).0, None, "steps in intervals {first} and {second}"); + } + } + + /// (i) A refresh published in three, four or five steps within 150 ms + /// at the very start of the probe (30 ms a sample): while it lasts it is + /// every interval observed, and at five steps it has five advances, but + /// the window is under 300 ms; by the time it is not, the rail has held + /// long enough that its advances are under 80% of the intervals. Frozen. + #[test] + fn i_a_refresh_in_three_to_five_steps_at_the_start_is_frozen() { + let step = Duration::from_millis(30); + let sizes = [61_764_485, 3_435, 1_207, 88, 19]; + for steps in 3..=5usize { + let sample = |i: usize| vec![("CPU Energy", 2_643_843_378 + sizes[..i.min(steps)].iter().sum::())]; + let (w, _) = scripted(step, |i| Some(sample(i))); + assert_eq!(w.verdict(), Some(CpuRail::Frozen), "a refresh in {steps} steps counted as a live rail: {w:?}"); + assert_eq!(w.rails, vec![(key("CPU Energy", 0), steps as u32)]); + assert_eq!(first_live(step, 200, sample).0, None, "a refresh in {steps} steps"); + } + } + + /// (j) The M3 Ultra's shape: `DIE_0_CPU Energy` and `DIE_1_CPU Energy` + /// each step twice within 64 ms (two consecutive 31 ms intervals), + /// together or one interval apart, amid unprefixed per-die channels that + /// repeat a name; at the start of the probe and inside it. Two advances + /// per die: Frozen. The old rule called each die Live. + #[test] + fn j_the_m3_ultra_two_steps_on_each_die_are_frozen() { + for (d0, d1) in [(1, 1), (1, 2), (6, 6), (6, 7)] { + let sample = |i: usize| { + let steps = |from: usize| i64::from(i >= from) * 1_344_869 + i64::from(i > from) * 2_113; + vec![ + ("DIE_0_CPU Energy", 9_494_336 + steps(d0)), + ("ECPUDTL00", 10 * i as i64), + ("DIE_1_CPU Energy", 10_212_656 + steps(d1)), + ("ECPUDTL00", 11 * i as i64), + ("GPU Energy", 3 * i as i64), + ] + }; + let (w, _) = scripted(INTERVAL, |i| Some(sample(i))); + assert_eq!(w.verdict(), Some(CpuRail::Frozen), "two steps on each die ({d0}, {d1}) counted as live: {w:?}"); + assert_eq!(w.rails, vec![(key("DIE_0_CPU Energy", 0), 2), (key("DIE_1_CPU Energy", 0), 2)]); + assert_eq!(first_live(INTERVAL, 200, sample).0, None, "dies stepping from {d0} and {d1}"); + } + } + + /// (k) A live rail that misses one interval in every five advances in + /// exactly 80% of them: Live, at the tenth interval (8 of 10, 310 ms). + /// One that missed two of its first five is not yet Live at the fifth + /// (3 of 5), nor at the ninth (7 of 9, 78%), though it advanced in every + /// interval since; it is at the tenth (8 of 10) — here sampled every + /// 100 ms, so the window never binds. And one that misses two in every + /// five (60%) is never Live: Frozen. + #[test] + fn k_a_live_rail_missing_one_interval_in_five_is_live_two_is_not_yet() { + let miss_one = |i: usize| vec![("CPU Energy", (1..=i).filter(|k| k % 5 != 0).count() as i64)]; + let (at, w) = first_live(INTERVAL, 40, miss_one); + assert_eq!(at, Some(10), "missing one in five must be Live at 8 of 10 intervals: {w:?}"); + let (w, taken) = scripted(INTERVAL, |i| Some(miss_one(i))); + assert_eq!((w.verdict(), w.intervals, taken), (Some(CpuRail::Live), 10, 11)); + assert_eq!(w.rails, vec![(key("CPU Energy", 0), 8)]); + + let miss_first_two = |i: usize| vec![("CPU Energy", (1..=i).filter(|k| *k != 1 && *k != 3).count() as i64)]; + let step = Duration::from_millis(100); + for n in [6, 10] { + let (at, w) = first_live(step, n, miss_first_two); + assert_eq!(at, None, "Live at {} of {} intervals: {w:?}", w.rails[0].1, w.intervals); + } + let (at, w) = first_live(step, 40, miss_first_two); + assert_eq!(at, Some(10), "8 of 10 intervals must be Live: {w:?}"); + + let miss_two = |i: usize| vec![("CPU Energy", (1..=i).filter(|k| k % 5 != 0 && k % 5 != 2).count() as i64)]; + let (w, _) = scripted(INTERVAL, |i| Some(miss_two(i))); + assert_eq!(w.verdict(), Some(CpuRail::Frozen), "a rail advancing in 60% of intervals counted as live: {w:?}"); + assert_eq!(first_live(INTERVAL, 1000, miss_two).0, None); + } + + /// (l) A loaded host, where each interval takes 100 ms: a live rail is + /// Live after exactly five intervals (500 ms). By 300 ms it has only + /// three advances. Sampling stops right there. + #[test] + fn l_a_live_rail_on_a_loaded_host_is_live_after_five_intervals() { + let step = Duration::from_millis(100); + let live = |i: usize| vec![("CPU Energy", 1_000 + 13 * i as i64)]; + let (at, w) = first_live(step, 40, live); + assert_eq!(at, Some(5), "{w:?}"); + let (w, taken) = scripted(step, |i| Some(live(i))); + assert_eq!((w.verdict(), w.intervals, taken, w.window), (Some(CpuRail::Live), 5, 6, Duration::from_millis(500))); } /// (e) Samples that cannot be taken keep the probe sampling to its cap of @@ -535,23 +757,35 @@ mod rail_liveness { assert_eq!((w.verdict(), w.intervals, taken), (None, 0, 201)); } - /// (f) Past the budget, the probe stops at once if it has seen two - /// intervals; with fewer it keeps sampling until it has two, or hits the cap. + /// (f) Past the budget, the probe stops at once if it has seen five + /// intervals; with fewer it keeps sampling until it has five, or hits + /// the cap of five budgets. #[test] - fn f_the_budget_stops_the_probe_only_once_two_intervals_are_seen() { - // Every tick jumps past the budget: it still takes three samples. - let (w, taken) = scripted(BUDGET + Duration::from_millis(1), |_| Some(vec![("CPU Energy", 42)])); + fn f_the_budget_stops_the_probe_only_once_five_intervals_are_seen() { + // Every tick takes 600 ms: past the budget after two, it still + // takes six samples, and stops at the fifth interval (3 s). + let (w, taken) = scripted(Duration::from_millis(600), |_| Some(vec![("CPU Energy", 42)])); assert_eq!(w.verdict(), Some(CpuRail::Frozen)); - assert_eq!((w.intervals, taken), (2, 3)); + assert_eq!((w.intervals, taken, w.window), (5, 6, Duration::from_millis(3000))); + // A live rail sampled that slowly is Live at its fifth advance. + let (w, taken) = scripted(Duration::from_millis(600), |i| Some(vec![("CPU Energy", i as i64)])); + assert_eq!((w.verdict(), w.intervals, taken), (Some(CpuRail::Live), 5, 6)); // One interval, then failures: past the budget it keeps sampling, to the cap. let (w, taken) = scripted(TICK, |i| (i < 2).then(|| vec![("CPU Energy", 42)])); assert_eq!((w.verdict(), w.intervals, taken), (Some(CpuRail::Frozen), 1, 201)); - // The second interval arrives at 2.5 s, past the budget: it stops right there. - let (w, taken) = scripted(TICK, |i| (i < 2 || i == 100).then(|| vec![("CPU Energy", 42)])); - assert_eq!((w.verdict(), w.intervals, taken), (Some(CpuRail::Frozen), 2, 101)); - // Within the budget, two intervals are not enough to stop. + // Four intervals, then failures: still to the cap. + let (w, taken) = scripted(TICK, |i| (i < 5).then(|| vec![("CPU Energy", i as i64)])); + assert_eq!((w.verdict(), w.intervals, taken), (Some(CpuRail::Frozen), 4, 201)); + // The fifth interval arrives at 2.5 s, past the budget: it stops right there. + let (w, taken) = scripted(TICK, |i| (i < 5 || i == 100).then(|| vec![("CPU Energy", 42)])); + assert_eq!((w.verdict(), w.intervals, taken), (Some(CpuRail::Frozen), 5, 101)); + // Within the budget, five intervals are not enough to stop. let (w, taken) = scripted(Duration::from_millis(100), |_| Some(vec![("CPU Energy", 42)])); assert_eq!((w.intervals, taken), (10, 11)); + // A tick of 2 s: the cap (5 s) stops it at three intervals, short of + // five, and a rail advancing in all three is still not Live. + let (w, taken) = scripted(Duration::from_secs(2), |i| Some(vec![("CPU Energy", i as i64)])); + assert_eq!((w.verdict(), w.intervals, taken), (Some(CpuRail::Frozen), 3, 4)); } /// (g) DIE_0 steps once and DIE_1 steps once, in different intervals: @@ -578,8 +812,9 @@ mod rail_liveness { /// A rail that goes DOWN between consecutive samples — a reset or a /// wrap — has not advanced. A rail falling at every sample is Frozen - /// with no advance at all; and one real step followed by a reset is - /// one advance, not the two a live rail needs. + /// with no advance at all; one real step followed by a reset is one + /// advance; and a rail that alternately rises and falls advances in + /// only half of its intervals, which is not the 80% a live rail needs. #[test] fn a_decrease_is_not_an_advance() { let (w, _) = scripted(TICK, |i| Some(vec![("CPU Energy", 1_000 - i as i64)])); @@ -589,6 +824,10 @@ mod rail_liveness { let (w, _) = scripted(TICK, |i| Some(vec![("CPU Energy", [500, 500, 500, 900, 900, 900, 10][i.min(6)])])); assert_eq!(w.verdict(), Some(CpuRail::Frozen), "one step and one reset counted as live: {w:?}"); assert_eq!(w.rails, vec![(key("CPU Energy", 0), 1)], "the reset counted as an advance"); + + let (w, _) = scripted(INTERVAL, |i| Some(vec![("CPU Energy", 500 + (i % 2) as i64)])); + assert_eq!(w.verdict(), Some(CpuRail::Frozen), "a rail rising and falling in turn counted as live: {w:?}"); + assert_eq!(w.rails, vec![(key("CPU Energy", 0), 17)], "each fall counted as an advance"); } /// A rail absent from the previous sample only sets its baseline in @@ -746,14 +985,30 @@ mod rail_liveness { } } - /// Two advances of the same CPU rail, in two intervals: Live, and the - /// probe stops right there. + /// The same CPU rail advancing in every interval, 25 ms of wall clock + /// apart: Live once the rule holds, at the 12th interval (300 ms), and + /// the probe stops right there. The window the rule saw is the + /// composition's own wall clock, from the first sample to the last. #[test] - fn probe_with_two_advances_is_live() { + fn probe_with_a_sustained_advance_is_live() { let f = fake(true, TICK, |i| Some(group_sample(&[("GPU Energy", 5), ("CPU Energy", 1_000 + 13 * i as i64)]))); assert_eq!(f.run.verdict(), Some(CpuRail::Live)); assert_eq!(status_of(f.run.verdict()), crate::IoReportStatus::Live); - assert_eq!((f.samples, f.ticks), (3, 2), "a live rail must stop the probe at its second advance"); + assert_eq!((f.samples, f.ticks), (13, 12), "a live rail must stop the probe as soon as the rule holds"); + assert_eq!((f.run.watch.intervals, f.run.watch.window), (12, Duration::from_millis(300))); + } + + /// The observed two-step refresh through the whole composition, beside + /// a moving GPU rail: Frozen, after the whole budget. + #[test] + fn probe_with_the_two_step_refresh_is_frozen() { + let f = fake(true, TICK, |i| { + Some(group_sample(&[("CPU Energy", two_step_refresh(i, 3, 4)), ("GPU Energy", 1_000 + 7 * i as i64)])) + }); + assert_eq!(f.run.verdict(), Some(CpuRail::Frozen), "the two-step refresh counted as live: {:?}", f.run.watch); + assert_eq!(status_of(f.run.verdict()), crate::IoReportStatus::CpuRailFrozen); + assert_eq!(f.run.watch.rails, vec![(key("CPU Energy", 0), 2)]); + assert_eq!((f.samples, f.ticks), (41, 40)); } /// Two dies, each stepping once, in different intervals: Frozen, and @@ -847,14 +1102,16 @@ mod rail_liveness { // `MCPU*`, `PACC_*`, `DIE__CPU Energy`) hold their value under full CPU load // on both an M5 Max and an M3 Ultra, and are refreshed only now and then (an M3 // Ultra's held for at least 22 minutes, then stepped by ~1.3 kJ per die in one -// sample), while `GPU Energy` still moves (steadily on the M5 Max, +// sample; an M5 Max's `CPU Energy` has been seen to publish one refresh in two +// steps 64 ms apart), while `GPU Energy` still moves (steadily on the M5 Max, // intermittently on the M3 Ultra). Summed, the rails report GPU-only energy, // zero, or a refresh of past CPU energy, and consumers such as eoc-cascade, // eoc-local and eoc-streaming tag what a hardware counter returns as Measured. // So the counter is constructed only when `ioreport_status()` is `Live` — the -// same CPU rail advanced in two separate sample intervals while the probing -// thread burned CPU — and otherwise `try_detect()` returns `Err(NoCounter)`, -// leaving the caller's own estimate. +// same CPU rail kept advancing while the probing thread burned CPU: in at least +// five separate sample intervals, at least 80% of those observed, over at least +// 300 ms — and otherwise `try_detect()` returns `Err(NoCounter)`, leaving the +// caller's own estimate. #[cfg(target_os = "macos")] #[allow(non_camel_case_types, unsafe_code)] mod macos_ioreport { @@ -1737,33 +1994,100 @@ mod macos_ioreport { struct OracleRun { verdict: Oracle, reads: u32, + /// Pairs of consecutive reads compared since the last baseline: + /// what the per-rail change counts are out of. + compared: u32, took: Duration, cpu_changes: Vec, gpu_changes: Vec, } - /// The fewest reads in which one rail can change twice, the oracle's - /// bar for a moving rail. The oracle reads past its window until it has - /// this many. - const ORACLE_MIN_READS: u32 = 3; + /// The fewest changes at which one rail counts as moving, and so the + /// fewest pairs of consecutive reads the oracle compares (six reads) + /// before its verdict can say anything: it reads past its window until + /// it has this many, so a live rail, changing at every read, can meet + /// its bar. Its own literal, not the implementation's constant. + const ORACLE_MIN_CHANGES: u32 = 5; /// However slow its reads, the oracle stops after this many windows. const ORACLE_WINDOW_CAP: u32 = 5; + /// The oracle's bar for a moving rail, over `compared` pairs of + /// consecutive reads: one rail changed at least [`ORACLE_MIN_CHANGES`] + /// times, at no fewer than 80% of them. A live rail changes at nearly + /// every read. A refresh — one step, or two 64 ms apart, or a few in a + /// burst — changes at a handful of reads in a window of 0.1-0.8 s reads, + /// so it cannot vouch for a Live verdict it happened to coincide with. + fn oracle_moved(cpu_changes: &[u32], compared: u32) -> bool { + cpu_changes.iter().any(|&n| n >= ORACLE_MIN_CHANGES && u64::from(n) * 5 >= u64::from(compared) * 4) + } + + /// What the oracle has counted, read by read: per rail, at how many + /// reads its value changed since the read before, and out of how many + /// pairs of reads compared since the last baseline. Pure, so the bar + /// runs under test on scripted reads. + #[derive(Debug, Default)] + struct OracleTally { + prev: Option<(Vec, Vec)>, + saw_cpu_rail: bool, + compared: u32, + cpu_changes: Vec, + gpu_changes: Vec, + } + + impl OracleTally { + /// Fold in one read: every channel whose name ends with `CPU + /// Energy` or `GPU Energy`, in sample order. + fn fold(&mut self, chans: Vec) { + let names = |v: &[OracleChannel]| v.iter().map(|c| c.name.clone()).collect::>(); + let (cpu, gpu): (Vec<_>, Vec<_>) = chans.into_iter().partition(|c| c.name.ends_with("CPU Energy")); + self.saw_cpu_rail |= !cpu.is_empty(); + match &self.prev { + Some((pc, pg)) if names(pc) == names(&cpu) && names(pg) == names(&gpu) => { + self.compared += 1; + for (n, (now, before)) in self.cpu_changes.iter_mut().zip(cpu.iter().zip(pc)) { + *n += u32::from(now.raw != before.raw); + } + for (n, (now, before)) in self.gpu_changes.iter_mut().zip(gpu.iter().zip(pg)) { + *n += u32::from(now.raw != before.raw); + } + } + // The first read, or rails that differ from the last read's: a new baseline. + _ => { + self.compared = 0; + self.cpu_changes = vec![0; cpu.len()]; + self.gpu_changes = vec![0; gpu.len()]; + } + } + self.prev = Some((cpu, gpu)); + } + + fn verdict(&self) -> Oracle { + if self.saw_cpu_rail { + Oracle::CpuRailMoved(oracle_moved(&self.cpu_changes, self.compared)) + } else { + Oracle::NoCpuRail + } + } + } + impl OracleRun { /// The oracle's evidence is only as good as its reads. Fewer than - /// three cannot show a rail changing twice, so they say nothing about - /// whether the CPU rail is frozen: that is a test FAILURE, never a - /// Frozen verdict. With no subscription there are no reads to have. + /// five compared pairs cannot show a rail changing five times, so + /// they say nothing about whether the CPU rail is frozen: that is a + /// test FAILURE, never a Frozen verdict. With no subscription there + /// are no reads to have. fn assert_evidence(&self) { if self.verdict == Oracle::NoIoReport { return; } assert!( - self.reads >= ORACLE_MIN_READS, - "the oracle completed only {} reads in {:?}, though it keeps reading past its window until \ - it has {ORACLE_MIN_READS}, for up to {ORACLE_WINDOW_CAP} windows: too few to see any rail \ - change twice, so there is no evidence about the CPU rail either way", + self.compared >= ORACLE_MIN_CHANGES, + "the oracle compared only {} pairs of consecutive reads ({} reads) in {:?}, though it keeps \ + reading past its window until it has {ORACLE_MIN_CHANGES}, for up to {ORACLE_WINDOW_CAP} \ + windows: too few for any rail to change {ORACLE_MIN_CHANGES} times, so there is no evidence \ + about the CPU rail either way", + self.compared, self.reads, self.took ); @@ -1777,20 +2101,19 @@ mod macos_ioreport { /// It reads its own all-channel subscription over and over, matches /// names with `ends_with("CPU Energy")` in any group (not the /// implementation's group filter and die-prefix stripping), and counts - /// per rail at how many reads the value changed since the read before. - /// `CpuRailMoved(true)` only if one CPU rail changed at two or more - /// reads: a live rail changes at nearly every read, a batch-refreshed one - /// at most once in a window this short, and one change is not a live - /// reading. Its own literal threshold, not the implementation's constant. - /// It reads until the window has passed AND it has - /// [`ORACLE_MIN_READS`] reads, for at most [`ORACLE_WINDOW_CAP`] - /// windows, so slow reads on a loaded host lengthen the window rather - /// than thin the evidence. + /// per rail at how many reads the value changed since the read before + /// ([`OracleTally`]). `CpuRailMoved(true)` only by [`oracle_moved`]: one + /// CPU rail changed at 80% or more of the compared reads, and at least + /// five times. It reads until the window has passed AND it has compared + /// [`ORACLE_MIN_CHANGES`] pairs of reads, for at most + /// [`ORACLE_WINDOW_CAP`] windows, so slow reads on a loaded host + /// lengthen the window rather than thin the evidence. fn oracle_cpu_rail(window: Duration) -> OracleRun { let Some(sub) = OracleSub::open() else { return OracleRun { verdict: Oracle::NoIoReport, reads: 0, + compared: 0, took: Duration::ZERO, cpu_changes: Vec::new(), gpu_changes: Vec::new(), @@ -1805,46 +2128,80 @@ mod macos_ioreport { } }); let started = Instant::now(); - let (mut reads, mut saw_cpu_rail) = (0u32, false); - let (mut cpu_changes, mut gpu_changes) = (Vec::new(), Vec::new()); - let mut prev: Option<(Vec, Vec)> = None; - let names = |v: &[OracleChannel]| v.iter().map(|c| c.name.clone()).collect::>(); - while (started.elapsed() < window || reads < ORACLE_MIN_READS) + let (mut reads, mut tally) = (0u32, OracleTally::default()); + while (started.elapsed() < window || tally.compared < ORACLE_MIN_CHANGES) && started.elapsed() < window * ORACLE_WINDOW_CAP { let Some(chans) = sub.read() else { continue; }; reads += 1; - let (cpu, gpu): (Vec<_>, Vec<_>) = chans.into_iter().partition(|c| c.name.ends_with("CPU Energy")); - saw_cpu_rail |= !cpu.is_empty(); - match &prev { - Some((pc, pg)) if names(pc) == names(&cpu) && names(pg) == names(&gpu) => { - for (n, (now, before)) in cpu_changes.iter_mut().zip(cpu.iter().zip(pc)) { - *n += u32::from(now.raw != before.raw); - } - for (n, (now, before)) in gpu_changes.iter_mut().zip(gpu.iter().zip(pg)) { - *n += u32::from(now.raw != before.raw); - } - } - // The first read, or rails that differ from the last read's: a new baseline. - _ => { - cpu_changes = vec![0; cpu.len()]; - gpu_changes = vec![0; gpu.len()]; - } - } - prev = Some((cpu, gpu)); + tally.fold(chans); } stop.store(true, Ordering::Relaxed); - let verdict = if saw_cpu_rail { - Oracle::CpuRailMoved(cpu_changes.iter().any(|&n| n >= 2)) - } else { - Oracle::NoCpuRail - }; - OracleRun { verdict, reads, took: started.elapsed(), cpu_changes, gpu_changes } + OracleRun { + verdict: tally.verdict(), + reads, + compared: tally.compared, + took: started.elapsed(), + cpu_changes: tally.cpu_changes, + gpu_changes: tally.gpu_changes, + } }) } + /// A scripted oracle read: `CPU Energy` at `cpu` and a `GPU Energy` + /// that moves at every read. + fn oracle_read(i: usize, cpu: i64) -> Vec { + let chan = |name: &str, raw: i64| OracleChannel { + group: "Energy Model".to_string(), + name: name.to_string(), + unit: "mJ".to_string(), + raw, + }; + vec![chan("CPU Energy", cpu), chan("GPU Energy", 1_000 + 7 * i as i64)] + } + + /// The oracle cannot vouch for a Live verdict with a refresh. Over a + /// window of scripted reads, the observed two-step refresh (+61,764,485 + /// then +3,435), a single step, and a five-step burst are no movement; + /// a rail changing at every read, or at four reads in five, is; one + /// changing at three reads in four is not, nor one that changed at + /// every one of only four compared pairs. The old bar (two changes) + /// called the two-step refresh movement. + #[test] + fn the_oracle_does_not_count_a_refresh_as_movement() { + let tally = |reads: usize, cpu: &dyn Fn(usize) -> i64| { + let mut t = OracleTally::default(); + for i in 0..reads { + t.fold(oracle_read(i, cpu(i))); + } + t + }; + let a = 2_643_843_378; + let two_step = |i: usize| if i < 4 { a } else if i < 5 { a + 61_764_485 } else { a + 61_764_485 + 3_435 }; + let t = tally(12, &two_step); + assert_eq!( + (t.verdict(), t.compared, t.cpu_changes.clone(), t.gpu_changes.clone()), + (Oracle::CpuRailMoved(false), 11, vec![2], vec![11]), + "the oracle counted the two-step refresh as a moving CPU rail" + ); + let one_step = |i: usize| if i < 4 { a } else { a + 61_764_485 }; + assert_eq!(tally(12, &one_step).verdict(), Oracle::CpuRailMoved(false)); + let burst = |i: usize| a + 100 * (i.clamp(3, 8) as i64 - 3); + let t = tally(20, &burst); + assert_eq!((t.verdict(), t.cpu_changes.clone()), (Oracle::CpuRailMoved(false), vec![5]), "a five-step burst"); + + let live = |i: usize| a + 13 * i as i64; + assert_eq!(tally(6, &live).verdict(), Oracle::CpuRailMoved(true), "5 changes in 5 compared pairs"); + assert_eq!(tally(5, &live).verdict(), Oracle::CpuRailMoved(false), "4 changes in 4 compared pairs"); + let four_in_five = |i: usize| a + (1..=i).filter(|k| k % 5 != 0).count() as i64; + assert_eq!(tally(21, &four_in_five).verdict(), Oracle::CpuRailMoved(true), "16 changes in 20"); + let three_in_four = |i: usize| a + (1..=i).filter(|k| k % 4 != 0).count() as i64; + assert_eq!(tally(21, &three_in_four).verdict(), Oracle::CpuRailMoved(false), "15 changes in 20"); + assert!(!oracle_moved(&[2], 11) && oracle_moved(&[0, 11], 11) && !oracle_moved(&[3, 3], 5)); + } + // ── The production status, on scripted IOReport leaves ─────────────── /// A sample the way the FFI returns one of the Energy Model group: @@ -1908,13 +2265,15 @@ mod macos_ioreport { /// sampler closure, [`production_probe`] with the real tick and clocks, /// the rule and the mapping — with a script standing in for IOReport's /// two FFI leaves. No subscription is `Unavailable`, with no sample - /// taken; a single batch refresh is `CpuRailFrozen`; two advances are - /// `Live`, with no sample past the third; one step on each of two dies, - /// in different intervals, is `CpuRailFrozen`. These hosts produce none - /// of the first three on demand, so no hardware test can reach them. - /// Every scripted step falls in the first two intervals, which the - /// probe always observes however loaded the host: it takes at least - /// three samples, and its first tick ends within the budget. + /// taken; a single batch refresh is `CpuRailFrozen`; the observed + /// refresh published in two steps is `CpuRailFrozen`; a rail advancing + /// at every sample is `Live`, once five intervals and 300 ms of the + /// real wall clock have passed and no later; one step on each of two + /// dies, in different intervals, is `CpuRailFrozen`. These hosts + /// produce none of the first four on demand, so no hardware test can + /// reach them. Every scripted step falls in the first five intervals, + /// which the probe always observes however loaded the host: it takes + /// at least six samples unless the rail proves live first. #[test] fn host_status_is_the_rule_on_the_production_composition() { let (status, served, ticks) = scripted_host_status(false, |_| panic!("sampled without a subscription")); @@ -1934,9 +2293,30 @@ mod macos_ioreport { // samples until it has passed. Load can only lengthen this. assert!(took >= Duration::from_secs(1), "the production probe gave a frozen verdict after {took:?}"); + // The M5 Max's two-step refresh: +61,764,485 then +3,435, in the + // third and fourth intervals. The old rule's two advances. + let two_step = |i: usize| { + let a = 2_643_843_378; + let cpu = if i < 3 { a } else if i < 4 { a + 61_764_485 } else { a + 61_764_485 + 3_435 }; + [("CPU Energy", cpu), ("MCPU0", cpu / 4), ("PCPU", cpu / 2), ("GPU Energy", 1_000 + 7 * i as i64)] + }; + let (status, served, _) = scripted_host_status(true, move |i| Some(group_sample(&two_step(i)))); + assert_eq!(status, IoReportStatus::CpuRailFrozen, "a refresh in two steps ({served} samples) must be frozen"); + + // A rail advancing at every sample. Each production tick burns 25 ms + // of this thread's CPU, so at least 25 ms of wall time: 300 ms have + // passed by the 12th interval at the latest (13 allowed, for clock + // granularity), and never before the 5th. let live = |i: usize| [("GPU Energy", 5), ("CPU Energy", 1_000 + 13 * i as i64)]; + let started = Instant::now(); let (status, served, _) = scripted_host_status(true, move |i| Some(group_sample(&live(i)))); - assert_eq!((status, served), (IoReportStatus::Live, 3), "two advances must be live at the third sample"); + let took = started.elapsed(); + assert_eq!(status, IoReportStatus::Live, "a rail advancing at every one of {served} samples must be live"); + assert!( + (6..=14).contains(&served) && took >= Duration::from_millis(300), + "a sustained advance was declared live after {served} samples in {took:?}: the rule needs five \ + intervals and 300 ms, and the probe stops as soon as it holds" + ); let dies = |i: usize| { [ @@ -1971,7 +2351,7 @@ mod macos_ioreport { walls.iter().max() ); assert_eq!(status, IoReportStatus::CpuRailFrozen); - assert!(ticks.len() >= 2 && ticks.len() == served - 1, "{} ticks for {served} samples", ticks.len()); + assert!(ticks.len() >= 5 && ticks.len() == served - 1, "{} ticks for {served} samples", ticks.len()); for (i, t) in ticks.iter().enumerate() { assert!( t.cpu >= TICK || t.wall >= TICK_WALL_CAP, @@ -1995,7 +2375,7 @@ mod macos_ioreport { /// spinning until this thread's own CPU clock has advanced 25 ms (or its /// 500 ms wall cap). Pinned per tick, by the tests' own clocks, in the /// production composition, over a frozen verdict's whole budget, which - /// always holds two ticks at least; and the contention must have been + /// always holds five ticks at least; and the contention must have been /// real: at least one tick shared its core. The probe starts only once /// every busy thread is running. #[test] @@ -2033,7 +2413,7 @@ mod macos_ioreport { ticks.iter().map(|t| (t.cpu, t.wall)).collect::>() ); assert_eq!(status, IoReportStatus::CpuRailFrozen); - assert!(ticks.len() >= 2 && ticks.len() == served - 1, "{} ticks for {served} samples", ticks.len()); + assert!(ticks.len() >= 5 && ticks.len() == served - 1, "{} ticks for {served} samples", ticks.len()); for (i, t) in ticks.iter().enumerate() { assert!( t.cpu >= TICK || t.wall >= TICK_WALL_CAP, @@ -2186,8 +2566,10 @@ mod macos_ioreport { /// The oracle gets at least three times the probe's budget, so a rail too /// slow for the probe but moving within the oracle's window is the only /// case where declining is accepted — and that fails safe: estimated, - /// never over-claimed. A single batch refresh in both windows does not - /// rescue a probe that called it live: the oracle counts it as one change. + /// never over-claimed. A batch refresh in both windows does not rescue + /// a probe that called it live, even one published in two or more + /// steps: the oracle counts a rail as moving only if it changed at 80% + /// or more of its reads, at least five times. #[test] fn the_ioreport_counter_is_never_selected_on_a_frozen_cpu_rail() { let status = crate::ioreport_status(); @@ -2198,8 +2580,8 @@ mod macos_ioreport { eprintln!( "ioreport liveness: status={status:?} oracle={:?} selected={selected} \ (oracle: {} reads over {:?} with a helper thread burning CPU; \ - changes per CPU rail {:?}, per GPU rail {:?})", - run.verdict, run.reads, run.took, run.cpu_changes, run.gpu_changes + changes per CPU rail {:?}, per GPU rail {:?}, of {} compared pairs)", + run.verdict, run.reads, run.took, run.cpu_changes, run.gpu_changes, run.compared ); run.assert_evidence(); match run.verdict { @@ -2231,8 +2613,8 @@ mod macos_ioreport { } Oracle::CpuRailMoved(true) => { // A live rail. The probe had a third of the oracle's time: if it - // saw two advances the counter must be selected; if not, - // declining is the safe side. + // saw the rail keep advancing the counter must be selected; if + // not, declining is the safe side. assert!( matches!(status, IoReportStatus::Live | IoReportStatus::CpuRailFrozen), "the oracle saw a CPU rail keep moving but the status is {status:?}" @@ -2257,10 +2639,11 @@ mod macos_ioreport { let cached = crate::ioreport_status(); let oracle = oracle_cpu_rail(crate::CPU_RAIL_LIVENESS_BUDGET * 3); eprintln!( - "ioreport probe: probed={probed:?} cached={cached:?} intervals={} sample calls={} ticks={} \ - advances per CPU rail {:?}; oracle={:?} ({} reads over {:?}, changes per CPU rail {:?}, per GPU \ - rail {:?})", + "ioreport probe: probed={probed:?} cached={cached:?} intervals={} over {:?} sample calls={} \ + ticks={} advances per CPU rail {:?}; oracle={:?} ({} reads over {:?}, changes per CPU rail {:?}, \ + per GPU rail {:?}, of {} compared pairs)", run.watch.intervals, + run.watch.window, run.calls.len(), run.ticks, run.watch.rails, @@ -2268,7 +2651,8 @@ mod macos_ioreport { oracle.reads, oracle.took, oracle.cpu_changes, - oracle.gpu_changes + oracle.gpu_changes, + oracle.compared ); oracle.assert_evidence(); if probed == Some(Some(CpuRail::Live)) || cached == IoReportStatus::Live { @@ -2765,14 +3149,16 @@ pub enum IoReportStatus { /// in it — an unfamiliar channel naming. The counter is not used. CpuRailAbsent, /// The CPU energy rail exists but did not keep advancing while CPU was - /// burned: it held, or stepped in only one sample interval (a refresh). + /// burned: it held, or stepped only in a burst (a refresh, which macOS 27 + /// can publish in two or more steps a few tens of milliseconds apart). /// The platform is not exposing live CPU energy to this process, so using /// the counter would report GPU-only energy, zero, or a stale refresh as /// the energy of CPU work. Observed on macOS 27.0.1 on both an M5 Max and /// an M3 Ultra. CpuRailFrozen, - /// The same CPU energy rail advanced in two separate sample intervals under - /// load; the counter is available. + /// The same CPU energy rail kept advancing under load — in at least five + /// separate sample intervals, at least 80% of those observed, over at least + /// 300 ms of wall time; the counter is available. Live, } @@ -2783,10 +3169,12 @@ pub enum IoReportStatus { /// 500 ms of wall time). A loaded host gives the probing thread less of a core /// in the same second, so the probe sees fewer intervals (on an M5 Max under /// 1 × ncpu busy spinners, often half as many or fewer), and a live rail has -/// fewer chances to show its two advances. That fails safe: declined, so -/// callers estimate; never over-claimed. If sampling is so slow that fewer -/// than two intervals fit in the budget, the probe keeps going until two -/// have, and never past five budgets. Paid once per process; see +/// fewer chances to show a sustained advance: five advancing intervals, at +/// least 80% of those observed, over at least 300 ms. That fails safe: +/// declined, so callers estimate; never over-claimed. If sampling is so slow +/// that fewer than five intervals fit in the budget, the probe keeps going +/// until five have, and never past five budgets. A live rail ends the probe +/// early, after 300 ms at the least. Paid once per process; see /// [`ioreport_status`]. #[cfg(target_os = "macos")] pub const CPU_RAIL_LIVENESS_BUDGET: std::time::Duration = std::time::Duration::from_millis(1000); @@ -2796,8 +3184,8 @@ pub const CPU_RAIL_LIVENESS_BUDGET: std::time::Duration = std::time::Duration::f /// `MacosIoReportCounter::try_detect` (and so [`detect`]) makes the IOReport /// counter available only when this is [`IoReportStatus::Live`]. The first /// call on macOS burns CPU on the calling thread for about -/// `CPU_RAIL_LIVENESS_BUDGET` of wall time (less when the rail proves live, at -/// most five budgets). Exposed so a caller or CLI can say *why* an Apple +/// `CPU_RAIL_LIVENESS_BUDGET` of wall time (less when the rail proves live, +/// though never under 300 ms; at most five budgets). Exposed so a caller or CLI can say *why* an Apple /// Silicon host is not metered by IOReport. pub fn ioreport_status() -> IoReportStatus { #[cfg(target_os = "macos")] -- 2.54.0 (Apple Git-157) From f84f0b2fd0122424c5676ad82a9d9f26e7bf4433 Mon Sep 17 00:00:00 2001 From: Transaction Science Date: Wed, 30 Sep 2026 20:17:08 -0400 Subject: [PATCH 09/22] eoc-cascade: a cache hit claims no measurement, and a total is measured only if every microjoule was read MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit c6378629 listed two defects here as not done. `JouleCost::zero()` was tagged `Measured`, and eoc-cache and the cascade's memoizer stamp every cache hit with it, so each hit claimed a measured 0 µJ although no meter was read for it. The receipt seals that tag. And `attach_cascade_cost` added the meter's delta over the whole resolve to the stage's own cost, then tagged the sum `Measured` as soon as the delta was positive. With a live counter (RAPL, NVML, a live IOReport rail) a 50 J estimate plus a 500 µJ reading came out as 50,000,500 µJ measured. A stage that meters itself on the cascade's counter reads a span nested inside the cascade's own span, so its reading was counted twice: 300 µJ inside a 500 µJ span came out as 800 µJ. The rule now, as the owner set it: a live vendor counter's figure is `Measured`, and nothing that was not read from one may be. So a total is measured only when every microjoule of it was read from a live counter. - `JouleCost::zero()` is `Estimated`. Its doc says what it is: a figure no counter read, which is what a cache hit costs by assumption. Every caller becomes honest at once: eoc-cache, the memoizer, and eoc-route-learned's two empty fallback responses, which also claimed a measured zero. No caller needed a measured zero; a counter that read but did not move has measured nothing. `JouleSource`'s and `JouleCost::measured`'s docs now say what measured means. - `attach_cascade_cost` takes the span only when it is live: both reads succeeded and the counter advanced. Otherwise it adds nothing, and the stage's cost stands, figure and tag, as before. - A live span and an estimated stage: the estimate is added, and the total is an estimate. The estimate may be energy the counter never saw (a remote backend), so it is added, not dropped. - A live span and a measured stage: the stage read a counter on this host during the resolve, so inside the span. The total is the larger of the two readings, measured, never their sum. That is exact when the stage read the cascade's counter, and a lower bound if it read another; the cost carries no counter identity, so the cascade cannot tell. - A live span and an estimated zero (a cache hit): the zero adds nothing, so the total is the span's reading alone and is measured. Every microjoule of it was read. Tagging it estimated would under-claim a real reading. The eoc CLI test's hardware-counter branch asserted the old rule (measured whenever the counter added anything). It now asserts the new one: the backend's estimate plus the reading, tagged estimated. That branch does not run on this Mac, where IOReport declines and the meter is the powermetrics stand-in ("eoc cost on this host: meter=powermetrics cost=50000000 µJ (estimated)"). Tests. The cascade is driven with a scripted counter that one test shares with a self-metering neural backend, as a host's one counter would be. An estimate plus a live 1,000 -> 1,500 span is `estimated(50_000_500)`, and its receipt verifies. A stage that read 1,100 -> 1,400 inside 1,000 -> 1,500 costs `measured(500)`, not 800. A cache hit inside a live 2,000 -> 2,250 span is `measured(250)`. A counter that holds, fails its first read, or fails its last leaves `estimated(42)` alone. Memoized and hand-inserted hits through a stub cascade are `estimated(0)`. A table test covers every branch of the cost rule and of the liveness check. Falsified on the uncommitted tree. Each mutant was restored byte-for-byte and re-touched so cargo rebuilt from the real source; a diff fingerprint taken before and after matched: - `zero()` back to `Measured` fails three tests, in eoc-core, eoc-cache and eoc-cascade; - the cache stamping hits `measured(0)` fails the hand-inserted hit (`left: JouleCost { microjoules: 0, source: Measured }`); - the memoizer stamping `measured(0)` fails the memoized hit; - tagging estimate + span `Measured` fails with `left: JouleCost { microjoules: 50000500, source: Measured }`; - summing a measured stage onto the span fails with `left: ... 800 ...`, `right: ... 500 ...`; - HEAD's rule, verbatim, fails three of the new tests; - counting a counter that held (`e >= s`) as live fails two. eoc-bench, a separate workspace that builds against eoc-core, eoc-cache and eoc-cascade, passes too. Co-Authored-By: Claude Opus 5.5 --- eoc/eoc-rs/crates/eoc-cache/src/lib.rs | 13 +- eoc/eoc-rs/crates/eoc-cascade/src/lib.rs | 231 +++++++++++++++++-- eoc/eoc-rs/crates/eoc-core/src/joule_cost.rs | 33 ++- eoc/eoc-rs/crates/eoc/src/main.rs | 14 +- 4 files changed, 248 insertions(+), 43 deletions(-) diff --git a/eoc/eoc-rs/crates/eoc-cache/src/lib.rs b/eoc/eoc-rs/crates/eoc-cache/src/lib.rs index 6b5a7d0a9..f165c7858 100644 --- a/eoc/eoc-rs/crates/eoc-cache/src/lib.rs +++ b/eoc/eoc-rs/crates/eoc-cache/src/lib.rs @@ -1,9 +1,10 @@ //! EOC stage 1 — cache. //! //! The cheapest stage: an LRU keyed by `QueryId` returning a previously -//! computed `Response`. Cache hits are reported with `JouleCost::zero()` — -//! the energy of a hashmap lookup is below the noise floor of any -//! contemporary hardware energy counter. +//! computed `Response`. Cache hits are reported with `JouleCost::zero()`: +//! zero micro-joules, tagged `Estimated`. The energy of a hashmap lookup is +//! below the noise floor of any contemporary hardware energy counter, and no +//! counter is read for a hit, so a hit never claims a measurement. #![forbid(unsafe_code)] @@ -56,7 +57,8 @@ impl LruCache { } /// Insert (or refresh) a `Response`. Re-stamps the response as `Cache` - /// stage with zero joule cost so subsequent hits look like hits. + /// stage with `JouleCost::zero()` (an estimated zero: no counter is read + /// for a hit) so subsequent hits look like hits. pub fn insert(&self, q: &Query, payload: impl Into) -> Response { let r = Response::new(q.id, payload.into(), StageKind::Cache, JouleCost::zero()); self.inner @@ -101,7 +103,8 @@ mod tests { let r = cache.try_resolve(&q).await.expect("hit"); assert_eq!(r.stage, StageKind::Cache); assert_eq!(r.joule_cost.microjoules, 0); - assert_eq!(r.joule_cost.source, JouleSource::Measured); + // No counter was read for the hit: its zero is an estimate. + assert_eq!(r.joule_cost.source, JouleSource::Estimated); } #[tokio::test] diff --git a/eoc/eoc-rs/crates/eoc-cascade/src/lib.rs b/eoc/eoc-rs/crates/eoc-cascade/src/lib.rs index e7e1a7215..a063a5ec5 100644 --- a/eoc/eoc-rs/crates/eoc-cascade/src/lib.rs +++ b/eoc/eoc-rs/crates/eoc-cascade/src/lib.rs @@ -1,9 +1,11 @@ //! EOC cascade — the four-stage memoizing pipeline. //! //! `Cascade::resolve` walks `cache → kv → graph → neural` in order. The -//! first stage to return `Some(Response)` wins. Joule cost is the sum of -//! the stage-level cost reported by the stage plus any cascade-level -//! overhead measured by the meter between the start and end of the call. +//! first stage to return `Some(Response)` wins. Its joule cost combines the +//! cost the stage reported with what the cascade's meter read over the whole +//! call, when that meter is live; it is `Measured` only when every +//! micro-joule of it was read from a live counter. Any micro-joule that was +//! estimated makes the whole an estimate. #![forbid(unsafe_code)] @@ -107,16 +109,17 @@ impl Cascade { r }; - // Re-derive the response with the cascade-attributed joule cost - // (stage-reported + measured overhead, if any). + // Re-derive the response with the cascade-attributed joule cost: the + // stage's own cost, combined with the meter's span when it is live. attach_cascade_cost(result, start, self.meter.as_ref()) } fn maybe_memoize(&self, _q: &Query, r: &Response) { if self.memoize { // Memoize as a *cache* entry so future hits report `Stage::Cache` - // and `JouleCost::zero()`. The original receipt is no longer - // valid (different stage + cost) — we mint a new one. + // and `JouleCost::zero()`, an estimated zero: no counter is read + // for a hit. The original receipt is no longer valid (different + // stage + cost) — we mint a new one. let cached = Response::new(r.query_id, r.payload.clone(), eoc_core::Stage::Cache, JouleCost::zero()); self.cache.insert_response(r.query_id, cached); @@ -125,29 +128,54 @@ impl Cascade { } fn attach_cascade_cost( - mut response: Response, + response: Response, start: Option, meter: &dyn JouleCounter, ) -> Response { let end = meter.read_microjoules().ok(); - let overhead = match (start, end) { - (Some(s), Some(e)) if e >= s => e - s, - _ => 0, - }; - let stage_cost = response.joule_cost.microjoules; - let total = stage_cost.saturating_add(overhead); - let source = if overhead > 0 { - JouleSource::Measured - } else { - response.joule_cost.source - }; - let joule_cost = JouleCost { - microjoules: total, - source, - }; + let joule_cost = cascade_cost(response.joule_cost, live_span(start, end)); // Rebuild the response so the receipt covers the final joule cost. - response = Response::new(response.query_id, response.payload, response.stage, joule_cost); - response + Response::new(response.query_id, response.payload, response.stage, joule_cost) +} + +/// What the cascade's meter read over one resolve: the delta between its two +/// reads, only when both succeeded and the counter advanced. A counter that +/// read but did not move — the stub, a frozen rail — measured nothing, and a +/// failed read leaves no span; neither is a figure. +fn live_span(start: Option, end: Option) -> Option { + match (start, end) { + (Some(s), Some(e)) if e > s => Some(e - s), + _ => None, + } +} + +/// The joule cost of one resolve, from the answering stage's own cost and +/// the cascade meter's live span over the whole resolve, if there was one. +/// +/// It is `Measured` only when every micro-joule of it was read from a live +/// counter; any micro-joule that was estimated makes the whole an estimate. +/// +/// - No live span: the cascade read nothing, adds nothing, and the stage's +/// cost stands as the stage reported it. +/// - A live span and a measured stage: the stage read a counter on this host +/// during the resolve, so inside the span. On the cascade's own counter the +/// span already holds that reading, and adding the two would count it +/// twice. The cost is the larger of the two, measured: exact when the stage +/// read the cascade's counter, a lower bound if it read another. +/// - A live span and an estimated stage: the estimate is energy no counter +/// read, possibly spent elsewhere (a remote backend), so it is added to the +/// span and the total is an estimate. An estimated zero, which is what a +/// cache hit costs, adds nothing: the cost is the span's reading alone, and +/// measured. +fn cascade_cost(stage: JouleCost, span: Option) -> JouleCost { + let Some(span) = span else { + return stage; + }; + match stage.source { + JouleSource::Measured => JouleCost::measured(stage.microjoules.max(span)), + JouleSource::Estimated if stage.microjoules == 0 => JouleCost::measured(span), + JouleSource::Estimated => JouleCost::estimated(stage.microjoules.saturating_add(span)), + } } #[cfg(test)] @@ -209,4 +237,157 @@ mod tests { let r2 = cascade.resolve(q).await; assert_eq!(r2.stage, StageKind::Cache); } + + /// A counter that reads the scripted values in turn, then the last one + /// again; `None` is a failed read. One instance can be shared by the + /// cascade and a stage, as one host counter would be. + struct Scripted { + reads: std::sync::Mutex>>, + } + + impl Scripted { + fn new(reads: &[Option]) -> Arc { + Arc::new(Self { + reads: std::sync::Mutex::new(reads.iter().copied().collect()), + }) + } + } + + impl JouleCounter for Scripted { + fn read_microjoules(&self) -> eoc_core::Result { + let mut r = self.reads.lock().unwrap(); + let next = if r.len() > 1 { + r.pop_front().unwrap() + } else { + r[0] + }; + next.ok_or(eoc_core::Error::NoCounter) + } + fn name(&self) -> &'static str { + "scripted" + } + } + + /// A neural backend that meters itself the way the local backends do: it + /// reads its counter before and after, and reports the delta as measured + /// when the counter advanced. + struct SelfMetered { + meter: Arc, + } + + #[async_trait::async_trait] + impl eoc_neural::NeuralBackend for SelfMetered { + async fn infer(&self, q: &Query) -> Response { + let start = self.meter.read_microjoules().ok(); + let end = self.meter.read_microjoules().ok(); + let cost = match (start, end) { + (Some(s), Some(e)) if e > s => JouleCost::measured(e - s), + _ => JouleCost::estimated(1), + }; + Response::new(q.id, "metered".to_string(), StageKind::Neural, cost) + } + } + + fn metered(meter: Arc, neural: Box) -> Cascade { + Cascade::new( + Arc::new(LruCache::new(8)), + Arc::new(KvStage::new(Box::new(MemoryKvBackend::new()))), + Arc::new(GraphStage::new()), + Arc::new(NeuralStage::new(neural)), + ) + .with_meter(meter) + } + + /// No meter is read for a cache hit, so it claims no measurement: not a + /// memoized answer, not an inserted one. + #[tokio::test] + async fn a_cache_hit_claims_no_measurement() { + let cascade = fixture(); + let q = Query::new("totally novel question"); + cascade.resolve(q.clone()).await; + let memoized = cascade.resolve(q).await; + assert_eq!(memoized.stage, StageKind::Cache); + assert_eq!(memoized.joule_cost, JouleCost::estimated(0)); + + let q = Query::new("inserted by hand"); + cascade.cache().insert(&q, "answer"); + let inserted = cascade.resolve(q).await; + assert_eq!(inserted.stage, StageKind::Cache); + assert_eq!(inserted.joule_cost, JouleCost::estimated(0)); + } + + /// A live counter advanced 500 µJ over the resolve, and the stage reported + /// a 50 J estimate. The total holds an estimate, so it is one. + #[tokio::test] + async fn an_estimate_plus_a_live_span_is_an_estimate() { + let meter = Scripted::new(&[Some(1_000), Some(1_500)]); + let cascade = metered(meter, Box::new(EchoBackend::new().with_cost(50_000_000))); + let r = cascade.resolve(Query::new("totally novel question")).await; + assert_eq!(r.stage, StageKind::Neural); + assert_eq!(r.joule_cost, JouleCost::estimated(50_000_500)); + assert!(r.receipt.verify(&r.query_id, &r.stage, &r.joule_cost, &r.payload)); + } + + /// The stage metered itself on the cascade's own counter: it read 300 µJ + /// (1_100 -> 1_400) inside the cascade's 500 µJ span (1_000 -> 1_500). The + /// span already holds those 300 µJ, so the resolve cost 500 µJ, measured, + /// never 800. + #[tokio::test] + async fn a_stage_metered_on_the_cascade_counter_is_counted_once() { + let meter = Scripted::new(&[Some(1_000), Some(1_100), Some(1_400), Some(1_500)]); + let shared: Arc = meter; + let cascade = metered(shared.clone(), Box::new(SelfMetered { meter: shared })); + let r = cascade.resolve(Query::new("totally novel question")).await; + assert_eq!(r.stage, StageKind::Neural); + assert_eq!(r.joule_cost, JouleCost::measured(500)); + } + + /// A cache hit adds no figure of its own, so on a live counter the cost is + /// the span's reading alone, every micro-joule of it read: measured. + #[tokio::test] + async fn a_cache_hit_on_a_live_counter_is_the_span_reading() { + let meter = Scripted::new(&[Some(2_000), Some(2_250)]); + let cascade = metered(meter, Box::new(EchoBackend::new())); + let q = Query::new("seen before"); + cascade.cache().insert(&q, "answer"); + let r = cascade.resolve(q).await; + assert_eq!(r.stage, StageKind::Cache); + assert_eq!(r.joule_cost, JouleCost::measured(250)); + } + + /// A counter that reads but does not move, or fails a read, measured + /// nothing: the stage's own cost stands, figure and tag. + #[tokio::test] + async fn a_span_that_is_not_live_adds_nothing() { + for reads in [vec![Some(7_000)], vec![None, Some(9_000)], vec![Some(9_000), None]] { + let meter = Scripted::new(&reads); + let cascade = metered(meter, Box::new(EchoBackend::new().with_cost(42))); + let r = cascade.resolve(Query::new("totally novel question")).await; + assert_eq!(r.joule_cost, JouleCost::estimated(42), "reads {reads:?}"); + } + } + + /// Every branch of the rule, on the figures alone. + #[test] + fn the_cost_is_measured_only_when_every_microjoule_was_read() { + let e = JouleCost::estimated; + let m = JouleCost::measured; + // No live span: the stage's cost, untouched. + assert_eq!(cascade_cost(e(40), None), e(40)); + assert_eq!(cascade_cost(m(40), None), m(40)); + assert_eq!(cascade_cost(e(0), None), e(0)); + // A live span and an estimate: summed, and an estimate. + assert_eq!(cascade_cost(e(40), Some(10)), e(50)); + // A live span and a measurement: the larger reading, never the sum. + assert_eq!(cascade_cost(m(40), Some(100)), m(100)); + assert_eq!(cascade_cost(m(400), Some(100)), m(400)); + // A live span and an estimated zero: the span's reading. + assert_eq!(cascade_cost(e(0), Some(100)), m(100)); + // A span is live only when the counter advanced. + assert_eq!(live_span(Some(5), Some(5)), None); + assert_eq!(live_span(Some(6), Some(5)), None); + assert_eq!(live_span(None, Some(5)), None); + assert_eq!(live_span(Some(5), None), None); + assert_eq!(live_span(Some(5), Some(8)), Some(3)); + } } diff --git a/eoc/eoc-rs/crates/eoc-core/src/joule_cost.rs b/eoc/eoc-rs/crates/eoc-core/src/joule_cost.rs index 68a831b00..be5be856d 100644 --- a/eoc/eoc-rs/crates/eoc-core/src/joule_cost.rs +++ b/eoc/eoc-rs/crates/eoc-core/src/joule_cost.rs @@ -5,9 +5,11 @@ use serde::{Deserialize, Serialize}; /// Where the joule reading came from. #[derive(Debug, Clone, Copy, PartialEq, Eq, Hash, Serialize, Deserialize)] pub enum JouleSource { - /// Reading from a hardware energy counter (RAPL, NVML, powermetrics). + /// Read from a live hardware energy counter (RAPL, NVML, or IOReport + /// while its CPU rail is live): the counter advanced over the span. Measured, - /// Synthesized estimate (no hardware counter available). + /// Anything else: a synthesized estimate, or a figure no live counter + /// read (no counter, a failed read, a counter that did not advance). Estimated, } @@ -21,11 +23,19 @@ pub struct JouleCost { } impl JouleCost { - /// A zero-cost reading — typical for in-process cache hits. + /// Zero micro-joules, tagged [`JouleSource::Estimated`]: a figure no + /// counter read. It is what a cache hit costs by assumption, since the + /// energy of an in-process lookup is below any counter's resolution and + /// no meter is read for it. + /// + /// It is never [`JouleSource::Measured`]. Measured means a live hardware + /// counter read the span and advanced over it; a zero that no counter read + /// is an estimate, and a counter that read but did not move has measured + /// nothing. pub fn zero() -> Self { Self { microjoules: 0, - source: JouleSource::Measured, + source: JouleSource::Estimated, } } @@ -37,7 +47,8 @@ impl JouleCost { } } - /// A measured cost — used when a hardware counter reading is attached. + /// A measured cost: the delta a live hardware counter (RAPL, NVML, a live + /// IOReport rail) read over the span of the work, and nothing else. pub fn measured(microjoules: u64) -> Self { Self { microjoules, @@ -60,3 +71,15 @@ impl std::fmt::Display for JouleCost { write!(f, "{} µJ ({tag})", self.microjoules) } } + +#[cfg(test)] +mod tests { + use super::*; + + /// A zero no counter read is an estimate, never a measurement. + #[test] + fn zero_claims_no_measurement() { + assert_eq!(JouleCost::zero(), JouleCost::estimated(0)); + assert_eq!(JouleCost::zero().to_string(), "0 µJ (estimated)"); + } +} diff --git a/eoc/eoc-rs/crates/eoc/src/main.rs b/eoc/eoc-rs/crates/eoc/src/main.rs index 0f3966308..3e4ff6914 100644 --- a/eoc/eoc-rs/crates/eoc/src/main.rs +++ b/eoc/eoc-rs/crates/eoc/src/main.rs @@ -167,7 +167,9 @@ mod tests { /// the meter [`super::detect`] picks is no hardware reading — the stub, or /// the powermetrics stand-in, which is what a Mac whose IOReport CPU rail is /// not live falls back to — the cost is the backend's own estimate: the - /// same non-zero figure, tagged estimated. Never a zero for the work, and + /// same non-zero figure, tagged estimated. Where it is a live hardware + /// counter, the cost is that estimate plus what the counter read, still + /// tagged estimated, because part of it is. Never a zero for the work, and /// never tagged measured. #[tokio::test] async fn work_on_an_unmetered_host_keeps_its_estimate() { @@ -186,14 +188,10 @@ mod tests { if matches!(meter, "stub" | "powermetrics") { assert_eq!(r.joule_cost, JouleCost::estimated(BURN_ESTIMATE_UJ), "meter={meter}"); } else { - // A hardware counter: the estimate plus what it measured, never less. + // A hardware counter: the estimate plus what it read, never less, + // and an estimate, since the backend's part of it is one. assert!(r.joule_cost.microjoules >= BURN_ESTIMATE_UJ, "{}", r.joule_cost); - assert_eq!( - r.joule_cost.source == JouleSource::Measured, - r.joule_cost.microjoules > BURN_ESTIMATE_UJ, - "{}", - r.joule_cost - ); + assert_eq!(r.joule_cost.source, JouleSource::Estimated, "{}", r.joule_cost); } } -- 2.54.0 (Apple Git-157) From 0db9b8be02db7897f23fe704d5216d3759839006 Mon Sep 17 00:00:00 2001 From: Transaction Science Date: Wed, 30 Sep 2026 20:17:24 -0400 Subject: [PATCH 10/22] eoc-carbon: an SCI score is only as measured as the joules it was computed from MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit `sci_from_eval_report` wrapped every eval report's energy in `JouleCost::measured`, whatever produced it. An eval run against the echo backend, a vendor API's token estimate or a stub-metered cascade came out as a measured energy term. It could not do better: `EvalReport` dropped the provenance of the costs it summed, and `SciScore` had nowhere to put one. The end-to-end test `sci_for_ai` did the same in miniature. Its comment said "pretend the meter reported 2000 J on the stub backend", and it built that figure with `JouleCost::measured`. What holds now: - `EvalReport` gains `joule_source`. It is `Measured` only when every case's cost was read from a live counter. One estimated case makes the total an estimate, and so does a run with no cases, since nothing was read. Each case is separate work that ran, so an unmeasured case is not covered by any other reading. A report serialized before the field existed reads back as `Estimated` (`#[serde(default)]`): nothing in it says it was measured. - `SciScore` gains `energy_source`, the input `JouleCost`'s provenance unchanged. It has the same serde default. The numbers do not change. - `sci_from_eval_report` builds the energy term with the report's own `joule_source`. - `sci_for_ai` uses `JouleCost::estimated` for its fixed 2000 J. No counter read that figure. It now asserts that the score says so. Tests: a measured backend gives a measured report. An estimated one gives an estimated report, and so does one that turns estimated from its third answer on. An empty run is estimated. A report and a score without provenance read back as estimated. A score carries its input's provenance, with the same SCI either way. Under `eval-bridge`, a report's score keeps the report's provenance. Falsified, each mutant restored byte-for-byte: - HEAD's `JouleCost::measured(total_microjoules)` in the bridge fails `an_eval_reports_score_keeps_the_reports_provenance` (`left: Measured`, `right: Estimated`). That test is behind the `eval-bridge` feature, which nothing in the workspace enables, so a plain `cargo test --workspace` never compiles it. It was run with `cargo test -p eoc-carbon --features eval-bridge`, 39 + 8 passing. - `SciScore` ignoring its input's provenance fails two tests, one in the default build. - the runner ignoring per-case provenance fails `the_report_is_measured_only_when_every_case_was`. - `sci_for_ai` tagging its figure measured again fails that test. Not done: `CarbonAccount` still drops provenance (`account_request` is the other public way in). And a report with zero correct answers has `joules_per_correct == u64::MAX` as a sentinel, which `sci_from_eval_report` multiplies by `correct.max(1)` into a u64::MAX µJ energy term. Co-Authored-By: Claude Opus 5.5 --- .../crates/eoc-carbon/src/sci_bridge.rs | 87 +++++++++++++++++- .../crates/eoc-carbon/tests/sci_for_ai.rs | 15 ++-- eoc/eoc-rs/crates/eoc-eval/src/runner.rs | 88 ++++++++++++++++++- 3 files changed, 180 insertions(+), 10 deletions(-) diff --git a/eoc/eoc-rs/crates/eoc-carbon/src/sci_bridge.rs b/eoc/eoc-rs/crates/eoc-carbon/src/sci_bridge.rs index 3fddedf3b..0640c36c1 100644 --- a/eoc/eoc-rs/crates/eoc-carbon/src/sci_bridge.rs +++ b/eoc/eoc-rs/crates/eoc-carbon/src/sci_bridge.rs @@ -24,12 +24,13 @@ use serde::{Deserialize, Serialize}; use crate::account::{CarbonAccount, account_request}; use crate::intensity::CarbonIntensity; use crate::pue::Pue; -use eoc_core::JouleCost; +use eoc_core::{JouleCost, JouleSource}; /// Inputs to one SCI computation. #[derive(Debug, Clone)] pub struct SciInputs<'a> { - /// Measured joules at the IT layer (from [`eoc_meter`]). + /// Joules at the IT layer, with their provenance: measured by a live + /// counter (see [`eoc_meter`]) or estimated. The score carries it. pub joule_cost: &'a JouleCost, /// Grid intensity (marginal preferred, average acceptable). pub intensity: &'a CarbonIntensity, @@ -57,6 +58,17 @@ pub struct SciScore { pub functional_units: f64, /// The SCI value (gCO2e per functional unit). pub sci: f64, + /// Where the energy term came from: the input's provenance, unchanged. + /// An SCI value is only as measured as the joules it was computed from. + /// A score serialized without this field reads back as `Estimated`. + #[serde(default = "unrecorded_energy_source")] + pub energy_source: JouleSource, +} + +/// The provenance of a score that recorded none: an estimate, never a +/// measurement. +fn unrecorded_energy_source() -> JouleSource { + JouleSource::Estimated } impl SciScore { @@ -77,6 +89,7 @@ impl SciScore { embodied_g_co2e: inputs.embodied_g_co2e, functional_units: r, sci, + energy_source: inputs.joule_cost.source, } } } @@ -86,7 +99,8 @@ impl SciScore { /// /// `EvalReport` carries `joules_per_correct` in micro-joules and a /// `correct` count, so we reconstruct the total IT-layer energy as -/// `joules_per_correct * correct`. +/// `joules_per_correct * correct`. Its provenance is the report's own +/// `joule_source`: measured only when every case of the run was. #[cfg(feature = "eval-bridge")] pub fn sci_from_eval_report( report: &eoc_eval::EvalReport, @@ -98,7 +112,10 @@ pub fn sci_from_eval_report( let total_microjoules = (report.joules_per_correct as u128) .saturating_mul(correct as u128) .min(u64::MAX as u128) as u64; - let cost = JouleCost::measured(total_microjoules); + let cost = JouleCost { + microjoules: total_microjoules, + source: report.joule_source, + }; SciScore::compute(SciInputs { joule_cost: &cost, intensity, @@ -134,4 +151,66 @@ mod tests { assert_eq!(s.functional_units, 1.0); assert!(s.sci > 0.0); } + + fn intensity() -> CarbonIntensity { + CarbonIntensity::new("US", 400.0, IntensityKind::Average, ProviderKind::Mock, Utc::now()) + } + + /// The score says where its energy term came from, and the same figure + /// yields the same numbers either way. + #[test] + fn the_score_carries_its_energy_provenance() { + let ci = intensity(); + let score = |cost: JouleCost| { + SciScore::compute(SciInputs { + joule_cost: &cost, + intensity: &ci, + pue: &Pue::new(1.2), + embodied_g_co2e: 0.0, + functional_units: 1.0, + }) + }; + let measured = score(JouleCost::measured(1_000_000)); + let estimated = score(JouleCost::estimated(1_000_000)); + assert_eq!(measured.energy_source, JouleSource::Measured); + assert_eq!(estimated.energy_source, JouleSource::Estimated); + assert_eq!(measured.sci, estimated.sci); + } + + /// A score written before `energy_source` existed recorded no + /// provenance, so it reads back as an estimate. + #[test] + fn a_score_without_provenance_reads_back_estimated() { + let json = r#"{"energy_kwh":1.0,"operational_g_co2e":1.0,"embodied_g_co2e":0.0, + "functional_units":1.0,"sci":1.0}"#; + let back: SciScore = serde_json::from_str(json).unwrap(); + assert_eq!(back.energy_source, JouleSource::Estimated); + } + + /// An eval report's SCI score is measured exactly when the report is. + #[cfg(feature = "eval-bridge")] + #[test] + fn an_eval_reports_score_keeps_the_reports_provenance() { + let report = |joule_source| eoc_eval::EvalReport { + harness: "fixed".into(), + model: "test".into(), + total_cases: 4, + correct: 2, + score: 0.5, + metric: eoc_eval::Metric::Accuracy, + joules_per_correct: 1_000_000, + joule_source, + latency_p50_ms: 1, + latency_p95_ms: 1, + }; + let ci = intensity(); + let pue = Pue::new(1.2); + let estimated = sci_from_eval_report(&report(JouleSource::Estimated), &ci, &pue, 0.0); + let measured = sci_from_eval_report(&report(JouleSource::Measured), &ci, &pue, 0.0); + assert_eq!(estimated.energy_source, JouleSource::Estimated); + assert_eq!(measured.energy_source, JouleSource::Measured); + // 2 correct × 1 J, × 1.2 PUE = 2.4 J either way. + assert!((estimated.energy_kwh - 2.4 / 3_600_000.0).abs() < 1e-15); + assert_eq!(estimated.sci, measured.sci); + } } diff --git a/eoc/eoc-rs/crates/eoc-carbon/tests/sci_for_ai.rs b/eoc/eoc-rs/crates/eoc-carbon/tests/sci_for_ai.rs index f1b8f10a3..f19cc23d6 100644 --- a/eoc/eoc-rs/crates/eoc-carbon/tests/sci_for_ai.rs +++ b/eoc/eoc-rs/crates/eoc-carbon/tests/sci_for_ai.rs @@ -1,14 +1,15 @@ -//! End-to-end: joules (from eoc-meter stub) → CarbonAccount → SCI value. +//! End-to-end: joules → CarbonAccount → SCI value. //! //! We exercise the joule pipeline by using `eoc_meter::detect()` which //! always returns *some* counter on every platform, then drive the SCI -//! computation with a deterministic test reading. +//! computation with a deterministic test figure. No counter read that +//! figure, so it is an estimate, and the score says so. use chrono::Utc; use eoc_carbon::intensity::{CarbonIntensity, IntensityKind, ProviderKind}; use eoc_carbon::pue::Pue; use eoc_carbon::sci_bridge::{SciInputs, SciScore}; -use eoc_core::JouleCost; +use eoc_core::{JouleCost, JouleSource}; #[test] fn full_request_to_sci_uses_meter_and_carbon_pipeline() { @@ -18,8 +19,10 @@ fn full_request_to_sci_uses_meter_and_carbon_pipeline() { let _ = counter.read_microjoules(); assert!(!counter.name().is_empty()); - // Step 2 — pretend the meter reported 2000 J on the stub backend. - let joule_cost = JouleCost::measured(2_000_000_000); + // Step 2 — a fixed 2000 J for the request. No counter read it, so it + // is an estimate: the stub reads 0, and a live counter would not read + // this figure. + let joule_cost = JouleCost::estimated(2_000_000_000); // Step 3 — grid intensity for the request's region. let intensity = CarbonIntensity::new( @@ -49,4 +52,6 @@ fn full_request_to_sci_uses_meter_and_carbon_pipeline() { assert!((sci.operational_g_co2e - 0.03667).abs() < 1e-4); assert_eq!(sci.embodied_g_co2e, 0.05); assert!((sci.sci - 0.08667).abs() < 1e-4); + // The score is no more measured than the joules it was computed from. + assert_eq!(sci.energy_source, JouleSource::Estimated); } diff --git a/eoc/eoc-rs/crates/eoc-eval/src/runner.rs b/eoc/eoc-rs/crates/eoc-eval/src/runner.rs index c3c818cc4..f910792ed 100644 --- a/eoc/eoc-rs/crates/eoc-eval/src/runner.rs +++ b/eoc/eoc-rs/crates/eoc-eval/src/runner.rs @@ -6,7 +6,7 @@ use std::sync::Arc; use serde::{Deserialize, Serialize}; -use eoc_core::Query; +use eoc_core::{JouleSource, Query}; use eoc_neural::NeuralBackend; use crate::error::Result; @@ -77,6 +77,8 @@ impl EvalRunner { let mut total_score = 0.0f64; let mut total_microjoules: u128 = 0; + // Whether every case's cost so far was read from a live counter. + let mut every_case_measured = true; let mut latencies: Vec = Vec::with_capacity(total_cases); for case in &cases { @@ -85,6 +87,7 @@ impl EvalRunner { let core_resp = self.backend.infer(&q).await; let latency_ms = start.elapsed().as_millis().min(u64::MAX as u128) as u64; total_microjoules += u128::from(core_resp.joule_cost.microjoules); + every_case_measured &= core_resp.joule_cost.source == JouleSource::Measured; latencies.push(latency_ms); let response = Response::from_core(core_resp, latency_ms); let s = self.harness.score(&response, &case.expected).await; @@ -107,6 +110,14 @@ impl EvalRunner { (total_microjoules / (correct as u128)).min(u64::MAX as u128) as u64 }; + // A total is only as good as its weakest part: one case that was not + // measured makes it an estimate, and a run with no cases read nothing. + let joule_source = if total_cases > 0 && every_case_measured { + JouleSource::Measured + } else { + JouleSource::Estimated + }; + latencies.sort_unstable(); let latency_p50_ms = percentile(&latencies, 0.50); let latency_p95_ms = percentile(&latencies, 0.95); @@ -119,6 +130,7 @@ impl EvalRunner { score, metric, joules_per_correct, + joule_source, latency_p50_ms, latency_p95_ms, }) @@ -144,12 +156,25 @@ pub struct EvalReport { pub metric: Metric, /// Micro-joules per correct case. `u64::MAX` when zero correct. pub joules_per_correct: u64, + /// Where the energy behind `joules_per_correct` came from: `Measured` + /// only when every case's cost was read from a live counter, `Estimated` + /// otherwise, including a run with no cases. A report serialized without + /// this field reads back as `Estimated`: nothing in it says it was + /// measured. + #[serde(default = "unrecorded_joule_source")] + pub joule_source: JouleSource, /// Latency p50 in milliseconds. pub latency_p50_ms: u64, /// Latency p95 in milliseconds. pub latency_p95_ms: u64, } +/// The provenance of a report that recorded none: an estimate, never a +/// measurement. +fn unrecorded_joule_source() -> JouleSource { + JouleSource::Estimated +} + /// Nearest-rank percentile (no interpolation). fn percentile(sorted: &[u64], q: f64) -> u64 { if sorted.is_empty() { @@ -248,6 +273,7 @@ mod tests { score: 1.0, metric: Metric::Accuracy, joules_per_correct: 100, + joule_source: JouleSource::Measured, latency_p50_ms: 1, latency_p95_ms: 1, }; @@ -255,5 +281,65 @@ mod tests { let back: EvalReport = serde_json::from_str(&json).unwrap(); assert_eq!(back.harness, "mmlu"); assert_eq!(back.correct, 1); + assert_eq!(back.joule_source, JouleSource::Measured); + } + + /// A report written before `joule_source` existed recorded no + /// provenance, so it reads back as an estimate, never a measurement. + #[test] + fn a_report_without_provenance_reads_back_estimated() { + let json = r#"{"harness":"mmlu","model":"m","total_cases":1,"correct":1, + "score":1.0,"metric":"Accuracy","joules_per_correct":100, + "latency_p50_ms":1,"latency_p95_ms":1}"#; + let back: EvalReport = serde_json::from_str(json).unwrap(); + assert_eq!(back.joule_source, JouleSource::Estimated); + } + + /// A backend whose every answer reports the given cost. + struct Costing(JouleCost); + #[async_trait] + impl NeuralBackend for Costing { + async fn infer(&self, q: &CoreQuery) -> CoreResponse { + CoreResponse::new(q.id, "yes".into(), Stage::Neural, self.0) + } + } + + /// One case's cost is measured exactly when its backend's is; a total + /// with one estimated case in it is an estimate. + #[tokio::test] + async fn the_report_is_measured_only_when_every_case_was() { + let run = |backend: Arc| async move { + let harness = Box::new(FixedHarness { n_cases: 4, correct: "yes" }); + EvalRunner::new(harness, backend).run().await.unwrap() + }; + let measured = run(Arc::new(Costing(JouleCost::measured(10)))).await; + assert_eq!(measured.joule_source, JouleSource::Measured); + let estimated = run(Arc::new(Costing(JouleCost::estimated(10)))).await; + assert_eq!(estimated.joule_source, JouleSource::Estimated); + + // Measured, measured, then estimated from the third answer on. + struct Mixed(AtomicUsize); + #[async_trait] + impl NeuralBackend for Mixed { + async fn infer(&self, q: &CoreQuery) -> CoreResponse { + let cost = if self.0.fetch_add(1, Ordering::Relaxed) < 2 { + JouleCost::measured(10) + } else { + JouleCost::estimated(10) + }; + CoreResponse::new(q.id, "yes".into(), Stage::Neural, cost) + } + } + let mixed = run(Arc::new(Mixed(AtomicUsize::new(0)))).await; + assert_eq!(mixed.joule_source, JouleSource::Estimated); + + let none = EvalRunner::new( + Box::new(FixedHarness { n_cases: 0, correct: "yes" }), + Arc::new(Costing(JouleCost::measured(10))), + ) + .run() + .await + .unwrap(); + assert_eq!(none.joule_source, JouleSource::Estimated, "no case ran, nothing was read"); } } -- 2.54.0 (Apple Git-157) From 6a8d7c68ab48c5c2d6a41d97b65f6d9c4d1bb9ac Mon Sep 17 00:00:00 2001 From: Transaction Science Date: Wed, 30 Sep 2026 20:17:36 -0400 Subject: [PATCH 11/22] =?UTF-8?q?eoc-streaming:=20a=20stream=20nothing=20m?= =?UTF-8?q?etered=20has=20no=20energy=20figure,=20never=200=20=C2=B5J?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit 467acb38 made a stream's energy measured only when its counter advanced. It left this as not done: `StreamAccount` had no way to say "not metered". So a stream metered by the stub, by a counter whose read failed, or by one that did not move reported `0 µJ (estimated)`. That is work that ran, presented as a figure of zero. Saying so honestly needed a change to a public type. Nothing outside this file uses `StreamAccount` or `StreamMeter`, in this workspace or any other in the monorepo, so the change is contained. The fix is the honest one, not a duration-based estimate: the account does not invent a figure. Any per-second or per-token coefficient would be a number nobody measured, and callers already hold their own estimates. - `StreamAccount.microjoules` is now `Option`: `Some` only when the counter read at the start and at the latest checkpoint and advanced between them, and `None` otherwise. The separate `measured: bool` is gone, since the `Option` says it. - `joule_cost()` returns `Option`: `Some(JouleCost::measured(..))` for a metered stream and `None` for one nothing metered. There is no estimated zero; a caller with no reading supplies its own estimate and tags it as one. - Token counts are kept, metered or not. A failed later read still keeps the last reading, and a failed start still never turns a later read into the counter's since-boot total. Tests: a counter that holds gives `{ tokens: 3, microjoules: None }` and no cost. A counter that advances 450 µJ gives `Some(measured(450))`. A failed start gives `None`. A stream that advanced 200 µJ keeps that figure when a later read fails. The stub gives five tokens and no figure, and so does `StreamAccount::default()`. Falsified, each mutant restored byte-for-byte: - recording a zero delta as the stream's figure fails three tests (`left: Some(0)`, `right: None`); - returning HEAD's `0 µJ (estimated)` for an unmetered stream fails three (`left: Some(JouleCost { microjoules: 0, source: Estimated })`). Co-Authored-By: Claude Opus 5.5 --- .../crates/eoc-streaming/src/account.rs | 102 ++++++++++-------- 1 file changed, 56 insertions(+), 46 deletions(-) diff --git a/eoc/eoc-rs/crates/eoc-streaming/src/account.rs b/eoc/eoc-rs/crates/eoc-streaming/src/account.rs index fea7019e4..97a4d8cd4 100644 --- a/eoc/eoc-rs/crates/eoc-streaming/src/account.rs +++ b/eoc/eoc-rs/crates/eoc-streaming/src/account.rs @@ -3,42 +3,41 @@ //! A [`StreamMeter`] samples a [`JouleCounter`](eoc_meter::JouleCounter) //! at the start of a stream and again at the end (or on each //! checkpoint). The difference is attributed to the stream and surfaced -//! as a [`JouleCost`](eoc_core::JouleCost). Token counts can be -//! accumulated alongside so the cascade sees both axes. +//! as a [`JouleCost`] when, and only when, the counter advanced over the +//! span. A stream the counter did not meter has no figure at all, never +//! 0 µJ: its work ran, and the caller keeps its own estimate. Token counts +//! are accumulated alongside, metered or not, so the cascade sees both axes. use std::sync::Arc; use std::sync::Mutex; -use eoc_core::{JouleCost, JouleSource}; +use eoc_core::JouleCost; use eoc_meter::JouleCounter; use crate::error::{StreamError, StreamResult}; /// Per-stream accounting record. -#[derive(Debug, Clone, Copy, Default)] +#[derive(Debug, Clone, Copy, Default, PartialEq, Eq)] pub struct StreamAccount { /// Tokens streamed so far. pub tokens: u64, - /// Cumulative micro-joule delta attributed to this stream. - pub microjoules: u64, - /// Whether `microjoules` is a hardware counter's reading of this span: - /// the counter read at the start and at the latest checkpoint, and - /// advanced between them. A zero delta, a failed read or the stub leaves - /// it `false`: a counter that did not move has not measured the work. - pub measured: bool, + /// A hardware counter's reading of this stream, in micro-joules: the + /// counter read at the start and at the latest checkpoint, and advanced + /// between them. `None` when the stream was not metered — the stub, a + /// failed read, a counter that did not move (a frozen rail, a + /// placeholder), or no checkpoint yet. Not metered is not 0 µJ: the work + /// ran, and no counter read what it cost. + pub microjoules: Option, } impl StreamAccount { - /// Convert to the canonical [`JouleCost`] type. - pub fn joule_cost(&self) -> JouleCost { - JouleCost { - microjoules: self.microjoules, - source: if self.measured { - JouleSource::Measured - } else { - JouleSource::Estimated - }, - } + /// The stream's cost as the canonical [`JouleCost`]: `Some`, and + /// [`JouleSource::Measured`](eoc_core::JouleSource::Measured), when a + /// counter metered it; `None` when nothing did. There is deliberately no + /// estimated zero here: a caller with no reading supplies its own + /// estimate (from tokens, or time) and tags it as one. + pub fn joule_cost(&self) -> Option { + self.microjoules.map(JouleCost::measured) } } @@ -67,9 +66,9 @@ impl std::fmt::Debug for StreamMeter { } impl StreamMeter { - /// Start metering a stream. Reads the counter once; if the read - /// fails the meter degrades to `Estimated` and continues. Nothing is - /// measured until a checkpoint sees the counter advance. + /// Start metering a stream. Reads the counter once; if the read fails + /// the stream is never metered, and it continues counting tokens. + /// Nothing is metered until a checkpoint sees the counter advance. pub fn start(counter: Arc) -> Self { let start_uj = counter.read_microjoules().ok(); Self { @@ -77,8 +76,7 @@ impl StreamMeter { start_uj, state: Arc::new(Mutex::new(StreamAccount { tokens: 0, - microjoules: 0, - measured: false, + microjoules: None, })), } } @@ -93,10 +91,10 @@ impl StreamMeter { /// Refresh the joule delta from the counter. Cheap and safe to /// call at every checkpoint; the cost is one counter read. /// - /// The delta is a measurement only if the counter advanced since the - /// start. A counter that reads but does not move — a frozen rail, a - /// placeholder — has not measured the work, and its zero is never - /// sealed as one. + /// The delta is a reading only if the counter advanced since the start. + /// A counter that reads but does not move — a frozen rail, a placeholder + /// — has not measured the work: the stream is left not metered, never + /// given its zero. pub fn checkpoint(&self) -> StreamResult { let (Some(start_uj), Ok(now_uj)) = (self.start_uj, self.counter.read_microjoules()) else { // No baseline, or no reading now: nothing to measure. Leave the @@ -105,8 +103,7 @@ impl StreamMeter { return Ok(*g); }; let mut g = self.state.lock().map_err(poisoned)?; - g.microjoules = now_uj.saturating_sub(start_uj); - g.measured = now_uj > start_uj; + g.microjoules = (now_uj > start_uj).then(|| now_uj - start_uj); Ok(*g) } @@ -130,6 +127,7 @@ fn poisoned(_: T) -> StreamError { #[cfg(test)] mod tests { use super::*; + use eoc_core::JouleSource; use eoc_meter::StubCounter; /// A counter that reads the scripted values in turn (then the last one @@ -157,16 +155,16 @@ mod tests { } } - /// A non-stub counter that reads but does not move has not measured the - /// stream: its zero is estimated, never sealed as a measurement. + /// A non-stub counter that reads but does not move has not metered the + /// stream: it has no figure, never a zero sealed as one. #[test] fn a_counter_that_did_not_advance_is_not_a_measurement() { let meter = StreamMeter::start(Scripted::new(&[Some(7_000)])); - assert!(!meter.snapshot().unwrap().measured, "measured before any checkpoint"); + assert_eq!(meter.snapshot().unwrap().microjoules, None, "metered before any checkpoint"); meter.on_token(3).unwrap(); let acct = meter.finish().unwrap(); - assert_eq!((acct.microjoules, acct.measured), (0, false)); - assert_eq!(acct.joule_cost().source, JouleSource::Estimated); + assert_eq!(acct, StreamAccount { tokens: 3, microjoules: None }); + assert_eq!(acct.joule_cost(), None); } /// A counter that advanced over the span is a measurement of it. @@ -174,10 +172,11 @@ mod tests { fn a_counter_that_advanced_is_a_measurement() { let meter = StreamMeter::start(Scripted::new(&[Some(7_000), Some(7_000), Some(7_450)])); let mid = meter.checkpoint().unwrap(); - assert_eq!((mid.microjoules, mid.measured), (0, false), "no advance yet"); + assert_eq!(mid.microjoules, None, "no advance yet"); let acct = meter.finish().unwrap(); - assert_eq!((acct.microjoules, acct.measured), (450, true)); - assert_eq!(acct.joule_cost(), JouleCost::measured(450)); + assert_eq!(acct.microjoules, Some(450)); + assert_eq!(acct.joule_cost(), Some(JouleCost::measured(450))); + assert_eq!(acct.joule_cost().unwrap().source, JouleSource::Measured); } /// Without a starting reading there is no span: a later reading is the @@ -186,18 +185,29 @@ mod tests { fn a_failed_start_measures_nothing() { let meter = StreamMeter::start(Scripted::new(&[None, Some(9_000_000)])); let acct = meter.finish().unwrap(); - assert_eq!((acct.microjoules, acct.measured), (0, false)); - assert_eq!(acct.joule_cost().source, JouleSource::Estimated); + assert_eq!(acct.microjoules, None); + assert_eq!(acct.joule_cost(), None); } + /// A counter that advanced once keeps that reading when a later read + /// fails, and never turns it into the counter's since-boot total. #[test] - fn stub_counter_estimated() { + fn a_failed_later_read_keeps_the_last_reading() { + let meter = StreamMeter::start(Scripted::new(&[Some(1_000), Some(1_200), None])); + assert_eq!(meter.checkpoint().unwrap().microjoules, Some(200)); + assert_eq!(meter.finish().unwrap().microjoules, Some(200)); + } + + /// A stream the stub "metered" streamed its tokens and has no energy + /// figure: not 0 µJ, which would present work that ran as free. + #[test] + fn an_unmetered_stream_has_no_figure() { let meter = StreamMeter::start(Arc::new(StubCounter)); meter.on_token(5).unwrap(); let acct = meter.finish().unwrap(); assert_eq!(acct.tokens, 5); - assert_eq!(acct.microjoules, 0); - assert!(!acct.measured); - assert_eq!(acct.joule_cost().source, JouleSource::Estimated); + assert_eq!(acct.microjoules, None); + assert_eq!(acct.joule_cost(), None); + assert_eq!(StreamAccount::default().joule_cost(), None); } } -- 2.54.0 (Apple Git-157) From 722d80d90e7f82ee539aa076cac3364b958cee8e Mon Sep 17 00:00:00 2001 From: Transaction Science Date: Wed, 30 Sep 2026 20:17:51 -0400 Subject: [PATCH 12/22] eoc-meter, eoc-local: say what a counter's provenance is, and what each backend reports without one MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Docs only. Each statement changed was false against the code. - eoc-meter's module doc said `StubCounter` "reports its source as `JouleSource::Estimated`". `JouleCounter` carries no source; it only reads. The doc now says consumers decide: eoc-cascade, eoc-streaming and eoc-local's `mlx`, `mlc` and `onnx` call a delta measured only when the counter advanced over their span. It also says what each does when nothing was read: no figure, the stage's own cost, or an estimate. - Where the IOReport counter is described (the module doc and `MacosIoReportCounter`): a live IOReport reading is a vendor energy counter's reading, so its delta is `Measured`, as RAPL's and NVML's are. That holds only while the rail is live, which is why the counter is never constructed when it is not. The struct's "Measured-grade" tag is replaced by that sentence. - The powermetrics stand-in's read comment said the error makes "the cascade attach `JouleSource::Estimated`". The cascade keeps the stage's own cost, whatever its tag; the comment now says no consumer gets a figure from it. The rung's doc, that it "never degrades to `StubCounter`", was already corrected in 3b827e2a. - eoc-local's "Joule attribution" said every backend reports the delta as measured and that a `StubCounter` yields zero cost. With nothing measured, `mlx` falls back to its per-token coefficient; `mlc` and `onnx` report 0 µJ, estimated; `llamacpp` reports the delta since its first read, estimated if any chunk saw a zero delta. Each is now stated per backend. The crate intro said local backends measure; they can, when the host has a live counter. - mlx's module doc said attribution "uses `powermetrics`". It reads whatever counter it is given, and the powermetrics stand-in never reads. The `joule_attribution` test's doc said the stub always yields 0, but its own mlx test asserts the per-token estimate. Checked: `cargo doc` raises no new warning on the crates touched, and `cargo test -p eoc-local --features mlc,gguf` passes. The `mlx` and `llamacpp` modules need native dependencies (mlx-rs, llama.cpp) that are not built here, so their doc changes are not compiled. They are comment text and an intra-doc link to `JouleCounter`, which that module imports. Not done, found while making these true. Both are behaviour, not docs: - `llamacpp` takes a failed first read as 0 (`unwrap_or(0)`), so a later successful read reports the counter's since-boot total, tagged measured. This is the defect 467acb38 fixed in eoc-streaming. - `mlc` and `onnx` report 0 µJ for inference that ran when nothing was measured. It is tagged estimated, but it is still a zero presented as a figure. Co-Authored-By: Claude Opus 5.5 --- eoc/eoc-rs/crates/eoc-local/src/lib.rs | 32 ++++++++++++++----- eoc/eoc-rs/crates/eoc-local/src/mlx.rs | 8 +++-- .../eoc-local/tests/joule_attribution.rs | 9 +++--- eoc/eoc-rs/crates/eoc-meter/src/lib.rs | 27 ++++++++++++---- 4 files changed, 55 insertions(+), 21 deletions(-) diff --git a/eoc/eoc-rs/crates/eoc-local/src/lib.rs b/eoc/eoc-rs/crates/eoc-local/src/lib.rs index 262421e8b..289467794 100644 --- a/eoc/eoc-rs/crates/eoc-local/src/lib.rs +++ b/eoc/eoc-rs/crates/eoc-local/src/lib.rs @@ -4,9 +4,9 @@ //! the same [`eoc_neural::NeuralBackend`] trait that //! [`eoc-vendor-api`](https://docs.rs/eoc-vendor-api) uses for commercial //! endpoints. Where vendor APIs *estimate* joule cost from token counts, -//! local backends *measure* it — the host hardware is right there, and -//! [`eoc_meter::JouleCounter`] readings taken on either side of the -//! inference give an honest reading. +//! local backends can *measure* it — the host hardware is right there, and +//! when the host has a live counter, [`eoc_meter::JouleCounter`] readings +//! taken on either side of the inference give an honest reading. //! //! ## Backends (all feature-gated) //! @@ -23,11 +23,27 @@ //! //! ## Joule attribution //! -//! Each backend takes a [`JouleCounter`](eoc_meter::JouleCounter), reads -//! it before and after inference, and reports the delta as -//! [`JouleSource::Measured`](eoc_core::JouleSource::Measured). When the -//! attached counter is [`StubCounter`](eoc_meter::StubCounter), cost is -//! zero — useful for tests, deterministic playback, and CI. +//! Each backend takes a [`JouleCounter`](eoc_meter::JouleCounter) and reads +//! it around the inference. +//! +//! `mlx`, `mlc` and `onnx` read it once before and once after. When both +//! reads succeed and the counter advanced, the delta is a live hardware +//! counter's reading, reported as +//! [`JouleSource::Measured`](eoc_core::JouleSource::Measured). Otherwise +//! nothing was measured: the [`StubCounter`](eoc_meter::StubCounter), which +//! always reads 0; a counter that read but did not move; or one whose read +//! fails, such as the powermetrics stand-in that [`eoc_meter::detect`] +//! selects on a Mac whose IOReport CPU rail is not live. The cost is then tagged +//! [`JouleSource::Estimated`](eoc_core::JouleSource::Estimated), and holds: +//! +//! - for `mlx`, its per-token coefficient (`fallback_joules_per_token` × +//! tokens), an estimated figure; +//! - for `mlc` and `onnx`, 0 µJ: they have no coefficient to estimate with. +//! +//! `llamacpp` reads the counter as it streams and reports the delta since +//! its first read (taken as 0 if that read fails), tagged estimated if any +//! streamed chunk saw a zero delta, and measured otherwise. With the stub +//! that is 0 µJ, estimated. //! //! ## WASM //! diff --git a/eoc/eoc-rs/crates/eoc-local/src/mlx.rs b/eoc/eoc-rs/crates/eoc-local/src/mlx.rs index d4ef866cd..694ad5b70 100644 --- a/eoc/eoc-rs/crates/eoc-local/src/mlx.rs +++ b/eoc/eoc-rs/crates/eoc-local/src/mlx.rs @@ -5,9 +5,11 @@ //! choice on M-series Macs for models that have been converted to MLX //! format. The Rust binding lives in [`mlx-rs`]. //! -//! Joule attribution on macOS uses `powermetrics`, which is already -//! wired into [`eoc_meter`]. `powermetrics` requires root in practice; -//! when readings fail, the backend tags joule cost as +//! Joule attribution reads whatever [`JouleCounter`] the backend is given +//! (on a Mac, [`eoc_meter::detect`] picks IOReport only when its CPU rail is +//! live; its `powermetrics` stand-in never reads). A delta from a counter +//! that advanced is reported as measured; when the reads fail or the counter +//! does not move, the backend tags joule cost as //! [`JouleSource::Estimated`](eoc_core::JouleSource::Estimated) and //! falls back to a coefficient-based estimate. //! diff --git a/eoc/eoc-rs/crates/eoc-local/tests/joule_attribution.rs b/eoc/eoc-rs/crates/eoc-local/tests/joule_attribution.rs index b8de2e0a4..b08f6f5e1 100644 --- a/eoc/eoc-rs/crates/eoc-local/tests/joule_attribution.rs +++ b/eoc/eoc-rs/crates/eoc-local/tests/joule_attribution.rs @@ -1,9 +1,10 @@ //! Joule-attribution integration test. //! -//! Verifies the contract: when the meter is a [`StubCounter`] the -//! reported joule cost is 0; when the meter is a non-stub counter -//! whose readings monotonically advance, the reported joule cost is -//! non-zero. +//! Verifies the contract: when the meter is a [`StubCounter`] nothing +//! was measured, so the cost is an estimate (0 µJ for `mlc`, the +//! per-token coefficient for `mlx`); when the meter is a non-stub +//! counter whose readings monotonically advance, the reported joule +//! cost is non-zero and measured. //! //! These tests exercise the *plumbing* — the backends themselves use //! their respective placeholder implementations in the reference diff --git a/eoc/eoc-rs/crates/eoc-meter/src/lib.rs b/eoc/eoc-rs/crates/eoc-meter/src/lib.rs index cd41e6259..cc8e691ba 100644 --- a/eoc/eoc-rs/crates/eoc-meter/src/lib.rs +++ b/eoc/eoc-rs/crates/eoc-meter/src/lib.rs @@ -3,7 +3,16 @@ //! The meter reads cumulative micro-joule counters from whatever hardware //! is available. Production deployments care a great deal which counter //! is in use; reference deployments fall back to `StubCounter`, which -//! always returns 0 and reports its source as `JouleSource::Estimated`. +//! always reads 0. +//! +//! A counter carries no provenance of its own: [`JouleCounter`] only reads, +//! and its consumers decide. eoc-cascade, eoc-streaming and eoc-local's +//! `mlx`, `mlc` and `onnx` backends call a delta `JouleSource::Measured` +//! only when the counter advanced over the span they read it across. +//! Anything else — a failed read, a counter that did not move, the stub, +//! which never moves — is no reading, and yields no measured figure: +//! eoc-streaming reports none, eoc-cascade keeps the answering stage's own +//! cost as the stage reported it, and those backends report an estimate. //! //! Hardware counter dependencies are **platform-conditional and //! feature-gated**: @@ -17,7 +26,10 @@ //! summed rails then advance through the GPU rail, a stale refresh, or not //! at all, and would pass that off as a reading of CPU work. When the CPU //! rail is not live the counter is declined and [`detect`] falls through to -//! the next rung. +//! the next rung. A live IOReport counter is a vendor energy counter, as +//! RAPL and NVML are, so a delta it reads is `Measured` — but only while the +//! rail is live: the counter exists only on a host whose rail the probe +//! found live, and on any other there is no IOReport reading to tag. //! - macOS `powermetrics`: detected when the binary is on the PATH, but the //! reference wrapper does not parse it, so every read returns //! `Err(NoCounter)`: no figure, and callers keep their own estimate. @@ -1233,10 +1245,13 @@ mod macos_ioreport { } } - /// Apple Silicon IOReport energy counter (SoC-total, Measured-grade). + /// Apple Silicon IOReport energy counter (SoC-total). /// /// Constructed only on a host whose CPU energy rail is live; see - /// [`crate::ioreport_status`]. + /// [`crate::ioreport_status`]. A live IOReport reading is a vendor + /// energy counter's reading, so the delta it gives is `Measured`, as + /// RAPL's and NVML's are; that holds only while the rail is live, which + /// is why the counter is never constructed when it is not. pub struct MacosIoReportCounter { state: Mutex, } @@ -3304,8 +3319,8 @@ mod macos_powermetrics { fn read_microjoules(&self) -> Result { // Real implementation would parse `powermetrics --samplers // cpu_power -i 100 -n 1 --hide-cpu-duty-cycle`. For the - // reference implementation we always error so the cascade - // attaches `JouleSource::Estimated` instead. + // reference implementation we always error, so no consumer + // gets a figure from it and each keeps its own estimate. Err(Error::NoCounter) } fn name(&self) -> &'static str { -- 2.54.0 (Apple Git-157) From 15cf07c2eeb562ab22a9d552df0925ec64fa84af Mon Sep 17 00:00:00 2001 From: Transaction Science Date: Wed, 30 Sep 2026 21:06:55 -0400 Subject: [PATCH 13/22] =?UTF-8?q?eoc-local:=20a=20span=20is=20measured=20o?= =?UTF-8?q?nly=20when=20both=20reads=20succeed=20and=20the=20counter=20adv?= =?UTF-8?q?anced,=20and=20inference=20that=20ran=20is=20never=200=20=C2=B5?= =?UTF-8?q?J?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit 722d80d9 listed two defects here as not done. `llamacpp` took a failed first counter read as 0 (`unwrap_or(0)`), so the next successful read reported the counter's whole since-boot total as the generation's energy. Its drain then tagged that figure measured unless some chunk had seen a zero delta. This is the defect 467acb38 fixed in eoc-streaming. And `mlc` and `onnx` reported 0 µJ for inference that ran whenever nothing was measured: tagged estimated, but still a zero presented as a figure. The rule now lives once, in a new `span` module, and every backend meters through it: - `MeterSpan::open` reads the counter; `reading` gives `Some(delta)` only when that opening read succeeded, the read now succeeds, and the counter advanced between them. A failed read at either end, a counter that held, one that went down, or the stub: `None`, never 0 and never a since-boot total. - `span_cost` makes a reading `JouleCost::measured` and asks for the estimate only when there was none. - `per_token_estimate` is tokens × `fallback_joules_per_token`, to the nearest µJ, and never 0: at least one token is counted, a coefficient that is not a finite positive number gives way to the default (0.05 J/token, mlx's figure since the start), and a product under half a µJ is 1 µJ. Per backend: - `llamacpp`: `TokenChunk::cumulative_microjoules` is now `Option`, the reading so far, `None` when nothing was measured. The inference's cost is the final chunk's reading, measured, or the per-token estimate over the tokens streamed. An earlier chunk's reading covers part of the generation, so it is never taken as the cost of the whole. The backend gains `fallback_joules_per_token` and its builder. Nothing outside this file uses `TokenChunk`. - `mlc` and `onnx` gain `fallback_joules_per_token` (default 0.05 J) and its builder, and estimate over the placeholder's own token count, as `mlx` does. - `mlx` already had the right rule; it now goes through the same functions, so a zero coefficient no longer yields 0 µJ. Factored so it tests without native code. Neither the `llamacpp` nor the `mlx` module binds a single symbol of its native crate (both generators are placeholders), so this crate's tests compile them without their features (`cfg(any(feature, test))`; mlx on macOS). Their metering now runs in every `cargo test --workspace`, where before it ran in none: llama.cpp is not built here, and `--features mlx` fails in mlx-sys's Metal kernel build on this toolchain. The `not(feature = "llamacpp")` error stub, unreachable since the module itself was gated on the feature, is gone. Tests, all on a scripted counter that reads a list (`None` is a failed read) and counts its reads: - span: an advance of 450 µJ reads 450; a failed first read followed by 9,000,000 reads nothing; so does a failed last read, a held counter, a counter that went down, and the stub. The estimate is 800,000 µJ for 8 × 0.1 J, 100,000 for 0 tokens at 0.1 J, the default for 0, -1, NaN and infinity, 1 µJ for 1e-12 J, and saturates at u64::MAX. - llamacpp, through `stream` and `infer`: the stub gives the 4-token estimate (200,000 µJ, estimated) and no chunk a reading; a failed first read then 9 J gives no chunk a reading and the estimate; 1,000 -> 1,100 ... 1,450 gives chunk readings 100, 200, 300, 400 and a measured 450 over exactly six reads; a held counter and a failed last read give the estimate. A table test pins that only the final chunk's reading is a cost. - mlc, onnx, mlx: measured 450 for an advance; the estimate (800,000 µJ) for a held counter, a failed first read and a failed last read; the stub's estimate, never 0. The `joule_attribution` integration test now asserts the mlc stub's estimate (500,000 µJ, estimated) where it asserted 0. Falsified, each mutant applied to the uncommitted tree, run, and restored byte-for-byte (sha256 checked): - a failed start taken as 0 fails 5 tests (llamacpp, mlc, mlx, onnx, span), the llamacpp one with every chunk at `Some(9000000)`; - `>=` in place of `>` (a held counter measured) fails 9; - llamacpp reading the counter's absolute value per chunk (HEAD's rule with a 0 baseline) fails 3; - a partial chunk reading kept as the cost fails the tally test; - mlc's and onnx's HEAD estimate of 0 fail 2 each; - the estimate without its 1 µJ floor, without the one-token minimum, or without the coefficient check fails the estimate test. One mutant survived and is equivalent: `final_reading = reading.or(previous)` cannot differ, since nothing sets the field before the final chunk. `cargo test -p eoc-local` passes (36 + 1), with `--features mlc,gguf` (47 + 3) and `--features onnx --lib` (43). The onnx leg runs `--lib` because the `joule_attribution` test binary cannot link ort-sys without ONNX Runtime installed when mlc is also enabled; that is unchanged. No clippy warning in this crate, before or after. Not done: the `llamacpp` error path still returns 0 µJ, estimated, with the error text. Whether any work ran before an error is unknown there. Co-Authored-By: Claude Opus 5.5 --- eoc/eoc-rs/crates/eoc-local/src/lib.rs | 48 ++-- eoc/eoc-rs/crates/eoc-local/src/llamacpp.rs | 232 +++++++++++++----- eoc/eoc-rs/crates/eoc-local/src/mlc.rs | 84 +++++-- eoc/eoc-rs/crates/eoc-local/src/mlx.rs | 67 +++-- eoc/eoc-rs/crates/eoc-local/src/onnx.rs | 87 +++++-- eoc/eoc-rs/crates/eoc-local/src/span.rs | 188 ++++++++++++++ .../eoc-local/tests/joule_attribution.rs | 19 +- 7 files changed, 557 insertions(+), 168 deletions(-) create mode 100644 eoc/eoc-rs/crates/eoc-local/src/span.rs diff --git a/eoc/eoc-rs/crates/eoc-local/src/lib.rs b/eoc/eoc-rs/crates/eoc-local/src/lib.rs index 289467794..308fe23ca 100644 --- a/eoc/eoc-rs/crates/eoc-local/src/lib.rs +++ b/eoc/eoc-rs/crates/eoc-local/src/lib.rs @@ -24,26 +24,27 @@ //! ## Joule attribution //! //! Each backend takes a [`JouleCounter`](eoc_meter::JouleCounter) and reads -//! it around the inference. +//! it around the inference, by one rule ([`span`]): //! -//! `mlx`, `mlc` and `onnx` read it once before and once after. When both -//! reads succeed and the counter advanced, the delta is a live hardware -//! counter's reading, reported as -//! [`JouleSource::Measured`](eoc_core::JouleSource::Measured). Otherwise -//! nothing was measured: the [`StubCounter`](eoc_meter::StubCounter), which -//! always reads 0; a counter that read but did not move; or one whose read -//! fails, such as the powermetrics stand-in that [`eoc_meter::detect`] -//! selects on a Mac whose IOReport CPU rail is not live. The cost is then tagged -//! [`JouleSource::Estimated`](eoc_core::JouleSource::Estimated), and holds: +//! - The inference is **measured** only when the counter read at the start, +//! read at the end, and advanced between them. The delta is then a live +//! hardware counter's reading, reported as +//! [`JouleSource::Measured`](eoc_core::JouleSource::Measured). +//! - Otherwise nothing was measured: the +//! [`StubCounter`](eoc_meter::StubCounter), which always reads 0; a counter +//! that read but did not move; or a read that failed at either end, such +//! as the powermetrics stand-in's, which [`eoc_meter::detect`] selects on a +//! Mac whose IOReport CPU rail is not live. A failed first read is never +//! taken as 0, which would report the counter's since-boot total. +//! - Inference that ran and that nothing measured is never 0 µJ. Every +//! backend then reports its per-token estimate, `fallback_joules_per_token` +//! (0.05 J by default, [`span::DEFAULT_FALLBACK_JOULES_PER_TOKEN`]) × the +//! tokens it generated, tagged +//! [`JouleSource::Estimated`](eoc_core::JouleSource::Estimated). //! -//! - for `mlx`, its per-token coefficient (`fallback_joules_per_token` × -//! tokens), an estimated figure; -//! - for `mlc` and `onnx`, 0 µJ: they have no coefficient to estimate with. -//! -//! `llamacpp` reads the counter as it streams and reports the delta since -//! its first read (taken as 0 if that read fails), tagged estimated if any -//! streamed chunk saw a zero delta, and measured otherwise. With the stub -//! that is 0 µJ, estimated. +//! `llamacpp` also reads the counter with each streamed token, so a chunk +//! carries the reading so far when there is one; the inference's cost is +//! the final read's. //! //! ## WASM //! @@ -65,10 +66,17 @@ pub mod tokenizer; #[cfg(any(feature = "gguf", feature = "llamacpp"))] pub mod gguf; -#[cfg(feature = "llamacpp")] +pub mod span; + +// The module binds no native symbol (its generator is a placeholder until an +// operator wires llama.cpp in), so this crate's tests compile it without the +// feature, and its metering is tested in every build. +#[cfg(any(feature = "llamacpp", test))] pub mod llamacpp; -#[cfg(all(feature = "mlx", target_os = "macos"))] +// Likewise binds no native symbol yet: compiled for this crate's tests on +// macOS without the feature. +#[cfg(all(any(feature = "mlx", test), target_os = "macos"))] pub mod mlx; #[cfg(feature = "mlc")] diff --git a/eoc/eoc-rs/crates/eoc-local/src/llamacpp.rs b/eoc/eoc-rs/crates/eoc-local/src/llamacpp.rs index e13994fbd..5958a36e0 100644 --- a/eoc/eoc-rs/crates/eoc-local/src/llamacpp.rs +++ b/eoc/eoc-rs/crates/eoc-local/src/llamacpp.rs @@ -11,15 +11,24 @@ //! | Linux + Intel Arc | SYCL | //! | otherwise | Vulkan (cross-vendor) or CPU | //! -//! Joule attribution: a [`JouleCounter`] reading is taken before and -//! after each inference. The delta is reported as -//! [`JouleSource::Measured`](eoc_core::JouleSource::Measured) when the -//! counter actually produced a number, else `Estimated`. +//! Joule attribution: a [`JouleCounter`] span is opened before generation +//! and read with each streamed token and once more at the end +//! ([`crate::span`]). The inference's cost is the final reading, +//! [`JouleSource::Measured`](eoc_core::JouleSource::Measured), only when +//! the counter read at the start, read at the end, and advanced between +//! them. Otherwise nothing was measured, and the cost is a per-token +//! estimate ([`LlamaCppBackend::fallback_joules_per_token`] × tokens +//! streamed), tagged estimated, never 0 µJ. //! //! Streaming: tokens are emitted through a `tokio::sync::mpsc` channel //! for incremental UIs. The non-streaming `infer` is implemented as //! "drain the stream". //! +//! The module's own logic needs no native code: the reference build's +//! generator below is a placeholder, so the module is compiled for this +//! crate's tests without the `llamacpp` feature, and its metering runs +//! under test on a scripted counter. +//! //! [`llama-cpp-2`]: https://crates.io/crates/llama-cpp-2 use std::path::PathBuf; @@ -34,6 +43,7 @@ use tokio::sync::mpsc; use crate::error::{LocalError, LocalResult}; use crate::model_registry::Quantization; use crate::sampling::{GreedySampler, Sampler}; +use crate::span::{self, MeterSpan}; /// llama.cpp backend. pub struct LlamaCppBackend { @@ -54,6 +64,9 @@ pub struct LlamaCppBackend { pub sampler: Box, /// Joule counter used to attribute energy cost. pub meter: Arc, + /// Estimated joules per streamed token, used when the counter did not + /// measure the inference ([`crate::span::per_token_estimate`]). + pub fallback_joules_per_token: f64, } /// Configuration for [`LlamaCppBackend`]. The runtime model object @@ -105,6 +118,7 @@ impl LlamaCppBackend { quantization: config.quantization, sampler: Box::new(GreedySampler), meter, + fallback_joules_per_token: span::DEFAULT_FALLBACK_JOULES_PER_TOKEN, }) } @@ -114,9 +128,16 @@ impl LlamaCppBackend { self } + /// Override the per-token estimate used when nothing was measured. + pub fn with_fallback_joules_per_token(mut self, j: f64) -> Self { + self.fallback_joules_per_token = j; + self + } + /// Streaming inference: emits one token per send. The receiver is - /// closed when generation completes. Each chunk includes the - /// running joule estimate. + /// closed when generation completes. Each chunk carries what the + /// counter has read since generation started, when it has read + /// anything ([`TokenChunk::cumulative_microjoules`]). /// /// Implemented as `spawn_blocking` so the synchronous llama.cpp /// call doesn't starve the tokio reactor. @@ -139,8 +160,12 @@ impl LlamaCppBackend { pub struct TokenChunk { /// Decoded UTF-8 fragment for this token. pub text: String, - /// Cumulative joule cost since the start of generation. - pub cumulative_microjoules: u64, + /// What the counter has read since generation started, in micro-joules: + /// `Some` only when it read at the start, read for this chunk, and + /// advanced between the two. `None` when nothing was measured — a failed + /// read, the stub, a counter that did not move — which is not 0 µJ: the + /// tokens were generated. + pub cumulative_microjoules: Option, /// Whether this is the last chunk. pub final_chunk: bool, } @@ -151,16 +176,12 @@ impl NeuralBackend for LlamaCppBackend { // Drain the stream into a single response. let mut rx = self.stream(q).await; let mut text = String::new(); - let mut last_cum = 0u64; - let mut source = JouleSource::Measured; + let mut tally = StreamTally::default(); while let Some(item) = rx.recv().await { match item { Ok(chunk) => { text.push_str(&chunk.text); - last_cum = chunk.cumulative_microjoules; - if chunk.cumulative_microjoules == 0 { - source = JouleSource::Estimated; - } + tally.observe(&chunk); } Err(e) => { return Response::new( @@ -175,19 +196,40 @@ impl NeuralBackend for LlamaCppBackend { } } } - Response::new( - q.id, - text, - Stage::Neural, - JouleCost { - microjoules: last_cum, - source, - }, - ) + Response::new(q.id, text, Stage::Neural, tally.cost(self.fallback_joules_per_token)) + } +} + +/// What a drained stream says about its cost: how many tokens it carried, +/// and the final chunk's reading. +#[derive(Debug, Default, Clone, Copy, PartialEq, Eq)] +struct StreamTally { + /// Chunks that carried a token. + tokens: u64, + /// The final chunk's reading: what the counter read over the whole + /// generation, when it read anything. `None` until the final chunk, and + /// after it when nothing was measured. + final_reading: Option, +} + +impl StreamTally { + fn observe(&mut self, chunk: &TokenChunk) { + if chunk.final_chunk { + self.final_reading = chunk.cumulative_microjoules; + } else if !chunk.text.is_empty() { + self.tokens += 1; + } + } + + /// The generation's cost: the final reading, measured, when the counter + /// measured the whole span; otherwise the per-token estimate. A reading + /// from an earlier chunk covers only part of the generation, so it is + /// never the cost of the whole. + fn cost(&self, joules_per_token: f64) -> JouleCost { + span::span_cost(self.final_reading, || span::per_token_estimate(self.tokens, joules_per_token)) } } -#[cfg(feature = "llamacpp")] fn run_llamacpp_blocking( prompt: &str, model_path: &std::path::Path, @@ -221,7 +263,7 @@ fn run_llamacpp_blocking( // plumbing. Operators wire in the real call against their pinned // llama-cpp-2 revision. - let start = meter.read_microjoules().unwrap_or(0); + let span = MeterSpan::open(meter); let mut out = String::new(); let n = max_tokens.min(64) as usize; for i in 0..n { @@ -243,15 +285,10 @@ fn run_llamacpp_blocking( break; } out.push_str(&frag); - let cum = meter - .read_microjoules() - .ok() - .map(|now| now.saturating_sub(start)) - .unwrap_or(0); if tx .blocking_send(Ok(TokenChunk { text: frag, - cumulative_microjoules: cum, + cumulative_microjoules: span.reading(meter), final_chunk: false, })) .is_err() @@ -259,37 +296,18 @@ fn run_llamacpp_blocking( break; } } - let cum = meter - .read_microjoules() - .ok() - .map(|now| now.saturating_sub(start)) - .unwrap_or(0); let _ = tx.blocking_send(Ok(TokenChunk { text: String::new(), - cumulative_microjoules: cum, + cumulative_microjoules: span.reading(meter), final_chunk: true, })); let _ = out; } -#[cfg(not(feature = "llamacpp"))] -#[allow(dead_code)] -fn run_llamacpp_blocking( - _prompt: &str, - _model_path: &std::path::Path, - _max_tokens: u32, - _meter: &dyn JouleCounter, - tx: mpsc::Sender>, -) { - let _ = tx.blocking_send(Err(LocalError::Backend { - backend: "llamacpp", - message: "feature `llamacpp` not enabled".into(), - })); -} - #[cfg(test)] mod tests { use super::*; + use crate::span::scripted::Scripted; use eoc_meter::StubCounter; use std::sync::Arc; @@ -303,21 +321,107 @@ mod tests { assert!(matches!(r, Err(LocalError::ModelNotFound(_)))); } - #[tokio::test] - async fn stub_counter_reports_zero_cost() { - // Construct a temp file so the existence check passes, then - // verify that with a StubCounter the reported joule cost is 0. + /// A backend on a dummy model file, metered by `meter`. The placeholder + /// generator streams four tokens for "hello world" (its header, the two + /// words, a filler), so the per-token estimate at 0.05 J/token is + /// 200,000 µJ. + fn backend(meter: Arc) -> (tempfile::TempDir, LlamaCppBackend) { let dir = tempfile::tempdir().unwrap(); let p = dir.path().join("dummy.gguf"); std::fs::write(&p, b"").unwrap(); - let cfg = LlamaCppConfig { - model_path: p, - ..LlamaCppConfig::default() - }; - let backend = LlamaCppBackend::new(cfg, Arc::new(StubCounter)).unwrap(); - let q = Query::new("hello world"); - let r = backend.infer(&q).await; - assert_eq!(r.joule_cost.microjoules, 0); + let cfg = LlamaCppConfig { model_path: p, ..LlamaCppConfig::default() }; + let b = LlamaCppBackend::new(cfg, meter).unwrap().with_fallback_joules_per_token(0.05); + (dir, b) + } + + const FOUR_TOKENS_UJ: u64 = 200_000; + + /// Every chunk of a stream metered by `meter`, in order. + async fn chunks(meter: Arc) -> Vec { + let (_dir, b) = backend(meter); + let mut rx = b.stream(&Query::new("hello world")).await; + let mut out = Vec::new(); + while let Some(item) = rx.recv().await { + out.push(item.expect("the placeholder never errors")); + } + out + } + + async fn cost(meter: Arc) -> JouleCost { + let (_dir, b) = backend(meter); + let r = b.infer(&Query::new("hello world")).await; assert_eq!(r.stage, Stage::Neural); + r.joule_cost + } + + /// The stub measured nothing, and the tokens were generated: the cost is + /// the per-token estimate, never 0 µJ. + #[tokio::test] + async fn the_stub_costs_the_per_token_estimate_never_zero() { + assert_eq!(cost(Arc::new(StubCounter)).await, JouleCost::estimated(FOUR_TOKENS_UJ)); + let streamed = chunks(Arc::new(StubCounter)).await; + assert_eq!(streamed.iter().filter(|c| !c.final_chunk).count(), 4); + assert!(streamed.iter().all(|c| c.cumulative_microjoules.is_none()), "{streamed:?}"); + } + + /// A failed first read is no baseline. The next reading is the counter's + /// since-boot total (9 J here), not the generation's energy: no chunk + /// carries it and the cost is not it. + #[tokio::test] + async fn a_failed_first_read_never_reports_the_since_boot_total() { + let streamed = chunks(Arc::new(Scripted::new(&[None, Some(9_000_000)]))).await; + assert!(streamed.iter().all(|c| c.cumulative_microjoules.is_none()), "{streamed:?}"); + let c = cost(Arc::new(Scripted::new(&[None, Some(9_000_000)]))).await; + assert_eq!(c, JouleCost::estimated(FOUR_TOKENS_UJ)); + } + + /// A counter that advanced over the whole generation measured it: the + /// final reading, measured. Each chunk carries the reading so far. + #[tokio::test] + async fn a_counter_that_advanced_measures_the_generation() { + let script = [Some(1_000), Some(1_100), Some(1_200), Some(1_300), Some(1_400), Some(1_450)]; + let meter = Arc::new(Scripted::new(&script)); + let streamed = chunks(meter.clone()).await; + assert_eq!(meter.reads(), 6, "one read to open, one per token, one at the end"); + let readings: Vec> = streamed.iter().map(|c| c.cumulative_microjoules).collect(); + assert_eq!(readings, vec![Some(100), Some(200), Some(300), Some(400), Some(450)]); + assert_eq!(cost(Arc::new(Scripted::new(&script))).await, JouleCost::measured(450)); + } + + /// A counter that read but did not move measured nothing. + #[tokio::test] + async fn a_counter_that_held_is_an_estimate() { + assert_eq!(cost(Arc::new(Scripted::new(&[Some(7_000)]))).await, JouleCost::estimated(FOUR_TOKENS_UJ)); + } + + /// The last read failed: the readings the earlier chunks carried cover + /// part of the generation, so none of them is its cost. + #[tokio::test] + async fn a_failed_last_read_is_an_estimate_not_a_partial_reading() { + let script = [Some(1_000), Some(1_100), Some(1_200), Some(1_300), Some(1_400), None]; + let streamed = chunks(Arc::new(Scripted::new(&script))).await; + assert_eq!(streamed.last().map(|c| (c.final_chunk, c.cumulative_microjoules)), Some((true, None))); + assert_eq!(cost(Arc::new(Scripted::new(&script))).await, JouleCost::estimated(FOUR_TOKENS_UJ)); + } + + /// The tally counts tokens and takes only the final chunk's reading. + #[test] + fn the_tally_costs_only_the_final_reading() { + let chunk = |text: &str, reading, final_chunk| TokenChunk { + text: text.to_string(), + cumulative_microjoules: reading, + final_chunk, + }; + let mut t = StreamTally::default(); + t.observe(&chunk("a", Some(100), false)); + t.observe(&chunk("b", Some(250), false)); + assert_eq!(t.cost(0.05), JouleCost::estimated(100_000), "no final chunk yet"); + t.observe(&chunk("", None, true)); + assert_eq!(t, StreamTally { tokens: 2, final_reading: None }); + assert_eq!(t.cost(0.05), JouleCost::estimated(100_000)); + let mut t = StreamTally::default(); + t.observe(&chunk("a", Some(100), false)); + t.observe(&chunk("", Some(300), true)); + assert_eq!(t.cost(0.05), JouleCost::measured(300)); } } diff --git a/eoc/eoc-rs/crates/eoc-local/src/mlc.rs b/eoc/eoc-rs/crates/eoc-local/src/mlc.rs index b45cd5884..1690dd37a 100644 --- a/eoc/eoc-rs/crates/eoc-local/src/mlc.rs +++ b/eoc/eoc-rs/crates/eoc-local/src/mlc.rs @@ -9,19 +9,23 @@ //! provides the *runtime loader* only. The expected on-disk layout is //! the standard `mlc-chat-config.json` + compiled library + weights. //! -//! Joule attribution piggy-backs on the `mlc_chat` runtime's own -//! reported timing where available; otherwise we sandwich the inference -//! call with [`JouleCounter`] reads exactly like the other backends. +//! Joule attribution sandwiches the inference call with [`JouleCounter`] +//! reads, by the rule every backend here shares ([`crate::span`]): the +//! delta is measured only when both reads succeed and the counter +//! advanced. Otherwise nothing was measured, and the cost is a per-token +//! estimate ([`MlcBackend::fallback_joules_per_token`]), tagged estimated, +//! as `mlx`'s is — never 0 µJ for inference that ran. use std::path::PathBuf; use std::sync::Arc; use async_trait::async_trait; -use eoc_core::{JouleCost, JouleSource, Query, Response, Stage}; +use eoc_core::{Query, Response, Stage}; use eoc_meter::JouleCounter; use eoc_neural::NeuralBackend; use crate::error::{LocalError, LocalResult}; +use crate::span::{self, MeterSpan}; /// MLC-LLM backend. pub struct MlcBackend { @@ -37,6 +41,9 @@ pub struct MlcBackend { pub max_tokens: u32, /// Joule counter for attribution. pub meter: Arc, + /// Estimated joules per output token, used when the counter did not + /// measure the inference ([`crate::span::per_token_estimate`]). + pub fallback_joules_per_token: f64, } impl MlcBackend { @@ -52,6 +59,7 @@ impl MlcBackend { device: default_device().to_string(), max_tokens: 512, meter, + fallback_joules_per_token: span::DEFAULT_FALLBACK_JOULES_PER_TOKEN, }) } @@ -73,6 +81,12 @@ impl MlcBackend { self.max_tokens = n; self } + + /// Override the per-token estimate used when nothing was measured. + pub fn with_fallback_joules_per_token(mut self, j: f64) -> Self { + self.fallback_joules_per_token = j; + self + } } fn default_device() -> &'static str { @@ -99,34 +113,29 @@ impl NeuralBackend for MlcBackend { // let response = mlc.generate(&q.prompt, GenerationOptions { max_tokens, ... })?; // // Reference build emits a placeholder and uses the meter - // sandwich for attribution. + // sandwich for attribution. Its token count, for the estimate, is + // the placeholder's own (as `mlx`'s is); a wired runtime passes the + // number of tokens it generated. - let start = self.meter.read_microjoules().ok(); + let span = MeterSpan::open(self.meter.as_ref()); + let generated_tokens = u64::from(self.max_tokens.min(64)); let payload = format!( "[mlc device={} model_dir={}]", self.device, self.model_dir.display() ); - let end = self.meter.read_microjoules().ok(); - let (microjoules, source) = match (start, end) { - (Some(s), Some(e)) if e >= s && (e - s) > 0 => (e - s, JouleSource::Measured), - _ => (0, JouleSource::Estimated), - }; - Response::new( - q.id, - payload, - Stage::Neural, - JouleCost { - microjoules, - source, - }, - ) + let cost = span::span_cost(span.reading(self.meter.as_ref()), || { + span::per_token_estimate(generated_tokens, self.fallback_joules_per_token) + }); + Response::new(q.id, payload, Stage::Neural, cost) } } #[cfg(test)] mod tests { use super::*; + use crate::span::scripted::Scripted; + use eoc_core::JouleCost; use eoc_meter::StubCounter; #[tokio::test] @@ -142,13 +151,38 @@ mod tests { assert!(matches!(r, Err(LocalError::ModelNotFound(_)))); } + /// Inference that ran and that nothing measured costs the per-token + /// estimate, tagged estimated: 8 tokens × 0.1 J = 800,000 µJ, never 0. #[tokio::test] - async fn stub_counter_reports_zero() { + async fn the_stub_costs_the_per_token_estimate_never_zero() { let dir = tempfile::tempdir().unwrap(); - let b = MlcBackend::new(dir.path(), Arc::new(StubCounter)).unwrap(); - let q = Query::new("hello"); - let r = b.infer(&q).await; - assert_eq!(r.joule_cost.microjoules, 0); + let b = MlcBackend::new(dir.path(), Arc::new(StubCounter)) + .unwrap() + .with_max_tokens(8) + .with_fallback_joules_per_token(0.1); + let r = b.infer(&Query::new("hello")).await; + assert_eq!(r.joule_cost, JouleCost::estimated(800_000)); assert_eq!(r.stage, Stage::Neural); + // The default coefficient, over the placeholder's 64-token cap. + let b = MlcBackend::new(dir.path(), Arc::new(StubCounter)).unwrap(); + assert_eq!(b.infer(&Query::new("hello")).await.joule_cost, JouleCost::estimated(64 * 50_000)); + } + + /// Each way a span can fail to measure gives the estimate; only a counter + /// that read twice and advanced gives a measured figure, its delta. + #[tokio::test] + async fn only_a_counter_that_advanced_is_measured() { + let dir = tempfile::tempdir().unwrap(); + let cost = |script: &'static [Option]| { + let b = MlcBackend::new(dir.path(), Arc::new(Scripted::new(script))) + .unwrap() + .with_max_tokens(8) + .with_fallback_joules_per_token(0.1); + async move { b.infer(&Query::new("hello")).await.joule_cost } + }; + assert_eq!(cost(&[Some(1_000), Some(1_450)]).await, JouleCost::measured(450)); + assert_eq!(cost(&[Some(7_000)]).await, JouleCost::estimated(800_000), "held"); + assert_eq!(cost(&[None, Some(9_000_000)]).await, JouleCost::estimated(800_000), "failed first read"); + assert_eq!(cost(&[Some(1_000), None]).await, JouleCost::estimated(800_000), "failed last read"); } } diff --git a/eoc/eoc-rs/crates/eoc-local/src/mlx.rs b/eoc/eoc-rs/crates/eoc-local/src/mlx.rs index 694ad5b70..a0f533020 100644 --- a/eoc/eoc-rs/crates/eoc-local/src/mlx.rs +++ b/eoc/eoc-rs/crates/eoc-local/src/mlx.rs @@ -7,11 +7,16 @@ //! //! Joule attribution reads whatever [`JouleCounter`] the backend is given //! (on a Mac, [`eoc_meter::detect`] picks IOReport only when its CPU rail is -//! live; its `powermetrics` stand-in never reads). A delta from a counter -//! that advanced is reported as measured; when the reads fail or the counter -//! does not move, the backend tags joule cost as +//! live; its `powermetrics` stand-in never reads), by the rule every backend +//! here shares ([`crate::span`]). A delta from a counter that read twice and +//! advanced is reported as measured; when a read fails or the counter does +//! not move, the backend tags joule cost as //! [`JouleSource::Estimated`](eoc_core::JouleSource::Estimated) and -//! falls back to a coefficient-based estimate. +//! falls back to a coefficient-based estimate, never 0 µJ. +//! +//! The module binds no `mlx-rs` symbol yet (its generator is a placeholder), +//! so this crate's tests compile it on macOS without the feature, and its +//! metering is tested in every macOS build. //! //! This module is hard-gated to `target_os = "macos"`. On any other //! platform compilation of `--features mlx` will fail with a clear @@ -25,11 +30,12 @@ use std::path::PathBuf; use std::sync::Arc; use async_trait::async_trait; -use eoc_core::{JouleCost, JouleSource, Query, Response, Stage}; +use eoc_core::{Query, Response, Stage}; use eoc_meter::JouleCounter; use eoc_neural::NeuralBackend; use crate::error::{LocalError, LocalResult}; +use crate::span::{self, MeterSpan}; /// Apple MLX backend. pub struct MlxBackend { @@ -64,7 +70,7 @@ impl MlxBackend { tokenizer_path, max_tokens: 512, meter, - fallback_joules_per_token: 0.05, + fallback_joules_per_token: span::DEFAULT_FALLBACK_JOULES_PER_TOKEN, }) } @@ -99,33 +105,22 @@ impl NeuralBackend for MlxBackend { // the trait + plumbing are exercised end-to-end. Operators // swap in the real mlx-rs symbols against their pinned rev. - let start = self.meter.read_microjoules().ok(); - let estimated_tokens = self.max_tokens.min(64) as u32; + let span = MeterSpan::open(self.meter.as_ref()); + let estimated_tokens = u64::from(self.max_tokens.min(64)); let payload = format!("[mlx model={} prompt={:?}]", self.model_path.display(), q.prompt); - let end = self.meter.read_microjoules().ok(); - let (microjoules, source) = match (start, end) { - (Some(s), Some(e)) if e >= s && (e - s) > 0 => (e - s, JouleSource::Measured), - _ => { - let j = (estimated_tokens as f64) * self.fallback_joules_per_token; - ((j * 1_000_000.0).max(0.0) as u64, JouleSource::Estimated) - } - }; - Response::new( - q.id, - payload, - Stage::Neural, - JouleCost { - microjoules, - source, - }, - ) + let cost = span::span_cost(span.reading(self.meter.as_ref()), || { + span::per_token_estimate(estimated_tokens, self.fallback_joules_per_token) + }); + Response::new(q.id, payload, Stage::Neural, cost) } } #[cfg(test)] mod tests { use super::*; + use crate::span::scripted::Scripted; + use eoc_core::{JouleCost, JouleSource}; use eoc_meter::StubCounter; #[tokio::test] @@ -147,6 +142,28 @@ mod tests { assert_eq!(r.joule_cost.microjoules, 800_000); } + /// Only a counter that read twice and advanced is a measurement; a + /// failed first read is no baseline, so a since-boot total is not one. + #[tokio::test] + async fn only_a_counter_that_advanced_is_measured() { + let dir = tempfile::tempdir().unwrap(); + let model = dir.path().join("model"); + std::fs::create_dir(&model).unwrap(); + let tok = model.join("tokenizer.json"); + for (script, want) in [ + (&[Some(1_000), Some(1_450)][..], JouleCost::measured(450)), + (&[Some(7_000)][..], JouleCost::estimated(800_000)), + (&[None, Some(9_000_000)][..], JouleCost::estimated(800_000)), + (&[Some(1_000), None][..], JouleCost::estimated(800_000)), + ] { + let b = MlxBackend::new(&model, &tok, Arc::new(Scripted::new(script))) + .unwrap() + .with_max_tokens(8) + .with_fallback_joules_per_token(0.1); + assert_eq!(b.infer(&Query::new("ping")).await.joule_cost, want, "{script:?}"); + } + } + #[tokio::test] async fn missing_model_returns_not_found() { let r = MlxBackend::new("/no/such/path", "/no/tok", Arc::new(StubCounter)); diff --git a/eoc/eoc-rs/crates/eoc-local/src/onnx.rs b/eoc/eoc-rs/crates/eoc-local/src/onnx.rs index d39b778f1..370b10887 100644 --- a/eoc/eoc-rs/crates/eoc-local/src/onnx.rs +++ b/eoc/eoc-rs/crates/eoc-local/src/onnx.rs @@ -11,7 +11,11 @@ //! is composed locally via the [`crate::sampling`] samplers. //! //! Joule attribution: pre/post counter reads sandwich the generation -//! call. +//! call, by the rule every backend here shares ([`crate::span`]): the +//! delta is measured only when both reads succeed and the counter +//! advanced. Otherwise nothing was measured, and the cost is a per-token +//! estimate ([`OnnxBackend::fallback_joules_per_token`]), tagged estimated, +//! as `mlx`'s is — never 0 µJ for inference that ran. //! //! [ONNX Runtime]: https://onnxruntime.ai //! [`ort`]: https://crates.io/crates/ort @@ -20,12 +24,13 @@ use std::path::PathBuf; use std::sync::Arc; use async_trait::async_trait; -use eoc_core::{JouleCost, JouleSource, Query, Response, Stage}; +use eoc_core::{Query, Response, Stage}; use eoc_meter::JouleCounter; use eoc_neural::NeuralBackend; use crate::error::{LocalError, LocalResult}; use crate::sampling::{GreedySampler, Sampler}; +use crate::span::{self, MeterSpan}; use crate::tokenizer::LocalTokenizer; /// Generation-loop configuration. @@ -100,6 +105,9 @@ pub struct OnnxBackend { pub sampler: Box, /// Joule counter. pub meter: Arc, + /// Estimated joules per output token, used when the counter did not + /// measure the inference ([`crate::span::per_token_estimate`]). + pub fallback_joules_per_token: f64, } impl OnnxBackend { @@ -120,6 +128,7 @@ impl OnnxBackend { generation_config: GenerationConfig::default(), sampler: Box::new(GreedySampler), meter, + fallback_joules_per_token: span::DEFAULT_FALLBACK_JOULES_PER_TOKEN, }) } @@ -140,6 +149,12 @@ impl OnnxBackend { self.sampler = sampler; self } + + /// Override the per-token estimate used when nothing was measured. + pub fn with_fallback_joules_per_token(mut self, j: f64) -> Self { + self.fallback_joules_per_token = j; + self + } } #[async_trait] @@ -164,51 +179,69 @@ impl NeuralBackend for OnnxBackend { // } // let text = self.tokenizer.decode(&tokens[input_ids.len()..], true)?; // - // Reference build sandwich-meters around a placeholder. + // Reference build sandwich-meters around a placeholder. Its token + // count, for the estimate, is the placeholder's own (as `mlx`'s is); + // the wired loop above passes the number of tokens it generated. - let start = self.meter.read_microjoules().ok(); + let span = MeterSpan::open(self.meter.as_ref()); + let generated_tokens = u64::from(self.generation_config.max_tokens.min(64)); let payload = format!( "[onnx ep={} model={}]", self.execution_provider.as_str(), self.model_path.display() ); - let end = self.meter.read_microjoules().ok(); - let (microjoules, source) = match (start, end) { - (Some(s), Some(e)) if e >= s && (e - s) > 0 => (e - s, JouleSource::Measured), - _ => (0, JouleSource::Estimated), - }; - Response::new( - q.id, - payload, - Stage::Neural, - JouleCost { - microjoules, - source, - }, - ) + let cost = span::span_cost(span.reading(self.meter.as_ref()), || { + span::per_token_estimate(generated_tokens, self.fallback_joules_per_token) + }); + Response::new(q.id, payload, Stage::Neural, cost) } } #[cfg(test)] mod tests { use super::*; + use crate::span::scripted::Scripted; + use eoc_core::JouleCost; use eoc_meter::StubCounter; - #[tokio::test] - async fn stub_counter_reports_zero_cost() { - let dir = tempfile::tempdir().unwrap(); - let model = dir.path().join("model.onnx"); + fn backend(dir: &std::path::Path, meter: Arc) -> OnnxBackend { + let model = dir.join("model.onnx"); std::fs::write(&model, b"").unwrap(); - let tok_path = dir.path().join("tokenizer.json"); + let tok_path = dir.join("tokenizer.json"); std::fs::write(&tok_path, b"{}").unwrap(); let tok = LocalTokenizer::from_path(&tok_path); - let b = OnnxBackend::new(&model, tok, Arc::new(StubCounter)).unwrap(); - let q = Query::new("hi"); - let r = b.infer(&q).await; - assert_eq!(r.joule_cost.microjoules, 0); + OnnxBackend::new(&model, tok, meter) + .unwrap() + .with_generation_config(GenerationConfig { max_tokens: 8, ..GenerationConfig::default() }) + .with_fallback_joules_per_token(0.1) + } + + /// Inference that ran and that nothing measured costs the per-token + /// estimate, tagged estimated: 8 tokens × 0.1 J = 800,000 µJ, never 0. + #[tokio::test] + async fn the_stub_costs_the_per_token_estimate_never_zero() { + let dir = tempfile::tempdir().unwrap(); + let r = backend(dir.path(), Arc::new(StubCounter)).infer(&Query::new("hi")).await; + assert_eq!(r.joule_cost, JouleCost::estimated(800_000)); assert_eq!(r.stage, Stage::Neural); } + /// Each way a span can fail to measure gives the estimate; only a counter + /// that read twice and advanced gives a measured figure, its delta. + #[tokio::test] + async fn only_a_counter_that_advanced_is_measured() { + let dir = tempfile::tempdir().unwrap(); + for (script, want) in [ + (&[Some(1_000), Some(1_450)][..], JouleCost::measured(450)), + (&[Some(7_000)][..], JouleCost::estimated(800_000)), + (&[None, Some(9_000_000)][..], JouleCost::estimated(800_000)), + (&[Some(1_000), None][..], JouleCost::estimated(800_000)), + ] { + let b = backend(dir.path(), Arc::new(Scripted::new(script))); + assert_eq!(b.infer(&Query::new("hi")).await.joule_cost, want, "{script:?}"); + } + } + #[tokio::test] async fn missing_model_returns_not_found() { let dir = tempfile::tempdir().unwrap(); diff --git a/eoc/eoc-rs/crates/eoc-local/src/span.rs b/eoc/eoc-rs/crates/eoc-local/src/span.rs new file mode 100644 index 000000000..d6e5cf6c4 --- /dev/null +++ b/eoc/eoc-rs/crates/eoc-local/src/span.rs @@ -0,0 +1,188 @@ +//! What a counter read over one span of inference, and what a backend +//! reports when it read nothing. +//! +//! Every backend in this crate meters the same way, so the rule lives here +//! once and is tested without any native runtime: +//! +//! - A span is **measured** only when the counter read at its start, read +//! again at its end, and advanced between the two. The reading is then the +//! delta, tagged [`JouleSource::Measured`](eoc_core::JouleSource::Measured). +//! - Anything else measured nothing: a failed read at either end, a counter +//! that read but did not move (the [`StubCounter`](eoc_meter::StubCounter), +//! a frozen rail, a placeholder). A failed first read is never taken as 0, +//! which would turn the next successful read into the counter's whole +//! since-boot total. +//! - A span that measured nothing still ran inference, so its cost is never +//! 0 µJ. The backend reports a per-token estimate instead, tagged +//! [`JouleSource::Estimated`](eoc_core::JouleSource::Estimated). + +use eoc_core::JouleCost; +use eoc_meter::JouleCounter; + +/// The joules per output token a backend estimates with when no live counter +/// measured its inference, unless it is given another coefficient. The same +/// figure `mlx` has always used; a reference value, to be replaced by one +/// calibrated on the deployment host. +pub const DEFAULT_FALLBACK_JOULES_PER_TOKEN: f64 = 0.05; + +/// One span of inference, opened by reading the counter. +#[derive(Debug, Clone, Copy, PartialEq, Eq)] +pub struct MeterSpan { + /// The counter's reading when the span opened; `None` when that read + /// failed, and then nothing over this span can be measured. + start: Option, +} + +impl MeterSpan { + /// Open a span: read `meter` once. + pub fn open(meter: &dyn JouleCounter) -> Self { + Self { start: meter.read_microjoules().ok() } + } + + /// What `meter` has read since the span opened: `Some(delta)` only when + /// the opening read succeeded, this read succeeds, and the counter + /// advanced between them. `None` otherwise — never 0, and never the + /// counter's since-boot total. + pub fn reading(&self, meter: &dyn JouleCounter) -> Option { + let start = self.start?; + let now = meter.read_microjoules().ok()?; + (now > start).then(|| now - start) + } +} + +/// The cost of a span of inference that ran: the counter's reading, +/// measured, when there is one; otherwise `estimate()`, tagged estimated. +/// The estimate is asked for only when nothing was read. +pub fn span_cost(reading: Option, estimate: impl FnOnce() -> u64) -> JouleCost { + match reading { + Some(uj) => JouleCost::measured(uj), + None => JouleCost::estimated(estimate()), + } +} + +/// A per-token estimate of inference that ran, in micro-joules: `tokens` × +/// `joules_per_token`, to the nearest micro-joule. Never 0, because the +/// inference ran: at least one token is counted (a pass that emitted nothing +/// still processed its prompt), a coefficient that is not a finite positive +/// number is not one to estimate with, so [`DEFAULT_FALLBACK_JOULES_PER_TOKEN`] +/// stands in for it, and a product under half a micro-joule is 1 µJ. +pub fn per_token_estimate(tokens: u64, joules_per_token: f64) -> u64 { + let per_token = if joules_per_token.is_finite() && joules_per_token > 0.0 { + joules_per_token + } else { + DEFAULT_FALLBACK_JOULES_PER_TOKEN + }; + let uj = (tokens.max(1) as f64 * per_token * 1_000_000.0).round(); + // `as` saturates at u64::MAX. + (uj as u64).max(1) +} + +/// Test support: a counter that reads a script. +#[cfg(test)] +pub(crate) mod scripted { + use std::sync::Mutex; + + use eoc_meter::JouleCounter; + + /// A counter that reads the scripted values in turn, then repeats the + /// last one; `None` is a failed read. Counts its reads. + #[derive(Debug)] + pub(crate) struct Scripted { + script: Vec>, + reads: Mutex, + } + + impl Scripted { + pub(crate) fn new(script: &[Option]) -> Self { + assert!(!script.is_empty(), "a script needs at least one read"); + Self { script: script.to_vec(), reads: Mutex::new(0) } + } + + /// How many times the counter has been read. + pub(crate) fn reads(&self) -> usize { + *self.reads.lock().unwrap() + } + } + + impl JouleCounter for Scripted { + fn read_microjoules(&self) -> eoc_core::Result { + let mut reads = self.reads.lock().unwrap(); + let value = self.script[(*reads).min(self.script.len() - 1)]; + *reads += 1; + value.ok_or(eoc_core::Error::NoCounter) + } + fn name(&self) -> &'static str { + "scripted" + } + } +} + +#[cfg(test)] +mod tests { + use super::scripted::Scripted; + use super::*; + use eoc_core::JouleSource; + use eoc_meter::StubCounter; + + fn read_span(script: &[Option]) -> Option { + let c = Scripted::new(script); + let span = MeterSpan::open(&c); + let reading = span.reading(&c); + assert_eq!(c.reads(), if script[0].is_some() { 2 } else { 1 }); + reading + } + + /// Both reads succeeded and the counter advanced: the delta. + #[test] + fn a_counter_that_advanced_reads_the_delta() { + assert_eq!(read_span(&[Some(1_000), Some(1_450)]), Some(450)); + } + + /// A failed first read is no baseline: a later reading of the counter's + /// since-boot total is not the span's energy. + #[test] + fn a_failed_first_read_measures_nothing() { + assert_eq!(read_span(&[None, Some(9_000_000)]), None); + } + + /// A failed last read leaves no reading, whatever the first read was. + #[test] + fn a_failed_last_read_measures_nothing() { + assert_eq!(read_span(&[Some(1_000), None]), None); + } + + /// A counter that read but did not move measured nothing; one that went + /// down (reset or wrapped) did not measure the span either. + #[test] + fn a_counter_that_did_not_advance_measures_nothing() { + assert_eq!(read_span(&[Some(7_000), Some(7_000)]), None); + assert_eq!(read_span(&[Some(7_000), Some(6_000)]), None); + let stub = StubCounter; + assert_eq!(MeterSpan::open(&stub).reading(&stub), None); + } + + /// A reading is measured and the estimate is not asked for; no reading is + /// the estimate, tagged estimated. + #[test] + fn a_span_costs_its_reading_or_the_estimate() { + assert_eq!(span_cost(Some(450), || panic!("estimated a measured span")), JouleCost::measured(450)); + let cost = span_cost(None, || 800_000); + assert_eq!(cost, JouleCost::estimated(800_000)); + assert_eq!(cost.source, JouleSource::Estimated); + } + + /// Tokens × coefficient, never 0: a run that emitted no token counts one, + /// a coefficient that is not finite and positive gives way to the + /// default, and a tiny product is 1 µJ. + #[test] + fn the_per_token_estimate_is_never_zero() { + assert_eq!(per_token_estimate(8, 0.1), 800_000); + assert_eq!(per_token_estimate(10, 0.05), 500_000); + assert_eq!(per_token_estimate(0, 0.1), 100_000); + for bad in [0.0, -1.0, f64::NAN, f64::INFINITY] { + assert_eq!(per_token_estimate(4, bad), 200_000, "coefficient {bad}"); + } + assert_eq!(per_token_estimate(1, 1e-12), 1); + assert_eq!(per_token_estimate(u64::MAX, 1e9), u64::MAX); + } +} diff --git a/eoc/eoc-rs/crates/eoc-local/tests/joule_attribution.rs b/eoc/eoc-rs/crates/eoc-local/tests/joule_attribution.rs index b08f6f5e1..e095ecf7e 100644 --- a/eoc/eoc-rs/crates/eoc-local/tests/joule_attribution.rs +++ b/eoc/eoc-rs/crates/eoc-local/tests/joule_attribution.rs @@ -1,10 +1,9 @@ //! Joule-attribution integration test. //! //! Verifies the contract: when the meter is a [`StubCounter`] nothing -//! was measured, so the cost is an estimate (0 µJ for `mlc`, the -//! per-token coefficient for `mlx`); when the meter is a non-stub -//! counter whose readings monotonically advance, the reported joule -//! cost is non-zero and measured. +//! was measured, so the cost is the backend's per-token estimate, never +//! 0 µJ; when the meter is a non-stub counter whose readings monotonically +//! advance, the reported joule cost is non-zero and measured. //! //! These tests exercise the *plumbing* — the backends themselves use //! their respective placeholder implementations in the reference @@ -49,14 +48,20 @@ impl JouleCounter for TickingCounter { #[cfg(feature = "mlc")] #[tokio::test] -async fn mlc_stub_counter_yields_zero_cost() { +async fn mlc_stub_counter_yields_an_estimate_never_zero() { use eoc_local::MlcBackend; use eoc_neural::NeuralBackend; let dir = tempfile::tempdir().unwrap(); - let b = MlcBackend::new(dir.path(), Arc::new(StubCounter)).unwrap(); + let b = MlcBackend::new(dir.path(), Arc::new(StubCounter)) + .unwrap() + .with_max_tokens(10) + .with_fallback_joules_per_token(0.05); let q = Query::new("hello"); let r = b.infer(&q).await; - assert_eq!(r.joule_cost.microjoules, 0); + // Nothing measured the inference, which ran: 10 * 0.05 J = 500,000 µJ, + // an estimate. + assert_eq!(r.joule_cost.microjoules, 500_000); + assert_eq!(r.joule_cost.source, JouleSource::Estimated); } #[cfg(feature = "mlc")] -- 2.54.0 (Apple Git-157) From 6c37e299806ae4c4f5f8162b0611eb80de137243 Mon Sep 17 00:00:00 2001 From: Transaction Science Date: Wed, 30 Sep 2026 21:07:16 -0400 Subject: [PATCH 14/22] eoc-carbon: an account and an SCI score are measured only where every part was, and a run with no correct answer has no score MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit 0db9b8be listed three things here as not done. `CarbonAccount` dropped provenance, and `account_request` is a public way in. A report with zero correct answers carries `joules_per_correct == u64::MAX` as a sentinel, which `sci_from_eval_report` multiplied by `correct.max(1)` into a u64::MAX µJ energy term (18.4 TJ). And the test that pins the bridge's provenance sat behind the `eval-bridge` feature, which nothing in the workspace enables, so `cargo test --workspace` never compiled it, nor any mutation it catches. What holds now: - `CarbonAccount` gains `it_energy_source`, the input `JouleCost`'s provenance unchanged, and `facility_energy_source`. The facility joules are `it_joules` × PUE, so they are measured only when every part of them was: the IT joules were measured, and the PUE added no overhead. A PUE above 1 is a published or assumed facility ratio applied to the reading, energy no counter on the host read; a figure containing any of it is an estimate. A PUE of 1 adds nothing, so measured IT joules stay measured, as an estimated zero adds nothing to a live span in eoc-cascade. Grid intensity has its own provenance (the reading's provider and kind) and does not enter the tag. Both fields read back as `Estimated` when absent. Nothing outside this crate builds a `CarbonAccount`. - `SciScore::energy_source` is now the account's facility provenance, because the score's energy term is facility energy. Before, it was the input's provenance, so measured IT joules at the default PUE of 1.58 gave a "measured" term of which 37% no counter read. This changes what 0db9b8be set: an SCI score at PUE > 1 is now estimated even from measured joules. The SCI numbers do not change. - The bridge's pure part is compiled in every build. `EvalEnergy` (micro-joules per correct answer, correct answers, provenance) and `sci_per_correct_answer` hold all the scoring, including the `JouleCost` construction 0db9b8be's mutation targeted. `sci_from_eval_report`, still behind `eval-bridge`, only copies the report's three fields across. - A run with no correct answer has no score per correct answer: `sci_per_correct_answer` and `sci_from_eval_report` return `None` for `correct == 0` and for the `u64::MAX` sentinel, whatever `correct` says beside it. The sentinel is never read as energy. `sci_from_eval_report` now returns `Option`; its only callers are this crate's tests. Tests in the default build: an account's (IT, facility) tags are (Measured, Measured) at PUE 1, (Measured, Estimated) at 1.2, (Estimated, Estimated) from an estimate at either, and (Measured, Measured) at a PUE of 0.5, which is clamped to 1; the figures are the same whichever tag. An account round-trips and an old one reads back estimated. A score is measured at PUE 1, estimated at 1.2 from measured joules, with the same SCI. An eval run's score keeps the run's provenance (2 × 1 J × 1.2 = 2.4 J over R = 2), and a run with 0 correct, or the sentinel with 3 correct, has none. `account_per_request` now asserts its tags. Under `eval-bridge`, a report's score equals its run's score for both provenances and both PUEs, and a 0-correct report has none. Falsified, each mutant restored byte-for-byte (sha256 checked); every one but the last runs in the default build: - 0db9b8be's mutant, `JouleCost::measured(total)` in the bridge, now fails `an_eval_runs_score_keeps_the_runs_provenance` in a plain `cargo test`; - an account dropping the IT provenance fails 1 test; - the facility tag ignoring the PUE fails 3, ignoring the IT provenance 3; - the score taking the input's provenance (0db9b8be's rule) fails 2; - removing the no-correct-answer guard fails 1, and so does checking `correct == 0` without the sentinel; - the `eval-bridge` wrapper tagging every report measured fails `an_eval_reports_score_is_its_runs_score` under `--features eval-bridge`. That wrapper's three-field copy is the only part a default build cannot see. `cargo test -p eoc-carbon` passes (42 + 8), and with `--features eval-bridge` (43 + 8). No clippy warning in this crate, before or after. Not done: `joules_per_correct` is an integer quotient, so the total reconstructed from it can fall short of the run's by less than one µJ per correct answer. Co-Authored-By: Claude Opus 5.5 --- eoc/eoc-rs/crates/eoc-carbon/src/account.rs | 78 ++++++- eoc/eoc-rs/crates/eoc-carbon/src/lib.rs | 2 +- .../crates/eoc-carbon/src/sci_bridge.rs | 203 +++++++++++++----- .../eoc-carbon/tests/account_per_request.rs | 3 + 4 files changed, 231 insertions(+), 55 deletions(-) diff --git a/eoc/eoc-rs/crates/eoc-carbon/src/account.rs b/eoc/eoc-rs/crates/eoc-carbon/src/account.rs index 730d0394a..1fca272ef 100644 --- a/eoc/eoc-rs/crates/eoc-carbon/src/account.rs +++ b/eoc/eoc-rs/crates/eoc-carbon/src/account.rs @@ -5,8 +5,16 @@ //! Inputs are typed: joules come from [`eoc_meter`] / [`eoc_core::JouleCost`], //! intensity comes from [`crate::intensity::CarbonIntensity`], overhead //! comes from [`crate::pue::Pue`]. +//! +//! An account keeps the provenance of its energy. Its IT-layer joules are +//! exactly as measured as the [`JouleCost`] they came from. Its facility +//! joules are measured only when every part of them was: the IT joules +//! were, and the PUE added no overhead. The overhead a PUE above 1 adds is a +//! published or assumed facility ratio applied to the reading, energy no +//! counter on the host read, so a facility figure that includes any of it +//! is an estimate. -use eoc_core::JouleCost; +use eoc_core::{JouleCost, JouleSource}; use serde::{Deserialize, Serialize}; use crate::intensity::CarbonIntensity; @@ -28,6 +36,36 @@ pub struct CarbonAccount { pub pue: f64, /// Final gCO2e number. pub g_co2e: f64, + /// Where `it_joules` came from: the input [`JouleCost`]'s provenance, + /// unchanged. An account serialized without this field reads back as + /// `Estimated`: nothing in it says it was measured. + #[serde(default = "unrecorded_source")] + pub it_energy_source: JouleSource, + /// Where `facility_joules`, and so the energy behind `g_co2e`, came from: + /// `Measured` only when every part of it was — the IT joules were + /// measured and the PUE added no overhead (a PUE of exactly 1). Any + /// overhead is a facility ratio applied to the reading, not a reading, + /// and makes the figure an estimate. Grid intensity has its own + /// provenance (the reading's provider and kind) and does not enter this + /// tag. Reads back as `Estimated` when absent. + #[serde(default = "unrecorded_source")] + pub facility_energy_source: JouleSource, +} + +/// The provenance of an account that recorded none: an estimate, never a +/// measurement. +fn unrecorded_source() -> JouleSource { + JouleSource::Estimated +} + +/// The provenance of `it × pue`: measured only when the IT joules were and +/// the PUE added nothing to them. +fn facility_source(it: JouleSource, pue: &Pue) -> JouleSource { + if it == JouleSource::Measured && pue.value() <= 1.0 { + JouleSource::Measured + } else { + JouleSource::Estimated + } } /// Anything that turns an `it_joules + intensity + pue` tuple into a @@ -62,6 +100,8 @@ impl CarbonAccounting for StandardAccounting { g_co2e_per_kwh: intensity.g_co2e_per_kwh, pue: pue.value(), g_co2e, + it_energy_source: joule_cost.source, + facility_energy_source: facility_source(joule_cost.source, pue), } } } @@ -104,4 +144,40 @@ mod tests { let acc = account_request(&JouleCost::zero(), &intensity(800.0), &Pue::new(1.4)); assert_eq!(acc.g_co2e, 0.0); } + + /// The IT joules keep their input's provenance. The facility joules are + /// measured only when every part of them was: measured IT joules and no + /// PUE overhead. The figures do not depend on the tags. + #[test] + fn an_account_is_measured_only_where_every_part_was() { + let ci = intensity(400.0); + let tags = |cost: JouleCost, pue: f64| { + let acc = account_request(&cost, &ci, &Pue::new(pue)); + (acc.it_energy_source, acc.facility_energy_source) + }; + use JouleSource::{Estimated, Measured}; + assert_eq!(tags(JouleCost::measured(1_000_000), 1.0), (Measured, Measured)); + assert_eq!(tags(JouleCost::measured(1_000_000), 1.2), (Measured, Estimated), "PUE overhead is not a reading"); + assert_eq!(tags(JouleCost::estimated(1_000_000), 1.0), (Estimated, Estimated)); + assert_eq!(tags(JouleCost::estimated(1_000_000), 1.2), (Estimated, Estimated)); + // A PUE below 1 is clamped to 1: no overhead, so nothing unread. + assert_eq!(tags(JouleCost::measured(1_000_000), 0.5), (Measured, Measured)); + + let m = account_request(&JouleCost::measured(1_000_000), &ci, &Pue::new(1.2)); + let e = account_request(&JouleCost::estimated(1_000_000), &ci, &Pue::new(1.2)); + assert_eq!((m.it_joules, m.facility_joules, m.g_co2e), (e.it_joules, e.facility_joules, e.g_co2e)); + } + + /// The provenance survives a round trip; an account written before it + /// existed recorded none, so it reads back as an estimate. + #[test] + fn an_account_without_provenance_reads_back_estimated() { + let acc = account_request(&JouleCost::measured(1_000_000), &intensity(400.0), &Pue::new(1.0)); + let back: CarbonAccount = serde_json::from_str(&serde_json::to_string(&acc).unwrap()).unwrap(); + assert_eq!(back, acc); + assert_eq!(back.facility_energy_source, JouleSource::Measured); + let old = r#"{"it_joules":1.0,"facility_joules":1.0,"g_co2e_per_kwh":400.0,"pue":1.0,"g_co2e":0.1}"#; + let back: CarbonAccount = serde_json::from_str(old).unwrap(); + assert_eq!((back.it_energy_source, back.facility_energy_source), (JouleSource::Estimated, JouleSource::Estimated)); + } } diff --git a/eoc/eoc-rs/crates/eoc-carbon/src/lib.rs b/eoc/eoc-rs/crates/eoc-carbon/src/lib.rs index ab1a31279..7fc6b8912 100644 --- a/eoc/eoc-rs/crates/eoc-carbon/src/lib.rs +++ b/eoc/eoc-rs/crates/eoc-carbon/src/lib.rs @@ -43,7 +43,7 @@ pub use error::{CarbonError, Result}; pub use intensity::{CarbonIntensity, ProviderKind, Zone}; pub use pue::{Pue, PueTable}; pub use scheduler::{DemandShifter, RegionScheduler, ShiftDecision}; -pub use sci_bridge::{SciInputs, SciScore}; +pub use sci_bridge::{EvalEnergy, SciInputs, SciScore, sci_per_correct_answer}; /// gCO2e per kWh in a "typical" world-average grid (IEA 2024). Useful as /// a last-resort fallback when even the IEA per-country table cannot be diff --git a/eoc/eoc-rs/crates/eoc-carbon/src/sci_bridge.rs b/eoc/eoc-rs/crates/eoc-carbon/src/sci_bridge.rs index 0640c36c1..12c2d2b62 100644 --- a/eoc/eoc-rs/crates/eoc-carbon/src/sci_bridge.rs +++ b/eoc/eoc-rs/crates/eoc-carbon/src/sci_bridge.rs @@ -15,9 +15,11 @@ //! correct answer for an eval). //! //! `eoc-eval` already references SCI; this module is the explicit -//! computation surface, and when the `eval-bridge` feature is on it -//! exposes a constructor that turns an [`eoc_eval::EvalReport`] into an -//! [`SciScore`] with `R = number of correct cases`. +//! computation surface. [`sci_per_correct_answer`] scores an eval run's +//! energy with `R = number of correct cases`, and when the `eval-bridge` +//! feature is on, `sci_from_eval_report` feeds it an +//! [`eoc_eval::EvalReport`]. A run with no correct answer has no score per +//! correct answer: both return `None` for it. use serde::{Deserialize, Serialize}; @@ -58,9 +60,13 @@ pub struct SciScore { pub functional_units: f64, /// The SCI value (gCO2e per functional unit). pub sci: f64, - /// Where the energy term came from: the input's provenance, unchanged. - /// An SCI value is only as measured as the joules it was computed from. - /// A score serialized without this field reads back as `Estimated`. + /// Where the energy term came from. The term is facility energy (IT + /// joules × PUE), so this is the account's + /// [`facility_energy_source`](CarbonAccount::facility_energy_source): + /// `Measured` only when the input joules were measured and the PUE + /// added no overhead, which no counter read. An SCI value is never more + /// measured than the joules it was computed from. A score serialized + /// without this field reads back as `Estimated`. #[serde(default = "unrecorded_energy_source")] pub energy_source: JouleSource, } @@ -89,40 +95,86 @@ impl SciScore { embodied_g_co2e: inputs.embodied_g_co2e, functional_units: r, sci, - energy_source: inputs.joule_cost.source, + energy_source: account.facility_energy_source, } } } +/// An eval run's energy, as `eoc_eval::EvalReport` carries it: micro-joules +/// per correct answer, the number of correct answers, and the provenance of +/// the joules. Kept free of `eoc-eval` so the scoring below is compiled, and +/// tested, in every build; the `eval-bridge` feature only fills it from a +/// report. +#[derive(Debug, Clone, Copy, PartialEq, Eq)] +pub struct EvalEnergy { + /// Micro-joules per correct answer. `u64::MAX` is the report's sentinel + /// for "no correct answer", not a figure. + pub joules_per_correct: u64, + /// Correct answers in the run. + pub correct: usize, + /// Where the joules came from: `Measured` only when every case's were. + pub joule_source: JouleSource, +} + +/// The SCI score per correct answer of an eval run: total IT energy +/// `joules_per_correct × correct`, with the run's own provenance (never +/// upgraded; see [`SciScore::energy_source`]), and `R = correct`. +/// +/// `None` when the run has no correct answer — `correct == 0`, or the +/// `u64::MAX` sentinel a report carries for that — since there is no figure +/// per correct answer to score. The sentinel is never read as energy. +pub fn sci_per_correct_answer( + energy: EvalEnergy, + intensity: &CarbonIntensity, + pue: &Pue, + embodied_g_co2e: f64, +) -> Option { + if energy.correct == 0 || energy.joules_per_correct == u64::MAX { + return None; + } + let total_microjoules = (energy.joules_per_correct as u128) + .saturating_mul(energy.correct as u128) + .min(u64::MAX as u128) as u64; + let cost = JouleCost { + microjoules: total_microjoules, + source: energy.joule_source, + }; + Some(SciScore::compute(SciInputs { + joule_cost: &cost, + intensity, + pue, + embodied_g_co2e, + functional_units: energy.correct as f64, + })) +} + /// `eoc-eval` bridge — turns an `EvalReport` into an SCI score with -/// `R = correct`. Compiled only with the `eval-bridge` Cargo feature. +/// `R = correct`. Compiled only with the `eval-bridge` Cargo feature; it +/// only hands the report's energy to [`sci_per_correct_answer`], which is +/// compiled and tested in every build. /// /// `EvalReport` carries `joules_per_correct` in micro-joules and a -/// `correct` count, so we reconstruct the total IT-layer energy as +/// `correct` count, so the total IT-layer energy is reconstructed as /// `joules_per_correct * correct`. Its provenance is the report's own -/// `joule_source`: measured only when every case of the run was. +/// `joule_source`: measured only when every case of the run was. `None` +/// for a run with no correct answer. #[cfg(feature = "eval-bridge")] pub fn sci_from_eval_report( report: &eoc_eval::EvalReport, intensity: &CarbonIntensity, pue: &Pue, embodied_g_co2e: f64, -) -> SciScore { - let correct = report.correct.max(1); - let total_microjoules = (report.joules_per_correct as u128) - .saturating_mul(correct as u128) - .min(u64::MAX as u128) as u64; - let cost = JouleCost { - microjoules: total_microjoules, - source: report.joule_source, - }; - SciScore::compute(SciInputs { - joule_cost: &cost, +) -> Option { + sci_per_correct_answer( + EvalEnergy { + joules_per_correct: report.joules_per_correct, + correct: report.correct, + joule_source: report.joule_source, + }, intensity, pue, embodied_g_co2e, - functional_units: correct as f64, - }) + ) } #[cfg(test)] @@ -156,25 +208,29 @@ mod tests { CarbonIntensity::new("US", 400.0, IntensityKind::Average, ProviderKind::Mock, Utc::now()) } + fn score(cost: JouleCost, pue: f64) -> SciScore { + SciScore::compute(SciInputs { + joule_cost: &cost, + intensity: &intensity(), + pue: &Pue::new(pue), + embodied_g_co2e: 0.0, + functional_units: 1.0, + }) + } + /// The score says where its energy term came from, and the same figure - /// yields the same numbers either way. + /// yields the same numbers either way. The term is facility energy, so + /// it is measured only when the joules were and the PUE added nothing. #[test] fn the_score_carries_its_energy_provenance() { - let ci = intensity(); - let score = |cost: JouleCost| { - SciScore::compute(SciInputs { - joule_cost: &cost, - intensity: &ci, - pue: &Pue::new(1.2), - embodied_g_co2e: 0.0, - functional_units: 1.0, - }) - }; - let measured = score(JouleCost::measured(1_000_000)); - let estimated = score(JouleCost::estimated(1_000_000)); - assert_eq!(measured.energy_source, JouleSource::Measured); - assert_eq!(estimated.energy_source, JouleSource::Estimated); - assert_eq!(measured.sci, estimated.sci); + assert_eq!(score(JouleCost::measured(1_000_000), 1.0).energy_source, JouleSource::Measured); + assert_eq!(score(JouleCost::estimated(1_000_000), 1.0).energy_source, JouleSource::Estimated); + assert_eq!( + score(JouleCost::measured(1_000_000), 1.2).energy_source, + JouleSource::Estimated, + "a PUE's overhead is not a reading" + ); + assert_eq!(score(JouleCost::measured(1_000_000), 1.2).sci, score(JouleCost::estimated(1_000_000), 1.2).sci); } /// A score written before `energy_source` existed recorded no @@ -187,30 +243,71 @@ mod tests { assert_eq!(back.energy_source, JouleSource::Estimated); } - /// An eval report's SCI score is measured exactly when the report is. + fn run(joules_per_correct: u64, correct: usize, joule_source: JouleSource) -> EvalEnergy { + EvalEnergy { joules_per_correct, correct, joule_source } + } + + /// An eval run's score is measured exactly when the run's joules were + /// (and the PUE adds nothing); the run's provenance is never upgraded. + /// Compiled in every build: the `eval-bridge` feature only fills the + /// run's energy from a report. + #[test] + fn an_eval_runs_score_keeps_the_runs_provenance() { + let ci = intensity(); + let per_correct = |source, pue: f64| { + sci_per_correct_answer(run(1_000_000, 2, source), &ci, &Pue::new(pue), 0.0).expect("two correct answers") + }; + assert_eq!(per_correct(JouleSource::Estimated, 1.0).energy_source, JouleSource::Estimated); + assert_eq!(per_correct(JouleSource::Measured, 1.0).energy_source, JouleSource::Measured); + assert_eq!(per_correct(JouleSource::Measured, 1.2).energy_source, JouleSource::Estimated); + // 2 correct × 1 J, × 1.2 PUE = 2.4 J either way, over R = 2. + let (e, m) = (per_correct(JouleSource::Estimated, 1.2), per_correct(JouleSource::Measured, 1.2)); + assert!((e.energy_kwh - 2.4 / 3_600_000.0).abs() < 1e-15, "{}", e.energy_kwh); + assert_eq!(e.functional_units, 2.0); + assert_eq!(e.sci, m.sci); + } + + /// A run with no correct answer has no score per correct answer. Its + /// `u64::MAX` sentinel is never read as a u64::MAX µJ energy term, + /// whatever `correct` says beside it. + #[test] + fn a_run_with_no_correct_answer_has_no_score() { + let ci = intensity(); + let pue = Pue::new(1.2); + for source in [JouleSource::Measured, JouleSource::Estimated] { + assert_eq!(sci_per_correct_answer(run(u64::MAX, 0, source), &ci, &pue, 0.0), None); + assert_eq!(sci_per_correct_answer(run(5_000_000, 0, source), &ci, &pue, 0.0), None); + assert_eq!(sci_per_correct_answer(run(u64::MAX, 3, source), &ci, &pue, 0.0), None); + } + assert!(sci_per_correct_answer(run(1, 1, JouleSource::Measured), &ci, &pue, 0.0).is_some()); + } + + /// The bridge scores a report exactly as the run's energy is scored, and + /// gives none for a report with no correct answer. #[cfg(feature = "eval-bridge")] #[test] - fn an_eval_reports_score_keeps_the_reports_provenance() { - let report = |joule_source| eoc_eval::EvalReport { + fn an_eval_reports_score_is_its_runs_score() { + let report = |correct, joules_per_correct, joule_source| eoc_eval::EvalReport { harness: "fixed".into(), model: "test".into(), total_cases: 4, - correct: 2, - score: 0.5, + correct, + score: correct as f64 / 4.0, metric: eoc_eval::Metric::Accuracy, - joules_per_correct: 1_000_000, + joules_per_correct, joule_source, latency_p50_ms: 1, latency_p95_ms: 1, }; let ci = intensity(); - let pue = Pue::new(1.2); - let estimated = sci_from_eval_report(&report(JouleSource::Estimated), &ci, &pue, 0.0); - let measured = sci_from_eval_report(&report(JouleSource::Measured), &ci, &pue, 0.0); - assert_eq!(estimated.energy_source, JouleSource::Estimated); - assert_eq!(measured.energy_source, JouleSource::Measured); - // 2 correct × 1 J, × 1.2 PUE = 2.4 J either way. - assert!((estimated.energy_kwh - 2.4 / 3_600_000.0).abs() < 1e-15); - assert_eq!(estimated.sci, measured.sci); + for pue in [Pue::new(1.0), Pue::new(1.2)] { + for source in [JouleSource::Estimated, JouleSource::Measured] { + let got = sci_from_eval_report(&report(2, 1_000_000, source), &ci, &pue, 0.1); + let want = sci_per_correct_answer(run(1_000_000, 2, source), &ci, &pue, 0.1); + assert_eq!(got, want); + assert!(got.is_some()); + } + } + assert_eq!(sci_from_eval_report(&report(0, u64::MAX, JouleSource::Measured), &ci, &Pue::new(1.2), 0.0), None); } } diff --git a/eoc/eoc-rs/crates/eoc-carbon/tests/account_per_request.rs b/eoc/eoc-rs/crates/eoc-carbon/tests/account_per_request.rs index 7fc0dbbfa..7ddac18cc 100644 --- a/eoc/eoc-rs/crates/eoc-carbon/tests/account_per_request.rs +++ b/eoc/eoc-rs/crates/eoc-carbon/tests/account_per_request.rs @@ -30,4 +30,7 @@ fn one_kilojoule_400g_pue_1_2() { assert!((acc.facility_joules - 1200.0).abs() < 1e-9); assert_eq!(acc.pue, 1.2); assert_eq!(acc.g_co2e_per_kwh, 400.0); + // The IT joules were read; the PUE's overhead was not. + assert_eq!(acc.it_energy_source, JouleSource::Measured); + assert_eq!(acc.facility_energy_source, JouleSource::Estimated); } -- 2.54.0 (Apple Git-157) From 2c353d68cd05ced8f57fade4515a5df43fbb1de0 Mon Sep 17 00:00:00 2001 From: Transaction Science Date: Wed, 30 Sep 2026 21:07:49 -0400 Subject: [PATCH 15/22] eoc-meter: RAPL and NVML are selected only when their counter keeps advancing, as IOReport's CPU rail is eoc-meter has both vendor counters: `LinuxRaplCounter` (Linux, package 0's powercap `energy_uj`) and `NvmlCounter` (feature `cuda`, GPU 0's total-energy counter). `detect()` selected RAPL whenever the file existed and NVML whenever it initialised. Neither was held to the rule every IOReport backend in the family now uses, so a counter that held, or was republished in lumps (a virtualised powercap tree can do either), was sealed as a reading by every consumer that tags a live counter's delta `Measured`. And `energy_uj` is root-only on most current kernels, so on those hosts RAPL was "selected" with every read failing. The rule is now one rule for every rung. `rail_liveness`, which held the IOReport rule, is compiled on every platform and generalised: a `RailWatch` carries which channels are rails (`default()`: IOReport's CPU energy aggregates; `every_channel()`: a single vendor counter's one channel), and `probe_counter` runs the same `watch` loop, stopping rule and thresholds over a counter that opens. A counter is live only if it advanced in at least five sample intervals, at least 80% of those observed, over at least 300 ms. A fall (RAPL's wrap at `max_energy_range_uj`) is no advance. - `CounterStatus` (`Unavailable`, `Frozen`, `Live`) is public, with `rapl_status()` and `nvml_status()`, each probed once per process and cached, as `ioreport_status()` is. Off Linux, or without `cuda`, they are `Unavailable` without probing. - RAPL's probe burns 25 ms of the calling thread's own CPU between samples, on its CPU clock, so the package counter is watched under load. Where `energy_uj` cannot be read it is `Unavailable` without burning anything. - NVML's probe waits 100 ms between samples instead: burning CPU does not load the GPU, and a powered GPU's own draw advances a live counter. That shows the counter is published continuously, not that it tracks a load. - `LinuxRaplCounter::try_detect` and `NvmlCounter::try_detect` go through `try_detect_given(status)`, which declines every status but `Live` before NVML is touched. `detect_with` takes each rung's status, asked only when the ladder reaches that rung. - The thread CPU clock, the wall clock and the CPU-burning tick moved out of `macos_ioreport` into a `clocks` module (macOS and Linux, the crate's second scoped `allow(unsafe_code)`, one `clock_gettime`), so RAPL's probe burns exactly what IOReport's does. Behaviour on a Linux host whose `energy_uj` is root-only: `detect()` now returns the stub, where it returned a RAPL counter whose every read failed. Consumers got no reading either way. On a host whose RAPL is readable, the first `detect()` now costs about 300 ms of CPU (a second if the counter is not live). Tests. The rule for a single counter runs on scripted samples and a scripted clock on every platform: advancing every 25 ms is live at 12 intervals, 13 samples (at 100 ms, at 5 intervals); a held counter is frozen after 41 samples; one step, a two-step refresh, and advancing in 2 of every 3 intervals are frozen, while advancing in 4 of every 5 is live; a wrap is no advance; a counter that never reads is sampled to the five-budget cap (201 samples) and is unavailable, and one that does not open is never sampled or ticked; a vendor watch counts every channel where IOReport's counts only `CPU Energy`. The RAPL module is compiled for tests on a Mac too, so its own probe runs on scripted reads (live; held, one step: frozen; EACCES on every read, or one read then none: unavailable), its sampler is pinned (raw value, saturating at `i64::MAX`), and only `Live` constructs it. Selection: a frozen or unavailable RAPL or NVML status never selects that rung, and a live RAPL status selects it exactly on Linux. Under `cuda`, a declined NVML status never even tries to initialise NVML (counted per thread), and `Live` tries once. On Linux, a hardware test checks the cached RAPL status against the counter itself across 200 ms of CPU work, and that `detect()` never picks a RAPL counter that is not live. Falsified, each mutant restored byte-for-byte (sha256 checked): - the gate admitting `Frozen` fails 2 tests; - a frozen verdict mapped to `Live` fails 3; - the vendor probe watching only CPU rails fails 4; - RAPL's constructor ignoring its status fails 1; - NVML's constructor ignoring its status fails 1 under `cuda` (it survived until the open attempt was counted: on a host with no NVIDIA GPU, opening fails either way); - two advances in place of five fails 4. What ran where. On this M5 Max (macOS 27.0.1): `cargo test -p eoc-meter` 57 + 1 passed, with `--features cuda` 58 + 1, and the whole eoc workspace 665 passed, 0 failed, 3 ignored. For Linux, `cargo check -p eoc-meter --tests --target x86_64-unknown-linux-gnu`, with and without `cuda`, is clean. The Linux production probe and the RAPL hardware test were type-checked, not run: there is no Linux host with readable RAPL here, and no NVIDIA GPU, so neither rung has been seen live. clippy raises no new warning; the crate's two existing ones (a manual C-string, a duplicated `link` attribute) predate this. Co-Authored-By: Claude Opus 5.5 --- eoc/eoc-rs/crates/eoc-meter/src/lib.rs | 775 ++++++++++++++++++++++--- 1 file changed, 682 insertions(+), 93 deletions(-) diff --git a/eoc/eoc-rs/crates/eoc-meter/src/lib.rs b/eoc/eoc-rs/crates/eoc-meter/src/lib.rs index cc8e691ba..b78c6dfc9 100644 --- a/eoc/eoc-rs/crates/eoc-meter/src/lib.rs +++ b/eoc/eoc-rs/crates/eoc-meter/src/lib.rs @@ -17,8 +17,16 @@ //! Hardware counter dependencies are **platform-conditional and //! feature-gated**: //! -//! - Linux RAPL is read via `/sys/class/powercap/intel-rapl:0/energy_uj`. -//! No external crate required. +//! Every hardware counter is held to one liveness rule before [`detect`] +//! selects it: probed once per process, it must keep advancing — in at least +//! five separate sample intervals, at least 80% of those observed, over at +//! least 300 ms. A counter that holds, or is republished in lumps, is +//! declined, because a lump landing in one span would be read as that span's +//! energy. A live counter's delta is a reading; nothing else is. +//! +//! - Linux RAPL is read via `/sys/class/powercap/intel-rapl:0/energy_uj` — +//! **only when that counter reads and is live** ([`rapl_status`]; the +//! probe burns CPU on the calling thread). No external crate required. //! - Apple Silicon IOReport Energy Model counters are read through Apple's //! IOReport C API (no sudo, no external binary) — **only when the CPU //! energy rail is live** ([`ioreport_status`]). A subscription can open @@ -33,13 +41,17 @@ //! - macOS `powermetrics`: detected when the binary is on the PATH, but the //! reference wrapper does not parse it, so every read returns //! `Err(NoCounter)`: no figure, and callers keep their own estimate. -//! - NVIDIA NVML is gated behind the `cuda` feature. Off by default. +//! - NVIDIA NVML is gated behind the `cuda` feature (off by default), and +//! selected only when GPU 0's total-energy counter is live +//! ([`nvml_status`]). //! - WASM and unknown targets fall through to `StubCounter`. -// `deny` not `forbid`: the whole crate is safe except the audited Apple-Silicon -// IOReport FFI module (`macos_ioreport`), which carries a scoped -// `#[allow(unsafe_code)]`. The on-die energy counters are unreachable without the -// `unsafe extern "C"` IOReport calls; everything else stays unsafe-free. +// `deny` not `forbid`: the whole crate is safe except two audited FFI modules, +// each with a scoped `#[allow(unsafe_code)]`: the Apple-Silicon IOReport module +// (`macos_ioreport`) — the on-die energy counters are unreachable without the +// `unsafe extern "C"` IOReport calls — and `clocks`, whose one call is +// `clock_gettime` for the probing thread's CPU clock. Everything else stays +// unsafe-free. #![deny(unsafe_code)] // `Error` is used by platform-conditional modules below. @@ -73,13 +85,15 @@ impl JouleCounter for StubCounter { } } -// The CPU-rail liveness rule, kept apart from the IOReport FFI so every part of -// it — the name rule, the pairing of samples, the per-rail count, when to stop -// sampling, and the verdict's mapping to a status — runs under test with -// scripted samples and a scripted clock. The FFI only takes a sample and lists -// its channels as (name, absolute raw integer); everything that decides lives -// here. Compiled for tests on every platform, so a Linux CI leg runs it too. -#[cfg(any(target_os = "macos", test))] +// The sustained-advance liveness rule, kept apart from every counter's FFI so +// every part of it — the name rule, the pairing of samples, the per-rail count, +// when to stop sampling, and the verdict's mapping to a status — runs under test +// with scripted samples and a scripted clock. The FFI only takes a sample and +// lists its channels as (name, absolute raw integer); everything that decides +// lives here. One rule for every vendor counter: IOReport watches the CPU energy +// rails of an Energy Model sample; RAPL and NVML hand their one counter as the +// only channel. Compiled on every platform, so a Linux CI leg runs its tests. +#[cfg_attr(not(target_os = "macos"), allow(dead_code))] mod rail_liveness { use std::time::Duration; @@ -175,11 +189,6 @@ mod rail_liveness { /// must count as one advance each, never as two advances of one rail. pub(crate) type RailKey = (String, u32); - /// The CPU rails in one sample, keyed by [`RailKey`], in sample order. - fn cpu_rails(sample: &[(String, i64)]) -> Vec<(RailKey, i64)> { - keyed(sample, is_cpu_energy_rail) - } - /// The channels of one sample whose names pass `keep`, keyed by /// [`RailKey`] (the ordinal counts same-named channels among those kept), /// in sample order. @@ -198,7 +207,7 @@ mod rail_liveness { /// The liveness rule over a sequence of samples. /// /// Each usable sample is compared with the PREVIOUS usable one — never - /// with the first — on each CPU rail's absolute raw integer, before unit + /// with the first — on each rail's absolute raw integer, before unit /// conversion (a small advance in a fine unit cannot truncate to zero). A /// rail advances in an interval when its value rose over that interval; so /// a rail advances at most once per interval, and each rail is counted on @@ -208,9 +217,15 @@ mod rail_liveness { /// is put to [`sustained`], over the intervals observed so far and the wall /// time since the first usable sample; the first time one passes, the watch /// says Live, and stays Live. - #[derive(Debug, Default)] + /// + /// Which channels of a sample are rails is the watch's `pick`: the CPU + /// energy aggregates for IOReport ([`RailWatch::default`]), every channel + /// for a single vendor counter ([`RailWatch::every_channel`]). + #[derive(Debug)] pub(crate) struct RailWatch { - /// The CPU rails of the last usable sample. + /// Which channels of a sample are watched as rails. + pick: fn(&str) -> bool, + /// The rails of the last usable sample. prev: Option>, /// The wall-clock reading of the first usable sample. first: Option, @@ -225,13 +240,33 @@ mod rail_liveness { live: bool, } + impl Default for RailWatch { + /// A watch of IOReport's CPU energy rails ([`is_cpu_energy_rail`]). + fn default() -> Self { + Self::of(is_cpu_energy_rail) + } + } + impl RailWatch { + /// A watch of the channels `pick` accepts. + fn of(pick: fn(&str) -> bool) -> Self { + Self { pick, prev: None, first: None, intervals: 0, window: Duration::ZERO, rails: Vec::new(), live: false } + } + + /// A watch of every channel of a sample: for a single vendor counter + /// (RAPL's package counter, NVML's total-energy counter), whose every + /// channel counts the domain it claims. + #[cfg_attr(not(any(target_os = "linux", feature = "cuda")), allow(dead_code))] // used by the RAPL and NVML rungs + pub(crate) fn every_channel() -> Self { + Self::of(|_| true) + } + /// Fold in one usable sample: every channel in it, as (name, absolute /// raw integer), and `at`, the wall-clock reading when the sample was /// in hand. A sample that could not be taken is simply not observed, /// so the next usable one pairs with the last usable one. pub(crate) fn observe(&mut self, sample: &[(String, i64)], at: Duration) { - let cur = cpu_rails(sample); + let cur = keyed(sample, self.pick); let first = *self.first.get_or_insert(at); self.window = at.saturating_sub(first); if let Some(prev) = &self.prev { @@ -291,13 +326,24 @@ mod rail_liveness { /// clock `now` read as it returns. [`probe_with`] runs it with the /// sampler, tick and wall clock it is given, and records each sample call. pub(crate) fn watch( + sample: impl FnMut() -> Option>, + tick: impl FnMut(), + now: impl Fn() -> Duration, + budget: Duration, + ) -> RailWatch { + watch_from(RailWatch::default(), sample, tick, now, budget) + } + + /// [`watch`], folding samples into the watch `w` it is given (so a + /// single vendor counter's probe watches every channel). + pub(crate) fn watch_from( + mut w: RailWatch, mut sample: impl FnMut() -> Option>, mut tick: impl FnMut(), now: impl Fn() -> Duration, budget: Duration, ) -> RailWatch { let started = now(); - let mut w = RailWatch::default(); if let Some(s) = sample() { w.observe(&s, now()); } @@ -460,6 +506,47 @@ mod rail_liveness { status == crate::IoReportStatus::Live } + /// One liveness probe of a single vendor energy counter (RAPL, NVML): + /// `open` it; if it opens, sample it — each sample its channels as + /// (name, absolute raw integer), every one of them a rail — `tick` + /// between samples, by the wall clock `now`, for `budget`, by the same + /// rule and stopping rule as the IOReport probe ([`watch`]). `None` when + /// it does not open: then nothing is sampled and nothing ticks. Generic + /// over each part so the tests drive this very function with a scripted + /// counter, tick and clock; each rung's cached probe passes the real ones. + #[cfg_attr(not(any(target_os = "linux", feature = "cuda")), allow(dead_code))] // used by the RAPL and NVML rungs + pub(crate) fn probe_counter( + open: impl FnOnce() -> Option, + mut sample: impl FnMut(&C) -> Option>, + tick: impl FnMut(), + now: impl Fn() -> Duration, + budget: Duration, + ) -> Option { + let counter = open()?; + Some(watch_from(RailWatch::every_channel(), || sample(&counter), tick, now, budget)) + } + + /// The status a counter probe gives its rung. No probe (the counter did + /// not open) and not one interval observed (no two usable samples in five + /// budgets) are both `Unavailable`; only a rail that kept advancing is + /// `Live`. + #[cfg_attr(not(any(target_os = "linux", feature = "cuda")), allow(dead_code))] // used by the RAPL and NVML rungs + pub(crate) fn counter_status(probed: Option<&RailWatch>) -> crate::CounterStatus { + match probed { + None => crate::CounterStatus::Unavailable, + Some(w) if w.intervals == 0 => crate::CounterStatus::Unavailable, + Some(w) if w.is_live() => crate::CounterStatus::Live, + Some(_) => crate::CounterStatus::Frozen, + } + } + + /// Whether a counter rung with this status may be selected: only when it + /// is live. The one gate the RAPL and NVML constructors pass through. + #[cfg_attr(not(any(target_os = "linux", feature = "cuda")), allow(dead_code))] // used by the RAPL and NVML rungs + pub(crate) fn admits_vendor_counter(status: crate::CounterStatus) -> bool { + status == crate::CounterStatus::Live + } + #[cfg(test)] mod tests { use super::*; @@ -1101,6 +1188,190 @@ mod rail_liveness { assert!(!admits_counter(declined), "{declined:?} admitted the counter"); } } + + // ── A single vendor counter (RAPL, NVML) ───────────────────────────── + + /// What a scripted counter probe did: its status, how many samples it + /// took and how many ticks it ran. `opens` is whether the counter + /// opens; `read(i)` is its i-th sample, `None` for a failed read. The + /// clock moves only with the tick, by `step`. + fn counter_probe(opens: bool, step: Duration, mut read: impl FnMut(usize) -> Option) -> (crate::CounterStatus, usize, u32) { + let clock = Cell::new(Duration::ZERO); + let (mut taken, mut ticks) = (0usize, 0u32); + let probed = probe_counter( + || opens.then_some("counter"), + |c: &&str| { + assert!(taken < 100_000, "the probe never stopped sampling"); + taken += 1; + read(taken - 1).map(|raw| vec![(c.to_string(), raw)]) + }, + || { + ticks += 1; + clock.set(clock.get() + step); + }, + || clock.get(), + BUDGET, + ); + (counter_status(probed.as_ref()), taken, ticks) + } + + /// A counter that advances at every sample is live once five intervals + /// and 300 ms have been watched: at 25 ms a sample, 12 intervals. + #[test] + fn a_vendor_counter_advancing_every_sample_is_live() { + let (status, taken, ticks) = counter_probe(true, TICK, |i| Some(81_000_000 + 1_400 * i as i64)); + assert_eq!((status, taken, ticks), (crate::CounterStatus::Live, 13, 12)); + // Slow samples: five intervals of 100 ms are 500 ms, past the window. + let (status, taken, _) = counter_probe(true, Duration::from_millis(100), |i| Some(i as i64)); + assert_eq!((status, taken), (crate::CounterStatus::Live, 6)); + } + + /// A counter that holds is frozen, watched for the whole budget. + #[test] + fn a_vendor_counter_that_holds_is_frozen() { + let (status, taken, _) = counter_probe(true, TICK, |_| Some(81_000_000)); + assert_eq!((status, taken), (crate::CounterStatus::Frozen, 41)); + } + + /// A counter republished in lumps — one step, or a refresh in two + /// steps 64 ms apart — is frozen: never the sustained advance of a live + /// count, and a lump landing in one caller's span would be sealed as + /// that span's reading. + #[test] + fn a_vendor_counter_republished_in_lumps_is_frozen() { + let (status, ..) = counter_probe(true, TICK, |i| Some(if i < 7 { 1_000 } else { 61_000 })); + assert_eq!(status, crate::CounterStatus::Frozen, "one step"); + let (status, ..) = counter_probe(true, Duration::from_millis(32), |i| Some(two_step_refresh(i, 3, 5))); + assert_eq!(status, crate::CounterStatus::Frozen, "a two-step refresh"); + // Advancing in two of every three intervals is not a live count... + let (status, ..) = counter_probe(true, TICK, |i| Some((i - i / 3) as i64)); + assert_eq!(status, crate::CounterStatus::Frozen, "advancing in 2 of every 3 intervals"); + // ...and a counter missing one interval in five still is live. + let (status, ..) = counter_probe(true, TICK, |i| Some((i - i / 5) as i64)); + assert_eq!(status, crate::CounterStatus::Live, "advancing in 4 of every 5 intervals"); + } + + /// A counter that wraps (RAPL's `energy_uj` rolls over at + /// `max_energy_range_uj`) fell over that interval, which is no advance; + /// it is live again from the next one. + #[test] + fn a_wrap_is_not_an_advance() { + let (status, ..) = counter_probe(true, TICK, |i| Some(if i < 3 { 262_000_000 + i as i64 } else { i as i64 })); + assert_eq!(status, crate::CounterStatus::Live); + let w = { + let mut w = RailWatch::every_channel(); + for (i, raw) in [(0, 900), (1, 950), (2, 10), (3, 60)] { + w.observe(&[("intel-rapl:0".to_string(), raw)], TICK * i); + } + w + }; + assert_eq!(w.rails, vec![(key("intel-rapl:0", 0), 2)], "the fall is no advance: {w:?}"); + } + + /// A counter that opens but never reads is sampled to the cap of five + /// budgets and is unavailable; one that does not open is unavailable + /// with nothing sampled and nothing ticked. + #[test] + fn a_vendor_counter_that_never_reads_or_opens_is_unavailable() { + let (status, taken, _) = counter_probe(true, TICK, |_| None); + assert_eq!((status, taken), (crate::CounterStatus::Unavailable, 201)); + let (status, taken, ticks) = counter_probe(false, TICK, |_| panic!("sampled a counter that did not open")); + assert_eq!((status, taken, ticks), (crate::CounterStatus::Unavailable, 0, 0)); + } + + /// Every channel of a vendor counter's sample is a rail; IOReport's + /// watch keeps only the CPU energy aggregates. + #[test] + fn a_vendor_counter_watches_every_channel() { + let sample = |i: i64| vec![("intel-rapl:0".to_string(), 10 * i), ("CPU Energy".to_string(), 5)]; + let (mut all, mut cpu) = (RailWatch::every_channel(), RailWatch::default()); + for i in 0..3 { + all.observe(&sample(i), TICK * i as u32); + cpu.observe(&sample(i), TICK * i as u32); + } + assert_eq!(all.rails, vec![(key("intel-rapl:0", 0), 2), (key("CPU Energy", 0), 0)]); + assert_eq!(cpu.rails, vec![(key("CPU Energy", 0), 0)]); + } + + /// Only a live counter admits its rung. + #[test] + fn only_a_live_vendor_counter_admits_its_rung() { + use crate::CounterStatus as S; + assert!(admits_vendor_counter(S::Live)); + for declined in [S::Unavailable, S::Frozen] { + assert!(!admits_vendor_counter(declined), "{declined:?} admitted the rung"); + } + } + } +} + +/// The clocks and the tick of the liveness probes that burn CPU (IOReport's, +/// RAPL's): this thread's CPU clock, a monotonic wall clock, and a tick that +/// spins until this thread has burned [`clocks::PROBE_TICK`] of its own CPU. +/// The one other module allowed `unsafe`: the thread CPU clock is only +/// reachable through `clock_gettime`. +#[cfg(any(target_os = "macos", target_os = "linux"))] +#[allow(unsafe_code)] +mod clocks { + use std::os::raw::{c_int, c_long}; + use std::sync::OnceLock; + use std::time::{Duration, Instant}; + + /// `struct timespec` on 64-bit macOS and Linux. + #[repr(C)] + struct Timespec { + tv_sec: c_long, + tv_nsec: c_long, + } + + /// `clockid_t`: an unsigned enum on macOS, an `int` on Linux. + #[cfg(target_os = "macos")] + type ClockId = std::os::raw::c_uint; + #[cfg(target_os = "linux")] + type ClockId = c_int; + + /// `CLOCK_THREAD_CPUTIME_ID`: 16 in macOS ``, 3 in Linux + /// ``. + #[cfg(target_os = "macos")] + const CLOCK_THREAD_CPUTIME_ID: ClockId = 16; + #[cfg(target_os = "linux")] + const CLOCK_THREAD_CPUTIME_ID: ClockId = 3; + + unsafe extern "C" { + fn clock_gettime(clock_id: ClockId, tp: *mut Timespec) -> c_int; + } + + /// CPU time the calling thread has consumed. Zero if the clock cannot be + /// read, which leaves a probe tick spinning to its wall cap: slower, but + /// still burning CPU. + pub(crate) fn thread_cpu_time() -> Duration { + let mut ts = Timespec { tv_sec: 0, tv_nsec: 0 }; + if unsafe { clock_gettime(CLOCK_THREAD_CPUTIME_ID, &mut ts) } != 0 { + return Duration::ZERO; + } + Duration::new(ts.tv_sec.max(0) as u64, ts.tv_nsec.clamp(0, 999_999_999) as u32) + } + + /// Wall time on the monotonic clock, from a fixed origin in this process. + pub(crate) fn wall_now() -> Duration { + static ORIGIN: OnceLock = OnceLock::new(); + ORIGIN.get_or_init(Instant::now).elapsed() + } + + /// The CPU the probing thread burns between two liveness samples: this + /// much of its own CPU time, measured on its CPU clock, so host load + /// stretches a tick's wall time but never shortens its CPU. + pub(crate) const PROBE_TICK: Duration = Duration::from_millis(25); + + /// However little CPU the probing thread gets, one tick ends after this + /// much wall time. + pub(crate) const PROBE_TICK_WALL_CAP: Duration = Duration::from_millis(500); + + /// One tick of a CPU-burning probe: spin until this thread's CPU clock has + /// advanced [`PROBE_TICK`], or [`PROBE_TICK_WALL_CAP`] of wall time has + /// passed. + pub(crate) fn cpu_burn_tick() { + crate::rail_liveness::spin_for_cpu(thread_cpu_time, wall_now, PROBE_TICK, PROBE_TICK_WALL_CAP); } } @@ -1128,11 +1399,11 @@ mod rail_liveness { #[allow(non_camel_case_types, unsafe_code)] mod macos_ioreport { use super::*; + use crate::clocks::{thread_cpu_time, wall_now}; use crate::rail_liveness::{self, ProbeRun, Sample}; - use std::os::raw::{c_int, c_long, c_uint, c_void}; + use std::os::raw::{c_int, c_long, c_void}; + use std::sync::Mutex; use std::sync::atomic::{AtomicUsize, Ordering}; - use std::sync::{Mutex, OnceLock}; - use std::time::{Duration, Instant}; type CFTypeRef = *const c_void; type CFStringRef = *const c_void; @@ -1187,34 +1458,6 @@ mod macos_ioreport { buffer_size: c_long, encoding: u32, ) -> bool; - fn clock_gettime(clock_id: c_uint, tp: *mut Timespec) -> c_int; - } - - /// `struct timespec` on 64-bit macOS. - #[repr(C)] - struct Timespec { - tv_sec: c_long, - tv_nsec: c_long, - } - - /// `CLOCK_THREAD_CPUTIME_ID` in macOS ``. - const CLOCK_THREAD_CPUTIME_ID: c_uint = 16; - - /// CPU time the calling thread has consumed. Zero if the clock cannot be - /// read, which leaves a probe tick spinning to its wall cap: slower, but - /// still burning CPU. - fn thread_cpu_time() -> Duration { - let mut ts = Timespec { tv_sec: 0, tv_nsec: 0 }; - if unsafe { clock_gettime(CLOCK_THREAD_CPUTIME_ID, &mut ts) } != 0 { - return Duration::ZERO; - } - Duration::new(ts.tv_sec.max(0) as u64, ts.tv_nsec.clamp(0, 999_999_999) as u32) - } - - /// Wall time on the monotonic clock, from a fixed origin in this process. - fn wall_now() -> Duration { - static ORIGIN: OnceLock = OnceLock::new(); - ORIGIN.get_or_init(Instant::now).elapsed() } #[link(name = "IOReport", kind = "dylib")] @@ -1434,19 +1677,9 @@ mod macos_ioreport { } } - /// The CPU the probing thread burns between two liveness samples: this - /// much of its own CPU time, measured on its CPU clock, so host load - /// stretches a tick's wall time but never shortens its CPU. - const PROBE_TICK: Duration = Duration::from_millis(25); - - /// However little CPU the probing thread gets, one tick ends after this - /// much wall time. - const PROBE_TICK_WALL_CAP: Duration = Duration::from_millis(500); - - /// One tick of the probe: spin until this thread's CPU clock has advanced - /// [`PROBE_TICK`], or [`PROBE_TICK_WALL_CAP`] of wall time has passed. + /// One tick of the probe: [`crate::clocks::cpu_burn_tick`]. fn probe_tick() { - rail_liveness::spin_for_cpu(thread_cpu_time, wall_now, PROBE_TICK, PROBE_TICK_WALL_CAP); + crate::clocks::cpu_burn_tick(); } /// How many times [`host_status`] — the probe behind the cached @@ -1727,6 +1960,7 @@ mod macos_ioreport { use super::script::{self, Script}; use super::*; use crate::rail_liveness::{CpuRail, RailKey, SampleCall}; + use std::time::{Duration, Instant}; use std::cell::RefCell; use std::rc::Rc; use std::sync::atomic::AtomicBool; @@ -2519,7 +2753,7 @@ mod macos_ioreport { let before = script::counter_opens(); let given = MacosIoReportCounter::try_detect_given(status); assert!(matches!(given, Err(Error::NoCounter)), "try_detect_given({status:?}) did not decline with NoCounter"); - let picked = crate::detect_with(|| status).name(); + let picked = crate::detect_with(|| CounterStatus::Unavailable, || CounterStatus::Unavailable, || status).name(); assert_ne!(picked, "ioreport", "detect_with({status:?}) selected the IOReport counter"); let (built, picked) = script::with_status(status, || (MacosIoReportCounter::try_detect().is_ok(), crate::detect().name())); @@ -2554,7 +2788,7 @@ mod macos_ioreport { if let Err(e) = given { assert!(matches!(e, Error::NoCounter), "try_detect_given({status:?}) declined with {e:?}"); } - let picked = crate::detect_with(|| status).name(); + let picked = crate::detect_with(|| CounterStatus::Unavailable, || CounterStatus::Unavailable, || status).name(); assert_eq!(picked == "ioreport", admitted, "detect_with({status:?}) selected {picked}"); let (built, picked) = script::with_status(status, || { assert_eq!(crate::ioreport_status(), status, "the installed status"); @@ -3251,23 +3485,115 @@ pub fn liveness_sampler_opens() -> usize { } -#[cfg(target_os = "linux")] +/// The liveness of a single vendor energy counter rung — RAPL's package +/// counter, NVML's total-energy counter — as its probe found it on this host, +/// by the rule IOReport's CPU rail is held to. Probed once per process and +/// cached; see [`rapl_status`] and [`nvml_status`]. Only +/// [`CounterStatus::Live`] admits the rung: a live vendor counter's reading is +/// a measurement, and nothing else from it is. +#[derive(Debug, Clone, Copy, PartialEq, Eq)] +pub enum CounterStatus { + /// The counter could not be opened or read — not this platform, the + /// feature not built, no such device, a locked-down sysfs (`energy_uj` + /// is root-only on most current kernels) — or it gave fewer than two + /// usable samples in five budgets. Never selected. + Unavailable, + /// The counter read, but did not keep advancing while the probe ran: it + /// held, or was republished in lumps. Never selected: a lump landing in + /// one caller's span would be sealed as that span's reading. + Frozen, + /// The counter kept advancing while the probe ran: in at least five + /// separate sample intervals, at least 80% of those observed, over at + /// least 300 ms. The rung is selected. + Live, +} + +/// How long, in wall time, a vendor counter's liveness probe watches a counter +/// that is not live before declining it: one second, as +/// `CPU_RAIL_LIVENESS_BUDGET`. A live counter is called live about 300 ms in. +/// Paid once per process, and only where the counter opens. +#[cfg_attr(not(any(target_os = "linux", feature = "cuda")), allow(dead_code))] // used by the RAPL and NVML rungs +const VENDOR_COUNTER_LIVENESS_BUDGET: std::time::Duration = std::time::Duration::from_millis(1000); + +/// This host's RAPL status, probed once per process and cached. +/// +/// [`detect`] selects the RAPL rung only when this is [`CounterStatus::Live`]: +/// the package-0 counter (`/sys/class/powercap/intel-rapl:0/energy_uj`) read +/// and kept advancing while the probing thread burned CPU, in at least five +/// sample intervals, at least 80% of them, over at least 300 ms. One that +/// holds, or is republished in lumps (a virtualised powercap tree can do +/// either), is declined. Where the counter is absent or unreadable it is +/// [`CounterStatus::Unavailable`] without burning anything. Always +/// `Unavailable` off Linux. The first call on a host whose counter reads burns +/// about 300 ms of the calling thread's CPU (a second if it is not live). +pub fn rapl_status() -> CounterStatus { + #[cfg(target_os = "linux")] + { + static STATUS: std::sync::OnceLock = std::sync::OnceLock::new(); + *STATUS.get_or_init(linux_rapl::host_status) + } + #[cfg(not(target_os = "linux"))] + { + CounterStatus::Unavailable + } +} + +/// This host's NVML status, probed once per process and cached. Always +/// [`CounterStatus::Unavailable`] without the `cuda` feature. +/// +/// [`detect`] selects the NVML rung only when this is [`CounterStatus::Live`]: +/// GPU 0's total-energy counter kept advancing, in at least five sample +/// intervals, at least 80% of them, over at least 300 ms. The probe cannot +/// load the GPU, so it waits between samples rather than burning CPU, and a +/// powered GPU's own draw advances a live counter; this shows the counter is +/// published continuously, not that it tracks a load. One that holds or steps +/// in lumps is declined. The first call where NVML opens waits about half a +/// second (a second if the counter is not live). +pub fn nvml_status() -> CounterStatus { + #[cfg(feature = "cuda")] + { + static STATUS: std::sync::OnceLock = std::sync::OnceLock::new(); + *STATUS.get_or_init(nvml::host_status) + } + #[cfg(not(feature = "cuda"))] + { + CounterStatus::Unavailable + } +} + +// Compiled for tests on every platform, so its sampler and gate run under test +// on a Mac too; selected only on Linux. +#[cfg(any(target_os = "linux", test))] +#[cfg_attr(not(target_os = "linux"), allow(dead_code))] mod linux_rapl { use super::*; + use crate::rail_liveness; + + /// Intel / AMD RAPL package-0 energy, in micro-joules, through powercap. + pub(crate) const RAPL_ENERGY_PATH: &str = "/sys/class/powercap/intel-rapl:0/energy_uj"; /// Reads Intel RAPL counters via the powercap sysfs interface. + /// + /// Constructed only on a host whose package counter the liveness probe + /// found live ([`crate::rapl_status`]); a live RAPL counter is a vendor + /// energy counter, so a delta it reads is a measurement. pub struct LinuxRaplCounter { path: std::path::PathBuf, } impl LinuxRaplCounter { - /// Try to construct a RAPL counter pointing at the package-0 domain. + /// The RAPL package-0 counter, on a host where it is live. + /// `Err(NoCounter)` when it is absent, unreadable, or does not keep + /// advancing under load; [`crate::rapl_status`] says which. pub fn try_detect() -> Result { - let path = std::path::PathBuf::from( - "/sys/class/powercap/intel-rapl:0/energy_uj", - ); - if path.exists() { - Ok(Self { path }) + Self::try_detect_given(crate::rapl_status()) + } + + /// The constructor behind [`Self::try_detect`], given this host's + /// status: every status the gate does not admit declines. + pub(crate) fn try_detect_given(status: CounterStatus) -> Result { + if rail_liveness::admits_vendor_counter(status) { + Ok(Self { path: std::path::PathBuf::from(RAPL_ENERGY_PATH) }) } else { Err(Error::NoCounter) } @@ -3286,6 +3612,103 @@ mod linux_rapl { "rapl" } } + + /// One liveness sample of the counter `read` reads: its raw `energy_uj` + /// as the one rail, named `intel-rapl:0`. `None` for a failed read, which + /// the probe skips. A wrap shows as a fall, which is no advance. + pub(crate) fn sample(read: impl Fn() -> Option) -> Option> { + let uj = read()?; + Some(vec![("intel-rapl:0".to_string(), i64::try_from(uj).unwrap_or(i64::MAX))]) + } + + /// The counter at `path`, read and parsed; `None` when it cannot be. + pub(crate) fn read_energy_uj(path: &std::path::Path) -> Option { + std::fs::read_to_string(path).ok()?.trim().parse::().ok() + } + + /// The RAPL probe on a given reader: it opens when the counter reads at + /// all, and is then sampled by [`rail_liveness::probe_counter`], with + /// `tick` between samples, by the clock `now`. + pub(crate) fn status_with( + read: impl Fn() -> Option, + tick: impl FnMut(), + now: impl Fn() -> std::time::Duration, + ) -> CounterStatus { + rail_liveness::counter_status( + rail_liveness::probe_counter( + || read().map(|_| ()), + |_| sample(&read), + tick, + now, + crate::VENDOR_COUNTER_LIVENESS_BUDGET, + ) + .as_ref(), + ) + } + + /// The real RAPL probe: the real counter file, the CPU-burning tick (a + /// package counter must advance while this thread burns CPU) and the + /// monotonic clock. Cached by [`crate::rapl_status`], its only caller. + #[cfg(target_os = "linux")] + pub(crate) fn host_status() -> CounterStatus { + let path = std::path::Path::new(RAPL_ENERGY_PATH); + status_with(|| read_energy_uj(path), crate::clocks::cpu_burn_tick, crate::clocks::wall_now) + } + + #[cfg(test)] + mod tests { + use super::*; + use std::cell::Cell; + use std::time::Duration; + + /// The RAPL probe on scripted reads, with a clock that only the tick + /// moves, by 25 ms. + fn scripted(read: impl Fn(usize) -> Option) -> CounterStatus { + let (clock, reads) = (Cell::new(Duration::ZERO), Cell::new(0usize)); + status_with( + || { + reads.set(reads.get() + 1); + read(reads.get() - 1) + }, + || clock.set(clock.get() + Duration::from_millis(25)), + || clock.get(), + ) + } + + /// A package counter that advances at every read is live; one that + /// holds, steps once, or cannot be read (root-only `energy_uj`) is + /// declined, and so is one that reads only once. + #[test] + fn the_rapl_probe_admits_only_a_counter_that_keeps_advancing() { + assert_eq!(scripted(|i| Some(81_000_000 + 1_400_000 * i as u64)), CounterStatus::Live); + assert_eq!(scripted(|_| Some(81_000_000)), CounterStatus::Frozen, "a held value"); + assert_eq!(scripted(|i| Some(if i < 9 { 1 } else { 2 })), CounterStatus::Frozen, "one step"); + assert_eq!(scripted(|_| None), CounterStatus::Unavailable, "EACCES on every read"); + assert_eq!(scripted(|i| (i == 0).then_some(5)), CounterStatus::Unavailable, "read once, then never"); + } + + /// The sampler hands the raw counter as one rail, and saturates + /// rather than wrapping a value past `i64::MAX`. + #[test] + fn the_rapl_sample_is_the_raw_counter() { + assert_eq!(sample(|| Some(42)), Some(vec![("intel-rapl:0".to_string(), 42)])); + assert_eq!(sample(|| Some(u64::MAX)), Some(vec![("intel-rapl:0".to_string(), i64::MAX)])); + assert_eq!(sample(|| None), None); + assert_eq!(read_energy_uj(std::path::Path::new("/no/such/powercap/energy_uj")), None); + } + + /// Only a live status constructs the counter. + #[test] + fn only_a_live_rapl_status_constructs_the_counter() { + assert!(LinuxRaplCounter::try_detect_given(CounterStatus::Live).is_ok()); + for status in [CounterStatus::Unavailable, CounterStatus::Frozen] { + assert!( + matches!(LinuxRaplCounter::try_detect_given(status), Err(Error::NoCounter)), + "{status:?} constructed the RAPL counter" + ); + } + } + } } #[cfg(target_os = "linux")] @@ -3335,23 +3758,81 @@ pub use macos_powermetrics::MacosPowermetricsCounter; #[cfg(feature = "cuda")] mod nvml { use super::*; + use crate::rail_liveness; use nvml_wrapper::Nvml; /// Reads cumulative GPU energy via NVML. + /// + /// Constructed only on a host whose total-energy counter the liveness + /// probe found live ([`crate::nvml_status`]); a live NVML counter is a + /// vendor energy counter, so a delta it reads is a measurement of GPU 0. pub struct NvmlCounter { _nvml: Nvml, device_index: u32, } impl NvmlCounter { - /// Attempt to detect NVML and pick GPU 0. + /// NVML on GPU 0, on a host where its counter is live. + /// `Err(NoCounter)` when the counter is not live; NVML's own error + /// when it does not initialise. pub fn try_detect() -> Result { - let nvml = Nvml::init().map_err(|e| Error::Backend(e.to_string()))?; - Ok(Self { - _nvml: nvml, - device_index: 0, - }) + Self::try_detect_given(crate::nvml_status()) } + + /// The constructor behind [`Self::try_detect`], given this host's + /// status: every status the gate does not admit declines before NVML + /// is touched. + pub(crate) fn try_detect_given(status: CounterStatus) -> Result { + if !rail_liveness::admits_vendor_counter(status) { + return Err(Error::NoCounter); + } + Self::open().ok_or(Error::NoCounter) + } + + /// NVML on GPU 0, when it initialises and the counter reads. + fn open() -> Option { + // Test builds only: count the attempt on this thread, so a test can + // pin that a declined status never gets this far. + #[cfg(test)] + OPEN_ATTEMPTS.with(|n| n.set(n.get() + 1)); + let nvml = Nvml::init().ok()?; + nvml.device_by_index(0).ok()?.total_energy_consumption().ok()?; + Some(Self { _nvml: nvml, device_index: 0 }) + } + + /// One liveness sample: the device's total-energy counter as its one + /// rail, in its raw millijoules. `None` for a failed read. + fn sample(&self) -> Option> { + let mj = self._nvml.device_by_index(self.device_index).ok()?.total_energy_consumption().ok()?; + Some(vec![(format!("nvml device {} total energy", self.device_index), i64::try_from(mj).unwrap_or(i64::MAX))]) + } + } + + #[cfg(test)] + thread_local! { + /// Attempts on this thread to initialise NVML for a counter. + static OPEN_ATTEMPTS: std::cell::Cell = const { std::cell::Cell::new(0) }; + } + + /// How long the NVML probe waits between samples. Burning this thread's + /// CPU does not load the GPU, so the probe waits; long enough that a + /// counter the driver refreshes up to every 100 ms still advances in + /// every interval. One refreshed less often fails the 80% share and is + /// declined — estimated, never over-claimed. + const NVML_PROBE_WAIT: std::time::Duration = std::time::Duration::from_millis(100); + + /// The real NVML probe. Cached by [`crate::nvml_status`], its only caller. + pub(crate) fn host_status() -> CounterStatus { + rail_liveness::counter_status( + rail_liveness::probe_counter( + NvmlCounter::open, + NvmlCounter::sample, + || std::thread::sleep(NVML_PROBE_WAIT), + crate::nvml_wall_now, + crate::VENDOR_COUNTER_LIVENESS_BUDGET, + ) + .as_ref(), + ) } impl JouleCounter for NvmlCounter { @@ -3370,6 +3851,44 @@ mod nvml { "nvml" } } + + #[cfg(test)] + mod tests { + use super::*; + + /// Only a live status gets as far as initialising NVML: every declined + /// status is `NoCounter` without an attempt to open it, whether or not + /// NVML would open on this host. Live tries, and constructs the + /// counter only where NVML opens. + #[test] + fn only_a_live_nvml_status_constructs_the_counter() { + let attempts = || OPEN_ATTEMPTS.with(|n| n.get()); + for status in [CounterStatus::Unavailable, CounterStatus::Frozen] { + let before = attempts(); + assert!( + matches!(NvmlCounter::try_detect_given(status), Err(Error::NoCounter)), + "{status:?} constructed the NVML counter" + ); + assert_eq!(attempts(), before, "under {status:?} the constructor tried to open NVML"); + } + let opens = NvmlCounter::open().is_some(); + let before = attempts(); + assert_eq!(NvmlCounter::try_detect_given(CounterStatus::Live).is_ok(), opens); + assert_eq!(attempts(), before + 1, "given Live, the constructor did not try to open NVML"); + eprintln!("nvml: opens={opens} status={:?}", crate::nvml_status()); + if !opens { + assert_eq!(crate::nvml_status(), CounterStatus::Unavailable, "a status for an NVML that does not open"); + } + } + } +} + +/// Wall time on a monotonic clock, from a fixed origin in this process: the +/// NVML probe's clock (it needs no thread CPU clock, and builds on every OS). +#[cfg(feature = "cuda")] +fn nvml_wall_now() -> std::time::Duration { + static ORIGIN: std::sync::OnceLock = std::sync::OnceLock::new(); + ORIGIN.get_or_init(std::time::Instant::now).elapsed() } #[cfg(feature = "cuda")] @@ -3377,24 +3896,34 @@ pub use nvml::NvmlCounter; /// Runtime-detect the best available counter on this host. /// -/// Preference order: NVML (if compiled with `cuda` and present) → RAPL on -/// Linux → IOReport on macOS (Energy-Model counters, no sudo; only when its -/// CPU energy rail is live — see [`ioreport_status`]) → powermetrics on -/// macOS → `StubCounter`. +/// Preference order: NVML (if compiled with `cuda`, present, and its counter +/// live — see [`nvml_status`]) → RAPL on Linux (only when its package counter +/// is live — see [`rapl_status`]) → IOReport on macOS (Energy-Model counters, +/// no sudo; only when its CPU energy rail is live — see [`ioreport_status`]) +/// → powermetrics on macOS → `StubCounter`. Every hardware rung is held to +/// the same sustained-advance rule; a rung whose counter is not live is +/// skipped, so callers estimate rather than seal what it read. pub fn detect() -> Box { - detect_with(ioreport_status) + detect_with(nvml_status, rapl_status, ioreport_status) } -/// The selection behind [`detect`], given where the IOReport status comes -/// from (asked only when the ladder reaches the IOReport rung). -#[cfg_attr(not(target_os = "macos"), allow(unused_variables))] -pub(crate) fn detect_with(ioreport: impl FnOnce() -> IoReportStatus) -> Box { +/// The selection behind [`detect`], given where each rung's status comes from +/// (each asked only when the ladder reaches its rung, and only on a platform +/// and build that has the rung). +// Each status is asked for only where its rung exists, so on any one platform +// and build some of them go unused. +#[allow(unused_variables)] +pub(crate) fn detect_with( + nvml: impl FnOnce() -> CounterStatus, + rapl: impl FnOnce() -> CounterStatus, + ioreport: impl FnOnce() -> IoReportStatus, +) -> Box { #[cfg(feature = "cuda")] - if let Ok(c) = NvmlCounter::try_detect() { + if let Ok(c) = NvmlCounter::try_detect_given(nvml()) { return Box::new(c); } #[cfg(target_os = "linux")] - if let Ok(c) = LinuxRaplCounter::try_detect() { + if let Ok(c) = LinuxRaplCounter::try_detect_given(rapl()) { return Box::new(c); } #[cfg(target_os = "macos")] @@ -3433,4 +3962,64 @@ mod tests { fn ioreport_is_unavailable_off_macos() { assert_eq!(ioreport_status(), IoReportStatus::Unavailable); } + + /// RAPL and NVML are selected only when their counter is live, and only + /// where their rung exists: RAPL on Linux, NVML with `cuda` where NVML + /// opens. A frozen or unavailable counter is never selected, whatever + /// the other rungs say. + #[test] + fn a_vendor_counter_is_selected_only_when_live() { + use CounterStatus as S; + let declined_ioreport = || IoReportStatus::CpuRailFrozen; + for status in [S::Unavailable, S::Frozen] { + let picked = detect_with(|| status, || status, declined_ioreport).name(); + assert!(!matches!(picked, "rapl" | "nvml"), "a {status:?} counter was selected: {picked}"); + let picked = detect_with(|| S::Unavailable, || status, declined_ioreport).name(); + assert_ne!(picked, "rapl", "a {status:?} RAPL counter was selected"); + } + let picked = detect_with(|| S::Unavailable, || S::Live, declined_ioreport).name(); + assert_eq!(picked == "rapl", cfg!(target_os = "linux"), "a live RAPL status selected {picked}"); + #[cfg(feature = "cuda")] + { + let opens = nvml::NvmlCounter::try_detect_given(S::Live).is_ok(); + let picked = detect_with(|| S::Live, || S::Unavailable, declined_ioreport).name(); + assert_eq!(picked == "nvml", opens, "a live NVML status selected {picked}"); + } + } + + /// Off Linux there is no RAPL rung to probe; without `cuda`, no NVML one. + #[test] + fn a_vendor_counter_with_no_rung_is_unavailable() { + #[cfg(not(target_os = "linux"))] + assert_eq!(rapl_status(), CounterStatus::Unavailable); + #[cfg(not(feature = "cuda"))] + assert_eq!(nvml_status(), CounterStatus::Unavailable); + } + + /// Against this host's RAPL counter, where there is one: the status the + /// probe cached agrees with what the counter does now. Live: it advances + /// across 200 ms of CPU work, and `detect()` selects it unless NVML is + /// live. Not live: `detect()` never selects it. Logs what it saw. + #[cfg(target_os = "linux")] + #[test] + fn the_rapl_rung_agrees_with_the_counter() { + let status = rapl_status(); + let path = std::path::Path::new(linux_rapl::RAPL_ENERGY_PATH); + let before = linux_rapl::read_energy_uj(path); + let started = std::time::Instant::now(); + let mut acc: u64 = 0; + while started.elapsed() < std::time::Duration::from_millis(200) { + acc = std::hint::black_box(acc.wrapping_add(1)); + } + let after = linux_rapl::read_energy_uj(path); + let picked = detect().name(); + eprintln!("rapl: status={status:?} before={before:?} after={after:?} detect()={picked}"); + if status == CounterStatus::Live { + let (b, a) = (before.expect("a live counter reads"), after.expect("a live counter reads")); + assert!(a != b, "a live RAPL counter did not move across 200 ms of CPU work: {b} -> {a}"); + assert!(picked == "rapl" || nvml_status() == CounterStatus::Live, "detect() picked {picked}"); + } else { + assert_ne!(picked, "rapl", "detect() selected a {status:?} RAPL counter"); + } + } } -- 2.54.0 (Apple Git-157) From b2e8464c793745959f563d2f9689ad439060801a Mon Sep 17 00:00:00 2001 From: Transaction Science Date: Wed, 30 Sep 2026 21:46:07 -0400 Subject: [PATCH 16/22] eoc-meter: IOReport is live only when every CPU rail it sums keeps advancing, each die's on its own The IOReport liveness rule said Live once ANY CPU rail sustained its advance (five intervals, 80% of them, 300 ms), but the counter it admits SUMS every CPU rail. On a two-die part (an M3 Ultra's `DIE_0_CPU Energy` and `DIE_1_CPU Energy`), one live die beside a frozen one would have selected a counter whose figure leaves the frozen die out, or carries its refreshes, while claiming the whole CPU, tagged Measured. RAPL and NVML were never exposed to this: each hands its one counter as the only channel, so "any" and "every" agree there. `RailWatch::observe` now says Live the first time EVERY rail seen in an interval passes `sustained`, and at least one does (`all` of no rails is vacuously true, and a sample with no CPU rail is Absent, never Live). Each rail is still counted on its own; a rail that joins late is held to every interval observed since the first, so it is live only once its own advances reach 80% of all of them. The rule stays one rule: for a single vendor counter, every rail is its one channel. `CpuRail`, `IoReportStatus` and the crate docs say so. The hardware oracle is held to the same property. `oracle_moved` required one CPU rail to change at 80% of its reads; it now requires every CPU rail it found to, so on a multi-die host the oracle can no longer vouch for a Live verdict that one die alone earned. Tests, all pure: - `m_every_die_must_sustain_its_own_advance`: two dies advancing at every 31 ms interval are Live at the tenth (310 ms), exactly where one die is (d); DIE_0 live with DIE_1 holding, stepping once, or stepping twice in a refresh is Frozen over the whole budget and over 1,000 samples; DIE_1 at 60% is Frozen; DIE_1 frozen for 20 intervals then live is Live at 80 of 100, never at DIE_0's tenth, and Frozen within one probe budget. - `probe_with_one_die_live_and_one_frozen_is_frozen`: the same through `probe_with`, both dies' own values handed to the rule; both live is Live at 12 intervals, as one die is. - `host_status_is_the_rule_on_the_production_composition` gains the mixed case: `CpuRailFrozen` through the real opener closure, tick and clocks. - (c) changes meaning: of two same-named `CPU Energy` rails, one live and one held was Live and is now Frozen; both live is Live. - The oracle test now pins `[0, 11]` and `[11, 2]` as not moving and `[11, 9]` as moving, and an empty set as not moving. - Single-die behaviour is unchanged: (a)-(l), the RAPL and NVML tests and the composition's single-rail cases pass untouched. Falsified, each mutant applied to the uncommitted tree, run, and restored byte-for-byte (sha256 checked): - `any` in place of `all` (HEAD's rule) fails 4: (m), with "DIE_0 live and DIE_1 frozen (holds) counted as live: the counter would leave DIE_1 out ... left: Some(Live) right: Some(Frozen)", plus (c), the probe_with mixed test and the production-composition test; - `all` without the non-empty guard fails 2 (`no_cpu_rail_is_absent`, `a_rail_new_in_a_sample_only_sets_its_baseline`); - the oracle back on `any` fails the oracle test. On hardware, macOS 27.0.1: the M5 Max (one `CPU Energy` rail) and the M3 Ultra (`DIE_0_`/`DIE_1_CPU Energy`) both report `status=CpuRailFrozen oracle=CpuRailMoved(false) selected=false`; the M3 Ultra's oracle saw changes per CPU rail [0, 0] over 19 compared pairs, the GPU rail [19]. eoc-meter's lib tests pass 59/59 on each. Not done: on a part whose idle die's rail is live but does not advance while the probe burns one thread elsewhere, every-die declines the rung. That fails safe (estimated, never over-claimed) and is not observable on these hosts, where every CPU rail is frozen to an unprivileged reader. Co-Authored-By: Claude Opus 5.5 --- eoc/eoc-rs/crates/eoc-meter/src/lib.rs | 248 +++++++++++++++++++------ 1 file changed, 194 insertions(+), 54 deletions(-) diff --git a/eoc/eoc-rs/crates/eoc-meter/src/lib.rs b/eoc/eoc-rs/crates/eoc-meter/src/lib.rs index b78c6dfc9..c247f2937 100644 --- a/eoc/eoc-rs/crates/eoc-meter/src/lib.rs +++ b/eoc/eoc-rs/crates/eoc-meter/src/lib.rs @@ -28,15 +28,17 @@ //! **only when that counter reads and is live** ([`rapl_status`]; the //! probe burns CPU on the calling thread). No external crate required. //! - Apple Silicon IOReport Energy Model counters are read through Apple's -//! IOReport C API (no sudo, no external binary) — **only when the CPU -//! energy rail is live** ([`ioreport_status`]). A subscription can open -//! while every CPU rail holds its value, refreshed only now and then; the -//! summed rails then advance through the GPU rail, a stale refresh, or not -//! at all, and would pass that off as a reading of CPU work. When the CPU +//! IOReport C API (no sudo, no external binary) — **only when every CPU +//! energy rail is live**, each die's on a multi-die part +//! ([`ioreport_status`]). A subscription can open while every CPU rail +//! holds its value, refreshed only now and then; the summed rails then +//! advance through the GPU rail, a stale refresh, or not at all, and would +//! pass that off as a reading of CPU work. The counter sums every die's CPU +//! rail, so one die live beside one frozen is not live either. When a CPU //! rail is not live the counter is declined and [`detect`] falls through to //! the next rung. A live IOReport counter is a vendor energy counter, as //! RAPL and NVML are, so a delta it reads is `Measured` — but only while the -//! rail is live: the counter exists only on a host whose rail the probe +//! rails are live: the counter exists only on a host whose rails the probe //! found live, and on any other there is no IOReport reading to tag. //! - macOS `powermetrics`: detected when the binary is on the PATH, but the //! reference wrapper does not parse it, so every read returns @@ -92,7 +94,9 @@ impl JouleCounter for StubCounter { // lists its channels as (name, absolute raw integer); everything that decides // lives here. One rule for every vendor counter: IOReport watches the CPU energy // rails of an Energy Model sample; RAPL and NVML hand their one counter as the -// only channel. Compiled on every platform, so a Linux CI leg runs its tests. +// only channel. Every rail watched must keep advancing, each on its own, because +// a counter's reading sums every rail it watches: IOReport's adds every die's +// `CPU Energy`. Compiled on every platform, so a Linux CI leg runs its tests. #[cfg_attr(not(target_os = "macos"), allow(dead_code))] mod rail_liveness { use std::time::Duration; @@ -140,15 +144,16 @@ mod rail_liveness { && window >= MIN_LIVE_WINDOW } - /// What the liveness probe saw of the CPU energy rail. + /// What the liveness probe saw of the CPU energy rails. #[derive(Debug, Clone, Copy, PartialEq, Eq)] pub(crate) enum CpuRail { - /// The same CPU energy rail kept advancing, by [`sustained`]: in at - /// least [`MIN_LIVE_ADVANCES`] separate intervals, at least 80% of - /// those observed, over at least [`MIN_LIVE_WINDOW`]. + /// Every CPU energy rail in the samples kept advancing, each by + /// [`sustained`]: in at least [`MIN_LIVE_ADVANCES`] separate intervals, + /// at least 80% of those observed, over at least [`MIN_LIVE_WINDOW`]. Live, - /// A CPU energy rail exists, but none kept advancing: it held, or - /// stepped only in a burst (a batch refresh, in one step or several). + /// A CPU energy rail exists, but not every one kept advancing: one + /// held, or stepped only in a burst (a batch refresh, in one step or + /// several), though another may have been live. Frozen, /// Samples were taken, and no `CPU Energy` channel was in them. Absent, @@ -215,8 +220,18 @@ mod rail_liveness { /// summed total is never consulted, because the GPU rail, which is live on /// an M5 Max, is exactly what corrupts it. After each interval every rail /// is put to [`sustained`], over the intervals observed so far and the wall - /// time since the first usable sample; the first time one passes, the watch - /// says Live, and stays Live. + /// time since the first usable sample; the first time EVERY rail seen in + /// an interval passes, the watch says Live, and stays Live. + /// + /// Every rail, not any one: the counter a Live verdict admits reads the + /// sum of every rail the watch picks. On a two-die part (an M3 Ultra's + /// `DIE_0_CPU Energy` and `DIE_1_CPU Energy`), one die live and the other + /// frozen would give a figure that leaves out the frozen die's energy, + /// or carries its refreshes, while claiming the whole CPU. A rail that + /// joins late is held to the intervals observed since the first, so it + /// is live only once its own advances are 80% of all of them. A single + /// vendor counter's one channel is every rail it has, so for RAPL and NVML + /// this is the rule it always was. /// /// Which channels of a sample are rails is the watch's `pick`: the CPU /// energy aggregates for IOReport ([`RailWatch::default`]), every channel @@ -236,7 +251,7 @@ mod rail_liveness { /// Every CPU rail seen in an interval, with the number of intervals in /// which it advanced. pub(crate) rails: Vec<(RailKey, u32)>, - /// Whether one rail has passed [`sustained`] after some interval. + /// Whether every rail has passed [`sustained`] after the same interval. live: bool, } @@ -282,18 +297,20 @@ mod rail_liveness { } } let (intervals, window) = (self.intervals, self.window); - self.live |= self.rails.iter().any(|(_, n)| sustained(*n, intervals, window)); + // Every rail, and at least one: `all` of no rails is vacuously + // true, and a sample with no rail in it is Absent, never Live. + self.live |= !self.rails.is_empty() && self.rails.iter().all(|(_, n)| sustained(*n, intervals, window)); } self.prev = Some(cur); } - /// Has one CPU rail kept advancing, by [`sustained`], after some + /// Has every rail kept advancing, each by [`sustained`], after some /// interval observed so far? pub(crate) fn is_live(&self) -> bool { self.live } - /// Keep sampling? Until the rail proves live; or until `budget` has + /// Keep sampling? Until the rails prove live; or until `budget` has /// elapsed **and** at least [`MIN_LIVE_ADVANCES`] intervals were /// observed, enough for a live rail to have shown its advances (slow or /// failed samples must not make a live rail look frozen); and never @@ -663,7 +680,8 @@ mod rail_liveness { /// (c) Two channels with the same name are two rails: stepping together /// once is one advance each — in the same interval or in different ones — - /// never two advances of one rail. And either of them can prove live. + /// never two advances of one rail. Both are summed by the counter, so + /// both must prove live: one live beside one that holds is Frozen. #[test] fn c_two_same_named_cpu_rails_are_counted_apart() { let (w, _) = scripted(TICK, |i| { @@ -678,12 +696,19 @@ mod rail_liveness { }); assert_eq!(w.verdict(), Some(CpuRail::Frozen), "two same-named rails stepping once each counted as live: {w:?}"); - // The first of two same-named rails is live; the second holds. At - // 25 ms a sample, the 300 ms window is what it waits for: 12 intervals. + // The first of two same-named rails is live; the second holds: + // Frozen, watched for the whole budget. let (w, taken) = scripted(TICK, |i| Some(vec![("CPU Energy", 100 + i as i64), ("CPU Energy", 200)])); + assert_eq!(w.verdict(), Some(CpuRail::Frozen), "one live rail beside one that holds counted as live: {w:?}"); + assert_eq!((w.intervals, taken), (40, 41)); + assert_eq!(w.rails, vec![(key("CPU Energy", 0), 40), (key("CPU Energy", 1), 0)]); + + // Both live. At 25 ms a sample, the 300 ms window is what they wait + // for: 12 intervals. + let (w, taken) = scripted(TICK, |i| Some(vec![("CPU Energy", 100 + i as i64), ("CPU Energy", 200 + 3 * i as i64)])); assert_eq!(w.verdict(), Some(CpuRail::Live), "{w:?}"); assert_eq!((w.intervals, taken), (12, 13)); - assert_eq!(w.rails, vec![(key("CPU Energy", 0), 12), (key("CPU Energy", 1), 0)]); + assert_eq!(w.rails, vec![(key("CPU Energy", 0), 12), (key("CPU Energy", 1), 12)]); } /// (d) A rail advancing in every interval, sampled every 31 ms, is not @@ -843,6 +868,74 @@ mod rail_liveness { assert_eq!((w.verdict(), w.intervals, taken, w.window), (Some(CpuRail::Live), 5, 6, Duration::from_millis(500))); } + /// (m) Every die must sustain its own advance, because the counter + /// sums every die's `CPU Energy`. On an M3 Ultra's two dies, sampled + /// every 31 ms beside a moving GPU rail and unprefixed per-die channels + /// that repeat a name: + /// - both dies advancing at every interval: Live, at the tenth interval + /// (310 ms), exactly where one die alone is (d); + /// - `DIE_0` advancing at every interval and `DIE_1` frozen — holding, + /// stepping once, or stepping twice in a refresh: Frozen, however + /// long it is watched. A rule that took any one live rail called each + /// of these Live at the tenth interval, and the counter it admitted + /// would leave `DIE_1` out of a figure claimed for the whole CPU; + /// - `DIE_0` live and `DIE_1` advancing in 60% of intervals: Frozen; + /// - `DIE_1` frozen for its first twenty intervals, then live: Live + /// only once its own advances reach 80% of every interval observed + /// (at the hundredth, 80 of 100), never at `DIE_0`'s tenth. + #[test] + fn m_every_die_must_sustain_its_own_advance() { + let dies = |die0: i64, die1: i64, i: usize| { + vec![ + ("DIE_0_CPU Energy", die0), + ("ECPUDTL00", 10 * i as i64), + ("DIE_1_CPU Energy", die1), + ("ECPUDTL00", 11 * i as i64), + ("GPU Energy", 3 * i as i64), + ] + }; + let live0 = |i: usize| 9_494_336 + 1_307 * i as i64; + + let both = |i: usize| dies(live0(i), 10_212_656 + 1_211 * i as i64, i); + let (at, w) = first_live(INTERVAL, 40, both); + assert_eq!(at, Some(10), "two live dies must be Live at the 10th interval (310 ms), as one is: {w:?}"); + let (w, taken) = scripted(INTERVAL, |i| Some(both(i))); + assert_eq!((w.verdict(), w.intervals, taken), (Some(CpuRail::Live), 10, 11)); + assert_eq!(w.rails, vec![(key("DIE_0_CPU Energy", 0), 10), (key("DIE_1_CPU Energy", 0), 10)]); + + // DIE_1 frozen: what it does, its value at sample i, its advances. + let frozen = [ + ("holds", (|_| 10_212_656) as fn(usize) -> i64, 0), + ("steps once", |i| 10_212_656 + i64::from(i >= 4) * 1_391_637, 1), + ("steps twice in a refresh", |i| 10_212_656 + i64::from(i >= 4) * 1_391_637 + i64::from(i >= 5) * 2_113, 2), + ]; + for (what, die1, steps) in frozen { + let mixed = |i: usize| dies(live0(i), die1(i), i); + let (w, taken) = scripted(INTERVAL, |i| Some(mixed(i))); + assert_eq!( + w.verdict(), + Some(CpuRail::Frozen), + "DIE_0 live and DIE_1 frozen ({what}) counted as live: the counter would leave DIE_1 out: {w:?}" + ); + assert_eq!(w.rails, vec![(key("DIE_0_CPU Energy", 0), 33), (key("DIE_1_CPU Energy", 0), steps)], "{what}"); + assert_eq!((w.intervals, taken), (33, 34), "{what}: the budget is 33 intervals of 31 ms"); + assert_eq!(first_live(INTERVAL, 1000, mixed).0, None, "DIE_1 {what}"); + } + + let sixty = |i: usize| dies(live0(i), 10_212_656 + (1..=i).filter(|k| k % 5 != 0 && k % 5 != 2).count() as i64, i); + let (w, _) = scripted(INTERVAL, |i| Some(sixty(i))); + assert_eq!(w.verdict(), Some(CpuRail::Frozen), "DIE_1 advancing in 60% of intervals counted as live: {w:?}"); + assert_eq!(first_live(INTERVAL, 1000, sixty).0, None); + + let late = |i: usize| dies(live0(i), 10_212_656 + i.saturating_sub(20) as i64, i); + let (at, w) = first_live(INTERVAL, 200, late); + assert_eq!(at, Some(100), "DIE_1 live from the 21st interval must be Live at 80 of 100: {w:?}"); + assert_eq!(w.rails, vec![(key("DIE_0_CPU Energy", 0), 199), (key("DIE_1_CPU Energy", 0), 179)]); + let (w, _) = scripted(INTERVAL, |i| Some(late(i))); + assert_eq!(w.verdict(), Some(CpuRail::Frozen), "DIE_1 live for only 13 of 33 intervals counted as live: {w:?}"); + assert_eq!(w.rails, vec![(key("DIE_0_CPU Energy", 0), 33), (key("DIE_1_CPU Energy", 0), 13)]); + } + /// (e) Samples that cannot be taken keep the probe sampling to its cap of /// five budgets, and end with no verdict: Unavailable. #[test] @@ -1131,6 +1224,33 @@ mod rail_liveness { } } + /// One die live, one frozen, through the whole composition: Frozen, + /// after the whole budget, with both dies' own values handed to the + /// rule. Both live: Live, where one die alone is. + #[test] + fn probe_with_one_die_live_and_one_frozen_is_frozen() { + let die = |i: usize, die1_live: bool| { + [ + ("DIE_0_CPU Energy", 9_494_336 + 1_307 * i as i64), + ("DIE_1_CPU Energy", 10_212_656 + if die1_live { 1_211 * i as i64 } else { 0 }), + ] + }; + let f = fake(true, TICK, |i| Some(group_sample(&die(i, false)))); + assert_eq!(f.run.verdict(), Some(CpuRail::Frozen), "one die live, one frozen, counted as live: {:?}", f.run.watch); + assert_eq!(status_of(f.run.verdict()), crate::IoReportStatus::CpuRailFrozen); + assert_eq!(f.run.watch.rails, vec![(key("DIE_0_CPU Energy", 0), 40), (key("DIE_1_CPU Energy", 0), 0)]); + assert_eq!((f.samples, f.ticks), (41, 40)); + for (i, call) in f.run.calls.iter().enumerate() { + let [(n0, v0), (n1, v1)] = die(i, false); + assert_eq!(call.handed, Some(vec![(key(n0, 0), v0), (key(n1, 0), v1)]), "sample {i}"); + } + + let f = fake(true, TICK, |i| Some(group_sample(&die(i, true)))); + assert_eq!(f.run.verdict(), Some(CpuRail::Live), "{:?}", f.run.watch); + assert_eq!(status_of(f.run.verdict()), crate::IoReportStatus::Live); + assert_eq!((f.samples, f.ticks), (13, 12)); + } + /// A clock pair for [`spin_for_cpu`] that only the spin moves: each read /// of the CPU clock advances it by `cpu_step`, and each read of the wall /// clock advances it by `wall_step`. The spin reads each once per turn, @@ -1390,11 +1510,12 @@ mod clocks { // intermittently on the M3 Ultra). Summed, the rails report GPU-only energy, // zero, or a refresh of past CPU energy, and consumers such as eoc-cascade, // eoc-local and eoc-streaming tag what a hardware counter returns as Measured. -// So the counter is constructed only when `ioreport_status()` is `Live` — the -// same CPU rail kept advancing while the probing thread burned CPU: in at least -// five separate sample intervals, at least 80% of those observed, over at least -// 300 ms — and otherwise `try_detect()` returns `Err(NoCounter)`, leaving the -// caller's own estimate. +// So the counter is constructed only when `ioreport_status()` is `Live` — every +// CPU rail in the samples (each die's, on a multi-die part) kept advancing while +// the probing thread burned CPU: each in at least five separate sample +// intervals, at least 80% of those observed, over at least 300 ms — and +// otherwise `try_detect()` returns `Err(NoCounter)`, leaving the caller's own +// estimate. Every rail, because the counter sums them all. #[cfg(target_os = "macos")] #[allow(non_camel_case_types, unsafe_code)] mod macos_ioreport { @@ -1490,7 +1611,7 @@ mod macos_ioreport { /// Apple Silicon IOReport energy counter (SoC-total). /// - /// Constructed only on a host whose CPU energy rail is live; see + /// Constructed only on a host whose every CPU energy rail is live; see /// [`crate::ioreport_status`]. A live IOReport reading is a vendor /// energy counter's reading, so the delta it gives is `Measured`, as /// RAPL's and NVML's are; that holds only while the rail is live, which @@ -2261,14 +2382,18 @@ mod macos_ioreport { /// However slow its reads, the oracle stops after this many windows. const ORACLE_WINDOW_CAP: u32 = 5; - /// The oracle's bar for a moving rail, over `compared` pairs of - /// consecutive reads: one rail changed at least [`ORACLE_MIN_CHANGES`] - /// times, at no fewer than 80% of them. A live rail changes at nearly - /// every read. A refresh — one step, or two 64 ms apart, or a few in a - /// burst — changes at a handful of reads in a window of 0.1-0.8 s reads, - /// so it cannot vouch for a Live verdict it happened to coincide with. + /// The oracle's bar for moving CPU rails, over `compared` pairs of + /// consecutive reads: EVERY CPU rail it found (each die's, on a + /// multi-die part) changed at least [`ORACLE_MIN_CHANGES`] times, at no + /// fewer than 80% of them. A live rail changes at nearly every read. A + /// refresh — one step, or two 64 ms apart, or a few in a burst — + /// changes at a handful of reads in a window of 0.1-0.8 s reads, so it + /// cannot vouch for a Live verdict it happened to coincide with. Every + /// rail, because the counter sums every die: one die moving beside one + /// that holds cannot vouch for a Live verdict either. fn oracle_moved(cpu_changes: &[u32], compared: u32) -> bool { - cpu_changes.iter().any(|&n| n >= ORACLE_MIN_CHANGES && u64::from(n) * 5 >= u64::from(compared) * 4) + !cpu_changes.is_empty() + && cpu_changes.iter().all(|&n| n >= ORACLE_MIN_CHANGES && u64::from(n) * 5 >= u64::from(compared) * 4) } /// What the oracle has counted, read by read: per rail, at how many @@ -2351,9 +2476,9 @@ mod macos_ioreport { /// names with `ends_with("CPU Energy")` in any group (not the /// implementation's group filter and die-prefix stripping), and counts /// per rail at how many reads the value changed since the read before - /// ([`OracleTally`]). `CpuRailMoved(true)` only by [`oracle_moved`]: one - /// CPU rail changed at 80% or more of the compared reads, and at least - /// five times. It reads until the window has passed AND it has compared + /// ([`OracleTally`]). `CpuRailMoved(true)` only by [`oracle_moved`]: + /// every CPU rail changed at 80% or more of the compared reads, and at + /// least five times. It reads until the window has passed AND it has compared /// [`ORACLE_MIN_CHANGES`] pairs of reads, for at most /// [`ORACLE_WINDOW_CAP`] windows, so slow reads on a loaded host /// lengthen the window rather than thin the evidence. @@ -2448,7 +2573,10 @@ mod macos_ioreport { assert_eq!(tally(21, &four_in_five).verdict(), Oracle::CpuRailMoved(true), "16 changes in 20"); let three_in_four = |i: usize| a + (1..=i).filter(|k| k % 4 != 0).count() as i64; assert_eq!(tally(21, &three_in_four).verdict(), Oracle::CpuRailMoved(false), "15 changes in 20"); - assert!(!oracle_moved(&[2], 11) && oracle_moved(&[0, 11], 11) && !oracle_moved(&[3, 3], 5)); + assert!(!oracle_moved(&[2], 11) && !oracle_moved(&[3, 3], 5) && !oracle_moved(&[], 11)); + // Two dies: both must have moved. + assert!(oracle_moved(&[11, 9], 11), "two moving dies"); + assert!(!oracle_moved(&[0, 11], 11) && !oracle_moved(&[11, 2], 11), "one die moving beside one that holds"); } // ── The production status, on scripted IOReport leaves ─────────────── @@ -2518,7 +2646,8 @@ mod macos_ioreport { /// refresh published in two steps is `CpuRailFrozen`; a rail advancing /// at every sample is `Live`, once five intervals and 300 ms of the /// real wall clock have passed and no later; one step on each of two - /// dies, in different intervals, is `CpuRailFrozen`. These hosts + /// dies, in different intervals, is `CpuRailFrozen`, and so is one die + /// advancing at every sample beside one that holds. These hosts /// produce none of the first four on demand, so no hardware test can /// reach them. Every scripted step falls in the first five intervals, /// which the probe always observes however loaded the host: it takes @@ -2575,6 +2704,12 @@ mod macos_ioreport { }; let (status, served, _) = scripted_host_status(true, move |i| Some(group_sample(&dies(i)))); assert_eq!(status, IoReportStatus::CpuRailFrozen, "one step on each die ({served} samples) must be frozen"); + + // DIE_0 advancing at every sample, DIE_1 held: the counter would + // sum a live die and a frozen one, so the whole is frozen. + let one_live_die = |i: usize| [("DIE_0_CPU Energy", 9_494_336 + 1_307 * i as i64), ("DIE_1_CPU Energy", 10_212_656)]; + let (status, served, _) = scripted_host_status(true, move |i| Some(group_sample(&one_live_die(i)))); + assert_eq!(status, IoReportStatus::CpuRailFrozen, "one live die beside a frozen one ({served} samples) must be frozen"); } /// Each production tick, in the production composition, burns 25 ms of @@ -2805,10 +2940,10 @@ mod macos_ioreport { // ── Against the hardware ───────────────────────────────────────────── /// The property that matters: the IOReport counter is **never** - /// constructed, and never picked by [`crate::detect`], when its CPU rail - /// is not live — then it would report GPU-only energy, zero, or a batch - /// refresh as the energy of CPU work, and consumers would tag it - /// Measured. Asserts the meter's decision against what the rail actually + /// constructed, and never picked by [`crate::detect`], when a CPU rail + /// it sums is not live — then it would report GPU-only energy, zero, a + /// batch refresh, or one die's energy as the energy of CPU work, and + /// consumers would tag it Measured. Asserts the meter's decision against what the rail actually /// did; every branch asserts, so there is no way to pass without /// checking anything, and an oracle with too few reads is a failure. /// @@ -2818,7 +2953,8 @@ mod macos_ioreport { /// never over-claimed. A batch refresh in both windows does not rescue /// a probe that called it live, even one published in two or more /// steps: the oracle counts a rail as moving only if it changed at 80% - /// or more of its reads, at least five times. + /// or more of its reads, at least five times, and the CPU as moving only + /// if every CPU rail it found did. #[test] fn the_ioreport_counter_is_never_selected_on_a_frozen_cpu_rail() { let status = crate::ioreport_status(); @@ -2866,7 +3002,7 @@ mod macos_ioreport { // not, declining is the safe side. assert!( matches!(status, IoReportStatus::Live | IoReportStatus::CpuRailFrozen), - "the oracle saw a CPU rail keep moving but the status is {status:?}" + "the oracle saw every CPU rail keep moving but the status is {status:?}" ); assert_eq!(selected, status == IoReportStatus::Live, "status={status:?}"); } @@ -2929,7 +3065,7 @@ mod macos_ioreport { ), Oracle::CpuRailMoved(true) => assert!( matches!(probed, Some(Some(CpuRail::Live | CpuRail::Frozen))), - "the oracle saw a CPU rail keep moving but the probe reported {probed:?}" + "the oracle saw every CPU rail keep moving but the probe reported {probed:?}" ), } } @@ -3397,17 +3533,21 @@ pub enum IoReportStatus { /// IOReport's Energy Model group was sampled, and no `CPU Energy` rail was /// in it — an unfamiliar channel naming. The counter is not used. CpuRailAbsent, - /// The CPU energy rail exists but did not keep advancing while CPU was + /// A CPU energy rail exists but did not keep advancing while CPU was /// burned: it held, or stepped only in a burst (a refresh, which macOS 27 - /// can publish in two or more steps a few tens of milliseconds apart). + /// can publish in two or more steps a few tens of milliseconds apart). On + /// a multi-die part, one die's frozen rail is enough, even beside a live + /// one: the counter sums every die, and would claim the frozen die's + /// energy as measured. /// The platform is not exposing live CPU energy to this process, so using /// the counter would report GPU-only energy, zero, or a stale refresh as /// the energy of CPU work. Observed on macOS 27.0.1 on both an M5 Max and /// an M3 Ultra. CpuRailFrozen, - /// The same CPU energy rail kept advancing under load — in at least five - /// separate sample intervals, at least 80% of those observed, over at least - /// 300 ms of wall time; the counter is available. + /// Every CPU energy rail (each die's, on a multi-die part) kept advancing + /// under load — each in at least five separate sample intervals, at least + /// 80% of those observed, over at least 300 ms of wall time; the counter is + /// available. Live, } @@ -3899,7 +4039,7 @@ pub use nvml::NvmlCounter; /// Preference order: NVML (if compiled with `cuda`, present, and its counter /// live — see [`nvml_status`]) → RAPL on Linux (only when its package counter /// is live — see [`rapl_status`]) → IOReport on macOS (Energy-Model counters, -/// no sudo; only when its CPU energy rail is live — see [`ioreport_status`]) +/// no sudo; only when every CPU energy rail is live — see [`ioreport_status`]) /// → powermetrics on macOS → `StubCounter`. Every hardware rung is held to /// the same sustained-advance rule; a rung whose counter is not live is /// skipped, so callers estimate rather than seal what it read. -- 2.54.0 (Apple Git-157) From 52d9d1e296005a0b772f093973f50df0d0761687 Mon Sep 17 00:00:00 2001 From: Transaction Science Date: Wed, 30 Sep 2026 21:46:07 -0400 Subject: [PATCH 17/22] eoc-agent: a step is charged what the meter measured only when both reads succeed and the counter advanced, never a since-boot total MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit `AgentLoop::read_meter` took a failed read as 0 (`unwrap_or(0)`), and `measured` charged `after - before`. A failed first read followed by a good one therefore charged the counter's whole since-boot total to one ReAct tool call or CodeAct execution, which on any real counter exhausts a joule budget at once. This is the defect 15cf07c2 fixed in eoc-local and 467acb38 in eoc-streaming. The same rule now holds here: - `read_meter` returns `Option`: `None` for a failed read, never 0. Its only callers are in this file; nothing else in the monorepo uses it. - `span_reading(start)` is `Some(delta)` only when the opening read succeeded, a read now succeeds, and the counter advanced. Without an opening reading the counter is not read again. - `measured` charges the larger of that delta and the step's own estimate, as before, when there is a delta; otherwise the estimate alone. Tests, on a scripted counter that reads a list (`None` is a failed read) and counts its reads: - a failed read is `None`, the next good one `Some`; - a failed first read then 9 J charges the estimate (250, or 0), with one read; - 1,000 -> 1,450 charges 450, or the estimate when larger; - a failed last read, a held counter, one that went down, and the stub charge the estimate; - through `ReActLoop`, with a 1 J budget and a counter that fails once then reads 9 J, the agent finishes `Done` having charged its two replies (2,000 µJ), and the tool step's span carries 0. Falsified, each mutant applied to the uncommitted tree, run, and restored byte-for-byte (sha256 checked): - the failed first read taken as 0 in `measured` fails 2: the unit test with `left: (9000000, 2) right: (250, 1)`, and the ReAct test, which stopped with `BudgetExhausted("joules")` after the tool step instead of `Done`; - `read_meter` as at HEAD (`Some(unwrap_or(0))`) fails 3. `cargo test -p eoc-agent` passes (25 unit, 9 integration); no new clippy warning (the two in `codeact.rs` and `react.rs`'s parsers predate this). Not done: ReAct and CodeAct pass an estimate of 0 µJ for a tool step, so an unmeasured tool step is charged 0 and its span carries 0, and `Span` has no field to say whether its figure was measured. Giving tool steps an estimate, and spans a provenance, are separate decisions. Co-Authored-By: Claude Opus 5.5 --- eoc/eoc-rs/crates/eoc-agent/src/agent.rs | 129 +++++++++++++++++++++-- eoc/eoc-rs/crates/eoc-agent/src/react.rs | 44 ++++++++ 2 files changed, 162 insertions(+), 11 deletions(-) diff --git a/eoc/eoc-rs/crates/eoc-agent/src/agent.rs b/eoc/eoc-rs/crates/eoc-agent/src/agent.rs index 4fd2c6ba8..28704b8fd 100644 --- a/eoc/eoc-rs/crates/eoc-agent/src/agent.rs +++ b/eoc/eoc-rs/crates/eoc-agent/src/agent.rs @@ -238,25 +238,43 @@ impl AgentLoop { } } - /// Take a meter reading. Returns 0 if the counter is unavailable. - pub fn read_meter(&self) -> u64 { - self.meter.read_microjoules().unwrap_or(0) + /// Take a meter reading: the counter's cumulative micro-joules, or `None` + /// when it could not be read. A failed read is not 0: subtracting a 0 + /// from the next successful read would charge the counter's whole + /// since-boot total to whatever ran in between. + pub fn read_meter(&self) -> Option { + self.meter.read_microjoules().ok() } - /// Wrap an async block, charging measured wall-energy from the meter - /// **plus** an upper-bound `estimated_microjoules` so loops degrade - /// gracefully on stub-only hosts. + /// What the meter measured over a span whose opening reading was + /// `start`: `Some(delta)` only when that read succeeded, a read now + /// succeeds, and the counter advanced between the two. `None` otherwise — + /// a failed read at either end, a counter that held or went down, the + /// [`StubCounter`] — never 0, and never the counter's since-boot total. + /// Without an opening reading the counter is not read again. + pub fn span_reading(&self, start: Option) -> Option { + let start = start?; + let now = self.read_meter()?; + (now > start).then(|| now - start) + } + + /// Run an async block as one step and say what to charge it: what the + /// meter measured over the step ([`Self::span_reading`]), or the larger of + /// that and `estimated_microjoules`, the step's own estimate; and when + /// the meter measured nothing, the estimate alone. The same rule + /// eoc-local's backends and eoc-streaming meter by: a span is measured + /// only when both reads succeed and the counter advanced. pub async fn measured(&self, estimated_microjoules: u64, f: F) -> (T, u64) where F: FnOnce() -> Fut, Fut: std::future::Future, { - let before = self.read_meter(); + let start = self.read_meter(); let out = f().await; - let after = self.read_meter(); - let measured = after.saturating_sub(before); - // Use the larger of measured vs estimated as the charge. - let charged = measured.max(estimated_microjoules); + let charged = match self.span_reading(start) { + Some(measured) => measured.max(estimated_microjoules), + None => estimated_microjoules, + }; (out, charged) } @@ -296,10 +314,99 @@ impl AgentLoop { } } +/// Test support: a counter that reads a script. +#[cfg(test)] +pub(crate) mod scripted { + use std::sync::Mutex; + + use eoc_meter::JouleCounter; + + /// A counter that reads the scripted values in turn, then repeats the + /// last one; `None` is a failed read. Counts its reads. + #[derive(Debug)] + pub(crate) struct Scripted { + script: Vec>, + reads: Mutex, + } + + impl Scripted { + pub(crate) fn new(script: &[Option]) -> Self { + assert!(!script.is_empty(), "a script needs at least one read"); + Self { script: script.to_vec(), reads: Mutex::new(0) } + } + + /// How many times the counter has been read. + pub(crate) fn reads(&self) -> usize { + *self.reads.lock().unwrap() + } + } + + impl JouleCounter for Scripted { + fn read_microjoules(&self) -> eoc_core::Result { + let mut reads = self.reads.lock().unwrap(); + let value = self.script[(*reads).min(self.script.len() - 1)]; + *reads += 1; + value.ok_or(eoc_core::Error::NoCounter) + } + fn name(&self) -> &'static str { + "scripted" + } + } +} + #[cfg(test)] mod tests { + use super::scripted::Scripted; use super::*; + /// What `measured` charges for one step with the estimate `estimate`, + /// metered by a counter reading `script`, and how many reads it took. + async fn charge(script: &[Option], estimate: u64) -> (u64, usize) { + let meter = Arc::new(Scripted::new(script)); + let loop_ = AgentLoop::with_meter(Budget::unlimited(10), meter.clone()); + let (out, charged) = loop_.measured(estimate, || async { 42 }).await; + assert_eq!(out, 42); + (charged, meter.reads()) + } + + /// A failed read is `None`, never 0. + #[test] + fn a_failed_read_is_no_reading() { + let loop_ = AgentLoop::with_meter(Budget::unlimited(1), Arc::new(Scripted::new(&[None, Some(9_000_000)]))); + assert_eq!(loop_.read_meter(), None); + assert_eq!(loop_.read_meter(), Some(9_000_000)); + } + + /// A failed first read is no baseline: the next read is the counter's + /// since-boot total (9 J here), and the step is charged its own estimate, + /// not that. The counter is not read a second time. + #[tokio::test] + async fn a_failed_first_read_never_charges_the_since_boot_total() { + assert_eq!(charge(&[None, Some(9_000_000)], 250).await, (250, 1)); + assert_eq!(charge(&[None, Some(9_000_000)], 0).await, (0, 1)); + } + + /// Both reads succeeded and the counter advanced: the step is charged + /// the delta, or its estimate when that is larger. + #[tokio::test] + async fn a_counter_that_advanced_charges_the_delta() { + assert_eq!(charge(&[Some(1_000), Some(1_450)], 0).await, (450, 2)); + assert_eq!(charge(&[Some(1_000), Some(1_450)], 100).await, (450, 2)); + assert_eq!(charge(&[Some(1_000), Some(1_450)], 1_000).await, (1_000, 2)); + } + + /// A failed last read, a counter that held, one that went down, and the + /// stub measured nothing: the step is charged its estimate. + #[tokio::test] + async fn a_span_that_measured_nothing_charges_the_estimate() { + assert_eq!(charge(&[Some(1_000), None], 250).await, (250, 2)); + assert_eq!(charge(&[Some(7_000), Some(7_000)], 250).await, (250, 2)); + assert_eq!(charge(&[Some(7_000), Some(6_000)], 250).await, (250, 2)); + let stub = AgentLoop::new(Budget::unlimited(1)); + assert_eq!(stub.measured(250, || async {}).await.1, 250); + assert_eq!(stub.span_reading(stub.read_meter()), None); + } + #[tokio::test] async fn budget_charge_llm_trips_token_cap() { let mut b = Budget::new(10, 1.0, 100, 100); diff --git a/eoc/eoc-rs/crates/eoc-agent/src/react.rs b/eoc/eoc-rs/crates/eoc-agent/src/react.rs index 319e3336d..ba82fa342 100644 --- a/eoc/eoc-rs/crates/eoc-agent/src/react.rs +++ b/eoc/eoc-rs/crates/eoc-agent/src/react.rs @@ -225,4 +225,48 @@ mod tests { let err = ReActStep::parse("Thought: nope\n").unwrap_err(); assert!(matches!(err, AgentError::Parse(_))); } + + /// A provider that replies with a script, 1,000 µJ a reply. + struct Replies(std::sync::Mutex>); + + #[async_trait] + impl LlmProvider for Replies { + async fn complete(&self, _prompt: &str) -> AgentResult { + let text = self.0.lock().unwrap().remove(0); + Ok(crate::agent::LlmReply::new(text, 5, 1_000)) + } + } + + struct Echo; + + #[async_trait] + impl crate::agent::Tool for Echo { + fn name(&self) -> &str { + "echo" + } + async fn run(&self, args: serde_json::Value) -> AgentResult { + Ok(args) + } + } + + /// A tool step whose first meter read fails is charged its own estimate + /// (0 µJ for a ReAct tool call), never the counter's since-boot total: + /// with a 1 J budget and a counter that reads 9 J once it reads at all, + /// the agent finishes, having charged only its two replies. + #[tokio::test] + async fn a_failed_first_read_never_exhausts_the_joule_budget() { + let meter = Arc::new(crate::agent::scripted::Scripted::new(&[None, Some(9_000_000)])); + let provider = Arc::new(Replies(std::sync::Mutex::new(vec!["Action: echo[hi]", "Action: Finish[done]"]))); + let mut tools = ToolBox::new(); + tools.insert(Echo); + let budget = Budget::new(1_000, 1.0, 10, 10); + let mut agent = ReActLoop::new(provider, tools, budget, "system"); + agent.inner = AgentLoop::with_meter(budget, meter.clone()); + let (out, stop) = agent.run("go").await.unwrap(); + assert_eq!((out.as_str(), stop), ("done", StopReason::Done)); + assert_eq!(meter.reads(), 1, "the counter was read again after a failed first read"); + assert_eq!(agent.budget().joules.used_microjoules, 2_000, "{:?}", agent.trace().spans); + let act = agent.trace().spans.iter().find(|s| s.kind == SpanKind::Act).expect("an act span"); + assert_eq!(act.microjoules, 0); + } } -- 2.54.0 (Apple Git-157) From 1535b8eb78e63c2e070edd0ef1f92dc6a789660f Mon Sep 17 00:00:00 2001 From: Transaction Science Date: Wed, 30 Sep 2026 21:46:07 -0400 Subject: [PATCH 18/22] =?UTF-8?q?eoc-local:=20a=20llamacpp=20generation=20?= =?UTF-8?q?that=20errors=20costs=20the=20per-token=20estimate=20over=20the?= =?UTF-8?q?=20tokens=20it=20streamed,=20never=200=20=C2=B5J?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit 15cf07c2 left one path at 0 µJ: when the stream carried an error, `infer` returned the error text with `JouleCost { microjoules: 0, source: Estimated }`. The prompt may have been processed and tokens generated before the error, so that is a zero presented as a figure for inference that may have run. The drain is now a function of the stream, `drain(rx, id, joules_per_token)`, which `infer` calls. On an error it returns the same error text, costed at `span::per_token_estimate` over the tokens streamed before the error — at least one, as every other unmeasured path — tagged estimated. A reading an earlier chunk carried covers only part of the generation, so it is not taken as the cost, as on the normal path. Test `an_error_costs_the_per_token_estimate_never_zero`, on an injected channel: three tokens then an error cost 150,000 µJ estimated at 0.05 J/token, with or without a reading on the earlier chunks, and with the payload "[llamacpp-error: backend error (llamacpp): decode failed]"; an error before any token costs one token, 50,000 µJ. Falsified, each mutant applied to the uncommitted tree, run, and restored byte-for-byte (sha256 checked): HEAD's 0 µJ fails it ("left: JouleCost { microjoules: 0, .. } right: 150000"); an estimate that ignores the tokens streamed fails it (50,000 for 150,000). `cargo test -p eoc-local` passes (37 + 1), and with `--features mlc,gguf` (48 + 3). No clippy warning in this crate. Co-Authored-By: Claude Opus 5.5 --- eoc/eoc-rs/crates/eoc-local/src/lib.rs | 3 +- eoc/eoc-rs/crates/eoc-local/src/llamacpp.rs | 83 +++++++++++++++------ 2 files changed, 61 insertions(+), 25 deletions(-) diff --git a/eoc/eoc-rs/crates/eoc-local/src/lib.rs b/eoc/eoc-rs/crates/eoc-local/src/lib.rs index 308fe23ca..0fedc4fc6 100644 --- a/eoc/eoc-rs/crates/eoc-local/src/lib.rs +++ b/eoc/eoc-rs/crates/eoc-local/src/lib.rs @@ -44,7 +44,8 @@ //! //! `llamacpp` also reads the counter with each streamed token, so a chunk //! carries the reading so far when there is one; the inference's cost is -//! the final read's. +//! the final read's. A generation that ends in an error is estimated over +//! the tokens streamed before it, at least one. //! //! ## WASM //! diff --git a/eoc/eoc-rs/crates/eoc-local/src/llamacpp.rs b/eoc/eoc-rs/crates/eoc-local/src/llamacpp.rs index 5958a36e0..731e09f0e 100644 --- a/eoc/eoc-rs/crates/eoc-local/src/llamacpp.rs +++ b/eoc/eoc-rs/crates/eoc-local/src/llamacpp.rs @@ -18,7 +18,9 @@ //! the counter read at the start, read at the end, and advanced between //! them. Otherwise nothing was measured, and the cost is a per-token //! estimate ([`LlamaCppBackend::fallback_joules_per_token`] × tokens -//! streamed), tagged estimated, never 0 µJ. +//! streamed), tagged estimated, never 0 µJ. A generation that ends in an +//! error is estimated the same way, over the tokens streamed before it (at +//! least one): inference may have run, so its cost is never 0 µJ either. //! //! Streaming: tokens are emitted through a `tokio::sync::mpsc` channel //! for incremental UIs. The non-streaming `infer` is implemented as @@ -35,7 +37,7 @@ use std::path::PathBuf; use std::sync::Arc; use async_trait::async_trait; -use eoc_core::{JouleCost, JouleSource, Query, Response, Stage}; +use eoc_core::{JouleCost, Query, QueryId, Response, Stage}; use eoc_meter::JouleCounter; use eoc_neural::NeuralBackend; use tokio::sync::mpsc; @@ -173,31 +175,36 @@ pub struct TokenChunk { #[async_trait] impl NeuralBackend for LlamaCppBackend { async fn infer(&self, q: &Query) -> Response { - // Drain the stream into a single response. - let mut rx = self.stream(q).await; - let mut text = String::new(); - let mut tally = StreamTally::default(); - while let Some(item) = rx.recv().await { - match item { - Ok(chunk) => { - text.push_str(&chunk.text); - tally.observe(&chunk); - } - Err(e) => { - return Response::new( - q.id, - format!("[llamacpp-error: {e}]"), - Stage::Neural, - JouleCost { - microjoules: 0, - source: JouleSource::Estimated, - }, - ); - } + drain(self.stream(q).await, q.id, self.fallback_joules_per_token).await + } +} + +/// Drain a generation's stream into the response for query `id`. The cost +/// of a stream that ends is [`StreamTally::cost`]. A stream that errors ends +/// there, with the error text; its inference may have run — the prompt +/// processed and every token streamed before the error generated — so its +/// cost is the per-token estimate over those tokens, at least one, tagged +/// estimated: never 0 µJ, and never a reading that covers only part of it. +async fn drain(mut rx: mpsc::Receiver>, id: QueryId, joules_per_token: f64) -> Response { + let mut text = String::new(); + let mut tally = StreamTally::default(); + while let Some(item) = rx.recv().await { + match item { + Ok(chunk) => { + text.push_str(&chunk.text); + tally.observe(&chunk); + } + Err(e) => { + return Response::new( + id, + format!("[llamacpp-error: {e}]"), + Stage::Neural, + JouleCost::estimated(span::per_token_estimate(tally.tokens, joules_per_token)), + ); } } - Response::new(q.id, text, Stage::Neural, tally.cost(self.fallback_joules_per_token)) } + Response::new(id, text, Stage::Neural, tally.cost(joules_per_token)) } /// What a drained stream says about its cost: how many tokens it carried, @@ -404,6 +411,34 @@ mod tests { assert_eq!(cost(Arc::new(Scripted::new(&script))).await, JouleCost::estimated(FOUR_TOKENS_UJ)); } + /// A stream that ends in an error, drained as `infer` drains one: `sent` + /// token chunks, each carrying `reading`, then the error. + async fn drain_with_error(sent: usize, reading: Option) -> Response { + let (tx, rx) = mpsc::channel(64); + for i in 0..sent { + let chunk = TokenChunk { text: format!("t{i} "), cumulative_microjoules: reading, final_chunk: false }; + tx.send(Ok(chunk)).await.unwrap(); + } + let error = LocalError::Backend { backend: "llamacpp", message: "decode failed".to_string() }; + tx.send(Err(error)).await.unwrap(); + drop(tx); + drain(rx, Query::new("hello world").id, 0.05).await + } + + /// An error ends the generation, but inference may have run: the cost is + /// the per-token estimate over the tokens streamed before the error, + /// tagged estimated — three tokens, 150,000 µJ at 0.05 J — and one token + /// (50,000 µJ) when the error came first. Never 0 µJ, and never a reading + /// an earlier chunk carried, which covers only part of the work. + #[tokio::test] + async fn an_error_costs_the_per_token_estimate_never_zero() { + let r = drain_with_error(3, None).await; + assert_eq!(r.payload, "[llamacpp-error: backend error (llamacpp): decode failed]"); + assert_eq!((r.stage, r.joule_cost), (Stage::Neural, JouleCost::estimated(150_000))); + assert_eq!(drain_with_error(3, Some(400)).await.joule_cost, JouleCost::estimated(150_000)); + assert_eq!(drain_with_error(0, None).await.joule_cost, JouleCost::estimated(50_000)); + } + /// The tally counts tokens and takes only the final chunk's reading. #[test] fn the_tally_costs_only_the_final_reading() { -- 2.54.0 (Apple Git-157) From 26165a77ecc338a791a143c512e6cf27c545996c Mon Sep 17 00:00:00 2001 From: Transaction Science Date: Wed, 30 Sep 2026 22:11:47 -0400 Subject: [PATCH 19/22] eoc-meter: RAPL is the sum of every package zone, each of which must be live and each wrapped at its own range The RAPL rung read one file, `/sys/class/powercap/intel-rapl:0/energy_uj`, and its liveness probe watched that one file. On a multi-socket server that figure is package 0's energy alone while the counter claims the CPU, so every other socket's energy was missing from every span it measured. And nothing makes zone 0 a package: `intel-rapl:` is only an identifier, and each zone says what it is in its `name` file. A host whose zone 0 is the platform zone (`psys`) or a `dram` zone had that domain read and probed as the package. proof-cascade does this right since 9334ec0c; this follows it. - The package set is listed from the powercap directory: every top-level `intel-rapl:` zone whose `name` reads `package-`. Left out wherever they sit, zone 0 included: `psys` (the platform zone, whose energy includes the packages', so summing it beside them counts them twice), `dram`, the `core`/`uncore`/`dram` sub-zones (`intel-rapl:0:0`), the control type (`intel-rapl`), and the MMIO interface (`intel-rapl-mmio:0`, also named `package-0`, which republishes a package). The listing reads the directory rather than trying fixed indices, and takes the root path as an argument, so the tests run it on fake trees. - Liveness: each package counter is its own rail in the sample, named for its zone, and the existing every-rail rule (the one IOReport's dies are held to) makes RAPL live only when EVERY package keeps advancing. A host with no package zone (only `psys`) is `Unavailable` with nothing read or burned, so no counter is constructed. - The probe's verdict is cached together with the package set it watched (`linux_rapl::host()`, one `OnceLock`). `rapl_status()` and the counter's constructor both read that cache, so the counter sums exactly the packages that were proved live. - The counter returns a running total: the first read's package counters summed, then each package's delta added, each taken modulo its own `max_energy_range_uj`. Summing the raw counters would make one package's wrap a fall of the sum, and with sockets at different ranges no single range is right. Before this, a wrap of the one zone was a fall, which callers take as no reading; now a wrap is read across. The files are read under the counter's lock, so two threads' reads are folded in the order they were taken, and an older read is never taken for a wrap. A package that cannot be read fails the read: no figure that leaves a package out. A package whose counter falls while its range is unknown (absent, or 0) fails that read and every later one, because the total would be short by energy no one can know. Tests, on fake powercap directories under the temp dir (removed when done), so they run on macOS: - a two-socket server (the control type, package-0 and package-1 with core and dram sub-zones, psys, a top-level dram, intel-rapl-mmio:0) lists packages 0 and 1 only, and with every zone moving, the counter's delta is the two packages' energy (250,000 + 400,000 uJ) and nothing else; - zone 0 named psys beside package zone 1: zone 1 alone is listed, sampled and summed; - a psys zone 0 moving at every tick beside a held package: Frozen; - both packages advancing: Live; package 1 held while package 0 and psys move: Frozen; scripted, either package held: Frozen; - per-zone wrap: package 0 wraps at 262,143,328,850 while package 1 advances, then package 1 wraps at 65,532,610,987: deltas of 1,700 and 2,500 uJ; taken at the other zone's range, package 1's wrap would be off by the difference of the two ranges; - a package with no range, absent or 0, is summed while it rises; its fall fails that read and the next; - an unreadable package fails the read; - an only-psys host, an empty directory and a missing one: Unavailable, no tick, no counter; a counter forced over no package never reads; - the name and directory rules: `package-`, `intel-rapl:`. On Linux, the hardware test reads every package in the cached set and, when it is Live, requires each one to move across 200 ms of CPU work. Falsified, each mutant applied to this commit, run, and restored with `git checkout` (blob hash equal to HEAD's): - back to `intel-rapl:0` only, whatever its name: 8 of the 12 RAPL tests fail; - every top-level zone summed (no name check): 5 fail; - every package wrapped at package 0's range: 1 fails, the per-zone wrap test; - the liveness sample holding the first package only: 3 fail. What ran where. On this M5 Max (macOS 27.0.1): `cargo test -p eoc-meter` 68 + 1 passed, with `--features cuda` 69 + 1; `cargo test --workspace` (with the eoc-agent commit that follows) 686 passed, 0 failed, 3 ignored. For x86_64-unknown-linux-gnu, `cargo check --all-targets` and `cargo clippy --all-targets` over every workspace member except the five that link aws-lc-sys (eoc-embeddings, -multimodal, -rerank, -tools, -vendor-api, whose build script needs `x86_64-linux-gnu-gcc`, not installed here) pass, with no warning in eoc-meter, with or without `cuda`. The sysfs reader, the host cache and the hardware test are type-checked, not run: there is no Linux host with readable RAPL here. Co-Authored-By: Claude Opus 5.5 --- eoc/eoc-rs/crates/eoc-meter/src/lib.rs | 716 +++++++++++++++++++++---- 1 file changed, 621 insertions(+), 95 deletions(-) diff --git a/eoc/eoc-rs/crates/eoc-meter/src/lib.rs b/eoc/eoc-rs/crates/eoc-meter/src/lib.rs index c247f2937..504476135 100644 --- a/eoc/eoc-rs/crates/eoc-meter/src/lib.rs +++ b/eoc/eoc-rs/crates/eoc-meter/src/lib.rs @@ -24,9 +24,13 @@ //! declined, because a lump landing in one span would be read as that span's //! energy. A live counter's delta is a reading; nothing else is. //! -//! - Linux RAPL is read via `/sys/class/powercap/intel-rapl:0/energy_uj` — -//! **only when that counter reads and is live** ([`rapl_status`]; the -//! probe burns CPU on the calling thread). No external crate required. +//! - Linux RAPL is read through powercap (`/sys/class/powercap`) as the sum +//! of every package zone — each top-level `intel-rapl:` zone whose `name` +//! reads `package-`, so every socket of a multi-socket server, and never +//! `psys` (the platform zone, whose energy includes the packages'), `dram` +//! or a sub-zone — each zone's wrap taken at its own `max_energy_range_uj`; +//! **only when every package counter reads and is live** ([`rapl_status`]; +//! the probe burns CPU on the calling thread). No external crate required. //! - Apple Silicon IOReport Energy Model counters are read through Apple's //! IOReport C API (no sudo, no external binary) — **only when every CPU //! energy rail is live**, each die's on a multi-die part @@ -93,10 +97,10 @@ impl JouleCounter for StubCounter { // with scripted samples and a scripted clock. The FFI only takes a sample and // lists its channels as (name, absolute raw integer); everything that decides // lives here. One rule for every vendor counter: IOReport watches the CPU energy -// rails of an Energy Model sample; RAPL and NVML hand their one counter as the -// only channel. Every rail watched must keep advancing, each on its own, because -// a counter's reading sums every rail it watches: IOReport's adds every die's -// `CPU Energy`. Compiled on every platform, so a Linux CI leg runs its tests. +// rails of an Energy Model sample; RAPL hands each package counter as its own +// channel, and NVML its one counter. Every rail watched must keep advancing, each +// on its own, because a counter's reading sums every rail it watches: IOReport's +// adds every die's `CPU Energy`, RAPL's every package's. Compiled on every platform, so a Linux CI leg runs its tests. #[cfg_attr(not(target_os = "macos"), allow(dead_code))] mod rail_liveness { use std::time::Duration; @@ -229,9 +233,10 @@ mod rail_liveness { /// frozen would give a figure that leaves out the frozen die's energy, /// or carries its refreshes, while claiming the whole CPU. A rail that /// joins late is held to the intervals observed since the first, so it - /// is live only once its own advances are 80% of all of them. A single - /// vendor counter's one channel is every rail it has, so for RAPL and NVML - /// this is the rule it always was. + /// is live only once its own advances are 80% of all of them. RAPL's + /// figure sums every package counter, so on a multi-socket server each + /// package is a rail, and every one must be live; NVML's one channel is + /// every rail it has. /// /// Which channels of a sample are rails is the watch's `pick`: the CPU /// energy aggregates for IOReport ([`RailWatch::default`]), every channel @@ -268,9 +273,9 @@ mod rail_liveness { Self { pick, prev: None, first: None, intervals: 0, window: Duration::ZERO, rails: Vec::new(), live: false } } - /// A watch of every channel of a sample: for a single vendor counter - /// (RAPL's package counter, NVML's total-energy counter), whose every - /// channel counts the domain it claims. + /// A watch of every channel of a sample: for a vendor counter (RAPL's + /// package counters, one channel a package; NVML's total-energy + /// counter), whose every channel counts the domain it claims. #[cfg_attr(not(any(target_os = "linux", feature = "cuda")), allow(dead_code))] // used by the RAPL and NVML rungs pub(crate) fn every_channel() -> Self { Self::of(|_| true) @@ -3634,9 +3639,10 @@ pub fn liveness_sampler_opens() -> usize { #[derive(Debug, Clone, Copy, PartialEq, Eq)] pub enum CounterStatus { /// The counter could not be opened or read — not this platform, the - /// feature not built, no such device, a locked-down sysfs (`energy_uj` - /// is root-only on most current kernels) — or it gave fewer than two - /// usable samples in five budgets. Never selected. + /// feature not built, no such device, no RAPL zone named `package-` + /// (a host that lists only `psys`), a locked-down sysfs (`energy_uj` is + /// root-only on most current kernels) — or it gave fewer than two usable + /// samples in five budgets. Never selected. Unavailable, /// The counter read, but did not keep advancing while the probe ran: it /// held, or was republished in lumps. Never selected: a lump landing in @@ -3658,19 +3664,22 @@ const VENDOR_COUNTER_LIVENESS_BUDGET: std::time::Duration = std::time::Duration: /// This host's RAPL status, probed once per process and cached. /// /// [`detect`] selects the RAPL rung only when this is [`CounterStatus::Live`]: -/// the package-0 counter (`/sys/class/powercap/intel-rapl:0/energy_uj`) read -/// and kept advancing while the probing thread burned CPU, in at least five -/// sample intervals, at least 80% of them, over at least 300 ms. One that -/// holds, or is republished in lumps (a virtualised powercap tree can do -/// either), is declined. Where the counter is absent or unreadable it is -/// [`CounterStatus::Unavailable`] without burning anything. Always -/// `Unavailable` off Linux. The first call on a host whose counter reads burns -/// about 300 ms of the calling thread's CPU (a second if it is not live). +/// every package counter — each top-level powercap zone whose `name` reads +/// `package-`, so each socket of a multi-socket server, and never `psys`, +/// `dram` or a sub-zone, wherever it sits — read, and each kept advancing +/// while the probing thread burned CPU, in at least five sample intervals, at +/// least 80% of them, over at least 300 ms. The counter's figure sums them +/// all, so one that holds, or is republished in lumps (a virtualised powercap +/// tree can do either), declines the rung. Where there is no package zone, or +/// a package counter is unreadable, it is [`CounterStatus::Unavailable`] +/// without burning anything. Always `Unavailable` off Linux. The first call on +/// a host whose counters read burns about 300 ms of the calling thread's CPU +/// (a second if they are not live); the package set it watched is cached +/// beside the status, and is the set the counter sums. pub fn rapl_status() -> CounterStatus { #[cfg(target_os = "linux")] { - static STATUS: std::sync::OnceLock = std::sync::OnceLock::new(); - *STATUS.get_or_init(linux_rapl::host_status) + linux_rapl::host().status } #[cfg(not(target_os = "linux"))] { @@ -3701,83 +3710,288 @@ pub fn nvml_status() -> CounterStatus { } } -// Compiled for tests on every platform, so its sampler and gate run under test -// on a Mac too; selected only on Linux. +// Compiled for tests on every platform, so its zone listing, sampler, counter +// and gate run under test on a Mac too, over fake powercap directories; +// selected only on Linux. #[cfg(any(target_os = "linux", test))] #[cfg_attr(not(target_os = "linux"), allow(dead_code))] mod linux_rapl { use super::*; use crate::rail_liveness; + use std::path::{Path, PathBuf}; + use std::sync::Mutex; + use std::time::Duration; - /// Intel / AMD RAPL package-0 energy, in micro-joules, through powercap. - pub(crate) const RAPL_ENERGY_PATH: &str = "/sys/class/powercap/intel-rapl:0/energy_uj"; + /// Where the kernel lists its powercap zones, RAPL's among them. + pub(crate) const POWERCAP_ROOT: &str = "/sys/class/powercap"; - /// Reads Intel RAPL counters via the powercap sysfs interface. + /// One RAPL package zone: a top-level `intel-rapl:` zone whose + /// `name` reads `package-`. + #[derive(Debug, Clone, Copy, PartialEq, Eq)] + pub(crate) struct Package { + /// `` in `intel-rapl:`. + pub(crate) index: u32, + /// The zone's `max_energy_range_uj`, the value its `energy_uj` wraps + /// at, when that reads as a number above zero. + pub(crate) max_energy_range_uj: Option, + } + + /// Whether a top-level zone's `name` file names a package: `package-`, + /// `` one or more digits (the file ends in a newline). Not `psys` (the + /// platform zone, whose energy includes the packages'), `dram`, `core` + /// or `uncore`. + pub(crate) fn is_package_name(name: &str) -> bool { + name.trim().strip_prefix("package-").is_some_and(|n| !n.is_empty() && n.bytes().all(|b| b.is_ascii_digit())) + } + + /// The index of a top-level RAPL zone directory, `intel-rapl:`. + /// `None` for anything else: a sub-zone (`intel-rapl:0:0`, a package's + /// core, uncore or dram), the control type itself (`intel-rapl`), and the + /// MMIO interface (`intel-rapl-mmio:0`, also named `package-0`, which + /// republishes a package's energy and would count it twice). + pub(crate) fn top_level_zone(dir: &str) -> Option { + let n = dir.strip_prefix("intel-rapl:")?; + if n.is_empty() || !n.bytes().all(|b| b.is_ascii_digit()) { + return None; + } + n.parse().ok() + } + + /// Zone `index`'s directory under the powercap directory `root`. + fn zone_dir(root: &Path, index: u32) -> PathBuf { + root.join(format!("intel-rapl:{index}")) + } + + /// The file at `path`, read and parsed as an unsigned integer; `None` when + /// it cannot be. + pub(crate) fn read_u64(path: &Path) -> Option { + std::fs::read_to_string(path).ok()?.trim().parse::().ok() + } + + /// The package zones under the powercap directory `root`, in index order: + /// every top-level zone whose `name` reads `package-`, with the range + /// its counter wraps at. Everything else is left out wherever it sits, + /// zone 0 included: `psys`, a `dram` zone, sub-zones, the MMIO interface. + /// Empty when no zone is a package, or `root` cannot be listed. + pub(crate) fn packages_under(root: &Path) -> Vec { + let Ok(entries) = std::fs::read_dir(root) else { + return Vec::new(); + }; + let mut packages: Vec = entries + .flatten() + .filter_map(|entry| { + let index = top_level_zone(entry.file_name().to_str()?)?; + let dir = entry.path(); + let name = std::fs::read_to_string(dir.join("name")).ok()?; + is_package_name(&name).then(|| Package { + index, + max_energy_range_uj: read_u64(&dir.join("max_energy_range_uj")).filter(|&m| m > 0), + }) + }) + .collect(); + packages.sort_by_key(|p| p.index); + packages + } + + /// Every package's `energy_uj` under `root`, in `packages` order. `None` + /// when there is no package, or when any one cannot be read: a figure that + /// left a package out would miss its energy while claiming the whole CPU. + pub(crate) fn read_packages(root: &Path, packages: &[Package]) -> Option> { + if packages.is_empty() { + return None; + } + packages.iter().map(|p| read_u64(&zone_dir(root, p.index).join("energy_uj"))).collect() + } + + /// One zone's energy between two reads, modulo its own wrap. A counter + /// that went down wrapped at `max_range`; without a known (non-zero) + /// range, or with an earlier read above it, the energy is unknown and + /// `None`. Undercounts a wrap by at most one counter unit, and cannot see + /// more than one wrap between two reads: about a quarter of an hour at + /// 300 W for a package's usual 262,143,328,850 µJ range. + pub(crate) fn zone_delta_uj(e0: u64, e1: u64, max_range: Option) -> Option { + if e1 >= e0 { + return Some(e1 - e0); + } + max_range.filter(|&m| m > 0)?.checked_sub(e0)?.checked_add(e1) + } + + /// A RAPL counter's running total across its reads. + #[derive(Debug)] + pub(crate) enum Running { + /// Not read yet. + Unread, + /// Each package's raw `energy_uj` at the last read, and the total so + /// far: the first read's counters summed, plus every package's energy + /// since, each taken modulo its own range. + At { last: Vec, total: u64 }, + /// A package's energy between two reads could not be known: its + /// counter went down and its range is unknown. The total no longer + /// holds that energy, so every read from then on fails rather than + /// hand out a figure short by it. + Lost(String), + } + + impl Running { + /// Fold in `now`, every package's raw counter read in `packages` order, + /// and return the running total. Each package's wrap is taken at ITS + /// OWN range; summing the raw counters instead would turn one + /// package's wrap into a fall of the sum. + pub(crate) fn advance(&mut self, packages: &[Package], now: &[u64]) -> Result { + match self { + Running::Lost(why) => Err(Error::Backend(why.clone())), + Running::Unread => { + let total = now + .iter() + .try_fold(0u64, |sum, &uj| sum.checked_add(uj)) + .ok_or_else(|| Error::Backend("the RAPL package counters' sum overflows u64".into()))?; + *self = Running::At { last: now.to_vec(), total }; + Ok(total) + } + Running::At { last, total } => { + if last.len() != now.len() || packages.len() != now.len() { + return Err(Error::Backend("a RAPL read did not cover every package".into())); + } + match energy_between(packages, last, now).and_then(|energy| { + total.checked_add(energy).ok_or_else(|| "the RAPL running total overflows u64".to_string()) + }) { + Ok(sum) => { + *total = sum; + last.copy_from_slice(now); + Ok(sum) + } + Err(why) => { + *self = Running::Lost(why.clone()); + Err(Error::Backend(why)) + } + } + } + } + } + } + + /// Every package's energy between the raw counters `last` and `now`, + /// summed, each taken modulo its own range ([`zone_delta_uj`]); why not, + /// when one package's energy is unknown. + fn energy_between(packages: &[Package], last: &[u64], now: &[u64]) -> std::result::Result { + let mut energy = 0u64; + for ((p, &e0), &e1) in packages.iter().zip(last).zip(now) { + let delta = zone_delta_uj(e0, e1, p.max_energy_range_uj).ok_or_else(|| { + format!( + "RAPL zone intel-rapl:{} went down from {e0} to {e1} uJ with no readable \ + max_energy_range_uj; its energy between the two reads is unknown", + p.index + ) + })?; + energy = energy.checked_add(delta).ok_or_else(|| "RAPL package energy overflows u64".to_string())?; + } + Ok(energy) + } + + /// Reads RAPL's package energy through powercap: the sum over every + /// package zone (`intel-rapl:` named `package-`), each zone's wrap + /// taken at its own `max_energy_range_uj`, as a running total. /// - /// Constructed only on a host whose package counter the liveness probe - /// found live ([`crate::rapl_status`]); a live RAPL counter is a vendor - /// energy counter, so a delta it reads is a measurement. + /// Constructed only on a host whose package counters the liveness probe + /// found live, every one of them ([`crate::rapl_status`]); a live RAPL + /// counter is a vendor energy counter, so a delta it reads is a + /// measurement. It sums exactly the packages the probe watched. pub struct LinuxRaplCounter { - path: std::path::PathBuf, + root: PathBuf, + packages: Vec, + running: Mutex, } impl LinuxRaplCounter { - /// The RAPL package-0 counter, on a host where it is live. - /// `Err(NoCounter)` when it is absent, unreadable, or does not keep - /// advancing under load; [`crate::rapl_status`] says which. + /// This host's RAPL package counters, on a host where every one is + /// live. `Err(NoCounter)` when there is no package zone, or one is + /// unreadable or does not keep advancing under load; + /// [`crate::rapl_status`] says which. pub fn try_detect() -> Result { Self::try_detect_given(crate::rapl_status()) } /// The constructor behind [`Self::try_detect`], given this host's - /// status: every status the gate does not admit declines. + /// status: every status the gate does not admit declines. The counter + /// sums the package zones the host's probe watched. pub(crate) fn try_detect_given(status: CounterStatus) -> Result { + Self::admitted(status, Path::new(POWERCAP_ROOT), host_packages) + } + + /// The counter over `packages` under the powercap directory `root`, + /// when `status` admits it; `packages` is asked for only then. + pub(crate) fn admitted(status: CounterStatus, root: &Path, packages: impl FnOnce() -> Vec) -> Result { if rail_liveness::admits_vendor_counter(status) { - Ok(Self { path: std::path::PathBuf::from(RAPL_ENERGY_PATH) }) + Ok(Self::over(root, packages())) } else { Err(Error::NoCounter) } } + + /// The counter over `packages` under `root`, ungated. + pub(crate) fn over(root: &Path, packages: Vec) -> Self { + Self { root: root.to_path_buf(), packages, running: Mutex::new(Running::Unread) } + } } impl JouleCounter for LinuxRaplCounter { + /// The running total. Fails — no figure, never a partial one — when + /// there is no package, when any package's counter cannot be read, or + /// once a package's energy between two reads is unknown. The counters + /// are read under the lock, so two threads' reads are folded in the + /// order they were taken, and an older read is never taken for a wrap. fn read_microjoules(&self) -> Result { - let s = std::fs::read_to_string(&self.path) - .map_err(|e| Error::Io(e.to_string()))?; - s.trim() - .parse::() - .map_err(|e| Error::Backend(e.to_string())) + if self.packages.is_empty() { + return Err(Error::NoCounter); + } + let mut running = self.running.lock().unwrap_or_else(std::sync::PoisonError::into_inner); + let now = read_packages(&self.root, &self.packages).ok_or_else(|| { + Error::Io(format!("a RAPL package zone's energy_uj under {} could not be read", self.root.display())) + })?; + running.advance(&self.packages, &now) } fn name(&self) -> &'static str { "rapl" } } - /// One liveness sample of the counter `read` reads: its raw `energy_uj` - /// as the one rail, named `intel-rapl:0`. `None` for a failed read, which - /// the probe skips. A wrap shows as a fall, which is no advance. - pub(crate) fn sample(read: impl Fn() -> Option) -> Option> { - let uj = read()?; - Some(vec![("intel-rapl:0".to_string(), i64::try_from(uj).unwrap_or(i64::MAX))]) + /// The name a package's counter has in a liveness sample: its zone, + /// `intel-rapl:`. + pub(crate) fn rail_name(index: u32) -> String { + format!("intel-rapl:{index}") } - /// The counter at `path`, read and parsed; `None` when it cannot be. - pub(crate) fn read_energy_uj(path: &std::path::Path) -> Option { - std::fs::read_to_string(path).ok()?.trim().parse::().ok() + /// One liveness sample of the package counters `read` reads, in + /// `packages` order: each package's raw `energy_uj` as its own rail, under + /// its zone's name, so the rule asks EVERY package to keep advancing (the + /// figure sums them all). `None` for a failed read, which the probe + /// skips. A wrap shows as a fall, which is no advance. + pub(crate) fn sample(packages: &[Package], read: impl Fn() -> Option>) -> Option> { + let raw = read()?; + if raw.len() != packages.len() { + return None; + } + Some(packages.iter().zip(raw).map(|(p, uj)| (rail_name(p.index), i64::try_from(uj).unwrap_or(i64::MAX))).collect()) } - /// The RAPL probe on a given reader: it opens when the counter reads at - /// all, and is then sampled by [`rail_liveness::probe_counter`], with - /// `tick` between samples, by the clock `now`. + /// The RAPL probe of `packages` on a given reader. With no package it is + /// `Unavailable`, with nothing read and nothing burned. Otherwise it opens + /// when every package counter reads, and is then sampled by + /// [`rail_liveness::probe_counter`], with `tick` between samples, by the + /// clock `now`: live only when every package counter keeps advancing. pub(crate) fn status_with( - read: impl Fn() -> Option, + packages: &[Package], + read: impl Fn() -> Option>, tick: impl FnMut(), - now: impl Fn() -> std::time::Duration, + now: impl Fn() -> Duration, ) -> CounterStatus { + if packages.is_empty() { + return CounterStatus::Unavailable; + } rail_liveness::counter_status( rail_liveness::probe_counter( || read().map(|_| ()), - |_| sample(&read), + |_| sample(packages, &read), tick, now, crate::VENDOR_COUNTER_LIVENESS_BUDGET, @@ -3786,26 +4000,324 @@ mod linux_rapl { ) } - /// The real RAPL probe: the real counter file, the CPU-burning tick (a + /// What one probe found under a powercap directory: its package zones, + /// and whether every one of their counters is live. + #[derive(Debug, Clone, PartialEq, Eq)] + pub(crate) struct Host { + pub(crate) packages: Vec, + pub(crate) status: CounterStatus, + } + + /// One probe of the package zones under `root`: list them + /// ([`packages_under`]), then probe all of their counters together + /// ([`status_with`]). + pub(crate) fn probe_root(root: &Path, tick: impl FnMut(), now: impl Fn() -> Duration) -> Host { + let packages = packages_under(root); + let status = status_with(&packages, || read_packages(root, &packages), tick, now); + Host { packages, status } + } + + /// This host's package zones and their liveness, probed once per process + /// and cached: the real powercap directory, the CPU-burning tick (a /// package counter must advance while this thread burns CPU) and the - /// monotonic clock. Cached by [`crate::rapl_status`], its only caller. + /// monotonic clock. Behind [`crate::rapl_status`] and the counter's + /// constructor, so the counter sums exactly the packages the probe found + /// live. #[cfg(target_os = "linux")] - pub(crate) fn host_status() -> CounterStatus { - let path = std::path::Path::new(RAPL_ENERGY_PATH); - status_with(|| read_energy_uj(path), crate::clocks::cpu_burn_tick, crate::clocks::wall_now) + pub(crate) fn host() -> &'static Host { + static HOST: std::sync::OnceLock = std::sync::OnceLock::new(); + HOST.get_or_init(|| probe_root(Path::new(POWERCAP_ROOT), crate::clocks::cpu_burn_tick, crate::clocks::wall_now)) + } + + /// The package zones the host's probe watched; none off Linux. + fn host_packages() -> Vec { + #[cfg(target_os = "linux")] + { + host().packages.clone() + } + #[cfg(not(target_os = "linux"))] + { + Vec::new() + } } #[cfg(test)] mod tests { use super::*; use std::cell::Cell; - use std::time::Duration; - /// The RAPL probe on scripted reads, with a clock that only the tick - /// moves, by 25 ms. - fn scripted(read: impl Fn(usize) -> Option) -> CounterStatus { + /// A package zone's range as the kernel commonly reports it (2^32 + /// units of 61.035 µJ), and a smaller one, so that a wrap taken at the + /// wrong zone's range shows. + const RANGE: u64 = 262_143_328_850; + const RANGE_B: u64 = 65_532_610_987; + + /// A fake powercap directory for one test, removed when dropped. + struct Powercap(PathBuf); + + impl Powercap { + fn new(tag: &str) -> Self { + let root = std::env::temp_dir().join(format!("eoc-meter-powercap-{tag}-{}", std::process::id())); + let _ = std::fs::remove_dir_all(&root); + std::fs::create_dir_all(&root).expect("create a fake powercap directory"); + Self(root) + } + + /// Add the zone directory `dir` (`intel-rapl:0`, `intel-rapl:0:1`, + /// `intel-rapl-mmio:0`, …) with its `name`, its `energy_uj` and, + /// when given, its `max_energy_range_uj`, newline-terminated as + /// the kernel writes them. + fn zone(&self, dir: &str, name: &str, energy_uj: u64, max: Option) -> &Self { + let d = self.0.join(dir); + std::fs::create_dir_all(&d).expect("create a fake zone"); + std::fs::write(d.join("name"), format!("{name}\n")).expect("write name"); + if let Some(m) = max { + std::fs::write(d.join("max_energy_range_uj"), format!("{m}\n")).expect("write max_energy_range_uj"); + } + self.set(dir, energy_uj); + self + } + + /// Set zone `dir`'s `energy_uj`. + fn set(&self, dir: &str, energy_uj: u64) { + std::fs::write(self.0.join(dir).join("energy_uj"), format!("{energy_uj}\n")).expect("write energy_uj"); + } + + fn root(&self) -> &Path { + &self.0 + } + } + + impl Drop for Powercap { + fn drop(&mut self) { + let _ = std::fs::remove_dir_all(&self.0); + } + } + + /// A two-socket server's powercap directory as the kernel lists it: + /// the control type, two packages each with core or dram sub-zones, + /// a platform (`psys`) zone, a top-level `dram` zone, and the MMIO + /// interface republishing package 0. + fn two_socket_server(tag: &str) -> Powercap { + let pc = Powercap::new(tag); + std::fs::create_dir_all(pc.root().join("intel-rapl")).expect("the control type"); + pc.zone("intel-rapl:0", "package-0", 1_000_000, Some(RANGE)) + .zone("intel-rapl:0:0", "core", 400_000, Some(RANGE)) + .zone("intel-rapl:0:1", "dram", 90_000, Some(RANGE_B)) + .zone("intel-rapl:1", "package-1", 5_000_000, Some(RANGE_B)) + .zone("intel-rapl:1:0", "dram", 70_000, Some(RANGE_B)) + .zone("intel-rapl:2", "psys", 80_000_000, Some(RANGE)) + .zone("intel-rapl:3", "dram", 3_000_000, Some(RANGE_B)) + .zone("intel-rapl-mmio:0", "package-0", 1_000_000, Some(RANGE)); + pc + } + + fn pkg(index: u32, max: u64) -> Package { + Package { index, max_energy_range_uj: Some(max) } + } + + /// The probe over a fake directory, run as the host's probe runs + /// (list, read, the rule), with a tick that sets each zone `step(n)` + /// names at the n-th tick, and a clock that moves 25 ms a tick. + fn probe_tree(pc: &Powercap, step: impl Fn(u64) -> Vec<(&'static str, u64)>) -> Host { + let ticks = Cell::new(0u32); + probe_root( + pc.root(), + || { + ticks.set(ticks.get() + 1); + for (dir, uj) in step(u64::from(ticks.get())) { + pc.set(dir, uj); + } + }, + || Duration::from_millis(25) * ticks.get(), + ) + } + + /// A package zone is named `package-`; `psys`, `dram`, `core` and + /// malformed names are not. A top-level zone is `intel-rapl:`; a + /// sub-zone, the control type and the MMIO interface are not. + #[test] + fn only_a_top_level_zone_named_package_n_is_a_package() { + for name in ["package-0", "package-1\n", "package-12"] { + assert!(is_package_name(name), "{name:?}"); + } + for name in ["psys", "psys\n", "dram", "core", "uncore", "package-", "package-x", "package-0a", "Package-0", ""] { + assert!(!is_package_name(name), "{name:?}"); + } + assert_eq!(top_level_zone("intel-rapl:0"), Some(0)); + assert_eq!(top_level_zone("intel-rapl:12"), Some(12)); + for dir in ["intel-rapl", "intel-rapl:", "intel-rapl:0:0", "intel-rapl:1:2", "intel-rapl-mmio:0", "intel-rapl:x", "dtpm"] { + assert_eq!(top_level_zone(dir), None, "{dir:?}"); + } + } + + /// A two-socket server: the figure is both packages' energy and + /// nothing else — not `psys` (which includes the packages, so it would + /// count them twice), not a `dram` zone at the top level or under a + /// package, not a core sub-zone, not the MMIO copy of package 0. + #[test] + fn a_two_socket_server_sums_its_two_packages_and_nothing_else() { + let pc = two_socket_server("two-socket"); + let packages = packages_under(pc.root()); + assert_eq!(packages, vec![pkg(0, RANGE), pkg(1, RANGE_B)]); + assert_eq!(read_packages(pc.root(), &packages), Some(vec![1_000_000, 5_000_000])); + let counter = LinuxRaplCounter::over(pc.root(), packages); + let before = counter.read_microjoules().expect("both packages read"); + assert_eq!(before, 1_000_000 + 5_000_000); + for (dir, uj) in [ + ("intel-rapl:0", 1_250_000), + ("intel-rapl:1", 5_400_000), + ("intel-rapl:0:0", 600_000), + ("intel-rapl:0:1", 190_000), + ("intel-rapl:1:0", 170_000), + ("intel-rapl:2", 99_000_000), + ("intel-rapl:3", 3_900_000), + ("intel-rapl-mmio:0", 1_250_000), + ] { + pc.set(dir, uj); + } + let after = counter.read_microjoules().expect("both packages read"); + assert_eq!(after - before, 250_000 + 400_000, "package 0's energy and package 1's, nothing else"); + } + + /// Zone 0 is the platform zone and the package is zone 1: zone 0 is + /// neither summed nor sampled, and the figure is zone 1's. + #[test] + fn a_zone_0_named_psys_is_left_out() { + let pc = Powercap::new("psys-at-0"); + pc.zone("intel-rapl:0", "psys", 70_000_000, Some(RANGE)) + .zone("intel-rapl:1", "package-0", 2_000_000, Some(RANGE)) + .zone("intel-rapl:1:0", "core", 1_000, Some(RANGE)); + let packages = packages_under(pc.root()); + assert_eq!(packages, vec![pkg(1, RANGE)]); + assert_eq!( + sample(&packages, || read_packages(pc.root(), &packages)), + Some(vec![("intel-rapl:1".to_string(), 2_000_000)]), + "the liveness sample holds the package alone" + ); + let counter = LinuxRaplCounter::over(pc.root(), packages); + let before = counter.read_microjoules().expect("the package reads"); + pc.set("intel-rapl:0", 90_000_000); + pc.set("intel-rapl:1", 2_300_000); + assert_eq!(counter.read_microjoules().expect("the package reads") - before, 300_000); + } + + /// A `psys` zone 0 that moves at every tick beside a package that + /// holds: frozen. The platform zone is not the package, and is never + /// watched. + #[test] + fn a_moving_psys_at_zone_0_never_makes_a_held_package_live() { + let pc = Powercap::new("psys-moving"); + pc.zone("intel-rapl:0", "psys", 70_000_000, Some(RANGE)).zone("intel-rapl:1", "package-0", 2_000_000, Some(RANGE)); + let host = probe_tree(&pc, |t| vec![("intel-rapl:0", 70_000_000 + 99_000 * t)]); + assert_eq!(host, Host { packages: vec![pkg(1, RANGE)], status: CounterStatus::Frozen }); + } + + /// Every package must keep advancing, because the figure sums them + /// all: both moving at every tick is live; package 1 holding is + /// frozen, though package 0 and `psys` move at every tick. + #[test] + fn every_package_counter_must_be_live() { + let both = two_socket_server("live-both"); + let host = probe_tree(&both, |t| vec![("intel-rapl:0", 1_000_000 + 40_000 * t), ("intel-rapl:1", 5_000_000 + 35_000 * t)]); + assert_eq!(host, Host { packages: vec![pkg(0, RANGE), pkg(1, RANGE_B)], status: CounterStatus::Live }); + let one = two_socket_server("live-one"); + let host = probe_tree(&one, |t| vec![("intel-rapl:0", 1_000_000 + 40_000 * t), ("intel-rapl:2", 80_000_000 + 99_000 * t)]); + assert_eq!(host.status, CounterStatus::Frozen, "package 1 held its value"); + } + + /// Each package wraps at its own range: package 0 rolls over at RANGE + /// while package 1 advances, then package 1 rolls over at RANGE_B. The + /// energy is each zone's delta modulo its own range, and the total + /// keeps rising across both wraps. + #[test] + fn each_package_wraps_at_its_own_range() { + let pc = Powercap::new("wrap"); + pc.zone("intel-rapl:0", "package-0", RANGE - 1_000, Some(RANGE)) + .zone("intel-rapl:1", "package-1", RANGE_B - 300, Some(RANGE_B)); + let counter = LinuxRaplCounter::over(pc.root(), packages_under(pc.root())); + let r0 = counter.read_microjoules().expect("read"); + pc.set("intel-rapl:0", 500); + pc.set("intel-rapl:1", RANGE_B - 100); + let r1 = counter.read_microjoules().expect("read across package 0's wrap"); + assert_eq!(r1 - r0, (1_000 + 500) + 200); + pc.set("intel-rapl:0", 2_500); + pc.set("intel-rapl:1", 400); + let r2 = counter.read_microjoules().expect("read across package 1's wrap"); + assert_eq!(r2 - r1, 2_000 + (100 + 400)); + // At the other zone's range, package 1's wrap would be off by the + // difference of the two ranges. + assert_eq!(zone_delta_uj(RANGE_B - 100, 400, Some(RANGE_B)), Some(500)); + assert_eq!(zone_delta_uj(RANGE_B - 100, 400, Some(RANGE)), Some(RANGE - RANGE_B + 500)); + } + + /// A package whose range is unknown (absent, or zero): summed while it + /// rises; once it goes down its energy since the last read is + /// unknown, so that read fails, and every later one too, since the + /// total would be short by that energy. + #[test] + fn a_fall_with_no_known_range_ends_the_figure() { + let pc = Powercap::new("no-range"); + pc.zone("intel-rapl:0", "package-0", 10_000, None).zone("intel-rapl:1", "package-1", 20_000, Some(0)); + let packages = packages_under(pc.root()); + assert_eq!(packages, vec![Package { index: 0, max_energy_range_uj: None }, Package { index: 1, max_energy_range_uj: None }]); + let counter = LinuxRaplCounter::over(pc.root(), packages); + let r0 = counter.read_microjoules().expect("read"); + pc.set("intel-rapl:0", 15_000); + pc.set("intel-rapl:1", 21_000); + assert_eq!(counter.read_microjoules().expect("read") - r0, 5_000 + 1_000); + pc.set("intel-rapl:0", 3_000); + assert!(matches!(counter.read_microjoules(), Err(Error::Backend(_)))); + pc.set("intel-rapl:0", 4_000); + assert!(matches!(counter.read_microjoules(), Err(Error::Backend(_))), "a total short by the lost energy"); + assert_eq!(zone_delta_uj(1_000, 10, Some(0)), None, "a zero range is no range"); + } + + /// A package that cannot be read fails the whole read: no figure that + /// leaves a package out. + #[test] + fn an_unreadable_package_fails_the_read() { + let pc = Powercap::new("unreadable"); + pc.zone("intel-rapl:0", "package-0", 10_000, Some(RANGE)).zone("intel-rapl:1", "package-1", 20_000, Some(RANGE)); + let packages = packages_under(pc.root()); + let counter = LinuxRaplCounter::over(pc.root(), packages.clone()); + std::fs::remove_file(pc.root().join("intel-rapl:1/energy_uj")).expect("remove"); + assert_eq!(read_packages(pc.root(), &packages), None); + assert!(matches!(counter.read_microjoules(), Err(Error::Io(_)))); + } + + /// A host whose only zone is the platform (`psys`) has no package: the + /// probe reads and burns nothing and finds RAPL unavailable, and no + /// RAPL counter is constructed; one forced over no package never + /// reads. So too an empty powercap directory, and a missing one. + #[test] + fn a_host_with_only_psys_has_no_rapl_counter() { + let pc = Powercap::new("only-psys"); + pc.zone("intel-rapl:0", "psys", 70_000_000, Some(RANGE)); + let ticks = Cell::new(0u32); + let host = probe_root(pc.root(), || ticks.set(ticks.get() + 1), || Duration::from_millis(25) * ticks.get()); + assert_eq!(host, Host { packages: Vec::new(), status: CounterStatus::Unavailable }); + assert_eq!(ticks.get(), 0, "burned CPU probing a host with no package"); + let asked = Cell::new(false); + let counter = LinuxRaplCounter::admitted(host.status, pc.root(), || { + asked.set(true); + host.packages.clone() + }); + assert!(matches!(counter, Err(Error::NoCounter))); + assert!(!asked.get(), "listed the packages of a declined counter"); + assert!(matches!(LinuxRaplCounter::over(pc.root(), Vec::new()).read_microjoules(), Err(Error::NoCounter))); + let empty = Powercap::new("empty"); + assert_eq!(probe_root(empty.root(), || panic!("ticked"), || Duration::ZERO).status, CounterStatus::Unavailable); + let missing = Path::new("/no/such/powercap"); + assert_eq!(probe_root(missing, || panic!("ticked"), || Duration::ZERO), Host { packages: Vec::new(), status: CounterStatus::Unavailable }); + } + + /// The RAPL probe on scripted reads of `packages`, with a clock that + /// only the tick moves, by 25 ms. + fn scripted(packages: &[Package], read: impl Fn(usize) -> Option>) -> CounterStatus { let (clock, reads) = (Cell::new(Duration::ZERO), Cell::new(0usize)); status_with( + packages, || { reads.set(reads.get() + 1); read(reads.get() - 1) @@ -3817,24 +4329,35 @@ mod linux_rapl { /// A package counter that advances at every read is live; one that /// holds, steps once, or cannot be read (root-only `energy_uj`) is - /// declined, and so is one that reads only once. + /// declined, and so is one that reads only once. Of two packages, + /// both must advance. #[test] - fn the_rapl_probe_admits_only_a_counter_that_keeps_advancing() { - assert_eq!(scripted(|i| Some(81_000_000 + 1_400_000 * i as u64)), CounterStatus::Live); - assert_eq!(scripted(|_| Some(81_000_000)), CounterStatus::Frozen, "a held value"); - assert_eq!(scripted(|i| Some(if i < 9 { 1 } else { 2 })), CounterStatus::Frozen, "one step"); - assert_eq!(scripted(|_| None), CounterStatus::Unavailable, "EACCES on every read"); - assert_eq!(scripted(|i| (i == 0).then_some(5)), CounterStatus::Unavailable, "read once, then never"); + fn the_rapl_probe_admits_only_counters_that_keep_advancing() { + let one = [pkg(0, RANGE)]; + assert_eq!(scripted(&one, |i| Some(vec![81_000_000 + 1_400_000 * i as u64])), CounterStatus::Live); + assert_eq!(scripted(&one, |_| Some(vec![81_000_000])), CounterStatus::Frozen, "a held value"); + assert_eq!(scripted(&one, |i| Some(vec![if i < 9 { 1 } else { 2 }])), CounterStatus::Frozen, "one step"); + assert_eq!(scripted(&one, |_| None), CounterStatus::Unavailable, "EACCES on every read"); + assert_eq!(scripted(&one, |i| (i == 0).then(|| vec![5])), CounterStatus::Unavailable, "read once, then never"); + let two = [pkg(0, RANGE), pkg(1, RANGE)]; + assert_eq!(scripted(&two, |i| Some(vec![1_000 + 7 * i as u64, 9_000 + 5 * i as u64])), CounterStatus::Live); + assert_eq!(scripted(&two, |i| Some(vec![1_000 + 7 * i as u64, 9_000])), CounterStatus::Frozen, "package 1 held"); + assert_eq!(scripted(&two, |i| Some(vec![1_000, 9_000 + 5 * i as u64])), CounterStatus::Frozen, "package 0 held"); + assert_eq!(scripted(&[], |_| panic!("read a host with no package")), CounterStatus::Unavailable); } - /// The sampler hands the raw counter as one rail, and saturates - /// rather than wrapping a value past `i64::MAX`. + /// The sampler hands each package's raw counter as its own rail, and + /// saturates rather than wrapping a value past `i64::MAX`. #[test] - fn the_rapl_sample_is_the_raw_counter() { - assert_eq!(sample(|| Some(42)), Some(vec![("intel-rapl:0".to_string(), 42)])); - assert_eq!(sample(|| Some(u64::MAX)), Some(vec![("intel-rapl:0".to_string(), i64::MAX)])); - assert_eq!(sample(|| None), None); - assert_eq!(read_energy_uj(std::path::Path::new("/no/such/powercap/energy_uj")), None); + fn the_rapl_sample_is_each_package_counter_raw() { + let two = [pkg(0, RANGE), pkg(3, RANGE)]; + assert_eq!( + sample(&two, || Some(vec![42, u64::MAX])), + Some(vec![("intel-rapl:0".to_string(), 42), ("intel-rapl:3".to_string(), i64::MAX)]) + ); + assert_eq!(sample(&two, || None), None); + assert_eq!(sample(&two, || Some(vec![42])), None, "a read that missed a package"); + assert_eq!(read_u64(Path::new("/no/such/powercap/energy_uj")), None); } /// Only a live status constructs the counter. @@ -4037,8 +4560,8 @@ pub use nvml::NvmlCounter; /// Runtime-detect the best available counter on this host. /// /// Preference order: NVML (if compiled with `cuda`, present, and its counter -/// live — see [`nvml_status`]) → RAPL on Linux (only when its package counter -/// is live — see [`rapl_status`]) → IOReport on macOS (Energy-Model counters, +/// live — see [`nvml_status`]) → RAPL on Linux (only when every package +/// counter is live — see [`rapl_status`]) → IOReport on macOS (Energy-Model counters, /// no sudo; only when every CPU energy rail is live — see [`ioreport_status`]) /// → powermetrics on macOS → `StubCounter`. Every hardware rung is held to /// the same sustained-advance rule; a rung whose counter is not live is @@ -4136,27 +4659,30 @@ mod tests { assert_eq!(nvml_status(), CounterStatus::Unavailable); } - /// Against this host's RAPL counter, where there is one: the status the - /// probe cached agrees with what the counter does now. Live: it advances - /// across 200 ms of CPU work, and `detect()` selects it unless NVML is - /// live. Not live: `detect()` never selects it. Logs what it saw. + /// Against this host's RAPL package counters, where there are any: the + /// status the probe cached agrees with what they do now. Live: every one + /// advances across 200 ms of CPU work, and `detect()` selects RAPL unless + /// NVML is live. Not live: `detect()` never selects it. Logs what it saw. #[cfg(target_os = "linux")] #[test] fn the_rapl_rung_agrees_with_the_counter() { let status = rapl_status(); - let path = std::path::Path::new(linux_rapl::RAPL_ENERGY_PATH); - let before = linux_rapl::read_energy_uj(path); + let host = linux_rapl::host(); + let root = std::path::Path::new(linux_rapl::POWERCAP_ROOT); + let before = linux_rapl::read_packages(root, &host.packages); let started = std::time::Instant::now(); let mut acc: u64 = 0; while started.elapsed() < std::time::Duration::from_millis(200) { acc = std::hint::black_box(acc.wrapping_add(1)); } - let after = linux_rapl::read_energy_uj(path); + let after = linux_rapl::read_packages(root, &host.packages); let picked = detect().name(); - eprintln!("rapl: status={status:?} before={before:?} after={after:?} detect()={picked}"); + eprintln!("rapl: status={status:?} packages={:?} before={before:?} after={after:?} detect()={picked}", host.packages); if status == CounterStatus::Live { - let (b, a) = (before.expect("a live counter reads"), after.expect("a live counter reads")); - assert!(a != b, "a live RAPL counter did not move across 200 ms of CPU work: {b} -> {a}"); + let (b, a) = (before.expect("live counters read"), after.expect("live counters read")); + for ((p, b), a) in host.packages.iter().zip(&b).zip(&a) { + assert!(a != b, "live package zone intel-rapl:{} did not move across 200 ms of CPU work: {b} -> {a}", p.index); + } assert!(picked == "rapl" || nvml_status() == CounterStatus::Live, "detect() picked {picked}"); } else { assert_ne!(picked, "rapl", "detect() selected a {status:?} RAPL counter"); -- 2.54.0 (Apple Git-157) From 371fbe5eaac8517a910431f3c8c86fe2187dbdf2 Mon Sep 17 00:00:00 2001 From: Transaction Science Date: Wed, 30 Sep 2026 22:11:47 -0400 Subject: [PATCH 20/22] =?UTF-8?q?eoc-agent:=20a=20tool=20step=20the=20mete?= =?UTF-8?q?r=20did=20not=20measure=20is=20charged=20its=20duration=20at=20?= =?UTF-8?q?a=20stated=20power,=20never=200=20=C2=B5J?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit ReAct and CodeAct passed `measured` an estimate of 0 µJ for a tool step. Where the meter measured nothing (the `StubCounter` that `AgentLoop::new` builds every loop with, a failed read, a counter that held), the step was charged 0 and its span carried 0, for a tool or a block of code that ran. 52d9d1e2 left this open. This crate has no energy model for a tool, and what a model step is charged is the provider's own figure (`LlmReply::microjoules`), which a tool has no counterpart of. So a tool step's estimate is time by power: - `duration_estimate(elapsed, watts)`: elapsed times watts, in micro-joules, rounded. Never 0, because the step ran: under half a micro-joule is 1 µJ. A power that is not finite and positive gives way to the default. - `DEFAULT_TOOL_WATTS` = 15 W, about one busy core: a reference figure, said to be one, to be replaced by one calibrated on the host, as eoc-local's `DEFAULT_FALLBACK_JOULES_PER_TOKEN` is. - `AgentLoop::measured_with(estimate, f)` times the step on the monotonic clock and makes the estimate from its duration once it has run. `measured` is now `measured_with` with a fixed estimate, its behaviour unchanged, and `measured_tool(f)` is `measured_with` at the default power. - ReAct and CodeAct charge tool steps through `measured_tool`, under the rule `measured` already had: the meter's reading when both reads succeed and the counter advanced, or the estimate if that is larger; otherwise the estimate. A tool that fails ran too, and is charged for it. Tests: - the estimate: 2 s at 15 W is 30,000,000 µJ, 20 ms is 300,000, 10 µs at 2.5 W is 25; zero and 1 ns are 1 µJ; zero, negative, NaN and infinite power fall back to 15 W; `Duration::MAX` saturates; - `measured_tool` over a step of at least 20 ms charges at least 300,000 µJ with the stub, with a failed first read, and with a held counter (and less than the 9 J since-boot total), and charges a 50 J reading when the counter advanced by that; - ReAct with the stub meter and a tool of at least 20 ms: the act span carries at least 300,000 µJ, and the budget the two replies plus that; - CodeAct with the stub meter and a runtime of at least 20 ms, once succeeding and once failing: the same; - the existing failed-first-read ReAct test now expects the tool step's non-zero estimate on the span and in the budget (2,000 plus the span), still far below the 9 J since-boot total. Its budget is 5 J instead of 1 J, so a slow scheduler cannot push a microsecond echo's estimate past it (1 J is 67 ms at 15 W). Falsified, each mutant applied to this commit, run, and restored with `git checkout`: - `measured_tool`'s estimate back to 0: 4 tests fail; - ReAct back to `measured(0, ..)`: 2 fail; - CodeAct back to `measured(0, ..)`: 1 fails. `cargo test -p eoc-agent`: 29 unit and 9 integration tests pass, and the eoc workspace 686 passed, 0 failed, 3 ignored, on an M5 Max (macOS 27.0.1). `cargo check` and `cargo clippy --all-targets` for x86_64-unknown-linux-gnu pass; clippy's two collapsible-if warnings in the ReAct and CodeAct parsers predate this. Not done: Plan-and-Execute charges each worker step `charge_tool(0)` without metering it, the same defect in a third loop; and `Span` still has no field to say whether its figure was measured or estimated. Co-Authored-By: Claude Opus 5.5 --- eoc/eoc-rs/crates/eoc-agent/src/agent.rs | 86 +++++++++++++++++++++- eoc/eoc-rs/crates/eoc-agent/src/codeact.rs | 48 +++++++++++- eoc/eoc-rs/crates/eoc-agent/src/react.rs | 48 ++++++++++-- 3 files changed, 172 insertions(+), 10 deletions(-) diff --git a/eoc/eoc-rs/crates/eoc-agent/src/agent.rs b/eoc/eoc-rs/crates/eoc-agent/src/agent.rs index 28704b8fd..87b024a20 100644 --- a/eoc/eoc-rs/crates/eoc-agent/src/agent.rs +++ b/eoc/eoc-rs/crates/eoc-agent/src/agent.rs @@ -11,6 +11,7 @@ use std::collections::HashMap; use std::sync::Arc; +use std::time::{Duration, Instant}; use async_trait::async_trait; use eoc_meter::{JouleCounter, StubCounter}; @@ -98,6 +99,23 @@ impl ToolBox { } } +/// The power a tool step is taken to draw while it runs when no live counter +/// measured it, in watts: about one busy core. A reference figure, to be +/// replaced by one calibrated on the deployment host. +pub const DEFAULT_TOOL_WATTS: f64 = 15.0; + +/// The estimate of a step that ran for `elapsed` at `watts`, in micro-joules: +/// `elapsed` × `watts`, to the nearest micro-joule. Never 0, because the step +/// ran: a product under half a micro-joule is 1 µJ, and a power that is not a +/// finite positive number is not one to estimate with, so +/// [`DEFAULT_TOOL_WATTS`] stands in for it. +pub fn duration_estimate(elapsed: Duration, watts: f64) -> u64 { + let watts = if watts.is_finite() && watts > 0.0 { watts } else { DEFAULT_TOOL_WATTS }; + let uj = (elapsed.as_secs_f64() * watts * 1_000_000.0).round(); + // `as` saturates at u64::MAX. + (uj as u64).max(1) +} + /// Combined per-loop budget. #[derive(Debug, Clone, Copy, Serialize, Deserialize)] pub struct Budget { @@ -268,16 +286,43 @@ impl AgentLoop { where F: FnOnce() -> Fut, Fut: std::future::Future, + { + self.measured_with(|_| estimated_microjoules, f).await + } + + /// [`Self::measured`], with the step's estimate made once it has run, by + /// `estimate`, from how long it ran (monotonic wall-clock time). + pub async fn measured_with(&self, estimate: E, f: F) -> (T, u64) + where + E: FnOnce(Duration) -> u64, + F: FnOnce() -> Fut, + Fut: std::future::Future, { let start = self.read_meter(); + let started = Instant::now(); let out = f().await; + let estimated = estimate(started.elapsed()); let charged = match self.span_reading(start) { - Some(measured) => measured.max(estimated_microjoules), - None => estimated_microjoules, + Some(measured) => measured.max(estimated), + None => estimated, }; (out, charged) } + /// Run a tool step and say what to charge it, by the rule of + /// [`Self::measured`], with the step's estimate its duration at + /// [`DEFAULT_TOOL_WATTS`] ([`duration_estimate`]). A tool that ran did + /// work, so a step the meter did not measure — the [`StubCounter`], a + /// failed read, a counter that held — is charged that estimate, never + /// 0 µJ, and a tool that failed is charged for the time it ran too. + pub async fn measured_tool(&self, f: F) -> (T, u64) + where + F: FnOnce() -> Fut, + Fut: std::future::Future, + { + self.measured_with(|elapsed| duration_estimate(elapsed, DEFAULT_TOOL_WATTS), f).await + } + /// Append a span to the trace. pub fn push_span(&mut self, span: Span) { self.trace.push(span); @@ -407,6 +452,43 @@ mod tests { assert_eq!(stub.span_reading(stub.read_meter()), None); } + /// A step's duration estimate is its duration at the given power, and + /// never 0: a step too short to cost half a micro-joule costs 1 µJ, and a + /// power that is no power gives way to the default. + #[test] + fn a_duration_estimate_is_time_by_power_and_never_zero() { + assert_eq!(duration_estimate(Duration::from_secs(2), 15.0), 30_000_000); + assert_eq!(duration_estimate(Duration::from_millis(20), DEFAULT_TOOL_WATTS), 300_000); + assert_eq!(duration_estimate(Duration::from_micros(10), 2.5), 25); + assert_eq!(duration_estimate(Duration::ZERO, 15.0), 1); + assert_eq!(duration_estimate(Duration::from_nanos(1), 15.0), 1); + for no_power in [0.0, -3.0, f64::NAN, f64::INFINITY] { + assert_eq!(duration_estimate(Duration::from_secs(1), no_power), 15_000_000, "{no_power}"); + } + assert_eq!(duration_estimate(Duration::MAX, 15.0), u64::MAX); + } + + /// A tool step the meter did not measure is charged its duration at the + /// default power, never 0: here the stub, a failed first read, and a + /// counter that held, each over a step that runs at least 20 ms. A step + /// the meter measured is charged the reading, or the estimate when that + /// is larger. + #[tokio::test] + async fn a_tool_step_is_charged_at_least_its_duration_estimate() { + let floor = duration_estimate(Duration::from_millis(20), DEFAULT_TOOL_WATTS); + let step = || async { tokio::time::sleep(Duration::from_millis(20)).await }; + let stub = AgentLoop::new(Budget::unlimited(1)); + let (_, charged) = stub.measured_tool(step).await; + assert!(charged >= floor, "the stub: charged {charged} µJ for 20 ms of work"); + for script in [&[None, Some(9_000_000)][..], &[Some(7_000), Some(7_000)][..]] { + let loop_ = AgentLoop::with_meter(Budget::unlimited(1), Arc::new(Scripted::new(script))); + let (_, charged) = loop_.measured_tool(step).await; + assert!(charged >= floor && charged < 9_000_000, "{script:?}: charged {charged} µJ"); + } + let big = AgentLoop::with_meter(Budget::unlimited(1), Arc::new(Scripted::new(&[Some(0), Some(50_000_000)]))); + assert_eq!(big.measured_tool(step).await.1, 50_000_000, "a reading above the estimate"); + } + #[tokio::test] async fn budget_charge_llm_trips_token_cap() { let mut b = Budget::new(10, 1.0, 100, 100); diff --git a/eoc/eoc-rs/crates/eoc-agent/src/codeact.rs b/eoc/eoc-rs/crates/eoc-agent/src/codeact.rs index df5f02598..009147660 100644 --- a/eoc/eoc-rs/crates/eoc-agent/src/codeact.rs +++ b/eoc/eoc-rs/crates/eoc-agent/src/codeact.rs @@ -168,9 +168,11 @@ impl Agent for CodeActLoop { return Ok((last_output, StopReason::Error("no code block in reply".into()))); } }; + // Charged what the meter measured, or at least the step's + // duration estimate: code that ran is never 0 µJ. let (out_res, microjoules) = self .inner - .measured(0, || async { self.runtime.run(&code).await }) + .measured_tool(|| async { self.runtime.run(&code).await }) .await; let out = match out_res { Ok(o) => o, @@ -218,6 +220,50 @@ mod tests { assert!(CodeActLoop::extract_final("not done").is_none()); } + /// A provider that replies with a script, 1,000 µJ a reply. + struct Replies(std::sync::Mutex>); + + #[async_trait] + impl LlmProvider for Replies { + async fn complete(&self, _prompt: &str) -> AgentResult { + let text = self.0.lock().unwrap().remove(0); + Ok(crate::agent::LlmReply::new(text, 5, 1_000)) + } + } + + /// A runtime that takes at least 20 ms, and fails or not as told. + struct SlowRuntime(bool); + + #[async_trait] + impl CodeRuntime for SlowRuntime { + async fn run(&self, code: &str) -> AgentResult { + tokio::time::sleep(std::time::Duration::from_millis(20)).await; + if self.0 { + Err(AgentError::Tool { tool: "runtime".into(), reason: format!("failed: {code}") }) + } else { + Ok(format!("ran {code}")) + } + } + } + + /// Code the meter does not measure (the host default, the stub) is + /// charged its duration estimate — at least 20 ms at the default power for + /// a runtime that runs at least 20 ms — on the span and against the + /// budget, whether it succeeded or failed: it ran either way. + #[tokio::test] + async fn unmetered_code_is_charged_its_duration_estimate() { + let floor = crate::agent::duration_estimate(std::time::Duration::from_millis(20), crate::agent::DEFAULT_TOOL_WATTS); + for fails in [false, true] { + let provider = Arc::new(Replies(std::sync::Mutex::new(vec!["```\nnoop()\n```", "FINAL: done"]))); + let mut agent = CodeActLoop::new(provider, Arc::new(SlowRuntime(fails)), Budget::unlimited(10)); + let (out, stop) = agent.run("go").await.unwrap(); + assert_eq!((out.as_str(), stop), ("done", StopReason::Done)); + let act = agent.trace().spans.iter().find(|s| s.kind == SpanKind::Act).expect("an act span"); + assert!(act.microjoules >= floor, "fails={fails}: 20 ms of code charged {} µJ", act.microjoules); + assert_eq!(agent.budget().joules.used_microjoules, 2_000 + act.microjoules); + } + } + #[test] fn parse_line_lenient() { let (n, a) = LineLangRuntime::parse_line("calc(1+1)").unwrap(); diff --git a/eoc/eoc-rs/crates/eoc-agent/src/react.rs b/eoc/eoc-rs/crates/eoc-agent/src/react.rs index ba82fa342..606046ddc 100644 --- a/eoc/eoc-rs/crates/eoc-agent/src/react.rs +++ b/eoc/eoc-rs/crates/eoc-agent/src/react.rs @@ -160,9 +160,11 @@ impl Agent for ReActLoop { }; let args = serde_json::json!({ "input": step.argument }); + // Charged what the meter measured, or at least the step's + // duration estimate: a tool that ran is never 0 µJ. let (result, microjoules) = self .inner - .measured(0, || async { tool.run(args).await }) + .measured_tool(|| async { tool.run(args).await }) .await; let observation = match result { Ok(v) => v.to_string(), @@ -249,24 +251,56 @@ mod tests { } } + /// A tool that takes at least 20 ms and echoes its arguments. + struct Slow; + + #[async_trait] + impl crate::agent::Tool for Slow { + fn name(&self) -> &str { + "slow" + } + async fn run(&self, args: serde_json::Value) -> AgentResult { + tokio::time::sleep(std::time::Duration::from_millis(20)).await; + Ok(args) + } + } + /// A tool step whose first meter read fails is charged its own estimate - /// (0 µJ for a ReAct tool call), never the counter's since-boot total: - /// with a 1 J budget and a counter that reads 9 J once it reads at all, - /// the agent finishes, having charged only its two replies. + /// (its duration at the default power), never the counter's since-boot + /// total: with a 5 J budget and a counter that reads 9 J once it reads at + /// all, the agent finishes, having charged its two replies and the tool + /// step's estimate, which is not 0. #[tokio::test] async fn a_failed_first_read_never_exhausts_the_joule_budget() { let meter = Arc::new(crate::agent::scripted::Scripted::new(&[None, Some(9_000_000)])); let provider = Arc::new(Replies(std::sync::Mutex::new(vec!["Action: echo[hi]", "Action: Finish[done]"]))); let mut tools = ToolBox::new(); tools.insert(Echo); - let budget = Budget::new(1_000, 1.0, 10, 10); + let budget = Budget::new(1_000, 5.0, 10, 10); let mut agent = ReActLoop::new(provider, tools, budget, "system"); agent.inner = AgentLoop::with_meter(budget, meter.clone()); let (out, stop) = agent.run("go").await.unwrap(); assert_eq!((out.as_str(), stop), ("done", StopReason::Done)); assert_eq!(meter.reads(), 1, "the counter was read again after a failed first read"); - assert_eq!(agent.budget().joules.used_microjoules, 2_000, "{:?}", agent.trace().spans); let act = agent.trace().spans.iter().find(|s| s.kind == SpanKind::Act).expect("an act span"); - assert_eq!(act.microjoules, 0); + assert!(act.microjoules > 0 && act.microjoules < 9_000_000, "{:?}", act); + assert_eq!(agent.budget().joules.used_microjoules, 2_000 + act.microjoules, "{:?}", agent.trace().spans); + } + + /// A tool step the meter does not measure (the host default, the stub) is + /// charged its duration estimate: at least 20 ms at the default power for + /// a tool that runs at least 20 ms, on the span and against the budget. + #[tokio::test] + async fn an_unmetered_tool_step_is_charged_its_duration_estimate() { + let provider = Arc::new(Replies(std::sync::Mutex::new(vec!["Action: slow[hi]", "Action: Finish[done]"]))); + let mut tools = ToolBox::new(); + tools.insert(Slow); + let mut agent = ReActLoop::new(provider, tools, Budget::unlimited(10), "system"); + let (out, stop) = agent.run("go").await.unwrap(); + assert_eq!((out.as_str(), stop), ("done", StopReason::Done)); + let act = agent.trace().spans.iter().find(|s| s.kind == SpanKind::Act).expect("an act span"); + let floor = crate::agent::duration_estimate(std::time::Duration::from_millis(20), crate::agent::DEFAULT_TOOL_WATTS); + assert!(act.microjoules >= floor, "a 20 ms tool step charged {} µJ", act.microjoules); + assert_eq!(agent.budget().joules.used_microjoules, 2_000 + act.microjoules); } } -- 2.54.0 (Apple Git-157) From 6cf8b60e4bc3f02e832c69e1f2f11f85f3cd7a7d Mon Sep 17 00:00:00 2001 From: Transaction Science Date: Wed, 30 Sep 2026 22:31:39 -0400 Subject: [PATCH 21/22] =?UTF-8?q?eoc-agent:=20a=20Plan-and-Execute=20step?= =?UTF-8?q?=20is=20metered=20like=20a=20tool=20step,=20never=20charged=200?= =?UTF-8?q?=20=C2=B5J?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Plan-and-Execute charged each worker step `charge_tool(0)` and put no figure on its span, and its plan step was charged nothing at all, under a comment saying it was charged a notional cost. A planner or worker that ran was 0 µJ in the trace and against the budget. 371fbe5e fixed this for ReAct and CodeAct and left it open here. Neither a `Planner` nor a `Worker` reports what its step cost: the traits return subgoals and text, and `LlmPlanner` and `LlmWorker` drop the reply's figure. So each step is metered in the loop, through `measured_tool`, under the rule ReAct and CodeAct now follow: the meter's reading when both reads succeed and the counter advanced, or the step's duration at `DEFAULT_TOOL_WATTS` if that is larger; otherwise that duration estimate, which is never 0. - The plan step's span carries its figure, and the figure is charged against the joule cap. It is not a tool call, so it takes none of the tool-call quota. A plan step that spends the cap stops the loop before any worker runs. - Each worker step's span carries its figure, charged with `charge_tool`. - A step that fails ran too. Before, a planner or worker error was returned at once, unrecorded and uncharged; now the span records `ERROR: ...` with the step's figure, the budget is charged, and then the error is returned. - The comment claiming planners self-meter through `charge_llm_step` is gone: a `Planner` has no access to the loop and cannot. Tests: - with the stub meter, a planner and two workers that each take at least 20 ms: every span carries at least 300,000 µJ, the budget is charged exactly the trace's total, and two tool calls; - with a scripted meter: 40 J over the plan step and 50 J over worker `b` are charged as read, and worker `a`, over which the counter held, its own estimate (above 0, below 1 J for an echo); the budget is 90 J plus that; - a worker that fails after 20 ms: the error is returned, its span says `ERROR:` and carries at least 300,000 µJ, the budget is charged that and one tool call; - a 1 µJ joule cap stops the loop after the plan step, before any worker. Falsified, each mutant applied to this commit, run, and restored with `git checkout`: - a worker step charged `charge_tool(0)` again, its span keeping the figure: 3 tests fail; - a worker step's figure 0 on the span and in the charge: 3 fail; - the plan step's figure 0: 3 fail; - the plan step not charged against the budget: 4 fail; - a failed worker step returned before it is recorded and charged: 1 fails. `cargo test -p eoc-agent`: 33 unit and 9 integration tests pass. clippy on eoc-agent adds no warning; its two collapsible-if warnings in the ReAct and CodeAct parsers predate this. Co-Authored-By: Claude Opus 5.5 --- .../crates/eoc-agent/src/plan_execute.rs | 168 +++++++++++++++--- 1 file changed, 146 insertions(+), 22 deletions(-) diff --git a/eoc/eoc-rs/crates/eoc-agent/src/plan_execute.rs b/eoc/eoc-rs/crates/eoc-agent/src/plan_execute.rs index 71ae4f0d7..127de4dc4 100644 --- a/eoc/eoc-rs/crates/eoc-agent/src/plan_execute.rs +++ b/eoc/eoc-rs/crates/eoc-agent/src/plan_execute.rs @@ -12,7 +12,7 @@ use async_trait::async_trait; use crate::agent::{Agent, AgentLoop, Budget, LlmProvider, StopReason}; use crate::error::{AgentError, AgentResult}; -use crate::trace::SpanKind; +use crate::trace::{Span, SpanKind}; /// Decomposes a goal into ordered subgoals. #[async_trait] @@ -110,34 +110,47 @@ impl PlanExecuteLoop { #[async_trait] impl Agent for PlanExecuteLoop { + /// Neither a [`Planner`] nor a [`Worker`] reports what its step cost, so + /// each step is metered here, by the rule of [`AgentLoop::measured_tool`]: + /// what the meter measured over it, or at least its duration at + /// [`DEFAULT_TOOL_WATTS`](crate::agent::DEFAULT_TOOL_WATTS) — never 0 µJ + /// for a step that ran. The plan step is charged against the joule cap, + /// and each worker step against the joule cap and the tool-call quota. A + /// step that fails ran too: it is recorded and charged, and then its + /// error is returned. async fn run(&mut self, goal: &str) -> AgentResult<(String, StopReason)> { - let subgoals = self.planner.plan(goal).await?; - // Charge a notional cost for the planner step; planners that - // wrap LlmProvider directly already self-meter via charge_llm_step, - // but the trait-level abstraction does not. - self.inner.push_span(crate::trace::Span::new( - 0, - SpanKind::Plan, - "plan_execute.plan", - goal, - subgoals.join(" | "), - )); + let (planned, microjoules) = self.inner.measured_tool(|| self.planner.plan(goal)).await; + let output = match &planned { + Ok(subgoals) => subgoals.join(" | "), + Err(e) => format!("ERROR: {e}"), + }; + self.inner.push_span( + Span::new(0, SpanKind::Plan, "plan_execute.plan", goal, output).with_microjoules(microjoules), + ); + let within_cap = self.inner.budget.joules.charge(microjoules); + let subgoals = planned?; + if !within_cap { + return Ok((String::new(), StopReason::BudgetExhausted("joules".into()))); + } let mut prior = String::new(); for (i, sub) in subgoals.iter().enumerate() { if let Some(stop) = self.inner.iteration_check(i) { return Ok((prior, stop)); } - let out = self.worker.execute(sub, &prior).await?; - self.inner.push_span(crate::trace::Span::new( - 0, - SpanKind::Execute, - "plan_execute.execute", - sub, - out.clone(), - )); - // Charge tool-call quota for the worker step. - if let Some(stop) = self.inner.budget.charge_tool(0) { + let (executed, microjoules) = + self.inner.measured_tool(|| self.worker.execute(sub, &prior)).await; + let output = match &executed { + Ok(out) => out.clone(), + Err(e) => format!("ERROR: {e}"), + }; + self.inner.push_span( + Span::new(0, SpanKind::Execute, "plan_execute.execute", sub, output) + .with_microjoules(microjoules), + ); + let stop = self.inner.budget.charge_tool(microjoules); + let out = executed?; + if let Some(stop) = stop { return Ok((out, stop)); } prior = out; @@ -183,4 +196,115 @@ mod tests { assert_eq!(out, "|a|b|c"); assert_eq!(reason, StopReason::Done); } + + const STEP: std::time::Duration = std::time::Duration::from_millis(20); + + /// What a step of at least [`STEP`] is charged at least, unmetered. + fn floor() -> u64 { + crate::agent::duration_estimate(STEP, crate::agent::DEFAULT_TOOL_WATTS) + } + + /// A planner that takes at least [`STEP`] to plan `a`, `b`. + struct SlowPlanner; + #[async_trait] + impl Planner for SlowPlanner { + async fn plan(&self, _goal: &str) -> AgentResult> { + tokio::time::sleep(STEP).await; + Ok(vec!["a".into(), "b".into()]) + } + } + + /// A worker that takes at least [`STEP`], and fails or not as told. + struct SlowWorker(bool); + #[async_trait] + impl Worker for SlowWorker { + async fn execute(&self, subgoal: &str, prior: &str) -> AgentResult { + tokio::time::sleep(STEP).await; + if self.0 { + Err(AgentError::Tool { tool: "worker".into(), reason: format!("failed: {subgoal}") }) + } else { + Ok(format!("{prior}|{subgoal}")) + } + } + } + + /// Steps the meter does not measure (the host default, the stub) are + /// charged their duration estimates, never 0: the plan step and each + /// worker step, each running at least 20 ms, carry at least 20 ms at the + /// default power on their spans, and the budget is charged exactly what + /// the spans carry, with one tool call a worker step. + #[tokio::test] + async fn unmetered_steps_are_charged_their_duration_estimates() { + let mut pe = PlanExecuteLoop::new(Arc::new(SlowPlanner), Arc::new(SlowWorker(false)), Budget::unlimited(10)); + let (out, stop) = pe.run("go").await.unwrap(); + assert_eq!((out.as_str(), stop), ("|a|b", StopReason::Done)); + let spans = &pe.trace().spans; + let kinds: Vec<_> = spans.iter().map(|s| s.kind).collect(); + assert_eq!(kinds, [SpanKind::Plan, SpanKind::Execute, SpanKind::Execute]); + for span in spans { + assert!(span.microjoules >= floor(), "a 20 ms step charged {} µJ: {span:?}", span.microjoules); + } + assert_eq!(pe.budget().joules.used_microjoules, pe.trace().total_microjoules()); + assert_eq!(pe.budget().tool_calls_used, 2); + } + + /// A step the meter measured is charged the counter's reading: 40 J over + /// the plan step and 50 J over worker `b`. Over worker `a` the counter + /// held, so `a` is charged its own estimate, which is not 0 and is far + /// below a joule for an echo. + #[tokio::test] + async fn a_metered_step_is_charged_the_counters_reading() { + let meter = Arc::new(crate::agent::scripted::Scripted::new(&[ + Some(0), + Some(40_000_000), + Some(40_000_000), + Some(40_000_000), + Some(40_000_000), + Some(90_000_000), + ])); + let planner = Arc::new(StaticPlanner(vec!["a".into(), "b".into()])); + let budget = Budget::unlimited(10); + let mut pe = PlanExecuteLoop::new(planner, Arc::new(EchoWorker), budget); + pe.inner = AgentLoop::with_meter(budget, meter.clone()); + let (out, stop) = pe.run("go").await.unwrap(); + assert_eq!((out.as_str(), stop), ("|a|b", StopReason::Done)); + assert_eq!(meter.reads(), 6); + let spans = &pe.trace().spans; + assert_eq!(spans[0].microjoules, 40_000_000, "{spans:?}"); + assert!((1..1_000_000).contains(&spans[1].microjoules), "{spans:?}"); + assert_eq!(spans[2].microjoules, 50_000_000, "{spans:?}"); + assert_eq!(pe.budget().joules.used_microjoules, 90_000_000 + spans[1].microjoules); + } + + /// A worker step that fails ran too: its span records the error and its + /// duration estimate, the budget is charged that and a tool call, and + /// then the error is returned. + #[tokio::test] + async fn a_failed_worker_step_is_recorded_and_charged() { + let planner = Arc::new(StaticPlanner(vec!["a".into(), "b".into()])); + let mut pe = PlanExecuteLoop::new(planner, Arc::new(SlowWorker(true)), Budget::unlimited(10)); + let err = pe.run("go").await.unwrap_err(); + assert!(matches!(err, AgentError::Tool { .. }), "{err:?}"); + let last = pe.trace().spans.last().expect("a span"); + assert_eq!(last.kind, SpanKind::Execute); + assert!(last.output.starts_with("ERROR: "), "{last:?}"); + assert!(last.microjoules >= floor(), "a failed 20 ms step charged {} µJ", last.microjoules); + assert_eq!(pe.budget().joules.used_microjoules, pe.trace().total_microjoules()); + assert_eq!(pe.budget().tool_calls_used, 1); + } + + /// A joule cap the plan step already spends stops the loop before any + /// worker runs. + #[tokio::test] + async fn a_plan_step_over_the_joule_cap_stops_the_loop() { + let mut pe = PlanExecuteLoop::new( + Arc::new(SlowPlanner), + Arc::new(SlowWorker(false)), + Budget::new(1_000, 0.000_001, 10, 10), + ); + let (out, stop) = pe.run("go").await.unwrap(); + assert_eq!((out.as_str(), stop), ("", StopReason::BudgetExhausted("joules".into()))); + assert_eq!(pe.trace().len(), 1); + assert_eq!(pe.budget().tool_calls_used, 0); + } } -- 2.54.0 (Apple Git-157) From 1352aeea07a77d26e91fc6d53e0fe327155dd692 Mon Sep 17 00:00:00 2001 From: Transaction Science Date: Wed, 30 Sep 2026 22:35:01 -0400 Subject: [PATCH 22/22] eoc-agent: every span says whether its joule figure was measured or estimated MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit A trace `Span` carried a micro-joule figure and nothing about where it came from. A provider's own figure, a tool step's duration estimate, a zero for an observation, and a delta a live counter read all looked the same, so nothing reading a trace could tell a measurement from an estimate. Every other eoc surface that reports energy says which it is (`JouleCost`, eoc-eval's `joule_source`); an agent trace did not. `Span` gains `joule_source: JouleSource`, eoc-core's tag, under eoc-core's rule: `Measured` only for what a live counter read over the step — both reads succeeded and the counter advanced — and `Estimated` for anything else. - The field is additive on the wire. It is `#[serde(default)]`, as eoc-eval's report field is: a trace written before it reads back with every span `Estimated`, since nothing in it says it was measured. Nothing in the monorepo outside eoc-agent uses eoc-agent, so no struct literal of `Span` breaks. - `Span::new` is `Estimated` (0 µJ that no counter read). `with_joule_cost` attaches a figure and its tag together, and `joule_cost()` reads them back. `with_microjoules` attaches a bare figure, which is an estimate: it no longer leaves an earlier `Measured` tag on a figure the counter did not read. - `AgentLoop::measured_with` and `measured_tool` return a `JouleCost` instead of a bare `u64`. The figure is unchanged — the reading, or the estimate when larger, or the estimate alone — and it is `Measured` exactly when it is the reading: a reading smaller than the estimate is not what is charged, so the estimate charged in its place is `Estimated`. `measured` still returns the bare figure, as on main. Both changed functions are new on this branch. - Model steps: `charge_llm_step` tags the provider's own figure `Estimated`; `LlmProvider`'s contract calls it an estimate and no counter here read it. This covers every loop's model steps (ReAct, CodeAct, Reflexion, ToT, LATS). - Tool steps: ReAct and CodeAct put `measured_tool`'s tag on the act span, and Plan-and-Execute on its plan and execute spans. - Other steps: ReAct's observation of an unknown tool is `Span::new`'s estimated 0, as nothing ran. Tests: - a new span and a bare figure are estimates; a measured cost survives `with_joule_cost`, and a bare figure attached after it is an estimate; - a trace round-trips through JSON with a `Measured` and an `Estimated` span, and a trace in the old format, without the field, reads back all `Estimated`; - `measured_tool`: the stub, a failed first or last read, a counter that held or went down, and one that advanced 10 µJ over a 20 ms step are all `Estimated` and charged at least the 300,000 µJ estimate; a 50 J reading is `Measured`. `measured_with` at a fixed estimate: a 450 µJ reading is `Measured` at estimates of 100 and 450, and an estimate of 1,000 is charged as `Estimated`; - `charge_llm_step` records the reply's figure as `Estimated`; - ReAct, with a counter that advances 50 J over the tool step: think spans `Estimated` at 1,000 µJ, the unknown tool's observation `Estimated` at 0, the act span `Measured` at 50 J, and the budget 3,000 µJ plus 50 J; with the stub, every span `Estimated`; - CodeAct, the same over code that succeeds and code that fails; - Plan-and-Execute: with the stub every span `Estimated`; with the scripted meter the plan step (40 J) and worker `b` (50 J) `Measured`, and worker `a`, over which the counter held, `Estimated`. Falsified, each mutant applied to this commit, run, and restored with `git checkout`: - every span pushed to a trace tagged `Measured`: 8 tests fail; - `Span::new` and `with_joule_cost` always `Measured`: 9 fail; - every span pushed to a trace tagged `Estimated`: 4 fail; - a span without the field reading back `Measured`: 1 fails; - `measured_with` tagging any reading `Measured`, including when the estimate is what is charged: 1 fails; - `charge_llm_step` tagging the provider's figure `Measured`: 5 fail; - `with_microjoules` keeping an earlier tag: 1 fails; - ReAct, CodeAct, and Plan-and-Execute each dropping the tag (`with_microjoules(cost.microjoules)`): 1 fails in each. `cargo test -p eoc-agent`: 38 unit and 9 integration tests pass. The eoc workspace (`cargo test --workspace`): 695 passed, 0 failed, 3 ignored, on an M5 Max (macOS 27.0.1). clippy for x86_64-unknown-linux-gnu on eoc-agent: the same two pre-existing warnings and no others. On macOS too clippy adds no warning (the two collapsible-if warnings predate this), and rustdoc reports nothing in the lines changed here. Co-Authored-By: Claude Opus 5.5 --- eoc/eoc-rs/crates/eoc-agent/src/agent.rs | 85 +++++++++++++++---- eoc/eoc-rs/crates/eoc-agent/src/codeact.rs | 40 ++++++++- .../crates/eoc-agent/src/plan_execute.rs | 31 ++++--- eoc/eoc-rs/crates/eoc-agent/src/react.rs | 46 +++++++++- eoc/eoc-rs/crates/eoc-agent/src/trace.rs | 79 +++++++++++++++-- 5 files changed, 238 insertions(+), 43 deletions(-) diff --git a/eoc/eoc-rs/crates/eoc-agent/src/agent.rs b/eoc/eoc-rs/crates/eoc-agent/src/agent.rs index 87b024a20..594624aa1 100644 --- a/eoc/eoc-rs/crates/eoc-agent/src/agent.rs +++ b/eoc/eoc-rs/crates/eoc-agent/src/agent.rs @@ -14,6 +14,7 @@ use std::sync::Arc; use std::time::{Duration, Instant}; use async_trait::async_trait; +use eoc_core::JouleCost; use eoc_meter::{JouleCounter, StubCounter}; use serde::{Deserialize, Serialize}; use serde_json::Value; @@ -281,18 +282,23 @@ impl AgentLoop { /// that and `estimated_microjoules`, the step's own estimate; and when /// the meter measured nothing, the estimate alone. The same rule /// eoc-local's backends and eoc-streaming meter by: a span is measured - /// only when both reads succeed and the counter advanced. + /// only when both reads succeed and the counter advanced. The figure + /// alone; [`Self::measured_with`] also says whether it was measured. pub async fn measured(&self, estimated_microjoules: u64, f: F) -> (T, u64) where F: FnOnce() -> Fut, Fut: std::future::Future, { - self.measured_with(|_| estimated_microjoules, f).await + let (out, cost) = self.measured_with(|_| estimated_microjoules, f).await; + (out, cost.microjoules) } /// [`Self::measured`], with the step's estimate made once it has run, by - /// `estimate`, from how long it ran (monotonic wall-clock time). - pub async fn measured_with(&self, estimate: E, f: F) -> (T, u64) + /// `estimate`, from how long it ran (monotonic wall-clock time), and the + /// figure's provenance: [`JouleCost::measured`] when the figure is what + /// the meter read, [`JouleCost::estimated`] when it is the estimate — + /// because the meter measured nothing, or read less than the estimate. + pub async fn measured_with(&self, estimate: E, f: F) -> (T, JouleCost) where E: FnOnce(Duration) -> u64, F: FnOnce() -> Fut, @@ -303,19 +309,20 @@ impl AgentLoop { let out = f().await; let estimated = estimate(started.elapsed()); let charged = match self.span_reading(start) { - Some(measured) => measured.max(estimated), - None => estimated, + Some(measured) if measured >= estimated => JouleCost::measured(measured), + _ => JouleCost::estimated(estimated), }; (out, charged) } - /// Run a tool step and say what to charge it, by the rule of - /// [`Self::measured`], with the step's estimate its duration at - /// [`DEFAULT_TOOL_WATTS`] ([`duration_estimate`]). A tool that ran did - /// work, so a step the meter did not measure — the [`StubCounter`], a - /// failed read, a counter that held — is charged that estimate, never - /// 0 µJ, and a tool that failed is charged for the time it ran too. - pub async fn measured_tool(&self, f: F) -> (T, u64) + /// Run a tool step and say what to charge it, and whether that was + /// measured, by the rule of [`Self::measured_with`], with the step's + /// estimate its duration at [`DEFAULT_TOOL_WATTS`] + /// ([`duration_estimate`]). A tool that ran did work, so a step the meter + /// did not measure — the [`StubCounter`], a failed read, a counter that + /// held — is charged that estimate, never 0 µJ, and a tool that failed is + /// charged for the time it ran too. + pub async fn measured_tool(&self, f: F) -> (T, JouleCost) where F: FnOnce() -> Fut, Fut: std::future::Future, @@ -330,6 +337,10 @@ impl AgentLoop { /// Charge an LLM step against the budget *and* record the span. /// Returns the stop reason if the budget is exhausted. + /// + /// The step's figure is the provider's own, [`LlmReply::microjoules`], + /// an estimate by [`LlmProvider`]'s contract that no counter here read, + /// so the span is [`JouleSource::Estimated`](eoc_core::JouleSource). pub fn charge_llm_step( &mut self, kind: SpanKind, @@ -339,7 +350,7 @@ impl AgentLoop { ) -> Option { let span = Span::new(self.trace.len(), kind, label, input, reply.text.clone()) .with_tokens(reply.tokens) - .with_microjoules(reply.microjoules); + .with_joule_cost(JouleCost::estimated(reply.microjoules)); self.push_span(span); self.budget.charge_llm(reply.tokens, reply.microjoules) } @@ -403,6 +414,7 @@ pub(crate) mod scripted { mod tests { use super::scripted::Scripted; use super::*; + use eoc_core::JouleSource; /// What `measured` charges for one step with the estimate `estimate`, /// metered by a counter reading `script`, and how many reads it took. @@ -479,14 +491,51 @@ mod tests { let step = || async { tokio::time::sleep(Duration::from_millis(20)).await }; let stub = AgentLoop::new(Budget::unlimited(1)); let (_, charged) = stub.measured_tool(step).await; - assert!(charged >= floor, "the stub: charged {charged} µJ for 20 ms of work"); + assert!(charged.microjoules >= floor, "the stub: charged {charged} for 20 ms of work"); for script in [&[None, Some(9_000_000)][..], &[Some(7_000), Some(7_000)][..]] { let loop_ = AgentLoop::with_meter(Budget::unlimited(1), Arc::new(Scripted::new(script))); let (_, charged) = loop_.measured_tool(step).await; - assert!(charged >= floor && charged < 9_000_000, "{script:?}: charged {charged} µJ"); + assert!(charged.microjoules >= floor && charged.microjoules < 9_000_000, "{script:?}: charged {charged}"); } let big = AgentLoop::with_meter(Budget::unlimited(1), Arc::new(Scripted::new(&[Some(0), Some(50_000_000)]))); - assert_eq!(big.measured_tool(step).await.1, 50_000_000, "a reading above the estimate"); + assert_eq!(big.measured_tool(step).await.1.microjoules, 50_000_000, "a reading above the estimate"); + } + + /// A step's figure is measured only when it is what a live counter read: + /// the counter advanced over the step, by at least the step's estimate. + /// The stub, a failed first or last read, a counter that held or went + /// down, and a counter that advanced by less than the estimate (10 µJ + /// over a 20 ms step, so the estimate is charged) all give an estimate. + #[tokio::test] + async fn a_step_is_measured_only_when_charged_a_live_counters_reading() { + let floor = duration_estimate(Duration::from_millis(20), DEFAULT_TOOL_WATTS); + let step = || async { tokio::time::sleep(Duration::from_millis(20)).await }; + let (_, stub) = AgentLoop::new(Budget::unlimited(1)).measured_tool(step).await; + assert_eq!(stub.source, JouleSource::Estimated, "the stub: {stub}"); + for script in [ + &[None, Some(9_000_000)][..], + &[Some(7_000), None][..], + &[Some(7_000), Some(7_000)][..], + &[Some(7_000), Some(6_000)][..], + &[Some(0), Some(10)][..], + ] { + let loop_ = AgentLoop::with_meter(Budget::unlimited(1), Arc::new(Scripted::new(script))); + let (_, cost) = loop_.measured_tool(step).await; + assert_eq!(cost.source, JouleSource::Estimated, "{script:?}: {cost}"); + assert!(cost.microjoules >= floor, "{script:?}: {cost}"); + } + let big = AgentLoop::with_meter(Budget::unlimited(1), Arc::new(Scripted::new(&[Some(0), Some(50_000_000)]))); + assert_eq!(big.measured_tool(step).await.1, JouleCost::measured(50_000_000)); + // A fixed estimate: the reading when it is at least the estimate. + for (script, estimate, cost) in [ + (&[Some(1_000), Some(1_450)], 100, JouleCost::measured(450)), + (&[Some(1_000), Some(1_450)], 450, JouleCost::measured(450)), + (&[Some(1_000), Some(1_450)], 1_000, JouleCost::estimated(1_000)), + (&[Some(7_000), Some(7_000)], 250, JouleCost::estimated(250)), + ] { + let loop_ = AgentLoop::with_meter(Budget::unlimited(1), Arc::new(Scripted::new(script))); + assert_eq!(loop_.measured_with(|_| estimate, || async {}).await.1, cost, "{script:?} {estimate}"); + } } #[tokio::test] @@ -504,5 +553,7 @@ mod tests { assert!(stop.is_none()); assert_eq!(loop_.trace.len(), 1); assert_eq!(loop_.trace.total_tokens(), 3); + // The provider's own figure: an estimate. + assert_eq!(loop_.trace.spans[0].joule_cost(), JouleCost::estimated(100)); } } diff --git a/eoc/eoc-rs/crates/eoc-agent/src/codeact.rs b/eoc/eoc-rs/crates/eoc-agent/src/codeact.rs index 009147660..004bec521 100644 --- a/eoc/eoc-rs/crates/eoc-agent/src/codeact.rs +++ b/eoc/eoc-rs/crates/eoc-agent/src/codeact.rs @@ -169,8 +169,9 @@ impl Agent for CodeActLoop { } }; // Charged what the meter measured, or at least the step's - // duration estimate: code that ran is never 0 µJ. - let (out_res, microjoules) = self + // duration estimate: code that ran is never 0 µJ. The span says + // which. + let (out_res, cost) = self .inner .measured_tool(|| async { self.runtime.run(&code).await }) .await; @@ -178,12 +179,12 @@ impl Agent for CodeActLoop { Ok(o) => o, Err(e) => format!("runtime_error: {e}"), }; - if let Some(stop) = self.inner.budget.charge_tool(microjoules) { + if let Some(stop) = self.inner.budget.charge_tool(cost.microjoules) { return Ok((last_output, stop)); } self.inner.push_span( crate::trace::Span::new(0, SpanKind::Act, "codeact.exec", code.clone(), out.clone()) - .with_microjoules(microjoules), + .with_joule_cost(cost), ); transcript.push_str(&format!("CODE:\n{code}\nOBSERVATION:\n{out}\n")); } @@ -202,6 +203,7 @@ impl Agent for CodeActLoop { #[cfg(test)] mod tests { use super::*; + use eoc_core::{JouleCost, JouleSource}; #[test] fn extract_code_finds_block() { @@ -261,6 +263,36 @@ mod tests { let act = agent.trace().spans.iter().find(|s| s.kind == SpanKind::Act).expect("an act span"); assert!(act.microjoules >= floor, "fails={fails}: 20 ms of code charged {} µJ", act.microjoules); assert_eq!(agent.budget().joules.used_microjoules, 2_000 + act.microjoules); + for span in &agent.trace().spans { + assert_eq!(span.joule_source, JouleSource::Estimated, "fails={fails}: {span:?}"); + } + } + } + + /// Code over which the counter advanced 50 J is measured, and charged + /// that, whether it succeeded or failed; the model steps around it (the + /// provider's own figures) are estimates. + #[tokio::test] + async fn metered_code_is_measured_and_model_steps_are_estimated() { + for fails in [false, true] { + let meter = Arc::new(crate::agent::scripted::Scripted::new(&[Some(0), Some(50_000_000)])); + let provider = Arc::new(Replies(std::sync::Mutex::new(vec!["```\nnoop()\n```", "FINAL: done"]))); + let budget = Budget::unlimited(10); + let mut agent = CodeActLoop::new(provider, Arc::new(SlowRuntime(fails)), budget); + agent.inner = AgentLoop::with_meter(budget, meter); + let (out, stop) = agent.run("go").await.unwrap(); + assert_eq!((out.as_str(), stop), ("done", StopReason::Done)); + let costs: Vec<_> = agent.trace().spans.iter().map(|s| (s.kind, s.joule_cost())).collect(); + assert_eq!( + costs, + [ + (SpanKind::Think, JouleCost::estimated(1_000)), + (SpanKind::Act, JouleCost::measured(50_000_000)), + (SpanKind::Think, JouleCost::estimated(1_000)), + ], + "fails={fails}" + ); + assert_eq!(agent.budget().joules.used_microjoules, 2_000 + 50_000_000); } } diff --git a/eoc/eoc-rs/crates/eoc-agent/src/plan_execute.rs b/eoc/eoc-rs/crates/eoc-agent/src/plan_execute.rs index 127de4dc4..361abe758 100644 --- a/eoc/eoc-rs/crates/eoc-agent/src/plan_execute.rs +++ b/eoc/eoc-rs/crates/eoc-agent/src/plan_execute.rs @@ -114,20 +114,21 @@ impl Agent for PlanExecuteLoop { /// each step is metered here, by the rule of [`AgentLoop::measured_tool`]: /// what the meter measured over it, or at least its duration at /// [`DEFAULT_TOOL_WATTS`](crate::agent::DEFAULT_TOOL_WATTS) — never 0 µJ - /// for a step that ran. The plan step is charged against the joule cap, + /// for a step that ran — and its span says which of the two it is + /// charged. The plan step is charged against the joule cap, /// and each worker step against the joule cap and the tool-call quota. A /// step that fails ran too: it is recorded and charged, and then its /// error is returned. async fn run(&mut self, goal: &str) -> AgentResult<(String, StopReason)> { - let (planned, microjoules) = self.inner.measured_tool(|| self.planner.plan(goal)).await; + let (planned, cost) = self.inner.measured_tool(|| self.planner.plan(goal)).await; let output = match &planned { Ok(subgoals) => subgoals.join(" | "), Err(e) => format!("ERROR: {e}"), }; self.inner.push_span( - Span::new(0, SpanKind::Plan, "plan_execute.plan", goal, output).with_microjoules(microjoules), + Span::new(0, SpanKind::Plan, "plan_execute.plan", goal, output).with_joule_cost(cost), ); - let within_cap = self.inner.budget.joules.charge(microjoules); + let within_cap = self.inner.budget.joules.charge(cost.microjoules); let subgoals = planned?; if !within_cap { return Ok((String::new(), StopReason::BudgetExhausted("joules".into()))); @@ -138,7 +139,7 @@ impl Agent for PlanExecuteLoop { if let Some(stop) = self.inner.iteration_check(i) { return Ok((prior, stop)); } - let (executed, microjoules) = + let (executed, cost) = self.inner.measured_tool(|| self.worker.execute(sub, &prior)).await; let output = match &executed { Ok(out) => out.clone(), @@ -146,9 +147,9 @@ impl Agent for PlanExecuteLoop { }; self.inner.push_span( Span::new(0, SpanKind::Execute, "plan_execute.execute", sub, output) - .with_microjoules(microjoules), + .with_joule_cost(cost), ); - let stop = self.inner.budget.charge_tool(microjoules); + let stop = self.inner.budget.charge_tool(cost.microjoules); let out = executed?; if let Some(stop) = stop { return Ok((out, stop)); @@ -170,6 +171,7 @@ impl Agent for PlanExecuteLoop { #[cfg(test)] mod tests { use super::*; + use eoc_core::{JouleCost, JouleSource}; struct StaticPlanner(Vec); #[async_trait] @@ -243,15 +245,17 @@ mod tests { assert_eq!(kinds, [SpanKind::Plan, SpanKind::Execute, SpanKind::Execute]); for span in spans { assert!(span.microjoules >= floor(), "a 20 ms step charged {} µJ: {span:?}", span.microjoules); + assert_eq!(span.joule_source, JouleSource::Estimated, "{span:?}"); } assert_eq!(pe.budget().joules.used_microjoules, pe.trace().total_microjoules()); assert_eq!(pe.budget().tool_calls_used, 2); } - /// A step the meter measured is charged the counter's reading: 40 J over - /// the plan step and 50 J over worker `b`. Over worker `a` the counter - /// held, so `a` is charged its own estimate, which is not 0 and is far - /// below a joule for an echo. + /// A step the meter measured is charged the counter's reading, and its + /// span says measured: 40 J over the plan step and 50 J over worker `b`. + /// Over worker `a` the counter held, so `a` is charged its own estimate, + /// which is not 0 and is far below a joule for an echo, and says + /// estimated. #[tokio::test] async fn a_metered_step_is_charged_the_counters_reading() { let meter = Arc::new(crate::agent::scripted::Scripted::new(&[ @@ -270,9 +274,10 @@ mod tests { assert_eq!((out.as_str(), stop), ("|a|b", StopReason::Done)); assert_eq!(meter.reads(), 6); let spans = &pe.trace().spans; - assert_eq!(spans[0].microjoules, 40_000_000, "{spans:?}"); + assert_eq!(spans[0].joule_cost(), JouleCost::measured(40_000_000), "{spans:?}"); assert!((1..1_000_000).contains(&spans[1].microjoules), "{spans:?}"); - assert_eq!(spans[2].microjoules, 50_000_000, "{spans:?}"); + assert_eq!(spans[1].joule_source, JouleSource::Estimated, "{spans:?}"); + assert_eq!(spans[2].joule_cost(), JouleCost::measured(50_000_000), "{spans:?}"); assert_eq!(pe.budget().joules.used_microjoules, 90_000_000 + spans[1].microjoules); } diff --git a/eoc/eoc-rs/crates/eoc-agent/src/react.rs b/eoc/eoc-rs/crates/eoc-agent/src/react.rs index 606046ddc..b7391730c 100644 --- a/eoc/eoc-rs/crates/eoc-agent/src/react.rs +++ b/eoc/eoc-rs/crates/eoc-agent/src/react.rs @@ -161,8 +161,9 @@ impl Agent for ReActLoop { let args = serde_json::json!({ "input": step.argument }); // Charged what the meter measured, or at least the step's - // duration estimate: a tool that ran is never 0 µJ. - let (result, microjoules) = self + // duration estimate: a tool that ran is never 0 µJ. The span + // says which. + let (result, cost) = self .inner .measured_tool(|| async { tool.run(args).await }) .await; @@ -170,7 +171,7 @@ impl Agent for ReActLoop { Ok(v) => v.to_string(), Err(e) => format!("ERROR: {e}"), }; - if let Some(stop) = self.inner.budget.charge_tool(microjoules) { + if let Some(stop) = self.inner.budget.charge_tool(cost.microjoules) { return Ok((self.last_output.clone(), stop)); } self.inner.push_span( @@ -181,7 +182,7 @@ impl Agent for ReActLoop { step.argument.clone(), observation.clone(), ) - .with_microjoules(microjoules), + .with_joule_cost(cost), ); self.transcript.push_str(&format!( "Thought: {th}\nAction: {ac}[{arg}]\nObservation: {obs}\n", @@ -206,6 +207,7 @@ impl Agent for ReActLoop { #[cfg(test)] mod tests { use super::*; + use eoc_core::{JouleCost, JouleSource}; #[test] fn parse_thought_action() { @@ -302,5 +304,41 @@ mod tests { let floor = crate::agent::duration_estimate(std::time::Duration::from_millis(20), crate::agent::DEFAULT_TOOL_WATTS); assert!(act.microjoules >= floor, "a 20 ms tool step charged {} µJ", act.microjoules); assert_eq!(agent.budget().joules.used_microjoules, 2_000 + act.microjoules); + for span in &agent.trace().spans { + assert_eq!(span.joule_source, JouleSource::Estimated, "{span:?}"); + } + } + + /// Each span says whether its figure was measured: a tool step over which + /// the counter advanced 50 J is measured, and charged that; a model step + /// (the provider's own figure) and an unknown tool's observation (nothing + /// ran, 0 µJ) are estimates. + #[tokio::test] + async fn a_span_is_measured_only_when_a_live_counter_read_it() { + let meter = Arc::new(crate::agent::scripted::Scripted::new(&[Some(0), Some(50_000_000)])); + let provider = Arc::new(Replies(std::sync::Mutex::new(vec![ + "Action: nope[x]", + "Action: echo[hi]", + "Action: Finish[done]", + ]))); + let mut tools = ToolBox::new(); + tools.insert(Echo); + let budget = Budget::unlimited(10); + let mut agent = ReActLoop::new(provider, tools, budget, "system"); + agent.inner = AgentLoop::with_meter(budget, meter.clone()); + let (out, stop) = agent.run("go").await.unwrap(); + assert_eq!((out.as_str(), stop), ("done", StopReason::Done)); + let costs: Vec<_> = agent.trace().spans.iter().map(|s| (s.kind, s.joule_cost())).collect(); + assert_eq!( + costs, + [ + (SpanKind::Think, JouleCost::estimated(1_000)), + (SpanKind::Observe, JouleCost::estimated(0)), + (SpanKind::Think, JouleCost::estimated(1_000)), + (SpanKind::Act, JouleCost::measured(50_000_000)), + (SpanKind::Think, JouleCost::estimated(1_000)), + ] + ); + assert_eq!(agent.budget().joules.used_microjoules, 3_000 + 50_000_000); } } diff --git a/eoc/eoc-rs/crates/eoc-agent/src/trace.rs b/eoc/eoc-rs/crates/eoc-agent/src/trace.rs index 2f1d8f6d5..6791af0a9 100644 --- a/eoc/eoc-rs/crates/eoc-agent/src/trace.rs +++ b/eoc/eoc-rs/crates/eoc-agent/src/trace.rs @@ -3,10 +3,12 @@ //! Every loop emits a [`Trace`] consisting of ordered [`Span`]s. A span //! captures *what kind of step* ran (think / act / observe / reflect), //! its inputs and outputs as opaque strings, and the joule + token cost -//! attributed to it. Traces are content-addressable via the existing +//! attributed to it, with whether that joule figure was measured or +//! estimated. Traces are content-addressable via the existing //! [`eoc_core::Receipt`] machinery — the cascade can memoize entire //! agent runs, not just individual responses. +use eoc_core::{JouleCost, JouleSource}; use serde::{Deserialize, Serialize}; /// Kind of step. @@ -60,10 +62,25 @@ pub struct Span { pub tokens: u64, /// Energy charged to this span (micro-joules). pub microjoules: u64, + /// Where `microjoules` came from: [`JouleSource::Measured`] only when it + /// is what a live hardware counter read over the step — both reads + /// succeeded and the counter advanced — and [`JouleSource::Estimated`] + /// otherwise: a provider's own figure, a duration estimate, a zero no + /// counter read. A span serialized without this field reads back as + /// `Estimated`: nothing in it says it was measured. + #[serde(default = "unrecorded_joule_source")] + pub joule_source: JouleSource, +} + +/// The provenance of a span that recorded none: an estimate, never a +/// measurement. +fn unrecorded_joule_source() -> JouleSource { + JouleSource::Estimated } impl Span { - /// Build a new span. + /// Build a new span, charged nothing yet: 0 µJ, which no counter read, + /// so [`JouleSource::Estimated`]. pub fn new( index: usize, kind: SpanKind, @@ -79,6 +96,7 @@ impl Span { output: output.into(), tokens: 0, microjoules: 0, + joule_source: JouleSource::Estimated, } } @@ -88,11 +106,25 @@ impl Span { self } - /// Attach an energy charge. - pub fn with_microjoules(mut self, microjoules: u64) -> Self { - self.microjoules = microjoules; + /// Attach an energy charge with no provenance, which makes it an + /// estimate: it replaces any [`JouleSource::Measured`] attached before, + /// since that tag belonged to a different figure. A figure a live counter + /// read is attached with [`Self::with_joule_cost`]. + pub fn with_microjoules(self, microjoules: u64) -> Self { + self.with_joule_cost(JouleCost::estimated(microjoules)) + } + + /// Attach an energy charge and its provenance. + pub fn with_joule_cost(mut self, cost: JouleCost) -> Self { + self.microjoules = cost.microjoules; + self.joule_source = cost.source; self } + + /// The energy charged to this span, with its provenance. + pub fn joule_cost(&self) -> JouleCost { + JouleCost { microjoules: self.microjoules, source: self.joule_source } + } } /// An ordered execution trace. @@ -157,4 +189,41 @@ mod tests { assert_eq!(t.total_tokens(), 12); assert_eq!(t.total_microjoules(), 250); } + + /// A span is measured only when given a measured cost. A new span and a + /// bare figure are estimates, and a bare figure attached after a measured + /// one does not inherit its tag. + #[test] + fn a_span_is_measured_only_when_given_a_measured_cost() { + let span = || Span::new(0, SpanKind::Act, "calc", "1+1", "2"); + assert_eq!(span().joule_cost(), JouleCost::estimated(0)); + assert_eq!(span().with_microjoules(50).joule_cost(), JouleCost::estimated(50)); + assert_eq!(span().with_joule_cost(JouleCost::measured(50)).joule_cost(), JouleCost::measured(50)); + assert_eq!(span().with_joule_cost(JouleCost::estimated(50)).joule_cost(), JouleCost::estimated(50)); + let relabelled = span().with_joule_cost(JouleCost::measured(50)).with_microjoules(70); + assert_eq!(relabelled.joule_cost(), JouleCost::estimated(70)); + } + + /// A span's provenance survives serialization, and a trace written + /// before spans carried one reads back with every span estimated. + #[test] + fn a_span_without_a_joule_source_reads_back_as_estimated() { + let mut t = Trace::new(); + t.push(Span::new(0, SpanKind::Act, "calc", "1+1", "2").with_joule_cost(JouleCost::measured(50))); + t.push(Span::new(0, SpanKind::Think, "t", "in", "out").with_microjoules(200)); + let json = serde_json::to_string(&t).unwrap(); + assert!(json.contains(r#""joule_source":"Measured""#), "{json}"); + let back: Trace = serde_json::from_str(&json).unwrap(); + let costs: Vec<_> = back.spans.iter().map(Span::joule_cost).collect(); + assert_eq!(costs, [JouleCost::measured(50), JouleCost::estimated(200)]); + + let old = r#"{"spans":[ + {"index":0,"kind":"Act","label":"calc","input":"1+1","output":"2","tokens":0,"microjoules":50}, + {"index":1,"kind":"Think","label":"t","input":"in","output":"out","tokens":3,"microjoules":200} + ]}"#; + let back: Trace = serde_json::from_str(old).unwrap(); + let costs: Vec<_> = back.spans.iter().map(Span::joule_cost).collect(); + assert_eq!(costs, [JouleCost::estimated(50), JouleCost::estimated(200)]); + assert_eq!(back.total_tokens(), 3); + } } -- 2.54.0 (Apple Git-157)