diff --git a/eoc/standards/sci-for-ai-energy-provenance-annex.md b/eoc/standards/sci-for-ai-energy-provenance-annex.md new file mode 100644 index 000000000..62470e7e6 --- /dev/null +++ b/eoc/standards/sci-for-ai-energy-provenance-annex.md @@ -0,0 +1,214 @@ +# Proposed Annex: the provenance and validity of E + +*Draft submission for filing to [github.com/Green-Software-Foundation/sci](https://github.com/Green-Software-Foundation/sci). Licensed CC-BY-4.0.* + +*Status: working draft v0.1. Authors: Transaction Science. Target: SCI-for-AI, as an annex to the treatment of E. Companion to [`sci-for-ai-submission.md`](sci-for-ai-submission.md), which maps EOC receipts onto the four-component formula and assumes E is already trustworthy; this annex is about whether it is.* + +--- + +## a. Summary + +`SCI = (E × I + M) per R` is well specified in every term but one. `I` has a +location-based methodology. `M` has a lifecycle methodology. `R` is +deployment-defined and the specification is explicit that it must be *stated +and reproducible*. **`E` has no comparable requirement.** A figure may be +labelled "measured" without saying what measured it, over what span, on which +silicon, or on whose behalf — and in current practice it frequently is. + +This annex proposes five requirements (E1–E5) that make an `E` figure +self-describing, and one refusal: **a figure taken over a span shorter than its +counter's update period is not a measurement of that span and MUST NOT be +reported as measured.** All five are machine-checkable, and a reference +implementation exists. + +**What this annex does not propose.** No change to `R`. SCI-for-AI already +supports diverse functional units — tokens for language models, inferences for +classifiers, FLOPs for training — so non-generative inference, including the +typed-decision models that emerged in September and October 2026 and emit no +tokens at all, is already expressible as `R = one inference`. The gap is not in +the unit of work. It is in the energy. + +## b. The problem, as evidenced + +Three independent observations, none of them about a careless implementer: + +1. **Published work that has a sensor reports only part of the machine.** A + survey of recent AI-energy measurement work finds the instrumented studies + are GPU-rail-only — no CPU package, no DRAM, no NIC, no storage, no + power-supply loss — and that none of them declares its sampling interval, + the device's update rate, or the integration method actually used. +2. **"Measured" is being applied to estimators.** A reference implementation in + this ecosystem stamps `measured` on the output of a lifecycle estimator + whose own documentation calls it an estimation. The alternative path uses + hard-coded kWh-per-token constants calibrated from a single figure at ±50%. + Neither is wrong as an estimate; both are wrong as a measurement. +3. **Per-call energy is being taken from provider SDKs**, which report a number + with no stated counter, period or boundary. The reporter cannot tell a + hardware reading from a model, and neither can the reader. + +### Why this is not pedantry + +The error is not symmetric. A figure used in a *budget* or a *ceiling* that +under-reports admits work the budget cannot cover, silently; one that +over-reports refuses work that would have fit, visibly. A figure used in a +*disclosure* that under-reports understates an obligation. In both cases the +unsafe direction is the one an uncharacterised counter takes. + +And counters do take it, by large factors. A 588-run study of one widely used +cumulative-energy counter measured it **under-reporting integrated power by +95.4% at 0.5 ms sampling**, recovering to 1.6% only at 1 s, and recommended +against unvalidated use. Three weeks later a separate paper built its entire +per-token energy decomposition on that same counter, specifically *because* +power sampling cannot resolve short decode windows. Neither cites the other. +A third observation, on different hardware, has the same counter class reading +4–30× the board's actual draw when polled at 100 ms. Three mutually +inconsistent characterisations of one counter class — which means the validity +bound is itself sampling-interval-dependent and cannot be assumed. + +We reproduced the shape on a fourth counter. Burning one thread's CPU and +sampling the macOS per-process energy counter 60 times per interval, the +fraction of intervals in which it advanced was **0% at 10 µs and 50 µs, 2% at +100 µs, 5% at 500 µs, 35% at 1 ms, 63% at 2 ms, 87% at 5 ms, and 100% at +10 ms**. Below 1 ms it does not lag smoothly — on the intervals where it moved +at all, the median step was ~20 mJ, two orders above the 1 ms step. It accrues +in lumps. A sampler reading it every 500 µs would see nothing most of the time +and a 20 mJ jump occasionally, and would report both as measurements. + +## c. Proposed requirements + +An `E` figure claiming to be **measured** MUST carry: + +**E1 — Source.** The named counter or instrument: `rapl`, `nvml`, `rocm-smi`, +`level-zero`, `ioreport`, `proc-rusage`, `tegra-ina`, `qualcomm-qpt`, +`ascend-smi`, `hl-smi`, `cnmon`, `external-meter`, … Not the word "measured". +Two figures from different sources are not interchangeable: they have different +domains, different periods and different attribution. + +**E2 — Period, and the span rule.** The counter's documented or measured update +period, and the span the figure covers. **If the span is shorter than the +period, the figure MUST be reported as modelled, not measured.** A source whose +period is *undeclared* can never support a measured claim at any span, because +the direction of its error is unknown. A per-operation figure for work shorter +than the period is sound only as the mean of many operations over a span longer +than it. + +**E3 — Attribution boundary.** Whether the figure counts **one principal's** +work (a per-process or per-container counter) or **the whole machine** (an SoC +rail, a package counter, a board shunt). These are different claims, and the +distinction is independent of how much of the *task* is covered. A machine-wide +figure attributed to one workload MUST state the condition under which that is +valid — normally exclusive use — and MUST report the magnitude of its idle +correction, so a reader can see whether the correction is a rounding error or +the result. + +**E4 — Fabric breakdown.** Where the work crosses silicon, `E` SHOULD be +reported per fabric (`cpu`, `gpu`, `npu`, `dram`, `package`, …) and MUST state +which fabrics are **not** covered. A total that silently omits a fabric the +work used is an understatement, and on accelerated inference the omitted +fabric is usually the dominant one. + +**E5 — Why a fabric is uncovered.** Where a fabric is not measured, the reason +MUST be one of five, because they are not interchangeable: + +| status | meaning | +|---|---| +| `frozen` | counter present, not advancing | +| `absent` | no such silicon on this host | +| `needs-privilege` | present, requires privilege the reporter lacks | +| `no-counter-exists` | present, and exposes **no energy counter at all** | +| `not-implemented` | a counter exists; this reporter has no reader for it | + +Only `no-counter-exists` is a property of the hardware. The other four are +properties of the deployment or the tooling, and they are fixable. + +This matters beyond accounting hygiene. The EU AI Act's GPAI Code of Practice +requires providers to document known or estimated energy consumption for +training *and* inference, while permitting an exemption where the provider +lacks "critical information from compute or hardware provider". That exemption +is a claim of `no-counter-exists`. E5 is what distinguishes it from the four +statuses that do not justify it — and an unprivileged on-die counter, which +needs no cooperation from the compute provider at all, narrows the set of +accelerators for which the claim can honestly be made. + +## d. Worked example + +A three-question typed decision served by a 421M-parameter decision model on an +Apple M5 Max, `R = one inference`: + +``` +E = 1.0413e-4 Wh (104 127 µJ) +R = 1 decision +source = { gpu: ioreport, cpu: proc-rusage } +period = 10 ms both; span = 2.606 s over 200 decisions → measured (E2 met) +fabrics = gpu 96 591 µJ attribution=machine, idle-corrected by 0.08% + cpu 7 536 µJ attribution=process, coverage=whole for CPU work + npu 0 µJ frozen (no ANE Energy aggregate on this part) + dram — not-implemented (rails exist; this reader omits them) +``` + +Two things the annex makes visible that a single number hides. First, the figure +is a **mean over 200 decisions**, because one decision takes 11.85 ms and a +10 ms-period counter cannot resolve it — under E2 a per-decision counter read +would have had to be reported as modelled. Second, 93% of the energy is on a +**machine-attributed** rail, so the figure is this workload's only under +exclusive use; the idle correction was 0.08%, which is what makes that credible +here and would not be on a shared host. + +The same decision class run on the CPU instead costs **0.486–0.555 J** — +roughly five times more energy — and is entirely **principal-attributed** and +whole-coverage. Under E3 those are different claims, and a reporter choosing +the cheaper figure is also choosing the weaker one. Without E3 the two look +like the same kind of number, and the cheaper one wins every comparison. + +## e. Conformance + +All five requirements are machine-checkable, and we have an implementation +rather than only a proposal: + +- A metering ladder that tags every figure with its source and refuses to + upgrade a model's output to a reading. +- A per-fabric survey that returns E1, E3, E4 and E5 for every + `(fabric, source)` pair on a host, probing presence so an unread accelerator + reports `not-implemented` rather than the false claim `absent`. +- The E2 refusal as a predicate over `(declared period, span)`, with the + counter characterisation above as its evidence. +- Measurement harnesses for both the counter characterisation and the + per-decision figures, so the numbers in §d are reproducible rather than + quoted. + +We propose filing this as an annex under the extension URN +`urn:gsf:sci-ai:ext:energy-provenance:v1`, additive and imposing nothing on +deployments that report estimates — which remain perfectly conformant, provided +they say so. + +## f. References + +- **ISO/IEC 21031:2024** — Software Carbon Intensity (SCI) specification. +- **GSF SCI for AI** — ratified 2025; functional units for language models, + classifiers and training. +- **GSF, SCI for AI and EU AI Act environmental compliance** — the policy + framing this annex is intended to support. +- **EU AI Act GPAI Code of Practice**, Model Documentation Form — energy + consumption for training and inference; the compute/hardware-provider + exemption. +- Dauner et al., *Evaluating the Influence of Measurement Frequency on + [GPU energy counters]*, ACM SIGENERGY Energy Informatics Review 6(2), + July 2026 — the 588-run study; 95.4% under-read at 0.5 ms. +- *Characterization of Request and Token Energy Costs for LLM Inference + Workloads*, arXiv:2608.28044 — builds on the same counter for the opposite + reason. +- **CEN/CLC/TR 18145:2025**, *Green and Sustainable AI* — the European + guidance this annex is also offered to. +- **AFNOR SPEC 2314**, *Référentiel général pour l'IA frugale* — prescribes + estimator tooling for claims of frugality; E1/E2 are what would let such a + claim be checked. +- **Blauer Engel DE-UZ 215** — software energy criteria requiring measurement + on a defined test system; the German UBA's own reporting names energy per + classification as a KPI. +- **UNE 0086:2025** — energy, carbon, water and performance of AI systems. + +--- + +*Working draft. The filing to `github.com/Green-Software-Foundation/sci` will +carry a JSON-Schema definition for the E-provenance block, the conformance +predicate for E2, and the reproducible harnesses behind §d.* diff --git a/eoc/standards/sci-for-ai-submission.md b/eoc/standards/sci-for-ai-submission.md index 1869b26cb..b5571826d 100644 --- a/eoc/standards/sci-for-ai-submission.md +++ b/eoc/standards/sci-for-ai-submission.md @@ -4,6 +4,8 @@ *Status: working draft v0.1. Authors: Transaction Science (steward of the EOC specification). Target SCI revision: SCI-for-AI v1.0 (ratified December 2025).* +*Companion: [`sci-for-ai-energy-provenance-annex.md`](sci-for-ai-energy-provenance-annex.md) proposes requirements on the **provenance and validity of E**. This document maps EOC receipt fields onto the formula and takes E as given; the annex is about whether a figure labelled "measured" earned the label.* + --- ## a. Summary diff --git a/jcp/jcp-rs/crates/jcp-energy/src/lib.rs b/jcp/jcp-rs/crates/jcp-energy/src/lib.rs index ed961003d..65fdbca26 100644 --- a/jcp/jcp-rs/crates/jcp-energy/src/lib.rs +++ b/jcp/jcp-rs/crates/jcp-energy/src/lib.rs @@ -1277,7 +1277,7 @@ mod ioreport { sub, channels: subscribed, prior_sample: None, - cumulative: IoReportReading { total_uj: 0, cpu_uj: 0 }, + cumulative: IoReportReading::default(), }), }) } @@ -1303,10 +1303,12 @@ mod ioreport { if let Some(prior) = state.prior_sample { let delta = IOReportCreateSamplesDelta(prior, current, std::ptr::null()); if !delta.is_null() { - let (total_uj, cpu_uj) = sum_energy_uj(delta); + let d = sum_energy_uj(delta); state.cumulative = IoReportReading { - total_uj: state.cumulative.total_uj.saturating_add(total_uj), - cpu_uj: state.cumulative.cpu_uj.saturating_add(cpu_uj), + total_uj: state.cumulative.total_uj.saturating_add(d.total_uj), + cpu_uj: state.cumulative.cpu_uj.saturating_add(d.cpu_uj), + gpu_uj: state.cumulative.gpu_uj.saturating_add(d.gpu_uj), + ane_uj: state.cumulative.ane_uj.saturating_add(d.ane_uj), }; CFRelease(delta); } @@ -1840,26 +1842,55 @@ mod ioreport { strip_die_prefix(name).eq_ignore_ascii_case("GPU Energy") } + /// Whether a channel is the ANE energy aggregate: `ANE Energy`, optionally + /// `DIE__`-prefixed. + /// + /// No Apple part seen here publishes one — see [`ane_uj_of`] for what is + /// counted instead. + fn is_ane_energy_rail(name: &str) -> bool { + strip_die_prefix(name).eq_ignore_ascii_case("ANE Energy") + } + + /// Whether a channel is a per-engine ANE rail — `ANE0`, `DIE_1_ANE0` — + /// rather than one of its own summands (`ANE0_0`, `ANE0_1`). + /// + /// On an M5 Max this is the *only* ANE channel there is: a single `ANE0` in + /// mJ, with no `ANE Energy` aggregate above it and no sub-rails below it. + fn is_ane_base_rail(name: &str) -> bool { + let base = strip_die_prefix(name); + let Some(rest) = base + .strip_prefix("ANE") + .or_else(|| base.strip_prefix("ane")) + else { + return false; + }; + !rest.is_empty() && rest.bytes().all(|b| b.is_ascii_digit()) + } + /// Over one delta sample: the energy of every Energy Model rail the /// counter sums ([`counts_once`]), and of the CPU energy rails alone /// ([`is_cpu_energy_rail`], a share of the first), in µJ. - unsafe fn sum_energy_uj(delta: CFDictionaryRef) -> (u64, u64) { + unsafe fn sum_energy_uj(delta: CFDictionaryRef) -> IoReportReading { let key = CFStringCreateWithCString( std::ptr::null_mut(), c"IOReportChannels".as_ptr() as *const _, K_CFSTRING_ENCODING_UTF8, ); if key.is_null() { - return (0, 0); + return IoReportReading::default(); } let arr = CFDictionaryGetValue(delta, key as *const _) as CFArrayRef; CFRelease(key as CFTypeRef); if arr.is_null() { - return (0, 0); + return IoReportReading::default(); } let count = CFArrayGetCount(arr); let mut total_uj: u64 = 0; let mut cpu_uj: u64 = 0; + let mut gpu_uj: u64 = 0; + let mut ane_aggregate_uj: u64 = 0; + let mut ane_base_uj: u64 = 0; + let mut saw_ane_aggregate = false; for i in 0..count { let item = CFArrayGetValueAtIndex(arr, i); if item.is_null() { @@ -1876,17 +1907,37 @@ mod ioreport { Some(s) => s, None => continue, }; + let unit = cfstring_to_string(IOReportChannelGetUnitLabel(item)).unwrap_or_default(); + if is_ane_base_rail(&name) { + ane_base_uj = ane_base_uj.saturating_add(energy_to_uj(raw, &unit)); + } if !counts_once(&name) { continue; } - let unit = cfstring_to_string(IOReportChannelGetUnitLabel(item)).unwrap_or_default(); let uj = energy_to_uj(raw, &unit); total_uj = total_uj.saturating_add(uj); if is_cpu_energy_rail(&name) { cpu_uj = cpu_uj.saturating_add(uj); } + if is_gpu_energy_rail(&name) { + gpu_uj = gpu_uj.saturating_add(uj); + } + if is_ane_energy_rail(&name) { + ane_aggregate_uj = ane_aggregate_uj.saturating_add(uj); + saw_ane_aggregate = true; + } } - (total_uj, cpu_uj) + // An `ANE Energy` aggregate, where a part has one, is already in + // `total_uj` via `counts_once`. Where there is none, the per-engine + // rails are the only ANE count there is, and `counts_once` excludes + // them — so they are added here, once, and not twice. + let ane_uj = if saw_ane_aggregate { + ane_aggregate_uj + } else { + total_uj = total_uj.saturating_add(ane_base_uj); + ane_base_uj + }; + IoReportReading { total_uj, cpu_uj, gpu_uj, ane_uj } } fn energy_to_uj(raw: c_long, unit: &str) -> u64 { @@ -2133,14 +2184,59 @@ mod ioreport { for name in ["GPU0", "GPU0_0", "GPU CS0_0", "GPU SRAM0_0", "GPU CS SRAM0_0"] { assert!(!counts_once(name), "{name} is summed beside GPU Energy"); } - // Neither chip has an `ANE Energy` aggregate, and its rails are - // not summed either: ANE energy is not counted at all. Pinned as it - // is, so a change to it is a decision, not an accident. + // Neither chip has an `ANE Energy` aggregate, so `counts_once` — + // which decides by name alone — says no to every ANE rail. That + // used to mean ANE energy was counted nowhere, and a Neural Engine + // workload metered as free. It is now counted in `sum_energy_uj` + // instead, which can see whether an aggregate is present in the + // sample and falls back to the per-engine rails when it is not + // ([`is_ane_base_rail`]). This gate stays as it is so the two + // paths cannot both count it. for name in ["ANE0", "ANE0_0", "ANE0_1"] { assert!(!counts_once(name), "{name} is summed"); } } + /// The per-engine ANE rails, and only those, are what the fallback + /// counts: `ANE0` yes, its own summands `ANE0_0`/`ANE0_1` no, because + /// adding both would double-count the engine. + #[test] + fn only_a_per_engine_ane_rail_is_the_fallback() { + assert!(is_ane_base_rail("ANE0")); + assert!(is_ane_base_rail("ANE1")); + assert!(is_ane_base_rail("DIE_1_ANE0")); + for name in ["ANE0_0", "ANE0_1", "ANE Energy", "ANE", "GPU0", "CPU Energy"] { + assert!(!is_ane_base_rail(name), "{name} must not be the fallback"); + } + // The aggregate is recognised where a part publishes one, so the + // fallback is not used then. + assert!(is_ane_energy_rail("ANE Energy")); + assert!(is_ane_energy_rail("DIE_0_ANE Energy")); + assert!(!is_ane_energy_rail("ANE0")); + } + + /// The fabrics must not bleed into one another: each classifier claims + /// only its own rails, so a per-fabric figure is that fabric's. + #[test] + fn the_fabric_classifiers_are_disjoint() { + let rails = [ + "CPU Energy", "DIE_0_CPU Energy", "GPU Energy", "DIE_1_GPU Energy", "ANE Energy", + "ANE0", "GPU0", "PACC_0", "DRAM0", "DCS0", + ]; + for name in rails { + let claims = [ + is_cpu_energy_rail(name), + is_gpu_energy_rail(name), + is_ane_energy_rail(name), + is_ane_base_rail(name), + ] + .iter() + .filter(|c| **c) + .count(); + assert!(claims <= 1, "{name} is claimed by {claims} fabrics"); + } + } + #[test] fn only_the_cpu_energy_aggregate_is_a_cpu_rail() { assert!(is_cpu_energy_rail("CPU Energy")); @@ -5091,15 +5187,687 @@ impl Meter { } } +// ─── Per-fabric energy ────────────────────────────────────────────────────── + +/// An on-die fabric whose energy the platform counts separately. +/// +/// The metering ladder ([`Meter`]) admits exactly one rung and reports one +/// number, which is right for debiting a budget and wrong for understanding +/// where the energy went. On Apple silicon the two are not the same question: +/// for an unprivileged reader on macOS 27 the `CPU Energy` rails hold their +/// value while the `GPU Energy` rail advances continuously, so the ladder +/// declines the whole IOReport rung and loses a live counter — the one that +/// reads the fabric every local inference actually runs on. +#[derive(Debug, Clone, Copy, PartialEq, Eq, PartialOrd, Ord, Hash)] +pub enum Fabric { + /// The CPU cores. + Cpu, + /// A GPU, discrete or integrated. + Gpu, + /// A neural / inference accelerator: Apple's Neural Engine, a Qualcomm or + /// Intel NPU, a Google TPU, Huawei Ascend, Intel Gaudi, Cambricon. + Npu, + /// Reconfigurable logic. + Fpga, + /// A DSP block, which on a mobile SoC often runs the quantised path. + Dsp, + /// Main memory. Metered separately where a rail exists, because on a large + /// model the data movement is a large share and attributing it to the + /// compute fabric overstates that fabric. + Dram, + /// The whole package or board, as an external meter or a board rail sees + /// it. Not a compute fabric: the sum, including losses the on-die counters + /// never see. + Package, +} + + +impl Fabric { + /// A stable lowercase name, for a receipt or a log. + pub fn as_str(self) -> &'static str { + match self { + Fabric::Cpu => "cpu", + Fabric::Gpu => "gpu", + Fabric::Npu => "npu", + Fabric::Fpga => "fpga", + Fabric::Dsp => "dsp", + Fabric::Dram => "dram", + Fabric::Package => "package", + } + } + + /// Every fabric class, for a caller surveying a host. + pub fn all() -> &'static [Fabric] { + &[ + Fabric::Cpu, + Fabric::Gpu, + Fabric::Npu, + Fabric::Fpga, + Fabric::Dsp, + Fabric::Dram, + Fabric::Package, + ] + } +} + +/// Whose work a figure counts — the axis [`Coverage`] cannot express. +/// +/// `Coverage` says whether a figure covers the whole *task*; this says whose +/// task. They are independent, and conflating them is what made a live GPU +/// rail unusable: it is a real reading of the whole GPU, and it cannot tell +/// one process's share from another's. +#[derive(Debug, Clone, Copy, PartialEq, Eq)] +pub enum Attribution { + /// This process's work alone. The kernel attributes it (macOS + /// `ri_energy_nj`), so another process's load does not inflate it. + Process, + /// Every process on the machine. An SoC energy rail counts the fabric, not + /// the caller, so a figure from one is only this work's energy to the + /// extent that nothing else was using that fabric. A caller that needs a + /// per-task figure from a machine-wide rail MUST bracket the span with + /// idle baselines and say so. + Machine, +} + +/// One fabric's energy over a span. +#[derive(Debug, Clone, Copy, PartialEq, Eq)] +pub struct FabricEnergy { + /// Which fabric. + pub fabric: Fabric, + /// Energy over the span, microjoules. + pub uj: u64, + /// Whose work it counts. + pub attribution: Attribution, + /// How long the span was. Reported because a vendor energy counter has an + /// update period, and a figure taken over a span shorter than the counter + /// can resolve is not a measurement of that span: a published study of one + /// such counter found it under-reporting integrated power by 95.4% at + /// 0.5 ms sampling, recovering to 1.6% only at 1 s. A caller MUST compare + /// this against the counter's period before grading the figure, and a + /// per-operation figure for work shorter than the period is sound only as + /// the mean of many operations over a span longer than it. + pub span_ns: u128, +} + +/// Reads each fabric the host exposes separately, over a span. +/// +/// Composes two counters rather than choosing between them, because on this +/// platform neither covers the work alone: the kernel's per-process counter +/// attributes CPU energy and explicitly excludes the GPU and the Neural +/// Engine, and IOReport's rails read those fabrics and cannot attribute. Used +/// together they cover the fabrics the work used, each figure carrying which +/// fabric it read and whose work it counts. +#[derive(Debug, Default)] +pub struct FabricMeter { + #[cfg(target_os = "macos")] + ioreport: Option, +} + +impl FabricMeter { + /// Open every per-fabric counter this host exposes. + pub fn detect() -> Self { + #[cfg(target_os = "macos")] + { + FabricMeter { ioreport: ioreport::IoReport::open() } + } + #[cfg(not(target_os = "macos"))] + { + FabricMeter::default() + } + } + + /// Which fabrics this meter can read, and whose work each figure counts. + pub fn readable(&self) -> Vec<(Fabric, Attribution)> { + let mut out = Vec::new(); + #[cfg(target_os = "macos")] + { + if process_energy::read().is_some() { + out.push((Fabric::Cpu, Attribution::Process)); + } + if self.ioreport.is_some() { + out.push((Fabric::Gpu, Attribution::Machine)); + out.push((Fabric::Npu, Attribution::Machine)); + } + } + out + } + + /// Run `work`, and report each fabric's energy over it. + /// + /// A fabric whose counter did not advance is reported as zero rather than + /// omitted: on an idle fabric that is the truth, and the caller can tell + /// "read zero" from "could not read" by [`Self::readable`]. + pub fn measure(&self, work: impl FnOnce() -> T) -> (T, Vec) { + let started = std::time::Instant::now(); + #[cfg(target_os = "macos")] + let cpu_before = process_energy::read().map(|r| r.energy_nj); + #[cfg(target_os = "macos")] + let io_before = self.ioreport.as_ref().and_then(|m| m.read()); + + let out = work(); + + let span_ns = started.elapsed().as_nanos(); + let mut readings = Vec::new(); + #[cfg(target_os = "macos")] + { + if let (Some(b), Some(a)) = (cpu_before, process_energy::read().map(|r| r.energy_nj)) { + readings.push(FabricEnergy { + fabric: Fabric::Cpu, + uj: a.saturating_sub(b) / 1_000, + attribution: Attribution::Process, + span_ns, + }); + } + let io_after = self.ioreport.as_ref().and_then(|m| m.read()); + if let (Some(b), Some(a)) = (io_before, io_after) { + readings.push(FabricEnergy { + fabric: Fabric::Gpu, + uj: a.gpu_uj.saturating_sub(b.gpu_uj), + attribution: Attribution::Machine, + span_ns, + }); + readings.push(FabricEnergy { + fabric: Fabric::Npu, + uj: a.ane_uj.saturating_sub(b.ane_uj), + attribution: Attribution::Machine, + span_ns, + }); + } + } + let _ = span_ns; + (out, readings) + } +} + +// ─── Heterogeneous fabric survey ──────────────────────────────────────────── + +/// The counter a fabric's energy figure came from. +/// +/// Named on every platform so a caller can match on it anywhere, and so a +/// receipt can say *which* counter produced a figure rather than only that it +/// was "hardware". Two figures graded `HwShunt` from different sources are not +/// interchangeable: they have different domains, different update periods and +/// different attribution. +#[derive(Debug, Clone, Copy, PartialEq, Eq, PartialOrd, Ord, Hash)] +pub enum FabricSource { + /// macOS `proc_pid_rusage` `ri_energy_nj` — this process's own CPU energy. + /// The only source here that attributes to a principal. + ProcRusage, + /// Apple silicon IOReport `Energy Model` rails. + IoReport, + /// macOS `powermetrics`, which exposes rails IOReport withholds from an + /// unprivileged reader. + Powermetrics, + /// Linux `powercap` RAPL (`intel-rapl`, `amd-rapl`): CPU package, and on + /// some parts DRAM and the integrated GPU. + Rapl, + /// NVIDIA `libnvidia-ml` total-energy counter. + Nvml, + /// AMD `rocm_smi` `rsmi_dev_energy_count_get`. + RocmSmi, + /// Intel oneAPI Level Zero Sysman `zesPowerGetEnergyCounter` — discrete and + /// integrated GPUs, and the Meteor-Lake-and-later NPU. + LevelZero, + /// NVIDIA Jetson INA3221 board rails over hwmon. + TegraIna, + /// Qualcomm Power Telemetry, the Snapdragon per-unit stack. + QualcommQpt, + /// Huawei `npu-smi` (Ascend). + AscendSmi, + /// Intel Gaudi `hl-smi`. + HabanaSmi, + /// Cambricon `cnmon`. + CambriconCnmon, + /// A meter outside the machine: a shunt, a PDU, a bench DMM. The only + /// source that sees conversion losses, and the only one that can be + /// calibrated against a traceable reference. + ExternalMeter, +} + +impl FabricSource { + /// A stable lowercase name, for a receipt or a log. + pub fn as_str(self) -> &'static str { + match self { + FabricSource::ProcRusage => "proc-rusage", + FabricSource::IoReport => "ioreport", + FabricSource::Powermetrics => "powermetrics", + FabricSource::Rapl => "rapl", + FabricSource::Nvml => "nvml", + FabricSource::RocmSmi => "rocm-smi", + FabricSource::LevelZero => "level-zero", + FabricSource::TegraIna => "tegra-ina", + FabricSource::QualcommQpt => "qualcomm-qpt", + FabricSource::AscendSmi => "ascend-smi", + FabricSource::HabanaSmi => "habana-smi", + FabricSource::CambriconCnmon => "cambricon-cnmon", + FabricSource::ExternalMeter => "external-meter", + } + } + + /// Whether this crate has a reader for this source, as opposed to only + /// knowing it exists. + /// + /// Stated rather than implied, because "we did not measure it" and "it + /// cannot be measured" are different facts and a survey that blurs them is + /// worthless. See [`FabricStatus::NotImplemented`]. + pub fn has_backend(self) -> bool { + match self { + FabricSource::ProcRusage | FabricSource::IoReport | FabricSource::Rapl => true, + FabricSource::Nvml => cfg!(feature = "nvml"), + _ => false, + } + } +} + +/// Whether this host can meter one fabric from one source — and when it +/// cannot, which of the five different reasons applies. +#[derive(Debug, Clone, PartialEq, Eq)] +pub enum FabricStatus { + /// A live counter. `resolution_ns` is the period over which the counter is + /// documented or measured to update; `None` means the period is + /// **undeclared**, which is not the same as fast. + Live { attribution: Attribution, resolution_ns: Option }, + /// The silicon and the counter are both here, and the counter is not + /// advancing — so a figure from it would be a reading of nothing. Apple's + /// CPU rails do this for an unprivileged reader. + Frozen, + /// This host has no such silicon. + Absent, + /// The silicon is here and the counter needs privilege this process does + /// not have. A legitimate gap, and a fixable one. + NeedsPrivilege, + /// The silicon is here, and it exposes **no energy counter at all**. Any + /// figure for it is modelled, by anyone, and that is a property of the + /// hardware rather than a gap in this crate. This is the status that makes + /// an energy-disclosure exemption legitimate. + NoCounterExists, + /// A counter exists and this build has no reader for it. Our gap, named. + NotImplemented, +} + +impl FabricStatus { + /// Whether a span of `span_ns` may be graded as a **hardware reading** from + /// this source. + /// + /// A vendor energy counter has an update period, and a figure taken over a + /// span shorter than that period is not a measurement of the span: a + /// 588-run study of one such counter measured it under-reporting + /// integrated power by 95.4% at 0.5 ms sampling, recovering to 1.6% only + /// at 1 s. Under-reporting is the unsafe direction at a ceiling, because it + /// admits work the budget cannot cover. + /// + /// A counter whose period is **undeclared** can never grade a span, at any + /// length: the error direction is unknown, and an unknown direction at a + /// gate has to be read as possibly-under-reporting. That is deliberately + /// strict — the alternative is to believe a number nobody characterised. + pub fn may_grade_span(&self, span_ns: u128) -> bool { + match self { + FabricStatus::Live { resolution_ns: Some(period), .. } => { + span_ns >= u128::from(*period) + } + _ => false, + } + } +} + +/// One `(fabric, source)` pair this crate knows about, and what this host can +/// do with it. +#[derive(Debug, Clone, PartialEq, Eq)] +pub struct FabricCapability { + /// The silicon. + pub fabric: Fabric, + /// The counter that would read it. + pub source: FabricSource, + /// What this host can actually do. + pub status: FabricStatus, + /// What the probe was, or why no counter exists — so a reader can check + /// the claim instead of trusting it. + pub note: &'static str, +} + +/// Which Apple silicon Energy Model rails are advancing on this host, per +/// fabric. +/// +/// A direct observation rather than a guess: two reads of the same +/// subscription with a CPU burn between them, so a rail that *would* advance +/// under load is seen to. The CPU rails are expected to be frozen here — they +/// hold their value for an unprivileged reader on macOS 27 — and the point of +/// probing rather than assuming is that this is a per-OS-version fact that +/// should be observed on the host it is claimed about. +#[cfg(target_os = "macos")] +fn ioreport_rail_survey() -> Vec<(Fabric, bool, &'static str)> { + const NOTES: [(Fabric, &str); 3] = [ + (Fabric::Cpu, "IOReport `Energy Model` / `CPU Energy` (per-die on a multi-die part)"), + (Fabric::Gpu, "IOReport `Energy Model` / `GPU Energy`"), + ( + Fabric::Npu, + "IOReport `Energy Model` / `ANE Energy` where a part has one, else the per-engine `ANE` rails", + ), + ]; + let Some(m) = ioreport::IoReport::open() else { + return NOTES.iter().map(|(f, n)| (*f, false, *n)).collect(); + }; + let Some(before) = m.read() else { + return NOTES.iter().map(|(f, n)| (*f, false, *n)).collect(); + }; + // Burn CPU so a live CPU rail has something to count, by this thread's own + // CPU clock rather than wall time (a loaded host can give a wall-clock + // spin almost no CPU). + let deadline = std::time::Instant::now() + std::time::Duration::from_millis(60); + let mut x: u64 = 0; + while std::time::Instant::now() < deadline { + x = x.wrapping_add(deadline.elapsed().subsec_nanos() as u64 ^ (x >> 7)); + } + std::hint::black_box(x); + let Some(after) = m.read() else { + return NOTES.iter().map(|(f, n)| (*f, false, *n)).collect(); + }; + vec![ + (Fabric::Cpu, after.cpu_uj > before.cpu_uj, NOTES[0].1), + (Fabric::Gpu, after.gpu_uj > before.gpu_uj, NOTES[1].1), + (Fabric::Npu, after.ane_uj > before.ane_uj, NOTES[2].1), + ] +} + +/// Every `(fabric, source)` pair this crate knows of, with this host's status +/// for each. +/// +/// # Why a survey rather than a single number +/// +/// A metering ladder picks one rung and reports one figure, which is what a +/// budget needs. It cannot answer the question a heterogeneous host actually +/// raises: *which of my accelerators can be metered at all, by whom, and how +/// well.* On a machine with a CPU, an integrated GPU, a discrete GPU and an +/// NPU, the honest answer differs per fabric and per vendor, and the gaps are +/// not all the same kind of gap. +/// +/// The five negative statuses are the point. A disclosure regime that lets a +/// provider skip energy reporting when it "lacks critical information from the +/// compute or hardware provider" is relying on exactly one of them — +/// [`FabricStatus::NoCounterExists`] — and this function distinguishes it from +/// [`FabricStatus::NeedsPrivilege`], [`FabricStatus::Frozen`] and +/// [`FabricStatus::NotImplemented`], which are not the hardware's fault and do +/// not excuse anything. +/// +/// Vendor facts below were checked in October 2026 and are the kind that go +/// stale; each carries its own note so a wrong one can be found and corrected. +pub fn survey() -> Vec { + let mut out = Vec::new(); + let mut push = |fabric, source, status, note| { + out.push(FabricCapability { fabric, source, status, note }) + }; + + // ── This process's own CPU energy — the only principal-attributed source. + #[cfg(target_os = "macos")] + push( + Fabric::Cpu, + FabricSource::ProcRusage, + match process_energy_status() { + CounterStatus::Live => FabricStatus::Live { + attribution: Attribution::Process, + // Measured on an M5 Max, macOS 27, by burning this thread's + // CPU for a fixed interval and sampling the counter 60 times + // per interval. The fraction of intervals in which it + // advanced: 0% at 10 and 50 µs, 2% at 100 µs, 5% at 500 µs, + // 35% at 1 ms, 63% at 2 ms, 87% at 5 ms, 100% at 10 ms and + // above. The crate's liveness criterion is 80%, so 5 ms is + // where it first clears; 10 ms is declared instead because + // over-declaring a period refuses spans that would have been + // fine, and under-declaring grades spans that were not — and + // at a ceiling only the second error is dangerous. + // + // Below 1 ms the counter does not merely lag, it batches: on + // the intervals where it did advance, the median step was + // ~20 mJ, the same accrual-in-lumps pathology this crate + // already refuses on IOReport's CPU rails. + resolution_ns: Some(10_000_000), + }, + _ => FabricStatus::Frozen, + }, + "proc_pid_rusage(RUSAGE_INFO_V6).ri_energy_nj, read in-process", + ); + #[cfg(not(target_os = "macos"))] + push( + Fabric::Cpu, + FabricSource::ProcRusage, + FabricStatus::Absent, + "ri_energy_nj is a macOS interface", + ); + + // ── Apple silicon SoC rails. + #[cfg(target_os = "macos")] + { + let rails = ioreport_rail_survey(); + for (fabric, live, note) in rails { + push( + fabric, + FabricSource::IoReport, + if live { + FabricStatus::Live { + attribution: Attribution::Machine, + // **1 s, not the 10 ms first declared here.** This + // rail has no fixed period: it republishes when there + // is activity to report and holds its value when there + // is not. Sampled every 2 ms on an M5 Max, the gap + // between actual changes was p50 12.7 ms / p90 91 ms / + // **max 992 ms** at idle, and p50 13.1 ms / p90 15 ms + // / max 42 ms under a sustained MLX matmul. + // + // So the tight figure is only available to a measurer + // who can guarantee the fabric stays busy, and a + // measurer cannot. The declared period has to be the + // idle worst case, because that is the condition a + // short span is most likely to fall inside — a span + // that lands entirely within one hold reads zero and + // would otherwise be graded as a measurement of zero. + // + // (An earlier note elsewhere in the fleet put the hold + // at 300–600 ms "under load", from a GGUF decode. + // That decode is dispatch-bound with idle gaps between + // tokens, so it was sampling the idle behaviour; the + // two observations agree once the cause is named.) + resolution_ns: Some(1_000_000_000), + } + } else { + FabricStatus::Frozen + }, + note, + ); + } + push( + Fabric::Dram, + FabricSource::IoReport, + FabricStatus::NotImplemented, + "`DRAM0` and `DCS0` rails exist in the Energy Model group; this crate does not sum them separately", + ); + push( + Fabric::Cpu, + FabricSource::Powermetrics, + FabricStatus::NeedsPrivilege, + "powermetrics exposes the CPU rails IOReport withholds, and needs root", + ); + push( + Fabric::Npu, + FabricSource::Powermetrics, + FabricStatus::NeedsPrivilege, + "`sudo powermetrics --samplers ane_power` is the ONLY route to Apple ANE energy, and it is machine-wide: `ANE0` is frozen unprivileged, ANE work runs in `aned` so it is not this process's CPU either, and the H1xANELoadBalancer clients carry no statistics. A span that used the ANE is therefore CPU+GPU only, and Partial", + ); + } + + // ── x86 CPU package energy. + push( + Fabric::Cpu, + FabricSource::Rapl, + if cfg!(target_os = "linux") { + match rapl_status() { + CounterStatus::Live => FabricStatus::Live { + attribution: Attribution::Machine, + // The MSR behind powercap updates about every millisecond. + resolution_ns: Some(1_000_000), + }, + CounterStatus::Unavailable => FabricStatus::Absent, + _ => FabricStatus::Frozen, + } + } else { + FabricStatus::Absent + }, + "/sys/class/powercap/intel-rapl:*/energy_uj, package zones only", + ); + push( + Fabric::Dram, + FabricSource::Rapl, + FabricStatus::NotImplemented, + "powercap exposes a top-level `dram` zone on some parts; this crate reads package zones only", + ); + + // ── Discrete accelerators, by vendor. + push( + Fabric::Gpu, + FabricSource::Nvml, + if cfg!(feature = "nvml") { + match nvml_status() { + CounterStatus::Live => FabricStatus::Live { + attribution: Attribution::Machine, + resolution_ns: Some(10_000_000), + }, + CounterStatus::Unavailable => FabricStatus::Absent, + _ => FabricStatus::Frozen, + } + } else if probe_path("/dev/nvidiactl") { + FabricStatus::NotImplemented + } else { + FabricStatus::Absent + }, + "nvmlDeviceGetTotalEnergyConsumption, Volta and later; build with --features nvml", + ); + push( + Fabric::Gpu, + FabricSource::RocmSmi, + if probe_path("/sys/class/kfd") { FabricStatus::NotImplemented } else { FabricStatus::Absent }, + "rsmi_dev_energy_count_get; probed by /sys/class/kfd", + ); + push( + Fabric::Gpu, + FabricSource::LevelZero, + if probe_path("/dev/dri") { FabricStatus::NotImplemented } else { FabricStatus::Absent }, + "zesPowerGetEnergyCounter; probed by /dev/dri", + ); + push( + Fabric::Npu, + FabricSource::LevelZero, + if probe_path("/dev/accel/accel0") { FabricStatus::NotImplemented } else { FabricStatus::Absent }, + "Intel NPU (Meteor Lake and later) via Level Zero Sysman; probed by /dev/accel/accel0", + ); + push( + Fabric::Package, + FabricSource::TegraIna, + if probe_path("/sys/bus/i2c/drivers/ina3221") { FabricStatus::NotImplemented } else { FabricStatus::Absent }, + "Jetson INA3221 board rails over hwmon; board rails, so Package not Gpu", + ); + push( + Fabric::Npu, + FabricSource::AscendSmi, + if probe_path("/dev/davinci0") { FabricStatus::NotImplemented } else { FabricStatus::Absent }, + "Huawei Ascend via npu-smi; probed by /dev/davinci0", + ); + push( + Fabric::Npu, + FabricSource::HabanaSmi, + if probe_path("/dev/hl0") { FabricStatus::NotImplemented } else { FabricStatus::Absent }, + "Intel Gaudi via hl-smi; probed by /dev/hl0", + ); + push( + Fabric::Npu, + FabricSource::CambriconCnmon, + if probe_path("/dev/cambricon_dev0") { FabricStatus::NotImplemented } else { FabricStatus::Absent }, + "Cambricon via cnmon; probed by /dev/cambricon_dev0", + ); + push( + Fabric::Npu, + FabricSource::QualcommQpt, + FabricStatus::NotImplemented, + "Qualcomm Power Telemetry attributes CPU/GPU/NPU on Snapdragon; Android-only, no desktop probe", + ); + + // ── Fabrics that expose nothing, anywhere. The honest exemption. + push( + Fabric::Dsp, + FabricSource::QualcommQpt, + FabricStatus::NotImplemented, + "Hexagon DSP energy is in the QPT stack; no reader here", + ); + push( + Fabric::Fpga, + FabricSource::ExternalMeter, + FabricStatus::NoCounterExists, + "FPGA energy is board-dependent; no vendor-portable on-die counter exists (Oct 2026)", + ); + push( + Fabric::Package, + FabricSource::ExternalMeter, + FabricStatus::NoCounterExists, + "a shunt, PDU or DMM is the only source that sees conversion losses, and the only one calibratable against a traceable reference; not an interface this crate can open", + ); + + out +} + +/// Whether a path exists, used only to decide whether silicon is present. +/// +/// A presence probe, never a reading: it answers "is there an accelerator +/// here", so the survey can say [`FabricStatus::NotImplemented`] — our gap — +/// instead of [`FabricStatus::Absent`], which would be a false claim about the +/// host. +fn probe_path(p: &str) -> bool { + std::path::Path::new(p).exists() +} + +/// The total of a per-fabric set, and the fabrics it covers. +/// +/// Returns the sum and the fabrics it is a sum *of*, so a caller states its +/// coverage instead of implying it. Summing a `Process` figure with a +/// `Machine` one is sound — they read disjoint fabrics — but the result +/// inherits the weaker attribution, which the second element makes visible. +pub fn total_uj(readings: &[FabricEnergy]) -> (u64, Vec) { + let mut sum = 0u64; + let mut fabrics: Vec = Vec::new(); + for r in readings { + sum = sum.saturating_add(r.uj); + if !fabrics.contains(&r.fabric) { + fabrics.push(r.fabric); + } + } + fabrics.sort(); + (sum, fabrics) +} + /// One cumulative read of the IOReport counter, in µJ since its first read. #[cfg_attr(not(target_os = "macos"), allow(dead_code))] -#[derive(Debug, Clone, Copy, PartialEq, Eq)] +#[derive(Debug, Clone, Copy, PartialEq, Eq, Default)] struct IoReportReading { /// Every Energy Model rail the counter sums, CPU included: the SoC total. total_uj: u64, /// The CPU energy rails alone (`CPU Energy`, or `DIE__CPU Energy` on /// multi-die parts), a share of `total_uj`. cpu_uj: u64, + /// The GPU energy rails alone (`GPU Energy`, or `DIE__GPU Energy`), a + /// share of `total_uj`. + /// + /// Read separately from `cpu_uj` because on Apple silicon the two rails do + /// not share a fate: for an unprivileged reader on macOS 27 the CPU rails + /// hold their value while the GPU rail advances continuously. A metering + /// ladder that admits or declines the whole counter cannot use the live + /// half, and GPU-resident work — every local inference on a Mac — is + /// exactly what the CPU rails would have missed anyway. + gpu_uj: u64, + /// The Neural Engine's energy, a share of `total_uj`. + /// + /// Counted from the `ANE Energy` aggregate where a part publishes one, and + /// otherwise from the per-engine `ANE` rails, which on every Apple part + /// seen here is the only ANE channel that exists. Before this it was + /// counted from neither, so ANE energy was absent from `total_uj` + /// altogether and a Neural Engine workload metered as though it were free. + ane_uj: u64, } /// The figure the IOReport rung gives one span, from its reads either side. @@ -5402,8 +6170,8 @@ mod tests { /// rise as the measured figure, covering the whole of the work. #[test] fn a_span_the_ioreport_cpu_rail_advanced_over_is_a_hardware_reading() { - let before = IoReportReading { total_uj: 10_000, cpu_uj: 4_000 }; - let after = IoReportReading { total_uj: 15_500, cpu_uj: 7_000 }; + let before = IoReportReading { total_uj: 10_000, cpu_uj: 4_000, gpu_uj: 0, ane_uj: 0 }; + let after = IoReportReading { total_uj: 15_500, cpu_uj: 7_000, gpu_uj: 0, ane_uj: 0 }; let e = ioreport_span(Some(before), Some(after), 1_000_000); assert_eq!(e.provenance, Provenance::HwShunt, "a live span was not graded as a reading: {e:?}"); assert_eq!(e.measured_uj, Some(5_500), "the reading is not the SoC total's rise: {e:?}"); @@ -5422,11 +6190,11 @@ mod tests { let elapsed_ns = 2_000_000; // 2 ms let model = time_power_uj(elapsed_ns, DEFAULT_ACTIVE_MILLIWATTS); assert!(model > 0); - let b = IoReportReading { total_uj: 10_000, cpu_uj: 4_000 }; + let b = IoReportReading { total_uj: 10_000, cpu_uj: 4_000, gpu_uj: 0, ane_uj: 0 }; let cases = [ - ("the GPU rail rose, the CPU rail held", Some(b), Some(IoReportReading { total_uj: 90_000, cpu_uj: 4_000 })), + ("the GPU rail rose, the CPU rail held", Some(b), Some(IoReportReading { total_uj: 90_000, cpu_uj: 4_000, gpu_uj: 0, ane_uj: 0 })), ("nothing moved", Some(b), Some(b)), - ("the read before failed", None, Some(IoReportReading { total_uj: 15_000, cpu_uj: 7_000 })), + ("the read before failed", None, Some(IoReportReading { total_uj: 15_000, cpu_uj: 7_000, gpu_uj: 0, ane_uj: 0 })), ("the read after failed", Some(b), None), ]; for (what, before, after) in cases { @@ -5449,8 +6217,8 @@ mod tests { let (meter, _) = Meter::detect_with(&Ladder { ioreport: &|| IoReportStatus::Live, ..Ladder::none() }); if matches!(meter, Meter::IoReport(_)) { let live = ioreport_span( - Some(IoReportReading { total_uj: 0, cpu_uj: 0 }), - Some(IoReportReading { total_uj: 2, cpu_uj: 1 }), + Some(IoReportReading { total_uj: 0, cpu_uj: 0, gpu_uj: 0, ane_uj: 0 }), + Some(IoReportReading { total_uj: 2, cpu_uj: 1, gpu_uj: 0, ane_uj: 0 }), 1, ); assert_eq!(meter.provenance(), Provenance::HwShunt, "the IOReport rung is not graded as a hardware reading"); @@ -6911,3 +7679,275 @@ mod tests { e.check_provenance().unwrap(); } } + +#[cfg(test)] +mod fabric_tests { + use super::*; + + /// A sum must say what it is a sum of, so a caller cannot imply coverage + /// it does not have. + #[test] + fn a_total_names_the_fabrics_it_covers() { + let r = [ + FabricEnergy { + fabric: Fabric::Cpu, + uj: 7_536, + attribution: Attribution::Process, + span_ns: 2_606_000_000, + }, + FabricEnergy { + fabric: Fabric::Gpu, + uj: 96_591, + attribution: Attribution::Machine, + span_ns: 2_606_000_000, + }, + ]; + // The measured figure for one typed decision on an M5 Max. + assert_eq!(total_uj(&r), (104_127, vec![Fabric::Cpu, Fabric::Gpu])); + // An empty set totals zero and claims nothing. + assert_eq!(total_uj(&[]), (0, vec![])); + } + + /// The two counters read disjoint fabrics, which is what makes summing + /// them sound; the names are stable because a receipt carries them. + #[test] + fn fabric_names_are_stable() { + assert_eq!(Fabric::Cpu.as_str(), "cpu"); + assert_eq!(Fabric::Gpu.as_str(), "gpu"); + assert_eq!(Fabric::Npu.as_str(), "npu"); + } + + /// On a host that exposes them, a span reports a figure per fabric, each + /// carrying its own attribution — and the GPU figure is machine-wide, so + /// it must never be reported as this process's. + #[cfg(target_os = "macos")] + #[test] + fn a_span_reports_each_fabric_with_its_own_attribution() { + let m = FabricMeter::detect(); + let readable = m.readable(); + if readable.is_empty() { + println!("SKIPPED: this host exposes no per-fabric counter"); + return; + } + let (out, readings) = m.measure(|| { + // Enough CPU work that the process counter has something to show. + let mut x = 0u64; + for i in 0..8_000_000u64 { + x = x.wrapping_add(i ^ (x >> 7)); + } + x + }); + assert_ne!(out, 0); + for r in &readings { + assert!(r.span_ns > 0, "a span must have a duration"); + match r.fabric { + Fabric::Cpu => assert_eq!(r.attribution, Attribution::Process), + // Every other fabric here is read from an SoC rail, which + // counts the silicon and cannot tell one process from another. + _ => assert_eq!( + r.attribution, + Attribution::Machine, + "an SoC rail cannot attribute to a process" + ), + } + } + // CPU work must show on the CPU fabric where that counter is live. + if readable.contains(&(Fabric::Cpu, Attribution::Process)) { + let cpu = readings.iter().find(|r| r.fabric == Fabric::Cpu).expect("a cpu figure"); + assert!(cpu.uj > 0, "8M iterations of CPU work metered as free: {cpu:?}"); + } + println!("per-fabric over the span: {readings:?} -> total {:?}", total_uj(&readings)); + } +} + +#[cfg(test)] +mod survey_tests { + use super::*; + + /// The survey must never say two different things about one + /// (fabric, source) pair, or a reader cannot act on it. + #[test] + fn the_survey_names_each_fabric_source_pair_once() { + let s = survey(); + assert!(!s.is_empty()); + let mut seen: Vec<(Fabric, FabricSource)> = Vec::new(); + for c in &s { + let key = (c.fabric, c.source); + assert!(!seen.contains(&key), "{:?}/{:?} appears twice", c.fabric, c.source); + seen.push(key); + assert!(!c.note.is_empty(), "{:?}/{:?} has no note", c.fabric, c.source); + } + } + + /// `Absent` is a claim about the host and `NotImplemented` is a claim about + /// us. A source we have no reader for must never be reported as `Live`, + /// because that would be a figure from nowhere. + #[test] + fn a_source_without_a_backend_is_never_live() { + for c in survey() { + if !c.source.has_backend() { + assert!( + !matches!(c.status, FabricStatus::Live { .. }), + "{:?}/{:?} is Live with no backend", + c.fabric, + c.source + ); + } + } + } + + /// The refusal the whole survey exists to support: an undeclared update + /// period can never grade a span, and a declared one grades only a span at + /// least that long. + #[test] + fn an_undeclared_update_period_can_never_grade_a_span() { + let undeclared = FabricStatus::Live { + attribution: Attribution::Machine, + resolution_ns: None, + }; + for span in [0u128, 1_000, 1_000_000, 1_000_000_000, u128::MAX] { + assert!( + !undeclared.may_grade_span(span), + "an uncharacterised counter graded a {span} ns span" + ); + } + + let declared = FabricStatus::Live { + attribution: Attribution::Machine, + resolution_ns: Some(10_000_000), // 10 ms + }; + // The 11.85 ms of one typed decision is above a 10 ms period; the + // 0.5 ms that under-reported by 95.4% in the published study is not. + assert!(declared.may_grade_span(11_850_000)); + assert!(declared.may_grade_span(10_000_000), "the boundary is inclusive"); + assert!(!declared.may_grade_span(9_999_999)); + assert!(!declared.may_grade_span(500_000)); + + // No negative status grades anything, whatever the span. + for st in [ + FabricStatus::Frozen, + FabricStatus::Absent, + FabricStatus::NeedsPrivilege, + FabricStatus::NoCounterExists, + FabricStatus::NotImplemented, + ] { + assert!(!st.may_grade_span(u128::MAX), "{st:?} graded a span"); + } + } + + /// Source names are stable and distinct: a receipt carries them, so two + /// sources sharing a name would make two different domains look like one. + #[test] + fn source_names_are_stable_and_distinct() { + let all = [ + FabricSource::ProcRusage, + FabricSource::IoReport, + FabricSource::Powermetrics, + FabricSource::Rapl, + FabricSource::Nvml, + FabricSource::RocmSmi, + FabricSource::LevelZero, + FabricSource::TegraIna, + FabricSource::QualcommQpt, + FabricSource::AscendSmi, + FabricSource::HabanaSmi, + FabricSource::CambriconCnmon, + FabricSource::ExternalMeter, + ]; + let mut names: Vec<&str> = all.iter().map(|s| s.as_str()).collect(); + names.sort_unstable(); + let n = names.len(); + names.dedup(); + assert_eq!(names.len(), n, "two sources share a name"); + for f in Fabric::all() { + assert!(!f.as_str().is_empty()); + } + } + + /// Exactly one source attributes to a principal. If a second ever claims + /// to, that is a real change and should be a deliberate one: an SoC rail + /// counts silicon and cannot tell one process from another. + #[test] + fn only_the_per_process_counter_attributes_to_a_principal() { + for c in survey() { + if let FabricStatus::Live { attribution: Attribution::Process, .. } = c.status { + assert_eq!( + c.source, + FabricSource::ProcRusage, + "{:?} claims per-process attribution", + c.source + ); + } + } + } + + /// What this host can actually meter, printed. Not an assertion about the + /// fleet — a record of the machine the test ran on. + #[test] + fn report_this_hosts_fabrics() { + println!("{:<9} {:<16} {:<48} note", "fabric", "source", "status"); + for c in survey() { + println!("{:<9} {:<16} {:<48} {}", c.fabric.as_str(), c.source.as_str(), format!("{:?}", c.status), c.note); + } + } +} + +#[cfg(all(test, target_os = "macos"))] +mod process_counter_resolution { + use super::*; + + /// The per-process CPU counter's declared period must be the measured one, + /// and it must be strict enough to refuse the spans the measurement showed + /// it cannot resolve. + /// + /// This is the counter our one principal-attributed figure comes from, so + /// if its period were left undeclared, the only whole-coverage, + /// process-attributed reading on Apple silicon could never be graded. + #[test] + fn the_per_process_counter_grades_only_spans_it_can_resolve() { + let Some(cap) = survey() + .into_iter() + .find(|c| c.source == FabricSource::ProcRusage && c.fabric == Fabric::Cpu) + else { + panic!("the survey must name the per-process CPU counter"); + }; + let FabricStatus::Live { attribution, resolution_ns } = cap.status else { + println!("SKIPPED: the per-process counter is not live on this host"); + return; + }; + assert_eq!(attribution, Attribution::Process); + assert_eq!(resolution_ns, Some(10_000_000), "the declared period is the measured one"); + + // The spans the measurement showed it cannot resolve are refused. + assert!(!cap.status.may_grade_span(50_000), "50 µs: measured 0% advancing"); + assert!(!cap.status.may_grade_span(500_000), "500 µs: measured 5% advancing"); + assert!(!cap.status.may_grade_span(1_000_000), "1 ms: measured 35% advancing"); + assert!(!cap.status.may_grade_span(5_000_000), "5 ms: 87%, below the declared period"); + + // The GPU rail's declared period is the idle worst case, so a span + // shorter than a second cannot be graded from it however busy the + // fabric happened to be. + if let Some(gpu) = survey() + .into_iter() + .find(|c| c.source == FabricSource::IoReport && c.fabric == Fabric::Gpu) + { + if let FabricStatus::Live { resolution_ns, .. } = gpu.status { + assert_eq!( + resolution_ns, + Some(1_000_000_000), + "the GPU rail holds up to 992 ms at idle; a shorter declared period grades spans it never covered" + ); + assert!(!gpu.status.may_grade_span(42_000_000), "42 ms was the max gap under load, not a safe span"); + assert!(!gpu.status.may_grade_span(992_000_000), "992 ms was the observed max hold"); + assert!(gpu.status.may_grade_span(2_606_000_000), "the 2.6 s span the figure was taken over"); + } + } + + // And the span the morning's figure was actually taken over is graded: + // 200 decisions over 2.606 s, which is why that figure stands. + assert!(cap.status.may_grade_span(2_606_000_000)); + // A single 11.85 ms decision is above the period too, but one decision + // is not how that figure was obtained — see the harness. + assert!(cap.status.may_grade_span(11_850_000)); + } +} diff --git a/jcp/tools/counter-resolution/README.md b/jcp/tools/counter-resolution/README.md new file mode 100644 index 000000000..e1525cb36 --- /dev/null +++ b/jcp/tools/counter-resolution/README.md @@ -0,0 +1,52 @@ +# What a counter can actually resolve + +Two harnesses behind the numbers `jcp-energy` declares. Both are the evidence +for a constant in the crate; neither is a test of the crate. + +## `counter_resolution.py` — the per-process CPU counter's update period + +`ri_energy_nj` is the only source in the survey that attributes energy to a +*principal* rather than to silicon, so whether it may grade a span decides +whether Apple silicon has any process-attributed reading at all. + +Burns this thread's CPU for a fixed interval, samples the counter 60 times per +interval, and reports the fraction of intervals in which it advanced — the +crate's own liveness criterion is 80%. + +Measured on an M5 Max, macOS 27: + +| interval | advanced | median step | clears 80% | +|---:|---:|---:|---| +| 10 µs | 0% | — | no | +| 50 µs | 0% | — | no | +| 100 µs | 2% | 20.4 mJ | no | +| 500 µs | 5% | 21.1 mJ | no | +| 1 ms | 35% | 2.8 mJ | no | +| 2 ms | 63% | 4.9 mJ | no | +| **5 ms** | **87%** | 9.6 mJ | **yes** | +| 10 ms | 100% | 19.0 mJ | yes | +| 50 ms | 100% | 92.9 mJ | yes | + +The crate declares **10 ms**, not the 5 ms where it first clears the criterion: +over-declaring a period refuses spans that would have been fine, while +under-declaring grades spans that were not, and at a ceiling only the second +error is dangerous. + +Note the shape below 1 ms. The counter does not lag smoothly — on the few +intervals where it advanced at all, the median step was ~20 mJ, two orders +above the 1 ms step. It accrues in lumps, which is the same +batched-refresh pathology the crate already refuses on IOReport's CPU rails. +A sampler that read it every 500 µs would see nothing most of the time and a +20 mJ jump occasionally, and would report both as measurements. + +This is also why a per-operation figure for an 11.85 ms decision is taken as +the mean of 200 decisions over a 2.6 s span rather than as 200 counter reads. + +## `ioreport_rails.py` — which Apple rails exist and advance + +Enumerates the IOReport `Energy Model` group unprivileged (364 rails on an +M5 Max) and prints each channel's name, unit and delta. What it establishes: +`GPU Energy` is live in **nJ**, every `CPU Energy` / `MCPU*` / `PACC*` / +`*_SRAM` rail is frozen for an unprivileged reader, and `ANE0` is the only ANE +channel — no `ANE Energy` aggregate, no sub-rails. Units differ *within one +sample*, so a summer that assumes one unit is wrong. diff --git a/jcp/tools/counter-resolution/counter_resolution.py b/jcp/tools/counter-resolution/counter_resolution.py new file mode 100644 index 000000000..445887a7f --- /dev/null +++ b/jcp/tools/counter-resolution/counter_resolution.py @@ -0,0 +1,32 @@ +"""At what interval does ri_energy_nj actually advance under continuous load? + +The crate's own liveness criterion: a rail is live when it advanced in >=80% of +sampled intervals. Applied here to find the shortest interval that clears it, +which is the counter's declarable update period. +""" +import ctypes, os, time, statistics +_l = ctypes.CDLL("/usr/lib/libSystem.B.dylib"); _b = ctypes.create_string_buffer(464); _p = os.getpid() +def nj(): + _l.proc_pid_rusage(ctypes.c_int(_p), ctypes.c_int(6), ctypes.byref(_b)) + return int.from_bytes(_b.raw[336:344], "little") + +def trial(interval_s, n=60): + """Burn for `interval_s` per sample; report the fraction that advanced.""" + adv = 0; deltas = [] + prev = nj() + for _ in range(n): + t_end = time.perf_counter() + interval_s + x = 0 + while time.perf_counter() < t_end: + x += 1 # burn this thread's CPU + cur = nj() + if cur > prev: adv += 1 + deltas.append(cur - prev) + prev = cur + nz = [d for d in deltas if d > 0] + return adv / n, (statistics.median(nz)/1e3 if nz else 0.0) + +print(f"{'interval':>10} {'advanced':>9} {'median ΔµJ':>11} live(>=80%)") +for us in (10, 50, 100, 250, 500, 1_000, 2_000, 5_000, 10_000, 20_000, 50_000): + frac, med = trial(us/1e6) + print(f"{us:>8}µs {frac*100:>8.0f}% {med:>11.1f} {'yes' if frac >= 0.80 else 'no'}") diff --git a/jcp/tools/counter-resolution/gpu_rail_period.py b/jcp/tools/counter-resolution/gpu_rail_period.py new file mode 100644 index 000000000..fb418c982 --- /dev/null +++ b/jcp/tools/counter-resolution/gpu_rail_period.py @@ -0,0 +1,48 @@ +"""How long does the GPU Energy rail hold its value? + +I declared 10 ms as its update period. PR #146 reports it "republishes in +steps and can hold 300-600 ms under load". Declaring too short a period is the +unsafe direction: it lets a span be graded that the counter never covered. +This measures the distribution of gaps between actual changes. +""" +import time, statistics, threading, sys +exec(open('rails.py').read().split('import time, subprocess, sys')[0]) + +def gpu_abs(): + """Absolute cumulative GPU nJ: delta against a fixed first sample.""" + return None + +def gaps(duration_s, label, load=None): + stop = threading.Event() + t = None + if load: + t = threading.Thread(target=load, args=(stop,), daemon=True); t.start() + time.sleep(0.3) # let the load spin up + prev_s = sample(); last_change = time.perf_counter(); g = []; steps = [] + t_end = time.perf_counter() + duration_s + while time.perf_counter() < t_end: + time.sleep(0.002) # 2 ms sampling: finer than any claim + cur_s = sample() + d = IOR.IOReportCreateSamplesDelta(prev_s, cur_s, None) + tot = sum(v for _g, _sg, n, u, v in rails(d) if n == "GPU Energy") + if tot > 0: + now = time.perf_counter() + g.append((now - last_change) * 1e3); steps.append(tot / 1e6) + last_change = now + prev_s = cur_s # only advance the baseline on a change + if load: + stop.set(); t.join(timeout=2) + if not g: + print(f"{label:<16} no change in {duration_s}s"); return + g.sort() + print(f"{label:<16} n={len(g):<4} gap ms: p50={statistics.median(g):7.1f} " + f"p90={g[int(.9*len(g))-1]:7.1f} max={g[-1]:7.1f} median step={statistics.median(steps):8.2f} mJ") + +def mlx_load(stop): + import mlx.core as mx + a = mx.random.normal((2048, 2048)); b = mx.random.normal((2048, 2048)) + while not stop.is_set(): + c = a @ b; mx.eval(c) + +gaps(6.0, "idle") +gaps(6.0, "mlx matmul", mlx_load) diff --git a/jcp/tools/counter-resolution/ioreport_rails.py b/jcp/tools/counter-resolution/ioreport_rails.py new file mode 100644 index 000000000..ce59c12fd --- /dev/null +++ b/jcp/tools/counter-resolution/ioreport_rails.py @@ -0,0 +1,80 @@ +"""Enumerate IOReport Energy Model rails on Apple Silicon, unprivileged. + +Decisive question: are the GPU and ANE energy rails readable without sudo on +macOS 27, and do they advance under a Metal workload? +""" +from ctypes import * + +CF = CDLL("/System/Library/Frameworks/CoreFoundation.framework/CoreFoundation") +IOR = CDLL("/usr/lib/libIOReport.dylib") + +CFStringRef = c_void_p; CFDictionaryRef = c_void_p; CFArrayRef = c_void_p +CF.CFStringCreateWithCString.restype = CFStringRef +CF.CFStringCreateWithCString.argtypes = [c_void_p, c_char_p, c_uint32] +CF.CFStringGetCStringPtr.restype = c_char_p +CF.CFStringGetCStringPtr.argtypes = [CFStringRef, c_uint32] +CF.CFStringGetCString.restype = c_bool +CF.CFStringGetCString.argtypes = [CFStringRef, c_char_p, c_long, c_uint32] +CF.CFDictionaryGetValue.restype = c_void_p +CF.CFDictionaryGetValue.argtypes = [CFDictionaryRef, c_void_p] +CF.CFArrayGetCount.restype = c_long +CF.CFArrayGetCount.argtypes = [CFArrayRef] +CF.CFArrayGetValueAtIndex.restype = c_void_p +CF.CFArrayGetValueAtIndex.argtypes = [CFArrayRef, c_long] + +def cfstr(s): return CF.CFStringCreateWithCString(None, s.encode(), 0x08000100) # kCFStringEncodingUTF8 +def pystr(ref): + if not ref: return None + p = CF.CFStringGetCStringPtr(ref, 0x08000100) + if p: return p.decode(errors="replace") + buf = create_string_buffer(512) + return buf.value.decode(errors="replace") if CF.CFStringGetCString(ref, buf, 512, 0x08000100) else None + +IOR.IOReportCopyChannelsInGroup.restype = CFDictionaryRef +IOR.IOReportCopyChannelsInGroup.argtypes = [CFStringRef, CFStringRef, c_uint64, c_uint64, c_uint64] +IOR.IOReportCreateSubscription.restype = c_void_p +IOR.IOReportCreateSubscription.argtypes = [c_void_p, CFDictionaryRef, POINTER(CFDictionaryRef), c_uint64, c_void_p] +IOR.IOReportCreateSamples.restype = CFDictionaryRef +IOR.IOReportCreateSamples.argtypes = [c_void_p, CFDictionaryRef, c_void_p] +IOR.IOReportCreateSamplesDelta.restype = CFDictionaryRef +IOR.IOReportCreateSamplesDelta.argtypes = [CFDictionaryRef, CFDictionaryRef, c_void_p] +for fn, rt in (("IOReportChannelGetChannelName", CFStringRef), ("IOReportChannelGetGroup", CFStringRef), + ("IOReportChannelGetSubGroup", CFStringRef), ("IOReportChannelGetUnitLabel", CFStringRef)): + getattr(IOR, fn).restype = rt; getattr(IOR, fn).argtypes = [CFDictionaryRef] +IOR.IOReportSimpleGetIntegerValue.restype = c_int64 +IOR.IOReportSimpleGetIntegerValue.argtypes = [CFDictionaryRef, c_int32] + +chans = IOR.IOReportCopyChannelsInGroup(cfstr("Energy Model"), None, 0, 0, 0) +if not chans: raise SystemExit("no Energy Model group") +subbed = CFDictionaryRef() +sub = IOR.IOReportCreateSubscription(None, chans, byref(subbed), 0, None) +if not sub: raise SystemExit("subscription refused (would mean privilege is required)") + +def sample(): return IOR.IOReportCreateSamples(sub, subbed, None) +def rails(delta): + out = [] + arr = CF.CFDictionaryGetValue(delta, cfstr("IOReportChannels")) + for i in range(CF.CFArrayGetCount(arr)): + ch = CF.CFArrayGetValueAtIndex(arr, i) + out.append((pystr(IOR.IOReportChannelGetGroup(ch)), pystr(IOR.IOReportChannelGetSubGroup(ch)), + pystr(IOR.IOReportChannelGetChannelName(ch)), pystr(IOR.IOReportChannelGetUnitLabel(ch)), + IOR.IOReportSimpleGetIntegerValue(ch, 0))) + return out + +import time, subprocess, sys +s0 = sample() +# Load the GPU (and ANE if asked) while the rails are watched. +load = subprocess.Popen([sys.argv[1], sys.argv[2]]) if len(sys.argv) > 2 else None +time.sleep(2.0) +s1 = sample() +if load: load.wait() +d = IOR.IOReportCreateSamplesDelta(s0, s1, None) +rs = rails(d) +print(f"subscription: OK (unprivileged, uid={__import__('os').getuid()}) rails in Energy Model: {len(rs)}") +agg = {} +for g, sg, name, unit, val in rs: + key = (sg or "-", name or "-", unit or "-") + a = agg.setdefault(key, [0, 0]); a[0] += val; a[1] += 1 +print(f"{'subgroup':<22}{'channel':<34}{'unit':<8}{'delta over 2s':>16}{'n':>5}") +for (sg, name, unit), (val, n) in sorted(agg.items(), key=lambda kv: -abs(kv[1][0]))[:28]: + print(f"{sg:<22}{name:<34}{unit:<8}{val:>16}{n:>5}") diff --git a/mesh/SPEC.md b/mesh/SPEC.md index 2fabaa08f..8dbcc4e63 100644 --- a/mesh/SPEC.md +++ b/mesh/SPEC.md @@ -80,6 +80,11 @@ A conforming request MUST carry both a `fidelity_floor` and a `joule_ceiling_uj` The pair is the contract: *give me an answer at least this good, for at most this much energy.* +A request MAY additionally carry a `decision` (§3.6), which says it is asking for +a bounded typed decision rather than generated text. Its absence is a generation, +and is omitted from the canonical form — so the `request_id` of every request +written before this field existed is unchanged. + ### §3.2 Fidelity Fidelity is an ordered scale `F0 … F4` — `F0` the coarsest sufficient answer @@ -125,13 +130,65 @@ IOReport), but a counter counts only while it is live: the whole request, such as a GPU counter on a host whose CPU has none, MUST NOT be graded `HwShunt`. +### §3.6 Typed decisions — the other unit of work + +A **decision model** takes state plus typed questions and returns a calibrated +probability per allowed answer, generating no text. The category converged +through 2026 on three primitives — a proposition (`noul`), a choice over 1–255 +named options, and a score over an ordered rubric — and backends score the +schema **non-autoregressively** rather than emitting tokens. + +Everything else in this spec measures work in tokens, and a decision's output +token count is zero. Measured on a 421M decision model: `output_tokens: 0`, with +132 input tokens of which **121 were the question schema and 11 the state under +judgement**. Three consequences follow, and a conforming implementation MUST +handle each. + +A request asking for a decision carries: + +| Field | Type | Meaning | +|---|---|---| +| `questions` | u32 | typed questions in the request | +| `options_scored` | u32 | options scored across all of them | +| `schema_tokens` | u32 | how many of `prompt_tokens` are schema, not state | + +1. **It MUST NOT be priced for tokens it cannot emit.** The estimate for a + decision MUST exclude `max_tokens`. Including it refuses work at a ceiling + that would in fact have covered it. +2. **Cost MUST be allowed to follow the schema.** A profile MAY declare a cost + per scored option; one question over 20 options is more work than one over 2 + at identical token counts, and no token count expresses the difference. + Declaring nothing reproduces the token-only model exactly, so an unmeasured + profile claims nothing. +3. **The efficiency figure MUST be defined.** `uj_per_accepted_token` (§6) is + undefined when no tokens were accepted, which is every request in this + category. A decision receipt MUST instead carry `uj_per_decision`, and + SHOULD carry `uj_per_option_scored`. The unit of work is the answered + question. + +`schema_tokens` is reported because a fixed schema is a **cacheable input +prefix** — distinct from `cache_hit_tokens`, which counts output tokens served +from cache, of which a decision has none. In the measured request 92% of the +input was schema, so a reader that cannot see the split cannot tell a cold +decision from a warm one. + +> **Why a decision receipt is worth more than a generation receipt.** A decision +> is a single forward pass with no sampling. Given the weight digest, revision +> and numeric precision, it is reproducible — so a receipt over one is verifiable +> **by re-execution**, which a receipt over sampled text can never be. A receipt +> for a decision SHOULD therefore bind the weight digest and the precision; the +> probabilities of a published decision model differ between `fp16` and `fp32` +> even where the selected answer agrees, so a receipt that omits precision does +> not reproduce. + --- ## §4 Routing — cheapest-sufficient For each registered backend the router computes an estimate -`est_total_uj = uj_per_token × (prompt_tokens + max_tokens)` and marks it -**admissible** iff: +`est_total_uj = uj_per_token × (prompt_tokens + max_tokens)`, or for a typed +decision (§3.6) `uj_per_token × prompt_tokens + uj_per_option_scored × +options_scored`, and marks it **admissible** iff: ``` backend.fidelity ≥ request.fidelity_floor AND est_total_uj ≤ request.joule_ceiling_uj @@ -207,7 +264,10 @@ and nulls; booleans are encoded as `0|1` uints and absent options dropped. | `fidelity_met` | `0|1` — did the served backend meet the floor | | `estimated_uj` | always present | | `measured_uj` | present **iff** provenance is `HwShunt` | -| `uj_per_accepted_token` | **the wedge metric** — `energy.uj() / tokens_accepted` | +| `uj_per_accepted_token` | **the wedge metric** for a generation — `energy.uj() / tokens_accepted`; omitted when no tokens were accepted | +| `questions_answered`, `options_scored`, `schema_tokens` | present **iff** the request was a typed decision (§3.6) | +| `uj_per_decision` | **the wedge metric** for a decision — `energy.uj() / questions_answered` | +| `uj_per_option_scored` | the finer decision figure — `energy.uj() / options_scored` | | `energy_provenance` | `HwShunt | ModelBased | Estimator` | | `sequence`, `prev` | hash-chain position and prior envelope's content id | | `nonce`, `steward` | per-receipt nonce and the sealing identity (COSE `kid`) | diff --git a/mesh/mesh-rs/crates/mesh-conformance/src/lib.rs b/mesh/mesh-rs/crates/mesh-conformance/src/lib.rs index c9de0120c..192d32a34 100644 --- a/mesh/mesh-rs/crates/mesh-conformance/src/lib.rs +++ b/mesh/mesh-rs/crates/mesh-conformance/src/lib.rs @@ -69,6 +69,7 @@ pub fn reference_scenario() -> ReferenceScenario { max_tokens: MAX_TOKENS, fidelity_floor: Fidelity::F1, joule_ceiling_uj: 8_000_000, + decision: None, }; // Empty prompt payload: the reference engine is synthetic (input-agnostic), diff --git a/mesh/mesh-rs/crates/mesh-core/examples/demo.rs b/mesh/mesh-rs/crates/mesh-core/examples/demo.rs index b66af044b..cfc27daaf 100644 --- a/mesh/mesh-rs/crates/mesh-core/examples/demo.rs +++ b/mesh/mesh-rs/crates/mesh-core/examples/demo.rs @@ -65,6 +65,7 @@ fn main() { max_tokens: 64, fidelity_floor: Fidelity::F1, joule_ceiling_uj: 8_000_000, // 8 J + decision: None, // a generation, not a typed decision }; // --- the caller mints a JCP grant: capability(infer over this request) ⊗ budget --- diff --git a/mesh/mesh-rs/crates/mesh-core/src/lib.rs b/mesh/mesh-rs/crates/mesh-core/src/lib.rs index 86e2960bc..c3310622c 100644 --- a/mesh/mesh-rs/crates/mesh-core/src/lib.rs +++ b/mesh/mesh-rs/crates/mesh-core/src/lib.rs @@ -236,6 +236,7 @@ mod tests { max_tokens, fidelity_floor: floor, joule_ceiling_uj: ceiling, + decision: None, } } diff --git a/mesh/mesh-rs/crates/mesh-engine/src/lib.rs b/mesh/mesh-rs/crates/mesh-engine/src/lib.rs index 831ddb18c..7b0c9efd2 100644 --- a/mesh/mesh-rs/crates/mesh-engine/src/lib.rs +++ b/mesh/mesh-rs/crates/mesh-engine/src/lib.rs @@ -173,6 +173,7 @@ impl ServingEngine for ReferenceEngine { cache_hit_tokens: if cache_hit { n } else { 0 }, spec_proposed, spec_accepted, + decision: None, }; Served { tokens, outcome } } @@ -192,6 +193,7 @@ mod tests { max_tokens, fidelity_floor: Fidelity::F0, joule_ceiling_uj: u64::MAX, + decision: None, } } diff --git a/mesh/mesh-rs/crates/mesh-openai/src/lib.rs b/mesh/mesh-rs/crates/mesh-openai/src/lib.rs index 46935d898..9d769fab7 100644 --- a/mesh/mesh-rs/crates/mesh-openai/src/lib.rs +++ b/mesh/mesh-rs/crates/mesh-openai/src/lib.rs @@ -108,6 +108,9 @@ pub fn to_serve_request( max_tokens, fidelity_floor, joule_ceiling_uj, + // The OpenAI-shaped surface describes a generation, never a typed + // decision: its request has no questions to score. + decision: None, }; Ok((serve, prompt)) } @@ -353,6 +356,9 @@ pub fn to_serve_request_responses( max_tokens, fidelity_floor, joule_ceiling_uj, + // The OpenAI-shaped surface describes a generation, never a typed + // decision: its request has no questions to score. + decision: None, }; Ok((serve, prompt)) } diff --git a/mesh/mesh-rs/crates/mesh-receipt/src/lib.rs b/mesh/mesh-rs/crates/mesh-receipt/src/lib.rs index 4a7aa34cd..a01ef6f74 100644 --- a/mesh/mesh-rs/crates/mesh-receipt/src/lib.rs +++ b/mesh/mesh-rs/crates/mesh-receipt/src/lib.rs @@ -82,6 +82,26 @@ impl MeshReceipt { self.energy.uj().checked_div(self.outcome.tokens_accepted as u64) } + /// The wedge metric for a backend that emits no tokens: microjoules per + /// answered decision (SPEC §3.6). `None` when the request was not a + /// decision, or answered no questions. + /// + /// A typed decision's `tokens_accepted` is zero, so + /// [`Self::uj_per_accepted_token`] is undefined for every request in that + /// category — the receipt would seal with no efficiency figure at all. The + /// unit of work here is the answered question, not the accepted token. + pub fn uj_per_decision(&self) -> Option { + let d = self.outcome.decision?; + self.energy.uj().checked_div(d.questions_answered as u64) + } + + /// Microjoules per option scored — the finer figure, since one question + /// over 20 options is more work than one over 2. + pub fn uj_per_option_scored(&self) -> Option { + let d = self.outcome.decision?; + self.energy.uj().checked_div(d.options_scored as u64) + } + /// Project to the JCR-1 `mesh/1` wire JSON. Booleans are encoded as `0|1` /// uints and absent options dropped, so the signed core stays within the /// det-CBOR type set (no bools, no nulls). `steward` is the endpoint @@ -111,6 +131,19 @@ impl MeshReceipt { if let Some(jpt) = self.uj_per_accepted_token() { obj.insert("uj_per_accepted_token".into(), json!(jpt)); } + // A typed decision (§3.6). Emitted only when the request was one, so a + // generation's signed core is byte-identical to before. + if let Some(d) = self.outcome.decision { + obj.insert("questions_answered".into(), json!(d.questions_answered)); + obj.insert("options_scored".into(), json!(d.options_scored)); + obj.insert("schema_tokens".into(), json!(d.schema_tokens)); + if let Some(jpd) = self.uj_per_decision() { + obj.insert("uj_per_decision".into(), json!(jpd)); + } + if let Some(jpo) = self.uj_per_option_scored() { + obj.insert("uj_per_option_scored".into(), json!(jpo)); + } + } obj.insert( "energy_provenance".into(), json!(self.energy.provenance.jcr1_label()), @@ -223,7 +256,7 @@ fn hex_lower(bytes: &[u8]) -> String { #[cfg(test)] mod tests { use super::*; - use mesh_schema::Provenance; + use mesh_schema::{DecisionOutcome, Provenance}; fn outcome(accepted: u32) -> ServeOutcome { ServeOutcome { @@ -251,6 +284,74 @@ mod tests { } } + /// A typed decision accepts no output tokens, so the token metric is + /// undefined and the decision metric is the one that exists. + /// + /// The figure is the **CPU-only, process-attributed** one: 486 000 µJ for a + /// three-question decision on an M5 Max, 162 mJ per question, from the + /// kernel's per-process counter over a span far longer than its measured + /// 10 ms period. + /// + /// Deliberately not the GPU figure. A GPU reading here comes from an SoC + /// rail that counts the whole machine, and on this host another process + /// was measured holding **99.5% of GPU time** in a 4 s window — so a rail + /// delta over a decision's span is an upper bound on the machine, not that + /// decision's energy. The per-process CPU counter is unaffected by that + /// contention, which is why the sound example uses it. + #[test] + fn a_decision_has_an_efficiency_figure_where_the_token_metric_has_none() { + let mut o = outcome(0); + o.decision = Some(DecisionOutcome { + questions_answered: 3, + options_scored: 8, // choice:3 + score:3 + noul:2 + schema_tokens: 121, + }); + let mut r = receipt(Energy::measured(486_000), 0); + r.outcome = o; + + // The defect: the documented "wedge metric" is undefined here. + assert_eq!(r.uj_per_accepted_token(), None); + // The fix: the unit of work is the answered question. + assert_eq!(r.uj_per_decision(), Some(162_000)); + assert_eq!(r.uj_per_option_scored(), Some(60_750)); + + // It survives the wire, and the wire says what it measured. + let wire = r.to_jcr1_wire("did:example:mesh-endpoint"); + assert_eq!(wire["uj_per_decision"], json!(162_000)); + assert_eq!(wire["questions_answered"], json!(3)); + assert_eq!(wire["schema_tokens"], json!(121)); + assert!( + wire.get("uj_per_accepted_token").is_none(), + "an undefined metric must be absent, not zero" + ); + + // And it still seals. + let sk = SigningKey::from_bytes(&[9u8; 32]); + let steward = "did:example:mesh-endpoint"; + let env = r.seal(&sk, steward).unwrap(); + assert!(r + .verify_envelope(&env, steward, &sk.verifying_key().to_bytes()) + .is_ok()); + } + + /// The addition must be invisible to a generation: no new keys, so every + /// receipt sealed before it verifies byte-identically. + #[test] + fn a_generation_receipt_gains_no_wire_keys() { + let r = receipt(Energy::measured(3_200_000), 64); + let wire = r.to_jcr1_wire("did:example:mesh-endpoint"); + for k in [ + "questions_answered", + "options_scored", + "schema_tokens", + "uj_per_decision", + "uj_per_option_scored", + ] { + assert!(wire.get(k).is_none(), "{k} must not appear for a generation"); + } + assert_eq!(r.uj_per_decision(), None); + } + #[test] fn measured_receipt_seal_verify_and_metric() { let sk = SigningKey::from_bytes(&[9u8; 32]); diff --git a/mesh/mesh-rs/crates/mesh-router/src/lib.rs b/mesh/mesh-rs/crates/mesh-router/src/lib.rs index cd9905309..76ce29248 100644 --- a/mesh/mesh-rs/crates/mesh-router/src/lib.rs +++ b/mesh/mesh-rs/crates/mesh-router/src/lib.rs @@ -53,6 +53,14 @@ pub struct CostProfile { /// says this backend gets more expensive per token as the prefix grows, /// which is what makes a long-context request refusable *before* it runs. pub uj_per_token_per_kctx: u64, + /// Microjoules per option scored in a typed decision (§3.6), on top of the + /// per-token cost of reading the request. + /// + /// Zero is the historical shape and the default: a backend that only + /// generates has no options to score, and every existing constructor and + /// sealed `wire()` string is unchanged. For a decision backend, zero means + /// the estimate ignores the only work the pass performs. + pub uj_per_option_scored: u64, /// The operating point these constants were measured at, if the operator /// declared one. See [`Calibration`]. /// @@ -169,12 +177,53 @@ impl CostProfile { /// Retained so existing callers are unchanged and the scalar case stays /// honestly expressible — some backends really are flat. pub fn linear(uj_per_token: u64) -> Self { - CostProfile { base_uj: 0, uj_per_token, uj_per_token_per_kctx: 0, calibration: None } + CostProfile { + base_uj: 0, + uj_per_token, + uj_per_token_per_kctx: 0, + uj_per_option_scored: 0, + calibration: None, + } } /// A profile whose per-token cost grows with the context length. pub fn context_scaled(base_uj: u64, uj_per_token: u64, uj_per_token_per_kctx: u64) -> Self { - CostProfile { base_uj, uj_per_token, uj_per_token_per_kctx, calibration: None } + CostProfile { + base_uj, + uj_per_token, + uj_per_token_per_kctx, + uj_per_option_scored: 0, + calibration: None, + } + } + + /// Declare what one scored option costs this backend, so a typed decision + /// can be priced on the work it does (§3.6). + /// + /// Zero — the default — reproduces the token-only model exactly, which for + /// a decision means pricing a pass that generates nothing by the tokens it + /// does not generate. + pub fn with_option_cost(mut self, uj_per_option_scored: u64) -> Self { + self.uj_per_option_scored = uj_per_option_scored; + self + } + + /// Estimated microjoules for a typed decision: the input it reads, plus the + /// schema it scores. + /// + /// `max_tokens` has no part in it. A decision model emits no tokens — a + /// measured three-question request reported `output_tokens: 0` — so the + /// per-token term covers only what it reads, and the work of scoring + /// `options_scored` options is the term that would otherwise be invisible. + pub fn estimate_decision_uj( + &self, + prompt_tokens: u64, + context_tokens: u64, + options_scored: u64, + ) -> u64 { + let read = self.estimate_uj(prompt_tokens, context_tokens); + let scored = self.uj_per_option_scored.saturating_mul(options_scored); + read.saturating_add(scored) } /// Whether this profile can represent a context-dependent cost at all. @@ -315,6 +364,16 @@ impl BackendSpec { /// backend — deliberately, since the alternative is to model the growth and /// then defend a second uncalibrated constant. fn estimate(&self, req: &ServeRequest) -> u64 { + if let Some(d) = req.decision { + // A typed decision generates nothing, so `max_tokens` is not part + // of its cost; what it costs is reading the request and scoring + // the schema. + return self.cost.estimate_decision_uj( + req.prompt_tokens as u64, + req.prompt_tokens as u64, + d.options_scored as u64, + ); + } let tokens = req.prompt_tokens as u64 + req.max_tokens as u64; self.cost.estimate_uj(tokens, req.prompt_tokens as u64) } @@ -430,6 +489,7 @@ impl Router { #[cfg(test)] mod tests { use super::*; + use mesh_schema::DecisionShape; use mesh_schema::ContentId; fn router() -> Router { @@ -448,9 +508,73 @@ mod tests { max_tokens, fidelity_floor: floor, joule_ceiling_uj: ceiling, + decision: None, } } + fn req_decision(prompt_tokens: u32, max_tokens: u32, options_scored: u32) -> ServeRequest { + ServeRequest { + prompt_hash: ContentId::of_bytes(b"state"), + prompt_tokens, + max_tokens, + fidelity_floor: Fidelity::F0, + joule_ceiling_uj: u64::MAX, + decision: Some(DecisionShape { questions: 3, options_scored, schema_tokens: 121 }), + } + } + + /// A typed decision must not be charged for tokens it cannot emit. + /// + /// This is the defect the decision shape exists to close: a decision model + /// generates nothing (`output_tokens: 0` measured), so a caller that leaves + /// `max_tokens` at a generation's default was being priced for an essay the + /// backend will never write — and refused at a ceiling that would in fact + /// have covered the work. + #[test] + fn a_decision_is_not_priced_for_tokens_it_never_emits() { + let spec = BackendSpec::with_cost( + "decider", + "laya-421m", + Fidelity::F1, + CostProfile::linear(10).with_option_cost(1_000), + ); + // 132 read tokens x 10 uJ + 8 options x 1000 uJ = 1320 + 8000. + let d = spec.estimate(&req_decision(132, 0, 8)); + assert_eq!(d, 9_320); + // The same request with a generation's max_tokens costs the same: the + // ceiling is irrelevant to a pass that emits nothing. + assert_eq!(spec.estimate(&req_decision(132, 4_096, 8)), d); + // A generation with that max_tokens is priced for all of it - proving + // the branch is what changed, not the profile. + assert_eq!( + spec.estimate(&req_ctx(Fidelity::F1, u64::MAX, 132, 4_096)), + (132 + 4_096) * 10 + ); + } + + /// Cost must follow the schema being scored, which is the dimension no + /// token count carries: one question over 20 options is more work than one + /// over 2, at identical token counts. + #[test] + fn a_decision_scales_with_the_options_it_scores() { + let spec = BackendSpec::with_cost( + "decider", + "laya-421m", + Fidelity::F1, + CostProfile::linear(10).with_option_cost(1_000), + ); + let few = spec.estimate(&req_decision(132, 0, 2)); + let many = spec.estimate(&req_decision(132, 0, 20)); + assert!(many > few, "scoring 20 options must cost more than scoring 2"); + assert_eq!(many - few, 18 * 1_000); + // Declaring no option cost reproduces the token-only model, so the + // change is opt-in and an unmeasured profile claims nothing. + let flat = + BackendSpec::with_cost("decider", "laya-421m", Fidelity::F1, CostProfile::linear(10)); + assert_eq!(flat.estimate(&req_decision(132, 0, 2)), 1_320); + assert_eq!(flat.estimate(&req_decision(132, 0, 20)), 1_320); + } + /// The historical scalar model must be reproduced exactly, so existing /// backends and their receipts do not shift under this change. #[test] @@ -558,6 +682,7 @@ mod tests { max_tokens, fidelity_floor: floor, joule_ceiling_uj: ceiling, + decision: None, } } diff --git a/mesh/mesh-rs/crates/mesh-schema/src/lib.rs b/mesh/mesh-rs/crates/mesh-schema/src/lib.rs index a44b98844..7cf348eaa 100644 --- a/mesh/mesh-rs/crates/mesh-schema/src/lib.rs +++ b/mesh/mesh-rs/crates/mesh-schema/src/lib.rs @@ -199,6 +199,39 @@ pub struct ServeRequest { pub fidelity_floor: Fidelity, /// Hard energy budget for this request, in microjoules. pub joule_ceiling_uj: u64, + /// The shape of a **typed decision**, when that is what is being asked for + /// rather than generated text (§3.6). + /// + /// `None` is the historical shape — a generation, priced per token — and is + /// skipped on the wire, so an existing request's [`ServeRequest::id`] and + /// every sealed conformance vector are byte-identical to before. + #[serde(default, skip_serializing_if = "Option::is_none")] + pub decision: Option, +} + +/// What a backend is being asked to decide, in the terms that determine its +/// energy (§3.6). +/// +/// A decision model takes state plus typed questions and returns a calibrated +/// probability per allowed answer, generating no text. Its work is therefore +/// not a token count: a non-autoregressive pass scores the *schema*, so cost +/// follows the questions asked and the options under each, and `max_tokens` is +/// zero. Pricing such a request per token prices the one thing it does not do. +#[derive(Clone, Copy, Debug, Default, PartialEq, Eq, Serialize, Deserialize)] +pub struct DecisionShape { + /// Typed questions in this request. + pub questions: u32, + /// Options scored across all of them — two 3-option `choice` questions and + /// one `noul` score 8. This is the dimension a token count cannot express. + pub options_scored: u32, + /// How many of `prompt_tokens` are the question schema rather than the + /// state under judgement. + /// + /// A fixed schema is a cacheable prefix, and it dominates: in a measured + /// three-question request to a 421M decision model, 121 of 132 input + /// tokens were schema and 11 were state. A router that cannot see the + /// split re-prices 92% of the request on every call as though it were new. + pub schema_tokens: u32, } impl ServeRequest { @@ -252,4 +285,32 @@ pub struct ServeOutcome { pub spec_proposed: u32, /// Speculative tokens the target model accepted. pub spec_accepted: u32, + /// What was decided, when the backend served a typed decision rather than + /// generating text (§3.6). + /// + /// `None` is the historical shape, whose unit of work is the accepted + /// output token, and is skipped on the wire so existing receipts seal to + /// the same bytes. + #[serde(default, skip_serializing_if = "Option::is_none")] + pub decision: Option, +} + +/// Per-request decision counters — the inputs to the µJ/decision metric +/// (§3.6). +/// +/// Every other field of [`ServeOutcome`] is a token count, and a decision's +/// `tokens_accepted` is zero, which leaves `uj_per_accepted_token` undefined. +/// The unit of work here is the answered question. +#[derive(Clone, Copy, Debug, Default, PartialEq, Eq, Serialize, Deserialize)] +pub struct DecisionOutcome { + /// Questions answered. The denominator of µJ/decision. + pub questions_answered: u32, + /// Options actually scored across them. + pub options_scored: u32, + /// Input tokens that were the question schema rather than the state. + /// + /// Distinct from [`ServeOutcome::cache_hit_tokens`], which counts *output* + /// tokens served from cache. A decision has no output tokens; what it can + /// reuse is this input prefix. + pub schema_tokens: u32, } diff --git a/mesh/tools/joules-per-decision/README.md b/mesh/tools/joules-per-decision/README.md new file mode 100644 index 000000000..a9f18a0c5 --- /dev/null +++ b/mesh/tools/joules-per-decision/README.md @@ -0,0 +1,231 @@ +# Joules per typed decision + +A measurement harness, and what it found: **Mesh cannot price, rate or report a +typed decision**, because every quantity in the schema is a token count and a +decision emits no tokens. + +Run it with `uv`: + +```sh +uv venv --python 3.12 .venv +uv pip install --python .venv/bin/python laya-mlx +.venv/bin/python joules_per_decision.py +``` + +## What a decision model is + +A decision model takes application *state* plus *typed questions* and returns +calibrated probabilities over a bounded answer set. It generates no text. The +category appeared between 2026-09-15 (TypeSafe's Jev) and 2026-10-01 +(Cloudflare's Clef), converging on three primitives: + +| primitive | returns | +|---|---| +| `noul` | P(true) for a proposition | +| `choice` | a probability per named option, 1–255 options | +| `score` | a probability per level of an ordered rubric | + +The schema is published as `System One` (OpenAPI 3.1, `api.typesafe.ai`), and +Jev, Liquid d1, Kev, Clef and Laya all implement it. Clef states that it scores +schema choices **non-autoregressively** rather than generating tokens. + +## Method + +The harness measures `laya-mlx` — an independent Apache-2.0 MLX port of Convai's +Laya (ModernBERT-large, 421M, FP16) — on Apple Silicon. + +Energy comes from `ri_energy_nj` in `proc_pid_rusage(getpid(), RUSAGE_INFO_V6)`, +the same counter `jcp-energy`'s rung 3 uses, read in-process so the reading is +`Coverage::Whole` for this process's CPU work. The ABI constants (`SIZE = 464`, +`ENERGY_NJ = 336`) are lifted from `jcp-energy`, which verified them with +`sizeof`/`offsetof` against the macOS 26, 26.5 and 27 SDKs; they are not +re-derived here. The harness applies `jcp-energy`'s own liveness rule before +trusting the counter — at least five advancing sample intervals and at least +80% of them, over at least 300 ms — and reports the result rather than assuming +it. + +Because the counter is per-process, the **energy** figures are unaffected by +other load on the host. The **latency** figures are not, so they are upper +bounds. + +## Results — M5 Max, macOS 27, 2026-10-04 + +| device | questions | p50 | p95 | J/decision | coverage | +|---|---|---:|---:|---:|---| +| CPU | 1 (`noul`) | 126.8 ms | 636.2 ms | **0.204** | Whole (CPU), HwShunt | +| CPU | 3 (`choice`+`score`+`noul`) | 629.1 ms | 1410.0 ms | **0.486** | Whole (CPU), HwShunt | +| GPU | 1 | 21.1 ms | 38.2 ms | 0.0079 | **floor only** | +| GPU | 3 | 17.1 ms | 22.8 ms | 0.0082 | **floor only** | + +Counter liveness: 25 advancing intervals of 25. + +Three questions cost *less* wall time than one, because all three ride a single +forward pass (`batch_size` defaults to 16) — the marginal cost of a question +inside a batch is near zero, which no per-token model predicts. + +The GPU rows are a **floor, not the decision's energy**. They are the host-side +CPU energy only; the GPU's share is unmetered. `jcp-energy` has no +whole-coverage rung for GPU-resident work on Apple Silicon and correctly refuses +to pass off IOReport's machine-wide GPU rail as one task's energy, so no +whole-task joule figure for the GPU path can be stated here. Every local +inference on a Mac is GPU-resident, so this is the gap that matters most. + +## What the measurements show about this spec + +Four findings, each checked against the code rather than inferred. + +1. **`uj_per_accepted_token` is undefined for the whole category.** + `MeshReceipt::uj_per_accepted_token` is documented as "the wedge metric" and + is `checked_div` by `outcome.tokens_accepted`. A decision's + `tokens_accepted` is 0 — measured `"output_tokens": 0` on every request — so + the metric returns `None`. The receipt seals, and the number that makes it + worth sealing does not exist. + +2. **The unit of work is implicitly the accepted output token.** Every field of + `ServeOutcome` is a token count: `tokens_in`, `tokens_out`, + `tokens_accepted`, `cache_hit_tokens`, `spec_proposed`, `spec_accepted`. A + decision's unit is the answered question, and nothing in the outcome can + carry it. + +3. **The cost model prices the wrong thing.** `CostProfile::estimate_uj` is + charged `tokens = prompt_tokens + max_tokens`. With `max_tokens = 0` a + decision collapses to `base_uj` plus its prompt, which ignores what the pass + actually does: score N questions × K options. And the split inside the + prompt is invisible — in the measured 3-question request, **121 of 132 input + tokens were the question schema and 11 were the state**. A fixed schema is a + cacheable prefix (`laya-mlx` exposes `cache_prompts`), so 92% of a real + decision request is re-priced on every call as though it were new input. + +4. **`fidelity_met` is a self-declaration sealed into a signed receipt.** + `mesh-core` computes `fidelity_met = chosen_fidelity >= req.fidelity_floor`, + where `chosen_fidelity` is the level the backend *advertises* (SPEC §3.2: + "ordered scale F0 … F4"). `Energy` in the same receipt carries a provenance + tag — `HwShunt | ModelBased | Estimator` — with a normative coupling rule. + Fidelity carries nothing. The receipt is honest about where its joules came + from and silent about where its quality claim came from. + + Decision models are what makes this fixable rather than inherent: for a + bounded question, calibration is measurable. Kev publishes Brier scores + (27B: 0.225) alongside accuracy; Clef trains against a Brier loss. A + fidelity rating can cite an evaluation instead of asserting a level. + +## Why a decision receipt is worth more than a generation receipt + +A decision is a single forward pass with no sampling. Given the weight hash, +revision, dtype and the fitted temperatures, it is reproducible — so a receipt +over one is verifiable **by re-execution**, which a receipt over sampled text +can never be. `laya-mlx` publishes pinned revisions and per-file checksums, and +reports 63/63 validation questions matching upstream's selected answer in both +FP32 and FP16, which is exactly the material a receipt would bind. + +The precision matters and must be bound: probabilities differ across FP16 and +FP32 even where the selected label agrees. A receipt that omits dtype does not +reproduce. + +## The defect this category ships with today + +Loading the published Laya checkpoint raises: + +``` +laya-mlx: this checkpoint ships temperatures outside [0.5, 5] which would +distort confidence; clamping choice:11+=0.1006. Treat confidence from the +affected buckets as uncalibrated. +``` + +Confirmed on the real weights: `temperature_by_options_raw["choice:11+"]` is +`0.10058280825614929`. A temperature of 0.1 sharpens logits roughly tenfold, so +the shipped checkpoint reports near-certainty on what may be close to a coin +flip, for every choice question with 11 or more options. The porter found it by +reproducing upstream numerically and clamped it; the publisher ships it. + +That is the argument for this work in one line: **a calibrated probability is a +claim, and this category ships claims with no receipt.** + +## The attribution trade — two fabrics, one decision + +`two_fabrics.py` measures a decision model on the CPU, where the kernel's +per-process counter attributes energy to a **principal**, and compares it with +the same class of work on the GPU, where the only counter is an SoC rail that +counts the silicon and cannot tell one process from another. + +Measured on an M5 Max, macOS 27, 2026-10-04. Both CPU rows are +`Attribution::Process`, `Coverage::Whole` for CPU work, over spans far longer +than the counter's declared 10 ms period: + +| model | params | runtime | fabric | J/decision | p50 | attribution | +|---|---:|---|---|---:|---:|---| +| Laya | 421M | MLX | CPU | **0.486** | 629 ms | Process, whole | +| GLiNER2.5-Decide | 340M | torch | CPU | **0.555** | 480 ms | Process, whole | +| Laya | 421M | MLX | GPU | **0.104** | 11.9 ms | **Machine** (GPU share) | + +With the fabric held constant the two models are close: GLiNER2.5-Decide spends +14% more energy per decision and returns it 24% faster — while scoring 60.1% +against Laya's 46.6% on Fastino's 17-dataset suite, so on this evidence the +better-scoring model is not the more expensive one in any meaningful sense. + +Moving the *same* model to the GPU costs **4.7× fewer joules and is 53× +faster** — and the figure stops being attributable. IOReport's `GPU Energy` +rail is a real reading of the whole GPU; it cannot say whose work it was. So +the cheap path produces a number that cannot be bound to a principal, and the +expensive path produces one that can. + +That is the trade, and it is not a defect in either counter. It is why +`Attribution` is a separate axis from `Coverage`: a receipt that wants to say +*this principal spent these joules* has to choose the CPU path today, and a +receipt that only needs *this much energy was spent here* can take the GPU. + +The CPU figure is complete, not merely attributable. Over the GLiNER run the +GPU rail, bracketed by idle baselines, moved by **−0.0013 J per decision** — +negative, meaning the idle estimate slightly exceeded the real draw, so GPU +involvement is below the measurement's own noise. The process-attributed CPU +number is therefore the whole task, which is exactly what the GPU rows cannot +claim. + +## ⚠ Withdrawn: the GPU figure is not this decision's energy + +`gpu_share.py` measures what the table above assumed away. `accumulatedGPUTime` +per IOUserClient (I/O Registry, IOAccelerator subtree) gives each process's GPU +time; the cumulative totals are useless because WindowServer has been running +for days, so what matters is the **delta over the measured span**. + +Measured on this host, over a 4.06 s span while an MLX matmul loop ran: + +| process | GPU time in span | share | +|---|---:|---:| +| com.apple.Safari | 2.470 s | **99.5%** | +| WindowServer | 0.007 s | 0.3% | +| PhotosReliveWidget | 0.005 s | 0.2% | +| **this process** | **0.000 s** | **0.0%** | + +So a single unrelated application held essentially the whole GPU for the +duration. A rail delta taken over that span is a reading of Safari, not of the +decision. The host was at load average 132 on 18 CPUs at the time, running +another session's VVC/VTM encodes — which is the normal state of this machine, +not an unlucky moment. + +**Consequences for the table above:** + +- The two **CPU** rows stand unchanged. The kernel's per-process counter + attributes to a principal, so other processes' load does not enter them. + That is the whole value of `Attribution::Process`, demonstrated the hard way. +- The **GPU** row is withdrawn as a per-decision energy figure. 0.104 J is an + upper bound on *machine* GPU energy over the span, and on this host the bound + is dominated by other processes. Its **latency** (11.9 ms p50) is unaffected. +- The idle-bracket correction of 0.08% that I read as evidence of cleanliness + proves nothing. A bracket subtracts *steady* background; it cannot subtract a + contending process whose work overlaps the span and not the brackets. + +The rule to apply, already shipped elsewhere in the fleet: a rail reading may +be attributed to this process only when this process holds ≥99% of the GPU time +advancing in the span; where it holds none, its GPU energy is exactly zero; and +otherwise **no GPU figure is issued** and the span is CPU-only and partial. By +that rule this measurement issues no GPU figure at all. + +This is also why time-share apportionment cannot rescue it: apportioning a rail +by GPU-time share was falsified elsewhere at 3–15× error, because a contending +job can draw an order of magnitude more watts per GPU-second than a decode, and +GPU time itself inflates ~5.4× under contention. + +**To get a sound GPU figure**, re-measure on a host with no display server and +no other GPU client — which on this fleet means a headless Linux box with NVML, +not a Mac. diff --git a/mesh/tools/joules-per-decision/gpu_share.py b/mesh/tools/joules-per-decision/gpu_share.py new file mode 100644 index 000000000..bd55c573d --- /dev/null +++ b/mesh/tools/joules-per-decision/gpu_share.py @@ -0,0 +1,49 @@ +"""My process's share of GPU time over a span — the #146 admissibility rule. + +accumulatedGPUTime is cumulative since client creation, so WindowServer always +dominates the totals. The question that decides whether a rail reading may be +attributed to this process is the DELTA over the measured span. +""" +import re, subprocess, time, os, sys, threading + +def gpu_times(): + out = subprocess.run(["ioreg","-r","-c","IOAccelerator","-l","-w0"], + capture_output=True, text=True, timeout=30).stdout + tot = {} + for b in out.split("IOUserClientCreator")[1:]: + m = re.search(r'pid (\d+), ([^"]+)', b) + if not m: continue + key = (int(m.group(1)), m.group(2).strip()) + g = sum(int(x) for x in re.findall(r'"accumulatedGPUTime"\s*=\s*(\d+)', b[:4000])) + if g: tot[key] = tot.get(key, 0) + g + return tot + +def load(stop): + import mlx.core as mx + a = mx.random.normal((2048,2048)); b = mx.random.normal((2048,2048)) + while not stop.is_set(): + mx.eval(a @ b) + +me = os.getpid() +stop = threading.Event() +t = threading.Thread(target=load, args=(stop,), daemon=True); t.start() +time.sleep(0.5) +before = gpu_times(); t0 = time.perf_counter() +time.sleep(4.0) +after = gpu_times(); span = time.perf_counter() - t0 +stop.set(); t.join(timeout=3) + +delta = {} +for k, v in after.items(): + d = v - before.get(k, 0) + if d > 0: delta[k] = d +s = sum(delta.values()) +mine = sum(v for (pid, _n), v in delta.items() if pid == me) +print(f"span {span:.2f}s clients advancing: {len(delta)} total GPU time in span: {s/1e9:.3f}s") +print(f"this process (pid {me}): {mine/1e9:.3f}s = {100*mine/s if s else 0:.1f}% of GPU time in the span") +for (pid, n), v in sorted(delta.items(), key=lambda kv: -kv[1])[:6]: + tag = " <-- me" if pid == me else "" + print(f" pid {pid:>7} {n[:34]:<34} {v/1e9:>8.3f}s {100*v/s:>5.1f}%{tag}") +print() +print("#146 rule:", "HwShunt (>=99%)" if s and mine/s >= 0.99 else + ("exactly 0 (no GPU time)" if mine == 0 else "NO GPU FIGURE — span is CPU only, Partial")) diff --git a/mesh/tools/joules-per-decision/joules_per_decision.py b/mesh/tools/joules-per-decision/joules_per_decision.py new file mode 100644 index 000000000..b151ba77d --- /dev/null +++ b/mesh/tools/joules-per-decision/joules_per_decision.py @@ -0,0 +1,80 @@ +"""Joules per typed decision, metered with the same counter jcp-energy's rung 3 uses. + +ABI constants are lifted from jcp-energy (sizeof/offsetof-verified against the +macOS 26, 26.5 and 27 SDKs), not re-derived here. +""" +import ctypes, os, time, statistics, json, warnings + +FLAVOR, SIZE, ENERGY_NJ, PENERGY_NJ = 6, 464, 336, 344 +_libc = ctypes.CDLL("/usr/lib/libSystem.B.dylib") +_buf = ctypes.create_string_buffer(SIZE) +_pid = os.getpid() + +def read_nj(): + rc = _libc.proc_pid_rusage(ctypes.c_int(_pid), ctypes.c_int(FLAVOR), ctypes.byref(_buf)) + if rc != 0: + raise OSError(f"proc_pid_rusage failed: rc={rc}") + e = int.from_bytes(_buf.raw[ENERGY_NJ:ENERGY_NJ+8], "little") + p = int.from_bytes(_buf.raw[PENERGY_NJ:PENERGY_NJ+8], "little") + return e, p + +def liveness(window_s=0.5, interval_s=0.02): + """jcp-energy's rule: >=5 advancing intervals AND >=80% of them, over >=300ms.""" + advanced = total = 0 + prev, _ = read_nj(); t_end = time.perf_counter() + window_s + while time.perf_counter() < t_end: + t1 = time.perf_counter() + interval_s + x = 0 + while time.perf_counter() < t1: # burn CPU + x += 1 + cur, _ = read_nj(); total += 1 + if cur > prev: advanced += 1 + prev = cur + ok = advanced >= 5 and total > 0 and advanced / total >= 0.80 + return ok, advanced, total + +import laya_mlx as laya +with warnings.catch_warnings(): + warnings.simplefilter("ignore") + agent = laya.load("aac6fef/laya-mlx", dtype="float16") + +STATE = "I was billed twice. Please refund the duplicate today." +Q1 = {"refund": {"type":"noul","instructions":"Does the customer ask for money back?"}} +Q3 = { + "department": {"type":"choice","instructions":"Which team should handle this request?", + "criteria":{"billing":"invoices, payments, refunds","technical":"bugs and outages","sales":"new purchases"}}, + "urgency": {"type":"score","instructions":"How urgent is this request?","criteria":["not urgent","soon","critical"]}, + "refund": {"type":"noul","instructions":"Does the customer ask for money back?"}} + +def bench(label, questions, n, device): + a = laya.load("aac6fef/laya-mlx", dtype="float16", device=device) if device else agent + for _ in range(3): a.predict(STATE, questions) # warm + e0, p0 = read_nj(); lat = [] + t0 = time.perf_counter() + for _ in range(n): + t = time.perf_counter(); r = a.predict(STATE, questions); lat.append((time.perf_counter()-t)*1e3) + wall = time.perf_counter() - t0 + e1, p1 = read_nj() + u = r["usage"] + return { + "label": label, "device": device or "default", "n": n, + "p50_ms": round(statistics.median(lat), 2), + "p95_ms": round(sorted(lat)[int(0.95*len(lat))-1], 2), + "wall_s": round(wall, 3), + "cpu_J_total": round((e1-e0)/1e9, 6), + "cpu_J_per_decision": round((e1-e0)/1e9/n, 6), + "pcore_J_per_decision": round((p1-p0)/1e9/n, 6), + "input_tokens": u["input_tokens"], "output_tokens": u["output_tokens"], + "state_tokens": u["state_tokens"], + } + +ok, adv, tot = liveness() +print(json.dumps({"counter":"ri_energy_nj via proc_pid_rusage(RUSAGE_INFO_V6)", + "live": ok, "advancing_intervals": adv, "intervals": tot, + "rule": "jcp-energy rung 3: >=5 advancing AND >=80%, >=300ms"}, indent=2)) +rows = [] +for dev in ("cpu", "gpu"): + for lbl, qs, n in (("1 question (noul)", Q1, 30), ("3 questions (choice+score+noul)", Q3, 30)): + try: rows.append(bench(lbl, qs, n, dev)) + except Exception as ex: rows.append({"label": lbl, "device": dev, "error": repr(ex)[:200]}) +print(json.dumps(rows, indent=2)) diff --git a/mesh/tools/joules-per-decision/two_fabrics.py b/mesh/tools/joules-per-decision/two_fabrics.py new file mode 100644 index 000000000..d960b7223 --- /dev/null +++ b/mesh/tools/joules-per-decision/two_fabrics.py @@ -0,0 +1,71 @@ +"""One bounded decision, two fabrics, two attributions. + +CPU: the kernel's per-process counter — Coverage::Whole for CPU work, +Attribution::Process, declared period 10 ms. +GPU: IOReport's `GPU Energy` rail — a real reading of the whole GPU that +cannot tell one process from another, so Attribution::Machine, bracketed by +idle baselines. +""" +import ctypes, os, time, json, statistics, warnings +warnings.filterwarnings("ignore") +exec(open('rails.py').read().split('import time, subprocess, sys')[0]) # IOReport + +_l = ctypes.CDLL("/usr/lib/libSystem.B.dylib"); _b = ctypes.create_string_buffer(464); _pid = os.getpid() +def cpu_nj(): + _l.proc_pid_rusage(ctypes.c_int(_pid), ctypes.c_int(6), ctypes.byref(_b)) + return int.from_bytes(_b.raw[336:344], "little") +def gpu_nj(s0, s1): + d = IOR.IOReportCreateSamplesDelta(s0, s1, None); t = 0 + for g, sg, name, unit, val in rails(d): + if name == "GPU Energy": + assert unit == "nJ"; t += val + return t + +PERIOD_NS = 10_000_000 # the measured, declared period of both counters + +STATE = ("Guest in room 1408 says the AC has been out since yesterday and they want to " + "move tonight or leave. They also asked for the incidentals hold to be released.") +SCHEMA = { + "intent": ["maintenance", "room_change", "checkout", "billing", "complaint", "amenity_request"], + "priority": ["low", "normal", "high", "urgent"], + "needs_human": ["yes", "no"], +} +N = 60 + +def idle(secs): + s0 = sample(); c0 = cpu_nj(); t = time.perf_counter(); time.sleep(secs) + s1 = sample(); c1 = cpu_nj(); dt = time.perf_counter() - t + return gpu_nj(s0, s1) / dt, (c1 - c0) / dt + +# ─── CPU: GLiNER2.5-Decide, 340M, Apache-2.0 ──────────────────────────────── +import torch +torch.set_num_threads(os.cpu_count() or 8) +from gliner2 import AutoExtractor +t0 = time.perf_counter() +m = AutoExtractor.from_pretrained("fastino/GLiNER2.5-Decide") +load_s = time.perf_counter() - t0 +try: m.model.to("cpu") +except Exception: pass + +for _ in range(3): m.classify_text(STATE, SCHEMA) +pre_g, pre_c = idle(1.0) +s0 = sample(); c0 = cpu_nj(); t0 = time.perf_counter(); lat = [] +for _ in range(N): + t = time.perf_counter(); out = m.classify_text(STATE, SCHEMA); lat.append((time.perf_counter()-t)*1e3) +span = time.perf_counter() - t0 +s1 = sample(); c1 = cpu_nj() +post_g, post_c = idle(1.0) +cpu_j = (c1 - c0) / 1e9 +gpu_leak = (gpu_nj(s0, s1) - (pre_g + post_g) / 2 * span) / 1e9 + +gliner = { + "model": "fastino/GLiNER2.5-Decide (340M, Apache-2.0, DeBERTa-v3-large)", + "fabric": "cpu", "attribution": "Process", "coverage": "Whole (CPU work)", + "load_s": round(load_s, 2), "decisions": N, "span_s": round(span, 3), + "p50_ms": round(statistics.median(lat), 2), "p95_ms": round(sorted(lat)[int(0.95*N)-1], 2), + "cpu_J_per_decision": round(cpu_j / N, 6), + "gpu_J_per_decision_leak": round(gpu_leak / N, 6), + "span_gradeable": span * 1e9 >= PERIOD_NS, + "answer": out, +} +print(json.dumps(gliner, indent=2, default=str))