33f0b89124
The owner's decision, applied: a vendor energy counter (RAPL, NVML, IOReport) is a real reading, and what separates one such reading from another is provenance, not accuracy. This crate gave RAPL and NVML `HwShunt` and gave the IOReport counter `ModelBased`, on the reasoning that Apple's Energy Model is a vendor model. That reasoning grades accuracy. What the counter could not establish was provenance: on macOS 27.0.1 its CPU rails are frozen for an unprivileged reader, and that is now caught by the liveness probe. A counter that has passed it is as much a reading as RAPL is. The grade. `AppleIoReportCounter` declares `HwShunt`, and stamps it on every reading through one constant, `READING_GRADE`, shared by `provenance()` and `read()`. It earns the grade by being live: the counter exists to read only where `admits()` let it through (CPU rail `Live`), and a declined counter produces no reading to carry any grade, so a frozen host still meters through the load-average estimator at `Estimator`. The rule, written once and shared (jouleclaw SPEC §1.3, §9.4; JCR-1 §2.3; the crate docs; vendor-energy PROTOCOL/README): a vendor counter's reading MAY be graded as a hardware reading only while the counter is demonstrably live for the domain the figure claims; the implementation MUST have observed the counter covering that domain advance under load (for IOReport: the CPU rail itself, in at least 5 intervals, 4 of every 5, over at least 300 ms while the probing thread burns CPU). A counter present but not live MUST NOT produce a hardware-graded figure; the implementation falls back to its estimator or reports the work unmetered, and SHOULD record why. A figure MUST NOT claim coverage of a domain its counter does not read; where the format has a coverage field, such a figure is Partial. No partial reading from the GPU rail. Where the CPU rail is frozen but `GPU Energy` still advances (the M5 Max), the counter stays declined. The GPU rail is machine-wide: every GPU client on the host moves it, display compositing included, with nothing of the metered work running. It does not cover part of the work; it covers other work, so no coverage field makes it honest. The rationale is in the module docs and SPEC §1.3. Docs and data that said otherwise: - SPEC §1.3 table and floors, §9.4; JCR-1 §2.3 and the crate README (toy vectors become measurements only off counters that were live). - JCR-1 vector 6's description said "measured on Apple IOReport". Its joules are toy values like every vector's. Only the `description` changed; it sits outside the signed core, which `verify.py` builds from `receipt` alone. Both verifiers pass 10/10 after the edit: `verify.py` (cbor2 6.1.4, cryptography 50.0.2, exit 0) and the independent `verify.go` (exit 0). `git diff` of the file is that one line; a field-by-field JSON diff finds only (6, description). The conformance README's slot 6 line follows. - vendor-energy conformance: README row 27 names `AppleIoReportCounter` and the `HwShunt` grade, with the liveness condition a filled fixture must meet; `apple-silicon-ioreport.json` is an unsigned, empty stub (`signed_by` null, every workload not run, no hash of it anywhere), so its `impl` and `provenance` are corrected in place. - MEASURED_ENERGY.md: a dated note that the 74.1 J / 25.5 W row was committed 2026-06-12, predates macOS 27, and recorded neither chip nor OS. The "reads CPU/GPU/ANE/DRAM with no sudo" claim is replaced by what macOS 27.0.1 does: the counter opens without sudo, but the CPU rails are frozen (all 645 rails on the M3 Ultra), so those hosts are estimated. - local-index / rerank tier SHOULDs, the CLI's `meter` help, meter.rs, apple.rs, energy_probe, Cargo feature comments, harness-econ docs. - l3-jouleclaw README: it claimed `HwShunt` on RAPL/NVML hosts, but the binding builds jouleclaw-energy with `ioreport` only, so off Apple Silicon the receipt is the token estimate. It now says what happens on each host. - mesh-schema's `HwShunt` definition includes vendor counters subject to liveness; mesh-meter, the mesh README and mesh-engine-gguf no longer say real silicon alone yields `HwShunt`. - The Live-branch assertions in mesh-meter, l3-jouleclaw and jouledesk-github expected `ModelBased` for IOReport; they expect `HwShunt`. jouledesk-github's fixtures that paired `apple-ioreport` with `ModelBased` now pair it with `HwShunt`, and keep a `ModelBased` case under a neutral meter name. Tests. `a_live_counters_readings_are_graded_as_hardware_readings` builds a counter on `Live` directly. That opens the real all-channel subscription, which opens here even though the gate declines this host. It reads it twice and asserts each reading is `HwShunt` over `SocTotal`. On this M5 Max (macOS 27.0.1): "an admitted IOReport counter: opens=true, 2 readings, each graded HwShunt", so the read path is covered on a frozen host. With `--features "ioreport rapl"`, `a_live_ioreport_reading_has_the_grade_of_a_rapl_reading` compares the grade with `RaplCounter`'s own. The used/declined tests assert `HwShunt` in their live branch. This host still takes the declined branch: "ioreport declined (CpuRailFrozen): 154 ms of CPU work metered by `apple-silicon` as 6200000 µJ, provenance estimator". Run on this M5 Max: jouleclaw-energy 24+1+1 (default), 76 lib + 1 + 1 (`ioreport`), 79 lib (`ioreport rapl`); jouleclaw-jcr1 50; harness-econ 14; local-index 41; rerank 37; cli 10; mesh-meter (`hw-meter-apple`) 10; mesh-engine-gguf 1; l3-jouleclaw 2 + 4 (2 ignored, need a model); jouledesk-github 12. All pass. Falsified, each mutant restored with `git checkout` and the tree confirmed clean: - `READING_GRADE` set back to `ModelBased`: the new test fails at the declared grade (left: ModelBased, right: HwShunt), 75 passed / 1 failed. - Only `read()`'s stamp set back to `ModelBased` (the declared grade left at `HwShunt`): the new test fails on the reading itself, "a live counter's reading is not graded as a hardware reading" (ModelBased, SocTotal) vs (HwShunt, SocTotal), 75 / 1. The read path is what the test pins. Co-Authored-By: Claude Opus 5.5 <[email protected]>
109 lines
5.1 KiB
Markdown
109 lines
5.1 KiB
Markdown
# JouleClaw vendor-energy measurement protocol
|
||
|
||
Honest energy accounting requires honest energy readings. The
|
||
[`EnergyCounter`] trait in this crate names three provenance tiers:
|
||
|
||
- **HwShunt** — a hardware reading: a shunt (board-level current
|
||
sense, IPMI sensor), or a vendor energy counter (Intel/AMD RAPL
|
||
MSR, NVML cumulative energy, Apple IOReport) while it is
|
||
demonstrably live for the domain the figure claims — the
|
||
measurer observed the counter covering that domain advance under
|
||
load. A counter that is present but not live (frozen, or
|
||
republished in lumps) gives no HwShunt figure. Authoritative.
|
||
- **ModelBased** — a vendor-published energy model (DCGM-X,
|
||
NVML `power.draw`, vendor whitepapers) consulted at run time.
|
||
Trustworthy if the model itself was validated by HwShunt.
|
||
- **Estimator** — anything derived from wall-clock × static
|
||
power ceiling. Useful as a floor; never as a ceiling.
|
||
|
||
The circuit breaker is only as honest as the **worst** counter in a
|
||
request's span — a single Estimator reading taints the whole walk.
|
||
That is by design.
|
||
|
||
## Why this file exists
|
||
|
||
The trait + provenance taxonomy is one half of the honesty story.
|
||
The other half is **actual measured numbers** for specific silicon
|
||
under specific workloads, so an operator can compute the energy
|
||
cost of a cascade walk before issuing it. This document specifies
|
||
*how* a third party with vendor hardware produces those numbers in
|
||
a way the rest of the JouleClaw ecosystem can consume.
|
||
|
||
Per-vendor fixture files live in `vendors/`. Each one ships
|
||
**empty** — values are `null` until a measurement run fills them.
|
||
A fixture with non-null values is a *standing claim* by its author
|
||
that the numbers were produced by the protocol below.
|
||
|
||
## Measurement protocol (vendor-agnostic)
|
||
|
||
Every workload measurement MUST capture:
|
||
|
||
1. **Hardware identification** —
|
||
`vendor`, `model`, `tdp_w` (datasheet), `host_os`.
|
||
2. **Counter** — which trait impl was used (`RaplCounter`,
|
||
`NvmlCounter`, `AppleIoReportCounter`, `AppleSiliconMeter`, or a
|
||
vendor-specific impl).
|
||
3. **Provenance** — `HwShunt` | `ModelBased` | `Estimator`. `HwShunt`
|
||
only for a counter observed live over the run (for Apple IOReport,
|
||
`ioreport_status()` returned `Live`); record in `method` how
|
||
liveness was established.
|
||
4. **Resolution** — `resolution_uj` (smallest delta the counter
|
||
meaningfully reports) and `min_window_ns` (smallest sampling
|
||
window below which the reading is noise-dominated). Honest
|
||
self-reporting; do not round below.
|
||
5. **Workload** — one of:
|
||
- `idle_1s` — power draw with no workload for one second.
|
||
- `tier_L0_lookup` — N = 10,000 keyed map lookups, no
|
||
mathematics. Floor energy.
|
||
- `tier_L1_local_index` — BM25 search over the seeded reference
|
||
corpus (`jouleclaw-local-index`'s in-memory fixture).
|
||
- `tier_L3_model_7b_token` — one decoded token of a 7B-param
|
||
model at fp16 with a 64-token prompt. The token-energy
|
||
baseline.
|
||
- `tier_L3_model_70b_token` — same, at 70B-param fp16.
|
||
6. **Method** — `cold` (counter zeroed before the workload starts)
|
||
or `warm` (counter sampled at workload boundaries).
|
||
7. **Joules per workload-unit** — `joules_uj_per_unit` for
|
||
token-shaped workloads; `joules_uj_total` otherwise.
|
||
8. **Sample count** — `n_samples` and `stddev_uj` so consumers
|
||
can reason about variance.
|
||
9. **Date** — ISO-8601 timestamp of the measurement run.
|
||
10. **Attestation** — `signed_by` (optional Ed25519 verifying key
|
||
hex of the operator), `attestation_url` (optional URL to the
|
||
public log entry / paper / blog post backing the claim).
|
||
|
||
## What a conformant fixture file looks like
|
||
|
||
See `vendors/intel-rapl-template.json` for the empty shape.
|
||
Every workload from §5 above MUST be measured (or explicitly
|
||
marked `"workload_not_run": true`); cherry-picking the favourable
|
||
ones isn't conformant. The whole point is to make the
|
||
expensive-tier-token energy visible alongside the floor.
|
||
|
||
## Why the fixtures ship empty
|
||
|
||
The workspace dev machine cannot produce HwShunt readings for
|
||
non-Intel CPU silicon, any NVIDIA GPU, any AMD GPU, any Apple
|
||
Silicon GPU, any TPU, or any Chinese accelerator. Pretending
|
||
otherwise — populating the fixtures with numbers from blog posts
|
||
or vendor whitepapers — would corrupt the provenance contract the
|
||
trait exists to enforce. Honest empty fixtures > dishonest filled
|
||
ones.
|
||
|
||
A third party with the hardware fills them, signs them, and PRs
|
||
them back. Anyone else holding the fixture-author's verifying key
|
||
can then verify the claim was actually made by that operator.
|
||
|
||
## Roadmap (not part of this conformance)
|
||
|
||
- A `jouleclaw-vendor-energy-attest` binary that takes a fixture
|
||
JSON, signs it via Ed25519, and emits a wire-form attestation
|
||
in the same shape as JCR-1 receipts (so the same Python/Go
|
||
verifiers can audit it).
|
||
- A registry of community-contributed fixtures with conflict
|
||
resolution (when two operators measure the same silicon and
|
||
disagree, both are kept; consumers decide which to trust).
|
||
- Cross-vendor parity: at least one HwShunt fixture per major
|
||
silicon lineage (Intel CPU / AMD CPU / NVIDIA GPU / AMD GPU /
|
||
Apple Silicon / Cloud TPU / Huawei Ascend / Cambricon /
|
||
Qualcomm Cloud AI).
|