jouleclaw-energy: IOReport only when its CPU rail keeps moving; consumers never zero-fill or relabel #117

Merged

dcharlot merged 23 commits from jouleclaw-energy/cpu-rail-liveness into main 2026-10-01 05:51:08 -04:00

Conversation 0 Commits 23 Files changed 65 +7887 -698

2026-09-30

Transaction Science

jouleclaw-energy: RAPL is the sum of every package zone, each of which must be live and each wrapped at its own range ...

`RaplCounter` read one file, `/sys/class/powercap/intel-rapl:0/energy_uj`, and
its liveness probe watched that one file, cached per path. On a multi-socket
server that is package 0's energy alone, graded `HwShunt` for the domain
`CpuPkg` while every other socket's energy is missing. And nothing makes zone 0
a package: `intel-rapl:<n>` is only an identifier, and each zone says what it
is in its `name` file, so a host whose zone 0 is the platform zone (`psys`) or
a `dram` zone had that domain read and probed as the package. proof-cascade
does this right since 9334ec0c; eoc-meter now does too (26165a77); this is the
same rule here.

- The package set is listed from the powercap directory: every top-level
  `intel-rapl:<n>` zone whose `name` reads `package-<digits>`. Left out wherever
  they sit, zone 0 included: `psys` (the platform zone, whose energy includes
  the packages', so summing it beside them counts them twice), `dram`, the
  `core`/`uncore`/`dram` sub-zones (`intel-rapl:0:0`), the control type, and
  the MMIO interface (`intel-rapl-mmio:0`, also named `package-0`, which
  republishes a package). The listing reads the directory, not fixed indices.
- Liveness: one channel per package, and the shared rule asks every channel to
  be live, so RAPL is live only when EVERY package keeps advancing while the
  probe burns CPU. A host with no package zone (only `psys`) is `Unavailable`
  with nothing opened or burned.
- The cache is per package set, keyed by the package zones' directories, not
  per file path. `RaplCounter::new()` lists the set and reuses a verdict for
  the same set; the counter sums exactly the packages that were probed.
- `read()` returns a running total: the first read's package counters summed,
  then each package's delta added, each taken modulo its own
  `max_energy_range_uj`. Summing the raw counters would make one package's
  wrap a fall of the sum, and with sockets at different ranges no single range
  is right. Before this, a wrap of the one zone was a fall, which `jcr1_mint`
  and the like take as no reading; now a wrap is read across. The files are
  read under the counter's lock, so two threads' reads are folded in the order
  they were taken, and an older read is never taken for a wrap. A package that
  cannot be read fails the read (`Platform`): no figure that leaves a package
  out. A package whose counter falls while its range is unknown (absent, or 0)
  makes that read `CounterWrap`, and every later one too, because the total
  would be short by energy no one can know.
- The read path is no longer compiled only for Linux: off Linux the counter is
  still never available (`rapl_liveness` is `Unavailable` there, with no
  probe), but the code it would run is the code the tests run on a Mac.

API: `RAPL_ENERGY_PATH` and `RaplCounter::with_path(&'static str)` are gone, as
a single `energy_uj` path cannot say which domain it is or stand for every
package; `RaplCounter::with_root(dir)` takes a powercap directory instead (a
container or chroot that mounts sysfs elsewhere), and `POWERCAP_ROOT` names
the default. `rapl_liveness()` takes no argument: it is this host's package
set. Nothing in the monorepo called the removed items. Docs that said a RAPL
figure is one file are corrected (`liveness.rs`, `detect_meter`, and the
`jcr1_mint` example's header).

Tests, on fake powercap directories under the temp dir (removed when done), so
they run on macOS:
- a two-socket server (the control type, package-0 and package-1 with core and
  dram sub-zones, psys, a top-level dram, intel-rapl-mmio:0) lists packages 0
  and 1 only, and with every zone moving the delta is the two packages' energy
  (250,000 + 400,000 uJ) and nothing else;
- zone 0 named psys beside package zone 1: zone 1 alone is listed, sampled and
  summed;
- per-zone wrap: package 0 wraps at 262,143,328,850 while package 1 advances,
  then package 1 wraps at 65,532,610,987: deltas of 1,700 and 2,500 uJ; taken
  at the other zone's range, package 1's wrap would be off by the difference
  of the two ranges;
- a package with no range, absent or 0, is summed while it rises; its fall is
  `CounterWrap`, and so is the next read;
- an unreadable package fails the read and the opener;
- an only-psys host, an empty directory and a missing one: no package, the
  real probe returns `Unavailable` in under 100 ms (no burn), and
  `RaplCounter::with_root` is unavailable and reads `NoCounter`;
- scripted: two packages advancing are Live, one held is Frozen;
- the cache: once per package set, and a set with one more package is another
  set; the key of a listed tree is its package directories;
- the production probe (real reads, real CPU burn, real clock) on a two-socket
  tree: held packages are Frozen after the whole budget; both packages advanced
  by a writer thread are Live inside it; package 0 and psys advanced while
  package 1 holds is Frozen.

Falsified, each mutant applied to this commit, run, and restored with
`git checkout` (blob hash equal to HEAD's):
- back to `intel-rapl:0` only, whatever its name: 8 of the 14 RAPL tests fail;
- every top-level zone summed (no name check): 5 fail;
- every package wrapped at package 0's range: 1 fails, the per-zone wrap test;
- the probe watching the first package alone: 1 fails, the production probe
  with package 1 held;
- the cache keyed by the first package's path alone: 1 fails.

What ran where, on this M5 Max (macOS 27.0.1): jouleclaw-energy `--features
rapl` 48 lib + 2 integration tests pass, `--features ioreport,rapl,nvml` 111 +
2, default features 34 + 2; mesh-meter 2 by default, 10 with
`--features hw-meter-rapl`, 12 with `--features hw-meter-apple`;
jouledesk-github 12; the `jcr1_mint` example compiles. For
x86_64-unknown-linux-gnu, `cargo check --all-targets --features rapl` and
`cargo clippy --all-targets` with `rapl` and with `rapl,nvml` pass with no
warning in a file this touches (clippy's existing ones are in jouleclaw-core,
the ledger tests and the IOReport FFI), and mesh-meter's clippy with
`hw-meter-rapl` is clean. The sysfs reads under the real powercap directory
and the Linux-only cached probe are type-checked, not run: there is no Linux
host with readable RAPL here.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

c07d4c5b47

• View at this point in history

2026-09-30 22:20:49 -04:00

Transaction Science

jouleclaw-energy: IOReport is live only when every CPU rail it sums is live ...

The IOReport counter sums every CPU energy aggregate in the Energy Model group:
`CPU Energy`, or on a multi-die part each `DIE_<n>_CPU Energy` (`counts_once`).
Its liveness rule (`RailWatch`) called the counter live once ANY one of those
rails sustained advances (five or more intervals, four in every five, over at
least 300 ms). On an M3 Ultra, `DIE_0_` live beside a frozen `DIE_1_` would
have passed, and every reading would have been graded HwShunt over SocTotal
while leaving the frozen die's energy out. The shared `liveness` rule that RAPL
and NVML are gated on already asks every channel the figure sums to be live,
for exactly this reason (one frozen GPU of two); IOReport was the one backend
that did not.

`RailWatch::is_live` now asks every CPU rail present in the samples to be
sustained, each on its own count, and no rail at all is never live (`all` over
nothing would be). A rail counts from the first sample it appears in, the
baseline included, so a die that drops out of later samples holds the counter
back instead of leaving the rule. Nothing else moves: the numbers, the keying
(name and ordinal), the probe loop, the cache and the gate are as they were, and
a single-die host (one `CPU Energy` rail, as on the M5 Max) is judged exactly as
before.

The trade-off, stated in the docs: the probe burns on one thread, so a die it
does not run on must still be seen advancing on its idle draw. If a live die's
idle draw cannot step its rail in four of every five ~31 ms intervals, the
counter is declined and the host estimated, which is the safe side. That cannot
be measured today: every CPU rail is frozen for an unprivileged reader on macOS
27.0.1, on both Macs here.

The test oracle follows the rule. `oracle_moved` said "moved" when one rail kept
changing, so the real-host check of the dangerous direction (probe Live, so the
oracle must have seen the rails move) would have let an any-rail probe through
on a mixed host. It now requires every CPU rail it finds to keep changing, and
the freshness test's Live check likewise requires every CPU rail to change
inside the probe's window.

Tests. One existing case asserted the defect itself: `DIE_0_` held beside an
advancing `DIE_1_` was expected Live. It now has both dies advancing. New, all
scripted:

- `both_dies_sustained_are_live`: Live at the 10th interval unloaded, the 5th
  loaded, and at the 10th with each die missing a different one in five.
- `one_die_sustained_beside_a_frozen_die_is_frozen`: `DIE_0_` sustained with
  `DIE_1_` held, the reverse, and `DIE_1_` moving only in a two-step refresh,
  all Frozen at both paces over 39 intervals.
- `watch_one_die_sustained_beside_a_frozen_die_is_frozen` and
  `the_production_status_of_one_live_die_beside_a_frozen_one_is_frozen`: the
  same through the probe loop, and through the real status function with only
  the IOReport group scripted. `CpuRailFrozen`, decided after the whole budget;
  both dies advancing is `Live`.
- `a_single_die_host_is_unchanged`: for six single-rail patterns at both paces,
  the verdict after every interval equals the shared one-channel `AdvanceWatch`,
  first live where the existing tests pin it.
- `a_die_that_drops_out_of_the_samples_holds_the_counter_back`, and
  `no_cpu_rail_is_absent_and_no_interval_is_no_verdict` extended over a whole
  window.

Falsified, each restored with git checkout: `all` back to `any` in `is_live`
(the old rule) fails four tests, the mixed ones going live at the 10th interval;
dropping the empty-rails guard fails `no_cpu_rail_is_absent...` and
`the_production_status_with_no_cpu_rail_is_absent`; counting rails only from
the first interval fails `a_die_that_drops_out_of_the_samples...`.

Checked: jouleclaw-energy with `--features ioreport` (92 lib tests, 2
integration) and default features (34, 2); mesh-meter with `--features
hw-meter-apple` (12); jouledesk-github (12). Clippy adds no warning on
jouleclaw-energy. On the M3 Ultra (macOS 27.0.1), the same lib test binary, run
once: 92 passed, `status=CpuRailFrozen oracle=CpuRailMoved(false)
selected=false`. The oracle found exactly two CPU rails across every group
there (`DIE_0_`, `DIE_1_`, both held), the rails the counter sums, and on the
M5 Max one (`CPU Energy`). Neither Mac can show one live die beside a frozen one
for real; that case is covered by the scripted tests only.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

c7a4d7102a

• View at this point in history

2026-09-30 21:29:12 -04:00

Transaction Science

l3-jouleclaw: a fabric-priced projection is graded by its weakest input ...

The L3 stage tagged every per-call figure a backend reported as ModelBased, on
the premise that it was "priced by THIS machine's IOReport calibration". It is
priced by whatever matmul figure each fabric member loaded: a calibration row at
the grade its writer declared, or a seeded figure, and the seeded GPU figures are
`Estimated` (a coarse model). A projection priced from an estimate was sealed as
a calibrated per-operation model. Joule-code SPEC §6 keeps those apart:
ModelBased is "computed from a calibrated per-operation model", Estimator "a
coarse constant estimator".

The fix needs each input's grade, which the ledger did not carry.

- jouleclaw-loader-gguf: `LedgerEntry` gains `projected_grade`, the
  `EnergyProvenance` of the cost row that priced `projected_uj`, taken from the
  same row, so the two are present or absent together. Additive; nothing else
  constructs the struct.
- l3-jouleclaw: `projection_grade(window)` grades a projection by its weakest
  input. ModelBased only when every member that ran was priced by a `Measured`
  or `Benchmarked` figure; Estimator when any was priced by an `Estimated`
  figure, when any ran unpriced (its work is missing from the sum), or when the
  window is empty (a figure no priced input accounts for). A projection is never
  a reading, so it is never HwShunt. The stage now opens the attribution window
  around the call itself (reset before, snapshot after), so a figure is graded
  by this call's inputs and not by entries an earlier call left behind.

Tests. The grading table covers Measured, Benchmarked, both, Estimated alone and
beside each calibrated grade, an unpriced member, and no input. In the stage, a
backend that runs a real matmul through the dispatcher with the fabric off, and
one that runs nothing, both report 0.25 J and get Estimator (main: ModelBased).
Two integration tests, each its own process, switch the fabric on with
`JOULECLAW_FABRIC=1` and point `JOULECLAW_FABRIC_CALIBRATION` at a file they
write, pricing `local/cpu-neon`'s matmul at 2.5e8 µJ/GFLOP: declared `Measured`,
the window carries `Measured` and the receipt says ModelBased, 1024 µJ;
declared `Estimated`, the window carries `Estimated` and the receipt says
Estimator, 1024 µJ. The machine's own ~/.jouleclaw/fabric-calibration.json is
neither read nor written by these tests.

Ran: `cargo test` in l3-jouleclaw (the model-backed tests stay #[ignore], no
model was loaded), `cargo test -p jouleclaw-loader-gguf --features fabric-meter
--lib` (304 passed), loader-gguf examples checked; clippy reports nothing new.

Not done: a projection under 1 µJ still truncates to a 0 µJ receipt figure, as
before; real hybrid-model calls are far above that.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

f521fb17be

• View at this point in history

2026-09-30 20:48:47 -04:00

Transaction Science

mesh-meter: a GPU-only reading is an estimate of the request, never HwShunt ...

Mesh SPEC §3.3 reads a mesh/1 figure as the energy of the whole request, and
mesh/1 has no coverage field. So a figure from a counter that does not cover the
whole request (a GPU counter, with the CPU that drives the GPU unmetered) must
not be graded HwShunt: it would claim the CPU it did not read. `HwMeter` mapped
every backend's grade through unchanged, so a live NVML counter, whose domain is
`Gpu`, would have sealed GPU-only joules as a measurement of the whole request.

The bridge now asks whether the reading's domain covers the whole request
(`covers_whole_request`): the whole SoC and a CPU package do; a GPU, NPU or ANE
alone, one core, DRAM, or an external rail of unknown scope do not. The match
is exhaustive, so a domain added to the family meter must be decided there
before it compiles. A reading that does not cover the request is graded as mesh
grades an estimate of the whole request, `Estimator` with no `measured_uj`, and
its figure is the counter's reading plus the bridge's own `EstimatorMeter` over
the same window for the rest of the host: a span is billed what the counter read
and an estimate of what it did not, never less than the counter read. Whole-
request backends pass through as before; the failed-read fallback continues
from the composed figure.

Tests. A stepping backend over `Gpu` at each grade (HwShunt, ModelBased,
Estimator) yields `Estimator`, no measured figure, an estimate above the GPU's
own step; the same backend over `SocTotal` passes through as a HwShunt
measurement of exactly its step, so only the domain differs. Every domain's
classification is listed. With `hw-meter-nvml`, the family's `NvmlCounter`
reports `Gpu` and is not whole-request.

Ran: `cargo test -p mesh-meter` with no features, `hw-meter`,
`hw-meter-apple` and `hw-meter-nvml`; `mesh-core --features hw-meter-apple`;
`mesh-conformance`; clippy clean on mesh-meter; `cargo check --tests` for
`x86_64-unknown-linux-gnu` with `hw-meter-rapl,hw-meter-nvml`.

Not done: a RAPL `CpuPkg` figure is still whole-request, which is right on a
host serving from its CPU and an undercount on one serving from a GPU it does
not read. The bridge cannot see where the work ran.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

55c82effd3

• View at this point in history

2026-09-30 20:44:15 -04:00

Transaction Science

jouleclaw-energy: RAPL and NVML read as hardware only while their counters are live ...

The crate's honesty rule, in its own docs and in JouleClaw SPEC §1.3: a vendor
energy counter (RAPL, NVML, IOReport) is a hardware reading only while it is
demonstrably live for the domain the figure claims. The IOReport backend has
been gated on that since its CPU rail was found frozen on macOS 27.0.1; RAPL and
NVML were not. `RaplCounter` was available as soon as `energy_uj` parsed once,
so a counter passed through by a hypervisor, or one that holds its value, would
have been selected and billed every span zero at HwShunt. `NvmlCounter` graded
every reading HwShunt, including its `power_usage()` integral, which the crate's
own Provenance docs and SPEC §1.3 call ModelBased (NVML `power.draw`), and it
never asked whether the cumulative-energy counter moved.

The rule now lives in one place, `liveness`, with the numbers the IOReport probe
uses: while the probing thread burns its own CPU time between samples, every
channel the figure sums must advance in at least five sample intervals, four in
every five of those observed, over at least 300 ms. The IOReport backend's
constants now alias these, so the backends cannot drift apart; what differs
there is only what is watched (any one CPU rail among many channels). Here every
channel must be live, since an NVML figure sums every GPU and one frozen device
would leave part of the claimed domain unread. `CounterLiveness` (`Unavailable`,
`Frozen`, `Live`) is exported so a caller can say why a host is not metered.

- RAPL: `RaplCounter` is available only when its `energy_uj` is readable and
  live, probed once per process per path and cached (`rapl_liveness`), 25 ms of
  thread CPU between samples. A root-only file (CVE-2020-8694) never gives a
  sample and is `Unavailable`; a held or lump-republished one is `Frozen`.
  Off Linux nothing is probed.
- NVML: the cumulative-energy counter must be supported on every GPU and live
  (100 ms ticks, two of its ~50 ms sample periods), probed once per process.
  Then readings are HwShunt. Otherwise the counter integrates `power_usage()`
  and grades it ModelBased: a Riemann sum advances whenever time passes, so no
  probe can vouch for it, and it is never HwShunt.
- `libc` becomes a Unix dependency (was macOS-only) for the probe's thread CPU
  clock on Linux; Cargo.lock does not change.

Tests. The rule is driven by scripts (`liveness`): live at the tenth 31 ms
interval and not before, a refresh in one to five steps frozen, four-in-five
kept and three-in-five refused, every channel required, no interval
unavailable, the probe stopping when live and capped when failing. RAPL's
production probe runs against real files on any OS: a file that holds its value
is Frozen after the full budget, a missing one Unavailable at once, and one a
second thread keeps advancing is Live in 0.3 to 1 s. Availability follows only
a Live verdict; the per-path cache probes once. NVML's mode and grade are
asserted for every (supported, liveness) pair, and a scripted two-GPU probe is
Frozen when one GPU holds.

Checked: `cargo test -p jouleclaw-energy` with default features, `rapl`,
`nvml`, `ioreport`, and all three; `cargo check --tests` and clippy on
`x86_64-unknown-linux-gnu` with `rapl,nvml`, and `cargo check --tests
--target x86_64-pc-windows-msvc --features nvml`. No NVIDIA GPU or readable
RAPL file was available, so neither real counter's probe has run on hardware.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

67cd4e7b73

• View at this point in history

2026-09-30 20:41:54 -04:00

Transaction Science

jouleclaw-compliance: a passport calls inference energy measured only on a live hardware reading ...

`PassportBuilder::build` accepted any pairing of `measurement_method` with
`compliance.eu_ai_act.inference_energy_measured`. That includes `IoReport`, which
`MeasurementMethod::provenance` itself grades `Estimator`, paired with
`inference_energy_measured: true`. A sealed passport could then disclose
estimator energy as measured inference energy under the EU AI Act field. It
also accepted `Estimated` with `true`.

The rationale beside `IoReport` was "estimator until the IOReport FFI ships".
The FFI shipped long ago. The real reason, after the previous commits: a vendor
energy counter is a real reading, and what decides its grade is provenance.
Was the counter live for the domain the figure claims? A bare `IoReport` records
no such observation, and where the CPU rail is frozen (macOS 27.0.1, for an
unprivileged reader) the host's figure is the estimator's. So `IoReport` stays
`Estimator`, and the stale wording is replaced in the variant doc, the crate
doc and the test.

The fix.
- New `MeasurementMethod::IoReportLive`: IOReport read while its CPU energy rail
  was observed live (`IoReportStatus::Live`). It is graded `HwShunt`, like RAPL
  and NVML. Display: "IOReport (live)". It is additive, so existing serialized
  passports still deserialize.
- `is_live_hardware_reading()` is true iff the method's grade is `HwShunt`.
- `build()` returns `PassportError::MeasuredWithoutHardwareReading { method,
  provenance }` when `inference_energy_measured` is true and the method is not
  a live hardware reading. That means a bare `IoReport` or `Estimated`.
  Unclaimed, every method still builds.

Tests: `a_measured_claim_needs_a_live_hardware_reading`. `IoReport` and
`Estimated` with a measured claim are refused with their method and grade, and
build without it. `IoReportLive`, `Rapl`, `Nvml` and `NvmlRapl` build with the
claim. The message names the method, the grade and the rule. The mapping test
covers `IoReportLive` → `HwShunt`, and the display test covers its label.
jouleclaw-compliance: 53 pass.

Falsified (restored with `git checkout`, tree clean): a gate that refuses
only `Estimated`, and so accepts `IoReport` with a measured claim, fails the
test with "IOReport paired with a measured claim was not refused". 52 / 1.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

ea61169ca6

• View at this point in history

2026-09-30 20:18:41 -04:00

Transaction Science

jouleclaw-loader-gguf: a tuned shape records the meter that measured it, and better evidence re-decides it ...

`TuningProfile::record` keeps the joules-minimal cost per shape, so a later,
noisier measurement can never regress a decision. That rule has one hole. A
cost too low to be true, recorded once, wins forever. A cost taken before the
IOReport counter had to show its CPU rail live is exactly that: on a frozen
rail a CPU window metered as a sliver of GPU energy. A later honest
measurement draws more joules, so under the old rule it could never displace
the bad one. Zeros are already refused (c0813bcf). But a zero the shape
already held, which an older build recorded, could never be replaced either.

The fix.
- A record now carries the meter that took it, as `Metered { meter,
  energy_provenance, domain }`. Records are ranked by that grade: a hardware
  reading, then a vendor model, then an estimate, then a cost recorded with no
  meter, which ranks last as the worst-counter rule ranks an unknown.
  `record_metered` replaces a lower-ranked record however many joules either
  drew, because the decision is re-made on better evidence. Within one rank
  the old joules-minimal rule holds, and a lower-ranked record never displaces
  a higher one. `record` (no meter) keeps its signature and ranks last. A zero
  the shape holds is replaced by any figure. A zero is still never recorded.
- `tune_shape_live` records through `record_global_metered` with
  `Metered::of(&counter)`. The counter is only ever the live IOReport counter,
  so that is `apple-ioreport` / `HwShunt` / `SocTotal`.
- The sealed `autotune/1` receipt carried no provenance, meter or domain. No
  published vector pins its bytes: there is no autotune/1 vector or verifier
  anywhere in the repo, only the crate's own round-trip test. So each sealed
  shape now carries `meter`, `energy_provenance` and `domain`, and a cost
  recorded with no meter is sealed as `unrecorded` / `Unrecorded` /
  `unrecorded`, not left blank. `global_snapshot_metered()` is the new
  preimage. `global_snapshot()` and `entries()` keep their shapes.

Tests: `a_cost_recorded_without_its_meter_is_replaced_by_a_metered_one`
covers an unrecorded 5 µJ CPU entry replaced by a metered 500 µJ GPU reading,
never displaced back, joules-minimal within a rank, and ranks climbing
Estimator → ModelBased → HwShunt. `a_zero_the_shape_holds_is_replaced` covers
both meter states. `a_meter_is_recorded_as_its_name_grade_and_domain`
(fabric-meter, macOS) checks the IOReport counter and the static estimator.
The seal test now checks the three fields for a metered and an unrecorded
shape. On this host `tune_shape_live` still records nothing ("ioreport=
CpuRailFrozen tuned=None recorded=None"). loader-gguf lib: 297 / 304 / 308
pass (default / `fabric-meter` / `rayon fabric-meter attest`).

Falsified (restored with `git checkout`, tree clean): with `insert` set back
to the joules-only rule, both new tests fail ("a metered measurement did not
replace an unrecorded one"; the held zero is never replaced). 10 / 2 of the
module's tests.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

9acf777ded

• View at this point in history

2026-09-30 20:18:18 -04:00

Transaction Science

jouleclaw-fabric: a calibration row loads at the grade it declares, with its method ...

The crate's own honesty rule: energy figures carry provenance (`Measured`,
`Benchmarked`, `Estimated`), and nothing upgrades one into another.
`apply_calibration_file` broke it. It loaded every row of
`~/.jouleclaw/fabric-calibration.json` as `Measured`, whatever the row's source,
and dropped the row's `method`, the field where the writer records that the
figure is an idle-subtraction model on a machine-wide counter. It also loaded
any figure, so a 0 µJ/GFLOP row made a member free to place on.

The fix.
- `CalibrationEntry` gains optional `provenance`, `date`, `os` and `chip`
  (serde default and skipped when absent), so a file written before them still
  loads.
- A row loads at the grade its writer declared. A row that declares none, which
  is every row written before this commit, loads at `UNDECLARED_ROW_PROVENANCE`
  = `Benchmarked`. It is this machine's recorded run, so it still replaces the
  seeded figure, but nothing says a live meter read it. Rows like that were
  written before the IOReport counter had to show its CPU rail live, and on a
  frozen rail they priced GPU-only energy, or zero.
- The row itself rides along on the figure as `EnergyCost::calibration`, so the
  method, the declared grade, and when and on what the figure was taken stay
  with it. `EnergyCost` loses `Copy` for this. Nothing in the workspace copied
  one.
- A row whose `uj_per_gflop` is not positive and finite (0, -0, negative, NaN,
  ±inf) is refused. The member keeps its figure, and the row comes back from
  `apply_calibration` / `apply_calibration_path` / `apply_calibration_file` as a
  `RejectedCalibration` with the reason. `discover()` ignores the list, as it
  ignored the file's errors before.
- The writers declare their grade. `marginal_energy::calibration_row` (under
  `fabric-meter`) grades a row from the meter that read it: `HwShunt` is
  `Measured`, which since the previous commit means a live counter; a vendor
  model is `Benchmarked`; an estimate is `Estimated`. Nothing is upgraded. It
  stamps the date (RFC 3339 UTC), the OS (`sw_vers`) and the chip (`sysctl
  machdep.cpu.brand_string`). `fabric_calibrate` and `mpp_energy` write through
  it, and their `method` also records the IOReport status. On this host:
  date 2026-10-01T00:15:52Z, os "macOS 27.0.1 (26A434)", chip "Apple M5 Max".
  Both examples still refuse to write here, because the counter is declined.
  Neither was run, and `~/.jouleclaw/fabric-calibration.json` was not touched.

Tests (jouleclaw-fabric 16, none of which reads the real calibration file or
sets the env var other tests' `discover()` reads):
- `a_calibration_row_loads_at_its_declared_grade_with_its_method`:
  Measured / Benchmarked / Estimated rows each load at their own grade, with
  the row carried.
- `a_row_that_declares_no_grade_loads_but_never_as_measured`: a file in the old
  format, with no new fields, loads through `apply_calibration_path`, replaces
  the seed, and is `Benchmarked`.
- `a_non_positive_or_non_finite_figure_is_refused`: six bad values refused,
  each saying why, and the seeded figure kept; a zero from a file too.
- `a_calibration_row_round_trips`.
loader-gguf: `a_calibration_row_is_dated_in_rfc3339_utc` (epoch, 2000-02-29,
2026, 2099) and, under `fabric-meter`,
`only_a_hardware_reading_is_a_measured_calibration_row`. loader-gguf lib: 297
(default), 304 (`fabric-meter`), 308 (`rayon fabric-meter attest`), all pass.

Falsified, each mutant restored with `git checkout` and the tree confirmed clean:
- Every row loaded as `Measured` regardless (the old behaviour): the
  declared-grade test fails (Some(Measured) vs Some(Benchmarked)) and so does
  the no-grade test. 14 passed / 2 failed.
- The positivity check dropped, so 0 is accepted: "uj_per_gflop 0 was
  accepted". 15 / 1.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

f9a91c2c6d

• View at this point in history

2026-09-30 20:17:51 -04:00

Transaction Science

jouleclaw-energy: a live IOReport reading is a hardware reading, and only a live one ...

The owner's decision, applied: a vendor energy counter (RAPL, NVML, IOReport)
is a real reading, and what separates one such reading from another is
provenance, not accuracy. This crate gave RAPL and NVML `HwShunt` and gave the
IOReport counter `ModelBased`, on the reasoning that Apple's Energy Model is a
vendor model. That reasoning grades accuracy. What the counter could not
establish was provenance: on macOS 27.0.1 its CPU rails are frozen for an
unprivileged reader, and that is now caught by the liveness probe. A counter
that has passed it is as much a reading as RAPL is.

The grade. `AppleIoReportCounter` declares `HwShunt`, and stamps it on every
reading through one constant, `READING_GRADE`, shared by `provenance()` and
`read()`. It earns the grade by being live: the counter exists to read only
where `admits()` let it through (CPU rail `Live`), and a declined counter
produces no reading to carry any grade, so a frozen host still meters through
the load-average estimator at `Estimator`.

The rule, written once and shared (jouleclaw SPEC §1.3, §9.4; JCR-1 §2.3;
the crate docs; vendor-energy PROTOCOL/README): a vendor counter's reading MAY
be graded as a hardware reading only while the counter is demonstrably live for
the domain the figure claims; the implementation MUST have observed the counter
covering that domain advance under load (for IOReport: the CPU rail itself, in
at least 5 intervals, 4 of every 5, over at least 300 ms while the probing
thread burns CPU). A counter present but not live MUST NOT produce a
hardware-graded figure; the implementation falls back to its estimator or
reports the work unmetered, and SHOULD record why. A figure MUST NOT claim
coverage of a domain its counter does not read; where the format has a coverage
field, such a figure is Partial.

No partial reading from the GPU rail. Where the CPU rail is frozen but `GPU
Energy` still advances (the M5 Max), the counter stays declined. The GPU rail
is machine-wide: every GPU client on the host moves it, display compositing
included, with nothing of the metered work running. It does not cover part of
the work; it covers other work, so no coverage field makes it honest. The
rationale is in the module docs and SPEC §1.3.

Docs and data that said otherwise:
- SPEC §1.3 table and floors, §9.4; JCR-1 §2.3 and the crate README (toy
  vectors become measurements only off counters that were live).
- JCR-1 vector 6's description said "measured on Apple IOReport". Its joules
  are toy values like every vector's. Only the `description` changed; it sits
  outside the signed core, which `verify.py` builds from `receipt` alone. Both
  verifiers pass 10/10 after the edit: `verify.py` (cbor2 6.1.4, cryptography
  50.0.2, exit 0) and the independent `verify.go` (exit 0). `git diff` of the
  file is that one line; a field-by-field JSON diff finds only (6,
  description). The conformance README's slot 6 line follows.
- vendor-energy conformance: README row 27 names `AppleIoReportCounter` and the
  `HwShunt` grade, with the liveness condition a filled fixture must meet;
  `apple-silicon-ioreport.json` is an unsigned, empty stub (`signed_by` null,
  every workload not run, no hash of it anywhere), so its `impl` and
  `provenance` are corrected in place.
- MEASURED_ENERGY.md: a dated note that the 74.1 J / 25.5 W row was committed
  2026-06-12, predates macOS 27, and recorded neither chip nor OS. The "reads
  CPU/GPU/ANE/DRAM with no sudo" claim is replaced by what macOS 27.0.1 does:
  the counter opens without sudo, but the CPU rails are frozen (all 645 rails
  on the M3 Ultra), so those hosts are estimated.
- local-index / rerank tier SHOULDs, the CLI's `meter` help, meter.rs,
  apple.rs, energy_probe, Cargo feature comments, harness-econ docs.
- l3-jouleclaw README: it claimed `HwShunt` on RAPL/NVML hosts, but the
  binding builds jouleclaw-energy with `ioreport` only, so off Apple Silicon
  the receipt is the token estimate. It now says what happens on each host.
- mesh-schema's `HwShunt` definition includes vendor counters subject to
  liveness; mesh-meter, the mesh README and mesh-engine-gguf no longer say
  real silicon alone yields `HwShunt`.
- The Live-branch assertions in mesh-meter, l3-jouleclaw and jouledesk-github
  expected `ModelBased` for IOReport; they expect `HwShunt`. jouledesk-github's
  fixtures that paired `apple-ioreport` with `ModelBased` now pair it with
  `HwShunt`, and keep a `ModelBased` case under a neutral meter name.

Tests. `a_live_counters_readings_are_graded_as_hardware_readings` builds a
counter on `Live` directly. That opens the real all-channel subscription, which
opens here even though the gate declines this host. It reads it twice and
asserts each reading is `HwShunt` over `SocTotal`. On this M5 Max (macOS 27.0.1):
"an admitted IOReport counter: opens=true, 2 readings, each graded HwShunt",
so the read path is covered on a frozen host. With `--features "ioreport
rapl"`, `a_live_ioreport_reading_has_the_grade_of_a_rapl_reading` compares the
grade with `RaplCounter`'s own. The used/declined tests assert `HwShunt` in
their live branch. This host still takes the declined branch: "ioreport
declined (CpuRailFrozen): 154 ms of CPU work metered by `apple-silicon` as
6200000 µJ, provenance estimator".

Run on this M5 Max: jouleclaw-energy 24+1+1 (default), 76 lib + 1 + 1
(`ioreport`), 79 lib (`ioreport rapl`); jouleclaw-jcr1 50; harness-econ 14;
local-index 41; rerank 37; cli 10; mesh-meter (`hw-meter-apple`) 10;
mesh-engine-gguf 1; l3-jouleclaw 2 + 4 (2 ignored, need a model);
jouledesk-github 12. All pass.

Falsified, each mutant restored with `git checkout` and the tree confirmed clean:
- `READING_GRADE` set back to `ModelBased`: the new test fails at the
  declared grade (left: ModelBased, right: HwShunt), 75 passed / 1 failed.
- Only `read()`'s stamp set back to `ModelBased` (the declared grade left at
  `HwShunt`): the new test fails on the reading itself, "a live counter's
  reading is not graded as a hardware reading" (ModelBased, SocTotal) vs
  (HwShunt, SocTotal), 75 / 1. The read path is what the test pins.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

33f0b89124

• View at this point in history

2026-09-30 20:17:13 -04:00

Transaction Science

jouleclaw-energy: a rail is live only if it keeps advancing, never on a two-step refresh ...

The CPU-rail liveness rule said Live once one CPU rail had advanced in two
separate sample intervals. A macOS 27 batch refresh can meet that on its own.
On this M5 Max (macOS 27.0.1, 2026-09-30) a watcher sampling the Energy Model
group alone every ~16 ms saw one refresh land in two steps:

    CPU Energy  2643843378 -> 2705607863  (+61764485)
    CPU Energy  2705607863 -> 2705611298  (+3435)      64 ms later

with MCPU0, MCPU1 and PCPU each stepping twice alongside. The probe's
intervals are ~31 ms (a 25 ms CPU tick and a ~6 ms group sample), so a probe
window that holds such a refresh sees two advancing intervals and says Live,
and that answer is cached for the process. It was seen to happen: under the
per-thread-cache mutant, two real probes said Live while the oracle saw no
sustained movement, at the moment of a refresh.

The rule is now the family rule for the five crates that probe this rail
(this commit changes this crate only):

- Live iff one CPU rail (same key: name + ordinal) advanced in at least
  MIN_LIVE_ADVANCES = 5 distinct intervals, those are at least four in five
  of all the intervals observed so far (advancing * 5 >= intervals * 4), and
  the samples so far span at least MIN_LIVE_WINDOW = 300 ms of wall-clock
  time from the first. It is evaluated after every interval, and is Live as
  soon as all three hold.
- The probe keeps sampling until Live, or until the 1 s budget has elapsed
  and at least 5 intervals were observed; hard cap 5 budgets.
- The rule is handed each sample's wall-clock time: `observe(sample, at)`,
  where the loop passes its clock's reading as each sample comes back, and
  the tests script it.

To be Live, a refresh would have to step in eight of the probe's first ten
~31 ms intervals, which is what a live rail does and a refresh does not. A
live rail (the M5 Max GPU rail changed in 13,125 of 13,614 samples 100 ms
apart, 96%) is Live after ten unloaded intervals (310 ms), or five 100 ms
intervals on a loaded host (500 ms). Everything else is unchanged: per-rail
keys, consecutive samples, a decrease or a first appearance is no advance,
one advance per rail per interval, the group-only subscription, the
thread-CPU tick, the pure verdict-to-status mapping, the OnceLock cache,
REAL_RUNS and GROUP_OPENS, the F9 seam (which also records the time handed
with each sample now), new_given/admits, counts_once, the consumers, and the
"ioreport liveness: status=" line. The docs that described the two-interval
rule (module doc, RailWatch, IoReportStatus, ioreport_status, detect_meter,
the Apple estimator) describe this one.

The test oracle is held to the same bar. It said CpuRailMoved(true) at two
changes, so a refresh landing in its window could vouch for a false Live.
Now it says so only if one rail changed at five or more of its reads and at
four in five or more of them, over its whole window. It reads on past its
3 s window until it has five reads (capped at four windows), and a CPU rail
read fewer times still fails the calling test as no evidence. The freshness
test's "cannot tell fresh from stale" disclosure is now bounded: a run whose
probe said Live must show the oracle seeing a CPU rail change inside the
probe's window (Live means the rule was handed one advancing in five or more
intervals over 300 ms or more), so the disclosure can only cover runs that
were not Live, and a stale sampler can never make one Live.

Tests. Those marked (*) fail under the old two-advance rule.
- (*) the_observed_two_step_refresh_is_frozen: the refresh above, with its
  steps in consecutive intervals and 64 ms apart.
- (*) a_refresh_in_three_to_five_quick_steps_is_frozen: 3, 4 and 5 steps in
  the first 150 ms, one per interval.
- (*) an_m3_ultra_two_die_two_step_refresh_is_frozen: DIE_0 and DIE_1 each
  stepping twice within 64 ms. A modelled shape: the M3 Ultra has been seen
  to refresh in one step per die.
- (*) the_production_status_of_a_two_step_refresh_is_frozen: the same
  refresh through the real uncached host status function (F9 seam).
- a_rail_advancing_every_interval_is_live_not_before_five_advances_and_300_ms:
  Live at interval 10 unloaded (310 ms) and 5 loaded (500 ms); not at 9
  (279 ms) or at 3 (300 ms).
- a_rail_missing_one_interval_in_five_is_live (Live at exactly 8 of 10) and
  a_rail_missing_two_intervals_in_five_is_not_yet_live.
- the_window_runs_from_the_first_sample_on_the_clock_handed_in, and
  two_rails_advancing_in_turn_are_not_one_live_rail.
- The watch-loop tests at 31 and 100 ms intervals and past the budget: the
  probe stops at five intervals, keeps going with fewer, stops at the cap.
- the_production_status_of_a_rail_that_keeps_advancing_is_live replaces "two
  advances -> Live after exactly 3 sample calls": Live after 6 to 13 sample
  calls, each sample's time handed to the rule inside the tests' own clock
  bracket, at least 300 ms watched, and not Live one sample earlier.
- the_oracle_says_moved_only_for_a_rail_that_kept_changing, and
  the_oracle_reads_on_past_its_window_until_a_live_rail_could_meet_its_bar.
The lib tests go from 66 to 76.

Mutations, each a temporary edit restored with git checkout (the tree clean
after each), lib tests with --features ioreport:
- the old rule (two advances, nothing else): 13 fail, all four (*) among them
- MIN_LIVE_ADVANCES = 2: 6 fail
- the four-in-five check removed: 3 fail
- the 300 ms window check removed: 6 fail
- the oracle back to ">= 2 changes": 2 fail
- also caught: the window measured from the clock's origin (3 fail), the
  loop handing the rule a zero time (3), ">" for ">=" in the share (2).

Runs. jouleclaw-energy lib and integration tests with and without
--features ioreport (76 + 1 + 1, and 24 + 1 + 1). Three times beside an
all-channel IOReport sampler looping in another process, all green:

  ioreport liveness: status=CpuRailFrozen oracle=CpuRailMoved(false) selected=false available=false oracle_reads=7
  ioreport liveness: status=CpuRailFrozen oracle=CpuRailMoved(false) selected=false available=false oracle_reads=5
  ioreport liveness: status=CpuRailFrozen oracle=CpuRailMoved(false) selected=false available=false oracle_reads=5

The oracle extended itself to five reads in up to 5.5 s where its 3 s window
gave fewer. On the M3 Ultra (macOS 27.0.1), 76 passed:

  ioreport liveness: status=CpuRailFrozen oracle=CpuRailMoved(false) selected=false available=false oracle_reads=16

The scripted sustained rail went Live after 13 sample calls there and in
every M5 Max run (rule window 302-322 ms). The consumers pass:
jouledesk-github (12), jouleclaw-loader-gguf --features fabric-meter (367),
jouleclaw-harness-econ (14), mesh-meter --features hw-meter-apple (10),
l3-jouleclaw-binding (6). clippy --no-deps --features ioreport gives the
same six warnings as before, none new; rustdoc adds none.

Not done here:
- The freshness test's Live assertion never runs on either Mac here, since
  both are CpuRailFrozen; it guards a live host.
- The budget rule stops at five intervals once 1 s has passed, even if the
  window is still under 300 ms. That only happens if the first sample took
  more than ~700 ms, and it declines a live rail, which is the safe side.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

962d34a766

• View at this point in history

2026-09-30 18:57:55 -04:00

Transaction Science

jouleclaw-energy: one tested composition is the host status, and only Live opens a counter ...

Round 5 left two dangerous survivors. Tests only ever saw CpuRailFrozen, the
one status either Mac produces, so these two passed 57/57 on both Macs:
- OWN1: a counter that declined only a frozen rail.
- OWN4: a host cache that took "no subscription" for a live rail.
It also left four fail-safe survivors:
- a constructor that re-implemented the probe, which went uncounted;
- a zero or stale transform between the probe's record and the rule;
- a tick that always ran to its wall cap;
- a latent flake on a host whose rail is live.
In the sibling crates, a wall-time median pin also failed on correct code under
concurrent all-channel IOReport sampling. This commit closes all of these.

The production chain
- probe_host_status() is the one production status. It bumps REAL_RUNS, then
  returns status_of(probe_with(open_probe_sampler, probe_tick, <monotonic
  clock>, CPU_RAIL_LIVENESS_BUDGET)). It has no branch of its own.
- probe_with opens a sampler and watches it. If nothing opened, or no
  interval was observed, it returns None, which status_of maps to Unavailable.
- crate::ioreport_status() now holds the process-wide OnceLock. It is the only
  production caller of probe_host_status.
- AppleIoReportCounter::new() is new_given(ioreport_status()). new_given
  declines, before opening anything, unless admits(status) says yes. admits is
  a pure, exhaustive match that is true for Live alone.
- GROUP_OPENS counts every call of the real group opener. Only the probe opens
  the group. So a constructor that re-implements the probe around the cache
  shows up there, even though it bypasses REAL_RUNS. Both counters are read
  through #[doc(hidden)] ioreport_probe_runs() and ioreport_group_opens().

Every status is tested
- admits: Live admits; Unavailable, CpuRailAbsent and CpuRailFrozen do not.
- new_given with Unavailable, CpuRailAbsent or CpuRailFrozen:
  - opens no subscription of any kind and samples nothing;
  - is unavailable and names the status;
  - read() returns NoCounter.
- new_given(Live) opens the all-channel subscription once and probes nothing.
- OWN1 now fails: "a counter given Unavailable opened a subscription or
  sampled: Record { group_opens: 0, all_channel_opens: 1, ... }".
- The once-per-process pin now lives in its own integration test,
  tests/ioreport_probe_once_per_process.rs, so it runs in a process of its own.
  A barrier releases 9 threads:
  - a third ask for the status first;
  - a third build a counter first;
  - a third call detect_meter first.
  REAL_RUNS and GROUP_OPENS must both end at 1. Without the ioreport backend
  the test asserts that nothing probed.
- In the lib test process the uncached status now runs on purpose, so the
  unit test there only checks that every thread sees the one cached status.

Fault injection inside the production code
- A test-only, thread-local seam sits in four places:
  - inside the real group opener;
  - inside the all-channel opener;
  - inside the FFI leaf of the real sampler, sample_raw, now split from the
    pure energy_model_channels filter;
  - at the top of RailWatch::observe.
- A test can script whether the group opens and what each sample holds.
  Everything between the leaf and the rule is production code, including the
  real 25 ms thread-CPU tick and the real clock.
- The real, uncached probe_host_status runs under these scripts:
  - The group does not open: Unavailable, one open call, zero sample calls.
    OWN4 now fails: "the production status with no Energy Model subscription".
  - The M5 Max's own single refresh: CpuRailFrozen.
  - A rail advancing twice: Live, after exactly 3 sample calls.
  - A constant rail with one failed sample: CpuRailFrozen. The failed sample is
    kept from the rule, and the verdict comes only after the whole budget.
  - Two dies stepping in different intervals: CpuRailFrozen. The script also
    has same-named sub-rails and an out-of-group channel named "CPU Energy"
    that moves in every sample. Each die reaches the rule at its own value.
  - No CPU rail: CpuRailAbsent.
- Every scripted run also checks two things:
  - The rule was handed exactly the Energy Model channels of each sample that
    came back.
  - Each tick between two sample calls burned at least 25 ms and under 50 ms of
    this thread's CPU on the tests' clock, unless it hit the 500 ms wall cap.

What the rule is handed
- The record is now taken at the rule's input (observe), not beside the
  sampler. A zero or stale transform anywhere between the FFI leaf and the rule
  now shows up in it.
- The freshness test runs the real uncached composition while an oracle reads
  the group on its own subscription. It checks:
  - the rule's input equals what the sampler read;
  - its CPU and GPU rails are the oracle's;
  - each bracketed sample equals the oracle where the rail held, and lies
    between the oracle's reads where it moved;
  - a rail the oracle saw change inside the window also changes in what the
    rule was handed.
- On a host whose rail is live, the probe stops after two intervals (~60 ms),
  and one such window often holds too few oracle reads. So the test runs the
  composition up to 8 times, until one run has evidence and a moving rail.
  - No evidence in 8 runs fails.
  - Evidence but no moving rail passes, and the test prints that such runs
    cannot tell a fresh sampler from a stale one.

Cost and clocks, with no wall-clock bound
- The group-only pin is structural. Every channel of every sample the real
  probe took is in the Energy Model group. Each sample holds exactly as many
  channels as the oracle's own group subscription: 364 on the M5 Max, 645 on
  the M3 Ultra.
- The cost pin is the median THREAD CPU per sample call, under 50 ms on the
  tests' clock. The wall-time median is gone.
- No hardware test asserts a wall-clock bound or ratio. The one wall-clock
  check left is a lower bound in a scripted test: a Frozen verdict comes only
  after the whole budget.
- The tests read their own CLOCK_THREAD_CPUTIME_ID, the literal 16 from the
  SDK's <_time.h>. Against it, the production clock:
  - does not advance over a 200 ms sleep (10.5 µs);
  - does not advance while a helper thread burns 200 ms (57 µs);
  - advances by what the tests' clock saw over this thread's own burn
    (100.001 ms for 100.0008 ms).
- Each production tick burns at least 25 ms and under 50 ms of this thread's
  CPU (unless it hits its 500 ms cap), and no more CPU than wall time.
- The probe's own record of its ticks and sample costs is gone: ProbeRun,
  Tick, DELIBERATE_PROBE, NO_ENERGY_MODEL_GROUP and PROBE_RUNS. The tests now
  measure the ticks themselves, between sample calls.

counts_once
- Ported from eoc-meter 7aa42418. Beside the CPU aggregates, counts_once now
  also drops:
  - per-cluster accumulators (PACC*, EACC*, MACC*);
  - CPU power managers (*CPM*), with their _SRAM rails.
  It matches on the die-stripped name. The test pins the real channel names of
  both chips.

Evidence (this commit's tree; M5 Max, macOS 27.0.1, hw.ncpu=18)
- Tests with --features ioreport:
  - lib: "test result: ok. 66 passed; 0 failed" (57 before);
  - ioreport_probe_once_per_process: 1 passed, "real probe runs=1, Energy
    Model group opens=1";
  - vendor_energy_conformance: 1 passed.
- Tests without the feature: lib 24, integration 1, conformance 1.
- Every probe status line said "ioreport liveness: status=CpuRailFrozen
  oracle=CpuRailMoved(false) selected=false available=false".
- Load runs, on the lib test binary built from this tree (sha256 df3f59b0…):
  - 5 runs while a helper process sampled IOReport over all ~11,900 channels in
    a tight loop (~185 ms a sample): 5/5, each "66 passed". Median sample-call
    thread CPU 6.95-8.65 ms, median tick 25.071-25.079 ms.
  - 5 runs under 18 busy-loop spinners (1 x ncpu, load1 14.2 -> 27.3): 5/5.
    Median sample-call thread CPU 6.59-6.82 ms.
  - Every helper was killed; kill -0 and pgrep found 0 left.
  - A light group-only watcher of mine was also running during these runs.
- M3 Ultra (10.10.10.2, 27.0.1), the same binaries, run with --nocapture and
  then deleted:
  - lib "test result: ok. 66 passed; 0 failed", and the once test "real probe
    runs=1, Energy Model group opens=1".
  - 645 channels per sample, equal to the group; median 7.10 ms; DIE_0 and
    DIE_1 keyed apart.
  - GPU Energy changed 6 times inside the window and 7 times in what the rule
    was handed: fresh.
- Simulated live host (a temporary edit makes is_live true after two
  intervals; restored with git checkout), freshness and group-only tests only:
  - 10/10 unloaded, 10/10 under the all-channel helper.
  - Under the helper, the median thread CPU over the 3 sample calls was
    6.1-8.3 ms; round 5's wall-time median was 100-202 ms there.
  - Every ~57 ms window held 3-4 oracle reads.
  - Where no rail moved in any of the 8 windows, the test said it cannot tell
    fresh from stale: 4 of 10 unloaded, 6 of 10 under the helper. In this
    simulation the CPU rail is really frozen. On a genuinely live host the rail
    itself moves in nearly every interval.
- Mutations: 22, each an exact-string edit of this tree, run with
  cargo test --no-fail-fast (lib and integration), then restored with
  git checkout (clean after each). All 22 failed:
  - new_given declines only CpuRailFrozen: "a counter given Unavailable opened
    a subscription or sampled".
  - no subscription -> Live inside probe_host_status: "the production status
    with no Energy Model subscription".
  - a single advance -> Live inside the production closure: "one batch refresh
    counted as a live rail" and "two dies' separate steps ... made a live rail".
  - the rule compares with the first sample: 11 tests fail.
  - a decrease counts as an advance: "a fall counted as an advance".
  - a no-op tick: "production tick 0 burned 416ns ...".
  - a tick that always runs to its wall cap: "production tick 0 burned
    455.483167ms ... a tick stops once it has burned 25ms".
  - a per-thread cache: "the real probe ran 10 time(s) and asked for the Energy
    Model group 10 time(s)".
  - a constructor that probes uncached: "ran 19 time(s) and asked ... 19".
  - a constructor that re-implements the probe through probe_with: "ran 1
    time(s) and asked for the Energy Model group 19 time(s)".
  - a zero or stale transform in the production sampler closure: "the rule was
    not handed exactly the Energy Model channels of each sample".
  - a zero transform inside watch() before observe: 7 tests fail.
  - zeroing inside the rule after the record: 6 tests fail.
  - the sampler reads constant 0: "("CPU Energy", 0) held at 2643843378 across
    the oracle's reads either side of it, but the rule was handed 0".
  - the thread clock is CLOCK_MONOTONIC: "this thread slept 200ms, and the
    production thread clock advanced 210.015ms".
  - admits accepts CpuRailAbsent: "CpuRailAbsent admitted the counter".
  - counts_once re-admits PACC_*: "PACC_0, a summand of CPU Energy on an M5
    Max, is summed beside it".
  - extras:
    - CLOCK_PROCESS_CPUTIME_ID: "advanced 1.767591s".
    - the probe on the all-channel subscription: 9 lib tests and the
      integration test fail.
    - each CPU rail handed to the rule as the sum of all CPU rails: fails on
      this one-rail M5 Max through the two-die script.
    - a tick of 30%: "burned 7.50375ms".
- Consumers, all passing:
  - jouledesk-github: 10/10, 12 passed each.
  - jouleclaw-loader-gguf with fabric-meter: 299 passed, 2 ignored.
  - jouleclaw-harness-econ: 14.
  - mesh-meter with hw-meter-apple: 10.
  - l3-jouleclaw: 2 + 4, with 2 + 2 ignored.
  - energy_probe prints "ioreport   : declined (CpuRailFrozen)".
  - ~/.jouleclaw/fabric-calibration.json is unchanged (2026-06-11 11:02:14).
- cargo check --all-targets, with and without ioreport, on
  x86_64/aarch64-unknown-linux-gnu and x86_64/aarch64-apple-darwin: 0 warnings.
- clippy --no-deps --all-targets --features ioreport shows only warnings that
  were already there:
  - the duplicated link attribute, now apple_ioreport.rs:112;
  - the b"IOReportChannels\0" literal in sum_energy_uj, now :823;
  - ledger.rs:631-634.
- rustdoc --document-private-items shows only its three earlier warnings.

Found, not fixed here: one batch refresh is two advances, so the rule can say
Live on this frozen host
- During the first mutation pass, the per-thread-cache mutant ran about a dozen
  real probes in one lib test process. Two of them, on two test threads, said
  Live, and the oracle tests failed:
  - "probe said Live (probed=CpuRailFrozen, cached=Live), the oracle did not
    see the CPU rail keep moving";
  - "left: [CpuRailFrozen, ...] right: [Live, ...]".
- CPU Energy moved in one batch refresh, 2,588,973,125 -> 2,643,843,378,
  between 17:54 and 17:59:38. That mutant ran from 17:58:24 to 17:58:34.
- On this chip CPU Energy is the sum MCPU0 + MCPU1 + PCPU: 311,045,470 +
  313,039,186 + 2,019,758,721 = 2,643,843,377, against 2,643,843,378.
- A watcher sampled the Energy Model group every ~16 ms: 20 minutes with no
  refresh, then the next one caught one. The refresh landed as two steps of
  every cluster and of CPU Energy, 64 ms apart:
    t=1076.463s CPU Energy 2643843378 -> 2705607863 (+61764485) sample#56065
    t=1076.528s CPU Energy 2705607863 -> 2705611298 (+3435)     sample#56068
  (MCPU0 +9470641 then +788, MCPU1 +9747542 then +844, PCPU +42546301 then
  +1802.)
- The probe's intervals are ~31 ms: a 25 ms tick plus a ~6 ms sample. A probe
  whose window holds such a refresh sees the same rail advance in two distinct
  intervals, which is Live under the current rule.
- A probe lands on a refresh rarely: its window is 1 s, and the two refreshes
  seen here came about 30-34 minutes apart. When it does, production caches
  Live for the whole process. The counter would then sum batch-refreshed CPU rails, plus the GPU,
  tagged ModelBased: the over-claim this branch exists to stop.
- The same mutant on the final tree left the lib tests at 66/66, with no
  refresh during that run.
- The rule itself (the same rail advancing in two distinct intervals) is the
  cross-crate rule every sibling crate shares, so it is left unchanged here.
  One direction would demand what a live rail shows and a refresh cannot: a
  rail that advances in most intervals over a minimum count, for example at
  least 5 intervals with 80% of them advancing. The M5 Max GPU rail changed in
  166 of 176 samples ~28 ms apart. A refresh gives at most 2 or 3 advancing
  intervals in a row.

Not done here
- Changing the liveness rule, per the finding above. It needs one decision
  across jouleclaw-energy, eoc-meter, proof-cascade, sandbox-meter and
  jcp-energy.
- The counter's own read() still samples every channel. It is declined on both
  Macs.
- No CI job runs these ioreport tests: .github/workflows is off-limits to this
  branch.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

c445d15028

• View at this point in history

2026-09-30 18:30:41 -04:00

Transaction Science

jouleclaw-loader-gguf: each test temp file gets a name no other test in the process can pick ...

The tier_store tests sometimes failed with "tier_store.rs:738:50: called
`Result::unwrap()` on an `Err` value: Io(Os { code: 2, kind: NotFound, ... })".

write_synth_gguf named its file jouleclaw-tier-store-<pid>-<nanos>.gguf. On
macOS SystemTime ticks in microseconds, and the harness starts these tests
together on parallel threads, so two tests could pick the same name. They then
shared one file, and the first to finish deleted it before the other had read
it.

Each such helper now takes a number from its own process-wide AtomicU64 instead
of the time, giving <prefix>-<pid>-<seq>. The crate's mellum, falcon_h1 and
hybrid_lm tests already name their files this way.

Four more helpers used the same timestamp naming with no per-test tag, and
several tests call each of them at once, so they get the same fix:
- lib.rs prefetch_policy_tests::write_synth_gguf (8 tests)
- gpu_buffer.rs aligned_temp_mmap (5 tests; it truncates and maps the file)
- mmap_advise.rs tempfile_compat (2 tests)
- moe.rs build_synthetic_store_with_experts (2 tests)
None of the four failed in the runs below, but they had the same race.

Only #[cfg(test)] code changes.

Evidence, on this M5 Max:
- Before, on the parent tree (3b260e0a): the lib test binary built with
  --features fabric-meter, run 30 times, passed 27 and failed 3. All three
  failures were at tier_store.rs:738:50 with NotFound, in
  pin_hot_changes_tier_to_hot, demote_to_cold_returns_to_cold and
  out_of_bounds_errors.
- After, the same build and the same 30 runs: 30/30, 299 passed each time.
- `cargo test -p jouleclaw-loader-gguf --lib --features fabric-meter`, 12
  consecutive runs: 12/12, "test result: ok. 299 passed; 0 failed; 2 ignored".
- With --features "rayon fabric-meter": 3/3, 302 passed.
- clippy --no-deps --tests --features fabric-meter: no warning on or next to
  any changed line. It exits 101 on three error-level lints that were already
  there, in files this commit does not touch: ssm.rs:406 (approx_constant) and
  gemma4.rs:562 and :582 ("this operation will always return zero").

Co-Authored-By: Claude Opus 5.5 <[email protected]>

0c2a43bbd5

• View at this point in history

2026-09-30 15:41:54 -04:00

Transaction Science

jouleclaw-energy: the probe's own record pins what it burns and samples, and only the cache runs it ...

Round 4 left one medium finding and four mutations that no test caught. This
commit closes all five. The hardware tests run in CI on this M5 Max, which is
often heavily loaded, so every new pin reads the probe's own record or a count.
None of them is a wall-clock ratio.

The once-per-process pin counts the real probe
- PROBE_RUNS (test builds only) is now incremented inside probe(), the function
  that opens the Energy Model subscription and samples it. Round 4 counted in
  probe_status, the wrapper that the cache called. So a counter constructor that
  called probe() directly, uncached, passed all 55 tests (O6).
- probe_status is gone; host_status caches probe() itself.
- Tests that run the probe on purpose, uncached, call the same probe() inside
  deliberately(). That is a test-only scope which marks one call, on one thread,
  as not the host's run. It exists only in test builds, so no production path
  can use it and still build.
- the_liveness_status_is_probed_once_and_cached still releases 6 threads
  through a barrier. Each thread calls ioreport_status() twice, builds two
  counters and calls detect_meter. The real probe must have run exactly once in
  the process. A constructor that probes uncached now fails it with "the real
  liveness probe ran 26 times".

The tick is pinned from the probe's own record
- probe_cpu_rail records each tick: the probing thread's CPU time, on its own
  CLOCK_THREAD_CPUTIME_ID, and the wall time. Every tick must show a whole
  PROBE_TICK (25 ms) of thread CPU or have hit the 500 ms wall cap, and there
  must be one tick before every re-sample.
- Round 4 pinned only the CPU used per interval, from the end of one sample to
  the end of the next. That includes the sample call's own 6-8 ms, so a tick
  that burned 25% or 30% of PROBE_TICK passed under load (O1, O1b). Both now
  fail at any load, for example: "tick 0 of 66 burned 6.251291ms ... a tick
  burns 25ms of thread CPU time".
- The record is checked against the tests' own thread clock, which is Mach
  thread_info, not clock_gettime. The thread must have used at least as much
  CPU as the probe says its ticks and its counted intervals used. A probe whose
  "thread CPU" clock was really the wall clock passes this only when the thread
  was almost never off the CPU. Under load it fails: "its ticks used
  804.630251ms ... the thread used only 566.975ms".

The no-subscription path goes through the pure mapping, and a test runs it
- probe() is now probe_with(open, probe_on), given the real opener and the real
  probe. probe_with maps "nothing opened" through status_of_run to Unavailable,
  the same as a probe that saw no interval.
- A test-only switch, NO_ENERGY_MODEL_GROUP, makes open_energy_model return None
  on one thread. a_probe_that_cannot_subscribe_to_the_group_is_unavailable runs
  the real probe() that way, on a host whose group opens, and asserts
  Unavailable with nothing probed. It also calls probe_with with an opener that
  opens nothing, and with a subscription that never returns a sample.
- Round 4's O2 (probe() maps a failed subscription straight to CpuRailAbsent)
  passed on both Macs. It now fails with "the real probe with no Energy Model
  subscription ... left: CpuRailAbsent". The same bypass returning Live fails
  too, placed in probe() or in probe_with.

The sampler is pinned to read afresh
- New test: the_probe_samples_afresh_across_its_window. While the real probe
  runs, an oracle thread reads the Energy Model group every 10 ms and records
  when each read ran. It uses its own subscription, opened with the raw FFI, not
  open_energy_model. The probe records each sample call's start, end and values
  (test builds only).
  - For every probe sample with an oracle read wholly before it and one wholly
    after it: a GPU Energy rail that held must read the oracle's value, and one
    that moved must read between the two oracle reads.
  - If the oracle saw a GPU Energy rail change between two reads strictly inside
    the probe's window, the probe's own samples must show it changing too.
  - Fewer than two oracle reads inside the window is no evidence, and fails.
- the_probe_sampler_reads_each_cpu_rail_at_its_own_value now holds GPU Energy
  to the same bracketing rule as the CPU rails, over two rounds on one
  subscription.
- Round 4's O3 (the sampler returns its first sample's values forever) passed
  on both Macs. It now fails. So do a sampler that is stale per probe run and
  one that is stale per subscription. Example: "probe sample 1: ("GPU Energy",
  0) moved 123056146417043 -> 123056172426143 across the oracle's reads either
  side of it, but the probe's sample read 123056142245337, outside them: not a
  fresh sample".
- What this test can show depends on the GPU:
  - The CPU rails are frozen on both Macs, so they never can.
  - On this M5 Max, GPU Energy changes every ~28 ms. It moved inside the window
    in every run here.
  - An M3 Ultra's GPU moves only sporadically: in 113 of 11,716 100 ms samples
    over 25 minutes one day, and in about a fifth of 30 ms intervals today. Today
    the M3 Ultra caught all three stale samplers.
  - In a window where the GPU does not move, the test prints "no GPU rail moved
    inside the window: this run cannot tell a fresh sampler from a stale one",
    and a stale sampler would pass that run.

Docs: the budget's doc no longer says "each still under a full tick of burn",
and the all-channel sample cost is now quoted as 0.1-1 s.

Evidence, on this exact tree
- M5 Max, macOS 27.0.1, 10 runs under 36 background spinners (2 x hw.ncpu),
  1-minute load average 23-57: 10/10 "test result: ok. 57 passed". All spinners
  were killed afterwards and none was left. Every run showed:
  - "ioreport liveness: status=CpuRailFrozen oracle=CpuRailMoved(false)
    selected=false available=false", with oracle_reads=8-13.
  - "the real probe ran 1 time(s)".
  - Probe: 7-12 intervals, median sample cost 6.98-7.79 ms, least tick
    25.0005 ms of thread CPU, least interval 31.34 ms.
  - Freshness: the oracle saw GPU Energy change 43-52 times inside the window,
    and the probe's own samples changed in every interval (6-13 changes).
- M5 Max, 5 more runs under 72 spinners (4 x hw.ncpu), load up to 75: 5/5.
- M3 Ultra (10.10.10.2), the same binary: 57 passed. DIE_0 and DIE_1 read equal
  to the oracle (35656018 and 38872869). GPU Energy moved 9 times inside the
  probe's window, and the probe's samples 9 times.
- 31 mutations. Each was applied, built, run, and restored with git checkout,
  with git status clean after each one.
  - These fail at least one test: baseline never moves; same-named rails
    counted under one key; a decrease, or a first appearance, counted as an
    advance; no subscription mapped to Live (in status_of_run, in probe_with, in
    probe); no subscription mapped to CpuRailAbsent (in probe_with, in probe);
    per-thread cache; no cache; constructor probing uncached; constant-0
    sampler; three stale samplers; tick sleeps; tick is a no-op (the burn call,
    or the whole tick); tick burns 30% or 25%; tick burns on a helper thread;
    all-channel subscription; liveness decided on the sum of every Energy Model
    rail, or on the GPU rail; advances summed over rails; a declined counter
    reading 0; the fallback metering 0.
  - Two pass on a quiet M5 Max, and each is caught where it differs. The first
    gives each CPU rail the sum of all CPU rails. It is equivalent on the M5
    Max's single rail, and on the M3 Ultra it fails: "DIE_0_CPU Energy held at
    35656018 ... but the probe's sampler read 74528887". The second makes the
    wall clock the probe's thread clock. It agrees with the thread clock on a
    quiet host, and fails 3/3 under 36 spinners.
- 24 no-feature tests pass. cargo check --tests --examples passes for
  aarch64-apple-darwin, x86_64-apple-darwin, aarch64-unknown-linux-gnu and
  x86_64-unknown-linux-gnu, with and without ioreport. clippy --no-deps shows
  only the warnings that were already there.

Not done here
- A path that re-implements probe(), by calling probe_with or probe_cpu_rail
  directly, is not counted.
- No CI job runs these ioreport tests; .github/workflows is outside this
  branch.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

01170e38ca

• View at this point in history

2026-09-30 15:37:11 -04:00

Transaction Science

jouleclaw-energy: the liveness probe burns its own CPU time, and its tests hold on a loaded host ...

The hardware tests for the CPU-rail liveness probe run in CI on the macOS runner,
which is this M5 Max, and it is often heavily loaded: load averages of 24-104 have
been seen. Round 3 left tests that only held on a quiet host, a cache that was
pinned per thread rather than per process, and rail values nobody checked. This
commit fixes those.

The probe's tick
- The tick now burns 25 ms of the probing thread's own CPU time, measured on
  CLOCK_THREAD_CPUTIME_ID. It gives up after 500 ms of wall time.
- The old tick spun on the wall clock. On a loaded host the scheduler gave it
  only part of each 25 ms. With 36 extra spinners on this Mac, that old tick
  left intervals with only 8.6-11.8 ms of thread CPU.
- The tick is still passed into watch(), so the scripted tests keep their
  scripted ticks.

What the hardware tests assert now
- The probe burns CPU. In every interval it counts, the probing thread must use
  at least half a tick (12.5 ms) of CPU time. Only the 500 ms wall cap could
  break that, and the assertion message says so. The probe records its thread
  CPU at each sample for this check.
- The probe subscribes to the Energy Model group only. The median wall time of
  the probe's own sample calls must be under 50 ms. A group sample takes about
  6-12 ms and an all-channel sample 130-800 ms. The probe records each call's
  duration, and the test prints the median.
- The interval-count assertions are gone: ">= 10 intervals" and ">= 2 intervals
  plus the full budget". What stays is "a frozen or absent verdict took at
  least the 1 s budget". It is a lower bound, and load can only make the probe
  slower. No hardware test asserts a ratio of CPU time to wall time.
- New test: the_probe_sampler_reads_each_cpu_rail_at_its_own_value. It checks
  the values the sampler reads, against an oracle on the oracle's own
  all-channel subscription.
  - The oracle reads every Energy Model CPU rail by name, once before and once
    after one probe sample.
  - If the oracle saw a rail hold, the probe's value must equal it. If the rail
    moved, the probe's value must lie between the two reads.
  - Before this, a sampler that returned constant 0, or gave each CPU rail the
    sum of all CPU rails, passed every test.

The oracle
- It keeps reading past its 3 s window, up to 4 windows, until it has at least
  2 reads after its baseline.
- With fewer reads it fails the test, with the read count and the reason. It
  never returns CpuRailMoved(false) on 0 or 1 reads.
- The "ioreport liveness:" line now ends with oracle_reads=N. The test that
  prints it keeps its name.
- Under load the oracle needed the extension: one run got 2 reads in 3.76 s.

The pure rule: each case has its own test
- A rail that goes down between two samples (a reset or wrap) has not advanced.
- A rail missing from the previous sample only sets its baseline.
- No verdict maps to Unavailable.
- A probe that cannot subscribe to the Energy Model group, or whose subscription
  never returns a sample, maps to Unavailable through status_of_run(). It is
  never Live and never CpuRailAbsent.

That last rule changes behaviour. A host where the group cannot be subscribed
but the all-channel subscription opens used to report CpuRailAbsent. It now
reports Unavailable, because the probe saw nothing. IoReportStatus's
documentation says this. The enum and its variants are unchanged.

The once-per-process cache
- A process-wide AtomicUsize, HOST_PROBE_RUNS (built only for tests), counts the
  runs of the real host probe. The thread-local counter is removed.
- the_liveness_status_is_probed_once_and_cached now starts 6 threads through a
  barrier. Each calls ioreport_status() twice, builds two counters and calls
  detect_meter. The test asserts the host probe ran exactly once in the process.
- This corrects 073fb7d8, which said the cache was pinned "once per process".
  Its thread-local counter only pinned "once on this thread": a per-thread
  cache passed all 50 tests.

Evidence
- M5 Max (macOS 27.0.1), clean test binary, 10 runs under 36 background
  spinners (2 x hw.ncpu). Load average went from 34.8 to 142.8 during the runs.
  The spinners were killed afterwards, and none was left.
  - Result: 10/10, "test result: ok. 55 passed".
  - Every run: "ioreport liveness: status=CpuRailFrozen
    oracle=CpuRailMoved(false) selected=false available=false oracle_reads=3..7".
  - Group probe: 5-10 intervals, median sample cost 7.16-10.24 ms.
  - Least thread CPU in any interval: 31.72-32.39 ms.
  - Oracle: 2-8 reads after the baseline.
- M3 Ultra (27.0.1), the same binary: 55 passed.
  - Median sample cost 6.75 ms over 33 calls.
  - Rails DIE_0_CPU Energy and DIE_1_CPU Energy.
  - Sampler values equal to the oracle's: 28237793 and 30845042.
  - Oracle: 16 reads.

Mutations. Each was a temporary edit, restored with git checkout, and git
status was clean afterwards.
- A decrease counts as an advance: 2 tests fail, "a fall counted as an advance".
- A rail new in a sample counts as an advance: 1 fails, "a new rail counted as
  an advance".
- No group subscription maps to Live: 1 fails.
- Per-thread cache: 1 fails, "the host liveness probe ran 13 times in this
  process". With no cache at all, 41 times.
- The sampler returns constant 0: 1 fails, "held at 1863990848 ... but the
  probe's sampler read 0".
- Each CPU rail gets the sum of all CPU rails:
  - On the M3 Ultra it fails, "("DIE_0_CPU Energy", 0) held at 28237793 ... but
    the probe's sampler read 59082835".
  - On the M5 Max it is an equivalent mutant, because that group has one CPU
    rail. Summing every channel named *CPU* into it does fail there: "read
    5567863491".
- The tick sleeps: fails, "interval 0 of 29 burned only 6.086583ms". A no-op
  tick fails the same way: "interval 0 of 114 ... 8.550958ms".
- The probe on the all-channel subscription: fails, "median sample call took
  538.447416ms".
- The oracle concludes after 1 read: 2 tests fail with the read-count message.
- The mutations whose old assertions were removed are still caught:
  - watch() stopping after 1 interval fails 4 tests.
  - A 10x-fast probe clock fails, "decided CpuRailFrozen after 211.462541ms,
    before its 1s budget".
  - PROBE_TICK = 250 ms fails the scripted test (a).
- The round-3 wall-clock tick passes on a quiet host. Under the same load it
  fails 3/3 on the new CPU pin.

Other checks
- No features: 24 passed.
- Linux x86_64 and aarch64, with and without ioreport: cargo check passes.
- Clippy: no new warnings.
- Consumers:
  - jouledesk-github: 20/20.
  - jouledesk: 18 + 4.
  - loader-gguf with rayon and fabric-meter: 302 passed. Runs with only
    fabric-meter hit a tier_store temp-file race (NotFound) twice in 12 runs,
    in a file this branch does not touch; the 10 other runs passed.
  - harness-econ: 14.
  - mesh-meter with hw-meter-apple: 10. mesh-core: 3.
  - l3-jouleclaw: 2 + 4.

Not done here
- The counter's own read() still samples every channel. It is declined on both
  Macs today.
- No CI job runs these tests: .github/workflows is off-limits to this branch.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

7d7e766b4a

• View at this point in history

2026-09-30 08:57:25 -04:00

Transaction Science

jouledesk-github: no unit test reads the model setting, and an unmetered reply names no grant or ledger ...

226a70cb reported "jouledesk-github 12" passing on the M5 Max. That held on
some runs only. Two defects in that commit's own change:

1. A test race on a process-global. The two on-host handle() tests, GitHub
   and Slack, each ran set_var("JOULEDESK_MODEL", "/nonexistent.gguf"), then
   handle(), then remove_var, and cargo runs them in parallel. When one
   removed the variable before the other's run_model read it, run_model fell
   back to DEFAULT_MODEL. This Mac has that model file, and llama-completion is
   on its PATH, so a unit test ran gemma-4-E2B for 2-3 s. The work was then
   metered and sealed, and the test's "not sealed" assertion failed. The
   same binary at e4b20074, run 20 times here: pass=15 fail=5, e.g.
     left: ("completed", true, 70845000)
     right: ("not sealed (no energy figure)", false, 0)
   and on the Slack side
     left: ("completed", true)
     right: ("not sealed (no energy figure)", false)
   The race was on main too, but main's loose assertions passed either way.
   This machine is also the macos CI runner for jouledesk-rs. The M3 Ultra has
   no model file, so it never showed there.

2. The reply for work with no energy figure still said "_Decided under a
   signed grant, ..._" and "ledger UNVERIFIED ✗"; Slack said "_Signed grant
   · ..._". seal_work returns before any Teammate, grant or ledger exists, so
   no grant decided anything and there was no ledger to be unverified.

The fix.
- The work path takes the model explicitly. do_work_with(question, model)
  and handle_with(event, handle, model), plus slack::handle_with(event,
  model), are pub(crate). run_model and model_label take the path they are
  given. The public do_work, handle and slack::handle keep their signatures.
  They read JOULEDESK_MODEL, else DEFAULT_MODEL, once at the entry point and
  pass it down. The model that runs is now also the model the reply names;
  before, run_model and model_label each read the variable separately.
- Under cfg(test), configured_model() panics. Every unit test passes a model
  path, NO_MODEL = "/nonexistent.gguf", so none can read the process-global
  or reach the default model and spawn llama-completion. No test calls
  set_var or remove_var any more.
- A reply for work with no figure now says it was not decided under a grant
  and has no ledger entry:
    _Worked on real hardware; not metered, so no grant decided it and nothing
    was sealed._
    - **energy:** not metered — `apple-silicon`: the work took 121 µs,
      shorter than the 100 ms window it can resolve; no energy figure was
      sealed
    - **decision:** not sealed (no energy figure) · no ledger entry
  In Slack:
    _Worked on real hardware · not metered · no grant decision · nothing
    sealed_ … • *decision:* not sealed (no energy figure) · no ledger entry
  A sealed reply is unchanged. From a real --dry-run with the default model
  on the M5 Max:
    _Decided under a signed grant, worked on real hardware, sealed to a
    ledger anyone can verify._
    - **energy:** 99956.5 mJ (99956487 µJ) — estimated by `apple-silicon`,
      provenance `estimator`
    - **decision:** completed · ledger verified ✓

Tests. assert_unmetered_reply: no "signed grant", no "UNVERIFIED" or "ledger
verified", and "no ledger entry". It is applied to the composed GitHub and
Slack replies and to both on-host handle() paths. Those paths also assert the
reply names `deterministic-extract`. The sealed-reply tests now assert the
grant clause and "ledger verified ✓".

After, on the M5 Max:
- The same test binary 20 times with JOULEDESK_MODEL unset: pass=20 fail=0,
  every run 12 passed in 1.22-1.27 s.
- 5 more times with JOULEDESK_MODEL set to the real gemma model: pass=5
  fail=0, 1.22-1.24 s. No run spent seconds in a model.
M3 Ultra: 12 passed. jouledesk: 18 + 4 passed. clippy --no-deps --tests on
jouledesk and jouledesk-github: nothing new; the unused libwebrtc patch
warning is the workspace's.

Falsified, each mutation applied, run, and restored with git checkout, with a
clean tree after:
- handle_with ignores its model and calls configured_model():
  a_mention_answered_too_fast_to_meter_is_posted_as_unmetered fails with "a
  unit test reached the configured model (/Volumes/.../gemma-4-E2B-q4_k_m.gguf
  unless JOULEDESK_MODEL); pass a model path".
- The GitHub unmetered reply says "Decided under a signed grant" and "ledger
  {verify}" again: 2 tests fail ("no grant decided this work: ...").
- The same in Slack ("Signed grant · ..."): 2 tests fail.

Not changed: Work::measured_uj and Handled::measured_uj keep their names,
although they carry an estimate when provenance says `estimator`. Renaming
them is an API decision.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

c8890926ef

• View at this point in history

2026-09-30 08:01:34 -04:00

Transaction Science

jouleclaw-energy: each probe sample is compared with the one before it, rail by rail ...

The family contract: measured when the platform exposes it, estimated when it
doesn't, and the provenance always says which; a reading never silently
upgrades its own provenance. On macOS 27.0.1 the IOReport CPU rails are
refreshed in rare batches, so this crate's liveness probe declines the
counter unless one CPU rail steps in two separate intervals. The rule was
right. What an independent verification of 1a477bc8 found is that the parts
around it could be broken without any test noticing.

What was wrong:
- Only the pure RailWatch was tested. The loop that pairs samples, where the
  original defect lived, and the FFI mapping that feeds it were not. Two
  mutations passed all 40 tests on both Macs: the real probe loop breaking
  after its first interval (probe_took fell to 138 ms), and every CPU rail
  collapsed into one synthetic "any rail advanced" rail at the FFI mapping. On
  a multi-die host whose dies refresh in different intervals, the second
  reports Live, which is the over-claim the rule exists to prevent. The
  family's shared mutation, comparing every sample with the first one, would
  have passed as well, because nothing tested the pairing.
- RailWatch keyed rails by name. The M3 Ultra's Energy Model group has 645
  channels and 264 duplicated names. The CPU aggregates are unique there
  today, but two same-named rails stepping in one refresh would have counted
  as two advances of one rail.
- The test oracle ran its CPU burn on the calling thread, in 50 ms slices
  between blocking all-channel reads of 0.1-0.8 s each. The window was under
  load 13-18% of the time on the M5 Max and 27% on the M3 Ultra.

Two claims were false. Its doc said it "burns CPU for `budget`, reading as it
goes". b9d7e945's message said the oracle "burns for three times the probe
budget". Both are true now, of a helper thread. b9d7e945 also said "Counting
per rail keeps two dies' separate jumps from adding up". That held only for
rails with different names.

The fix follows the family test design.
- D1, the sample layer. RailWatch::observe takes a whole sample as (channel
  name, absolute raw integer) and keeps the CPU rails: `CPU Energy`,
  optionally `DIE_<n>_`-prefixed. It keys each rail by (name, ordinal among
  same-named CPU rails in that sample) and compares it with the PREVIOUS
  sample. A rail can advance at most once per interval, and only by going up.
  The FFI side, sample_energy_model, takes one plain IOReportCreateSamples
  (absolute values; the probe no longer uses delta samples) and extracts
  (name, raw) for every Energy Model channel. It decides nothing. A sample
  with no channel array is a failed sample, not a sample with no CPU rail.
- D2, the loop. It is generic: watch(sample, tick, now, budget). It takes a
  baseline, then ticks and samples again while RailWatch::wants_more, and
  skips failed samples; the next good sample is compared with the last good
  one. The real probe passes the FFI sampler, a 25 ms spin and Instant.
  Behaviour change: a failed first sample used to end the probe at once with
  no verdict. Failed samples are now retried up to the five-budget cap, so a
  host whose Energy Model group subscribes but never samples pays up to 5 s,
  once per process, before Unavailable.
- D3: status_of maps a verdict to an IoReportStatus.
- D4: the real probe counts its runs in a per-thread counter (PROBE_RUNS).
  Tests that probe on purpose, in parallel, do not disturb it.
- The oracle's burn runs on a helper thread. It starts before the oracle's
  subscription opens and stops after its last read. The calling thread only
  reads, back to back. The oracle still has its own all-channel subscription,
  absolute values, ends_with("CPU Energy") in any group, and its own ">= 2
  changes" threshold.

Tests (50 with --features ioreport, was 40; 24 without, unchanged):
- Scripted, no hardware, through watch() with a scripted sampler and clock:
  (a) the M5 Max refresh 480,083,160 -> 541,893,638 mJ, beside a GPU rail
  moving every sample, is Frozen; (b) the M3 Ultra's two dies stepping in one
  sample, beside duplicated ECPUDTL00 names, is Frozen; (c) two channels
  named `CPU Energy` stepping together once is Frozen; (d) a rail stepping
  every sample is Live after exactly 2 intervals, with sampling stopped at 3
  samples and 2 ticks; (e) samples that always fail run to the 5-budget cap
  with no verdict (Unavailable), and a failed baseline followed by good
  samples still gets a verdict; (f) at 600 ms a sample, the probe keeps going
  past the budget with one interval and stops at two, Frozen; at 3 s a sample
  it stops at the cap with one; (g) DIE_0 and DIE_1 stepping once each, in
  different intervals, is Frozen.
- Rule tests now feed samples. New: two_same_named_rails_stepping_once_are_
  not_a_live_rail, only_a_cpu_rail_going_up_is_an_advance, and
  each_verdict_maps_to_its_status (D3).
- D4, the_liveness_status_is_probed_once_and_cached: three
  ioreport_status() calls, new(), default() and detect_meter() ran the real
  probe at most once on the test's thread. It replaces a timing check.
- D5, the_liveness_probe_never_reports_a_frozen_rail_as_live: wherever the
  probe could sample and found no live rail, it ran its whole budget and
  compared at least 10 intervals. It prints the count.
- the_liveness_probe_subscribes_to_the_energy_model_group_only also asserts:
  - the rails the rule watched, as (name, ordinal), are exactly the group's
    CPU rails, with the test's own name matching;
  - unless the verdict is Live, at least 2 intervals and the whole budget.
- D6: a_declined_counter_yields_no_reading_and_the_fallback_says_estimator
  now also runs 150 ms of CPU work through detect_meter(). The figure must
  be above one quantum of the meter, with Estimator provenance on both
  readings.

Observed. M5 Max (macOS 27.0.1), three runs, 50 passed each:
  ioreport probe subscription: status=CpuRailFrozen after 30 intervals in
    1.096744083s; 364 channels per sample, 0 outside the Energy Model group
    (all-channel subscription: 11885); one sample 18.000625ms (all-channel
    449.545167ms); rails watched [("CPU Energy", 0)]
  ioreport oracle: 8 reads over a 3s window, CPU burned on a helper thread
    for 3.085035917s, changes per CPU rail [0] -> CpuRailMoved(false)
  ioreport probe: probed=CpuRailFrozen cached=CpuRailFrozen
    oracle=CpuRailMoved(false) intervals=Some(30) probe_took=1.087961791s
  ioreport declined (CpuRailFrozen): 150.009083ms of CPU work metered by
    `apple-silicon` as 6000000 µJ, provenance estimator then estimator
  ioreport liveness: status=CpuRailFrozen oracle=CpuRailMoved(false)
    selected=false available=false
Intervals were 28-30 across the three runs. The oracle got 3-8 reads, since
an all-channel sample took 0.45-1.07 s at the time.
M3 Ultra (same test binary, run by scp, then deleted), 50 passed:
  ... after 32 intervals in 1.100072167s; 645 channels per sample, 0
    outside ...; rails watched [("DIE_0_CPU Energy", 0), ("DIE_1_CPU
    Energy", 0)]
  ioreport oracle: 17 reads over a 3s window, CPU burned on a helper thread
    for 3.031725917s, changes per CPU rail [0, 0] -> CpuRailMoved(false)
  ioreport declined (CpuRailFrozen): 150.007459ms of CPU work metered by
    `apple-silicon` as 1452465 µJ, provenance estimator then estimator
In every run the cache test saw 0 probe runs on its own thread, because
another test's thread filled the cache. It pins "at most once"; the D4
mutation below shows 6.

Falsified on the final code, each mutation applied, run, and restored with
git checkout, with a clean tree after:
- Compare every sample with the first (prev.clone() + get_or_insert). 7
  fail, e.g. "one refresh counted as a live rail: RailWatch { intervals: 5,
  ... rails: [(("CPU Energy", 0), 4)] }".
- The round-2 survivor M7, the loop stopping after its first interval. 5
  fail, including both hardware probe tests: "the probe decided
  CpuRailFrozen on 1 intervals in ...: it did not re-sample the Energy Model
  group every ~25ms".
- The real probe's clock running 10x fast. 2 fail: "the probe decided
  CpuRailFrozen on 3 intervals in 225.471167ms".
- The real probe given a tenth of its budget. The same 2 fail.
- The round-2 survivor M8, the FFI collapsing the CPU rails into a synthetic
  "total". 3 fail on both Macs: "the probe's rule did not watch the group's
  CPU rails one by one" and "the probe found no CPU rail, the oracle found
  one".
- The same collapse into one rail named `CPU Energy`. It fails on the M3
  Ultra ("rails watched [("CPU Energy", 0)]" / "did not watch the group's
  CPU rails one by one"). It passes on the M5 Max, where the group has one
  CPU rail, so the collapse changes nothing there.
- The rule keying every CPU rail as one. 5 fail, (b), (c) and (g) among them.
- Keying by name alone (ordinal 0). (c) and the same-named rule test fail
  ("left: Some(Live) right: Some(Frozen)").
- The GPU rail counted as a CPU rail. 8 fail.
- A failed sample ending the probe. (e) fails: "gave up at 37ms after 2
  samples, before the 5s cap".
- Stopping at the budget with fewer than 2 intervals. 3 fail, (f) among them.
- Frozen mapped to Live. 4 fail, D3 among them.
- No per-process cache. D4 fails: "the real liveness probe ran 6 times: the
  status is not cached".
- PROBE_TICK at 250 ms. D5 fails: "the probe decided CpuRailFrozen on 4
  intervals", and (a).
- The probe on the all-channel subscription. The group test fails ("11521
  channels outside the Energy Model group") and so does D5 ("on 2 intervals
  in 1.569546042s").
- The estimator apportioning whole seconds only. D6 fails: "150.009209ms of
  CPU work metered as 0 µJ by `apple-silicon`: a zero or a floor, not a
  figure".
One mutation I wrote was wrong and proved nothing. It moved the real probe's
clock a budget ahead, but watch() takes its own origin from that clock, so a
constant offset cancels. It passed, and it was replaced by the 10x clock.

clippy --no-deps --tests, with and without ioreport: nothing new. The
collapsible-if my first draft added is fixed. What remains was there before:
the duplicated #[link], sum_energy_uj's b"..\0", ledger.rs's constant asserts
and meter.rs's vec-init-then-push. cargo check --tests --examples for
x86_64- and aarch64-unknown-linux-gnu, with and without ioreport: clean.

Dependants, re-run against this commit on the M5 Max:
- jouleclaw-loader-gguf --features "rayon fabric-meter": 302 passed, 2
  ignored ("tune_shape_live on this host: ioreport=CpuRailFrozen tuned=None
  recorded=None").
- jouleclaw-harness-econ: 14 passed.
- mesh-meter --features hw-meter-apple: 10 passed. mesh-core
  --features hw-meter-apple: 3 passed.
- l3-jouleclaw: 2 + 4 passed ("estimated by apple-silicon from host load").
- jouledesk + jouledesk-github: 18 + 4 + 12 passed.
- The energy_probe example prints "ioreport   : declined (CpuRailFrozen)"
  and "--- estimated ---".
~/.jouleclaw/fabric-calibration.json is untouched: Jun 11 11:02, sha256
792b00e6...b488.

Not done here:
- No CI job runs this crate with --features ioreport. The workflows are
  outside this branch; see the report.
- The counter's own read() still samples every channel. It is declined on
  both Macs.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

073fb7d81b

• View at this point in history

2026-09-30 08:01:34 -04:00

Transaction Science

l3-jouleclaw: the receipt says which meter ran and why, and an estimate is never "host-metered" ...

The family contract: measured when the platform exposes it, estimated when it
doesn't, and the provenance always says which. The L3 binding's `run` binary
meters the host across a model inference and puts the figure on a Joule Code
receipt.

The defect (b9d7e945, unchanged since origin/main 54e89b56):
- It printed every counter figure that was not HwShunt as "(<provenance>,
  host-metered)". Since b9d7e945 declines IOReport on both of our Macs, the
  counter it gets is the load-average estimator, so the printout read
  "(Estimator, host-metered)": an estimate presented as metered.
- The comment said the estimator runs "only if IOReport will not open". It now
  also runs when IOReport opens but its CPU rail does not advance, which is the
  case on both machines, and nothing said which.
- The delta was `a.saturating_sub(b)` with the after-reading's provenance: a
  counter going backwards gave a 0, and so did one that registered nothing.
  A before/after pair with different provenance was labelled with the second.

The fix.
- receipt_energy() takes the counter's delta at its provenance only if both
  reads succeeded, agree about provenance, and the delta is positive (no
  inference runs on zero energy). Otherwise the receipt carries the token-count
  estimate, which is what it already did without a counter. HwShunt carries
  measured_uj; every other provenance carries none, as the receipt schema
  requires.
- The printout adds "meter : <name> (IOReport: <ioreport_status()>)", and the
  energy line names how the figure was obtained: "measured by <meter>" (HwShunt),
  "host-metered by <meter> (energy model)" (ModelBased), "estimated by <meter>
  from host load" (Estimator), or "estimated from token counts".
- host_counter returns the meter as an EnergyMeter so the printout can name it,
  and its comment says when IOReport is used.

Observed, M5 Max, a real model (`JOULE_CODE_L3_ENGINE=llama`, gemma-4-E2B q4_k_m):
  meter        : apple-silicon (IOReport: CpuRailFrozen)
  tokens       : in=280 out=255
  energy       : 180.091 J  (Estimator, estimated by apple-silicon from host load)

Tests (the `run` bin, 4; the lib's 2 and the 4 model-gated ignored tests are
unchanged). CI's joule-code-conformance job runs `cargo test` here on its Linux
runner, where there is no counter:
- a_counter_reading_keeps_its_provenance, an_estimate_is_never_printed_as_metered.
- without_two_readings_the_receipt_is_the_token_estimate: missing reads, a
  counter going backwards, one that registered nothing (a held ModelBased rail
  included), readings that disagree about provenance.
- on_this_host_the_estimator_fallback_runs_unless_ioreport_is_live: IOReport
  only where Live; on Apple Silicon otherwise the estimator meters a 20 ms burn
  (non-zero, Estimator, "estimated by apple-silicon from host load"); with no
  counter (Linux) the receipt is exactly the token estimate. M5 Max and, from
  the same binary, M3 Ultra: "ioreport=CpuRailFrozen meter=apple-silicon
  energy=Energy { measured_uj: None, estimated_uj: 670069, provenance:
  Estimator }" (M3 Ultra: 198232).

Falsified, each mutation applied to this commit and restored with git checkout:
- FX1, an estimate printed "host-metered by <meter>": 2 tests fail ("left:
  host-metered by apple-silicon").
- FX2, a zero delta taken as the figure: fails on "Some((400, Estimator)) →
  Some((400, Estimator))".
- FX3, disagreeing provenance ignored: fails on "Some((100, ModelBased)) →
  Some((400, Estimator))".
- FX4, IOReport used although declined: "IOReport is used only when its CPU
  rail is live".

clippy --no-deps --tests --bins: nothing in src/bin/run.rs (the lib's one
warning predates this). `cargo check --tests --bins --target
x86_64-unknown-linux-gnu`: clean.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

e4b200748c

• View at this point in history

2026-09-30 01:58:06 -04:00

Transaction Science

mesh-meter: a declined or failed family meter is estimated, never a zero ...

The family contract: measured when the platform exposes it, estimated when it
doesn't, and the provenance always says which. mesh-meter's HwMeter bridges
jouleclaw-energy's detect_meter() into Mesh's Meter/Session, and seals each
span's Energy into the mesh/1 receipt.

What was asked: confirm that on a host where b9d7e945 declines the IOReport
counter, the bridge takes the no-hardware path and not a zero ModelBased
reading. It does, as long as the composite reads. detect_meter() puts the Apple
load estimator behind the declined counter, and every span says Estimator (on
both Macs: "meter=apple-silicon energy=Energy { measured_uj: None,
estimated_uj: 2000000, provenance: Estimator }").

What the check found. The bridge turned any failed read into `Sample { uj: 0,
provenance: Estimator }`, and a Session's delta is `end - start` at the end
sample's provenance:
- An unavailable counter with nothing behind it (CompositeMeter over the
  IOReport counter alone, declined) reads Err every time. Every span came out
  `estimated_uj: 0`: a zero, billed as an estimate.
- A span whose opening read failed and whose closing read succeeded was billed
  the counter's whole cumulative value at the counter's provenance (with a
  RAPL-sized history, 1,000,000,001,000 µJ tagged ModelBased in the test).
- A span whose closing read failed was billed 0.

The fix. When a read of the backend fails, the bridge takes this crate's
no-hardware path from then on: EstimatorMeter, tagged Estimator, continuing
from the backend's last good reading, so a span across the failure is billed
an estimate of its own length. It never bills zero, never the counter's
history, and never runs backwards. The fallback is sticky: once a read has
failed, the backend is not trusted again in this meter. Mixing its readings
back in would bill whichever span straddled the switch the difference between
two unrelated counters. `active_name()`/`name()` report "estimator" once it
has fallen back. The module and HwMeter docs now say the bridge can resolve to
an estimator (on Apple Silicon whose CPU rail does not advance, it does).

mesh-core, the only consumer (its demo): `--features hw-meter-apple` (and
-rapl, which the README documents, and -nvml) built the backend but did not
turn on mesh-core's own `hw-meter` feature, which is what the demo checks, so
the demo ran Mesh's 15 W EstimatorMeter and never the family meter. Each
platform feature now implies hw-meter, and a compile_error! stops a build
where one is on without it. `cargo run -p mesh-core --example demo --features
hw-meter-apple` now prints "meter: apple-silicon" and seals
"energy_provenance": "Estimator" (it printed "meter: estimator" before).

Tests (mesh-meter: 2 without features, 8 with hw-meter, 10 with hw-meter-apple,
M5 Max; the hw-meter-apple binary also on the M3 Ultra, 10 passed; mesh-core
3/3 with hw-meter-apple):
- an_unavailable_counter_takes_the_no_hardware_path_never_a_modelbased_zero: a
  backend that is not available and whose read would claim a ModelBased zero.
- the_real_ioreport_counter_when_declined_takes_the_no_hardware_path
  (hw-meter-apple, macOS): the real AppleIoReportCounter bridged alone;
  available exactly when ioreport_status() is Live, and otherwise an
  Estimator span, no measured figure, non-zero. Both machines: "ioreport=
  CpuRailFrozen available=false meter=estimator energy=Energy { measured_uj:
  None, estimated_uj: 305126 .. }" (M3 Ultra: 450329).
- on_this_host_the_bridge_is_as_honest_as_the_family_meter (hw-meter-apple):
  detect() on this host; IOReport only when Live, else Estimator.
- a_read_that_fails_mid_span_is_estimated_never_zero,
  a_span_that_opens_on_a_failed_read_is_not_billed_the_counters_history,
  a_backend_that_never_reads_is_estimated,
  once_a_read_fails_the_bridge_estimates_and_never_runs_backwards,
  a_backend_that_reads_passes_through_unchanged: a scripted backend.
These run only with hw-meter; CI's mesh-rs job builds without features.

Falsified, each mutation applied to this commit and restored with git checkout:
- FM1, main's snapshot (a failed read is `uj: 0`, Estimator): 6 tests fail,
  among them the real declined counter ("meter=none .. estimated_uj: 0") and
  the history case ("estimated_uj: 1000000001000, provenance: ModelBased").
- FM2, the backend trusted again after a failure: 2 fail, e.g. "the backend is
  not trusted again" (left: ModelBased).
- FM3, hw-meter-apple no longer implying hw-meter: mesh-core does not compile
  ("a hw-meter-* feature must enable mesh-core's `hw-meter` feature").

clippy --no-deps --tests --examples on mesh-meter and mesh-core, with no
features, hw-meter, hw-meter-apple and hw-meter-rapl: clean.

Not done here, deliberately:
- The mesh-rs CI job runs `cargo test --workspace` with no features, so none
  of the hw tests above run in CI. Adding a feature leg means editing
  .github/workflows.
- mesh-core's module doc still says Session makes the figure "measured, not
  self-reported"; with the default EstimatorMeter it is an estimate, as the
  receipt's provenance says. That predates this branch.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

66efc6c569

• View at this point in history

2026-09-30 01:55:19 -04:00

Transaction Science

jouleclaw-harness-econ: the local-model examples say which meter ran and never zero-fill a read ...

The family contract: measured when the platform exposes it, estimated when it
doesn't, and the provenance always says which. The three examples that meter a
real local model's decode with jouleclaw-energy's detect_meter() broke it in
their output, and turned a failed read into a figure.

The defect (b9d7e945, unchanged since origin/main 54e89b56):
- local_model_econ, swebench_local_coder and swebench_local_suite read the meter
  with `.unwrap_or(0)` before the decode and `.unwrap_or(e0)` after. A failed
  first read billed the dispatch the meter's whole cumulative value; a failed
  second read billed it zero.
- local_model_econ printed "{J} J measured" per dispatch and "(Estimator,
  energy-only $)" for the average; swebench_local_coder printed "{J} J
  measured"; swebench_local_suite said "Energy is real inference measured on
  this machine". Those labels were fixed strings, not the meter's provenance.
  Since b9d7e945 the meter on both of our Macs is the load-average estimator,
  so every "measured" is an estimate called a measurement. Where IOReport's
  CPU rail is live, the hard-coded "Estimator" would understate a reading.
- The docs: Cargo.toml said the `ioreport` dev-feature "selects the
  Apple-Silicon load-derived energy estimator" and that the IOReport meter
  "lives in a separate repo's sandbox-meter"; the examples said energy "is the
  honest Provenance::Estimator figure". It is IOReport where its CPU rail is
  live, else the estimator.

The fix.
- Two small, documented functions in the library, so the rule is tested where
  CI runs (tests inside examples/ never run under `cargo test --workspace`):
  window_joules(before, after) gives the delta and the readings' provenance,
  or why there is no figure: a failed read, readings that disagree about
  provenance, a counter that went backwards, or a zero (no decode runs on zero
  energy). joules_obtained(provenance) is "measured" for a reading and
  "estimated" for an estimate. The crate already depended on jouleclaw-energy
  for Provenance; no dependency is added.
- The three examples meter every dispatch through window_joules. They stop
  with the reason if the meter put no figure on a dispatch, since the report
  exists to print that figure. They print each dispatch's joules with
  joules_obtained, the meter that ran and its provenance. local_model_econ's
  average states the dispatches' own provenance.
- The false doc strings are corrected: Cargo.toml, the three examples' module
  docs, and swebench_local_suite's closing line.

Observed, M5 Max, `cargo run --release -p jouleclaw-harness-econ --example
local_model_econ` with its real Nemotron-3-Nano-4B snapshot:
  dispatch 1: 6 tok in 0.84s (7.1 tok/s), 33.720 J estimated by apple-silicon (estimator)
  dispatch 2: 6 tok in 0.98s (6.1 tok/s), 39.320 J estimated by apple-silicon (estimator)
  dispatch 3: 6 tok in 0.85s (7.1 tok/s), 33.840 J estimated by apple-silicon (estimator)
  avg real-model dispatch  : 35.627 J  (Estimator, energy-only $)
The report's own energy_provenance ("(Estimator)") is derived from the tier,
not the meter, and is unchanged.

Tests: 14 lib tests pass (M5 Max), 3 new, none feature- or host-gated:
a_window_is_the_delta_at_its_readings_provenance,
a_window_without_two_good_readings_is_never_zero (a failed read on either side,
backwards, disagreeing provenance, a zero), an_estimate_is_never_reported_as_
measured. The WIP version of this change had put the label test inside
examples/local_model_econ.rs, where `cargo test --workspace` compiles it but
never runs it.

Falsified, each mutation applied to this commit and restored with git checkout:
- FH1, a failed first read taken as a 0 µJ reading (the old unwrap_or(0)):
  a_window_without_two_good_readings_is_never_zero fails.
- FH2, a zero window accepted as a figure: the same test fails.
- FH3, an estimate called "measured": an_estimate_is_never_reported_as_measured
  fails.

clippy --no-deps --tests --examples: no warning in the lines this commit
touches; swebench_local_suite's unread `text` field and empty-format-literal
warnings are pre-existing. `cargo check --tests --examples` for
aarch64-unknown-linux-gnu (+fp16, CI's target): clean.

Not done here, deliberately:
- EconReport::energy_provenance comes from the tier (skill/cache hits are
  HwShunt, model tiers Estimator), not from any meter. A skill hit's joules are
  a constant (SKILL_JOULES), so a report whose rows are all skill hits says
  HwShunt for a figure nothing measured. That is not an IOReport consumer; it
  needs its own change.
- meter_probe (example) prints the meter and provenance it gets and is not
  wrong; it does not print ioreport_status().

Co-Authored-By: Claude Opus 5.5 <[email protected]>

060a858620

• View at this point in history

2026-09-30 01:52:51 -04:00

Transaction Science

jouleclaw-loader-gguf: a meter that did not measure records nothing and writes nothing ...

The family contract: measured when the platform exposes it, estimated when it
doesn't, and the provenance always says which. This crate's energy paths
price work as marginal draw, `ΔE_busy − P_idle·Δt`, on the IOReport counter,
and three of them turned a counter that measured nothing into a figure that is
then treated as measured.

The defect (b9d7e945, unchanged since origin/main 54e89b56):
- fabric_dispatch::tune_shape_live read the counter with `.unwrap_or(0)` /
  `.unwrap_or(e0)`, clamped the marginal with `.max(0.0)` and truncated the
  per-op figure to u64. On a CPU rail that does not advance (macOS 27.0.1, both
  our Macs) every backend came out at 0 µJ per op, joules_minimal picked one,
  and TuningProfile::record kept it: a recorded 0 beats every later
  measurement, so the shape was pinned to that backend for the process.
- examples/fabric_calibrate wrote ~/.jouleclaw/fabric-calibration.json, which
  jouleclaw-fabric loads as Measured placement costs, with idle
  `(e1 - e0) / t` (0 on a frozen rail) and each member's marginal clamped to
  `.max(0.0)`: zeros written as measurements.
- examples/mpp_energy overlaid its matmul2d figure into the same file with an
  unclamped marginal (negative on a frozen rail), printed a CPU/GPU ratio of
  0/0 = NaN, and ignored whether its CPU matmul ran at all.
b9d7e945 now declines the counter on those hosts, so today these paths stop
at `is_available()`. That covers today's hosts. A counter that is available but
does not see a window would still have written zeros.

The fix.
- New module marginal_energy, pure arithmetic over raw readings, which the
  three paths share: idle_draw_uw refuses an idle window that did not advance
  (an SoC draws power doing nothing, so a counter that holds is not metering),
  marginal_uj refuses a busy window at or below idle and is never clamped to
  zero, per_op_uj refuses a per-op cost that would round to 0, energy_ratio is
  None unless both figures are positive and finite, and refusal() renders what
  an example prints before it exits non-zero.
- tune_shape_live: a failed read, an idle baseline that did not advance, or
  either backend's window drawing nothing above idle voids the tune (None,
  nothing recorded), through the new tune_measured: an unmeasured backend
  neither wins nor loses against a measured one.
- TuningProfile::record (and so record_global and tune_and_record) never
  records a zero-joule cost; tune_and_record returns None for one.
- fabric_calibrate and mpp_energy refuse to write, exit 1, and print the file,
  the reason and jouleclaw_energy::ioreport_status() when the counter is
  unavailable, the idle baseline did not advance, or any marginal is <= 0. They
  refuse before any write, so no partial file. mpp_energy's ratio is n/a unless
  both runs were measured, and its CPU run now asserts the matmul ran, as its
  GPU run already did (a refused shape would meter an empty loop, and that run
  is one of the refusal gates).
- energy_autotune (example) says why a shape was not tuned, with the status.

Observed, M5 Max and M3 Ultra (macOS 27.0.1), examples run with HOME set to a
fresh directory and JOULECLAW_FABRIC_CALIBRATION unset:
  refusing to write <tmp>/.jouleclaw/fabric-calibration.json: the energy
  counter is not available on this host (apple-ioreport declined)
  IOReport status: CpuRailFrozen; nothing was written (its figures are loaded
  as measurements)
exit 1 for fabric_calibrate and mpp_energy on both, no file under the temp
HOME. The real ~/.jouleclaw/fabric-calibration.json kept its 2026-06-11
11:02:14 mtime and SHA-256 (792b00e6...) on the M5 Max, and its mtime on the
M3 Ultra. energy_autotune: "not tuned: the meter produced no figure (IOReport
CpuRailFrozen)", global profile empty.

Tests. Default features, what CI's jouleclaw-rs job runs: 294 lib tests pass
(the marginal_energy and energy_autotune tests below are not feature-gated).
With `--features "rayon fabric-meter"`: 302 pass on the M5 Max and, from the
same binary, on the M3 Ultra.
- a_counter_that_holds_at_idle_has_no_idle_baseline, a_busy_window_at_or_below_
  idle_is_refused_never_clamped_to_zero, a_per_op_cost_never_rounds_to_zero,
  the_cpu_gpu_ratio_is_undefined_unless_both_are_measured,
  a_refusal_names_the_file_the_reason_and_the_ioreport_status, a_refusal_says_why.
- a_zero_joule_cost_is_never_recorded (incl. through record_global),
  tune_and_record_records_nothing_when_the_winner_drew_zero_joules,
  an_unmeasured_candidate_voids_the_tune.
- a_live_tune_records_only_what_the_meter_measured (fabric-meter, on this
  host): calls tune_shape_live itself and asserts it records exactly what it
  returns, never a zero, and None where ioreport_status() is not Live. Both
  machines: "ioreport=CpuRailFrozen tuned=None recorded=None".

Falsified, each mutation applied to this commit and restored with git checkout:
- FL1, record accepts a zero: "the meter did not see the work: nothing to record".
- FL2, a held idle counter as a 0 W baseline: "left: Ok(0.0) right:
  Err(IdleDidNotAdvance { secs: 3.0 })".
- FL3, the marginal clamped to zero again: "left: Ok(0.0) right:
  Err(NoMarginal { marginal_uj: 0.0 })".
- FL4, the raw ratio: "left: Some(NaN) right: None".
- FL5, an unmeasured backend taken as 0 µJ: "left: Some(Cost { backend: Cpu,
  uj: 0, ns: 0 }) right: None".
- FL6, tune_shape_live returning and recording a zero cost when the counter is
  declined: the host test fails, "a live tune records exactly what it
  returns", tuned=Some(Cost { uj: 0 .. }) recorded=None (the record gate
  dropped it; the test caught the returned zero).
- FL7, the refusal without the IOReport status, and FL8, a per-op cost
  rounding to 0 accepted: their tests fail.

clippy --no-deps --tests on the crate (with and without fabric-meter, plus the
three energy examples): no warning in the lines this commit adds. The crate
does not pass clippy on its own: two deny-level lints elsewhere
(clippy::erasing_op at gemma4.rs:562 and :582, clippy::approx_constant in a lib
test) stop it before the tests and examples, so they were checked with those
two allowed. The pre-existing warnings in the touched files (mpp_energy's
double_parens, fabric_calibrate's chunks_exact_mut, fabric_dispatch's
too_many_arguments) are unchanged, and mpp_energy's unused-Result warning is
gone. `cargo check --tests --examples` for aarch64-unknown-linux-gnu (CI's
jouleclaw-rs target, with +fp16): clean.

Not done here, deliberately:
- jouleclaw-fabric loads every calibration row as Measured, and the file
  carries no date, OS or provenance. A file written on a live counter is still
  loaded at that grade on a host where the counter is now declined.
- TuningProfile and the autotune/1 receipt carry no provenance field, so a
  sealed tuning profile cannot say how its joules were obtained. Both are wire
  or format changes.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

c0813bcfb7

• View at this point in history

2026-09-30 01:50:18 -04:00

Transaction Science

jouledesk-github: an estimate is sealed and posted as an estimate, and no zero is ...

The family contract: measured when the platform exposes it, estimated when it
doesn't, and the provenance always says which; a reading never silently
upgrades its own provenance. jouledesk-github broke it in the ledger and in
every reply it posts.

The defect, on origin/main (54e89b56) and still after b9d7e945:
- do_work read jouleclaw-energy's detect_meter() around the work and floored
  the delta to the meter's resolution, `(after - before).max(1000)`, falling
  back to that 1000 µJ when a read failed. A counter that read nothing, which is
  what the IOReport CPU rail does on macOS 27.0.1, came out as 1000 µJ of work.
- seal() committed that figure through Teammate::tag, which seals every task's
  efficiency/1 as Energy::measured: HwShunt, a hardware reading, whatever
  metered it.
- The GitHub and Slack replies said "measured via `<meter>`" regardless.
Since b9d7e945 declines IOReport on both of our Macs, the meter that runs is
the load-average estimator (apple-silicon), and the three together sealed an
estimate as a hardware reading and posted "measured via `apple-silicon`,
provenance `estimator`".

The fix.
- jouledesk: Teammate::tag_with_provenance(kind, intent, provenance) seals the
  task's energy at how it was obtained, using the schema's own constructors
  (Energy::measured / modeled / estimator). Teammate::tag is unchanged: it
  still seals HwShunt, and the signed conformance vector (examples/vector.rs,
  tests/pack.rs) is built through it. EnergyProvenance re-exports
  jcp_schema::Provenance so a binding need not depend on jcp-schema.
- jouledesk-github seals the meter's figure at the meter's provenance, and the
  replies say "estimated by `<meter>`" for an estimate and "measured via" only
  for a reading, naming the meter and its provenance either way.
- The floor is gone. The figure is the delta as read. And no zero becomes a
  figure: the delta counts only if both reads succeeded, agree about
  provenance, the counter did not go backwards, the window was at least the
  active meter's own min_window_ns (the estimator declares 100 ms, IOReport
  10 ms), and it is non-zero, since no work runs on zero energy. Otherwise the
  work is not sealed and the reply says "not metered — `<meter>`: <why>; no
  energy figure was sealed". The same gate refuses a zero in
  jouleclaw-loader-gguf's tuner (next commit).

Why the window gate: with the floor removed, the no-model fallback (a
deterministic extract that takes microseconds) sealed "0.0 mJ (0 µJ) —
estimated by `apple-silicon`" on the M5 Max. The estimator integrates at 1 ms
steps, so such a window reads 0, or one whole millisecond of its watts when it
straddles a tick. Neither is a figure for the work, and the estimator says so
itself through its 100 ms minimum window.

What a reply says now. A real mention with the default model (gemma-4-E2B,
--dry-run, M5 Max):
  - **energy:** 134774.9 mJ (134774859 µJ) — estimated by `apple-silicon`,
    provenance `estimator`
  - **decision:** completed · ledger verified ✓
The no-model fallback (M5 Max; the M3 Ultra says "7 µs"):
  - **energy:** not metered — `apple-silicon`: the work took 71 µs, shorter
    than the 100 ms window it can resolve; no energy figure was sealed
It used to seal 1000 µJ as HwShunt and post "measured via".

Tests (jouledesk 18 + pack 4, jouledesk-github 12; all pass on the M5 Max; the
jouledesk-github binary also on the M3 Ultra, 12 passed):
- a_tags_energy_is_sealed_at_the_provenance_it_was_obtained_at (jouledesk):
  each provenance, read back through the verified efficiency/1 chain; plain
  tag still seals HwShunt.
- a_figure_is_the_delta_as_read_never_floored: 250 µJ stays 250.
- a_window_without_two_good_readings_has_no_figure: a failed read, a counter
  going backwards, readings that disagree about provenance.
- a_window_the_meter_cannot_resolve_is_never_a_zero: a 200 µs window, with or
  without a step in it, and a zero delta over a second.
- an_estimate_is_sealed_and_posted_as_an_estimate,
  a_reading_is_sealed_at_its_own_provenance,
  work_with_no_figure_is_neither_sealed_nor_given_one, and Slack's
  slack_posts_an_estimate_as_an_estimate: what is sealed, read back from the
  verified chain, and what each reply says.
- On this host, not skippable: a_mention_answered_too_fast_to_meter_is_posted_
  as_unmetered and the Slack equivalent run the whole handle() path and assert
  the reply names IOReport exactly when ioreport_status() is Live;
  on_this_host_work_the_meter_can_resolve_is_sealed_at_its_provenance burns CPU
  for the real meter's min window + 50 ms and asserts the figure is non-zero and
  sealed at that meter's provenance. Both machines:
  "ioreport=CpuRailFrozen sealed Estimator 6000000 µJ ... estimated by
  `apple-silicon`" (M3 Ultra: 1492895 µJ).
The old handles_a_mention_and_seals_measured_energy asserted
`measured_uj >= 1000` and "measured via": it held only because of the floor.

Falsified, each mutation applied to this commit and restored with git checkout:
- FJ1, the floor back (`uj.max(1_000)`): a_figure_is_the_delta_as_read_never_
  floored fails.
- FJ2, no min-window gate: three tests fail; the fallback's window then reads
  "it registered no energy across 11 µs of work", so the zero gate still holds
  it back.
- FJ3, a zero delta accepted as a figure: a_window_the_meter_cannot_resolve_is_
  never_a_zero fails.
- FJ4, seal through plain tag (every figure HwShunt): three tests fail; the
  on-host one sealed "HwShunt 5379036 µJ" for apple-silicon's estimate.
- FJ5, an estimate posted "measured via": three tests fail, GitHub and Slack.
- FJ6, tag_with_provenance sealing Energy::measured for an estimator:
  a_tags_energy_is_sealed_at_the_provenance_it_was_obtained_at fails.

clippy --no-deps --tests on jouledesk and jouledesk-github: nothing new (the
unused libwebrtc patch warning is the workspace's). jcp-efficiency is a new
dev-dependency of jouledesk-github, already in the graph through jouledesk.

Not done here, deliberately:
- Work::measured_uj and Handled::measured_uj are public Rust fields that now
  carry an estimate when provenance says `estimator` (documented). The sealed
  efficiency/1 is right (measured_uj: None for an estimate); renaming the Rust
  fields is an API question.
- Teammate::tag still seals a skill's declared cost as a HwShunt reading. The
  signed jouledesk vector is built through it, so changing it means
  regenerating a signed vector.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

226a70cb64

• View at this point in history

2026-09-30 01:39:31 -04:00

Transaction Science

jouleclaw-energy: the liveness probe samples the Energy Model group alone ...

The family liveness rule says the probe re-samples the CPU energy rail after
every ~25 ms of CPU burn on the probing thread. In this crate that was not
true. The probe (b9d7e945) sampled the counter's own subscription, which holds
every IOReport channel, and one IOReportCreateSamples over every channel costs
0.12-0.8 s on the M5 Max (11,885 channels; 0.4-0.8 s yesterday, 117-144 ms
today) and ~0.15 s on the M3 Ultra (14,342). So the "1 s of burn, polled every
25 ms" probe spent much of its second waiting inside IOReport, compared the
rail a handful of times, and overran its budget by up to a sample. It still
declined correctly on both machines; it just was not the probe it said it was.

The fix. The probe now opens its own subscription to the Energy Model group
(IOReportCopyChannelsInGroup("Energy Model") then IOReportCreateSubscription),
samples that, and drops it. The CPU rails in it are the same channels the
counter reads, at the same absolute values. If the group cannot be subscribed,
the probe opens the counter's all-channel subscription only to tell
Unavailable (IOReport does not open) from CpuRailAbsent (no such group), as
eoc-meter does (c6378629). The counter's own read path is unchanged: it still
subscribes to every channel and filters the group when it sums. The rule
itself (RailWatch: the same rail advancing in two separate intervals) is
unchanged.

Measured with this commit, by the new test, from what the probe itself sampled:
  M5 Max:   364 channels per sample, 0 outside the group (all-channel: 11,885);
            one sample 5.9 ms (all-channel 117 ms); 33 intervals in 1.08 s.
  M3 Ultra: 645 channels per sample, 0 outside the group (all-channel: 14,342);
            one sample 8.6 ms (all-channel 146 ms); 32 intervals in 1.10 s.
The uncached probe (open + 1 s of burn) took 1.10 s on the M5 Max and 1.13 s on
the M3 Ultra; on the all-channel subscription it took 1.32 s on the M5 Max
this morning, 1.9 s when the core commit measured it.

To test what the probe really sampled rather than a helper beside it, the
sampling loop now returns a ProbeRun: the RailWatch, the channels per sample,
and how many were outside the Energy Model group. host_status caches
probe_status, which is probe(budget).0.

Tests.
- the_liveness_probe_subscribes_to_the_energy_model_group_only (new): runs the
  same probe host_status runs, and asserts every channel it sampled was in the
  Energy Model group, that it sampled fewer channels than one all-channel
  sample holds, and that the group's CPU rails are exactly those in the
  counter's subscription. Where the group cannot be subscribed, the status
  must say CpuRailAbsent or Unavailable, matching whether IOReport opens. It
  asserts on channels, not timings; the timings are printed.
- the_liveness_probe_never_reports_a_frozen_rail_as_live now runs the real
  uncached probe_status instead of a probe on the all-channel subscription.
40 passed on both machines, both "status=CpuRailFrozen
oracle=CpuRailMoved(false) selected=false available=false".

Falsified, each mutation applied to this commit and restored with git checkout:
- FP1, the probe back on the counter's all-channel subscription (probe()
  calling open()): "the probe sampled 11521 channels outside the Energy Model
  group (11885 per sample)"; it managed 6 intervals in 1.45 s.
- FP2, the group subscription built from IOReportCopyAllChannels: the same
  failure, 11521 of 11885.
- FP3, the family rule's threshold set to 1 (LIVE_RAIL_ADVANCES = 1): four rule
  tests fail, e.g. a_single_jump_is_a_batch_update_not_a_live_rail
  ("assertion failed: !w.is_live()") and a_rail_that_keeps_advancing_is_live
  ("one advance is not yet live").

Not done here, deliberately:
- The counter's own read() still samples every channel (0.1-0.8 s per read on
  the M5 Max). It is declined on both machines, so nothing pays that today;
  changing what a live counter's read costs is a separate change.
- The oracle in the tests still samples every channel, deliberately: it shares
  no subscription with the probe.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

1a477bc8d3

• View at this point in history

2026-09-30 01:32:54 -04:00

2026-09-29

Transaction Science

jouleclaw-energy: never use IOReport unless its CPU rail keeps moving ...

This crate's contract, in its own docs: every reading carries a Provenance
"declaring how the value was obtained", and "implementations that overstate
their honesty are non-conformant". The family contract (jcp-energy):
"measured when the platform exposes it, estimated when it doesn't, and the
provenance always says which ... A reading never silently upgrades its own
provenance." The IOReport counter broke both on current macOS.

Ground truth, measured today by reading the channels, not by re-running tests:

- M5 Max, macOS 27.0.1: `CPU Energy` (the only channel in any group whose name
  ends in "CPU Energy") held 480,083,160 mJ across a 3 s CPU burn while
  `GPU Energy` advanced ~892 mJ. M3 Ultra, same OS: `DIE_0_CPU Energy`,
  `DIE_1_CPU Energy` and `GPU Energy` all held. All are format 1 (Simple).
- The CPU rail is not permanently frozen; it is batch-updated, tens of
  minutes apart. Watched for 25 minutes (10:51-11:16) through an
  Energy-Model-only subscription sampled every 100 ms and an all-channel one
  every 2 s: on the M3 Ultra both dies' CPU rails changed exactly once, in
  the same sample, at 11:09:05 in both subscriptions (+1,344,869 and
  +1,391,637 mJ); on the M5 Max the CPU rail did not change in 13,614
  samples while `GPU Energy` changed in 13,125 of them. Yet the M5 Max rail
  read 541,893,638 mJ by 10:47, up from 480,083,160 at ~10:30.
- A live rail looks nothing like that: the M5 Max GPU rail changed in 166 of
  176 samples ~28 ms apart.
- One IOReportCreateSamples over the counter's all-channel subscription costs
  ~0.35-0.7 s on the M5 Max (11,860 channels), ~0.135 s on the M3 Ultra
  (14,335 channels).

The defect. `AppleIoReportCounter::new()` was available whenever a
subscription opened, `detect_meter()` put it first (since b11af6ce), and
`read()` summed every Energy Model rail, skipped non-positive deltas, and
labelled the total SocTotal / ModelBased. On the M5 Max a CPU burn was
metered as GPU energy — main's `measures_energy_after_cpu_burn` passes there
only through the GPU rail. On the M3 Ultra nothing moves and the same test
fails on main: "expected measured energy to advance after a CPU burn:
before=0 after=0"; a consumer flooring that zero to the meter's resolution
sealed it as a ModelBased reading.

The fix enforces the contract where the counter is opened.
`AppleIoReportCounter::new()` (so `Default`, `detect_meter()` and every direct
caller) returns an available counter only when `ioreport_status()` is `Live`.
The status is probed once per process and cached (OnceLock): open a
subscription, burn CPU on the calling thread, re-sample after every 25 ms of
burn for the 1 s budget, and decide on the raw integer of each CPU rail
(`CPU Energy`, or `DIE_<n>_CPU Energy`) per consecutive sample interval —
never on the summed total, because the live GPU rail is exactly what
corrupts the total. Otherwise the counter is declined: `is_available()` is
false, `status()` says why, and every `read()` returns
`Err(EnergyError::NoCounter(SocTotal))` — never a number. `detect_meter()`
then falls through to `apple::AppleSiliconMeter`, whose readings say
`Estimator` end to end (CompositeMeter passes the reading through; nothing
in this crate re-tags it). The probe costs ~1.9 s once per process on the
M5 Max. New public API, all additive: `IoReportStatus` { Unavailable,
CpuRailAbsent, CpuRailFrozen, Live }, `jouleclaw_energy::ioreport_status()`
(compiled on every target; `Unavailable` without the backend, no probe),
`AppleIoReportCounter::status()`, `apple_ioreport::CPU_RAIL_LIVENESS_BUDGET`.

One deliberate difference from the jcp-energy reference (dfec5d44): a single
advance does not make a rail live. The same CPU rail must advance in at least
two separate sample intervals (`RailWatch`). Because the rail is
batch-updated, a probe whose window straddles an update sees one large jump —
tens of minutes of the whole machine's CPU energy — and the reference's
"advanced at all" rule calls that live and selects the counter for the rest
of the process. That happened here: at ~10:46, while a falsification run
re-probed without the cache, fresh probes under the reference rule reported
the M5 Max rail Live and the oracle, a moment later, saw it hold. Counting per
rail keeps two dies' separate jumps from adding up. A live rail passes in its
first two intervals; the probe samples past the budget until it has two, up
to five budgets, so one slow sample cannot make a live rail look frozen.

Docs made false by the change are corrected in the files touched: the module
and `new()` / `is_available()` docs; the `detect_meter()` comment that said
the counter is unavailable only "if IOReport can't open"; the `apple`
estimator's doc, which said IOReport was not yet wired and told a maintainer
to re-tag the estimator ModelBased once it was — it is now the path that runs
on macOS 27, so that instruction would relabel an estimate as a reading; and
the `energy_probe` example, which printed "--- measured ---" over whatever it
picked and now prints why IOReport was declined and "--- estimated ---" for
an Estimator. No grade changes.

Tests. `measures_energy_after_cpu_burn` — which asserted only that the summed
total advanced, and returned early, passing, when IOReport was unavailable —
is replaced. The new tests assert against an independent oracle: it opens its
own subscription, reads each CPU rail's ABSOLUTE value (no delta samples),
matches names with `ends_with("CPU Energy")` in any group, burns for three
times the probe budget, and returns NoIoReport / NoCpuRail /
CpuRailMoved(bool), true only if a rail's value changed at two or more
separate reads, so one batch jump landing in both windows cannot vouch for a
false Live. No test returns early without asserting.

- the_ioreport_counter_is_never_used_on_a_frozen_cpu_rail: neither available
  nor selected by detect_meter when the oracle saw the rail hold; logs
  "ioreport liveness: status=.. oracle=.. selected=.. available=..".
- the_liveness_probe_never_reports_a_frozen_rail_as_live: runs the probe
  uncached on a fresh subscription beside the cached status; asserts on every
  outcome, not only Live.
- a_used_ioreport_counter_meters_cpu_work: when available, its own CPU rail is
  live and 200 ms of CPU meters non-zero ModelBased energy; when not, the
  status is not Live.
- a_declined_counter_yields_no_reading_and_the_fallback_says_estimator.
- the_liveness_status_is_probed_once_and_cached.
- Rule tests with no hardware: a_single_jump_is_a_batch_update_not_a_live_rail,
  one_jump_on_each_die_is_not_a_live_rail, a_rail_that_keeps_advancing_is_live,
  no_cpu_rail_is_absent_and_no_interval_is_no_verdict,
  the_probe_samples_until_live_or_long_enough_to_tell,
  only_the_cpu_energy_aggregate_is_a_cpu_rail; and, where the backend is not
  built, without_the_ioreport_backend_the_status_is_unavailable.

Observed with this commit, one test binary run on both machines:
  M5 Max:   ioreport liveness: status=CpuRailFrozen oracle=CpuRailMoved(false)
            selected=false available=false (oracle: changes per CPU rail [0])
            test result: ok. 39 passed; 0 failed
  M3 Ultra: ioreport liveness: status=CpuRailFrozen oracle=CpuRailMoved(false)
            selected=false available=false (oracle: changes per CPU rail
            [0, 0], the DIE_0_/DIE_1_ path)
            test result: ok. 39 passed; 0 failed
With `--features ioreport` 39 lib tests (main: 29); without it 24 (main: 23).

Falsified on the M5 Max, each mutation applied to this commit and restored
with git checkout:
- F1, the probe always says Live: "the oracle did not see the CPU rail keep
  moving under 3s of CPU burn, yet status=Live" (and the probe test).
- F2, main's ungated new(): "IOReport counter used although its CPU rail did
  not keep moving under 3s of CPU burn — it would report GPU-only energy, or
  zero, as the energy of CPU work (available=true, selected=true)".
- F3, no per-process cache: "a cached status and a declined counter took
  6.389182s: the liveness probe ran again".
- F4, liveness decided on every Energy Model rail, as the total sees it: the
  GPU rail makes the M5 Max probe say Live; both liveness tests fail.
- F5, the reference's single-advance rule: four rule tests fail, e.g.
  one_jump_on_each_die_is_not_a_live_rail "left: Some(Live) right:
  Some(Frozen)".
- F6, advances counted across rails, not per rail:
  one_jump_on_each_die_is_not_a_live_rail fails.

Dependants, per workspace with -p: mesh-meter and mesh-core with
hw-meter-apple (2 and 3 passed), jouledesk-github (4), l3-jouleclaw-binding
(2; 4 ignored, they need a model), jouleclaw-harness-econ (11),
jouleclaw-loader-gguf with fabric-meter (357 passed, 14 ignored) — all pass.
clippy -p jouleclaw-energy --no-deps --tests, with and without ioreport: no
new warnings (the duplicated #[link] attribute, sum_energy_uj's b"..\0",
ledger.rs's constant asserts and meter.rs's vec-init-then-push are on main).
cargo check --tests --examples for x86_64- and aarch64-unknown-linux-gnu,
with and without ioreport: clean.

Not done here, deliberately:
- The consumers (jouledesk-github, mesh-meter, l3-jouleclaw, loader-gguf's
  fabric_dispatch / fabric_calibrate / mpp_energy, harness-econ) are handled
  next, on this branch. On macOS 27 they now get no IOReport counter.
- Reporting the live GPU rail as an explicitly partial reading instead of
  declining, and which grade IOReport readings carry, are David's decisions.
- A per-read check for a rail that stops moving after selection needs a
  witness (CPU Stats residency); the platform-level freeze is caught here.
- `counts_once` would sum `PACC_<n>` sub-rails (no "cpu" in the name) on top
  of `CPU Energy`, a latent double count once the rail is live; fixing it
  needs a channel dump from a host whose CPU rail is live.
- The counter subscribes to all ~12-14k channels, so every read() costs one
  0.35-0.7 s sample on the M5 Max; an Energy-Model-only subscription samples
  in ~8 ms. Worth doing, but it changes what every read costs.
- jcp-energy (the reference) and the sibling backends still accept a single
  advance as live.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

b9d7e9450f

• View at this point in history

2026-09-29 11:22:20 -04:00