grade labels and fallback windows: every figure says how it was obtained #222

Merged

dcharlot merged 11 commits from fix/grade-labels-and-windows into main 2026-10-12 09:08:53 -04:00

Conversation 0 Commits 11 Files changed 24 +728 -173

2026-10-12

Transaction Science

jouleclaw-energy: the process-counter test asserts against an oracle, not a joule floor ...

The test required 200 ms of one thread's CPU to read at least 0.1 J. What
200 ms of CPU costs depends on the core class that ran it: on an
efficiency core at a load average of 114-186 it read 0.094-0.099 J, the
counter agreeing with an independent oracle each time, and the test
failed on the host's load, not on the meter.

The burn is now held to an oracle's window around it (no more than it,
and at least three quarters of it — the counter's own reads may each miss
a scheduling quantum of the reading thread), the sleep to under a tenth
of the burn, and the parent's window while only a child works to under a
tenth of what the child burned. A counter that reads wall time at a fixed
power still fails the sleep check. 7 of 7 runs passed at load ~186,
four of them on burns the old floor would have failed.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

2e13655f29

• View at this point in history

2026-10-12 08:48:53 -04:00

2026-10-11

Transaction Science

jouleclaw-fabric: one calibration-file path for the placer and its writers ...

Fabric::apply_calibration_file and mpp_energy honoured
JOULECLAW_FABRIC_CALIBRATION, but fabric_calibrate always wrote
~/.jouleclaw/fabric-calibration.json, so a calibration written under the
variable was not the file the placer loaded. calibration_file_path() is
now the one place the path is decided, and the reader and both writers
use it; the rule is tested without touching the process environment.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

18124713ac

• View at this point in history

2026-10-11 22:23:23 -04:00

Transaction Science

mesh-meter: the span meter's fallback window no longer holds its GPU waits ...

ProcessMeter's docs justified billing an estimated fallback span with this
crate's EstimatorMeter by the host meter's window taking in the span meter's
GPU-rail waits. jouleclaw-energy now reads the host meter inside the GPU
snapshots, so that window is the work's own and the reason no longer holds.
The behaviour is unchanged (an estimated request is still billed by this
crate's estimator over the work's window, as the rest of a partial span is);
the comments now say so instead of describing the old window.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

9a6c28ff39

• View at this point in history

2026-10-11 22:23:23 -04:00

Transaction Science

jouleclaw-energy: a fallback span's window is the work's, not the GPU waits ...

ProcessSpanMeter read the host meter first at a span's start and last at its
end, outside both GPU snapshots. A snapshot can wait for the GPU rail to
republish (up to GPU_IDLE_WINDOW on a quiet GPU, RAIL_EDGE_WAIT on a busy
one), so a span that fell back to the host meter bracketed those waits: its
time-based estimate billed the meter's waiting, not the work (an ~11 us serve
in mesh-openai's metal example was billed ~1.5-9.5 J).

The host meter is now read between the GPU snapshot and the CPU reading at
each end: start = GPU snapshot, host meter, CPU; end = CPU, host meter, GPU
snapshot. The CPU window stays innermost (neither the snapshots' CPU time nor
the host meter's reads are in it), and the fallback window is the work's own.
The readings go through a small SpanReads seam so a test can drive the order
on a clock of its own: with snapshots waiting 0 to 1 s, a fallback span's
estimate equals the work's duration (plus one host read) and the CPU window
equals the work exactly.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

4b0d932bfc

• View at this point in history

2026-10-11 22:23:23 -04:00

Transaction Science

jouleclaw-tokenpower-bench: define the grades as the family does ...

EnergyProvenance::ModelBased was defined as "derived from vendor TDP +
utilization counters (NVML / IOReport / ROCm / RAPL)". An uncalibrated
TDP x utilization model is an estimate (JCP SPEC 5.1 rungs 5-6: only time x
a power calibrated on the host is ModelBased), so a runner following that
definition would claim a better grade than its figure has. The same text also
filed live vendor energy counters (RAPL, NVML total energy, IOReport) under
ModelBased, and defined HwShunt as an in-line shunt only.

The three definitions now follow the family: HwShunt is a shunt or a vendor
energy counter read while live; ModelBased is the device's own sampled power
integrated over time or wall time x a host-calibrated power; Estimator is a
constant or an uncalibrated model such as TDP x utilization. The crate assigns
no grade itself (callers pass the tag, and nothing in the tree uses it or
publishes its output), so there is no code path or benchmark output to
regrade.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

f9ba141751

• View at this point in history

2026-10-11 22:23:23 -04:00

Transaction Science

jouleclaw-llm: grade a backend's figure by what the backend declares ...

LlmCheapTier::report_provenance graded any Some(joules) as ModelBased, so a
backend that reports a bare constant (verification-tier's StaticBackend
says 2.0 J every call) would have been graded a calibrated model's figure
had it been wired to the L3 tier. A number is not evidence of a meter.

LlmBackend gains energy_provenance(), defaulting to Estimator: a backend
that read a live counter over the call says HwShunt, one with a model
calibrated on the host says ModelBased, and everything else stays an
estimate. report_provenance (now a &self method; nothing outside the crate
called it) returns that grade for a reported figure and Estimator for the
typical-cost fallback. StaticBackend keeps the default, and its comment no
longer calls its constant "real joules"; a test pins its grade.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

c70a112877

• View at this point in history

2026-10-11 22:23:23 -04:00

Transaction Science

joule-code-remediate, sandbox-meter: name the probe an attribution was read by ...

joule-code-remediate's meter source said "attributed(ModelBased over IOReport
HwShunt)" for every attributed run, whatever counter its windows were read
by. Today only the Apple Silicon attribution path can attribute, so the label
happened to fit, but it was a constant, not a record: an attribution read by
RAPL, or by several probes, would still have claimed IOReport.

sandbox-meter's Attribution now carries the window's own Measurement::source
(the probe that read it, "composite" for several, "estimator" for an
estimate), so a caller names the reading by what read it. joule-code-remediate
collects the sources of its attributed windows and names them; the meter
source string moves into EnergyTally::source, where a test pins that an
attribution read by "rapl-powercap" says so and never says IOReport.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

ccb382ba08

• View at this point in history

2026-10-11 22:23:23 -04:00

Transaction Science

jouleclaw-harness-econ: report a ModelBased figure as modeled, not measured ...

joules_obtained() printed "measured" for ModelBased as well as HwShunt, so
the local-model examples called a vendor energy model's figure (power x
time) a measurement. It now follows the grade: "measured" for a hardware
reading only, "modeled" for a model's figure, "estimated" for an estimate.
The test that pinned ModelBased to "measured" now pins "modeled".

Co-Authored-By: Claude Opus 5.5 <[email protected]>

d84155efea

• View at this point in history

2026-10-11 22:23:23 -04:00

Transaction Science

jouledesk-github: say a ModelBased figure was modeled, not measured ...

obtained() put "measured via <meter>" on both HwShunt and ModelBased figures,
so a reply posted a vendor model's figure to GitHub or Slack as a
measurement. The phrase now follows the grade: a hardware reading is
"measured via" its meter, a model's figure "modeled by" it, an estimate
"estimated by" it. The reply still names the provenance label beside it.

The test that pinned "measured via `vendor-model`, provenance `model_based`"
now pins "modeled by" and that the word "measured" does not appear; the
Slack reply gets the same check, and the host test splits its ModelBased and
HwShunt arms.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

2ab6b9debe

• View at this point in history

2026-10-11 22:23:23 -04:00

Transaction Science

joulehook-meter: read Mesh's own provenance labels when bridging usage.energy ...

The bridge matched only the lower-case spellings "hw_shunt" / "model_based",
but Mesh writes usage.energy.provenance with its wire labels
(mesh_schema::Provenance::jcr1_label: "HwShunt", "ModelBased", "Estimator"),
so every Mesh HwShunt or ModelBased figure fell through to the estimator
floor and was sealed into efficiency/1 under-graded.

Both spellings are now read as the same grade (the lower-case one is
JouleClaw's energy-crate display form); any other label still falls to
Estimator. The new tests feed the bridge real Mesh output
(mesh_openai::chat_completion_response with mesh_schema's jcr1_label, as
dev-dependencies) and check each grade reaches the verified receipt
unchanged. Mesh's usage.energy carries no coverage label and efficiency/1
has no coverage field, so there is no coverage label to map. The proxy's
mock upstream now writes Mesh's spelling.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

a3bacf032b

• View at this point in history

2026-10-11 22:23:22 -04:00

Transaction Science

jouleclaw-compliance: grade NVML by what was read, not by counter class ...

MeasurementMethod::Nvml (and NvmlRapl) mapped to HwShunt unconditionally,
but NVML yields a hardware reading only from its cumulative-energy counter
read while the liveness probe judged it live; everywhere else
jouleclaw-energy integrates the sampled power draw over wall time, a vendor
power model graded ModelBased. A passport could therefore call a power x time
figure a live hardware reading and claim its inference energy "measured".

Following the IoReport / IoReportLive split, a bare Nvml / NvmlRapl now
records no liveness and is graded ModelBased (a sum is no better than its
weakest part); NvmlLive / NvmlLiveRapl name the live counter and keep
HwShunt. MeasurementMethod::nvml_from_reading maps a reading's own grade to
the method, never better.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

0f1bae8e1d

• View at this point in history

2026-10-11 22:23:22 -04:00