a process reads its own energy only once the reading thread has left its core #229

Merged

dcharlot merged 5 commits from fix/in-process-self-read-fold into main 2026-10-13 02:22:46 -04:00

Conversation 0 Commits 5 Files changed 14 +2374 -69

dcharlot commented 2026-10-13 00:57:04 -04:00

Owner

Copy link

Why

The kernel adds a thread's CPU energy to its process's count (ri_energy_nj) only when that thread leaves its core. So a process reading its own count from a running thread does not see that thread's latest energy:

• At a span's start, up to a scheduling quantum of energy spent before the span lands inside it and is sealed as a hardware reading.
• At a span's end, the tail is missing.

#214 measured this and fixed it in sandbox-meter. Four other crates carry their own in-process counter and read it with no check, and they meter short spans: proof attempts, cascade lookups, flow scans, served requests.

Measured on an idle M3 Ultra, before the fix. In every one of the four crates:

• 39–40 of 40 spans of 1 ms read no advance;
• 37–40 of 40 sleeps of 2 ms taken right after an 8 ms burn were sealed with the burn (40–53 mJ for a sleep).

An idle host is the worst case, because nothing takes the reading thread off its core but its own quantum. A loaded M5 Max showed 60–78%.

What

Each crate keeps its own code, since the standards do not depend on each other. The rule is the same in all four.

The rule. Before a self-read, sleep (50 µs, then 200 µs, 1 ms, 5 ms) until the reading thread's own energy count has moved across a sleep. That count is proc_pidinfo flavour 34, a private-header flavour, used only where it reads in the expected layout. A read that is not seen to fold is no reading: the span takes the crate's existing estimate path. It is never a zero, and never a short figure under a hardware label. Reads of another process (a child, a tree) are not folded.

Per crate:

• jouleclaw-energy. ProcessCpuCounter, and through it ProcessSpanMeter's CPU half, mesh's ProcessMeter and the fabric calibration. Adds EnergyError::NotFolded.
• jcp-energy. The ProcessEnergy rung. An unverified end falls to the default-power estimate.
• eoc-meter. The in-process counter. A failed read leaves no span, and every consumer already keeps its estimate then.
• proof-cascade. The reads behind measure_marginal. That measurement was already one span around a batch.

Docs: each crate's module docs, jouleclaw/SPEC.md §1.3, and the eoc SCI annex. The annex's "accrues in lumps" observation now carries its mechanism and the remedy; its measured figures stand.

Evidence (M3 Ultra, idle unless stated)

Crate	1 ms burn, before → after	2 ms sleep after an 8 ms burn, before → after	Loaded repeats (40 busy loops)
jouleclaw-energy	40/40 no advance → 40/40 within the oracle	40/40 read the burn → 40/40 clean	110/110
jcp-energy	40/40 estimated → 40/40 HwShunt, ≥985/1000 of the oracle	39–40/40 HwShunt on the burn → 40/40, ≤4/1000	110/110
eoc-meter	39–40/40 wrong → 40/40	37–40/40 read the burn → 40/40	110/110
proof-cascade	40/40 not read → 40/40	40/40 read the burn → 40/40	110/110

• Oracle. Each new test folds its own way (sleep until the process switch count moves) and reads ri_energy_nj from a raw buffer. It shares no code with the meter.
• M5 Max. jouleclaw-energy was also measured on a loaded M5 Max, before and after. The other three were measured on the M3 Ultra only.
• Cost per read. About 80 µs idle; a median of 150–340 µs under 40 busy loops. An unverified read costs all four sleeps.
• Mutation checks. An always-pass fold, and a skipped fold, each fail tests in every crate.
• Suites, on the build host:
  • jouleclaw-energy with ioreport: 129;
  • jcp-energy: 112;
  • eoc-meter: 86;
  • proof-cascade: 111;
  • the jcp, eoc and proof workspaces, mesh, and loader-gguf with fabric-meter pass;
  • jcp-conformance and proof-conformance pass;
  • proof gen-vectors output is byte-identical to proof/conformance/v1;
  • Linux cross-checks are clean.

Not covered

• Other threads of the process are still unfolded at a read, by up to a quantum each at each end. This is documented, and it cannot be fixed from the reading thread.
• Sub-quantum spans and the meter's own cost. The eoc cascade's cache hits and jcp-intercept's per-step spans now read correctly, but the reading is mostly the meter's own closing wake-up (tens of µJ, about 0.17 ms). Charging cache hits at a host-calibrated per-hit figure, and opening a span only for the neural stage, is the follow-up to this PR.

🤖 Generated with Claude Code

## Why The kernel adds a thread's CPU energy to its process's count (`ri_energy_nj`) only when that thread leaves its core. So a process reading its **own** count from a running thread does not see that thread's latest energy: - At a span's start, up to a scheduling quantum of energy spent **before** the span lands inside it and is sealed as a hardware reading. - At a span's end, the tail is missing. #214 measured this and fixed it in sandbox-meter. Four other crates carry their own in-process counter and read it with no check, and they meter short spans: proof attempts, cascade lookups, flow scans, served requests. **Measured on an idle M3 Ultra, before the fix.** In every one of the four crates: - 39–40 of 40 spans of 1 ms read no advance; - 37–40 of 40 sleeps of 2 ms taken right after an 8 ms burn were sealed with the burn (40–53 mJ for a sleep). An idle host is the worst case, because nothing takes the reading thread off its core but its own quantum. A loaded M5 Max showed 60–78%. ## What Each crate keeps its own code, since the standards do not depend on each other. The rule is the same in all four. **The rule.** Before a self-read, sleep (50 µs, then 200 µs, 1 ms, 5 ms) until the reading thread's own energy count has moved across a sleep. That count is `proc_pidinfo` flavour 34, a private-header flavour, used only where it reads in the expected layout. A read that is not seen to fold is no reading: the span takes the crate's existing estimate path. It is never a zero, and never a short figure under a hardware label. Reads of **another** process (a child, a tree) are not folded. **Per crate:** - **jouleclaw-energy.** `ProcessCpuCounter`, and through it `ProcessSpanMeter`'s CPU half, mesh's `ProcessMeter` and the fabric calibration. Adds `EnergyError::NotFolded`. - **jcp-energy.** The `ProcessEnergy` rung. An unverified end falls to the default-power estimate. - **eoc-meter.** The in-process counter. A failed read leaves no span, and every consumer already keeps its estimate then. - **proof-cascade.** The reads behind `measure_marginal`. That measurement was already one span around a batch. **Docs:** each crate's module docs, `jouleclaw/SPEC.md` §1.3, and the eoc SCI annex. The annex's "accrues in lumps" observation now carries its mechanism and the remedy; its measured figures stand. ## Evidence (M3 Ultra, idle unless stated) | Crate | 1 ms burn, before → after | 2 ms sleep after an 8 ms burn, before → after | Loaded repeats (40 busy loops) | |---|---|---|---| | jouleclaw-energy | 40/40 no advance → 40/40 within the oracle | 40/40 read the burn → 40/40 clean | 110/110 | | jcp-energy | 40/40 estimated → 40/40 `HwShunt`, ≥985/1000 of the oracle | 39–40/40 `HwShunt` on the burn → 40/40, ≤4/1000 | 110/110 | | eoc-meter | 39–40/40 wrong → 40/40 | 37–40/40 read the burn → 40/40 | 110/110 | | proof-cascade | 40/40 not read → 40/40 | 40/40 read the burn → 40/40 | 110/110 | - **Oracle.** Each new test folds its own way (sleep until the process switch count moves) and reads `ri_energy_nj` from a raw buffer. It shares no code with the meter. - **M5 Max.** jouleclaw-energy was also measured on a loaded M5 Max, before and after. The other three were measured on the M3 Ultra only. - **Cost per read.** About 80 µs idle; a median of 150–340 µs under 40 busy loops. An unverified read costs all four sleeps. - **Mutation checks.** An always-pass fold, and a skipped fold, each fail tests in every crate. - **Suites, on the build host:** - jouleclaw-energy with `ioreport`: 129; - jcp-energy: 112; - eoc-meter: 86; - proof-cascade: 111; - the jcp, eoc and proof workspaces, mesh, and loader-gguf with `fabric-meter` pass; - `jcp-conformance` and `proof-conformance` pass; - proof `gen-vectors` output is byte-identical to `proof/conformance/v1`; - Linux cross-checks are clean. ## Not covered - **Other threads** of the process are still unfolded at a read, by up to a quantum each at each end. This is documented, and it cannot be fixed from the reading thread. - **Sub-quantum spans and the meter's own cost.** The eoc cascade's cache hits and jcp-intercept's per-step spans now read correctly, but the reading is mostly the meter's own closing wake-up (tens of µJ, about 0.17 ms). Charging cache hits at a host-calibrated per-hit figure, and opening a span only for the neural stage, is the follow-up to this PR. 🤖 Generated with [Claude Code](https://claude.com/claude-code)

dcharlot added 5 commits 2026-10-13 00:57:05 -04:00

jouleclaw-energy: a process reads itself only once the reading thread has left its core ... 6fe45b175f

The kernel adds a thread's CPU energy to its process's count
(proc_pid_rusage ri_energy_nj) only when the thread leaves a core. A
process reading its own count does not take the reading thread off its
core, so the read is short of what that thread spent since it last left:
a span's first read put up to a scheduling quantum of energy from before
the span inside it, and its last read left the span's tail out. Both were
sealed HwShunt.

Measured against a reader that sleeps before its own reads and checks
the process's context-switch count rose, 40 spans of each kind a run,
three runs a host (macOS 27.0.1):
- 1 ms of the reading thread's CPU: on an M5 Max at a load average of
  about 90, 25, 31 and 27 spans read no advance or under three quarters
  of the work; on an idle M3 Ultra, all 40 read no advance, every run;
- a 2 ms sleep straight after 8 ms of CPU: on the M5 Max 25, 26 and 31
  spans read the CPU before them as the sleep's energy, up to 18 mJ; on
  the M3 Ultra all 40 read the whole burn, 42 to 53 mJ for a sleep.

ProcessCpuCounter now sleeps before each read of its own process (50 us,
then 200 us, 1 ms, 5 ms) and reads only once the reading thread's own
energy count (proc_pidinfo PROC_PIDTHREADCOUNTS, layout-gated: 88 bytes
and two performance levels on both hosts) rose across a sleep. A read
whose thread was not seen to leave is EnergyError::NotFolded, no reading:
measure() gives no figure, read() fails, and a ProcessSpanMeter span
falls back to the host meter. The fold is inside process::energy_nj, the
one call span.rs makes for the CPU half, so it holds whatever order
span.rs takes its readings in. The counter is Unavailable where the
thread's count does not read, and the liveness probe still reads with no
sleep.

With it, all 40 spans of each kind agreed on both hosts: at least 95% of
the work on the loaded M5 Max and 97% on the quiet M3 Ultra, at most
0.5% of the burn before a sleep; and under 40 busy loops on the M3 Ultra
(220 runs), at least 84% and at most 2% (the oracle's window also holds
both readers' own reads, which do not shrink with the work). A read
costs about 80 us on the quiet M3 Ultra, about 200 to 330 us at the
median under the busy loops, and 81 to 154 us at the median on the M5
Max at a load average of 135, against 1 us.

tests/process_cpu_short_span.rs is the test that fails without the fold.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

jcp-energy: the per-process rung reads its own count only once the metering thread has left its core ... 1108d507f9

SPEC §5.1 rung 3 already says it: the kernel adds a thread's energy to
its process's count (proc_pid_rusage ri_energy_nj) only when the thread
leaves a core, a process's read of its own count does not take the
reading thread off its core, and an implementation that relies on the
thread having left MUST check that it did. The reference meter did not.
Each end of a span was read from a thread still on its core, so a span
held up to a scheduling quantum of what the thread spent before it and
lacked its own tail, and was graded HwShunt.

Measured on an idle M3 Ultra (macOS 27.0.1) against the tests' own
reader, which sleeps before each read until the process's
context-switch count rose; 40 spans of each kind a run, six runs:
- 1 ms of the metering thread's CPU: all 40 showed no rise and were the
  default-power estimate, every run;
- a 2 ms sleep straight after 8 ms of CPU: 39 or 40 were graded HwShunt
  on the burn before them, about 41 mJ for a sleep.

Now ProcessEnergy::read, the read at each end of a Meter span and of a
FabricMeter span, sleeps (50 us, then 200 us, 1 ms, 5 ms) until the
metering thread's own energy count (proc_pidinfo PROC_PIDTHREADCOUNTS,
decoded only in the one layout known: 88 bytes, two performance levels
here) rose across a sleep, and only then reads the process. An end whose
thread was not seen to leave is not read: the Meter span is the
default-power estimate (process_span's existing path, Estimator, no
measured figure) and the FabricMeter span reports no CPU figure. The
rung opens only where the thread's count reads. The liveness probe still
samples with no sleep. The layout decoding and the rule are pure and
compiled on every platform.

With it, on that host, idle and under 40 busy loops (110 runs): every
span of both kinds was read, the work at 84% or more of the tests' own
window and the sleep at 6% or less of the window that held the burn. A
read costs about 80 us idle and 169 us at the median under the busy
loops, against under 1 us.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

eoc-meter: the process counter reads its own count only once the reading thread has left its core ... 0f6a9a3bb3

The kernel adds a thread's CPU energy to its process's count
(proc_pid_rusage ri_energy_nj) only when the thread leaves a core, and a
process's read of its own count does not take the reading thread off its
core. MacosProcessEnergyCounter read the count as it stood, so a span
bracketed by two reads held up to a scheduling quantum of what the
thread spent before the span and lacked the span's own tail, and every
consumer here tags a positive delta of the counter Measured.

Measured on an M3 Ultra (macOS 27.0.1, load average about 14) against
the tests' own reader, which sleeps before each read until the process's
context-switch count rose; three runs of 40 spans of each kind:
- 1 ms of the reading thread's CPU between two reads: 39, 40 and 40
  spans read nothing or under two thirds of the work;
- a 2 ms sleep between two reads, straight after 8 ms of CPU: 40, 40 and
  37 spans read the burn before them, about 41 mJ for a sleep.

read_microjoules now sleeps (50 us, then 200 us, 1 ms, 5 ms) until the
reading thread's own energy count (proc_pidinfo PROC_PIDTHREADCOUNTS,
read only in the one layout known: 88 bytes, two performance levels
here) rose across a sleep, and only then reads the process. A read whose
thread was not seen to leave is an error, not a short count: eoc-cascade,
eoc-agent and eoc-local all take a failed read as no span and keep their
estimate. process_energy_status is Unavailable, with no probe run, where
the thread's count does not read, and the liveness probe still samples
with no sleep. The rule and the layout decoding are pure and compiled on
every platform.

With it, on that host, quiet and under 40 busy loops (110 runs): every
1 ms span read 79% or more of the tests' window around it (98% when
quiet) and no sleep read more than 3% of the window that held the burn.
A read costs 83 us at the median on the quiet host and 226 us under the
busy loops, against under 1 us; detect_in_process's docs say so, and
that a batch of short operations is bracketed by one span, not one each.

tests/process_energy_short_span.rs is the test that fails without it.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

proof-cascade: the meter reads its own process only once the measuring thread has left its core ... 60bcca7594

measure_marginal's process-counter rung brackets its window with two
reads of this process's CPU energy (proc_pid_rusage ri_energy_nj). The
kernel adds a thread's energy to that count only when the thread leaves
a core, and a process's read of its own count does not take the reading
thread off its core. So the first read left up to a scheduling quantum
of what the measuring thread spent before the window (the liveness
probe's burn, the caller's own work) to land inside it, and the figure
is sealed HwShunt.

Measured on an M3 Ultra (macOS 27.0.1) with the meter's own read at each
end of a window on the reading thread, against the tests' own reader,
which sleeps before each read until the process's context-switch count
rose; three runs of 40 windows of each kind:
- 1 ms of the reading thread's CPU: all 40 read no advance, every run;
- a 2 ms sleep straight after 8 ms of CPU: all 40 read the burn before
  them, every run, 40 to 48 mJ for a sleep.

Each of the measurement's two reads now sleeps (50 us, then 200 us,
1 ms, 5 ms) until the measuring thread's own energy count (proc_pidinfo
PROC_PIDTHREADCOUNTS, read only in the one layout known: 88 bytes, two
performance levels here) rose across a sleep, and only then reads the
process. A read whose thread was not seen to leave gives the measurement
no figure, by the path an unreadable counter already takes: the work
does not run, measure_marginal returns None and the caller keeps its
estimate. The process counter is Unavailable where the thread's count
does not read (proof --measure says so), and the liveness probe still
samples with no sleep. The rule and the layout decoding are pure and
compiled on every platform.

The measurement was already one window around a batch: a second of
calls on every spare core, two reads in all. Its workers have exited and
been joined before the closing read, and a thread's exit puts its last
time on a core into the count before the join returns (measured: a 5 ms
burn read the same at the join whether or not the thread slept first),
so only the measuring thread needed taking off its core.

With it, on that host: quiet, every window read 98% or more of the
tests' window and no sleep more than 1% of the window that held the
burn; under 40 busy loops (110 runs), 82% or more and 4% or less. A
read costs about 80 us quiet and 190 us at the median under the busy
loops, against under 1 us.

process_window_read_nj is a doc-hidden hook, like the crate's other
test hooks: tests/process_energy_short_span.rs, the test that fails
without the check, needs a process of its own. The published
conformance vectors are unchanged (gen-vectors into a scratch directory
is byte-identical to conformance/v1).

Co-Authored-By: Claude Opus 5.5 <[email protected]>

eoc annex: the per-process counter's lumps are the kernel's fold, and a checked read is exact ... ac93d9e56e

The annex reported that the macOS per-process energy counter "accrues in
lumps" under sub-millisecond sampling, and left it there. The mechanism
is now established: a thread's energy is added to its process's count
when the thread leaves its core, so a reader on a running thread sees
nothing until its quantum ends. A reader that first leaves its core and
checks its own thread's count moved reads 1 ms spans to within a few
percent of an independent read, where unchecked it read none of them on
an idle host. Say so beside the measured figures, which stand.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

dcharlot merged commit a4dc260c5d into main 2026-10-13 02:22:46 -04:00

dcharlot referenced this pull request from a commit 2026-10-13 02:22:48 -04:00

Merge pull request 'a process reads its own energy only once the reading thread has left its core' (#229) from fix/in-process-self-read-fold into main

dcharlot referenced this pull request 2026-10-13 02:27:32 -04:00

eoc-cascade: a hit reads no meter and is charged a calibrated estimate; the span opens at the neural stage #233

dcharlot referenced this pull request from a commit 2026-10-13 02:44:02 -04:00

jcp-intercept: a reading meter meters a seam as one span, divided among its steps by CPU time

dcharlot referenced this pull request 2026-10-13 02:44:05 -04:00

jcp-intercept: a reading meter meters a seam as one span, divided among its steps by CPU time #236

Sign in to join this conversation.

Reviewers

No reviewers

Labels

Clear labels

No items

No labels

Milestone

Clear milestone

No items

No milestone

Projects

Clear projects

No items

No project

Assignees

Clear assignees

No assignees

1 participant

Notifications

Subscribe

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference

Transaction-Science/open-standards!229

No description provided.