eoc-meter: a live per-process CPU energy counter for in-process cascades #124

Merged

dcharlot merged 4 commits from eoc-meter/process-energy into main 2026-10-01 06:22:08 -04:00

Conversation 0 Commits 4 Files changed 4 +687 -27

dcharlot commented 2026-09-30 21:29:56 -04:00

Owner

Copy link

Stacked on #116 (base eoc-meter/cpu-rail-liveness). Merge #116 first; this PR then retargets to main.

On macOS 27 an unprivileged process sees IOReport's CPU rails republished only in lumps, so #116 makes eoc-meter decline there. The kernel keeps a per-process counter, ri_energy_nj from proc_pid_rusage(getpid(), RUSAGE_INFO_V6), and that one is live. It is a public libproc call, with no privilege.

The counter

• MacosProcessEnergyCounter (process-energy) reports cumulative µJ, gated on the sustained-advance rule; its figures are Measured.
  • The layout is checked at compile time: 464 bytes, energy at offset 336.
• Domain: this process's threads' CPU energy only. Not child processes, GPU, ANE or DRAM, and it includes other threads' concurrent work.
• detect() never selects it. That is tested across all 36 status combinations and on hardware. Callers opt in with detect_in_process() or try_new().
• Adopted only by the eoc CLI's query cascade. Its cache, KV store, graph and echo backend all run on this process's CPU.
  • eoc-agent is not adopted, because its shell tool runs a subprocess.
  • eoc-local's docs now say which backends the counter covers: onnx on the CPU provider, and llamacpp with n_gpu_layers 0. mlx, mlc and default llamacpp offload to the GPU.

Measured (macOS 27.0.1)

	M5 Max	M3 Ultra
status	Live (20/20 intervals)	Live (20/20)
200 ms of one thread's CPU	0.601 J	1.330 J
500 ms asleep	512 µJ	184 µJ
a child's work (its own count) vs this process around the wait	child 0.700 J, this process 0.0017 J	child 1.726 J, this process 0.0002 J

The CLI prints eoc in-process cost: 273344 µJ (measured).

Mutations caught (6):

• the counter added to detect();
• a constant at the read;
• a constant at the source;
• the parent pid in the counter;
• the parent pid in the probe;
• liveness ignored.

The workspace is green: 675 passed, 0 failed.

Updates

• Rebased onto #116's final head: the every-rail rule, RAPL package zones, and the eoc-agent fixes.
• 3513b9cc: the in-process burn test's backend is built only on macOS, where its one test runs. This clears a Linux clippy warning.
• f4847eaf: the burn test's floor is 0.1 J. User-interactive QoS is capped to default for a test binary, and 200 ms of CPU costs 0.18–1.24 J depending on the core class.
• Tests: the workspace at 705, and tests/process_energy.rs at 4/4.

🤖 Generated with Claude Code

> **Stacked on #116** (base `eoc-meter/cpu-rail-liveness`). Merge #116 first; this PR then retargets to `main`. On macOS 27 an unprivileged process sees IOReport's CPU rails republished only in lumps, so #116 makes eoc-meter decline there. The kernel keeps a per-process counter, `ri_energy_nj` from `proc_pid_rusage(getpid(), RUSAGE_INFO_V6)`, and that one is **live**. It is a public libproc call, with no privilege. ### The counter - **`MacosProcessEnergyCounter`** (`process-energy`) reports cumulative µJ, gated on the sustained-advance rule; its figures are **Measured**. - The layout is checked at compile time: 464 bytes, energy at offset 336. - **Domain:** this process's threads' CPU energy only. Not child processes, GPU, ANE or DRAM, and it includes other threads' concurrent work. - **`detect()` never selects it.** That is tested across all 36 status combinations and on hardware. Callers opt in with `detect_in_process()` or `try_new()`. - **Adopted only by the `eoc` CLI's query cascade.** Its cache, KV store, graph and echo backend all run on this process's CPU. - eoc-agent is not adopted, because its shell tool runs a subprocess. - eoc-local's docs now say which backends the counter covers: `onnx` on the CPU provider, and `llamacpp` with `n_gpu_layers` 0. mlx, mlc and default llamacpp offload to the GPU. ### Measured (macOS 27.0.1) | | M5 Max | M3 Ultra | |---|---|---| | status | `Live` (20/20 intervals) | `Live` (20/20) | | 200 ms of one thread's CPU | 0.601 J | 1.330 J | | 500 ms asleep | 512 µJ | 184 µJ | | a child's work (its own count) vs this process around the wait | child 0.700 J, this process 0.0017 J | child 1.726 J, this process 0.0002 J | The CLI prints `eoc in-process cost: 273344 µJ (measured)`. **Mutations caught (6):** - the counter added to `detect()`; - a constant at the read; - a constant at the source; - the parent pid in the counter; - the parent pid in the probe; - liveness ignored. The workspace is green: 675 passed, 0 failed. ### Updates - **Rebased onto #116's final head:** the every-rail rule, RAPL package zones, and the eoc-agent fixes. - **`3513b9cc`:** the in-process burn test's backend is built only on macOS, where its one test runs. This clears a Linux clippy warning. - **`f4847eaf`:** the burn test's floor is 0.1 J. User-interactive QoS is capped to default for a test binary, and 200 ms of CPU costs 0.18–1.24 J depending on the core class. - **Tests:** the workspace at 705, and `tests/process_energy.rs` at 4/4. 🤖 Generated with [Claude Code](https://claude.com/claude-code)

dcharlot added 2 commits 2026-09-30 21:29:56 -04:00

eoc-meter: this process's own CPU energy is a live counter, and detect() never selects it ... 2ea6efe95d

On macOS 27, `proc_pid_rusage(getpid(), RUSAGE_INFO_V6)` (libproc: public,
no privilege, no entitlement) returns `ri_energy_nj`, the cumulative CPU
energy the kernel attributes to this process's threads. Unlike IOReport's CPU
rails, it is live for an unprivileged reader: it advanced in 20 of 20
intervals here, under 25 ms of CPU each. It is the first live CPU energy
counter eoc-meter has on a Mac.

Its domain is the trap, and the design follows from it: this process's
threads' CPU energy, all of them, and nothing else. Not a child process (a
child's energy is its own, and is not added to the parent's when reaped), not
the GPU, the Neural Engine or DRAM. eoc's counter brackets stages that may run
in a subprocess (eoc-tools' shell tool, HumanEval's python) or on the GPU
(mlx, mlc, llama.cpp's default offload). A span over such work would read only
this process's share and be tagged measured: a figure for work the counter did
not see. So:

- `MacosProcessEnergyCounter` (`JouleCounter`, cumulative µJ =
  `ri_energy_nj` / 1000, name `process-energy`) is constructed only where it
  is live. Its doc states the domain exactly.
- `process_energy_status()` probes it once per process and caches the answer,
  by the same sustained-advance rule and `probe_counter` the RAPL and NVML
  rungs use, burning this thread's CPU between samples (that burn is this
  process's work, so a live counter advances in every interval). It is
  `Unavailable` off macOS.
- `detect()` does **not** select it, under any status of any rung. A caller
  whose metered work runs on the CPU in this process opts in with
  `detect_in_process()` (this counter where live, otherwise `detect()`) or
  `MacosProcessEnergyCounter::try_new()`.
- A span the counter did not advance over is still no measurement; the
  consumers enforce that (eoc-cascade's `live_span`, eoc-streaming, eoc-local).

The FFI is hand-rolled: `struct rusage_info_v6` field for field from the
macOS 27 SDK, with compile-time asserts that it is 464 bytes and
`ri_energy_nj` is at byte 336, as a C program compiled against the SDK prints.
It is the crate's third scoped `allow(unsafe_code)` module.

Tests. In the library, on scripted reads: the probe calls a counter that
advances every read live; a constant, one step, or a failing
`proc_pid_rusage` is declined. Only `Live` constructs the counter, which then
reads this pid. A pid that does not exist reads nothing. `detect_with` never
returns the process counter for any of the 36 combinations of NVML, RAPL and
IOReport status, and `detect()` never does here; `detect_in_process` picks it
exactly for `Live`, on macOS.

Against the hardware, in an integration test binary of its own (`tests/
process_energy.rs`), because the counter's domain is the whole process and
other tests in the library binary burn CPU on other threads; its tests take
one lock. Each figure is checked against an oracle the test reads itself:
`proc_pid_rusage` into a raw buffer at the SDK's offsets. No test can pass
without asserting.

- live: the status agrees with the oracle (live if it advanced in 80% of 20
  intervals), and on Apple Silicon both must say live;
  `detect_in_process()` picks it; `detect()` does not.
- 200 ms of one thread's CPU reads 0.2-20 J, inside the oracle's bracket.
  The burn is 200 ms of the thread's own CPU, by its CPU clock, at
  user-interactive QoS. A 200 ms wall-time spin failed on this M5 Max under a
  load average near 70 ("174844 µJ for 200 ms of one thread's CPU work": it
  got 90 ms of CPU). And the bound is for a performance core: 200 ms of CPU on
  an M3 Ultra efficiency core (background QoS) reads 0.013 J, against 0.72 J
  at default QoS and 1.10 J at user-interactive (a C probe, not this test);
- 500 ms asleep reads under 0.1 J;
- a child process (`yes` to /dev/null, run at least 300 ms and until its own
  counter reads 0.5 J) is not attributed: the parent's counter, read before
  the spawn and after the reap, moves under 0.1 J and under a fifth of the
  child's own, while the reaped child's CPU time shows it worked.

Measured, verbatim from the test logs:

M5 Max (macOS 27.0.1, load average ~50):
  process-energy oracle: 20 of 20 intervals advanced
  status=Live oracle_live=true detect_in_process()=process-energy detect()=powermetrics
  one-thread burn of 200.001792ms thread CPU in 205.349583ms wall read 600514 µJ = 0.601 J (3.00 J per CPU-second)
  500 ms asleep read 512 µJ = 0.0005 J
  child ran 315.771709ms, its own counter read Some(700069134) nJ, its CPU (reaped) 289.941208ms; the parent's counter moved 1708 µJ = 0.0017 J
  (three runs before it: burns of 0.483, 0.444, 0.408 J; asleep 4894, 2735, 47 µJ; the parent moved 14311, 1404, 1199 µJ while a child read 0.54-0.61 J)

M3 Ultra (macOS 27.0.1, test binaries built for apple-m3, copied, run, deleted):
  process-energy oracle: 20 of 20 intervals advanced
  status=Live oracle_live=true detect_in_process()=process-energy detect()=powermetrics
  one-thread burn of 200.003584ms thread CPU in 200.008542ms wall read 1329909 µJ = 1.330 J (6.65 J per CPU-second)
  500 ms asleep read 184 µJ = 0.0002 J
  child ran 356.917167ms, its own counter read Some(1725566789) nJ, its CPU (reaped) 356.596542ms; the parent's counter moved 187 µJ = 0.0002 J

So the counter is live on both chips. The first M3 Ultra run failed
`a_child_processs_work_is_not_attributed` with "the child did not work:
120.042625ms of CPU": the child was stopped as soon as its counter read 0.5 J,
which the M3 Ultra's `yes` reached in 120 ms (0.716 J), under the 200 ms CPU
floor the test then demanded. A flaw in the test, not the counter; the parent
had moved 843 µJ. The child now runs at least 300 ms, and the floor is 150 ms.
The library's 62 tests passed there too.

Falsified, each mutant restored byte-for-byte (sha256 checked), run against
the library and the hardware binary:
- the process counter as a `detect()` rung fails 3 tests (`detect_with(
  Unavailable, Unavailable, Unavailable)` picks it; `detect()` picks it);
- a constant reading (counter only) fails the burn test ("0 µJ for 200 ms of
  one thread's CPU work"); a constant at the source makes the probe decline,
  and fails 4;
- reading the parent's pid fails 2 (the burn is not the parent's); probing
  the parent's pid fails 4;
- the constructor ignoring its status fails 2.

clippy raises no new warning. `cargo check --tests` for
x86_64-unknown-linux-gnu, with and without `cuda`, is clean.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

eoc: the CLI's cascade is metered by this process's own counter, because all of its stages run in-process ... d1e02cfaf1

`detect_in_process()` is for callers whose metered work runs on the CPU in
this process, and only for them. Each place eoc picks a counter was checked
against that:

- The `eoc` CLI's query cascade: adopted. Every stage it builds runs on the
  CPU in this process: the LRU cache, the in-memory KV store, the in-memory
  graph, and the echo backend, which formats a string. None spawns a
  subprocess or offloads to the GPU, and they run on this process's tokio
  workers. `build_cascade_with` now meters with `detect_in_process()`, and
  its doc says a backend passed to it must keep to the same.
- `eoc meter` still reports `detect()`'s counter and its idle draw, and now
  also names the counter `eoc query` meters with.
- Not adopted anywhere else, because no other eoc code picks a counter:
  eoc-cascade and eoc-agent default to the stub and take the caller's;
  eoc-streaming and eoc-local's backends take the caller's. eoc-agent must not
  be given this one in any case: its tools include eoc-tools' shell, a
  subprocess.
- eoc-local's crate doc now says which backends this counter covers. All four
  run in this process, but `mlx` (GPU, Neural Engine), `mlc` (GPU),
  `llamacpp` (GPU by default, `n_gpu_layers` 999) and `onnx` with a CoreML,
  CUDA, DirectML or TensorRT provider offload work it does not see. It covers
  `onnx` with the CPU provider and `llamacpp` with `n_gpu_layers` 0.

What the CLI reports on this M5 Max:

  counter: powermetrics
  query  : metered by process-energy (its stages all run on the CPU in this process)
  eoc query "what is 2+2"           -> stage kv, cost 50 µJ (estimated)
  eoc query "tell me something new" -> stage neural, cost 50000000 µJ (estimated)

Both lookups finish inside one update of the counter, so neither span
advanced and each stage's own figure stands, as eoc-cascade requires.

Tests. `work_on_an_unmetered_host_keeps_its_estimate` now expects the CLI's
meter to be `process-energy` exactly when it is live (and IOReport's rule
otherwise); its ~100 ms burn with a 50 J estimate costs the estimate plus the
reading, tagged estimated: "meter=process-energy cost=50230593 µJ
(estimated)" in one run. A new test, `in_process_work_is_measured_by_the_process_counter`,
runs ~100 ms of CPU work through the CLI cascade from a backend that claims
nothing (an estimated zero): it reads as measured, between 1 mJ and 10 J. The
floor is low on purpose, since a wall-time burn on a loaded host or an
efficiency core can draw little; the size of a reading is pinned in
eoc-meter's hardware test. Here: "eoc in-process cost: 273344 µJ (measured)"
and "meter=process-energy cost=50270761 µJ (estimated)". The two burning tests
take one lock, since each would otherwise read the other's burn: run in
parallel they read 380637 µJ each.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

dcharlot force-pushed eoc-meter/process-energy from d1e02cfaf1 to 974d929a8e 2026-09-30 21:48:27 -04:00 Compare

dcharlot force-pushed eoc-meter/process-energy from 974d929a8e to 511a180df1 2026-09-30 22:14:30 -04:00 Compare

dcharlot force-pushed eoc-meter/process-energy from 511a180df1 to 3513b9cc5d 2026-09-30 22:41:14 -04:00 Compare

dcharlot added 1 commit 2026-09-30 22:49:59 -04:00

eoc-meter: the one-thread burn test's floor matches what 200 ms of CPU can cost ... f4847eafce

The process-energy burn test asked for user-interactive QoS so its burn
would run on the fast cores, and it asserted at least 0.2 J. The QoS call
returns 0 but changes nothing: the kernel caps a process with no app role,
such as a test binary, at default QoS. rusage books all of the burn as
default QoS (measured in jcp-energy's matching test, 22e5a6ad).

The burn already counts 200 ms of the thread's own CPU time, not wall time,
but the energy those 200 ms cost still depends on which core class ran
them, and at what clock. On an M5 Max at default QoS it read 0.18-1.24 J,
so a 0.2 J floor sits at the bottom of the real range.

This drops the ineffective QoS call and its declarations, records why in
the test's doc, and lowers the floor to 0.1 J: half the lowest reading,
and the sleeping test's ceiling. A counter that reads wall time at a fixed
power still cannot pass both tests, and a counter reading 0 still fails.

On this M5 Max, 4/4 pass: the burn read 0.504 J, and 500 ms asleep 21 µJ.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

dcharlot referenced this pull request 2026-10-01 01:21:07 -04:00

eoc-agent: a local model's measured step is a measured span #127

dcharlot changed target branch from eoc-meter/cpu-rail-liveness to main 2026-10-01 06:21:22 -04:00

dcharlot merged commit 2dafc2ce5a into main 2026-10-01 06:22:08 -04:00

dcharlot referenced this pull request from a commit 2026-10-01 06:22:09 -04:00

Merge pull request 'eoc-meter: a live per-process CPU energy counter for in-process cascades' (#124) from eoc-meter/process-energy into main

Sign in to join this conversation.

Reviewers

No reviewers

Labels

Clear labels

No items

No labels

Milestone

Clear milestone

No items

No milestone

Projects

Clear projects

No items

No project

Assignees

Clear assignees

No assignees

1 participant

Notifications

Subscribe

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference

Transaction-Science/open-standards!124

No description provided.