jouleclaw-energy: per-process GPU energy on Apple Silicon; a shared GPU is never split by time #146

Merged

dcharlot merged 14 commits from jouleclaw-energy/apple-gpu-share into main 2026-10-05 20:52:16 -04:00

Conversation 0 Commits 14 Files changed 19 +2710 -57

dcharlot commented 2026-10-04 16:22:11 -04:00

Owner

Copy link

Per-process GPU energy on Apple Silicon, without root, alongside the per-process CPU counter, so a Metal inference span is metered on both.

What the hardware offers (M5 Max, macOS 27.0.1, unprivileged; measured)

• GPU rail. IOReport Energy Model / GPU Energy (nJ) is live; every mJ channel (CPU Energy, GPU0, ANE0, DRAM0, …) is frozen for an unprivileged reader. The rail republishes in steps, sometimes holding 300–600 ms under load, so a snapshot reads it on its next update.
• Per-process GPU time. Each GPU client in the I/O Registry's IOAccelerator subtree carries IOUserClientCreator (its pid) and AppUsage.accumulatedGPUTime (ns). This is read through IOKit, unprivileged.
• Not usable.
  • task_power_info_v2.gpu_energy.task_gpu_utilisation is always 0.
  • task_energy equals the CPU counter ri_energy_nj.
  • The Neural Engine has no unprivileged live source: ANE0 is frozen, and ANE work runs in aned, whose clients carry no statistics.

The rule

• This process used no GPU time: its GPU energy is exactly 0. That is graded HwShunt and needs no rail, so other clients can't disturb it, and CPU-only work keeps its CPU grade.

• It had ≥99% of all clients' GPU time: it was effectively the GPU's sole user, and its GPU figure is the rail's reading over the span, graded HwShunt. As with RAPL's package figure, idle and static draw are included.

• Otherwise, no GPU figure (GpuShared). Time share is not energy share:

  • a GGUF decode drew ~1.9 W per GPU-second, beside a job drawing ~17 W;
  • apportioned figures came out 3–15× off;
  • the decode's own GPU time inflated 5.4× under contention (9.81 s vs 1.83 s for the same 63 tokens).

  SPEC §1.3 now says the rail MUST NOT be split by GPU time.

• A client that vanishes mid-span (its process exited) makes every share unknown, so there is no GPU figure.

• A span (ProcessSpanMeter) sums CPU and GPU at the weaker grade, as Whole, or Partial if the caller declares ANE use. When the GPU has no figure, the span is the CPU figure alone: HwShunt, Partial, with the reason recorded (SpanSource::CpuOnly). The host meter is the fallback only when the CPU counter itself gives none.

• Liveness: the family rule (≥5 advances, ≥80% of intervals, a window of ≥300 ms) over the GPU rail and this process's own GPU time, while the caller's closure loads the GPU. It is probed once per process. A CPU-only load returns Frozen.

Wiring

• backend-apple: gpu_liveness_load(), a small Metal dispatch for the probe (objc2-metal, as the fabric uses).
• loader-gguf fabric-meter: the live tune falls back to process CPU+GPU spans when IOReport is declined, as it always is on macOS 27. Before this it returned None. Only Whole spans compete; a Partial span cancels the tune.
• harness-econ: span_joules bills only a Whole figure.
• mesh-meter: a process's GPU domain never covers a whole request.
• runtime: a new gguf_energy example prints CPU and GPU joules per token, with grade, coverage and share.

Measured

• Hardware test, sole user: a 1.5 s burn read 39.860 J, with 1603.6 of 1603.6 ms of GPU time. The test's own IOReport read and ioreg gave the same numbers.
• Hardware test, beside a burning child: GpuShared, so the span is CPU only and Partial. CPU-only work stays Whole with a GPU figure of exactly 0. With a child exiting mid-span, the span is CPU only and Partial, and records why.
• TinyLlama 1.1B Q8_0 decode, 63 tokens, on the GPU (Q8_PACKED=1):

  Condition	J/token	CPU	GPU	Grade	Coverage	GPU share	Rail
  Sole user	0.128	0.069	0.060	HwShunt	Whole	99.67%	3.77 J
  Beside a GPU burner	0.053	0.053	none (GPU shared)	HwShunt	Partial	n/a	n/a

Verified

• cargo test -p jouleclaw-energy --features ioreport passes all 121 tests, including the GPU hardware test, on this Mac.
• cargo check --workspace --all-targets passes for jouleclaw-rs. The harness-econ, loader-gguf (fabric-meter) and mesh-meter tests pass.
• The energy crate builds for Linux and Windows, where the counter is unavailable.
• Mutations, all caught: grade always HwShunt; vanished client ignored; liveness gate admitting anything; the probe dropping this process's own GPU time; probes run around the cache; the exact-zero return deleted; the zero waiting on other clients; the GpuShared check removed; CPU-only replaced by the fallback meter.

🤖 Generated with Claude Code

Per-process **GPU** energy on Apple Silicon, without root, alongside the per-process CPU counter, so a Metal inference span is metered on both. ## What the hardware offers (M5 Max, macOS 27.0.1, unprivileged; measured) - **GPU rail.** IOReport `Energy Model` / `GPU Energy` (nJ) is live; every mJ channel (`CPU Energy`, `GPU0`, `ANE0`, `DRAM0`, …) is frozen for an unprivileged reader. The rail republishes in steps, sometimes holding 300–600 ms under load, so a snapshot reads it on its next update. - **Per-process GPU time.** Each GPU client in the I/O Registry's IOAccelerator subtree carries `IOUserClientCreator` (its pid) and `AppUsage.accumulatedGPUTime` (ns). This is read through IOKit, unprivileged. - **Not usable.** - `task_power_info_v2.gpu_energy.task_gpu_utilisation` is always 0. - `task_energy` equals the CPU counter `ri_energy_nj`. - The Neural Engine has no unprivileged live source: `ANE0` is frozen, and ANE work runs in `aned`, whose clients carry no statistics. ## The rule - **This process used no GPU time:** its GPU energy is **exactly 0**. That is graded HwShunt and needs no rail, so other clients can't disturb it, and CPU-only work keeps its CPU grade. - **It had ≥99% of all clients' GPU time:** it was effectively the GPU's sole user, and its GPU figure is the rail's reading over the span, graded **HwShunt**. As with RAPL's package figure, idle and static draw are included. - **Otherwise, no GPU figure (`GpuShared`).** Time share is not energy share: - a GGUF decode drew ~1.9 W per GPU-second, beside a job drawing ~17 W; - apportioned figures came out 3–15× off; - the decode's own GPU time inflated 5.4× under contention (9.81 s vs 1.83 s for the same 63 tokens). SPEC §1.3 now says the rail MUST NOT be split by GPU time. - **A client that vanishes mid-span** (its process exited) makes every share unknown, so there is no GPU figure. - **A span** (`ProcessSpanMeter`) sums CPU and GPU at the weaker grade, as **Whole**, or **Partial** if the caller declares ANE use. When the GPU has no figure, the span is **the CPU figure alone: HwShunt, Partial**, with the reason recorded (`SpanSource::CpuOnly`). The host meter is the fallback only when the CPU counter itself gives none. - **Liveness:** the family rule (≥5 advances, ≥80% of intervals, a window of ≥300 ms) over the GPU rail and this process's own GPU time, while the caller's closure loads **the GPU**. It is probed once per process. A CPU-only load returns Frozen. ## Wiring - **backend-apple:** `gpu_liveness_load()`, a small Metal dispatch for the probe (`objc2-metal`, as the fabric uses). - **loader-gguf `fabric-meter`:** the live tune falls back to process CPU+GPU spans when IOReport is declined, as it always is on macOS 27. Before this it returned `None`. Only Whole spans compete; a Partial span cancels the tune. - **harness-econ:** `span_joules` bills only a Whole figure. - **mesh-meter:** a process's GPU domain never covers a whole request. - **runtime:** a new `gguf_energy` example prints CPU and GPU joules per token, with grade, coverage and share. ## Measured - **Hardware test, sole user:** a 1.5 s burn read 39.860 J, with 1603.6 of 1603.6 ms of GPU time. The test's own IOReport read and `ioreg` gave the same numbers. - **Hardware test, beside a burning child:** `GpuShared`, so the span is CPU only and Partial. CPU-only work stays Whole with a GPU figure of exactly 0. With a child exiting mid-span, the span is CPU only and Partial, and records why. - **TinyLlama 1.1B Q8_0 decode, 63 tokens, on the GPU (`Q8_PACKED=1`):** | Condition | J/token | CPU | GPU | Grade | Coverage | GPU share | Rail | |---|---|---|---|---|---|---|---| | Sole user | **0.128** | 0.069 | 0.060 | HwShunt | Whole | 99.67% | 3.77 J | | Beside a GPU burner | 0.053 | 0.053 | none (GPU shared) | HwShunt | Partial | n/a | n/a | ## Verified - `cargo test -p jouleclaw-energy --features ioreport` passes all 121 tests, including the GPU hardware test, on this Mac. - `cargo check --workspace --all-targets` passes for jouleclaw-rs. The harness-econ, loader-gguf (`fabric-meter`) and mesh-meter tests pass. - The energy crate builds for Linux and Windows, where the counter is unavailable. - **Mutations, all caught:** grade always HwShunt; vanished client ignored; liveness gate admitting anything; the probe dropping this process's own GPU time; probes run around the cache; the exact-zero return deleted; the zero waiting on other clients; the `GpuShared` check removed; CPU-only replaced by the fallback meter. 🤖 Generated with [Claude Code](https://claude.com/claude-code)

dcharlot added 13 commits 2026-10-04 16:22:11 -04:00

jouleclaw-energy: per-process GPU energy on Apple Silicon (WIP: counter, apportionment, span meter) ... 5d12f05bb3

The IOReport GPU rail (Energy Model / GPU Energy, nJ, live unprivileged)
apportioned by this process's share of every IOAccelerator client's
accumulatedGPUTime (I/O Registry). HwShunt only as the sole user (>=99%),
ModelBased otherwise; a vanished client leaves no figure. ProcessSpanMeter
combines it with the process-cpu counter, falling back to the host meter.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

jouleclaw-energy: read the GPU rail on a republish; hardware test against ioreg + IOReport oracles ... 8a936666a6

The GPU rail at times holds for 300-600 ms and catches up in one step
(7 of 37 ~315 ms windows read zero on a loaded M5 Max), so each snapshot
reads it on its next change and the clients right after; a snapshot whose
rail held for 1 s bounds no figure.

tests/process_gpu.rs: liveness once per process (and Frozen under a CPU
load), a GPU burn checked against the test's own IOReport read and ioreg's
text, a zero-GPU span, a contending child process, and a child exiting
inside a span (no GPU figure; the span meter falls back and says why).

Co-Authored-By: Claude Opus 5.5 <[email protected]>

backend-apple: gpu_liveness_load; runtime: gguf_energy example (CPU+GPU joules per token) ... 36973aeb77

jouleclaw-energy holds no Metal, so the Metal owner supplies the GPU
counter's liveness load: ~25 ms of a small FMA kernel, waited on.

gguf_energy meters a GGUF prefill and decode with ProcessSpanMeter and
prints joules per token, CPU and GPU apart, with the GPU figure's grade
and this process's share of the GPU's time.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

Wire the per-process GPU counter: fabric live tune, mesh-meter; idle-aware rail reads ... ef5e2c9b51

- jouleclaw-energy: a snapshot takes a held GPU rail as current once no
  client has accrued GPU time for 100 ms (a quiet GPU may not republish
  for over a second); a rail held for 1 s while the GPU is busy is
  EnergyError::NotCurrent, no figure.
- loader-gguf: tune_shape_live meters each backend on this process's CPU
  and GPU counters where the IOReport counter is declined (macOS 27.0.1),
  recorded as process-cpu+gpu at the weakest span's grade.
- mesh-meter: ProcessGpu does not cover a whole request.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

harness-econ: span_joules bills a ProcessSpanMeter figure only when it covers the whole dispatch ... 81af1452a6

Co-Authored-By: Claude Opus 5.5 <[email protected]>

jouleclaw-energy: document the per-process sources; keep GPU snapshot overhead out of a span's CPU window ... 83be3adbff

MEASURED_ENERGY.md and SPEC.md list the per-process CPU and GPU counters,
their grades and when they give no figure.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

jouleclaw-energy: module docs resolve in their own scope; IOKit linked once ... 070c82232b

Co-Authored-By: Claude Opus 5.5 <[email protected]>

jouleclaw-energy: record the GGUF decode measurement (sole user HwShunt, contended ModelBased) ... cb24ea769c

tinyllama-1.1b Q8_0, Q8_PACKED=1, 63 decode tokens on an M5 Max:
sole GPU user 0.130 J/token (CPU 0.076 + GPU 0.054, HwShunt, 100.00%);
beside another GPU job 0.222 J/token (GPU 0.171, ModelBased, 11.5%).
On a quiet GPU the rail did not change in 4 s at idle.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

jouleclaw-energy test: a process's GPU time runs ~0.6 ms a command buffer past the buffers' own GPU spans ... bf5159721a

Seen as the sole user: 1618.3 ms of accumulatedGPUTime for 1500.9 ms of
GPUStartTime..GPUEndTime over ~190 buffers. The ceiling allows 2 ms a buffer.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

jouleclaw-energy: no time-share GPU figure; an exact zero; the CPU alone, Partial ... cfe38e0220

The time-share model is falsified by this branch's own measurements (a
GGUF decode at ~1.9 W per GPU-second beside a ~17 W job; contended
figures 3-15x the sole-user figure), so it is no longer a figure:

- apportion(): this process's own GPU time first, from its own clients
  (one gone: GpuClientVanished). None spent: 0 uJ, exact, HwShunt, with
  no rail, currency or other-client check. Spent, but under 99% of every
  client's time: EnergyError::GpuShared { self_ns, all_ns }. ModelBased
  is no longer a GPU grade; GpuShare.provenance is now a method
  (HwShunt), rail_uj / all_ns are None for an exact zero.
- combine(): CPU figure and no GPU figure is SpanSource::CpuOnly, the CPU
  alone, HwShunt, Partial, with why; the fallback meter only when the CPU
  counter gives none.
- fabric live tune: only spans metered Whole compete; a Partial one voids
  the tune. harness-econ refuses CpuOnly as a dispatch total.
- gguf_energy prints the CPU-only line; the hardware test expects
  GpuShared and CpuOnly under contention and exact zeros beside a
  burning child and across its exit.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

jouleclaw-energy: MEASURED_ENERGY and SPEC: no split of a shared GPU rail; re-measured ... a75a801039

tinyllama-1.1b Q8_0, Q8_PACKED=1, 63 decode tokens, M5 Max:
sole GPU user 0.128 J/token = CPU 0.069 + GPU 0.060, HwShunt, Whole
(99.67% of the GPU time); beside a compute burner, CPU alone 0.053
J/token, HwShunt, Partial (GPU shared: 9.81 s of 13.98 s).

Co-Authored-By: Claude Opus 5.5 <[email protected]>

jouleclaw-energy test: the oracle ignores a client that came and went inside the counter's span ... 738299a3d9

A host process closing a GPU client inside the oracle's window failed the
self-zero check beside the child, which needs no other client. The oracle
now sums the clients it saw at both reads (and new ones), and checks that
any it saw vanish was absent from the counter's start snapshot.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

jouleclaw-energy test: pass the CPU tick directly ... a408d9712b

Co-Authored-By: Claude Opus 5.5 <[email protected]>

dcharlot referenced this pull request from a commit 2026-10-05 09:09:47 -04:00

jouleclaw-energy test: two dies' steps fall within the probe's guaranteed samples

dcharlot added 1 commit 2026-10-05 09:09:47 -04:00

jouleclaw-energy test: two dies' steps fall within the probe's guaranteed samples ... d5a952e794

the_production_status_of_two_dies_stepping_apart... put DIE_1's step at
sample 6 and required 8 samples. The probe stops once its budget is spent
and it has five intervals (six samples). On the loaded macOS CI runner
its ticks ran slowly, it stopped at six samples, and the test failed
before the second step was ever followed (#146's macOS energy job). It
passed 5 of 5 runs locally.

The steps are now at samples 2 and 4, both followed within the six
samples the probe always takes, and the test asserts at least six. It
still checks what it set out to check: two dies stepping in different
intervals are each held to the rule at their own value, and never
summed.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

dcharlot referenced this pull request 2026-10-05 09:16:51 -04:00

jcp-gateway: make enforcement discoverable, and refusable #157

dcharlot merged commit b83ebc8da9 into main 2026-10-05 20:52:16 -04:00

dcharlot referenced this pull request from a commit 2026-10-05 20:52:17 -04:00

Merge pull request 'jouleclaw-energy: per-process GPU energy on Apple Silicon; a shared GPU is never split by time' (#146) from jouleclaw-energy/apple-gpu-share into main

dcharlot referenced this pull request 2026-10-08 20:23:18 -04:00

mesh: an in-process engine is metered by this process's own CPU and GPU energy #193

Sign in to join this conversation.

Reviewers

No reviewers

Labels

Clear labels

No items

No labels

Milestone

Clear milestone

No items

No milestone

Projects

Clear projects

No items

No project

Assignees

Clear assignees

No assignees

1 participant

Notifications

Subscribe

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference

Transaction-Science/open-standards!146

No description provided.