mesh: an in-process engine is metered by this process's own CPU and GPU energy #193

Merged

dcharlot merged 1 commit from mesh-meter/process-span into main 2026-10-09 02:41:26 -04:00

Conversation 0 Commits 1 Files changed 12 +505 -10

2026-10-08

Transaction Science

mesh: an in-process engine is metered by this process's own CPU and GPU energy ...

Mesh's headline figure, microjoules per accepted token, is to be measured
at the meter. Yet every real serve through mesh was estimated: the GGUF
serving binaries were hardwired to EstimatorMeter, and HwMeter bridges
the host meter, which on macOS 27 is the estimator (IOReport's CPU rails
are frozen).

- mesh-meter `ProcessMeter`, behind `hw-meter`, for an engine that runs
  in this process. Each serve is metered by jouleclaw's ProcessSpanMeter:
  this process's own CPU energy plus its own GPU energy.
  - A span that covers the whole request is graded as its counters
    grade it: HwShunt for a live reading. Whole means the process was
    effectively the GPU's sole user and used no Neural Engine; a
    process with no GPU time spent exactly 0 J on it, so CPU-only
    engines qualify.
  - Any other span is an estimate of the whole request, Estimator: the
    reading plus EstimatorMeter for the rest of the host, as HwMeter
    bills a counter that reads part of a request. `mesh/1` has no
    coverage field.
- `Meter::open_span` and `Window`. The default is None, so existing
  meters keep snapshot semantics. ProcessMeter grades over the whole
  span, because its GPU grade depends on the process's share over it.
  Session uses a span window where a meter keeps one.
- mesh-engine-gguf, with `metal`: serve_falcon and serve_openai_falcon
  meter with ProcessMeter (the Metal liveness load comes from
  jouleclaw-backend-apple). New `serve_gemma_metered` serves Gemma 4
  through mesh with ProcessMeter.
- Fixes mesh-engine-gguf under `gguf`: it no longer compiled on main.
  ServeOutcome and ServeRequest gained a `decision` field and three
  initializers lacked it, unseen because CI never built the feature.
  standards-build now runs `cargo test -p mesh-engine-gguf --features
  gguf` on the mesh-rs leg.

Measured on an M5 Max:
- mesh-core tests/process_meter.rs: a ReferenceEngine serve sealed
  HwShunt 1,051,668 µJ (4,108 µJ per accepted token), within the
  1,090,946 µJ of this process's CPU energy the test read around the
  serve.
- serve_gemma_metered, Gemma 4 E2B on CPU, 3 x 16 tokens: HwShunt
  12.52 J, 12.16 J and 12.54 J (0.78, 0.76 and 0.78 J per accepted
  token). Every envelope verifies.

Mutation caught: billing a whole span's figure plus the rest-of-host
estimate fails the unit test and the end-to-end bracket ("the receipt
read 55691805 µJ, more than this process's 966177 µJ").

Co-Authored-By: Claude Opus 5.5 <[email protected]>

1fb670a512

• View at this point in history

2026-10-08 20:21:15 -04:00