mesh: an in-process engine is metered by this process's own CPU and GPU energy #193
Loading…
Reference in a new issue
No description provided.
Delete branch "mesh-meter/process-span"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Mesh's headline figure, microjoules per accepted token, is meant to be measured at the meter. Until now every real serve through mesh was estimated:
EstimatorMeter;HwMeterbridges the host meter, and on macOS 27 that is the estimator, because IOReport's CPU rails are frozen.This PR makes an in-process serve a measured one wherever the counters can back it.
ProcessMeter(mesh-meter, behindhw-meter)It is for an engine that runs in this process. Each serve is metered by jouleclaw's
ProcessSpanMeter(#146): this process's own CPU energy plus its own GPU energy.HwShuntfor a live reading). Whole means the process was effectively the GPU's sole user (≥99% of GPU time) and used no Neural Engine. A process with no GPU time spent exactly 0 J on it, so CPU-only engines qualify.Estimator, as an estimate of the whole request, becausemesh/1has no coverage field (SPEC §3.3). That covers a shared GPU, a GPU with no figure, and declared ANE use. The figure is the reading plusEstimatorMeterfor the rest of the host, the ruleHwMeteralready applies to a counter that reads part of a request.Meter::open_spanandWindow. The default isNone, so every existing meter keeps snapshot semantics.ProcessMetergrades over the whole span, because its GPU grade depends on the process's share across it.Sessionuses a span window where a meter keeps one; mesh-core's API is unchanged.Serving binaries (mesh-engine-gguf,
metal)serve_falconandserve_openai_falconmeter withProcessMeter. The Metal liveness load comes fromjouleclaw-backend-apple. Withoutmetalthey keep the estimator.serve_gemma_meteredserves Gemma 4 through mesh withProcessMeterand prints each sealed receipt.Build fix: mesh-engine-gguf under
ggufdid not compile onmainServeOutcomeandServeRequestgained adecisionfield. Three initializers in mesh-engine-gguf lacked it, and nothing noticed because CI never built that feature. They now setdecision: None, since a token-generating engine answers no typed decision. standards-build's mesh-rs leg now runscargo test -p mesh-engine-gguf --features gguf, through thefeature_testshook, so this cannot recur silently.Measured (M5 Max, macOS 27)
ReferenceEngine(mesh-coretests/process_meter.rs)serve_gemma_metered)The test's figure lies within the 1,090,946 µJ of this process's CPU energy the test read around the serve, through its own
proc_pid_rusageoracle. Every envelope verifies.Verified
hw-meter-apple: 17 tests passed.cargo test --workspace: 52 passed.cargo test -p mesh-engine-gguf --features gguf: passes.cargo check -p mesh-engine-gguf --features metal --bins: clean.🤖 Generated with Claude Code
Mesh's headline figure, microjoules per accepted token, is to be measured at the meter. Yet every real serve through mesh was estimated: the GGUF serving binaries were hardwired to EstimatorMeter, and HwMeter bridges the host meter, which on macOS 27 is the estimator (IOReport's CPU rails are frozen). - mesh-meter `ProcessMeter`, behind `hw-meter`, for an engine that runs in this process. Each serve is metered by jouleclaw's ProcessSpanMeter: this process's own CPU energy plus its own GPU energy. - A span that covers the whole request is graded as its counters grade it: HwShunt for a live reading. Whole means the process was effectively the GPU's sole user and used no Neural Engine; a process with no GPU time spent exactly 0 J on it, so CPU-only engines qualify. - Any other span is an estimate of the whole request, Estimator: the reading plus EstimatorMeter for the rest of the host, as HwMeter bills a counter that reads part of a request. `mesh/1` has no coverage field. - `Meter::open_span` and `Window`. The default is None, so existing meters keep snapshot semantics. ProcessMeter grades over the whole span, because its GPU grade depends on the process's share over it. Session uses a span window where a meter keeps one. - mesh-engine-gguf, with `metal`: serve_falcon and serve_openai_falcon meter with ProcessMeter (the Metal liveness load comes from jouleclaw-backend-apple). New `serve_gemma_metered` serves Gemma 4 through mesh with ProcessMeter. - Fixes mesh-engine-gguf under `gguf`: it no longer compiled on main. ServeOutcome and ServeRequest gained a `decision` field and three initializers lacked it, unseen because CI never built the feature. standards-build now runs `cargo test -p mesh-engine-gguf --features gguf` on the mesh-rs leg. Measured on an M5 Max: - mesh-core tests/process_meter.rs: a ReferenceEngine serve sealed HwShunt 1,051,668 µJ (4,108 µJ per accepted token), within the 1,090,946 µJ of this process's CPU energy the test read around the serve. - serve_gemma_metered, Gemma 4 E2B on CPU, 3 x 16 tokens: HwShunt 12.52 J, 12.16 J and 12.54 J (0.78, 0.76 and 0.78 J per accepted token). Every envelope verifies. Mutation caught: billing a whole span's figure plus the rest-of-host estimate fails the unit test and the end-to-end bracket ("the receipt read 55691805 µJ, more than this process's 966177 µJ"). Co-Authored-By: Claude Opus 5.5 <[email protected]>