722d80d9 listed two defects here as not done.
`llamacpp` took a failed first counter read as 0 (`unwrap_or(0)`), so the next
successful read reported the counter's whole since-boot total as the
generation's energy. Its drain then tagged that figure measured unless some
chunk had seen a zero delta. This is the defect 467acb38 fixed in
eoc-streaming. And `mlc` and `onnx` reported 0 µJ for inference that ran
whenever nothing was measured: tagged estimated, but still a zero presented as
a figure.
The rule now lives once, in a new `span` module, and every backend meters
through it:
- `MeterSpan::open` reads the counter; `reading` gives `Some(delta)` only when
that opening read succeeded, the read now succeeds, and the counter advanced
between them. A failed read at either end, a counter that held, one that
went down, or the stub: `None`, never 0 and never a since-boot total.
- `span_cost` makes a reading `JouleCost::measured` and asks for the estimate
only when there was none.
- `per_token_estimate` is tokens × `fallback_joules_per_token`, to the nearest
µJ, and never 0: at least one token is counted, a coefficient that is not a
finite positive number gives way to the default (0.05 J/token, mlx's figure
since the start), and a product under half a µJ is 1 µJ.
Per backend:
- `llamacpp`: `TokenChunk::cumulative_microjoules` is now `Option<u64>`, the
reading so far, `None` when nothing was measured. The inference's cost is
the final chunk's reading, measured, or the per-token estimate over the
tokens streamed. An earlier chunk's reading covers part of the generation,
so it is never taken as the cost of the whole. The backend gains
`fallback_joules_per_token` and its builder. Nothing outside this file uses
`TokenChunk`.
- `mlc` and `onnx` gain `fallback_joules_per_token` (default 0.05 J) and its
builder, and estimate over the placeholder's own token count, as `mlx` does.
- `mlx` already had the right rule; it now goes through the same functions, so
a zero coefficient no longer yields 0 µJ.
Factored so it tests without native code. Neither the `llamacpp` nor the `mlx`
module binds a single symbol of its native crate (both generators are
placeholders), so this crate's tests compile them without their features
(`cfg(any(feature, test))`; mlx on macOS). Their metering now runs in every
`cargo test --workspace`, where before it ran in none: llama.cpp is not built
here, and `--features mlx` fails in mlx-sys's Metal kernel build on this
toolchain. The `not(feature = "llamacpp")` error stub, unreachable since the
module itself was gated on the feature, is gone.
Tests, all on a scripted counter that reads a list (`None` is a failed read)
and counts its reads:
- span: an advance of 450 µJ reads 450; a failed first read followed by
9,000,000 reads nothing; so does a failed last read, a held counter, a
counter that went down, and the stub. The estimate is 800,000 µJ for 8 × 0.1
J, 100,000 for 0 tokens at 0.1 J, the default for 0, -1, NaN and infinity,
1 µJ for 1e-12 J, and saturates at u64::MAX.
- llamacpp, through `stream` and `infer`: the stub gives the 4-token estimate
(200,000 µJ, estimated) and no chunk a reading; a failed first read then 9 J
gives no chunk a reading and the estimate; 1,000 -> 1,100 ... 1,450 gives
chunk readings 100, 200, 300, 400 and a measured 450 over exactly six reads;
a held counter and a failed last read give the estimate. A table test pins
that only the final chunk's reading is a cost.
- mlc, onnx, mlx: measured 450 for an advance; the estimate (800,000 µJ) for a
held counter, a failed first read and a failed last read; the stub's
estimate, never 0. The `joule_attribution` integration test now asserts the
mlc stub's estimate (500,000 µJ, estimated) where it asserted 0.
Falsified, each mutant applied to the uncommitted tree, run, and restored
byte-for-byte (sha256 checked):
- a failed start taken as 0 fails 5 tests (llamacpp, mlc, mlx, onnx, span),
the llamacpp one with every chunk at `Some(9000000)`;
- `>=` in place of `>` (a held counter measured) fails 9;
- llamacpp reading the counter's absolute value per chunk (HEAD's rule with a
0 baseline) fails 3;
- a partial chunk reading kept as the cost fails the tally test;
- mlc's and onnx's HEAD estimate of 0 fail 2 each;
- the estimate without its 1 µJ floor, without the one-token minimum, or
without the coefficient check fails the estimate test.
One mutant survived and is equivalent: `final_reading = reading.or(previous)`
cannot differ, since nothing sets the field before the final chunk.
`cargo test -p eoc-local` passes (36 + 1), with `--features mlc,gguf` (47 +
3) and `--features onnx --lib` (43). The onnx leg runs `--lib` because the
`joule_attribution` test binary cannot link ort-sys without ONNX Runtime
installed when mlc is also enabled; that is unchanged. No clippy warning in
this crate, before or after.
Not done: the `llamacpp` error path still returns 0 µJ, estimated, with the
error text. Whether any work ran before an error is unknown there.
Co-Authored-By: Claude Opus 5.5 <[email protected]>
15cf07c2ee