agent-joule/crates/joule_coded/tests
Repository files (latest commit first)
Filename Latest commit message Latest commit date
David Charlot 6362476bc9 joule_coded: Measure the tokens the metered path was reporting as zero
Two defects in the run identity and the headline metric, both found by
running the new agent adapter and reading its output rather than trusting
it.

`RunIdentity::detect` finds the model through the environment, which is
how the daemon is configured. The `agent` subcommand takes `--model`
directly, so every run started that way reported `no model` while holding
a loaded one. An identity that is wrong exactly when a model was used is
worse than no identity: it would have labelled every benchmark row as
modelless. It now stamps the model it actually loaded.

The metered loop passed `None` for a generation step's token count, so
`summary.tokens` was 0, `microjoules_per_token` was `None`, and the JSON
reported `output_tokens: 0` — which reads as "generated nothing". Every
run down this path silently disabled joules-per-token, the metric this
project exists to report.

The counts were never missing, only discarded. `l3_generate_counted`
already returns them, and its own doc says it exists so that
µJ/accepted-token is "measured rather than left at zero or invented" —
the metered path just called `l3_generate` instead. It now takes a
generator returning `(text, tokens_out)` and records the count on the
step.

First measured figures from the agent path, on a 2.6B local model:
192 output tokens, 6.49 kJ, 111 s — 33.8 J/token at 1.73 tokens/sec. The
joules figure is now real. The token rate is roughly twenty times slower
than this model should run on this hardware, which is a separate thread
to pull.

Co-Authored-By: Claude Opus 5 <[email protected]>
2026-09-07 06:33:46 -04:00
..
acp.rs joule_coded: serve on-device inference as an OpenAI-compatible metered endpoint 2026-07-02 11:01:27 -04:00
consensus.rs joule_coded: BFT consensus rounds across the fleet 2026-06-17 01:31:58 -04:00
dap_step.rs joule_coded: Clear the last three lints blocking the clippy gate 2026-08-29 21:05:13 -04:00
debug_inspect.rs joule_coded: Clear the last three lints blocking the clippy gate 2026-08-29 21:05:13 -04:00
fanout.rs joule_coded: serve on-device inference as an OpenAI-compatible metered endpoint 2026-07-02 11:01:27 -04:00
headless_run.rs joule_coded: Measure the tokens the metered path was reporting as zero 2026-09-07 06:33:46 -04:00