eoc-agent: a local model's measured step is a measured span #127

Merged

dcharlot merged 1 commit from eoc-agent/model-span-provenance into main 2026-10-01 09:50:27 -04:00

Conversation 0 Commits 1 Files changed 5 +244 -20

dcharlot commented 2026-10-01 01:21:07 -04:00

Owner

Copy link

Stacked on #124 (eoc per-process energy). Retarget to main once #124 merges.

What was wrong. LlmReply carried a bare microjoules, and AgentLoop::charge_llm_step tagged every model step Estimated. A backend that ran the model in this process and measured it had no way to say so, so its measured figure was recorded as an estimate. Such a backend is an eoc-local backend whose MeterSpan is backed by #124's per-process counter.

What changed (eoc-agent):

• LlmReply carries a JouleCost. LlmReply::new(text, tokens, µJ) is the provider's estimate, exactly as before. LlmReply::with_cost(text, tokens, cost) gives a figure with its grade.
• charge_llm_step records the provider's cost as graded, and charges the budget that figure.
• New NeuralProvider<B: NeuralBackend> wraps any in-process backend as an LlmProvider. The reply's energy is the backend's own JouleCost. A Response has no token count, so tokens are counted from the text: approx_tokens (one per four characters, rounded up), or the model's tokenizer through with_token_counter. Tokens feed only the token budget.
• A model served over the network still reports an estimate, because it ran on hardware no counter here reads. The trait doc says so.

Verified

• cargo test -p eoc-agent: 42 lib tests plus the integration tests pass.
• cargo check --workspace --all-targets passes for eoc-rs.
• Clippy and rustdoc report nothing new.
• An MlcBackend metered by an advancing counter gives a measured span of exactly what the counter read. A held, failed or stub counter gives the backend's estimate, never 0. A whole ReAct run records the model step measured, and the budget total is that figure.
• Falsified: reverting charge_llm_step to always-estimated fails both end-to-end tests.

🤖 Generated with Claude Code

Stacked on #124 (eoc per-process energy). Retarget to `main` once #124 merges. **What was wrong.** `LlmReply` carried a bare `microjoules`, and `AgentLoop::charge_llm_step` tagged every model step `Estimated`. A backend that ran the model in this process and measured it had no way to say so, so its measured figure was recorded as an estimate. Such a backend is an `eoc-local` backend whose `MeterSpan` is backed by #124's per-process counter. **What changed** (`eoc-agent`): - `LlmReply` carries a `JouleCost`. `LlmReply::new(text, tokens, µJ)` is the provider's estimate, exactly as before. `LlmReply::with_cost(text, tokens, cost)` gives a figure with its grade. - `charge_llm_step` records the provider's cost as graded, and charges the budget that figure. - New `NeuralProvider<B: NeuralBackend>` wraps any in-process backend as an `LlmProvider`. The reply's energy is the backend's own `JouleCost`. A `Response` has no token count, so tokens are counted from the text: `approx_tokens` (one per four characters, rounded up), or the model's tokenizer through `with_token_counter`. Tokens feed only the token budget. - A model served over the network still reports an estimate, because it ran on hardware no counter here reads. The trait doc says so. **Verified** - `cargo test -p eoc-agent`: 42 lib tests plus the integration tests pass. - `cargo check --workspace --all-targets` passes for eoc-rs. - Clippy and rustdoc report nothing new. - An `MlcBackend` metered by an advancing counter gives a measured span of exactly what the counter read. A held, failed or stub counter gives the backend's estimate, never 0. A whole ReAct run records the model step measured, and the budget total is that figure. - **Falsified:** reverting `charge_llm_step` to always-estimated fails both end-to-end tests. 🤖 Generated with [Claude Code](https://claude.com/claude-code)

dcharlot added 1 commit 2026-10-01 01:21:07 -04:00

eoc-agent: a local model's measured step is a measured span ... 8894a1a663

LlmReply carried a bare `microjoules`, and charge_llm_step tagged every
model step Estimated. A backend that ran a model in this process and read
it on a live counter (an eoc-local backend's MeterSpan, now backed by
the per-process counter) had no way to say so. Its measured figure was
recorded as an estimate.

- LlmReply carries a JouleCost. LlmReply::new is the provider's estimate,
  as before, and LlmReply::with_cost gives a figure with its grade.
- charge_llm_step records the provider's cost as graded and charges the
  budget that figure.
- NeuralProvider wraps any in-process NeuralBackend as an LlmProvider.
  The reply's energy is the backend's own JouleCost. Tokens are counted
  from the text (approx_tokens, or the model's tokenizer through
  with_token_counter) and feed only the token budget.

A model served over the network still reports an estimate: it ran on
hardware no counter here reads.

Tests: an MlcBackend metered by an advancing counter gives a measured
span, and a held, failed or stub counter gives the estimate. A whole
ReAct run records the model step measured. Reverting charge_llm_step to
always-estimated fails both end-to-end tests.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

dcharlot force-pushed eoc-agent/model-span-provenance from 8894a1a663 to 30370b1996 2026-10-01 01:29:14 -04:00 Compare

dcharlot changed target branch from eoc-meter/process-energy to main 2026-10-01 09:50:17 -04:00

dcharlot merged commit 89f1c9e7b0 into main 2026-10-01 09:50:27 -04:00

dcharlot referenced this pull request from a commit 2026-10-01 09:50:29 -04:00

Merge pull request 'eoc-agent: a local model's measured step is a measured span' (#127) from eoc-agent/model-span-provenance into main

Sign in to join this conversation.

Reviewers

No reviewers

Labels

Clear labels

No items

No labels

Milestone

Clear milestone

No items

No milestone

Projects

Clear projects

No items

No project

Assignees

Clear assignees

No assignees

1 participant

Notifications

Subscribe

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference

Transaction-Science/open-standards!127

No description provided.