`sci_from_eval_report` wrapped every eval report's energy in
`JouleCost::measured`, whatever produced it. An eval run against the echo
backend, a vendor API's token estimate or a stub-metered cascade came out as a
measured energy term. It could not do better: `EvalReport` dropped the
provenance of the costs it summed, and `SciScore` had nowhere to put one. The
end-to-end test `sci_for_ai` did the same in miniature. Its comment said
"pretend the meter reported 2000 J on the stub backend", and it built that
figure with `JouleCost::measured`.
What holds now:
- `EvalReport` gains `joule_source`. It is `Measured` only when every case's
cost was read from a live counter. One estimated case makes the total an
estimate, and so does a run with no cases, since nothing was read. Each case
is separate work that ran, so an unmeasured case is not covered by any other
reading. A report serialized before the field existed reads back as
`Estimated` (`#[serde(default)]`): nothing in it says it was measured.
- `SciScore` gains `energy_source`, the input `JouleCost`'s provenance
unchanged. It has the same serde default. The numbers do not change.
- `sci_from_eval_report` builds the energy term with the report's own
`joule_source`.
- `sci_for_ai` uses `JouleCost::estimated` for its fixed 2000 J. No counter
read that figure. It now asserts that the score says so.
Tests: a measured backend gives a measured report. An estimated one gives an
estimated report, and so does one that turns estimated from its third answer
on. An empty run is estimated. A report and a score without provenance read
back as estimated. A score carries its input's provenance, with the same SCI
either way. Under `eval-bridge`, a report's score keeps the report's
provenance.
Falsified, each mutant restored byte-for-byte:
- HEAD's `JouleCost::measured(total_microjoules)` in the bridge fails
`an_eval_reports_score_keeps_the_reports_provenance` (`left: Measured`,
`right: Estimated`). That test is behind the `eval-bridge` feature, which
nothing in the workspace enables, so a plain `cargo test --workspace` never
compiles it. It was run with `cargo test -p eoc-carbon --features
eval-bridge`, 39 + 8 passing.
- `SciScore` ignoring its input's provenance fails two tests, one in the
default build.
- the runner ignoring per-case provenance fails
`the_report_is_measured_only_when_every_case_was`.
- `sci_for_ai` tagging its figure measured again fails that test.
Not done: `CarbonAccount` still drops provenance (`account_request` is the
other public way in). And a report with zero correct answers has
`joules_per_correct == u64::MAX` as a sentinel, which `sci_from_eval_report`
multiplies by `correct.max(1)` into a u64::MAX µJ energy term.
Co-Authored-By: Claude Opus 5.5 <[email protected]>
0db9b8be02