All checks were successful
repository checks / lockfiles, tests in CI, workflow hygiene, spec and section references, public text (pull_request) Successful in 55s
standards workspaces build + test / which workspaces this change touches (pull_request) Successful in 17s
standards workspaces build + test / mesh-rs (mesh/mesh-rs, -p mesh-engine-gguf -p mesh-openai --features mesh-engine-gguf/gguf,mesh-openai/server, mesh-rs, linux-arm64) (pull_request) Successful in 1m12s
standards workspaces build + test / jouleclaw-rs (-nodebug, jouleclaw/jouleclaw-rs, -p jouleclaw-energy --features rapl,nvml, jouleclaw-rs, linux-arm64, -C target-feature=+fp16) (pull_request) Successful in 3m18s
standards workspaces build + test / joule-code-rs (joule-code/joule-code-rs, joule-code-rs, linux-arm64, true) (pull_request) Successful in 4m14s
standards workspaces build + test / jcp-rs (jcp-conformance, jcp/jcp-rs, -p jcp-energy --features nvml, jcp-rs, linux) (pull_request) Successful in 1m37s
standards workspaces build + test / macOS energy backends (pull_request) Successful in 7m41s
standards workspaces build + test / joulehook-rs (joulehook/joulehook-rs, joulehook-rs, linux) (pull_request) Successful in 1m10s
standards workspaces build + test / jouledesk-rs (jouledesk/jouledesk-rs, jouledesk-rs, macos) (pull_request) Successful in 2m50s
standards workspaces build + test / sandbox-rs (sandbox-conformance, sandbox/sandbox-rs, sandbox-rs, linux) (pull_request) Successful in 2m16s
standards workspaces build + test / joule-ui-rs (joule-ui/joule-ui-rs, joule-ui-rs, linux) (pull_request) Successful in 8m12s
`Efficiency::wire()` sealed `energy_uj` and `provenance` and nothing about
coverage. So a GPU-only reading (a hardware counter that reads the
accelerator and not the CPU work that drove it) reached a verifier as the
whole unit's energy. The receipt also sealed a `uj_per_accepted_token` over
that reading, which presented it as what the work cost per token. jcp SPEC
§5.2 already said a `Partial` figure MUST NOT be presented as a task total,
and listed the receipts that carry coverage. efficiency/1 was not among them.
`aggregate()` then summed energy across receipts with no coverage and no
provenance, and its per-unit figure came out as clean as a whole one. §5.3
says aggregation takes the floor of both and is never a step at which a
grade improves.
Now:
- The receipt carries `energy_coverage: "Partial"` when the figure is
partial, and omits it when the figure is whole. A whole receipt, and every
receipt sealed before the field existed, keeps its bytes.
- A partial receipt seals no `uj_per_accepted_token`. A partial figure
divided by tokens is not what the work cost per token, nor a bound on it,
since a partial figure can fall short of the work or exceed it.
- `read_payload`, split out of `verify` so the payload rules can be replayed
without a signature, refuses:
- a partial receipt carrying the headline;
- any `energy_coverage` but "Partial" ("Whole" included; absence is Whole);
- a provenance label that is not one of the three.
- `Totals` carries `provenance_floor` and `coverage`. A total over any partial
receipt is partial, gives no per-unit energy, and reports value per joule
as its worst value (0.0).
SPEC §5.2 now lists efficiency/1 with these rules. A published pack,
`conformance/v1/efficiency.json` (from `gen_efficiency_vectors`), carries the
readings and refusals and the totals sets must sum to. `tests/efficiency_pack.rs`
replays it from disk.
Tests: 3 new unit tests and 4 pack replays. All 9 mutations of the new rules
are killed. The pack alone catches the headline, coverage-floor and
provenance-floor mutations.
Co-Authored-By: Claude Opus 5.5 <[email protected]>
85 lines
3.4 KiB
Rust
85 lines
3.4 KiB
Rust
//! Replay the published `efficiency/1` payload pack from disk.
|
|
//!
|
|
//! jcp SPEC §5.2-5.3 states the rules and `gen_efficiency_vectors` emits the
|
|
//! vectors. This reads those bytes back, the same bytes a third party would,
|
|
//! and checks the reference reader reproduces every recorded verdict. The
|
|
//! refusals are the half that matters: a reader that accepts a partial figure
|
|
//! presented as a task total has failed in the direction that flatters.
|
|
|
|
use jcp_efficiency::{aggregate, read_payload};
|
|
use serde_json::{json, Value};
|
|
|
|
fn pack() -> Value {
|
|
let p = std::path::Path::new(env!("CARGO_MANIFEST_DIR")).join("../../../conformance/v1/efficiency.json");
|
|
// Assert, not skip: a `return` here would pass green in the one case this
|
|
// exists to catch, the pack going missing.
|
|
assert!(p.exists(), "conformance pack missing at {} — it is what this test replays", p.display());
|
|
let bytes = std::fs::read(&p).unwrap_or_else(|e| panic!("read {}: {e}", p.display()));
|
|
serde_json::from_slice(&bytes).expect("pack parses")
|
|
}
|
|
|
|
#[test]
|
|
fn the_pack_declares_its_profile_and_spec() {
|
|
let v = pack();
|
|
assert_eq!(v["profile"], "efficiency/1");
|
|
assert!(v["spec"].as_str().unwrap().contains("5.2"));
|
|
}
|
|
|
|
#[test]
|
|
fn every_positive_payload_reads_as_recorded() {
|
|
let v = pack();
|
|
let cases = v["payload"]["positive"].as_array().expect("positives");
|
|
assert!(cases.len() >= 2, "expected the published positive cases");
|
|
for c in cases {
|
|
let note = c["note"].as_str().unwrap_or("?");
|
|
let r = read_payload(&c["input"]).unwrap_or_else(|e| panic!("{note}: must read: {e}"));
|
|
let got = json!({
|
|
"energy_uj": r.energy_uj,
|
|
"provenance": r.provenance,
|
|
"energy_coverage": r.energy_coverage.label(),
|
|
"uj_per_accepted_token": r.uj_per_accepted_token,
|
|
});
|
|
assert_eq!(got, c["expected"], "{note}");
|
|
}
|
|
}
|
|
|
|
#[test]
|
|
fn every_negative_payload_is_refused_for_its_recorded_reason() {
|
|
let v = pack();
|
|
let cases = v["payload"]["negative"].as_array().expect("negatives");
|
|
assert!(cases.len() >= 3, "expected the published refusals");
|
|
for c in cases {
|
|
let note = c["note"].as_str().unwrap_or("?");
|
|
let because = c["expected"]["refused"].as_str().expect("a refusal reason");
|
|
match read_payload(&c["input"]) {
|
|
Ok(r) => panic!("{note}: accepted, as {r:?}"),
|
|
Err(e) => assert!(e.to_string().contains(because), "{note}: refused for '{e}', not '{because}'"),
|
|
}
|
|
}
|
|
}
|
|
|
|
#[test]
|
|
fn every_aggregate_sums_to_its_recorded_totals() {
|
|
let v = pack();
|
|
let cases = v["aggregate"]["cases"].as_array().expect("aggregate cases");
|
|
assert!(cases.len() >= 2);
|
|
for c in cases {
|
|
let note = c["note"].as_str().unwrap_or("?");
|
|
let rs: Vec<_> = c["input"]
|
|
.as_array()
|
|
.expect("payloads")
|
|
.iter()
|
|
.map(|p| read_payload(p).unwrap_or_else(|e| panic!("{note}: {e}")))
|
|
.collect();
|
|
let t = aggregate(&rs);
|
|
let got = json!({
|
|
"completed_uj": t.completed_uj,
|
|
"wasted_uj": t.wasted_uj,
|
|
"coverage": t.coverage.label(),
|
|
"provenance_floor": t.provenance_floor.map(|p| p.jcr1_label()),
|
|
"uj_per_completed_unit": t.uj_per_completed_unit(),
|
|
"tokens_per_completed_unit": t.tokens_per_completed_unit(),
|
|
});
|
|
assert_eq!(got, c["expected"], "{note}");
|
|
}
|
|
}
|