joule-code-conformance: the header undersold the harness by a whole suite #367

Merged

dcharlot merged 1 commit from fix/joule-code-conformance-header into main 2026-10-20 15:40:34 -04:00

Conversation 0 Commits 1 Files changed 2 +16 -10

dcharlot commented 2026-10-20 13:51:58 -04:00

Owner

Copy link

Found while hand-checking a candidate my own cross-standard audit had flagged — and the audit was wrong. verify_cascade and verify_gated_cascade both exist, so joule-code's C4 and C5 are implemented. But reading the crate to disprove the flag turned up three stale claims in its own documentation.

It says four suites; there are five

"runs the four suites: C1 content addressing, C2 signing, C3 receipt, C4 cascade"

That omits C5 gated cascade, whose six vectors demonstrably run (c5-ungranted-l4-denied-even-when-it-would-close, c5-exhausted-budget-stops-the-escape-hatch-on-the-second-run, …). A reader of the header would not know C5 is covered at all.

This is the understating direction of the defect family the last several changes here have been about: documentation narrower than the code. It is the benign direction — better than claiming coverage that doesn't exist (#357) — but it still misleads, and in a conformance harness the header is how an implementer learns what the pack proves.

It implies something is outstanding when nothing is

Both the lib header and main.rs explain SKIP as "derived fields pending regeneration". But conformance/v1/regeneration.md says:

Status: all fields are real. Every derived field in this pack has been computed by gen-vectors and verified by the harness (all vectors pass, none skipped). There are no placeholders.

And the run agrees: 24 passed, 0 failed, 0 skipped.

So two documents disagreed about whether the pack has a backlog. The SKIP mechanism is kept — a future input change re-opens that state, and is_placeholder is still live — but it no longer reads as though something were outstanding. All three documents now say the same thing.

Three cargo doc warnings were the docs' own fault

/// issuer key = [1;32], agent key = [2;32], steward key = [7;32],

Those are Rust array literals for the fixed test keys, written without backticks — so rustdoc parsed [1;32], [2;32], [7;32] each as an intra-doc link and could not resolve any of them. Backticked.

cargo doc --no-deps for this crate goes from 3 warnings to 0. (Unlike the eight I deliberately left in jouleclaw-energy in #353, these are neither platform-gated nor another author's cfg-split docs — they are a plain typo in prose this PR is already correcting.)

Verification

• joule-code-conformance -- verify ../conformance/v1 — 24 passed, 0 failed, 0 skipped
• cargo test -p joule-code-conformance --locked — exit 0
• cargo clippy -p joule-code-conformance --all-targets --no-deps --locked -- -D warnings — 0 findings
• cargo doc --no-deps -p joule-code-conformance — 0 warnings

On the audit that led here

Worth stating plainly: the cross-standard audit that flagged this flagged three standards, and every one was a false positive, now each checked by hand:

• joule-code C4/C5 — implemented as verify_cascade / verify_gated_cascade;
• sense C1–C5 — implemented as named fixture functions (c1_replay_digest, c4_run, c5_verify) plus #[test]s;
• map C1–C5 — implemented via run(); its header states "five positive suites and a negative pack … there are no placeholder SKIPs", and the F0–F4 ids my audit matched are the fidelity cascade, not its C classes.

So the audit produced zero real findings. A grep-level comparison of spec clause ids against ids in source cannot distinguish named differently from absent, and it is recorded as unreliable rather than kept as a tool. Disproving its flags by hand was still worth the time, because that is what surfaced the three stale claims above — and because the one genuine instance of this defect class (#357's C25) was found by reading, not by grepping.

🤖 Generated with Claude Code

Found while hand-checking a candidate **my own cross-standard audit had flagged** — and the audit was wrong. `verify_cascade` and `verify_gated_cascade` both exist, so joule-code's C4 and C5 *are* implemented. But reading the crate to disprove the flag turned up three stale claims in its own documentation. ## It says four suites; there are five > "runs the four suites: C1 content addressing, C2 signing, C3 receipt, C4 cascade" That omits **C5 gated cascade**, whose six vectors demonstrably run (`c5-ungranted-l4-denied-even-when-it-would-close`, `c5-exhausted-budget-stops-the-escape-hatch-on-the-second-run`, …). A reader of the header would not know C5 is covered at all. This is the *understating* direction of the defect family the last several changes here have been about: documentation narrower than the code. It is the benign direction — better than claiming coverage that doesn't exist (#357) — but it still misleads, and in a conformance harness the header is how an implementer learns what the pack proves. ## It implies something is outstanding when nothing is Both the lib header and `main.rs` explain SKIP as *"derived fields pending regeneration"*. But `conformance/v1/regeneration.md` says: > **Status: all fields are real.** Every derived field in this pack has been computed by `gen-vectors` and verified by the harness (all vectors pass, none skipped). There are no placeholders. And the run agrees: **24 passed, 0 failed, 0 skipped.** So two documents disagreed about whether the pack has a backlog. The SKIP mechanism is **kept** — a future input change re-opens that state, and `is_placeholder` is still live — but it no longer reads as though something were outstanding. All three documents now say the same thing. ## Three `cargo doc` warnings were the docs' own fault ``` /// issuer key = [1;32], agent key = [2;32], steward key = [7;32], ``` Those are Rust **array literals** for the fixed test keys, written without backticks — so rustdoc parsed `[1;32]`, `[2;32]`, `[7;32]` each as an intra-doc link and could not resolve any of them. Backticked. `cargo doc --no-deps` for this crate goes from **3 warnings to 0**. (Unlike the eight I deliberately left in `jouleclaw-energy` in #353, these are neither platform-gated nor another author's cfg-split docs — they are a plain typo in prose this PR is already correcting.) ## Verification - `joule-code-conformance -- verify ../conformance/v1` — **24 passed, 0 failed, 0 skipped** - `cargo test -p joule-code-conformance --locked` — exit **0** - `cargo clippy -p joule-code-conformance --all-targets --no-deps --locked -- -D warnings` — **0** findings - `cargo doc --no-deps -p joule-code-conformance` — **0** warnings ## On the audit that led here Worth stating plainly: the cross-standard audit that flagged this flagged **three standards, and every one was a false positive**, now each checked by hand: - **`joule-code`** C4/C5 — implemented as `verify_cascade` / `verify_gated_cascade`; - **`sense`** C1–C5 — implemented as named fixture functions (`c1_replay_digest`, `c4_run`, `c5_verify`) plus `#[test]`s; - **`map`** C1–C5 — implemented via `run()`; its header states "five positive suites and a negative pack … there are no placeholder SKIPs", and the `F0–F4` ids my audit matched are the fidelity cascade, not its `C` classes. So the audit produced **zero real findings**. A grep-level comparison of spec clause ids against ids in source cannot distinguish *named differently* from *absent*, and it is recorded as unreliable rather than kept as a tool. Disproving its flags by hand was still worth the time, because that is what surfaced the three stale claims above — and because the one genuine instance of this defect class (#357's `C25`) was found by reading, not by grepping. 🤖 Generated with [Claude Code](https://claude.com/claude-code)

dcharlot added 1 commit 2026-10-20 13:51:58 -04:00

joule-code-conformance: the header undersold the harness by a whole suite ... 81ba291c56

Found while hand-checking a candidate my own cross-standard audit had flagged.
The audit was wrong — `verify_cascade` and `verify_gated_cascade` both exist, so
C4 and C5 are implemented — but reading the crate to disprove the flag turned up
three stale claims in its own documentation.

**It says four suites; there are five.** "runs the four suites: C1 content
addressing, C2 signing, C3 receipt, C4 cascade" omits C5 gated cascade, whose
six vectors demonstrably run (`c5-ungranted-l4-denied-even-when-it-would-close`
and friends). A reader of the header would not know C5 is covered at all. This
is the understating direction of the same defect the last few changes here have
been about: documentation narrower than the code.

**It implies something is outstanding when nothing is.** Both the lib header
and `main.rs` explain SKIP as "derived fields pending regeneration", while
`conformance/v1/regeneration.md` says "Status: all fields are real ... all
vectors pass, none skipped. There are no placeholders." The run agrees: 24
passed, 0 failed, 0 skipped. The SKIP mechanism is kept, because a future input
change re-opens that state — but it no longer reads as a backlog. All three
documents now say the same thing.

**And three `cargo doc` warnings were the docs' own fault.** `issuer key =
[1;32], agent key = [2;32], steward key = [7;32]` are Rust array literals for
the fixed test keys, written without backticks, so rustdoc parsed each as an
intra-doc link and could not resolve it. Backticked; `cargo doc --no-deps` for
this crate goes from 3 warnings to 0.

Verified: pack 24 passed / 0 failed / 0 skipped, tests pass, clippy `-D
warnings` clean, `cargo doc` clean.

Co-Authored-By: Claude Opus 5 <[email protected]>

dcharlot merged commit 9de63a06d4 into main 2026-10-20 15:40:34 -04:00

dcharlot referenced this pull request from a commit 2026-10-20 15:40:36 -04:00

Merge pull request 'joule-code-conformance: the header undersold the harness by a whole suite' (#367) from fix/joule-code-conformance-header into main

Sign in to join this conversation.

Reviewers

No reviewers

Labels

Clear labels

No items

No labels

Milestone

Clear milestone

No items

No milestone

Projects

Clear projects

No items

No project

Assignees

Clear assignees

No assignees

1 participant

Notifications

Subscribe

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference

Transaction-Science/open-standards!367

No description provided.