joule-code: close the two gaps its conformance gate cannot see #210

Merged

dcharlot merged 2 commits from dcharlot/joule-code-pack-test into main 2026-10-13 02:40:37 -04:00 AGit

Conversation 0 Commits 2 Files changed 2 +176 -2

dcharlot commented 2026-10-10 17:22:45 -04:00

Owner

Copy link

Revives #97 (1083 commits behind) by cherry-pick onto current main, plus one correction. Re-verified against main: the pinned PUBLISHED_VECTORS=24 still matches the published pack with skip==0, and both tests pass including the tamper case. The second commit fixes a claim in the added workflow comment — it named 'a tests/pack.rs in each conformance crate' as the mechanism, which is a convention rather than a rule, and auditing by that filename produces false coverage gaps. Supersedes #97, which I will close.

Revives #97 (1083 commits behind) by cherry-pick onto current main, plus one correction. Re-verified against main: the pinned PUBLISHED_VECTORS=24 still matches the published pack with skip==0, and both tests pass including the tamper case. The second commit fixes a claim in the added workflow comment — it named 'a tests/pack.rs in each conformance crate' as the mechanism, which is a convention rather than a rule, and auditing by that filename produces false coverage gaps. Supersedes #97, which I will close.

dcharlot added 2 commits 2026-10-10 17:22:45 -04:00

joule-code: close the two gaps its conformance gate cannot see ... 692d769f2e

`joule-code-conformance` ships a complete pack verifier — `verify_pack` reads
all six files of `joule-code/conformance/v1`, recomputes every derived field
(each `op_id` is BLAKE3 over canonical CBOR of its `CodingOp`, each binding and
receipt signature a real Ed25519 signature over the spec's preimage) and returns
a FAIL count that `main()` turns into a non-zero exit. A dedicated workflow,
`joule-code-conformance.yml`, already runs it. The pack is not unchecked, and
this commit is not what makes it checked.

It closes the two gaps that gate cannot see.

**It cannot distinguish a verified pack from a skipped one.** The gate is
`grep -Eq '0 failed'` over the summary line, and `verify_pack` is documented to
let SKIPs pass. So `0 passed, 0 failed, 24 skipped` satisfies that grep exactly
as well as `24 passed, 0 failed, 0 skipped` — verified by running the three
summary strings through the gate's own expression. A pack that shrank to one
vector passes it too. This pins the count and requires `skip == 0`, which is
the distinction the grep cannot draw. The count is pinned rather than bounded
because `>= 24` would let a case be deleted as long as another was added.

**It is `paths:`-filtered to `joule-code/**`, and this crate reaches outside
it.** `joule-code-conformance` depends on `jcp-schema`, `jcp-grant` and
`jcp-receipt` by relative path across workspaces, and `joule-code-joulecontract`
on `joulecontract-schema`. A change in `jcp` to canonical CBOR or the receipt
preimage would change the `op_id`s and signatures this pack pins, and would not
touch `joule-code/**` — so that workflow would not run. `standards-build.yml`
carries a deliberate "NO `paths:` filter" and runs `cargo test --workspace`,
confirmed by running that exact command rather than reasoning about it.

The second test is the load-bearing one. A verifier reporting PASS on a pack it
never read is worse than none, because it manufactures confidence — a tampered
copy of `map`'s pack once still reported 15/15. So the tamper case does not
trust the report: it flips one hex digit of a signature in a scratch copy and
requires the verifier to notice. The offset is found by scanning to the value's
opening quote, since the pack is column-aligned and the run of spaces is not
something a test should depend on.

Falsified, each assertion separately: corrupting a published signature fails
the verify test; claiming 25 vectors fails the count; neutering the flip to a
no-op fails the tamper test with "the verifier is not checking what it
publishes". The pack was restored afterwards and is byte-identical to main.

One honest limit: I could not force a SKIP to confirm that assertion fires, so
unlike the other three it is reasoned rather than falsified.

Corrects the workflow comment claiming nothing in CI validates the published
`conformance/v1` files on disk. That stopped being true with the `tests/pack.rs`
suites, and a missing `conformance:` key is not evidence a pack is unchecked —
`joule-code` and `sandbox` are both covered without one.

Co-Authored-By: Claude Opus 5 <[email protected]>

ci: the comment named a mechanism that is a convention, not a rule ... 37f5bda4ba

The commit before this one (revived from #97) added a note explaining why a
missing `conformance:` key does not mean a standard's pack is unchecked. Its
reasoning is right and its mechanism was wrong: it said the checking is done
"by a `tests/pack.rs` in each conformance crate". Several crates do use that
name, but it is a convention — some read the pack from a test named something
else, and some from another crate entirely.

That matters because the wrong mechanism invites the wrong audit. Counting
files named `tests/pack.rs` and finding crates without one looks like a
coverage gap and is not one; I did exactly that earlier today and was about to
report three standards as unchecked before a note from the last time it
happened stopped me. So the comment now names the question — which tests read
the published bytes — and the grep that answers it.

No count in the text on purpose: any "N of M conformance crates" would be
stale by the next crate added, and a stale number in a comment is how the
first wrong conclusion got its confidence.

Co-Authored-By: Claude Opus 5 <[email protected]>

dcharlot referenced this pull request 2026-10-10 17:22:59 -04:00

joule-code: close the two gaps its conformance gate cannot see #97

dcharlot referenced this pull request 2026-10-11 22:59:51 -04:00

wai-vector: the default stroke set and the test repertoire, as WSF1 sources expanded by Annex A, and the text vectors (D7-5) #224

dcharlot merged commit 1370f77bd8 into main 2026-10-13 02:40:37 -04:00

dcharlot referenced this pull request from a commit 2026-10-13 02:40:37 -04:00

Merge pull request 'joule-code: close the two gaps its conformance gate cannot see' (#210) from dcharlot/joule-code-pack-test into main

dcharlot referenced this pull request from a commit 2026-10-14 02:13:13 -04:00

Merge origin/main (at bd3983c6, with #229-#232, #235, #142, #209, #210) into wai/d2c-decoders-on-ops

Sign in to join this conversation.

Reviewers

No reviewers

Labels

Clear labels

No items

No labels

Milestone

Clear milestone

No items

No milestone

Projects

Clear projects

No items

No project

Assignees

Clear assignees

No assignees

1 participant

Notifications

Subscribe

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference

Transaction-Science/open-standards!210

No description provided.