joulecontract: the published negative vectors were never actually run #94

Merged

dcharlot merged 2 commits from joulecontract/run-negative-vectors into main 2026-09-18 11:35:25 -04:00

Conversation 0 Commits 2 Files changed 1 +139

2026-09-17

David Charlot

joulecontract: the negative replay could pass with the pack missing ...

The replay test opened with the same `if !dir.exists() { return; }` the other
two vector-pack tests had — and that early return fires *before* the
`replayed >= 11` guard at the end of the function. So a missing or moved pack
reported green, which is precisely the case that guard was written to catch. I
had described the test as anti-vacuous; with that return in place it was not.

`main` has since replaced the same pattern in `vector_pack_tests` with an
assertion, on the grounds that this crate is not published and so only ever runs
in-repo where the pack is always present. That reasoning applies unchanged here,
so the replay now asserts too, and the three pack tests in this file agree.

Verified in both directions rather than one:
  - pack present: 10 passed
  - pack moved aside: the replay FAILS, naming the path it could not find
Before this change the second case passed. The pack was restored and confirmed
byte-identical with `git status`, not by assumption.

Co-Authored-By: Claude Opus 5 <[email protected]>

7eee267a06

• View at this point in history

2026-09-17 21:07:54 -04:00

David Charlot

joulecontract: the published negative vectors were never actually run ...

The pack had two tests: every file's content address against `MANIFEST.json`,
and the positive DSSE envelope. Neither *executed* a negative vector. All eleven
`negative-*.json` were published, digest-pinned, and never read by a verifier —
so the vectors written to catch a verifier that stops demoting were exactly the
ones that could not catch it.

This replays each through `validate_sealed` under the context they were generated
in, and compares all four recorded fields: the demotion list, the maximum
profile, the rendered §15 claim string, and whether the claimed profile is
permitted.

**Equality, not "it demoted".** These are degradation vectors, not accept/reject
ones. Each records the verdict a conformant verifier must produce —
`metered_unattested` demotes Energy to `ModelBased`, `generator_unattested` caps
Governance at `declared`. Asserting merely that something was demoted would pass
a verifier that demoted the wrong axis, or by the wrong amount, or that refused
the receipt outright — and refusing is not the specified behaviour either, so the
replay also asserts each vector still validates structurally.

**A wrong assumption the vectors corrected.** My first version asserted every
negative vector must leave its claimed profile impermissible. Two do not:
`recourse_undeclared` and `static_only_frame` record a *disclosure* demotion
rather than a cap, and `static_only` is prohibited only at JR-4 while these
vectors claim JR-3. The honest verdict there is "permitted, and here is what you
are trusting". Asserting `!permits` would have pressured two published vectors
toward a stricter reading than the spec takes, so the invariant is the demotion
list instead.

**Falsified, not assumed.** Removing the `EngagementUndisclosed` demotion rule
from the verifier leaves the *entire* pre-existing workspace suite green, and
fails this replay naming the vector and the axis. `StaticOnly`,
`EngagementUndisclosed` and `IndependentKeyOnly` appear in no unit test in
`joulecontract-receipt` at all — the published pack was their only coverage, and
it was not running. (A probe against `MeteredUnattested` is *not* silent: two
unit tests catch that one. The gap is uneven, which is itself worth knowing.)

The replay asserts it saw at least eleven vectors, so a wrong path, a renamed
prefix or an empty glob fails rather than passing having checked nothing — the
same vacuity that let the original gap hide. It also pins the three context
scalars it replays under, so a regenerated pack under a different context is a
failure rather than a silent comparison between two different questions.

`joulecontract-rs` is already in the CI matrix, so these vectors now run on every
change.

Verified: 15 test binaries green across the workspace, 1 new. The verifier crate
is byte-identical to `main` — the falsification probes were reverted and the
revert checked with `git diff --quiet`.

Co-Authored-By: Claude Opus 5 <[email protected]>

27fa4e5e59

• View at this point in history

2026-09-17 21:07:54 -04:00