jouleclaw: a claim is judged on what it cites, so a trusted source cannot be credited with what a weak one says #355

Merged

dcharlot merged 1 commit from fix/claims-grounded-by-their-own-citations into main 2026-10-20 08:28:14 -04:00

Conversation 0 Commits 1 Files changed 6 +515 -14

2026-10-20

Transaction Science

jouleclaw: a claim is judged on what it cites, so a trusted source cannot be credited with what a weak one says ...

jouleclaw-diagnose checked grounding and authority without reading what the
answer cites:
- grounding passed if ANY retrieved item entailed the claim;
- authority passed if ANY retrieved item for the sub-query met the tier;
- `cited_item_ids` appeared nowhere in the crate.

So an answer could cite a primary source that does not support a claim,
supported only by an uncited forum post, and pass both: grounded by the post,
authoritative by the source. That is citation laundering. One weak source
makes the claim, and a trusted one is credited with it. The report also
listed I1 (every claim has provenance) and I4 (cited sources meet the tier)
as verified on every run, though nothing checked either.

`citation_violations` now judges each claim on its own segment's citations,
using the same notion of support as grounding (every model that judged the
pair entailed it):
- a supported claim that cites nothing is `uncited` (Major);
- one supported only by items it does not cite is `misattributed` (Major);
- it is `laundering` (Critical, not retrievable, so the answer is refused)
  when it also cites an item more authoritative than any that supports it;
- authority is judged on the most authoritative cited item that supports the
  claim: an `authority` violation scoped to the claim (Critical,
  retrievable) when even that one falls short.

Recovery drops laundering and hedges the rest. `VerifyInputs::with_citations`
supplies the citations, and the edge pipeline passes the draft's at both
verify calls. I1 and I4 are listed only when the citations were judged.

jouleclaw-compose attributed a segment to its most authoritative cited item
(#344) whether or not that item supported the claim, so the displayed source
could still be the laundered one. Where the verifier judged the claim
against its citations, the attribution now goes to the most authoritative
cited item that supports it. Where none does, the segment is attributed to
none, with a caveat saying so. Where nothing was judged, the most
authoritative cited item stands as before.

Tests: 8 new across diagnose and compose, including the attack end to end
through `verify`. All 11 mutations of the new logic are killed.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

e541ab6d79

• View at this point in history

2026-10-20 04:27:42 -04:00