open-standards/jouleclaw/jouleclaw-rs/crates/jouleclaw-evals
Repository files (latest commit first)
Filename Latest commit message Latest commit date
Transaction Science 49c6eaa0de
Some checks failed
repository checks / lockfiles, advisories, tests in CI, workflow hygiene, spec and section references, public text (push) Has been cancelled
jouleclaw public surface must stay documented / Public-surface audit (jouleclaw-rs) (push) Has been cancelled
jouleclaw: memory, poisoning and errand benchmarks, scored from both sides
jouleclaw-evals::memory runs fixed, deterministic cases against the
notebook. Poisoning plants notes from a sender, from the agent, from the
person's signature over swapped words and from no words, some on topics
the person already has a note on, and counts attacks blocked, the
person's notes accepted, and whether any of theirs was displaced after
maintenance: 6 of 6 blocked, 3 of 3 kept, none displaced. Long-term
recall changes the person's facts over time (an update, a correction, a
trip that ends) and counts current answers and stale ones: 7 of 7
current, none stale. Errands are graded from their trace: done (an
interrupted errand is not), within the grant, every consequential step
cited (JCP §4.14).

The poisoning benchmark discriminates: a notebook that takes the
sender's words as the person's fails it. Falsified through the
benchmarks: a notebook that trusts anyone, no supersession, no expiry,
an interrupted errand counted done, and uncited steps ignored each turn
a test red.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-11-12 04:12:31 -05:00
..
src jouleclaw: memory, poisoning and errand benchmarks, scored from both sides 2026-11-12 04:12:31 -05:00
Cargo.toml jouleclaw: memory, poisoning and errand benchmarks, scored from both sides 2026-11-12 04:12:31 -05:00