0055f59b3c
JCR-1 ships ten conformance vectors with three independent verifier
implementations byte-equal. The triangle primitives (conservation,
cone, oracle, elicitor) had no equivalent — implementers in other
languages had to read the Rust source to know what "compliant" means.
This commit closes that gap. Each primitive gets:
conformance/vectors.json — the canonical fixture set
conformance/README.md — the conformance contract spec
tests/conformance.rs — Rust verifier wired into cargo test
Vectors:
conservation/1 — 5 vectors: conforming, below_spec, hallucination,
missing_delta, plus a multi-finding case
cone/1 — 6 vectors: every datum_default band entry,
jouleclaw_cascade_default L0, escalate-lut-to-
formulaic, escalate-from-apex (refused)
oracle/1 — 5 vectors: always_pass, always_fail, composite
all-pass with chain dedup, composite short-
circuit on first fail, empty-composite trivial-
pass
elicitor/1 — 5 vectors: empty-walker-done, single-required-
ask, two-required-fill-first-ask-second, fully-
filled-done, optional-slot-never-asked
Each verifier asserts byte-equality between the reference impl's
serde_json output and the vector's `expected_*`. Adding a vector
extends the conformance contract for every implementer.
The Python and Go reference verifiers (the JCR-1 multi-language
pattern) are roadmap, not part of this commit — the vectors are
language-neutral so other-language verifiers can be added without
changing the contract.
4 new tests, one per primitive — all green.
Co-Authored-By: Claude Opus 4.7 <[email protected]>
51 lines
2 KiB
Rust
51 lines
2 KiB
Rust
//! Elicitor conformance — the reference [`SlotWalker`] must produce
|
|
//! the final [`ElicitorStep`] listed in `conformance/vectors.json`
|
|
//! after processing the submit sequence for every fixture, byte-equal
|
|
//! under serde_json.
|
|
|
|
use jouleclaw_elicitor::{Answer, Elicitor, ElicitorStep, RawRequest, Slot, SlotWalker};
|
|
use serde_json::Value;
|
|
|
|
const VECTORS: &str = include_str!("../conformance/vectors.json");
|
|
|
|
fn check_vector(v: &Value) {
|
|
let n = v["n"].as_i64().expect("n");
|
|
let desc = v["description"].as_str().expect("description");
|
|
let input = &v["input"];
|
|
let expected = &v["expected_final_step"];
|
|
|
|
let request = RawRequest::new(
|
|
input["request_text"]
|
|
.as_str()
|
|
.expect("request_text is string"),
|
|
);
|
|
let slots: Vec<Slot> = serde_json::from_value(input["slots"].clone()).expect("slots parse");
|
|
let mut walker = SlotWalker::new(&request, slots);
|
|
|
|
let mut step = walker.next();
|
|
for sub in input["submits"].as_array().expect("submits array") {
|
|
let answer: Answer = serde_json::from_value(sub.clone()).expect("answer parse");
|
|
step = walker.submit(answer).expect("submit succeeds for valid vectors");
|
|
}
|
|
|
|
let got: Value = serde_json::to_value(&step).expect("step serialises");
|
|
assert_eq!(
|
|
&got, expected,
|
|
"vector {n} ({desc}): final step mismatch.\n got = {got}\n expected = {expected}"
|
|
);
|
|
|
|
// Round-trip the step through ElicitorStep to confirm it parses
|
|
// back identically — the conformance contract is bidirectional.
|
|
let _back: ElicitorStep = serde_json::from_value(got).expect("step parses back");
|
|
}
|
|
|
|
#[test]
|
|
fn every_elicitor_vector_matches_the_reference_walker() {
|
|
let suite: Value = serde_json::from_str(VECTORS).expect("vectors.json parses");
|
|
let count = suite["vector_count"].as_u64().expect("vector_count") as usize;
|
|
let vectors = suite["vectors"].as_array().expect("vectors array");
|
|
assert_eq!(vectors.len(), count, "vector_count mismatch");
|
|
for v in vectors {
|
|
check_vector(v);
|
|
}
|
|
}
|