0055f59b3c
JCR-1 ships ten conformance vectors with three independent verifier
implementations byte-equal. The triangle primitives (conservation,
cone, oracle, elicitor) had no equivalent — implementers in other
languages had to read the Rust source to know what "compliant" means.
This commit closes that gap. Each primitive gets:
conformance/vectors.json — the canonical fixture set
conformance/README.md — the conformance contract spec
tests/conformance.rs — Rust verifier wired into cargo test
Vectors:
conservation/1 — 5 vectors: conforming, below_spec, hallucination,
missing_delta, plus a multi-finding case
cone/1 — 6 vectors: every datum_default band entry,
jouleclaw_cascade_default L0, escalate-lut-to-
formulaic, escalate-from-apex (refused)
oracle/1 — 5 vectors: always_pass, always_fail, composite
all-pass with chain dedup, composite short-
circuit on first fail, empty-composite trivial-
pass
elicitor/1 — 5 vectors: empty-walker-done, single-required-
ask, two-required-fill-first-ask-second, fully-
filled-done, optional-slot-never-asked
Each verifier asserts byte-equality between the reference impl's
serde_json output and the vector's `expected_*`. Adding a vector
extends the conformance contract for every implementer.
The Python and Go reference verifiers (the JCR-1 multi-language
pattern) are roadmap, not part of this commit — the vectors are
language-neutral so other-language verifiers can be added without
changing the contract.
4 new tests, one per primitive — all green.
Co-Authored-By: Claude Opus 4.7 <[email protected]>
114 lines
3.8 KiB
JSON
114 lines
3.8 KiB
JSON
{
|
|
"spec": "JouleClaw Elicitor v1 — deterministic structured intake",
|
|
"schema_id": "elicitor/1",
|
|
"note": "SlotWalker.next() and .submit() drive a deterministic state machine: pending required slots become Ask gaps in declared order; once every required slot has been filled (in any order), the walker emits Done with the filled list in submit-order. Every implementer MUST produce the same ElicitorStep (byte-equal under serde_json) for every (slots, submit-sequence) pair below.",
|
|
"vector_count": 5,
|
|
"vectors": [
|
|
{
|
|
"n": 1,
|
|
"description": "Empty slot list → Done immediately with no filled answers.",
|
|
"input": {
|
|
"request_text": "anything",
|
|
"slots": [],
|
|
"submits": []
|
|
},
|
|
"expected_final_step": {
|
|
"kind": "done",
|
|
"draft": {"summary": "anything", "filled": []}
|
|
}
|
|
},
|
|
{
|
|
"n": 2,
|
|
"description": "Single required Text slot, unanswered → Ask gap for it.",
|
|
"input": {
|
|
"request_text": "hello",
|
|
"slots": [
|
|
{"id": "msg", "name": "Message", "prompt": "Your message", "kind": {"kind": "text"}, "required": true}
|
|
],
|
|
"submits": []
|
|
},
|
|
"expected_final_step": {
|
|
"kind": "ask",
|
|
"gap": {
|
|
"slot_id": "msg",
|
|
"name": "Message",
|
|
"prompt": "Your message",
|
|
"kind": {"kind": "text"},
|
|
"required": true
|
|
}
|
|
}
|
|
},
|
|
{
|
|
"n": 3,
|
|
"description": "Two required slots, fill the first → Ask for the second.",
|
|
"input": {
|
|
"request_text": "demo",
|
|
"slots": [
|
|
{"id": "a", "name": "A", "prompt": "What is a?", "kind": {"kind": "text"}, "required": true},
|
|
{"id": "b", "name": "B", "prompt": "What is b?", "kind": {"kind": "integer"}, "required": true}
|
|
],
|
|
"submits": [
|
|
{"slot_id": "a", "value": {"kind": "text", "text": "alpha"}}
|
|
]
|
|
},
|
|
"expected_final_step": {
|
|
"kind": "ask",
|
|
"gap": {
|
|
"slot_id": "b",
|
|
"name": "B",
|
|
"prompt": "What is b?",
|
|
"kind": {"kind": "integer"},
|
|
"required": true
|
|
}
|
|
}
|
|
},
|
|
{
|
|
"n": 4,
|
|
"description": "Two required slots, fill both → Done with both answers in submit-order.",
|
|
"input": {
|
|
"request_text": "demo",
|
|
"slots": [
|
|
{"id": "a", "name": "A", "prompt": "What is a?", "kind": {"kind": "text"}, "required": true},
|
|
{"id": "b", "name": "B", "prompt": "What is b?", "kind": {"kind": "boolean"}, "required": true}
|
|
],
|
|
"submits": [
|
|
{"slot_id": "a", "value": {"kind": "text", "text": "alpha"}},
|
|
{"slot_id": "b", "value": {"kind": "boolean", "b": true}}
|
|
]
|
|
},
|
|
"expected_final_step": {
|
|
"kind": "done",
|
|
"draft": {
|
|
"summary": "demo",
|
|
"filled": [
|
|
{"slot_id": "a", "value": {"kind": "text", "text": "alpha"}},
|
|
{"slot_id": "b", "value": {"kind": "boolean", "b": true}}
|
|
]
|
|
}
|
|
}
|
|
},
|
|
{
|
|
"n": 5,
|
|
"description": "Optional slot is never asked about; once the one required slot is filled the walker is Done even if the optional remains empty.",
|
|
"input": {
|
|
"request_text": "demo",
|
|
"slots": [
|
|
{"id": "opt", "name": "Opt", "prompt": "Optional?", "kind": {"kind": "text"}, "required": false},
|
|
{"id": "req", "name": "Req", "prompt": "Required?", "kind": {"kind": "text"}, "required": true}
|
|
],
|
|
"submits": [
|
|
{"slot_id": "req", "value": {"kind": "text", "text": "yes"}}
|
|
]
|
|
},
|
|
"expected_final_step": {
|
|
"kind": "done",
|
|
"draft": {
|
|
"summary": "demo",
|
|
"filled": [
|
|
{"slot_id": "req", "value": {"kind": "text", "text": "yes"}}
|
|
]
|
|
}
|
|
}
|
|
}
|
|
]
|
|
}
|