0055f59b3c
JCR-1 ships ten conformance vectors with three independent verifier
implementations byte-equal. The triangle primitives (conservation,
cone, oracle, elicitor) had no equivalent — implementers in other
languages had to read the Rust source to know what "compliant" means.
This commit closes that gap. Each primitive gets:
conformance/vectors.json — the canonical fixture set
conformance/README.md — the conformance contract spec
tests/conformance.rs — Rust verifier wired into cargo test
Vectors:
conservation/1 — 5 vectors: conforming, below_spec, hallucination,
missing_delta, plus a multi-finding case
cone/1 — 6 vectors: every datum_default band entry,
jouleclaw_cascade_default L0, escalate-lut-to-
formulaic, escalate-from-apex (refused)
oracle/1 — 5 vectors: always_pass, always_fail, composite
all-pass with chain dedup, composite short-
circuit on first fail, empty-composite trivial-
pass
elicitor/1 — 5 vectors: empty-walker-done, single-required-
ask, two-required-fill-first-ask-second, fully-
filled-done, optional-slot-never-asked
Each verifier asserts byte-equality between the reference impl's
serde_json output and the vector's `expected_*`. Adding a vector
extends the conformance contract for every implementer.
The Python and Go reference verifiers (the JCR-1 multi-language
pattern) are roadmap, not part of this commit — the vectors are
language-neutral so other-language verifiers can be added without
changing the contract.
4 new tests, one per primitive — all green.
Co-Authored-By: Claude Opus 4.7 <[email protected]>
95 lines
3.1 KiB
JSON
95 lines
3.1 KiB
JSON
{
|
|
"spec": "JouleClaw Information Conservation Law v1",
|
|
"schema_id": "conservation/1",
|
|
"measure_id": "canonical-json-byte-count",
|
|
"note": "I(output) ≥ I(spec) AND I(output) ≤ I(spec) + I(delta) + I(substrate.used) (within residual_tolerance). Every implementer MUST produce the same Vec<VerificationFinding> ordering as the reference DeterministicVerifier, byte-equal under serde_json's serialization with the same field set.",
|
|
"vector_count": 5,
|
|
"vectors": [
|
|
{
|
|
"n": 1,
|
|
"description": "Conforming — output exactly equals spec, no delta, no substrate.",
|
|
"input": {
|
|
"spec_bytes_utf8": "hello",
|
|
"output_bytes_utf8": "hello",
|
|
"delta_fields": {},
|
|
"substrate_id": "",
|
|
"substrate_arrangement_hash": "",
|
|
"substrate_used_info": 0,
|
|
"residual_tolerance": 0,
|
|
"required_slots": []
|
|
},
|
|
"expected_findings": [
|
|
{"kind": "conforming"}
|
|
]
|
|
},
|
|
{
|
|
"n": 2,
|
|
"description": "BelowSpec — output shorter than spec; shortfall reported.",
|
|
"input": {
|
|
"spec_bytes_utf8": "the quick brown fox jumps over the lazy dog",
|
|
"output_bytes_utf8": "hi",
|
|
"delta_fields": {},
|
|
"substrate_id": "",
|
|
"substrate_arrangement_hash": "",
|
|
"substrate_used_info": 0,
|
|
"residual_tolerance": 0,
|
|
"required_slots": []
|
|
},
|
|
"expected_findings": [
|
|
{"kind": "below_spec", "shortfall": 41}
|
|
]
|
|
},
|
|
{
|
|
"n": 3,
|
|
"description": "Hallucination — output longer than spec + delta + substrate budget.",
|
|
"input": {
|
|
"spec_bytes_utf8": "hi",
|
|
"output_bytes_utf8": "hi there, this is a much longer response than the budget allows",
|
|
"delta_fields": {},
|
|
"substrate_id": "",
|
|
"substrate_arrangement_hash": "",
|
|
"substrate_used_info": 0,
|
|
"residual_tolerance": 0,
|
|
"required_slots": []
|
|
},
|
|
"expected_findings": [
|
|
{"kind": "hallucination", "unaccounted": 61}
|
|
]
|
|
},
|
|
{
|
|
"n": 4,
|
|
"description": "MissingDelta — required slot 'name' was never filled.",
|
|
"input": {
|
|
"spec_bytes_utf8": "hello, name",
|
|
"output_bytes_utf8": "hello, name",
|
|
"delta_fields": {},
|
|
"substrate_id": "",
|
|
"substrate_arrangement_hash": "",
|
|
"substrate_used_info": 0,
|
|
"residual_tolerance": 0,
|
|
"required_slots": ["name"]
|
|
},
|
|
"expected_findings": [
|
|
{"kind": "missing_delta", "missing": ["name"]}
|
|
]
|
|
},
|
|
{
|
|
"n": 5,
|
|
"description": "Multi-finding — MissingDelta + BelowSpec together (required slot empty AND output undersized).",
|
|
"input": {
|
|
"spec_bytes_utf8": "the quick brown fox jumps over the lazy dog",
|
|
"output_bytes_utf8": "hi",
|
|
"delta_fields": {},
|
|
"substrate_id": "",
|
|
"substrate_arrangement_hash": "",
|
|
"substrate_used_info": 0,
|
|
"residual_tolerance": 0,
|
|
"required_slots": ["name"]
|
|
},
|
|
"expected_findings": [
|
|
{"kind": "missing_delta", "missing": ["name"]},
|
|
{"kind": "below_spec", "shortfall": 41}
|
|
]
|
|
}
|
|
]
|
|
}
|