0055f59b3c
JCR-1 ships ten conformance vectors with three independent verifier
implementations byte-equal. The triangle primitives (conservation,
cone, oracle, elicitor) had no equivalent — implementers in other
languages had to read the Rust source to know what "compliant" means.
This commit closes that gap. Each primitive gets:
conformance/vectors.json — the canonical fixture set
conformance/README.md — the conformance contract spec
tests/conformance.rs — Rust verifier wired into cargo test
Vectors:
conservation/1 — 5 vectors: conforming, below_spec, hallucination,
missing_delta, plus a multi-finding case
cone/1 — 6 vectors: every datum_default band entry,
jouleclaw_cascade_default L0, escalate-lut-to-
formulaic, escalate-from-apex (refused)
oracle/1 — 5 vectors: always_pass, always_fail, composite
all-pass with chain dedup, composite short-
circuit on first fail, empty-composite trivial-
pass
elicitor/1 — 5 vectors: empty-walker-done, single-required-
ask, two-required-fill-first-ask-second, fully-
filled-done, optional-slot-never-asked
Each verifier asserts byte-equality between the reference impl's
serde_json output and the vector's `expected_*`. Adding a vector
extends the conformance contract for every implementer.
The Python and Go reference verifiers (the JCR-1 multi-language
pattern) are roadmap, not part of this commit — the vectors are
language-neutral so other-language verifiers can be added without
changing the contract.
4 new tests, one per primitive — all green.
Co-Authored-By: Claude Opus 4.7 <[email protected]>
73 lines
3 KiB
JSON
73 lines
3 KiB
JSON
{
|
|
"spec": "JouleClaw Oracle v1 — closure-gate Verdict",
|
|
"schema_id": "oracle/1",
|
|
"note": "Verdict { Pass { chain: [oracle_id, ...] } | Fail { reasons: [FailReason, ...] } }. CompositeOracle runs its members in registration order and short-circuits on the first Fail. Every implementer MUST produce the same Verdict (byte-equal under serde_json) for every (oracle_program, candidate) pair below.",
|
|
"vector_count": 5,
|
|
"vectors": [
|
|
{
|
|
"n": 1,
|
|
"description": "AlwaysPass — emits Pass with the oracle's id in the chain.",
|
|
"input": {
|
|
"candidate": {"id": "c1", "content": "ok", "language": "rust", "source_tier": "lut", "provenance": "lut:t1"},
|
|
"oracle_program": [{"kind": "always_pass"}]
|
|
},
|
|
"expected_verdict": {"kind": "pass", "chain": ["unattested-pass"]}
|
|
},
|
|
{
|
|
"n": 2,
|
|
"description": "AlwaysFail — emits Fail with a single Custom reason.",
|
|
"input": {
|
|
"candidate": {"id": "c2", "content": "ok", "language": "rust", "source_tier": "lut", "provenance": "lut:t1"},
|
|
"oracle_program": [{"kind": "always_fail", "reason": "no oracle in this env"}]
|
|
},
|
|
"expected_verdict": {
|
|
"kind": "fail",
|
|
"reasons": [{"kind": "custom", "subkind": "no-oracle", "message": "no oracle in this env"}]
|
|
}
|
|
},
|
|
{
|
|
"n": 3,
|
|
"description": "Composite with two AlwaysPass oracles — chain contains both ids in registration order.",
|
|
"input": {
|
|
"candidate": {"id": "c3", "content": "ok", "language": "rust", "source_tier": "lut", "provenance": "lut:t1"},
|
|
"oracle_program": [
|
|
{"kind": "composite", "id": "compile-and-test"},
|
|
{"kind": "always_pass"},
|
|
{"kind": "always_pass"}
|
|
]
|
|
},
|
|
"expected_verdict": {
|
|
"kind": "pass",
|
|
"chain": ["compile-and-test", "unattested-pass"]
|
|
}
|
|
},
|
|
{
|
|
"n": 4,
|
|
"description": "Composite with AlwaysPass then AlwaysFail — short-circuits on first Fail (the second oracle is not consulted; only the first AlwaysPass and the AlwaysFail contribute).",
|
|
"input": {
|
|
"candidate": {"id": "c4", "content": "ok", "language": "rust", "source_tier": "lut", "provenance": "lut:t1"},
|
|
"oracle_program": [
|
|
{"kind": "composite", "id": "lint-then-test"},
|
|
{"kind": "always_pass"},
|
|
{"kind": "always_fail", "reason": "test failed"},
|
|
{"kind": "always_pass"}
|
|
]
|
|
},
|
|
"expected_verdict": {
|
|
"kind": "fail",
|
|
"reasons": [{"kind": "custom", "subkind": "no-oracle", "message": "test failed"}]
|
|
}
|
|
},
|
|
{
|
|
"n": 5,
|
|
"description": "Empty composite — trivially passes with an empty chain.",
|
|
"input": {
|
|
"candidate": {"id": "c5", "content": "ok", "language": "rust", "source_tier": "lut", "provenance": "lut:t1"},
|
|
"oracle_program": [
|
|
{"kind": "composite", "id": "empty"}
|
|
]
|
|
},
|
|
"expected_verdict": {"kind": "pass", "chain": ["empty"]}
|
|
}
|
|
]
|
|
}
|