6a18101d12
engine::answer reads wai.quantum.answer/1: view statements over an OpenQASM 3 program (Histogram, Verdict, Equivalence, Bloch, Estimate, Receipt and the rest, fourteen in all), each view's numbers bound to an engine call written with @ (@Sample, @Check, @Seal, eleven in all), so a model's answer shows numbers the engine computed. One statement a line, the program in a named qasm fence, root first, forward references, and a later declaration standing as the edit. The reader streams: a snapshot says what is ready, pending or dropped, and the finished answer is the whole text's at every split (tested at every split point). Every problem has a stable code: a program's keep the reader's Q codes, a statement's are the new J100 to J112, each dropped, defaulted or warned, and only the dropped ones are errors a model is sent back. eval carries the calls out on the engine (a run's time left out), seals what @Seal names in a job's form, and checks a stored answer's receipt marker by running it again. The conformance set (quantum-conformance/answers/, 20 cases, every code) passes natively, under wasm32-wasip2 and in the browser, where the stored case written natively verifies to the same bytes. wai-web gains AnswerStream and the answer and telemetry calls (pins 24 native, 16 wasm32). engine::telemetry defines the correctness record (codes, verdict level, repairs, the person's accept or reject, the receipt) and turns the failures among records into an eval corpus in the conformance format. The specification (extensions/quantum-answers.md) is a draft whose token comparison is still to come. Co-Authored-By: Claude Opus 5.5 <[email protected]>
2.5 KiB
Answer conformance
Answers in the answer view layer (quantum-answers.md,
wai_quantum::engine::answer), each with what the layer must make of it.
| Path | What |
|---|---|
cases/*.wqa |
The answers: a program, a sample, a verdict and a seal; an intent named by a reference program; the equivalence ladder; an intent that fails; expectation values, amplitudes and a Bloch vector; an engine call named apart from its view; no root; and one case for each code of the layer (J100 to J112) and for a program the reader refuses (Q103). bell-crlf is bell with CRLF line ends. bell-stored is bell with the receipt marker its seal gives under the key whose seed is 32 bytes of 3, and bell-stored-changed the same with one sample's shots changed. |
manifest.json |
Each case: its file, what it covers, and what it must give (the manifest's expect says what each field means). |
bundle.json |
The manifest's cases with every answer inline, for builds that read no files. |
A case is held to the codes of its errors (the diagnostics whose outcome is
dropped or refused, the ones a model is sent back) and of its warnings, in
the order of their lines; to its root; and to the order its views are drawn
in. A case with run has its views carried out on the engine and held to what
they show: a sample's total and its outcomes, a verdict's level and whether it
holds, an equivalence's decision, expectation values, a Bloch vector, the
names a seal covers and its receipt. A stored case's receipt is checked by
running the answer again: the engine is deterministic, so the same answer
gives the same sealed bytes on every build, and a changed one does not verify.
Native
cd wai/wai-quantum
cargo test --release --features full --test answer_conformance
Two tests: every case against its manifest, and the bundle (and the stored cases) held equal to the files. After changing a case or the manifest, write them:
WAI_ANSWER_BUNDLE_WRITE=1 cargo test --release --features full --test answer_conformance the_bundle_is_the_corpus
WASI (the edge)
CARGO_TARGET_WASM32_WASIP2_RUNNER=wasmtime \
cargo test --target wasm32-wasip2 --features full --test answer_conformance
Browser
wai-web's wasm32 tests (--features quantum_engine, under
wasm-bindgen-test-runner) read every case of the bundle with
engine_answer_parse and verify the stored cases with engine_answer_verify.
CI: wai-quantum-verdict.yml (native and wasm32-wasip2) and
wai-web-wasm.yml (the browser build).