open-standards/wai/quantum-conformance/answers

Answer conformance

Answers in the answer view layer (quantum-answers.md, wai_quantum::engine::answer), each with what the layer must make of it.

Path	What
cases/*.wqa	The answers: a program, a sample, a verdict and a seal; an intent named by a reference program; the equivalence ladder; an intent that fails; expectation values, amplitudes and a Bloch vector; an engine call named apart from its view; no root; and one case for each code of the layer (J100 to J112) and for a program the reader refuses (Q103). bell-crlf is bell with CRLF line ends. bell-stored is bell with the receipt marker its seal gives under the key whose seed is 32 bytes of 3, and bell-stored-changed the same with one sample's shots changed.
manifest.json	Each case: its file, what it covers, and what it must give (the manifest's expect says what each field means).
bundle.json	The manifest's cases with every answer inline, for builds that read no files.

A case is held to the codes of its errors (the diagnostics whose outcome is dropped or refused, the ones a model is sent back) and of its warnings, in the order of their lines; to its root; and to the order its views are drawn in. A case with run has its views carried out on the engine and held to what they show: a sample's total and its outcomes, a verdict's level and whether it holds, an equivalence's decision, expectation values, a Bloch vector, the names a seal covers and its receipt. A stored case's receipt is checked by running the answer again: the engine is deterministic, so the same answer gives the same sealed bytes on every build, and a changed one does not verify.

Native

cd wai/wai-quantum
cargo test --release --features full --test answer_conformance

Two tests: every case against its manifest, and the bundle (and the stored cases) held equal to the files. After changing a case or the manifest, write them:

WAI_ANSWER_BUNDLE_WRITE=1 cargo test --release --features full --test answer_conformance the_bundle_is_the_corpus

WASI (the edge)

CARGO_TARGET_WASM32_WASIP2_RUNNER='wasmtime run -W nan-canonicalization=y -W relaxed-simd-deterministic=y' \
  cargo test --target wasm32-wasip2 --features full --test answer_conformance

Browser

wai-web's wasm32 tests (--features quantum_engine, under wasm-bindgen-test-runner) read every case of the bundle with engine_answer_parse and verify the stored cases with engine_answer_verify.

CI: wai-quantum-verdict.yml (native and wasm32-wasip2) and wai-web-wasm.yml (the browser build).