open-standards/wai/wai-quantum/docs/verdicts.md

# Verdicts: how far the engine checked

Every answer WAI Quantum gives about a program states one verification level.
"Valid" is never the claim: the engine says how far it checked, and repairs
only what it flagged (`wai_quantum::engine::verdict`, `engine::repair`).

## The ladder

| Level | Rank | The engine |
|---|---|---|
| `refused` | 0 | did not read the program, or refused what was asked of it; the refusal ends with its code |
| `parsed` | 1 | read the program and ran nothing |
| `executed` | 2 | ran the program by a method of a stated accuracy class, and checked no intent: none was given, or one could not be decided |
| `evidence` | 3 | checked every intent, at least one by a check that is evidence and not proof (random product states, a statistical test of samples) |
| `proof` | 4 | checked every intent by proof: a tableau, every basis state, checked rewrites, or the exact distribution |

A level says how far the engine checked, not that the program is good.
`holds` says whether the intents held: `true`, `false`, or `null` when one
was not decided. A failure decided by proof is a verdict at `proof` with
`holds: false`. With several intents, the level is the weakest check among
them, and one unchecked intent leaves it at `executed`.

## The layer, and its limits

A level is how far the engine checked. Beside it a verdict names what did
the checking, and what that does not show: its `layer` and that layer's
`limits`, for each intent's check and for the verdict (the weakest of its
checks' layers). A classical checker decides only where it can simulate, so a
verdict never names a stronger layer than the one that held, and says
`unchecked` where none did.

| Layer | The engine | Strength | Its limits |
|---|---|---|---|
| `exact_simulation` | computed the program's exact state, distribution or expectation value and compared; for an energy-landscape job, every arrangement | proof | up to 26 qubits without noise, 16 under the job's noise, every basis state up to 8 qubits, every arrangement up to 16 variables; in double precision; of the program as written, not of a device |
| `stabilizer` | compared stabilizer tableaus | proof | Clifford circuits only, at any width |
| `pass_equivalence` | checked each rewrite or compile pass where it acts, and added the distances | proof | what the rewrite rules reach; an approximate compile holds within its stated bound |
| `reading` | read the two programs: they write different qubits to the bits | proof | decides a difference only |
| `certificate` | held the answer itself to the checks its problem states | evidence | the answer passes those checks; nothing shows no better answer exists |
| `random_states` | ran the two circuits on random product states | evidence | up to 22 qubits, on the inputs drawn |
| `agreement` | held the counts the job returned to the exact distribution it computed of the program | evidence | only where it can simulate the program; a test that does not reject is no proof |
| `samples` | held samples to a stated distribution, by a stated test at a stated level or within a stated distance | evidence | no more than the samples can show; the distribution is the one the intent states |
| `physical_consistency` | held the counts the job returned to a conserved quantity or a set of outcomes | evidence | a necessary condition on the samples returned: nothing of the outcomes' weights, nor of an outcome too rare to be sampled |
| `unchecked` | decided nothing | none | the answer stands on the run alone, by a method of the stated accuracy class |

The result's accuracy class (`exact`, `bit-exact`, `single`, `bounded`,
`estimate`, `heuristic`, `measured`, `simulated`) is under
`executed.accuracy`, beside the layer.

## What a verdict records

A verdict (`"schema": "wai.quantum.verdict/1"`) holds:

- `parsed`: whether the reader took the program, with its warnings and notes
  and their codes, or its refusal;
- `executed`: the method the run took and its accuracy class, and the
  distance a compile put between the program and what ran;
- `intent`: each intent's check, with its `outcome` (`holds`, `fails`,
  `unchecked`), its `strength` (`proof`, `evidence`, `none`), `how` it was
  decided and `why`; for a distribution, its total variation `distance`,
  the `tolerance`, the distribution computed (up to 64 outcomes) and, when it
  fails, the largest gaps;
- `critique`: the program's smells, each with its lines;
- `repairs`: each repair made before the run, with the code it answered,
  the rule, the lines it flagged and every line it changed, before and after;
- `refusal`: for a job that did not run, its message, its code and every
  code it gives;
- `level`, `rank`, `holds`, and `statement`, the level in one plain line;
- `layer` and `limits`, the layer that decided it and what that layer does
  not show; each check under `intent` has its own.

## Asking for one

A job asks for a verdict with `"intent"` (one intent or a list) or
`"verdict": true`. Its result then carries `verdict`, which the receipt seals
with the rest of the result. A job that asks for neither gives the result it
gave before verdicts existed, byte for byte; a test pins it.

The intents:

- `{"distribution": {"00": 0.5, "11": 0.5}, "tolerance": 1e-9}`: the exact
  distribution of the measured bits, keyed by the classical bits with the
  highest bit first. The engine computes it by the state vector (up to 26
  qubits), or under the job's noise by the density matrix, in double
  precision, within the memory the job was admitted with. The plan admits the
  check's memory beside the method's own.
- `{"equivalent": "<the reference program>"}`: the same circuit up to a
  global phase, by the equivalence ladder: equal tableaus for Clifford
  circuits at any width, every basis state up to 8 qubits, checked rewrites
  of one followed by the other's inverse to nothing (all proofs), then random
  product states up to 22 qubits (evidence).
- `{"prepares": "<the reference program>"}`: the same state from |0…0⟩, with
  the same qubits measured into the same bits.

- `{"sampled": {"00": 0.5, "11": 0.5}, "test": "chi_squared", "alpha": 0.01,
  "shots": 4000, "seed": 7}`: the program run as the job runs it (its method,
  noise, options and device target) for that many shots, at the intent's seed
  (or the job's), and the counts held to the distribution by the stated test,
  `chi_squared` (Pearson's) or `g_test` (the likelihood ratio), at the
  significance level `alpha`, with its statistic, degrees of freedom and
  p-value (`engine::stats`). The test, `alpha` and `shots` are required. It is
  evidence, never proof: one holding leaves the level at `evidence`, and an
  outcome of probability 0 that was sampled fails it outright.

- `{"expectation": {"observable": "Z0 Z1", "value": 1.0}, "tolerance": 1e-9}`:
  the observable's exact value on the program's state, by the state vector,
  or under the job's noise by the density matrix, held to `value` within the
  tolerance.

Those are of the program, or of a run made for it. Four more are of the
answer the job returned, its counts, and so can be held to a job on a
provider's device, where no run here stands for the device's:

- `{"outcomes": ["00", "11"], "tolerance": 0}`: the outcomes the answer may
  be. Where the run is the program's own (not a device's, nor a device
  model's) and the engine can compute its distribution, it is decided there,
  by proof; otherwise on the counts: no more than `tolerance` of the samples
  lie outside (evidence, `physical_consistency`).
- `{"conserves": {"weight": 2}}` or `{"conserves": {"parity": 0}}`: a
  quantity every outcome keeps, the sum of its digits or that sum modulo 2,
  as a problem's symmetry gives it. Decided as `outcomes` is.
- `{"counts": {"00": 0.5, "11": 0.5}, "test": "chi_squared", "alpha": 0.01}`,
  or with `"tolerance"` (a total variation distance) in place of the test:
  the job's own counts held to a stated distribution. Evidence (`samples`).
- `{"agrees": "simulation", "test": "chi_squared", "alpha": 0.01}`, or with
  `"tolerance"`: the job's own counts held to the exact distribution the
  engine computes of the program. Evidence (`agreement`), and only where the
  engine can simulate the program.

An intent of the answer is `unchecked` when the job returned no counts, or
when its counts leave out some of its shots.

Any intent may carry a `note`, a `name`, and `"require": "proof"`, which
counts evidence (random product states, a statistical test, samples) as
unchecked.

On a device target the run is the target's noisy model of the device; a
distribution intent is checked on the program's own exact distribution, and
a sampled intent on samples of the target's run. The intent is checked against the program the
job ran (the repaired one, when it asked to be repaired), before any compile;
a compiled run states the compile's distance under `executed`.

An intent that does not read is refused before anything runs, with the code
`J020`.

## Other jobs

- **A provider's job** takes `intent` as a circuit job does. It is read
  before the job is sent, so a job is never paid for and then refused its
  verdict. The program's intents (`distribution`, `expectation`,
  `equivalent`, `prepares`) are checked on the program; the answer's
  (`outcomes`, `conserves`, `counts`, `agrees`) on the counts the device
  returned. A `sampled` intent is refused there (`J020`): it runs the program
  again on the engine, which is no device. On the command line a provider's
  run prints its verdict and ends with its job's own status, whatever its
  intent: the verdict is in what it prints and in its JSON.
- **A qudit job** (`wai.quantum.qudit/1`) takes `intent` of its counts:
  `outcomes`, `conserves` and `counts`, in its digits, `parity` modulo its
  dimension. Its verdict is sealed with its counts, and `wai-quantum qudit
  check` holds it to the request run again.
- **An energy-landscape job** states what its answer must do in its program
  (its checks and its weights) and seals its own word. Read as a verdict,
  `solved` with every arrangement checked holds by `exact_simulation`,
  `solved` on the program's checks alone holds by `certificate`, `missed` and
  `failed` fail, and `unchecked` is `unchecked`.
- **A wasmQ job**'s form holds no intent. It is held to one by a sign-off.

## A sign-off

A job whose form holds no intent, or one another host sealed, is signed off
by a job about it (`engine::signoff`, `wai-quantum signoff JOB --intent
FILE`). Its request is `{"signoff": {"job": {…}, "intent": …}}`; its result
(`"format": "wai.quantum.signoff/1"`) names the job by the digest its own
receipt signs and by its key, repeats its accuracy class, and holds the
verdict: the job's own where the sign-off gives no intent, otherwise the
intent checked against the job (a job with counts and no program the engine
reads is held to its counts). Only a job whose receipt verifies is signed
off, and never one whose sealed verdict or compile differs from what the job
bears out: signed off, that verdict would be sealed again under another key.
The sign-off's receipt seals the job, the intent and the verdict together.
`wai-quantum attest JOB --trust KEYS` (the MCP tool `attest_quantum_job`)
signs off a job another host ran after verifying it as `wai-quantum verify`
does, its key held to a trust list.

## A grade

A grade holds a program to a task's cases and seals what it found as a job
(`engine::grade`, `wai-quantum grade FILE --task T`, the MCP tool
`grade_quantum_program`; quantum-providers §8.10). Its request is
`{"grade": {"task": {…}, "program": "…"}}`. The task is in the form the
course keeps its exercises in: an `intent`, or `cases` (each with its name,
a generator's `inputs` and its intent), a `job` (shots, seed, noise) and
`constraints`, with no solution in it, since the receipt seals the task
whole. Each case is run with its intent and graded `passed` (its intents
hold, as proof or as evidence), `failed` (one does not, or the reader
refuses the program at its inputs) or `unchecked` (no layer decided it), its
verdict and layer beside the grade. The result (`"format":
"wai.quantum.grade/1"`) also gives each constraint measured, what went wrong
and where to look, whether the program passes, and one verdict of every
case's checks, whose layer is the weakest of any case's. A verifier makes
the grade again and works that verdict out again; a grade whose rows,
summary or constraints the checks and the program do not bear out differs.
`wai-quantum check` (`check_quantum_program`) gives the same grade unsealed,
with the reader's diagnostics, the fixes the engine would make and what to
do next: the inner loop, before a grade is sealed.

The job service grades too (`POST /v1/grade`, and the same tool on its MCP
server), sealing with its own key; it holds each case to its bounds before
any case runs, the program written out at that case's inputs, so a wide case
of a generator is refused at that case with the local command. The Python
and npm packages have `grade`, `check` and `attest` as well.

A whole task set is graded at once with
`wai-quantum grade --set FILE --programs ANSWERS` (or
`--course --programs DIR`; `engine::grade::batch`):
one sealed grade a task, a task with no program listed as missing and one
whose grade is refused said so, beside the set's digest and a summary
(`"format": "wai.quantum.grades/1"`, quantum-providers §8.10, *A task set
graded*). The batch itself is not sealed; `wai-quantum verify` checks each
grade and makes it again, and works the digest and the summary out again
from the entries. That a task had no program is the grader's statement.

## Verifying a verdict

A receipt shows the verdict is the one the host sealed. `wai-quantum verify`
(and every surface that verifies a job) then works the verdict out again and
says, under `answer`, how that went (`again`):

| `again` | What it means |
|---|---|
| `reproduced` | every check was decided again here and is the same |
| `partly` | some were; the rest could not be decided here, and are the host's statement |
| `host_statement` | none could be decided here: the verdict is the host's word under its seal |
| `differs` | the verdict is not the one the job bears out: the job is not valid, though its seal holds |

A verdict differs when its level, `holds` or layer is not what its own checks
conclude; when a check names a strength or a layer its method does not have;
when it names a method or an accuracy class the result does not; when it
checks other intents than the request gives; or when a check decided here has
another outcome, or only a weaker strength; a sign-off differs when it
names another job or another accuracy class than the job it holds gives; and
a grade differs when a case's row, its summary or a constraint is not what
its checks and the program give. A
verifier decides a check again within the state vector of 22 qubits (11
under noise); one past that is the host's statement. So is a sampled check
whose samples, drawn again here, are others, when its own sealed samples bear
its outcome out; and a check another version of the engine sealed as a proof
that this one reaches only as evidence. A verdict sealed before layers were named is held to its
level and its checks, and verifies as it did.

## Repairs

`engine::repair` reads the program, takes the reader's refusal (the reader
names one at a time) and, where the fix is mechanical, makes it on the lines
the diagnostic flags, reads the program again, and goes on to the next:

| Code | The rule |
|---|---|
| `Q004` (expected `;`) | a statement that ends its line without `;` gets one |
| `Q102` (no register named …) | the register is declared before the line, with the fewest qubits (bits) its indexed uses need, or the size given (`sizes`) |
| `Q320` (`measure` with no target) | the one bit register of the measured register's size takes it |
| `Q310` (unbounded `while`) | with the bound chosen (`max_iterations`), `@wai.max_iterations N` on a line before it |
| `Q500` (an angle the bit-exact method refuses) | with the bits chosen (`dyadic_bits`), each such angle written as the nearest `τ·m/2^K`, its rounding stated on its line |

It never guesses what a program means. A code with no rule, or a case that is
not the simple one, comes back as "needs a person" with the reader's advice.
A repair someone else wrote (`proposed`) is held to the same lines: a change
to a line no diagnostic flags, or a line added away from one, is rejected
(`J021`) and nothing is changed. The engine finds the flagged lines itself,
by reading the program, never from what the caller says. Every rule's own
edit is held to its lines the same way.

A job may ask to be repaired first (`"repair": true`, or the choices); its
result's `repaired` holds the program that ran and each repair, and its
verdict lists them.

## Refusals

A refusal of what is asked of a program ends with a job-layer code, beside
the reader's: [job-refusals.md](job-refusals.md) has the table. A refused job
still states its level: `refused` when the program does not read, `parsed`
when it reads and the job was refused.

## On every surface

- **Command line:** `wai-quantum run FILE --intent ref.qasm` (or a
  distribution, an intent, a list, or a project's `intent.json`, in JSON)
  prints the verdict and exits 0 when the intent holds, 1 when it fails, 2
  when it is unchecked; `--verdict` asks for one with no intent, `--repair`
  repairs first. `wai-quantum repair FILE` repairs on the flagged lines and
  checks a proposal (`--proposed FILE`). `wai-quantum test` runs a project's
  program with its intents, so the verdict is sealed in the run's receipt.
- **MCP servers (both):** `run_quantum_circuit` takes `intent` and `repair`
  and always states its verdict (an agent's run asks for one, so the request
  it seals says so); `repair_qasm` repairs and checks proposals; every answer
  about a program carries its verdict (`check_equivalent`'s own `verdict`
  came first and keeps its meaning, so its level is under `verification`);
  every refusal carries its codes as data; every tool declares a versioned
  output schema (`wai-quantum://schemas/<tool>/1`). The resource
  `wai-quantum://docs/verdicts` gives this page's contract as data.
- **Job service:** `POST /v1/run` and `/v1/jobs` take `intent` and `repair`,
  checked within the service's bounds; `POST /v1/repair`; every refusal's
  body carries its `code`.
- **Studio and the desktop app:** every run and every assistant proposal
  shows its level as a badge, with the engine's statement.
- **Browser engine:** `engine_verdict(job, intent, memory_mb)` and
  `engine_repair(src, options)`.
- **Python:** `wai_quantum.run(..., intent=...)`, `job.verdict`,
  `wai_quantum.verdict(job, intent)` and `wai_quantum.repair(program, ...)`.

A sealed job is signed off by `wai-quantum signoff JOB --intent FILE`, the
local MCP server's `signoff_quantum_job`, `wai_quantum.signoff(job, intent)`
and the browser engine's `QuantumEngine.signoff(request)`. A sealed verdict
is worked out again by `wai-quantum verify`, the job service's
`POST /v1/verify`, both MCP servers' verify tools, `wai_quantum.verify`
(`.answer`) and the browser's `engine_verify`: each gives the same `answer`.
The job service works a check out again only within what it takes of an
intent's check; past that it still judges the receipt, and says the verdict
is the host's statement. The resource `wai-quantum://docs/verdicts` lists
the layers with their limits.

## Conformance

[`quantum-conformance/intents/`](../../quantum-conformance/intents/) holds
programs, their intents and the verdict each must get, every level of the
ladder and every intent covered, with distances known by hand. It runs
natively, under wasm32-wasip2 and in the browser build.
[`quantum-conformance/verdicts/`](../../quantum-conformance/verdicts/) holds
sealed jobs and what a verifier must say of each one's verdict: one that
holds, one that fails, one no layer decided, one past what a verifier
simulates, and ones a host sealed that the job does not bear out. The
contract is [quantum-providers §8.9](../../extensions/quantum-providers.md).