open-standards/wai/quantum-cross-simulator/README.md

# Cross-simulator conformance corpus

WAI Quantum's engine, run beside other simulators on a published set of circuits, every answer
held to a reference within a tolerance derived from the arithmetic that produced it. One
command regenerates every result here on a clean machine.

```sh
bash wai/quantum-cross-simulator/reproduce.sh --tier full   # the whole corpus, nightly in CI
bash wai/quantum-cross-simulator/reproduce.sh               # the CI tier, a few minutes
bash wai/quantum-cross-simulator/reproduce.sh --self-test   # every injected fault must be caught
bash wai/quantum-cross-simulator/reproduce.sh --tier gpu    # the GPU members, on a host with a GPU
bash wai/quantum-cross-simulator/reproduce.sh --perf --out perf.json
```

It needs macOS on arm64 or Linux on x86_64, `curl`, `tar`, a C compiler and rustup; no account
and no key. It fetches the pinned
uv release (by SHA-256), CPython 3.12.16 through it, and the engines of
[`env/requirements.lock`](env/requirements.lock), every wheel checked against its hash. It builds
the engine from this checkout with `--locked`, runs every check, prints each failure with its
measured value and tolerance, and ends with `PASS` or `FAIL` (exit 0 or 1). `QCORPUS_CACHE` says
where the downloads go.

## What is compared

| engine | version | methods compared here |
|---|---|---|
| wai-quantum (this repository) | the checkout | bit-exact fixed point (`exact`), double-precision state vector, stabilizer tableau, matrix product state, tensor network, density matrix, error-correction frames and models, and the whole job path with method `auto`; and, each against an exact reference of its own (below), Pauli propagation, the stabilizer-rank and cut-rank sums, trajectories, Pauli frames under noise, mitigation (`zne`, `vd`, `readout`), pulses, two decoders (exact matching, union-find), and on a GPU the fabric's members (WebGPU, Metal, CUDA) and the single-precision state vector and cut-rank table |
| Qiskit Aer <!-- allow-name: an open-source simulator compared, named as data --> | 0.18.2, with Qiskit 2.6.2 | state vector (built in code, and read from the published OpenQASM 3 text), stabilizer, matrix product state, density matrix |
| Stim <!-- allow-name: an open-source simulator compared, named as data --> | 1.16.0 | tableau, sampler, detector error models, the reference generator of the error-correction circuits |
| quimb <!-- allow-name: an open-source library compared, named as data --> | 1.15.0 | dense state, tensor network contracted, matrix product state |
| MQT DDSIM <!-- allow-name: an open-source simulator compared, named as data --> | 2.7.0, with mqt.core 3.11.0 | decision-diagram state vector and sampler |
| qsim <!-- allow-name: an open-source simulator compared, named as data --> | qsimcirq 0.22.1, with Cirq 1.7.0 | single-precision state vector and sampler |

Each engine runs every job in a process of its own ([`tools/engines.py`](tools/engines.py)). A
third-party engine takes each gate by its own name for it where it has one, so a gate it
defines differently shows; where it has none, the adapter says so in the result and gives it the
gate's matrix. The engine under test is driven through
[`wai-quantum-cli/examples/cross_simulator.rs`](../wai-quantum-cli/examples/cross_simulator.rs),
which calls what the command line and the job service call.

## The corpus

[`corpus/manifest.json`](corpus/manifest.json) lists every circuit with its parameters, the
reason it is there, what it is held to and the SHA-256 of its OpenQASM 3 text;
[`tools/corpus.py`](tools/corpus.py) writes it, and a run fails if the generator no longer gives
those bytes. Programs up to 16 KiB are committed in [`corpus/qasm/`](corpus/qasm/), and the
error-correction circuits, which are the pinned reference generator's, in
[`corpus/qec/`](corpus/qec/); a run checks that the generator still writes them. Random
circuits draw from SHA-256 in counter mode, and every angle is written exactly: `m/2^k` of a
turn, a rational fraction of a turn, or a decimal number of radians.

| family | what it exposes |
|---|---|
| bit order | which index an X on qubit 0 reads as; named classical bits out of order; a state whose every amplitude differs, so any exchange of qubits shows; an empty register; index bit 19 |
| GHZ, QFT, Grover, phase estimation | closed forms: two amplitudes of 1/√2; e^{2πixy/2^n}/2^(n/2) with every controlled phase to 2π/2^n; sin²((2k+1)θ); an outcome of probability 1, and for a phase of 1/3 the closed law, which the bit-exact method must refuse to round |
| random Clifford+T, dyadic, arbitrary angles | universal circuits in the table's gates, rotations that become many table phases in fixed point, and decimal angles no dyadic grid holds |
| mirror | U then U†, at widths past the integer reference: the answer is \|0…0⟩ exactly |
| small angles | phases of 2π/2^k for k up to 32, the table's last entry, and below it to k = 60, each with a check of whether the engine held the phase or dropped it; p(2π/2^19) applied 2^18 times (a half turn, whose fixed-point result the extension states) and 2^17 times (a quarter turn), each held to the state with its phase |
| gate conventions | every gate of OpenQASM 3's standard library, and modifiers, on a state with no symmetry; rotations by whole turns, whose sign is the convention; a controlled global phase |
| measurement in the middle | teleportation, reset of an entangled qubit, measuring twice, feedback, a GHZ state by fusion, repeat-until-success: exact laws from every branch |
| stabilizer circuits | GHZ and cluster states to 5000 qubits; random Clifford layers at 63, 64, 65, 127, 128 and 129 qubits (either side of a machine word) and to 2000; a Clifford circuit as a state vector and a tableau; outcomes at 40 qubits |
| noise | depolarizing channels after X and cx chains, whose expectation values are closed forms |
| wide circuits | GHZ to 100 qubits and the QFT at 30 and 40 by matrix product states and contraction, and at 30 at a bond far above what it reaches; a shallow brickwork at 26; a 40-qubit mirror |
| error correction | the rotated surface-code memory experiment in Z (d = 3 … 15) and X, the unrotated layout, repetition codes to d = 25, a noiseless run, a colour code whose cycle turns the axes with `C_XYZ`, and hand-written circuits for inverted targets, tags, flip noise, merging, Pauli observables, a correlated error, a general Pauli channel, and two circuits that must be refused |
| past 14 qubits | the QFT at 18, 20 and 22, Clifford+T at 16, decimal rotations at 18 and dyadic ones at 20, each held whole to the integer state |
| Pauli propagation | random rotations, Clifford+T, a kicked Ising chain and dyadic rotations, 6 to 12 qubits, with observables of weight 1 to 6 and a coefficient |
| stabilizer rank | Clifford+T circuits with 1 to 15 T gates (Toffoli and CCZ among them), summed exactly and sampled |
| cut rank | a Clifford circuit and a layer of `u` gates at decimal angles, 8 to 20 qubits, and the same sampled |
| noise | GHZ, decimal rotations and a Clifford circuit under depolarizing, dephasing, amplitude-damping and readout noise |
| mitigation | zero-noise extrapolation by folding, virtual distillation, readout inversion |
| pulses | a resonant drive, a drive 13 MHz off resonance, two drives about orthogonal axes, an envelope of explicit samples |
| decoders | the repetition code at d = 3 and the rotated surface code at d = 3 and 5: every single fault, pairs of faults, and seeded sets of two to six |

## How a result is judged

The references are exact or as good as exact, and share no code with any engine
([`tools/exact.py`](tools/exact.py), standard library only):

- **A state vector in integers.** Amplitudes are integers at scale 2^-256; sine and cosine come
  from an integer π and an integer series. Every run checks that it gives the extension's phase
  table, P(1) … P(32) rounded to the nearest 2^-30, entry for entry against the referee's, and
  that its states meet the closed forms within 2^40 units of 2^-256: the QFT's amplitudes, GHZ's,
  Grover's sin²((2k+1)θ) (computed in integers), phase estimation's outcome of probability 1, and
  its law for a phase of 1/3.
- **Exact distributions.** A program that measures in the middle is walked branch by branch.
- **An integer stabilizer tableau**, and the reduced row echelon form of a group with its signs, so
  that two descriptions of a stabilizer state compare as bytes.
- **The referee** ([`../quantum-conformance/tools/referee.py`](../quantum-conformance/tools/referee.py)) reruns the bit-exact method's
  op-log in its own integer arithmetic and must give the engine's integers and hash, on every
  circuit the method runs whose op-log is at most 2^30 amplitude updates (its length times 2^n,
  a few minutes of the referee's time): every one to 14 qubits, and the shorter op-logs past it. Beside it, the op-log run in double precision must
  give the circuit's exact state, modulo a global phase, within that run's rounding and the
  distance the reader's rule moves angles: so the integers are tied to the op-log, and the op-log
  to the circuit.

Each tolerance comes from [`tools/tolerance.py`](tools/tolerance.py), and none is fitted to a
result:

- **The bit-exact method**: the extension's published bound (quantum-sim.md §5, at most 1 + √2/2
  units of 2^-30 per component per inexact gate), summed over the op-log the engine ran.
- **Floating point**: a gate of width w costs C(w)·u in the Euclidean norm, C(w) = (2√2 + (2^w − 1)
  + 2 + π)·2^(w/2), from the complex products, the row sum, the matrix entries and the angle's
  rounding; u = 2^-53 in double and 2^-24 in single precision; widths are each engine's fusion.
  Contractions, decision diagrams (whose merging tolerance, 2^-42, is read from mqt.core's header)
  and matrix product states (with LAPACK's unstated SVD constant taken as 16·m, and the weight
  their truncation can drop) are stated in the same file.
- **Matrix product states**: the rounding of each two-site update's SVD (a gate on qubits d apart
  takes 2d − 1 of them, its routing swaps included), and 2√w for each truncation of weight w, as
  each engine reports what it dropped: the engine under test its summed weight over its
  truncations, Aer every truncation from its log, and quimb run with an absolute cutoff whose
  most per update is known. A row whose bound is not below half the amplitudes it checks is
  recorded and not judged.
- **Sampled counts**: every outcome's count against its exact probability by Bernstein's
  inequality, and the whole distribution against the exact one in L1 distance, twice the total
  variation (the bound of Weissman and others, 2003, which holds for any number of outcomes); an
  outcome of probability 0
  that appears at all fails. A Clifford circuit's outcomes at 40 qubits: every shot inside the
  affine space its stabilizers fix, no outcome more often than its probability allows, and each
  bit the space leaves free reading 1 half the time.
- **Error-correction models**: the same symptom sets, probabilities within 16u per noise site of
  the circuit; detection rates against the rates the model gives, by Bernstein's inequality.
- Every sampled test of a run (counts, stabilizer outcomes, detection rates, and the estimates of
  the methods below) takes a share of one family-wise error rate of 10^-6, a quarter to each kind,
  so a correct engine fails a run on sampling with probability at most 10^-6.

### The engine's other methods

Each is held to an exact reference, and to the accuracy the method itself states: its bound where
it states one, the rounding of its arithmetic (derived in [`tools/tolerance.py`](tools/tolerance.py))
where it states exactness, and, for an estimate, a tail bound on its samples that needs no normal
approximation. Two more references serve them, standard library only: **an integer density
matrix** (`exact.Density`: ρ as a state of 2n qubits in integers at scale 2^-256, each channel a
sum of Kraus terms, the noise placed where the engine's model places it), and **the least-weight
correction of a syndrome, by exhaustion** ([`tools/decoding.py`](tools/decoding.py): shortest paths
for each observable mask, then every pairing of the fired detectors).

| method (engine) | reference | pass criterion |
|---|---|---|
| Pauli propagation (`wai.pauli`) | `<P>` of the integer state | at each threshold of its ladder (0, where only exact zeros drop, and 10^-5 or 10^-3 with the three decades above), \|value − exact\| ≤ the truncation bound it states + its rounding, \|coeff\|·√T·(16R + T)·u for T terms at the peak and at most R rotations (the coefficients' 2-norm is the observable's, and a rotation turns pairs of them); at threshold 0 the bound must be 0 |
| stabilizer rank, exact (`wai.stabrank`) | the integer state's amplitudes; `<P>` | the sum's one evaluation in double precision: (2t + 2n + 13)·u·sec(π/8)^t, with 2t T gates for an expectation value's sandwich |
| stabilizer rank, sampled | the integer state's amplitudes | each draw states a bound that holds with a stated confidence; over 10 seeds the bound may fail no more often than a binomial count at that confidence allows |
| cut rank (`wai.cutrank`) | the integer state's amplitudes (every one at 8 qubits, 256 chosen at 12, 64 at 16 and 20); the exact distribution | (11n + 2)·u·2^(n/2) per amplitude, √m of it over m amplitudes in the 2-norm modulo a global phase; counts as every sampler's |
| density matrix (`wai.density_matrix`), noisy | the integer density matrix: `<P>`, the law of the bits with readout flips | √(2^n)·16·C(4)·u per superoperator it applies; counts as every sampler's |
| trajectories (`wai.trajectories`), Pauli frames (`wai.stabilizer` with noise) | the same | the mean of N samples each in [−\|c\|, \|c\|] within Hoeffding's bound, \|c\|·√(2 ln(2/α)/N), and a trajectory's own rounding; counts as every sampler's |
| zero-noise extrapolation (`wai.mitigate`, `zne`) | each folded program's noisy value from the integer density matrix, and the Richardson extrapolation of those exact values | each scale's value from its shots within Hoeffding's bound; the extrapolation within Σ\|c_i\| times those bounds; the ideal value it states within the state vector's rounding |
| virtual distillation (`vd`) | Tr(ρ²) and Tr(ρ²P)/Tr(ρ²) from the integer density matrix | the purity it states within twice the density matrix's rounding; the ratio within what Hoeffding's bounds on its numerator and denominator allow |
| readout inversion (`readout`) | the law of the bits before the flips | every quasi-probability within Hoeffding's bound on its shots, each shot's share spanning (1 − 2p)^-k(1 − p)^(k−1) |
| pulses (`wai.pulse`) | the Rabi formula, r²/g²·sin²(πgt), in integers | P(0) and P(1) within the bound the run states (its series' truncation, proved, and its rounding, estimated) |
| exact matching (`wai.matching`) | the least weight of a correction for each observable mask | the prediction's least weight within the weights' rounding of the least of all: (detectors + 1)/16384 for each fired detector |
| union-find (`wai.union_find`) | the same | every single edge of the graph corrected exactly, as it states; how often it agrees with a least-weight correction is recorded, not judged (it states no least weight) |
| the fabric's members (`wai.fabric[…]`, gpu tier) | the CPU's bit-exact integers, which the state checks hold to the integer reference | the same integers and the same hash, bit for bit |
| single precision on the GPU (`wai.statevector_single`, `wai.cutrank_single`, gpu tier) | the integer state's amplitudes | the double-precision bounds with u = 2^-24 |

A state is held to its reference modulo a global phase, which no measurement sees; the
distance with the phase is recorded beside it (`strict`). Where the phase is the result (the
repeated phases), the verdict is the distance with it. The bounds are worst cases and so are
loose: each result records how far inside its bound an engine is, and the table below gives the
largest such ratio. Below an engine's bound a distance cannot tell a small-angle gate from its
absence, so each small-angle circuit also asks of every engine whether it held the gate's phase
or dropped it, exactly, in integers.

The engine under test is held to answering. A crash, a time-out, or a refusal that
[`corpus/expected-refusals.json`](corpus/expected-refusals.json) does not list for that circuit in
those words fails the run; an answer to a circuit that file says it must refuse fails it too. The
file lists the bit-exact method's refusals by its reader's rule (an angle no grid of 2^32 holds
within 10^-14 of a turn), computed from the circuits' exact angles, and four error-correction
circuits. Another engine's refusal or crash is recorded as `not run`, with its words.

A run can be seen to fail. `--self-test` runs one circuit for each fault `compare.py` can inject,
and passes only if each fails the run on the check it breaks, on the engine it was put into, and
on no other engine:

- an amplitude moved by 10^-9; the qubits of a state reversed; one fixed-point bit flipped; a
  stabilizer's sign flipped; a mechanism's probability raised by one part in 10^6;
- 5% of an outcome's shots moved to another that can occur (caught by the per-outcome test, on a
  distribution with variance); every outcome's count moved by a tenth of itself, up and down in
  turn (caught only by the L1 distance); every shot of a 40-qubit Clifford sampler given to one
  outcome; a detector's count raised by a quarter; a sampler's answer with no observable;
- a phase dropped from the bit-exact op-log, its integers and hash made again to match (caught
  only by the op-log against the circuit); a small-angle gate dropped from a double-precision
  state, inside its bound (caught only by the held-or-dropped check); a half turn on a repeated
  phase (caught only by the distance with the phase); a global phase on the bit-exact state
  (caught by the closed-form amplitudes, which it was once exempt from); a phase of 0.5 rad on a
  matrix-product amplitude (caught by a bound charged what that engine truncated);
- the engine under test refusing, crashing on a job or on a model, hanging, taking an angle it
  must refuse, or refusing in words no list holds, or in other words than its list's;
- Pauli propagation's values moved by twice their truncation bound (and 10^-9); an exact
  stabilizer-rank amplitude moved by 10^-9; a cut-rank amplitude moved by 10^-8; trajectories'
  means moved by 0.05; a noisy density matrix's values moved by 10^-8; Pauli frames' values
  negated; an extrapolation replaced by its value at scale 1; readout's
  quasi-probabilities replaced by the raw frequencies; the purity raised by 10^-6; a pulse's P(1)
  moved by ten times its bound; every prediction of exact matching, and of union-find, flipped;
  and in the gpu tier, one bit of each GPU member's state flipped.

## Results

The committed runs, one file per host, are in [`results/`](results/): every check with its
measured value and tolerance, the host and every version. Each file names the git trees of the
engine and of the tools that produced it, as they were on disk (`git rev-parse
<commit>:wai/wai-quantum` gives the same id for any commit holding the same engine). A cell
below is checks passed of those made, with the largest ratio of measured value to tolerance
among the passes in brackets.

### What agreed

On both hosts, every engine met its reference within its tolerance on every circuit it ran, with
the exceptions listed next. That covers the engine under test's bit-exact, double-precision,
stabilizer (tableau and Pauli frames), matrix-product, tensor-network, density-matrix,
error-correction, Pauli-propagation, stabilizer-rank, cut-rank, trajectory, mitigation (three
methods), pulse and decoding (two decoders) methods, and eleven methods of the others, on states
(held whole to the integer state to 22 qubits), sampled counts, stabilizer groups to 5000 qubits,
amplitudes to 100 qubits, noisy expectation values and laws under every channel of the noise
model, mitigated values, pulse populations, decoded syndromes to distance 5, and detector error
models and detection rates to distance 15. The engine under
test answered every circuit, and refused only those
[`corpus/expected-refusals.json`](corpus/expected-refusals.json) lists, in the words it lists.

The bit-exact method's integers were reproduced bit for bit by the referee on every circuit whose
op-log is at most 2^30 amplitude updates: every one to 14 qubits, the QFT at 18 and 20, Clifford+T
at 16 and the mirror at 16 (the 22-qubit QFT, the 20-qubit mirror and the 20-qubit dyadic
rotations are past it, and recorded as not rerun); its op-log, run in double precision, gave each
circuit's exact state; and its
distance from the ideal state stayed inside the extension's published bound everywhere. On
`repeated_phase_k19`, p(2π/2^19) applied 2^18 times to |1⟩, its amplitude ends with modulus
1.0000188705 and 8.99·10^-6 rad from the half turn (a distance of 2.09·10^-5 from −|1⟩), where the
extension states modulus 1.0000189 and within 10^-5 rad. These are integers, the same on both hosts.

### Disagreements with the references

- **The engine under test, wrong; fixed.** `quantum_frame::surface_code_memory` documents that
  its text is the reference generator's, byte for byte. At p = 0 it was not: it wrote every noise
  instruction with probability 0 and the first round's resets on two lines, where the reference
  writes no noise instruction and one reset line (`surface_z_d5_noiseless`). The two circuits
  meant the same, so models and samples agreed; the text did not. Fixed in the engine, with a
  test against the reference's text (`wai-quantum/tests/frame_reference.rs`).
- **Another engine.** `ddsim.statevector` on `measure_twice` (h, then the same qubit measured
  into two bits) returns 00 and 10, where a repeated measurement repeats: 00 and 11. On
  `repeat_until_success` it raises an error and gives no counts. Each claim about it, with the
  smaller programs that show what the error needs, is a case of a reproducer committed beside
  [`corpus/known-disagreements.json`](corpus/known-disagreements.json)
  ([`corpus/reproducers/`](corpus/reproducers/)), which every run that holds the circuit reruns;
  the two checks are reported as `known`. Only other engines may be listed there.
- **A single-precision engine, past its bound at 20 qubits, on one host.** On macOS on arm64, on
  `dyadic_n20_k16` (148 rotations by m/2^16 of a turn and cz gates), the single-precision state
  vector is 2.3·10^-4 from the integer state, past the 1.9·10^-4 that its gates allow
  (tolerance.py), and nearly all of it is its norm, 1.00023; the circuit library's own complex64
  simulator is as far there. The norm grows in runs of the whole circuit prefix, not in a gate
  applied by hand in single precision to the state before it; the same construction at 16
  qubits is within its bound; and on Linux on x86_64, with the same pinned versions, the
  distance is 5.5·10^-7. Not yet explained; each claim is a case of a reproducer
  (`corpus/reproducers/qsim_single_precision_norm.py`), and the check is reported as `known`
  where it shows (on Linux the listed disagreement is recorded as no longer showing).
- **The reference, wrong once.** While this corpus was written, the integer reference's `ryy`
  used the wrong fixed rotation. It showed as every engine agreeing with every other and none
  with the reference, and was corrected before any result here was recorded. A reference is
  examined first when that pattern appears.

### Conventions the corpus shows

- **Global phase.** Every engine keeps OpenQASM 3's global phases except the bit-exact method,
  which lowers `rz`, `rx`, `ry` and `U` through phase gates and so differs from the standard's
  matrices by a global phase (a `strict` distance of up to 1.97 on a rotation by a whole turn).
  No measurement sees it, and under a control the method is right (`controlled_rz_full_turn`,
  `gate_crz`, `gate_cu`). A result on this method states the phase (`accuracy.global_phase`,
  exactly, in 2^33rds of a turn), and the corpus holds the method's state to the reference's with that
  phase, nothing fitted.
- **Angles below the table.** The bit-exact method reads an angle within 10^-14 of a turn of a
  dyadic one as that dyadic. A phase of 2π/2^k is therefore refused for k = 33 … 46 (`[Q500]`),
  and for k ≥ 47 runs as no phase at all, labelled bit-exact: its op-log is the circuit's with the
  angle moved by 2^-k of a turn, which the op-log check allows by the reader's rule, and its state
  is within 2.3·10^-14 of the ideal one, far inside the method's resolution. The result and the
  plan say so (`accuracy.below_table`, with how many and the largest), and an angle moved to a
  dyadic neighbour other than zero is stated too (`accuracy.moved_angles`). Method `auto` sends
  k = 33 … 46 to double precision.
- **Held or dropped.** Every double- and single-precision engine here held the phase at every k
  where its answer can say. The decision-diagram engine dropped it at k = 47, 48, 52 and 60, each
  an angle below its parameter tolerance of 10^-13 rad, and held it at k ≤ 40. At k = 60 the phase
  moves the state by 2.7·10^-18, less than half of most answers' own rounding: those are recorded
  as unable to say, and the single-precision engine's answer, whose rounding there was smaller,
  held it.
- **Bit order.** Every engine's answers were brought to one convention (qubit 0 the lowest bit of
  an index, classical bit 0 the last character of an outcome) by its adapter, and the bit-order
  family holds each adapter to it; the self-test's reversed state fails there.

### Run time and memory

The performance tables below are one shared host each, on one day; read the Limits before any
figure. Each row is one engine on one case, as data:

- **threads**: what the engine was given. `all (32)` is every CPU the process may run on, which
  the engine under test's state vector and tensor network use; `1 (no threads)` a method with no
  threads of its own (the engine under test's bit-exact, stabilizer, matrix-product-state and
  error-correction methods); a number, the count passed in the engine's own setting; `not set`,
  an engine the adapter gives no thread setting; `1 (pinned)`, the process held to one CPU
  (Linux only).
- **ops**: the operations the engine ran, where it reports them: the driver's count for the
  engine under test, and the gates of Aer's circuit after its conversion.
- **seconds**: the engine's own call, the median of the runs with the minimum in brackets. The
  list above each table says where each engine's timer starts and stops.
- **peak MiB**: the peak resident set of the process that ran the engine (the driver for the
  engine under test; for another engine, its adapter, which holds the interpreter and its
  packages), as the system counts it for that process alone; the baseline beside it is the same
  measure given a two-qubit job.

The last table runs the same harness on the engine under test built before and after a gate
under many controls stood whole on its state vector, one build straight after the other on one
host; `ops` gives the operations each build ran. `wai.exact_whole` is the bit-exact method asked
to hold such a gate whole (`"options": {"controls": "whole"}`), which gives other bits than its
default; `wai.exact`, its default, runs standard gates and gives the bits it gave before. Past 16
controls the standard form refuses (`[Q208]`), so `grover_n20` has no "before".

<!-- results: written by tools/report.py from results/*.json; do not edit by hand -->

### `conformance-linux-x86_64.json`

Host: AMD RYZEN AI MAX+ 395 w/ Radeon 8060S, Ubuntu 26.04.1 LTS, 32 logical CPUs, 123 GiB, x86_64. Tier full, 4600.4 s. Checks: info 19, known 12, not run 17, pass 1970, refused 11.

| family | `wai.exact` | `wai.statevector` | `wai.stabilizer` | `wai.auto` | `wai.mps` | `wai.tensor_network` | `wai.density_matrix` | `wai.qec` | `wai.pauli` | `wai.stabrank` | `wai.cutrank` | `wai.trajectories` | `wai.mitigate` | `wai.pulse` | `wai.matching` | `wai.union_find` | `aer.statevector` | `aer.qasm_import` | `aer.stabilizer` | `aer.mps` | `aer.density_matrix` | `stim.tableau` | `quimb.dense` | `quimb.tensor_network` | `quimb.mps` | `ddsim.statevector` | `qsim.statevector` | `pairs and references` | `references` |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| bit_order | 24/24 (0.068) | 8/8 (0.015) | 12/12 | 8/8 (0.0069) |  |  |  |  |  |  |  |  |  |  |  |  | 8/8 (0.016) | 6/6 (0.016) | 5/5 |  |  | 7/7 | 6/6 (0.019) | 6/6 (0.031) |  | 8/8 (0.00015) | 8/8 (0.013) | 15/15 (0.0084) | 5/5 |
| clifford_large | 4/4 (0.012) | 1/1 (0.00077) | 34/34 (0.43) | 1/1 (7.8e-05) |  |  |  |  |  |  |  |  |  |  |  |  | 1/1 (0.00084) | 1/1 (0.00084) | 16/16 (0.51) |  |  | 19/19 (0.3) | 1/1 (0.0017) | 1/1 (4.4e-08) |  | 1/1 (2.4e-07) | 1/1 (0.00052) |  | 15/15 |
| cutrank |  |  |  |  |  |  |  |  |  |  | 6/6 (0.56) |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |
| decoders |  |  |  |  |  |  |  |  |  |  |  |  |  |  | 3/3 | 3/3 |  |  |  |  |  |  |  |  |  |  |  |  |  |
| dynamic |  | 12/12 (0.29) | 8/8 (0.21) | 12/12 (0.29) |  |  |  |  |  |  |  |  |  |  |  |  | 12/12 (0.36) |  |  |  |  | 8/8 (0.29) |  |  |  | 8/8 (0.22) · 11 known | 6 not run |  |  |
| gate_conventions | 188/188 (0.069) | 47/47 (0.017) |  | 47/47 (0.066) |  |  |  |  |  |  |  |  |  |  |  |  | 47/47 (0.018) | 47/47 (0.018) |  |  |  |  | 46/46 (0.017) · 1 not run | 46/46 (0.032) · 1 not run |  | 47/47 (0.00015) | 46/46 (0.026) · 1 not run |  |  |
| ghz | 20/20 (0.042) | 7/7 (0.21) | 2/2 (0.022) | 7/7 (0.022) |  |  |  |  |  |  |  |  |  |  |  |  | 7/7 (0.058) | 5/5 (0.011) |  |  |  | 2/2 (0.29) | 5/5 (0.0094) | 5/5 (0.041) |  | 7/7 (0.017) | 7/7 (0.14) |  | 5/5 |
| grover | 16/16 (0.029) | 6/6 (0.12) |  | 6/6 (0.14) |  |  |  |  |  |  |  |  |  |  |  |  | 6/6 (0.25) | 4/4 (0.0043) |  |  |  |  | 4/4 (0.0079) | 4/4 (0.00031) |  | 6/6 (0.16) | 6/6 (0.27) |  | 4/4 |
| mirror | 8/8 (0.007) · 1 not run | 3/3 (0.00036) |  |  |  |  |  |  |  |  |  |  |  |  |  |  | 3/3 (0.00068) |  |  |  |  |  | 3/3 (0.00036) |  |  | 2/2 (7e-05) · 1 not run | 3/3 (0.00057) |  |  |
| mitigation |  |  |  |  |  |  |  |  |  |  |  |  | 18/18 (0.23) |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |
| noise |  |  |  |  |  |  | 3/3 (0.00093) |  |  |  |  |  |  |  |  |  |  |  |  |  | 3/3 (0.00093) |  |  |  |  |  |  |  |  |
| noisy |  |  | 11/11 (0.34) |  |  |  | 16/16 (0.24) |  |  |  |  | 16/16 (0.41) |  |  |  |  |  |  |  |  | 3/3 (0.0003) |  |  |  |  |  |  |  |  |
| other |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  | 3/3 |
| pauli |  |  |  |  |  |  |  |  | 81/81 (0.11) |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |
| pulse |  |  |  |  |  |  |  |  |  |  |  |  |  | 8/8 (0.0013) |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |
| qec |  |  |  |  |  |  |  | 29/29 (0.6) · 2 refused as listed |  |  |  |  |  |  |  |  |  |  |  |  |  | 22/22 (0.53) · 2 not run |  |  |  |  |  | 44/44 (0.016) |  |
| qft | 36/36 (0.042) · 1 not run | 12/12 (0.58) |  | 11/11 (0.66) |  |  |  |  |  |  |  |  |  |  |  |  | 12/12 (0.61) | 6/6 (0.009) |  |  |  |  | 10/10 (0.0062) | 5/5 (0.0057) |  | 12/12 (0.64) | 12/12 (0.63) | 15/15 (0.0027) | 8/8 |
| qpe | 12/12 (0.017) · 2 refused as listed | 5/5 (0.0091) |  | 5/5 (0.013) |  |  |  |  |  |  |  |  |  |  |  |  | 5/5 (0.0058) | 5/5 (0.0075) |  |  |  |  | 5/5 (0.0071) | 4/4 (0.0014) |  | 5/5 (4.8e-06) | 5/5 (0.0034) |  | 5/5 |
| random_clifford_t | 16/16 (0.039) | 4/4 (0.0047) |  | 4/4 (0.017) |  |  |  |  |  |  |  |  |  |  |  |  | 4/4 (0.0042) | 3/3 (0.0042) |  |  |  |  | 4/4 (0.003) | 3/3 (0.00027) |  | 4/4 (3e-05) | 4/4 (0.0023) |  |  |
| random_dyadic | 11/11 (0.0076) · 1 not run | 3/3 (0.0019) |  | 3/3 (0.0016) |  |  |  |  |  |  |  |  |  |  |  |  | 3/3 (0.0027) | 2/2 (0.0036) |  |  |  |  | 3/3 (0.0019) | 2/2 (0.00016) |  | 3/3 (6.8e-06) | 3/3 (0.004) · 1 known |  |  |
| random_rotations | 4 refused as listed | 6/6 (0.48) |  | 6/6 (0.48) |  |  |  |  |  |  |  |  |  |  |  |  | 6/6 (0.49) | 3/3 (0.0034) |  |  |  |  | 4/4 (0.0021) | 3/3 (0.00014) |  | 6/6 (0.55) · 1 not run | 6/6 (0.4) | 6/6 (0.0017) |  |
| small_angle | 62/62 (0.9) · 3 refused as listed | 32/32 (0.016) | 2/2 | 18/18 (0.098) |  |  |  |  |  |  |  |  |  |  |  |  | 32/32 (0.018) | 29/29 (0.016) | 1/1 |  |  | 1/1 | 30/30 (0.019) | 29/29 (0.016) |  | 29/29 (0.15) | 33/33 (0.0075) |  | 1/1 |
| stabrank |  |  |  |  |  |  |  |  |  | 9/9 (0.039) |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |
| wide_mps |  |  |  |  | 6/6 (0.00017) | 5/5 (0.00057) |  |  |  |  |  |  |  |  |  |  |  |  |  | 4/4 (0.00023) · 1 not run |  |  |  | 5/5 (0.043) | 5/5 (7.3e-05) |  |  |  |  |
| wide_tn |  |  |  |  | 1/1 (0.00093) | 1/1 (6.4e-07) |  |  |  |  |  |  |  |  |  |  |  |  |  | 1/1 (0.0021) |  |  |  | 1/1 (0.02) | 1/1 (0.0011) |  |  | 21/21 (0.02) |  |

Known disagreements of other engines with the exact reference (`corpus/known-disagreements.json`):

- `repeat_until_success`, `ddsim.statevector`: ValueError: [changePermutation] Target must be an injective mapping using only keys and logical qubits from the source permutation. | known: MQT DDSIM 2.7.0's qasm_simulator raises ValueError [changePermutation] on this program (two qubits; qubit 1 measured three times in the middle, with gates between and an x conditioned on each reading; four classical bits) and gives no counts. It raises the same without the conditioned x, and on qubit 1 measured, h, measured again, then qubit 0 measured into a third bit; with no third bit it gives counts. Each is a case of the reproducer.
- `measure_twice`, `ddsim.statevector`: 10: 49842 shots, probability 0; 11: 0 shots, expected 50000.0 ± 1095.4 | known: MQT DDSIM 2.7.0's qasm_simulator, given h q[0]; c[0] = measure q[0]; c[1] = measure q[0]; with nothing between the two measurements, returns 00 and 10: c[0] is never set. With x in place of h it gives 10 on every shot; with a barrier between the measurements, 00 and 11; with c[1] measured first and c[0] second, ValueError [changePermutation]. Each is a case of the reproducer, with the circuit built in Qiskit 2.6.2.
- `measure_twice`, `ddsim.statevector`: known: MQT DDSIM 2.7.0's qasm_simulator, given h q[0]; c[0] = measure q[0]; c[1] = measure q[0]; with nothing between the two measurements, returns 00 and 10: c[0] is never set. With x in place of h it gives 10 on every shot; with a barrier between the measurements, 00 and 11; with c[1] measured first and c[0] second, ValueError [changePermutation]. Each is a case of the reproducer, with the circuit built in Qiskit 2.6.2.
- `measure_twice`, `ddsim.statevector`, reproducer (corpus/reproducers/ddsim_repeated_measurement.py): h, then c[0] and c[1] measured from the qubit with nothing between: only 00 and 10 come — saw ['00', '10']
- `measure_twice`, `ddsim.statevector`, reproducer (corpus/reproducers/ddsim_repeated_measurement.py): x, then the same two measurements: 10 on every shot — saw ['10']
- `measure_twice`, `ddsim.statevector`, reproducer (corpus/reproducers/ddsim_repeated_measurement.py): h, the two measurements with a barrier between: 00 and 11 — saw ['00', '11']
- `measure_twice`, `ddsim.statevector`, reproducer (corpus/reproducers/ddsim_repeated_measurement.py): h, then c[1] measured first and c[0] second: an error naming changePermutation — saw ValueError: [changePermutation] Target must be an injective mapping using only keys and logical qubits from the source permutation.
- `repeat_until_success`, `ddsim.statevector`, reproducer (corpus/reproducers/ddsim_feedback_rounds.py): repeat_until_success: an error naming changePermutation, and no counts — saw ValueError: [changePermutation] Target must be an injective mapping using only keys and logical qubits from the source permutation.
- `repeat_until_success`, `ddsim.statevector`, reproducer (corpus/reproducers/ddsim_feedback_rounds.py): the same three rounds without the conditioned x: the same error — saw ValueError: [changePermutation] Target must be an injective mapping using only keys and logical qubits from the source permutation.
- `repeat_until_success`, `ddsim.statevector`, reproducer (corpus/reproducers/ddsim_feedback_rounds.py): qubit 1 measured, h, measured again, then qubit 0 measured into a third bit: the same error — saw ValueError: [changePermutation] Target must be an injective mapping using only keys and logical qubits from the source permutation.
- `repeat_until_success`, `ddsim.statevector`, reproducer (corpus/reproducers/ddsim_feedback_rounds.py): the same without qubit 0's measurement into a third bit: counts — saw counts over 4 outcomes
- `dyadic_n20_k16`, `qsim.statevector`, reproducer (corpus/reproducers/qsim_single_precision_norm.py): the same construction on 16 qubits stays within its bound — saw 6.33e-07

<details><summary>Refusals by the engine under test, each listed in <code>corpus/expected-refusals.json</code> with its words (11)</summary>

- `qpe_t6_third`, `wai.exact`: the angle 2.0943951023931953 is not a dyadic multiple of 2π (2πm/2^k, k ≤ 32) [Q500]
- `qpe_t10_third`, `wai.exact`: the angle 2.0943951023931953 is not a dyadic multiple of 2π (2πm/2^k, k ≤ 32) [Q500]
- `rotations_n6`, `wai.exact`: the angle 1.6459015101789476 is not a dyadic multiple of 2π (2πm/2^k, k ≤ 32) [Q500]
- `rotations_n10`, `wai.exact`: the angle 0.49840613019150304 is not a dyadic multiple of 2π (2πm/2^k, k ≤ 32) [Q500]
- `rotations_n12`, `wai.exact`: the angle 1.437905822610216 is not a dyadic multiple of 2π (2πm/2^k, k ≤ 32) [Q500]
- `below_grid_k33`, `wai.exact`: the angle 0.0000000007314590396335798 is not a dyadic multiple of 2π (2πm/2^k, k ≤ 32) [Q500]
- `below_grid_k40`, `wai.exact`: the angle 0.000000000005714523747137342 is not a dyadic multiple of 2π (2πm/2^k, k ≤ 32) [Q500]
- `tiny_radians`, `wai.exact`: the angle 0.000000000001 is not a dyadic multiple of 2π (2πm/2^k, k ≤ 32) [Q500]
- `rotations_n18`, `wai.exact`: the angle 1.586539928488264 is not a dyadic multiple of 2π (2πm/2^k, k ≤ 32) [Q500]
- `qec_refused_channel_with_no_independent_form`, `wai.qec`: line 2: PAULI_CHANNEL_1(0.1, 0.1, 0): its exclusive outcomes have no form as independent X, Y and Z flips, which this simulator and a detector error model both need
- `qec_refused_random_detector`, `wai.qec`: a detector or observable is not deterministic (it anticommutes with op 0)

</details>

<details><summary>Checks other engines did not make, and why (17)</summary>

- `rotations_wide_n20`, `ddsim.statevector`: did not finish within 900 s
- `mirror_n20`, `wai.exact`: not rerun: 8386 ops on 2^20 amplitudes is past the referee's 2^30 updates
- `mirror_n20`, `ddsim.statevector`: did not finish within 900 s
- `gate_gphase_controlled`, `quimb.dense`: no gate gphase
- `gate_gphase_controlled`, `quimb.tensor_network`: no gate gphase
- `gate_gphase_controlled`, `qsim.statevector`: a global phase gate
- `teleport`, `qsim.statevector`: measurement in the middle (or one qubit measured twice)
- `reset_after_superposition`, `qsim.statevector`: measurement in the middle (or one qubit measured twice)
- `measure_twice`, `qsim.statevector`: measurement in the middle (or one qubit measured twice)
- `feedback_flip`, `qsim.statevector`: measurement in the middle (or one qubit measured twice)
- `ghz_by_fusion`, `qsim.statevector`: measurement in the middle (or one qubit measured twice)
- `repeat_until_success`, `qsim.statevector`: measurement in the middle (or one qubit measured twice)
- `mps_ghz_n100`, `aer.mps`: CircuitTooWideForTarget: 'Number of qubits (100) in circuit-46 is greater than maximum (63) in the coupling_map'
- `qft_n22_x2796203`, `wai.exact`: not rerun: 298 ops on 2^22 amplitudes is past the referee's 2^30 updates
- `dyadic_n20_k16`, `wai.exact`: not rerun: 1038 ops on 2^20 amplitudes is past the referee's 2^30 updates
- `qec_refused_channel_with_no_independent_form`, `stim.tableau`: ValueError: Encountered the operation PAULI_CHANNEL_1 during error analysis, but this operation requires the `approximate_disjoint_errors` option to be enabled. If you're calling from python, using stim.Circuit.detector_error_model, you need to add the argument approximate_disjoint_errors=True.  If 
- `qec_refused_random_detector`, `stim.tableau`: ValueError: The circuit contains non-deterministic detectors.  To make an SVG picture of the problem, you can use the python API like this:     your_circuit.diagram('detslice-with-ops-svg', tick=range(0, 5), filter_coords=['D0', ]) or the command line API like this:     stim diagram --in your_circui

</details>

<details><summary>Recorded and not judged (16)</summary>

- `below_grid_k47`, `ddsim.statevector`: holds the phase of its small-angle gate: the share of the step from the state without that gate to the ideal (1 holds it, 0 drops it) — measured -4.3e-78. drops it: the angle, 4.46e-14 rad, is below its parameter tolerance of 1e-13 rad, where it may read an angle as 0
- `below_grid_k48`, `ddsim.statevector`: holds the phase of its small-angle gate: the share of the step from the state without that gate to the ideal (1 holds it, 0 drops it) — measured -4.3e-78. drops it: the angle, 2.23e-14 rad, is below its parameter tolerance of 1e-13 rad, where it may read an angle as 0
- `below_grid_k52`, `ddsim.statevector`: holds the phase of its small-angle gate: the share of the step from the state without that gate to the ideal (1 holds it, 0 drops it) — measured -4.3e-78. drops it: the angle, 1.4e-15 rad, is below its parameter tolerance of 1e-13 rad, where it may read an angle as 0
- `below_grid_k60`, `wai.statevector`: holds the phase of its small-angle gate: the share of the step from the state without that gate to the ideal (1 holds it, 0 drops it) — measured 1. neither can be told: its distance from the ideal state, 2.22e-16, is more than half the gate's effect, 2.72e-18
- `below_grid_k60`, `aer.statevector`: holds the phase of its small-angle gate: the share of the step from the state without that gate to the ideal (1 holds it, 0 drops it) — measured -21. neither can be told: its distance from the ideal state, 6.12e-17, is more than half the gate's effect, 2.72e-18
- `below_grid_k60`, `aer.qasm_import`: holds the phase of its small-angle gate: the share of the step from the state without that gate to the ideal (1 holds it, 0 drops it) — measured -21. neither can be told: its distance from the ideal state, 6.12e-17, is more than half the gate's effect, 2.72e-18
- `below_grid_k60`, `quimb.dense`: holds the phase of its small-angle gate: the share of the step from the state without that gate to the ideal (1 holds it, 0 drops it) — measured 1. neither can be told: its distance from the ideal state, 2.22e-16, is more than half the gate's effect, 2.72e-18
- `below_grid_k60`, `quimb.tensor_network`: holds the phase of its small-angle gate: the share of the step from the state without that gate to the ideal (1 holds it, 0 drops it) — measured 1. neither can be told: its distance from the ideal state, 1.11e-16, is more than half the gate's effect, 2.72e-18
- `below_grid_k60`, `ddsim.statevector`: holds the phase of its small-angle gate: the share of the step from the state without that gate to the ideal (1 holds it, 0 drops it) — measured -4.3e-78. drops it: the angle, 5.45e-18 rad, is below its parameter tolerance of 1e-13 rad, where it may read an angle as 0
- `decode_repetition_d3`, `wai.union_find`: predictions that are a least-weight correction's, of 129 syndromes (no claim of least weight) — measured 129
- `decode_surface_z_d3`, `wai.union_find`: predictions that are a least-weight correction's, of 2494 syndromes (no claim of least weight) — measured 2434
- `decode_surface_z_d5`, `wai.union_find`: predictions that are a least-weight correction's, of 6492 syndromes (no claim of least weight) — measured 6485
- `dyadic_n20_k16`, `qsim.statevector`: a listed disagreement no longer shows — measured False. qsim 0.22.1 (single precision, gates fused to two qubits, one thread), on 148 rotations by m/2^16 of a turn and cz gates on 20 qubits, gives a state 2.3e-4 from the exact one, past the single-precision bound of 1.9e-4 that 148 gates allow (tools/tolerance.py), and almost all of it is the norm: 1.000
- `dyadic_n20_k16`, `qsim.statevector`: reproducer (corpus/reproducers/qsim_single_precision_norm.py): the state of all 148 gates is farther from the double-precision state than the single-precision bound of 0.000194, and its norm is past 1 by about as much — measured {'distance': 5.54e-07, 'norm_minus_1': 1e-08}. the claim no longer holds
- `dyadic_n20_k16`, `qsim.statevector`: reproducer (corpus/reproducers/qsim_single_precision_norm.py): its run of the first 60 gates has a norm larger than its run of 59 by more than 10^-5, where the 60th gate applied by hand in single precision to the 59 gates' state moves the norm by less than 10^-6 — measured {'run_60_minus_run_59': -1.8e-08, 'by_hand': 2.6774e-08}. the claim no longer holds
- `dyadic_n20_k16`, `qsim.statevector`: reproducer (corpus/reproducers/qsim_single_precision_norm.py): another single-precision state vector of the same circuit (the circuit library's own, complex64) is as far from the double-precision state, within 1% — measured 4.8e-07. the claim no longer holds

</details>

Versions: cirq-core 1.7.0, mqt.core 3.11.0, mqt.ddsim 2.7.0, numpy 2.5.4, python 3.12.16, qiskit 2.6.2, qiskit-aer 0.18.2, qsimcirq 0.22.1, quimb 1.15.0, scipy 1.18.1, stim 1.16.0.

### `conformance-macos-arm64.json`

Host: Apple M3 Ultra, macOS 27.2.1, 32 logical CPUs, 256 GiB, arm64. Tier full, 3936.0 s. Checks: info 15, known 16, not run 16, pass 1973, refused 11.

| family | `wai.exact` | `wai.statevector` | `wai.stabilizer` | `wai.auto` | `wai.mps` | `wai.tensor_network` | `wai.density_matrix` | `wai.qec` | `wai.pauli` | `wai.stabrank` | `wai.cutrank` | `wai.trajectories` | `wai.mitigate` | `wai.pulse` | `wai.matching` | `wai.union_find` | `aer.statevector` | `aer.qasm_import` | `aer.stabilizer` | `aer.mps` | `aer.density_matrix` | `stim.tableau` | `quimb.dense` | `quimb.tensor_network` | `quimb.mps` | `ddsim.statevector` | `qsim.statevector` | `pairs and references` | `references` |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| bit_order | 24/24 (0.068) | 8/8 (0.015) | 12/12 | 8/8 (0.0069) |  |  |  |  |  |  |  |  |  |  |  |  | 8/8 (0.016) | 6/6 (0.016) | 5/5 |  |  | 7/7 | 6/6 (0.019) | 6/6 (0.031) |  | 8/8 (0.00015) | 8/8 (0.013) | 15/15 (0.0084) | 5/5 |
| clifford_large | 4/4 (0.012) | 1/1 (0.00077) | 34/34 (0.43) | 1/1 (7.8e-05) |  |  |  |  |  |  |  |  |  |  |  |  | 1/1 (0.00084) | 1/1 (0.00084) | 16/16 (0.51) |  |  | 19/19 (0.34) | 1/1 (0.0017) | 1/1 (4.8e-08) |  | 1/1 (2.4e-07) | 1/1 (0.0024) |  | 15/15 |
| cutrank |  |  |  |  |  |  |  |  |  |  | 6/6 (0.56) |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |
| decoders |  |  |  |  |  |  |  |  |  |  |  |  |  |  | 3/3 | 3/3 |  |  |  |  |  |  |  |  |  |  |  |  |  |
| dynamic |  | 12/12 (0.29) | 8/8 (0.21) | 12/12 (0.29) |  |  |  |  |  |  |  |  |  |  |  |  | 12/12 (0.36) |  |  |  |  | 8/8 (0.31) |  |  |  | 8/8 (0.25) · 11 known | 6 not run |  |  |
| gate_conventions | 188/188 (0.069) | 47/47 (0.017) |  | 47/47 (0.066) |  |  |  |  |  |  |  |  |  |  |  |  | 47/47 (0.018) | 47/47 (0.018) |  |  |  |  | 46/46 (0.017) · 1 not run | 46/46 (0.032) · 1 not run |  | 47/47 (0.00015) | 46/46 (0.026) · 1 not run |  |  |
| ghz | 20/20 (0.042) | 7/7 (0.21) | 2/2 (0.022) | 7/7 (0.022) |  |  |  |  |  |  |  |  |  |  |  |  | 7/7 (0.058) | 5/5 (0.011) |  |  |  | 2/2 (0.29) | 5/5 (0.0094) | 5/5 (0.041) |  | 7/7 (0.058) | 7/7 (0.14) |  | 5/5 |
| grover | 16/16 (0.029) | 6/6 (0.12) |  | 6/6 (0.14) |  |  |  |  |  |  |  |  |  |  |  |  | 6/6 (0.25) | 4/4 (0.0043) |  |  |  |  | 4/4 (0.0079) | 4/4 (0.00032) |  | 6/6 (0.2) | 6/6 (0.27) |  | 4/4 |
| mirror | 8/8 (0.007) · 1 not run | 3/3 (0.00036) |  |  |  |  |  |  |  |  |  |  |  |  |  |  | 3/3 (0.0011) |  |  |  |  |  | 3/3 (0.00049) |  |  | 2/2 (0.00016) · 1 not run | 3/3 (0.00059) |  |  |
| mitigation |  |  |  |  |  |  |  |  |  |  |  |  | 18/18 (0.23) |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |
| noise |  |  |  |  |  |  | 3/3 (0.00093) |  |  |  |  |  |  |  |  |  |  |  |  |  | 3/3 (0.00093) |  |  |  |  |  |  |  |  |
| noisy |  |  | 11/11 (0.34) |  |  |  | 16/16 (0.24) |  |  |  |  | 16/16 (0.41) |  |  |  |  |  |  |  |  | 3/3 (0.0003) |  |  |  |  |  |  |  |  |
| other |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  | 3/3 |
| pauli |  |  |  |  |  |  |  |  | 81/81 (0.11) |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |
| pulse |  |  |  |  |  |  |  |  |  |  |  |  |  | 8/8 (0.0013) |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |
| qec |  |  |  |  |  |  |  | 29/29 (0.6) · 2 refused as listed |  |  |  |  |  |  |  |  |  |  |  |  |  | 22/22 (0.52) · 2 not run |  |  |  |  |  | 44/44 (0.016) |  |
| qft | 36/36 (0.042) · 1 not run | 12/12 (0.58) |  | 11/11 (0.66) |  |  |  |  |  |  |  |  |  |  |  |  | 12/12 (0.61) | 6/6 (0.009) |  |  |  |  | 10/10 (0.0062) | 5/5 (0.0093) |  | 12/12 (0.62) | 12/12 (0.63) | 15/15 (0.0027) | 8/8 |
| qpe | 12/12 (0.017) · 2 refused as listed | 5/5 (0.0091) |  | 5/5 (0.013) |  |  |  |  |  |  |  |  |  |  |  |  | 5/5 (0.005) | 5/5 (0.0074) |  |  |  |  | 5/5 (0.0068) | 4/4 (0.0011) |  | 5/5 (7e-06) | 5/5 (0.0037) |  | 5/5 |
| random_clifford_t | 16/16 (0.039) | 4/4 (0.0047) |  | 4/4 (0.017) |  |  |  |  |  |  |  |  |  |  |  |  | 4/4 (0.0042) | 3/3 (0.0042) |  |  |  |  | 4/4 (0.003) | 3/3 (0.00026) |  | 4/4 (3.7e-05) | 4/4 (0.0022) |  |  |
| random_dyadic | 11/11 (0.008) · 1 not run | 3/3 (0.0019) |  | 3/3 (0.0016) |  |  |  |  |  |  |  |  |  |  |  |  | 3/3 (0.0031) | 2/2 (0.0038) |  |  |  |  | 3/3 (0.0014) | 2/2 (0.00016) |  | 3/3 (8e-06) | 2/2 (0.004) · 5 known |  |  |
| random_rotations | 4 refused as listed | 6/6 (0.48) |  | 6/6 (0.48) |  |  |  |  |  |  |  |  |  |  |  |  | 6/6 (0.49) | 3/3 (0.0039) |  |  |  |  | 4/4 (0.0021) | 3/3 (0.00014) |  | 6/6 (0.48) | 6/6 (0.4) | 10/10 (0.067) |  |
| small_angle | 62/62 (0.9) · 3 refused as listed | 32/32 (0.016) | 2/2 | 18/18 (0.098) |  |  |  |  |  |  |  |  |  |  |  |  | 32/32 (0.018) | 29/29 (0.016) | 1/1 |  |  | 1/1 | 30/30 (0.019) | 29/29 (0.016) |  | 29/29 (0.15) | 33/33 (0.0075) |  | 1/1 |
| stabrank |  |  |  |  |  |  |  |  |  | 9/9 (0.039) |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |
| wide_mps |  |  |  |  | 6/6 (0.00017) | 5/5 (0.00057) |  |  |  |  |  |  |  |  |  |  |  |  |  | 4/4 (0.00023) · 1 not run |  |  |  | 5/5 (0.043) | 5/5 (7.3e-05) |  |  |  |  |
| wide_tn |  |  |  |  | 1/1 (0.00093) | 1/1 (6.4e-07) |  |  |  |  |  |  |  |  |  |  |  |  |  | 1/1 (0.0033) |  |  |  | 1/1 (0.019) | 1/1 (0.0006) |  |  | 21/21 (0.015) |  |

Known disagreements of other engines with the exact reference (`corpus/known-disagreements.json`):

- `repeat_until_success`, `ddsim.statevector`: ValueError: [changePermutation] Target must be an injective mapping using only keys and logical qubits from the source permutation. | known: MQT DDSIM 2.7.0's qasm_simulator raises ValueError [changePermutation] on this program (two qubits; qubit 1 measured three times in the middle, with gates between and an x conditioned on each reading; four classical bits) and gives no counts. It raises the same without the conditioned x, and on qubit 1 measured, h, measured again, then qubit 0 measured into a third bit; with no third bit it gives counts. Each is a case of the reproducer.
- `dyadic_n20_k16`, `qsim.statevector`: single precision, fusion width 2 | known: qsim 0.22.1 (single precision, gates fused to two qubits, one thread), on 148 rotations by m/2^16 of a turn and cz gates on 20 qubits, gives a state 2.3e-4 from the exact one, past the single-precision bound of 1.9e-4 that 148 gates allow (tools/tolerance.py), and almost all of it is the norm: 1.00023. Its run of the first 60 gates has a norm larger than its run of 59 by 6e-5, where the 60th gate applied by hand in single precision to the 59 gates' state moves the norm by under 10^-6. Cirq 1.7.0's own complex64 simulator gives the same distance. On 16 qubits the same construction stays within its bound. Each is a case of the reproducer, which builds the circuit with the corpus's own adapter.
- `measure_twice`, `ddsim.statevector`: 10: 49817 shots, probability 0; 11: 0 shots, expected 50000.0 ± 1095.4 | known: MQT DDSIM 2.7.0's qasm_simulator, given h q[0]; c[0] = measure q[0]; c[1] = measure q[0]; with nothing between the two measurements, returns 00 and 10: c[0] is never set. With x in place of h it gives 10 on every shot; with a barrier between the measurements, 00 and 11; with c[1] measured first and c[0] second, ValueError [changePermutation]. Each is a case of the reproducer, with the circuit built in Qiskit 2.6.2.
- `measure_twice`, `ddsim.statevector`: known: MQT DDSIM 2.7.0's qasm_simulator, given h q[0]; c[0] = measure q[0]; c[1] = measure q[0]; with nothing between the two measurements, returns 00 and 10: c[0] is never set. With x in place of h it gives 10 on every shot; with a barrier between the measurements, 00 and 11; with c[1] measured first and c[0] second, ValueError [changePermutation]. Each is a case of the reproducer, with the circuit built in Qiskit 2.6.2.
- `measure_twice`, `ddsim.statevector`, reproducer (corpus/reproducers/ddsim_repeated_measurement.py): h, then c[0] and c[1] measured from the qubit with nothing between: only 00 and 10 come — saw ['00', '10']
- `measure_twice`, `ddsim.statevector`, reproducer (corpus/reproducers/ddsim_repeated_measurement.py): x, then the same two measurements: 10 on every shot — saw ['10']
- `measure_twice`, `ddsim.statevector`, reproducer (corpus/reproducers/ddsim_repeated_measurement.py): h, the two measurements with a barrier between: 00 and 11 — saw ['00', '11']
- `measure_twice`, `ddsim.statevector`, reproducer (corpus/reproducers/ddsim_repeated_measurement.py): h, then c[1] measured first and c[0] second: an error naming changePermutation — saw ValueError: [changePermutation] Target must be an injective mapping using only keys and logical qubits from the source permutation.
- `repeat_until_success`, `ddsim.statevector`, reproducer (corpus/reproducers/ddsim_feedback_rounds.py): repeat_until_success: an error naming changePermutation, and no counts — saw ValueError: [changePermutation] Target must be an injective mapping using only keys and logical qubits from the source permutation.
- `repeat_until_success`, `ddsim.statevector`, reproducer (corpus/reproducers/ddsim_feedback_rounds.py): the same three rounds without the conditioned x: the same error — saw ValueError: [changePermutation] Target must be an injective mapping using only keys and logical qubits from the source permutation.
- `repeat_until_success`, `ddsim.statevector`, reproducer (corpus/reproducers/ddsim_feedback_rounds.py): qubit 1 measured, h, measured again, then qubit 0 measured into a third bit: the same error — saw ValueError: [changePermutation] Target must be an injective mapping using only keys and logical qubits from the source permutation.
- `repeat_until_success`, `ddsim.statevector`, reproducer (corpus/reproducers/ddsim_feedback_rounds.py): the same without qubit 0's measurement into a third bit: counts — saw counts over 4 outcomes
- `dyadic_n20_k16`, `qsim.statevector`, reproducer (corpus/reproducers/qsim_single_precision_norm.py): the state of all 148 gates is farther from the double-precision state than the single-precision bound of 0.000194, and its norm is past 1 by about as much — saw {'distance': 0.000232443, 'norm_minus_1': 0.000232442}
- `dyadic_n20_k16`, `qsim.statevector`, reproducer (corpus/reproducers/qsim_single_precision_norm.py): the same construction on 16 qubits stays within its bound — saw 6.448e-06
- `dyadic_n20_k16`, `qsim.statevector`, reproducer (corpus/reproducers/qsim_single_precision_norm.py): its run of the first 60 gates has a norm larger than its run of 59 by more than 10^-5, where the 60th gate applied by hand in single precision to the 59 gates' state moves the norm by less than 10^-6 — saw {'run_60_minus_run_59': 6.0414e-05, 'by_hand': 2.6285e-08}
- `dyadic_n20_k16`, `qsim.statevector`, reproducer (corpus/reproducers/qsim_single_precision_norm.py): another single-precision state vector of the same circuit (the circuit library's own, complex64) is as far from the double-precision state, within 1% — saw 0.000232436

<details><summary>Refusals by the engine under test, each listed in <code>corpus/expected-refusals.json</code> with its words (11)</summary>

- `qpe_t6_third`, `wai.exact`: the angle 2.0943951023931953 is not a dyadic multiple of 2π (2πm/2^k, k ≤ 32) [Q500]
- `qpe_t10_third`, `wai.exact`: the angle 2.0943951023931953 is not a dyadic multiple of 2π (2πm/2^k, k ≤ 32) [Q500]
- `rotations_n6`, `wai.exact`: the angle 1.6459015101789476 is not a dyadic multiple of 2π (2πm/2^k, k ≤ 32) [Q500]
- `rotations_n10`, `wai.exact`: the angle 0.49840613019150304 is not a dyadic multiple of 2π (2πm/2^k, k ≤ 32) [Q500]
- `rotations_n12`, `wai.exact`: the angle 1.437905822610216 is not a dyadic multiple of 2π (2πm/2^k, k ≤ 32) [Q500]
- `below_grid_k33`, `wai.exact`: the angle 0.0000000007314590396335798 is not a dyadic multiple of 2π (2πm/2^k, k ≤ 32) [Q500]
- `below_grid_k40`, `wai.exact`: the angle 0.000000000005714523747137342 is not a dyadic multiple of 2π (2πm/2^k, k ≤ 32) [Q500]
- `tiny_radians`, `wai.exact`: the angle 0.000000000001 is not a dyadic multiple of 2π (2πm/2^k, k ≤ 32) [Q500]
- `rotations_n18`, `wai.exact`: the angle 1.586539928488264 is not a dyadic multiple of 2π (2πm/2^k, k ≤ 32) [Q500]
- `qec_refused_channel_with_no_independent_form`, `wai.qec`: line 2: PAULI_CHANNEL_1(0.1, 0.1, 0): its exclusive outcomes have no form as independent X, Y and Z flips, which this simulator and a detector error model both need
- `qec_refused_random_detector`, `wai.qec`: a detector or observable is not deterministic (it anticommutes with op 0)

</details>

<details><summary>Checks other engines did not make, and why (16)</summary>

- `mirror_n20`, `wai.exact`: not rerun: 8386 ops on 2^20 amplitudes is past the referee's 2^30 updates
- `mirror_n20`, `ddsim.statevector`: did not finish within 900 s
- `gate_gphase_controlled`, `quimb.dense`: no gate gphase
- `gate_gphase_controlled`, `quimb.tensor_network`: no gate gphase
- `gate_gphase_controlled`, `qsim.statevector`: a global phase gate
- `teleport`, `qsim.statevector`: measurement in the middle (or one qubit measured twice)
- `reset_after_superposition`, `qsim.statevector`: measurement in the middle (or one qubit measured twice)
- `measure_twice`, `qsim.statevector`: measurement in the middle (or one qubit measured twice)
- `feedback_flip`, `qsim.statevector`: measurement in the middle (or one qubit measured twice)
- `ghz_by_fusion`, `qsim.statevector`: measurement in the middle (or one qubit measured twice)
- `repeat_until_success`, `qsim.statevector`: measurement in the middle (or one qubit measured twice)
- `mps_ghz_n100`, `aer.mps`: CircuitTooWideForTarget: 'Number of qubits (100) in circuit-46 is greater than maximum (63) in the coupling_map'
- `qft_n22_x2796203`, `wai.exact`: not rerun: 298 ops on 2^22 amplitudes is past the referee's 2^30 updates
- `dyadic_n20_k16`, `wai.exact`: not rerun: 1038 ops on 2^20 amplitudes is past the referee's 2^30 updates
- `qec_refused_channel_with_no_independent_form`, `stim.tableau`: ValueError: Encountered the operation PAULI_CHANNEL_1 during error analysis, but this operation requires the `approximate_disjoint_errors` option to be enabled. If you're calling from python, using stim.Circuit.detector_error_model, you need to add the argument approximate_disjoint_errors=True.  If 
- `qec_refused_random_detector`, `stim.tableau`: ValueError: The circuit contains non-deterministic detectors.  To make an SVG picture of the problem, you can use the python API like this:     your_circuit.diagram('detslice-with-ops-svg', tick=range(0, 5), filter_coords=['D0', ]) or the command line API like this:     stim diagram --in your_circui

</details>

<details><summary>Recorded and not judged (12)</summary>

- `below_grid_k47`, `ddsim.statevector`: holds the phase of its small-angle gate: the share of the step from the state without that gate to the ideal (1 holds it, 0 drops it) — measured -4.3e-78. drops it: the angle, 4.46e-14 rad, is below its parameter tolerance of 1e-13 rad, where it may read an angle as 0
- `below_grid_k48`, `ddsim.statevector`: holds the phase of its small-angle gate: the share of the step from the state without that gate to the ideal (1 holds it, 0 drops it) — measured -4.3e-78. drops it: the angle, 2.23e-14 rad, is below its parameter tolerance of 1e-13 rad, where it may read an angle as 0
- `below_grid_k52`, `ddsim.statevector`: holds the phase of its small-angle gate: the share of the step from the state without that gate to the ideal (1 holds it, 0 drops it) — measured -4.3e-78. drops it: the angle, 1.4e-15 rad, is below its parameter tolerance of 1e-13 rad, where it may read an angle as 0
- `below_grid_k60`, `wai.statevector`: holds the phase of its small-angle gate: the share of the step from the state without that gate to the ideal (1 holds it, 0 drops it) — measured 1. neither can be told: its distance from the ideal state, 2.22e-16, is more than half the gate's effect, 2.72e-18
- `below_grid_k60`, `aer.statevector`: holds the phase of its small-angle gate: the share of the step from the state without that gate to the ideal (1 holds it, 0 drops it) — measured -21. neither can be told: its distance from the ideal state, 6.12e-17, is more than half the gate's effect, 2.72e-18
- `below_grid_k60`, `aer.qasm_import`: holds the phase of its small-angle gate: the share of the step from the state without that gate to the ideal (1 holds it, 0 drops it) — measured -21. neither can be told: its distance from the ideal state, 6.12e-17, is more than half the gate's effect, 2.72e-18
- `below_grid_k60`, `quimb.dense`: holds the phase of its small-angle gate: the share of the step from the state without that gate to the ideal (1 holds it, 0 drops it) — measured 1. neither can be told: its distance from the ideal state, 2.22e-16, is more than half the gate's effect, 2.72e-18
- `below_grid_k60`, `quimb.tensor_network`: holds the phase of its small-angle gate: the share of the step from the state without that gate to the ideal (1 holds it, 0 drops it) — measured 1. neither can be told: its distance from the ideal state, 1.11e-16, is more than half the gate's effect, 2.72e-18
- `below_grid_k60`, `ddsim.statevector`: holds the phase of its small-angle gate: the share of the step from the state without that gate to the ideal (1 holds it, 0 drops it) — measured -4.3e-78. drops it: the angle, 5.45e-18 rad, is below its parameter tolerance of 1e-13 rad, where it may read an angle as 0
- `decode_repetition_d3`, `wai.union_find`: predictions that are a least-weight correction's, of 129 syndromes (no claim of least weight) — measured 129
- `decode_surface_z_d3`, `wai.union_find`: predictions that are a least-weight correction's, of 2494 syndromes (no claim of least weight) — measured 2434
- `decode_surface_z_d5`, `wai.union_find`: predictions that are a least-weight correction's, of 6492 syndromes (no claim of least weight) — measured 6485

</details>

Versions: cirq-core 1.7.0, mqt.core 3.11.0, mqt.ddsim 2.7.0, numpy 2.5.4, python 3.12.16, qiskit 2.6.2, qiskit-aer 0.18.2, qsimcirq 0.22.1, quimb 1.15.0, scipy 1.18.1, stim 1.16.0.

### `performance-linux-x86_64.json`

Host: AMD RYZEN AI MAX+ 395 w/ Radeon 8060S, Ubuntu 26.04.1 LTS, 32 logical CPUs, 123 GiB, x86_64. Each figure is the engine's own call, the median of 5 runs (the minimum in brackets). Memory is the peak resident set of the process that ran the engine, and the baseline the same given a two-qubit job. Ops are the operations the engine ran, where it reports them.

- `aer.*` seconds: the simulator's own time for the experiment, as its result states it (time_taken); the conversion of the circuit to its own form is outside it.
- `ddsim.*` seconds: the run call, which converts the circuit to its own form inside it.
- `qsim.*` seconds: the simulate call, which converts the circuit to its own form inside it.
- `quimb.*` seconds: the building of the circuit object, which applies each gate as it is added, and the read-out.
- `stim.*` seconds: the tableau's or sampler's run, after the circuit text is parsed.
- `wai.*` seconds: the engine's run, after the program is read and lowered (the driver's own clock).

| case | engine | threads | ops | seconds, median (min) | peak MiB (baseline) | load average before → after |
|---|---|---|---|---|---|---|
| qft_n20 | `wai.statevector` | all (32) | 230 | 0.0363 (0.0347) | 26 (5) | 13.9 → 13.9 |
| qft_n20 | `aer.statevector` | 32 | 231 | 0.819 (0.737) | 116 (100) | 13.9 → 13.7 |
| qft_n20 | `qsim.statevector` | 32 |  | 4.78 (3.16) | 196 (188) | 14.6 → 28.9 |
| qft_n20 | `ddsim.statevector` | not set |  | 0.349 (0.329) | 132 (99) | 28.0 → 28.0 |
| qft_n20 | `wai.exact` | 1 (no threads) | 250 | 0.27 (0.26) | 38 (5) | 28.0 → 27.2 |
| qft_n20 | `quimb.dense` | not set |  | 4.92 (3.99) | 291 (195) | 27.2 → 31.9 |
| qft_n20 | `wai.statevector` | 1 (pinned) | 230 | 0.372 (0.352) | 23 (5) | 31.9 → 31.2 |
| qft_n20 | `aer.statevector` | 1 (pinned) | 231 | 0.445 (0.294) | 116 (100) | 31.2 → 31.4 |
| qft_n20 | `qsim.statevector` | 1 (pinned) |  | 0.0454 (0.0434) | 195 (188) | 31.4 → 29.3 |
| qft_n20 | `ddsim.statevector` | 1 (pinned) |  | 0.562 (0.5) | 132 (99) | 29.3 → 27.7 |
| qft_n20 | `wai.exact` | 1 (pinned) | 250 | 0.393 (0.349) | 38 (5) | 27.7 → 27.7 |
| qft_n20 | `quimb.dense` | 1 (pinned) |  | 1.37 (1.26) | 276 (195) | 27.7 → 25.0 |
| qft_n24 | `wai.statevector` | all (32) | 324 | 0.45 (0.441) | 279 (5) | 25.0 → 26.2 |
| qft_n24 | `aer.statevector` | 32 | 325 | 3.9 (1.6) | 356 (100) | 26.2 → 29.5 |
| qft_n24 | `qsim.statevector` | 32 |  | 2.56 (1.91) | 315 (188) | 29.5 → 30.9 |
| qft_n24 | `ddsim.statevector` | not set |  | 6.22 (5.34) | 612 (99) | 30.9 → 25.7 |
| qft_n24 | `wai.exact` | 1 (no threads) | 348 | 6.6 (6.48) | 518 (5) | 25.7 → 18.8 |
| qft_n24 | `wai.statevector` | 1 (pinned) | 324 | 5.82 (5.35) | 263 (5) | 18.8 → 20.5 |
| qft_n24 | `aer.statevector` | 1 (pinned) | 325 | 8.17 (6.6) | 356 (100) | 20.5 → 21.2 |
| qft_n24 | `qsim.statevector` | 1 (pinned) |  | 1.9 (1.63) | 315 (188) | 21.2 → 19.6 |
| qft_n24 | `ddsim.statevector` | 1 (pinned) |  | 7.36 (6.23) | 612 (99) | 19.6 → 18.5 |
| qft_n24 | `wai.exact` | 1 (pinned) | 348 | 7.8 (7) | 518 (5) | 18.5 → 14.9 |
| qft_n26 | `wai.statevector` | all (32) | 377 | 2.14 (2.05) | 1047 (5) | 14.9 → 19.9 |
| qft_n26 | `aer.statevector` | 32 | 378 | 3.97 (3.7) | 1124 (100) | 19.9 → 23.6 |
| qft_n26 | `qsim.statevector` | 32 |  | 5.62 (3.93) | 700 (188) | 23.6 → 31.5 |
| qft_n28 | `wai.statevector` | all (32) | 434 | 14.2 (12.4) | 4118 (5) | 31.5 → 44.7 |
| qft_n28 | `aer.statevector` | 32 | 435 | 16.8 (15.3) | 4198 (100) | 44.7 → 41.8 |
| qft_n28 | `qsim.statevector` | 32 |  | 12.7 (11.9) | 2234 (188) | 41.8 → 38.5 |
| rotations_n20 | `wai.statevector` | all (32) | 590 | 0.0963 (0.0761) | 27 (5) | 38.5 → 38.5 |
| rotations_n20 | `aer.statevector` | 32 | 591 | 1.36 (1.06) | 120 (100) | 38.5 → 40.0 |
| rotations_n20 | `qsim.statevector` | 32 |  | 1.42 (1.22) | 195 (188) | 40.0 → 38.4 |
| rotations_n20 | `quimb.dense` | not set |  | 15.7 (9.5) | 290 (195) | 38.4 → 54.5 |
| rotations_n20 | `wai.statevector` | 1 (pinned) | 590 | 0.613 (0.481) | 23 (5) | 54.5 → 52.2 |
| rotations_n20 | `aer.statevector` | 1 (pinned) | 591 | 0.876 (0.709) | 120 (100) | 52.2 → 50.1 |
| rotations_n20 | `qsim.statevector` | 1 (pinned) |  | 0.0458 (0.0437) | 196 (188) | 50.1 → 45.3 |
| rotations_n20 | `quimb.dense` | 1 (pinned) |  | 2.25 (2.2) | 276 (195) | 45.3 → 38.4 |
| rotations_n24 | `wai.statevector` | all (32) | 710 | 0.892 (0.869) | 282 (5) | 38.4 → 37.3 |
| rotations_n24 | `aer.statevector` | 32 | 711 | 2.65 (2.44) | 356 (100) | 37.3 → 38.9 |
| rotations_n24 | `qsim.statevector` | 32 |  | 1.71 (1.52) | 315 (188) | 38.9 → 38.1 |
| rotations_n24 | `wai.statevector` | 1 (pinned) | 710 | 10.4 (9.72) | 263 (5) | 38.1 → 21.1 |
| rotations_n24 | `aer.statevector` | 1 (pinned) | 711 | 15.1 (13.3) | 356 (100) | 21.1 → 13.5 |
| rotations_n24 | `qsim.statevector` | 1 (pinned) |  | 1.54 (1.52) | 316 (188) | 13.5 → 13.6 |
| rotations_n26 | `wai.statevector` | all (32) | 770 | 4.12 (3.94) | 1050 (5) | 13.6 → 23.3 |
| rotations_n26 | `aer.statevector` | 32 | 771 | 8.7 (8) | 1123 (100) | 23.3 → 33.4 |
| rotations_n26 | `qsim.statevector` | 32 |  | 4.15 (4.07) | 700 (188) | 33.4 → 35.5 |
| grover_n10 | `wai.statevector` | all (32) | 114 | 0.000998 (0.000871) | 6 (5) | 35.5 → 35.5 |
| grover_n10 | `wai.exact` | 1 (no threads) | 26686 | 0.0363 (0.0331) | 17 (5) | 35.5 → 35.5 |
| grover_n10 | `wai.exact_whole` | 1 (no threads) | 114 | 0.000195 (0.000162) | 6 (5) | 35.5 → 35.5 |
| grover_n10 | `aer.statevector` | 32 | 123 | 0.000298 (0.000293) | 100 (100) | 35.5 → 35.5 |
| grover_n10 | `qsim.statevector` | 32 |  | 0.0014 (0.00129) | 188 (188) | 35.5 → 33.5 |
| grover_n10 | `ddsim.statevector` | not set |  | 0.00883 (0.00609) | 100 (99) | 33.5 → 33.5 |
| grover_n10 | `wai.statevector` | 1 (pinned) | 114 | 0.000446 (0.000321) | 6 (5) | 33.5 → 33.5 |
| grover_n10 | `wai.exact` | 1 (pinned) | 26686 | 0.0507 (0.0317) | 17 (5) | 33.5 → 33.5 |
| grover_n10 | `wai.exact_whole` | 1 (pinned) | 114 | 0.000273 (0.000271) | 6 (5) | 33.5 → 31.6 |
| grover_n10 | `aer.statevector` | 1 (pinned) | 123 | 0.000298 (0.000289) | 100 (100) | 31.6 → 31.6 |
| grover_n10 | `qsim.statevector` | 1 (pinned) |  | 0.00137 (0.00128) | 187 (188) | 31.6 → 29.7 |
| grover_n10 | `ddsim.statevector` | 1 (pinned) |  | 0.00742 (0.0058) | 100 (99) | 29.7 → 27.8 |
| grover_n14 | `wai.statevector` | all (32) | 158 | 0.00241 (0.00227) | 6 (5) | 27.8 → 27.8 |
| grover_n14 | `wai.exact` | 1 (no threads) | 557146 | 11.4 (10.6) | 242 (5) | 27.8 → 14.9 |
| grover_n14 | `wai.exact_whole` | 1 (no threads) | 158 | 0.00354 (0.00352) | 6 (5) | 14.9 → 14.9 |
| grover_n14 | `aer.statevector` | 32 | 167 | 0.00298 (0.00286) | 100 (100) | 14.9 → 14.9 |
| grover_n14 | `qsim.statevector` | 32 |  | 0.0773 (0.0661) | 188 (188) | 14.9 → 14.7 |
| grover_n14 | `ddsim.statevector` | not set |  | 0.00672 (0.0066) | 99 (99) | 14.7 → 14.7 |
| grover_n14 | `wai.statevector` | 1 (pinned) | 158 | 0.00225 (0.00156) | 6 (5) | 14.7 → 14.7 |
| grover_n14 | `wai.exact` | 1 (pinned) | 557146 | 17.4 (13.8) | 242 (5) | 14.7 → 12.9 |
| grover_n14 | `wai.exact_whole` | 1 (pinned) | 158 | 0.00343 (0.0034) | 6 (5) | 12.9 → 12.9 |
| grover_n14 | `aer.statevector` | 1 (pinned) | 167 | 0.00296 (0.00281) | 100 (100) | 12.9 → 12.9 |
| grover_n14 | `qsim.statevector` | 1 (pinned) |  | 0.0018 (0.0018) | 187 (188) | 12.9 → 12.7 |
| grover_n14 | `ddsim.statevector` | 1 (pinned) |  | 0.0073 (0.00706) | 100 (99) | 12.7 → 12.7 |
| grover_n16 | `wai.statevector` | all (32) | 180 | 0.00857 (0.0083) | 7 (5) | 12.7 → 12.7 |
| grover_n16 | `wai.exact_whole` | 1 (no threads) | 180 | 0.0169 (0.0162) | 7 (5) | 12.7 → 12.7 |
| grover_n16 | `aer.statevector` | 32 | 189 | 0.462 (0.437) | 101 (100) | 12.7 → 15.2 |
| grover_n16 | `qsim.statevector` | 32 |  | 0.328 (0.288) | 189 (188) | 15.2 → 15.1 |
| grover_n16 | `ddsim.statevector` | not set |  | 0.00875 (0.0085) | 101 (99) | 15.1 → 15.1 |
| grover_n16 | `wai.statevector` | 1 (pinned) | 180 | 0.00965 (0.0096) | 7 (5) | 15.1 → 15.1 |
| grover_n16 | `wai.exact_whole` | 1 (pinned) | 180 | 0.0281 (0.0269) | 8 (5) | 15.1 → 15.1 |
| grover_n16 | `aer.statevector` | 1 (pinned) | 189 | 0.0186 (0.0182) | 101 (100) | 15.1 → 14.6 |
| grover_n16 | `qsim.statevector` | 1 (pinned) |  | 0.00342 (0.00234) | 188 (188) | 14.6 → 14.4 |
| grover_n16 | `ddsim.statevector` | 1 (pinned) |  | 0.0115 (0.0113) | 102 (99) | 14.4 → 14.2 |
| grover_n20 | `wai.statevector` | all (32) | 224 | 0.0268 (0.0259) | 25 (5) | 14.2 → 14.2 |
| grover_n20 | `wai.exact_whole` | 1 (no threads) | 224 | 0.34 (0.339) | 38 (5) | 14.2 → 14.2 |
| grover_n20 | `aer.statevector` | 32 | 233 | 0.72 (0.61) | 124 (100) | 14.2 → 16.7 |
| grover_n20 | `qsim.statevector` | 32 |  | 0.429 (0.316) | 195 (188) | 16.7 → 16.6 |
| grover_n20 | `ddsim.statevector` | not set |  | 0.0332 (0.0325) | 132 (99) | 16.6 → 16.5 |
| grover_n20 | `wai.statevector` | 1 (pinned) | 224 | 0.204 (0.204) | 22 (5) | 16.5 → 16.5 |
| grover_n20 | `wai.exact_whole` | 1 (pinned) | 224 | 0.586 (0.583) | 38 (5) | 16.5 → 16.5 |
| grover_n20 | `aer.statevector` | 1 (pinned) | 233 | 0.35 (0.35) | 124 (100) | 16.5 → 16.2 |
| grover_n20 | `qsim.statevector` | 1 (pinned) |  | 0.0161 (0.0133) | 195 (188) | 16.2 → 15.5 |
| grover_n20 | `ddsim.statevector` | 1 (pinned) |  | 0.0423 (0.032) | 132 (99) | 15.5 → 15.5 |
| clifford_n1000 | `wai.stabilizer` | 1 (no threads) | 35082 | 0.000968 (0.000961) | 90 (5) | 15.5 → 15.6 |
| clifford_n1000 | `stim.tableau` | not set |  | 0.002 (0.00126) | 40 (24) | 15.6 → 15.6 |
| clifford_n1000 | `aer.stabilizer` | 32 | 35083 | 0.0184 (0.0161) | 207 (121) | 15.6 → 15.7 |
| clifford_n5000 | `wai.stabilizer` | 1 (no threads) | 58265 | 0.00666 (0.00655) | 142 (5) | 15.9 → 15.6 |
| clifford_n5000 | `stim.tableau` | not set |  | 0.00839 (0.00829) | 65 (24) | 15.6 → 15.6 |
| surface_z_d5_sample | `wai.qec` | 1 (no threads) |  | 0.0762 (0.0737) | 19 (4) | 15.6 → 15.7 |
| surface_z_d5_sample | `stim.tableau` | not set |  | 0.348 (0.337) | 160 (38) | 15.7 → 15.7 |
| surface_z_d5_sample | `wai.qec` | 1 (pinned) |  | 0.0949 (0.0812) | 19 (4) | 15.7 → 15.7 |
| surface_z_d5_sample | `stim.tableau` | 1 (pinned) |  | 0.292 (0.273) | 160 (38) | 15.7 → 15.5 |
| surface_z_d5_model | `wai.qec` | 1 (no threads) |  | 0.00185 (0.00182) | 7 (5) | 15.5 → 15.5 |
| surface_z_d5_model | `stim.tableau` | not set |  | 0.00321 (0.00319) | 26 (25) | 15.5 → 15.5 |
| surface_z_d11_sample | `wai.qec` | 1 (no threads) |  | 0.817 (0.811) | 163 (4) | 15.5 → 15.5 |
| surface_z_d11_sample | `stim.tableau` | not set |  | 3.19 (3.05) | 853 (38) | 15.5 → 16.8 |
| surface_z_d11_sample | `wai.qec` | 1 (pinned) |  | 1.06 (1.06) | 163 (4) | 16.8 → 16.8 |
| surface_z_d11_sample | `stim.tableau` | 1 (pinned) |  | 4.05 (3.33) | 853 (38) | 16.8 → 16.8 |
| surface_z_d11_model | `wai.qec` | 1 (no threads) |  | 0.0442 (0.0373) | 26 (5) | 16.8 → 16.8 |
| surface_z_d11_model | `stim.tableau` | not set |  | 0.0542 (0.0509) | 40 (25) | 16.8 → 16.8 |
| qft_n40_amplitudes | `wai.mps` | 1 (no threads) |  | 0.0114 (0.0107) | 11 (7) | 16.8 → 16.8 |
| qft_n40_amplitudes | `aer.mps` | 32 | 861 | 0.00371 (0.00352) | 105 (99) | 16.8 → 16.6 |
| qft_n40_amplitudes | `quimb.mps` | not set |  | 1.98 (1.91) | 206 (200) | 16.6 → 18.2 |
| qft_n40_amplitudes | `wai.tensor_network` | all (32) |  | 0.155 (0.15) | 12 (7) | 18.2 → 17.9 |
| qft_n40_amplitudes | `quimb.tensor_network` | not set |  | 1.08 (0.961) | 220 (205) | 17.9 → 17.8 |

### `performance-macos-arm64.json`

Host: Apple M3 Ultra, macOS 27.2, 32 logical CPUs, 256 GiB, arm64. Each figure is the engine's own call, the median of 5 runs (the minimum in brackets). Memory is the peak resident set of the process that ran the engine, and the baseline the same given a two-qubit job. Ops are the operations the engine ran, where it reports them.

- `aer.*` seconds: the simulator's own time for the experiment, as its result states it (time_taken); the conversion of the circuit to its own form is outside it.
- `ddsim.*` seconds: the run call, which converts the circuit to its own form inside it.
- `qsim.*` seconds: the simulate call, which converts the circuit to its own form inside it.
- `quimb.*` seconds: the building of the circuit object, which applies each gate as it is added, and the read-out.
- `stim.*` seconds: the tableau's or sampler's run, after the circuit text is parsed.
- `wai.*` seconds: the engine's run, after the program is read and lowered (the driver's own clock).

| case | engine | threads | ops | seconds, median (min) | peak MiB (baseline) | load average before → after |
|---|---|---|---|---|---|---|
| qft_n20 | `wai.statevector` | all (32) | 230 | 0.0326 (0.0307) | 28 (4) | 84.8 → 84.9 |
| qft_n20 | `aer.statevector` | 32 | 231 | 0.0841 (0.0824) | 101 (83) | 84.9 → 84.9 |
| qft_n20 | `qsim.statevector` | 32 |  | 0.0905 (0.0771) | 197 (188) | 84.9 → 90.7 |
| qft_n20 | `ddsim.statevector` | not set |  | 0.145 (0.142) | 119 (86) | 89.8 → 89.8 |
| qft_n20 | `wai.exact` | 1 (no threads) | 250 | 0.242 (0.231) | 37 (3) | 89.8 → 89.8 |
| qft_n20 | `quimb.dense` | not set |  | 1.6 (1.49) | 256 (158) | 89.8 → 87.8 |
| qft_n24 | `wai.statevector` | all (32) | 324 | 0.585 (0.54) | 308 (4) | 87.8 → 88.6 |
| qft_n24 | `aer.statevector` | 32 | 325 | 1.02 (0.944) | 341 (83) | 88.6 → 88.7 |
| qft_n24 | `qsim.statevector` | 32 |  | 0.825 (0.777) | 317 (188) | 88.7 → 89.6 |
| qft_n24 | `ddsim.statevector` | not set |  | 2.24 (2.2) | 599 (86) | 89.6 → 87.0 |
| qft_n24 | `wai.exact` | 1 (no threads) | 348 | 5.76 (5.49) | 518 (3) | 87.0 → 91.1 |
| qft_n26 | `wai.statevector` | all (32) | 377 | 2.66 (2.16) | 1070 (4) | 91.1 → 107.2 |
| qft_n26 | `aer.statevector` | 32 | 378 | 2.28 (2.08) | 1109 (83) | 107.2 → 108.9 |
| qft_n26 | `qsim.statevector` | 32 |  | 2.77 (2.72) | 701 (188) | 108.9 → 108.4 |
| qft_n28 | `wai.statevector` | all (32) | 434 | 10.2 (10.2) | 4143 (4) | 108.4 → 129.7 |
| qft_n28 | `aer.statevector` | 32 | 435 | 7.04 (7.02) | 4181 (83) | 129.7 → 133.5 |
| qft_n28 | `qsim.statevector` | 32 |  | 9.45 (9.41) | 2238 (188) | 133.5 → 107.2 |
| rotations_n20 | `wai.statevector` | all (32) | 590 | 0.0699 (0.0663) | 29 (4) | 107.2 → 107.2 |
| rotations_n20 | `aer.statevector` | 32 | 591 | 0.195 (0.166) | 102 (83) | 107.2 → 107.1 |
| rotations_n20 | `qsim.statevector` | 32 |  | 0.143 (0.0835) | 199 (188) | 107.1 → 100.2 |
| rotations_n20 | `quimb.dense` | not set |  | 2.84 (2.82) | 236 (158) | 100.2 → 99.3 |
| rotations_n24 | `wai.statevector` | all (32) | 710 | 0.925 (0.901) | 303 (4) | 99.3 → 102.6 |
| rotations_n24 | `aer.statevector` | 32 | 711 | 1.58 (1.51) | 342 (83) | 102.6 → 102.4 |
| rotations_n24 | `qsim.statevector` | 32 |  | 0.601 (0.585) | 318 (188) | 102.4 → 100.1 |
| rotations_n26 | `wai.statevector` | all (32) | 770 | 2.36 (2.3) | 1074 (4) | 100.1 → 98.1 |
| rotations_n26 | `aer.statevector` | 32 | 771 | 3.89 (3.83) | 1110 (83) | 98.1 → 94.5 |
| rotations_n26 | `qsim.statevector` | 32 |  | 1.99 (1.98) | 702 (188) | 94.5 → 91.2 |
| grover_n10 | `wai.statevector` | all (32) | 114 | 0.000183 (0.000175) | 4 (4) | 91.2 → 91.2 |
| grover_n10 | `wai.exact` | 1 (no threads) | 26686 | 0.0264 (0.0264) | 17 (3) | 91.2 → 91.2 |
| grover_n10 | `wai.exact_whole` | 1 (no threads) | 114 | 0.00015 (0.00015) | 4 (3) | 91.2 → 91.2 |
| grover_n10 | `aer.statevector` | 32 | 123 | 0.000252 (0.000226) | 84 (83) | 91.2 → 89.9 |
| grover_n10 | `qsim.statevector` | 32 |  | 0.00337 (0.00307) | 189 (188) | 89.9 → 88.5 |
| grover_n10 | `ddsim.statevector` | not set |  | 0.00703 (0.00571) | 87 (86) | 88.5 → 88.5 |
| grover_n14 | `wai.statevector` | all (32) | 158 | 0.00223 (0.00221) | 4 (4) | 88.5 → 88.5 |
| grover_n14 | `wai.exact` | 1 (no threads) | 557146 | 9.18 (9.1) | 261 (3) | 88.5 → 73.9 |
| grover_n14 | `wai.exact_whole` | 1 (no threads) | 158 | 0.00322 (0.0032) | 4 (3) | 73.9 → 73.9 |
| grover_n14 | `aer.statevector` | 32 | 167 | 0.00316 (0.00313) | 84 (83) | 73.9 → 75.7 |
| grover_n14 | `qsim.statevector` | 32 |  | 0.0138 (0.0123) | 189 (188) | 75.7 → 71.6 |
| grover_n14 | `ddsim.statevector` | not set |  | 0.00643 (0.00622) | 88 (86) | 71.6 → 71.6 |
| grover_n16 | `wai.statevector` | all (32) | 180 | 0.011 (0.0106) | 7 (4) | 71.6 → 71.6 |
| grover_n16 | `wai.exact_whole` | 1 (no threads) | 180 | 0.0148 (0.0146) | 6 (3) | 71.6 → 71.6 |
| grover_n16 | `aer.statevector` | 32 | 189 | 0.0194 (0.016) | 86 (83) | 71.6 → 73.7 |
| grover_n16 | `qsim.statevector` | 32 |  | 0.0151 (0.0134) | 189 (188) | 73.7 → 73.4 |
| grover_n16 | `ddsim.statevector` | not set |  | 0.00702 (0.00667) | 89 (86) | 73.4 → 73.4 |
| grover_n20 | `wai.statevector` | all (32) | 224 | 0.0301 (0.0286) | 28 (4) | 73.4 → 79.4 |
| grover_n20 | `wai.exact_whole` | 1 (no threads) | 224 | 0.322 (0.319) | 36 (3) | 79.4 → 79.4 |
| grover_n20 | `aer.statevector` | 32 | 233 | 0.0588 (0.0546) | 108 (83) | 79.4 → 79.4 |
| grover_n20 | `qsim.statevector` | 32 |  | 0.0254 (0.0247) | 197 (188) | 79.4 → 74.8 |
| grover_n20 | `ddsim.statevector` | not set |  | 0.0159 (0.0157) | 119 (86) | 74.8 → 74.8 |
| clifford_n1000 | `wai.stabilizer` | 1 (no threads) | 35082 | 0.00106 (0.00103) | 95 (3) | 74.8 → 74.8 |
| clifford_n1000 | `stim.tableau` | not set |  | 0.0017 (0.00163) | 46 (27) | 74.8 → 74.8 |
| clifford_n1000 | `aer.stabilizer` | 32 | 35083 | 0.00963 (0.00854) | 193 (99) | 74.8 → 69.7 |
| clifford_n5000 | `wai.stabilizer` | 1 (no threads) | 58265 | 0.0127 (0.0123) | 168 (3) | 77.3 → 77.3 |
| clifford_n5000 | `stim.tableau` | not set |  | 0.0441 (0.0331) | 74 (27) | 77.3 → 77.3 |
| surface_z_d5_sample | `wai.qec` | 1 (no threads) |  | 0.0897 (0.0872) | 18 (3) | 77.3 → 76.9 |
| surface_z_d5_sample | `stim.tableau` | not set |  | 0.224 (0.219) | 123 (38) | 76.9 → 76.9 |
| surface_z_d5_model | `wai.qec` | 1 (no threads) |  | 0.00231 (0.00226) | 6 (3) | 76.9 → 76.9 |
| surface_z_d5_model | `stim.tableau` | not set |  | 0.00279 (0.00269) | 28 (28) | 76.9 → 76.9 |
| surface_z_d11_sample | `wai.qec` | 1 (no threads) |  | 0.997 (0.981) | 163 (3) | 76.9 → 77.1 |
| surface_z_d11_sample | `stim.tableau` | not set |  | 2.18 (2.18) | 938 (38) | 77.1 → 75.9 |
| surface_z_d11_model | `wai.qec` | 1 (no threads) |  | 0.0264 (0.0243) | 33 (3) | 75.9 → 74.7 |
| surface_z_d11_model | `stim.tableau` | not set |  | 0.04 (0.0399) | 47 (28) | 74.7 → 74.7 |
| qft_n40_amplitudes | `wai.mps` | 1 (no threads) |  | 0.0167 (0.0159) | 10 (5) | 74.7 → 74.7 |
| qft_n40_amplitudes | `aer.mps` | 32 | 861 | 0.00552 (0.00523) | 86 (83) | 74.7 → 74.7 |
| qft_n40_amplitudes | `quimb.mps` | not set |  | 1.93 (1.92) | 167 (160) | 74.7 → 64.9 |
| qft_n40_amplitudes | `wai.tensor_network` | all (32) |  | 0.197 (0.194) | 12 (5) | 65.7 → 65.7 |
| qft_n40_amplitudes | `quimb.tensor_network` | not set |  | 0.934 (0.916) | 180 (167) | 65.7 → 70.7 |

### `before-after-macos-arm64.json`

Host: Apple M3 Ultra, macOS 27.2, 32 logical CPUs, 256 GiB, arm64. The same harness and circuits, with the engine under test built before and after a gate under many controls stood whole on its state vector, one build straight after the other: each figure is the engine's own call, the median of 5 runs (the minimum in brackets), and ops are the operations it ran.

`wai.exact_whole`'s before rows ran a build with no such option, so they are standard gates, as their ops show. The last column is the one-minute load as each build's row began.

| case | engine | threads | before: ops | before: seconds, median (min) | after: ops | after: seconds, median (min) | load average, before build → after build |
|---|---|---|---|---|---|---|---|
| qft_n20 | `wai.statevector` | all (32) | 230 | 0.0227 (0.0223) | 230 | 0.0326 (0.0307) | 30.9 → 84.8 |
| qft_n20 | `aer.statevector` | 32 | 231 | 0.0708 (0.0672) | 231 | 0.0841 (0.0824) | 30.9 → 84.9 |
| qft_n20 | `qsim.statevector` | 32 |  | 0.0564 (0.0557) |  | 0.0905 (0.0771) | 30.9 → 84.9 |
| qft_n20 | `ddsim.statevector` | not set |  | 0.14 (0.14) |  | 0.145 (0.142) | 29.4 → 89.8 |
| qft_n20 | `wai.exact` | 1 (no threads) | 250 | 0.22 (0.214) | 250 | 0.242 (0.231) | 32.9 → 89.8 |
| qft_n20 | `quimb.dense` | not set |  | 1.41 (1.38) |  | 1.6 (1.49) | 32.9 → 89.8 |
| qft_n24 | `wai.statevector` | all (32) | 324 | 0.474 (0.468) | 324 | 0.585 (0.54) | 32.5 → 87.8 |
| qft_n24 | `aer.statevector` | 32 | 325 | 0.565 (0.532) | 325 | 1.02 (0.944) | 32.5 → 88.6 |
| qft_n24 | `qsim.statevector` | 32 |  | 0.555 (0.528) |  | 0.825 (0.777) | 33.5 → 88.7 |
| qft_n24 | `ddsim.statevector` | not set |  | 2.12 (2.12) |  | 2.24 (2.2) | 39.0 → 89.6 |
| qft_n24 | `wai.exact` | 1 (no threads) | 348 | 5.01 (5) | 348 | 5.76 (5.49) | 38.2 → 87.0 |
| qft_n26 | `wai.statevector` | all (32) | 377 | 2.1 (2.06) | 377 | 2.66 (2.16) | 42.0 → 91.1 |
| qft_n26 | `aer.statevector` | 32 | 378 | 1.84 (1.76) | 378 | 2.28 (2.08) | 48.6 → 107.2 |
| qft_n26 | `qsim.statevector` | 32 |  | 2.11 (2.09) |  | 2.77 (2.72) | 51.3 → 108.9 |
| qft_n28 | `wai.statevector` | all (32) | 434 | 9.44 (9.32) | 434 | 10.2 (10.2) | 44.6 → 108.4 |
| qft_n28 | `aer.statevector` | 32 | 435 | 6.72 (6.58) | 435 | 7.04 (7.02) | 57.7 → 129.7 |
| qft_n28 | `qsim.statevector` | 32 |  | 8.98 (8.9) |  | 9.45 (9.41) | 63.9 → 133.5 |
| rotations_n20 | `wai.statevector` | all (32) | 590 | 0.0619 (0.0609) | 590 | 0.0699 (0.0663) | 55.2 → 107.2 |
| rotations_n20 | `aer.statevector` | 32 | 591 | 0.13 (0.123) | 591 | 0.195 (0.166) | 55.2 → 107.2 |
| rotations_n20 | `qsim.statevector` | 32 |  | 0.0568 (0.0534) |  | 0.143 (0.0835) | 55.2 → 107.1 |
| rotations_n20 | `quimb.dense` | not set |  | 2.61 (2.6) |  | 2.84 (2.82) | 53.9 → 100.2 |
| rotations_n24 | `wai.statevector` | all (32) | 710 | 0.982 (0.909) | 710 | 0.925 (0.901) | 53.7 → 99.3 |
| rotations_n24 | `aer.statevector` | 32 | 711 | 1.3 (1.17) | 711 | 1.58 (1.51) | 59.2 → 102.6 |
| rotations_n24 | `qsim.statevector` | 32 |  | 0.526 (0.463) |  | 0.601 (0.585) | 59.4 → 102.4 |
| rotations_n26 | `wai.statevector` | all (32) | 770 | 2.32 (2.3) | 770 | 2.36 (2.3) | 53.0 → 100.1 |
| rotations_n26 | `aer.statevector` | 32 | 771 | 3.92 (3.63) | 771 | 3.89 (3.83) | 55.1 → 98.1 |
| rotations_n26 | `qsim.statevector` | 32 |  | 1.77 (1.66) |  | 1.99 (1.98) | 57.8 → 94.5 |
| grover_n10 | `wai.statevector` | all (32) | 8290 | 0.0156 (0.0152) | 114 | 0.000183 (0.000175) | 49.9 → 91.2 |
| grover_n10 | `wai.exact` | 1 (no threads) | 26686 | 0.0251 (0.0247) | 26686 | 0.0264 (0.0264) | 49.9 → 91.2 |
| grover_n10 | `wai.exact_whole` | 1 (no threads) | 26686 | 0.0249 (0.0248) | 114 | 0.00015 (0.00015) | 49.9 → 91.2 |
| grover_n10 | `aer.statevector` | 32 | 123 | 0.000224 (0.000219) | 123 | 0.000252 (0.000226) | 47.0 → 91.2 |
| grover_n10 | `qsim.statevector` | 32 |  | 0.00315 (0.00293) |  | 0.00337 (0.00307) | 47.0 → 89.9 |
| grover_n10 | `ddsim.statevector` | not set |  | 0.0054 (0.00457) |  | 0.00703 (0.00571) | 44.2 → 88.5 |
| grover_n14 | `wai.statevector` | all (32) | 131214 | 3 (2.97) | 158 | 0.00223 (0.00221) | 44.2 → 88.5 |
| grover_n14 | `wai.exact` | 1 (no threads) | 557146 | 8.48 (8.39) | 557146 | 9.18 (9.1) | 46.0 → 88.5 |
| grover_n14 | `wai.exact_whole` | 1 (no threads) | 557146 | 8.45 (8.37) | 158 | 0.00322 (0.0032) | 35.3 → 73.9 |
| grover_n14 | `aer.statevector` | 32 | 167 | 0.00318 (0.00303) | 167 | 0.00316 (0.00313) | 35.0 → 73.9 |
| grover_n14 | `qsim.statevector` | 32 |  | 0.0114 (0.0102) |  | 0.0138 (0.0123) | 34.7 → 75.7 |
| grover_n14 | `ddsim.statevector` | not set |  | 0.0059 (0.00558) |  | 0.00643 (0.00622) | 34.6 → 71.6 |
| grover_n16 | `wai.statevector` | all (32) | 524452 | 16.3 (15.5) | 180 | 0.011 (0.0106) | 34.6 → 71.6 |
| grover_n16 | `wai.exact_whole` | 1 (no threads) | 2490472 | 151 (151) | 180 | 0.0148 (0.0146) | 35.1 → 71.6 |
| grover_n16 | `aer.statevector` | 32 | 189 | 0.0168 (0.0137) | 189 | 0.0194 (0.016) | 66.7 → 71.6 |
| grover_n16 | `qsim.statevector` | 32 |  | 0.0174 (0.0158) |  | 0.0151 (0.0134) | 66.7 → 73.7 |
| grover_n16 | `ddsim.statevector` | not set |  | 0.00663 (0.00625) |  | 0.00702 (0.00667) | 70.6 → 73.4 |
| grover_n20 | `wai.statevector` | all (32) |  | not run: line 34: modifiers add at most 16 controls here [Q208] | 224 | 0.0301 (0.0286) | 70.6 → 73.4 |
| grover_n20 | `wai.exact_whole` | 1 (no threads) |  | not run: line 34: modifiers add at most 16 controls here [Q208] | 224 | 0.322 (0.319) | 70.6 → 79.4 |
| grover_n20 | `aer.statevector` | 32 | 233 | 0.063 (0.0566) | 233 | 0.0588 (0.0546) | 70.6 → 79.4 |
| grover_n20 | `qsim.statevector` | 32 |  | 0.032 (0.0248) |  | 0.0254 (0.0247) | 70.6 → 79.4 |
| grover_n20 | `ddsim.statevector` | not set |  | 0.0179 (0.0166) |  | 0.0159 (0.0157) | 74.4 → 74.8 |
| clifford_n1000 | `wai.stabilizer` | 1 (no threads) | 35082 | 0.00102 (0.00101) | 35082 | 0.00106 (0.00103) | 78.1 → 74.8 |
| clifford_n1000 | `stim.tableau` | not set |  | 0.00163 (0.0016) |  | 0.0017 (0.00163) | 78.1 → 74.8 |
| clifford_n1000 | `aer.stabilizer` | 32 | 35083 | 0.0088 (0.00803) | 35083 | 0.00963 (0.00854) | 78.1 → 74.8 |
| clifford_n5000 | `wai.stabilizer` | 1 (no threads) | 58265 | 0.0124 (0.0112) | 58265 | 0.0127 (0.0123) | 85.1 → 77.3 |
| clifford_n5000 | `stim.tableau` | not set |  | 0.0389 (0.0299) |  | 0.0441 (0.0331) | 85.6 → 77.3 |
| surface_z_d5_sample | `wai.qec` | 1 (no threads) |  | 0.0913 (0.0879) |  | 0.0897 (0.0872) | 85.6 → 77.3 |
| surface_z_d5_sample | `stim.tableau` | not set |  | 0.228 (0.222) |  | 0.224 (0.219) | 85.6 → 76.9 |
| surface_z_d5_model | `wai.qec` | 1 (no threads) |  | 0.00231 (0.00223) |  | 0.00231 (0.00226) | 86.8 → 76.9 |
| surface_z_d5_model | `stim.tableau` | not set |  | 0.00329 (0.00271) |  | 0.00279 (0.00269) | 86.8 → 76.9 |
| surface_z_d11_sample | `wai.qec` | 1 (no threads) |  | 0.968 (0.962) |  | 0.997 (0.981) | 86.8 → 76.9 |
| surface_z_d11_sample | `stim.tableau` | not set |  | 2.21 (2.19) |  | 2.18 (2.18) | 88.7 → 77.1 |
| surface_z_d11_model | `wai.qec` | 1 (no threads) |  | 0.0251 (0.0245) |  | 0.0264 (0.0243) | 87.2 → 75.9 |
| surface_z_d11_model | `stim.tableau` | not set |  | 0.0443 (0.0405) |  | 0.04 (0.0399) | 87.2 → 74.7 |
| qft_n40_amplitudes | `wai.mps` | 1 (no threads) |  | 0.0163 (0.0161) |  | 0.0167 (0.0159) | 89.1 → 74.7 |
| qft_n40_amplitudes | `aer.mps` | 32 | 861 | 0.00552 (0.00538) | 861 | 0.00552 (0.00523) | 89.1 → 74.7 |
| qft_n40_amplitudes | `quimb.mps` | not set |  | 1.96 (1.93) |  | 1.93 (1.92) | 89.1 → 74.7 |
| qft_n40_amplitudes | `wai.tensor_network` | all (32) |  | 0.188 (0.187) |  | 0.197 (0.194) | 86.7 → 65.7 |
| qft_n40_amplitudes | `quimb.tensor_network` | not set |  | 0.985 (0.972) |  | 0.934 (0.916) | 85.4 → 65.7 |

<!-- end of results -->

## Limits

- **The bounds are loose.** They are worst cases, so an error smaller than a bound passes, and
  a double-precision engine usually measures a hundred times or more below its bound. The
  ratios in the tables say how much room each check left. The sharp checks are the exact ones:
  the referee's bit-for-bit rerun, the bit-exact op-log against the circuit, the held-or-dropped
  check of each small-angle gate, stabilizer groups as bytes, outcomes of probability 0, and
  symptom sets.
- **The integer state reference stops at 22 qubits**, in the full tier, and time stops it, not
  memory. On the build host (one core of an M3 Ultra) one circuit's reference took 34 s and
  0.25 GiB for the QFT at 20 qubits (231 gates), 81 s for 148 dyadic rotations at 20, and 146 s
  and 0.94 GiB for the QFT at 22; each two qubits more take four to five times as long, so 26
  qubits would fit in about 15 GiB and take hours a circuit. Wider circuits are held to closed
  forms (QFT, GHZ), to mirror circuits, and to each other. The referee reruns the bit-exact
  method's op-log where that is at most 2^30 amplitude updates; a longer one is recorded as not
  rerun, with its length.
- **Sampling is checked by marginals and by L1 distance.** Each outcome's count is held to its
  probability, and the whole distribution to the exact one; the independence of shots is not
  tested. At the family-wise rate of 10^-6 a test fails on a deviation of about six standard
  deviations: on 100,000 shots of a fair outcome, about 1,000 shots, and in L1 distance about
  0.02, or 0.045 over 257 outcomes at 200,000 shots. A smaller fault passes; each check records its
  bound. At 40 qubits a sampler confined to a smaller space inside the right one, with every bit
  still free, is not seen.
- **Not compared yet:** mitigation by `pea`, `pec` and `cdr` (their values rest on a learned
  noise model or a fitted line: no exact value holds them, beyond the ideal each states); the
  decoders but exact matching and union-find (correlated matching, BP-OSD, the sliding window, the
  code-capacity decoders), and union-find's logical rate beside matching's (recorded, not
  judged); pulses with decay, three levels or couplings (the Rabi formula covers one undamped
  qubit on two levels); the fabric's members other than the GPU ones (the neural, tensor,
  dataflow, FPGA, DSP and vector members), the `statevector` and `pauli` methods placed on an
  accelerator (no host here has a member certified for the dense f64 state), and the CUDA member
  (no nightly host has the device); single-precision GPU state vectors of the other engines.
- **What the engine under test refuses** is listed in
  [`corpus/expected-refusals.json`](corpus/expected-refusals.json), with the words it must give.
  Its frame simulator refuses a Pauli channel whose exclusive outcomes no independent flips can
  give (`PAULI_CHANNEL_1(0.1, 0.1, 0)`, whose Z would need a rate below zero), with its line: a
  detector error model of independent mechanisms cannot state it either. A general Pauli channel
  it can take (`qec_pauli_channel`) it writes as the independent flips with the channel's law, and
  its model matches the other engine's mechanism by mechanism. A job
  naming method `mps` runs under a meter of the bonds it holds (`mps_qft_n30_bond256`) and stops
  with `[J011]` if those pass one job's work; one naming `auto` keeps to the estimate at the bond
  asked for.
- **What the adapters do not drive.** The qsim adapter runs no measurement in the middle of a
  circuit, so those entries are `not run` for it; that is this adapter's limit. `aer.mps` is
  not run at 100 qubits: its target holds 63. An engine that has not answered a circuit within
  900 s is stopped, with every process it started, and is recorded as not run for it, or as a
  failure when it is the engine under test.
- **Platforms.** The pinned wheels exist together for macOS on arm64 and Linux on x86_64 only.
- **Performance** is one shared host a file, on one day. Other work on the host moves a timing
  by several times; the load average before and after each case is beside it. Timers do not all
  cover the same work: the list above each table says what each covers.
- **One author** wrote the adapters, the references and the tolerances. A reproduction by
  someone else, and adapters reviewed by each engine's own users, are what this page still needs.

## A provider of the open protocol

[`provider/seeded_statevector_provider.py`](provider/seeded_statevector_provider.py) wraps a
third-party state-vector simulator as a `process` provider of `wai.quantum.provider/1`
([the extension](../extensions/quantum-providers.md)). It is also the template for a device of
your own: replace the `Device` class's `run` with a call to your device.

```sh
wai-quantum providers add sim --kind process -- python3 -I provider/seeded_statevector_provider.py
wai-quantum providers test sim        # the protocol's conformance vectors
wai-quantum run bell.qasm --provider sim --shots 1000 --json > job.json
wai-quantum verify job.json
```

[`provider/test.sh`](provider/test.sh) does exactly that in a directory of its own, as part of
`reproduce.sh` and of CI: every conformance vector passes, the counts are sealed as `measured` on a
`simulator`, the receipt verifies, and the same job gives the same counts twice (the device is
seeded; on the two hosts here it gave the same counts, too).

## CI

`.github/workflows/wai-quantum-cli.yml` runs the CI tier on Linux (the circuits marked `ci` in the
manifest: every kind of check but the pairwise comparison of wide circuits), the self-test, the
provider template, and `tools/report.py --check`, which holds this page's results section to
`results/`, on every change.

`.github/workflows/wai-quantum-cross-simulator.yml` runs the rest every night, with no person:
the full tier on Linux, each job stopped at 900 s or when its processes hold more than 10 GiB
(`--memory-gb`), the whole run at 190 minutes (at two jobs the full tier took 66 minutes on the
build host and 77 on the Linux host of the results above), then the self-test; and the gpu tier on the macOS
host (WebGPU on Metal, the Metal member, the single-precision methods) and on the Linux host with
an integrated GPU (WebGPU on each Vulkan adapter, the GPU's and the software driver's). Each host
declares the members it must have (`QCORPUS_GPU_EXPECT`), and a declared member that ran no check
fails the run; a member it lacks is recorded as not run, with why; then the tier's fault must be
caught. The integer reference and the referee run in the harness's own process, outside the
guard: it held 7.6 GB at its peak on the build host, with the 22-qubit QFT.

## Files

| path | what |
|---|---|
| `reproduce.sh` | the one command |
| `env/requirements.in`, `env/requirements.lock` | the pinned engines, and every wheel's hash for macOS and Linux |
| `corpus/` | the manifest, the committed programs and error-correction circuits, the hand-written ones, the refusals the engine under test may give, and the known disagreements with their reproducers |
| `tools/corpus.py` | the circuits |
| `tools/exact.py` | the references: the integer state vector and stabilizer tableau, the integer density matrix, Pauli expectation values, the Rabi formula |
| `tools/decoding.py` | the decoders' reference: the least-weight correction of a syndrome, by exhaustion |
| `tools/tolerance.py` | every tolerance and where it comes from |
| `tools/engines.py` | one adapter per engine and method |
| `tools/compare.py` | the run: references, tolerances, injected faults, the results file |
| `tools/perf.py` | run time and peak memory |
| `tools/report.py` | this page's results section |
| `tools/before_after.py` | one host's timings of two builds side by side, from `tools/perf.py` run on each |
| `provider/` | the provider template and its test |
| [`../wai-quantum/examples/fabric_state.rs`](../wai-quantum/examples/fabric_state.rs) | the gpu tier's driver: a program's bit-exact state on the CPU and on every GPU member a build finds |
| `results/` | the committed runs |