wai-quantum: a cross-simulator conformance corpus, reproducible with one command, and the surface-code generator's text at p = 0 #662

Merged

dcharlot merged 9 commits from wai/quantum-cross-simulator-corpus into main 2026-11-07 01:27:16 -05:00

Conversation 0 Commits 9 Files changed 173 +64132 -16

dcharlot commented 2026-11-05 16:56:41 -05:00

Owner

Copy link

Phase 0 of the platform blueprint: the engine run beside other simulators on a published set of circuits, every answer held to an exact reference within a tolerance derived from the arithmetic that produced it, and all of it regenerated by one command on a clean machine.

What is here

• wai/quantum-cross-simulator/: the corpus (circuits in families, each with its parameters, its reason and what it must give; corpus/manifest.json pins every one's SHA-256), integer references (tools/exact.py), derived tolerances (tools/tolerance.py), one adapter per engine and method (tools/engines.py), the run (tools/compare.py), run time and memory (tools/perf.py), the page (README.md, its results section written from results/*.json by tools/report.py), and the one command, reproduce.sh.
• corpus/expected-refusals.json: the refusals the engine under test may or must give, with their words. corpus/known-disagreements.json: another engine's disagreements, each with a reproducer in corpus/reproducers/.
• wai/wai-quantum-cli/examples/cross_simulator.rs: the engine's side, one job in and one answer out, through the calls the command line and the job service make.
• provider/seeded_statevector_provider.py: a third-party simulator as a process provider of wai.quantum.provider/1, and the template to copy. wai-quantum providers test passes 31 of 31, and run --provider seals its counts in a receipt that verify accepts.
• CI (wai-quantum-cli.yml, the Linux leg): the CI tier, the self-test, the provider template, and report.py --check.

After review

An independent review confirmed 18 findings on the first head (3e6ab36d2). Each fix was first shown as a planted fault that the earlier tools let pass, then as a failure of the check it breaks; the planted faults are now the self-test's, which requires each to fail its own check on its own engine and no other.

finding	what was planted, and what the earlier tools did	now
A crash, refusal or hang of the engine under test is not run or limit, and the run passes (two findings)	the engine refusing every MPS job, crashing on the job path, hanging in the QEC sampler (its driver was left running), refusing a model in words no list holds, crashing on the random-detector circuit ("both refuse" passed), answering with no observable: each PASS	each fails the run; corpus/expected-refusals.json lists the refusals that are meant, with their words; a time-out kills the whole process group
The bit-exact method's must-refuse check and a closed-form row cannot fail; checks on it never run (three findings)	a reader that rounds angles to 2^-32 (rotations_n6 accepted), a refusal in other words, a 0.5 rad global phase on qft_wide_n20: each PASS	each fails; the must-refuse list is the reader's rule on exact angles, checked against the file; no row is exempt; gate_gphase_controlled runs on every state engine (quimb.* and qsim.statevector record not run)
repeated_phase_k19 cannot see its phase	a half turn on the state (a phase ignored, doubled or of the wrong sign): PASS	the entry is held to the state with its phase, beside a quarter turn a wrong sign cannot pass, and to the extension's stated modulus and angle; the README's figure is relabelled
aer.mps's bound is ten times the amplitude it checks	a 0.5 rad phase on every amplitude of the 40-qubit QFT: PASS (bound 9.6e-6 on 9.5e-7)	FAIL (bound 1.37e-11, from aer.mps's own log of its updates, bonds and truncations); quimb.mps runs with an absolute cutoff whose drop is known; a row whose bound cannot see a wrong phase is recorded, not passed
Small-angle entries pass with the gate dropped	a phase dropped from the bit-exact op-log, integers and hash made again (small_angle_k32); the state without the gate on the double state vector (below_grid_k48): each PASS	each fails: the op-log's ideal state is held to the circuit's, and every engine is checked held or dropped, exactly. The decision-diagram engine's drops at k ≥ 47 are recorded, below its parameter tolerance
The 2% sampling fault passes on a real distribution; the self-test's counts fault is caught by the probability-0 branch only (two findings)	with the Bernstein bound disabled, the self-test's counts fault still "caught"; every outcome ±10% on qft_counts_n8, a 40-qubit sampler on one outcome, an empty count map: each PASS	the counts fault moves 5% between two outcomes that can occur and must fail the per-outcome check by name; an L1 test (Weissman et al.) catches the ±10%; the 40-qubit entry checks shots returned, the most frequent outcome and each free bit. A 2% move on a fair coin is below any test at the run's error rate, and the README states each test's power
The page ranks the engine against named projects	—	the ranking prose is gone; the tables are data, each with where its timer runs; wai/CONTRIBUTING.md records the corpus's place under the public-text rule
The account of the other engine's failures misdescribes one program and has three unreproduced claims	—	repeat_until_success is described as it is (no reset); every claim is a case of a reproducer in corpus/reproducers/, rerun with the circuit; one claim is new: the error needs no conditioned x
aer.stabilizer's tableau time is mostly per-instruction handling	—	aer.*'s figure is the simulator's own experiment time (time_taken), the whole call kept beside it; each engine's timer is stated
Linux peak memory has a floor set by perf.py	—	each figure is the peak of the process that ran the engine, as the system counts it for that process alone (VmHWM on Linux, from the driver itself for the engine under test)
The threads column says 32 for engines given no setting	—	threads says what each was given: all, 1 (no threads), the count, or not set; fusion and ops are recorded
The write-up states three things no run performs	—	the phase table is checked entry for entry against the referee's, mirror_n10 by the referee, and ops come from the driver into the timing files; QPE and Grover closed forms are checked in integers
CI skips referee changes, and its comment claims families it does not run	—	the referee is on both path lists; the CI tier has every kind of check but the wide pairwise one, and the comment says so
Provenance names HEAD's trees for a dirty checkout	an edited tools/ tree recorded as HEAD's id	the tree as it is on disk, marked (uncommitted) when it differs from HEAD's, and the toolchain the checkout picks

Results

file	host	pass	fail	known	refused as listed	not run	recorded	time
conformance-linux-x86_64.json	AMD RYZEN AI MAX+ 395 w/ Radeon 8060S, Ubuntu 26.04.1 LTS	1637	0	11	12	12	12	2705 s
conformance-macos-arm64.json	Apple M3 Ultra, macOS 27.2	1638	0	11	12	11	12	2100 s

The first head's files had 1363 and 1368 passes, limit 3, and no op-log, closed-form-in-integers, held-or-dropped, L1 or reproducer rows. What moved in the published tables:

• New rows: the bit-exact op-log against the circuit, on every circuit the method runs; QPE (outcome of probability 1, and the law for 1/3), Grover (in integers) and the phase table, checked in the reference; held-or-dropped on every small-angle circuit; L1 on every sampled distribution; shots, most frequent outcome and free bits at 40 qubits; the reproducers' cases. The largest measured/tolerance ratios in the sampled families rise to about 0.6, the L1 rows'.
• Refusals: the bit-exact method's refusals of non-dyadic angles were counted as passes (random_rotations 3/3); they are now refused as listed, with the four error-correction refusals.
• Matrix product states: bounds from each engine's own report tighten by three to seven orders; aer.mps's largest ratio goes from 3.6e-11 to 2.3e-4 of its bound.
• Recorded, not judged: the decision-diagram engine drops the phase at k = 47, 48, 52 and 60, each below its parameter tolerance of 1e-13 rad; at k = 60 most double-precision answers' own rounding exceeds the gate's effect.
• repeated_phase_k19: modulus 1.0000188705 and 8.99e-6 rad from the half turn (2.09e-5 from −|1⟩), against the extension's 1.0000189 and 1e-5; the page called 1.89e-5 the distance, which was the modulus excess.
• Not run: three new rows (gate_gphase_controlled, which quimb.* and qsim.statevector have no gate for). On Linux the decision-diagram engine again passes 900 s on mirror_n20; rotations_wide_n20 now finishes.
• Timings: aer.*'s figure is the simulator's own experiment time, so its tableau case (clifford_n1000) reads 0.022 s on Linux (was 0.296) and 0.0076 s on macOS (was 0.199), its whole call beside it. Linux memory no longer sits on the harness's floor: the engine under test's baselines are about 5 MiB (were 48), surface_z_d5_model 6.7 MiB for it and 26 MiB for stim.tableau (were 94 each). Threads say what each engine was given. The tables carry no ranking prose.

The planted-fault runs on the earlier tools and these, side by side, are logged (not committed) at os-wt-qcorpus-old/_demo/demos-2026-11-06.log on the dev Mac.

An engine bug, fixed in its own commit

112cc14bd: quantum_frame::surface_code_memory documents that its text is the reference generator's, byte for byte. At p = 0 it was not: it wrote every noise instruction with probability 0 and the first round's resets on two lines. The circuits meant the same, so models and samples were unaffected; only the text differed. A fixture test against the reference's text failed before the change and passes after. The fixture count pinned in wai-quantum-processors.yml and wai-quantum-dsp.yml goes from 2 to 3.

Reproducing

bash wai/quantum-cross-simulator/reproduce.sh --tier full
bash wai/quantum-cross-simulator/reproduce.sh --self-test

macOS on arm64 or Linux on x86_64, with curl, tar, a C compiler and rustup. It fetches a pinned uv, CPython 3.12.16 and every wheel by hash, and builds the engine --locked. Each results file names the git trees it was produced from, as they were on disk (git rev-parse <commit>:wai/wai-quantum), and the toolchain the checkout picks.

Notes for review

• The engine fix touches wai/wai-quantum/src, so this pull request fires every wai-quantum workflow.
• The CI step downloads about 250 MB of wheels on a runner without ~/.cache. The full tier is not in CI.
• wai/CONTRIBUTING.md records how this corpus meets the public-text rule: results per engine as data, projects named only on allow-name: lines, no ranking.
• Left for the stacked reader-scaling pull request: gates under many controls, and the three behaviours the corpus showed silent (the bit-exact method's global phase, angles below its table, the matrix-product estimate at max_bond 256), and C_XYZ, E and PAULI_CHANNEL_1.

🤖 Generated with Claude Code

Phase 0 of the platform blueprint: the engine run beside other simulators on a published set of circuits, every answer held to an exact reference within a tolerance derived from the arithmetic that produced it, and all of it regenerated by one command on a clean machine. ## What is here - `wai/quantum-cross-simulator/`: the corpus (circuits in families, each with its parameters, its reason and what it must give; `corpus/manifest.json` pins every one's SHA-256), integer references (`tools/exact.py`), derived tolerances (`tools/tolerance.py`), one adapter per engine and method (`tools/engines.py`), the run (`tools/compare.py`), run time and memory (`tools/perf.py`), the page (`README.md`, its results section written from `results/*.json` by `tools/report.py`), and the one command, `reproduce.sh`. - `corpus/expected-refusals.json`: the refusals the engine under test may or must give, with their words. `corpus/known-disagreements.json`: another engine's disagreements, each with a reproducer in `corpus/reproducers/`. - `wai/wai-quantum-cli/examples/cross_simulator.rs`: the engine's side, one job in and one answer out, through the calls the command line and the job service make. - `provider/seeded_statevector_provider.py`: a third-party simulator as a `process` provider of `wai.quantum.provider/1`, and the template to copy. `wai-quantum providers test` passes 31 of 31, and `run --provider` seals its counts in a receipt that `verify` accepts. - CI (`wai-quantum-cli.yml`, the Linux leg): the CI tier, the self-test, the provider template, and `report.py --check`. ## After review An independent review confirmed 18 findings on the first head (`3e6ab36d2`). Each fix was first shown as a planted fault that the earlier tools let pass, then as a failure of the check it breaks; the planted faults are now the self-test's, which requires each to fail its own check on its own engine and no other. | finding | what was planted, and what the earlier tools did | now | |---|---|---| | A crash, refusal or hang of the engine under test is `not run` or `limit`, and the run passes (two findings) | the engine refusing every MPS job, crashing on the job path, hanging in the QEC sampler (its driver was left running), refusing a model in words no list holds, crashing on the random-detector circuit ("both refuse" passed), answering with no observable: each PASS | each fails the run; `corpus/expected-refusals.json` lists the refusals that are meant, with their words; a time-out kills the whole process group | | The bit-exact method's must-refuse check and a closed-form row cannot fail; checks on it never run (three findings) | a reader that rounds angles to 2^-32 (`rotations_n6` accepted), a refusal in other words, a 0.5 rad global phase on `qft_wide_n20`: each PASS | each fails; the must-refuse list is the reader's rule on exact angles, checked against the file; no row is exempt; `gate_gphase_controlled` runs on every state engine (`quimb.*` and `qsim.statevector` record `not run`) | | `repeated_phase_k19` cannot see its phase | a half turn on the state (a phase ignored, doubled or of the wrong sign): PASS | the entry is held to the state with its phase, beside a quarter turn a wrong sign cannot pass, and to the extension's stated modulus and angle; the README's figure is relabelled | | aer.mps's bound is ten times the amplitude it checks | a 0.5 rad phase on every amplitude of the 40-qubit QFT: PASS (bound 9.6e-6 on 9.5e-7) | FAIL (bound 1.37e-11, from `aer.mps`'s own log of its updates, bonds and truncations); `quimb.mps` runs with an absolute cutoff whose drop is known; a row whose bound cannot see a wrong phase is recorded, not passed | | Small-angle entries pass with the gate dropped | a phase dropped from the bit-exact op-log, integers and hash made again (`small_angle_k32`); the state without the gate on the double state vector (`below_grid_k48`): each PASS | each fails: the op-log's ideal state is held to the circuit's, and every engine is checked held or dropped, exactly. The decision-diagram engine's drops at k ≥ 47 are recorded, below its parameter tolerance | | The 2% sampling fault passes on a real distribution; the self-test's counts fault is caught by the probability-0 branch only (two findings) | with the Bernstein bound disabled, the self-test's counts fault still "caught"; every outcome ±10% on `qft_counts_n8`, a 40-qubit sampler on one outcome, an empty count map: each PASS | the counts fault moves 5% between two outcomes that can occur and must fail the per-outcome check by name; an L1 test (Weissman et al.) catches the ±10%; the 40-qubit entry checks shots returned, the most frequent outcome and each free bit. A 2% move on a fair coin is below any test at the run's error rate, and the README states each test's power | | The page ranks the engine against named projects | — | the ranking prose is gone; the tables are data, each with where its timer runs; `wai/CONTRIBUTING.md` records the corpus's place under the public-text rule | | The account of the other engine's failures misdescribes one program and has three unreproduced claims | — | `repeat_until_success` is described as it is (no reset); every claim is a case of a reproducer in `corpus/reproducers/`, rerun with the circuit; one claim is new: the error needs no conditioned x | | `aer.stabilizer`'s tableau time is mostly per-instruction handling | — | `aer.*`'s figure is the simulator's own experiment time (`time_taken`), the whole call kept beside it; each engine's timer is stated | | Linux peak memory has a floor set by perf.py | — | each figure is the peak of the process that ran the engine, as the system counts it for that process alone (`VmHWM` on Linux, from the driver itself for the engine under test) | | The threads column says 32 for engines given no setting | — | threads says what each was given: `all`, `1 (no threads)`, the count, or `not set`; fusion and ops are recorded | | The write-up states three things no run performs | — | the phase table is checked entry for entry against the referee's, `mirror_n10` by the referee, and ops come from the driver into the timing files; QPE and Grover closed forms are checked in integers | | CI skips referee changes, and its comment claims families it does not run | — | the referee is on both path lists; the CI tier has every kind of check but the wide pairwise one, and the comment says so | | Provenance names HEAD's trees for a dirty checkout | an edited tools/ tree recorded as HEAD's id | the tree as it is on disk, marked `(uncommitted)` when it differs from HEAD's, and the toolchain the checkout picks | ## Results | file | host | pass | fail | known | refused as listed | not run | recorded | time | |---|---|---|---|---|---|---|---|---| | `conformance-linux-x86_64.json` | AMD RYZEN AI MAX+ 395 w/ Radeon 8060S, Ubuntu 26.04.1 LTS | 1637 | 0 | 11 | 12 | 12 | 12 | 2705 s | | `conformance-macos-arm64.json` | Apple M3 Ultra, macOS 27.2 | 1638 | 0 | 11 | 12 | 11 | 12 | 2100 s | The first head's files had 1363 and 1368 passes, `limit` 3, and no op-log, closed-form-in-integers, held-or-dropped, L1 or reproducer rows. What moved in the published tables: - **New rows:** the bit-exact op-log against the circuit, on every circuit the method runs; QPE (outcome of probability 1, and the law for 1/3), Grover (in integers) and the phase table, checked in the reference; held-or-dropped on every small-angle circuit; L1 on every sampled distribution; shots, most frequent outcome and free bits at 40 qubits; the reproducers' cases. The largest measured/tolerance ratios in the sampled families rise to about 0.6, the L1 rows'. - **Refusals:** the bit-exact method's refusals of non-dyadic angles were counted as passes (`random_rotations 3/3`); they are now `refused as listed`, with the four error-correction refusals. - **Matrix product states:** bounds from each engine's own report tighten by three to seven orders; aer.mps's largest ratio goes from 3.6e-11 to 2.3e-4 of its bound. - **Recorded, not judged:** the decision-diagram engine drops the phase at k = 47, 48, 52 and 60, each below its parameter tolerance of 1e-13 rad; at k = 60 most double-precision answers' own rounding exceeds the gate's effect. - **`repeated_phase_k19`:** modulus 1.0000188705 and 8.99e-6 rad from the half turn (2.09e-5 from −|1⟩), against the extension's 1.0000189 and 1e-5; the page called 1.89e-5 the distance, which was the modulus excess. - **Not run:** three new rows (`gate_gphase_controlled`, which `quimb.*` and `qsim.statevector` have no gate for). On Linux the decision-diagram engine again passes 900 s on `mirror_n20`; `rotations_wide_n20` now finishes. - **Timings:** `aer.*`'s figure is the simulator's own experiment time, so its tableau case (`clifford_n1000`) reads 0.022 s on Linux (was 0.296) and 0.0076 s on macOS (was 0.199), its whole call beside it. Linux memory no longer sits on the harness's floor: the engine under test's baselines are about 5 MiB (were 48), `surface_z_d5_model` 6.7 MiB for it and 26 MiB for `stim.tableau` (were 94 each). Threads say what each engine was given. The tables carry no ranking prose. The planted-fault runs on the earlier tools and these, side by side, are logged (not committed) at `os-wt-qcorpus-old/_demo/demos-2026-11-06.log` on the dev Mac. ## An engine bug, fixed in its own commit `112cc14bd`: `quantum_frame::surface_code_memory` documents that its text is the reference generator's, byte for byte. At p = 0 it was not: it wrote every noise instruction with probability 0 and the first round's resets on two lines. The circuits meant the same, so models and samples were unaffected; only the text differed. A fixture test against the reference's text failed before the change and passes after. The fixture count pinned in `wai-quantum-processors.yml` and `wai-quantum-dsp.yml` goes from 2 to 3. ## Reproducing ```sh bash wai/quantum-cross-simulator/reproduce.sh --tier full bash wai/quantum-cross-simulator/reproduce.sh --self-test ``` macOS on arm64 or Linux on x86_64, with curl, tar, a C compiler and rustup. It fetches a pinned uv, CPython 3.12.16 and every wheel by hash, and builds the engine `--locked`. Each results file names the git trees it was produced from, as they were on disk (`git rev-parse <commit>:wai/wai-quantum`), and the toolchain the checkout picks. ## Notes for review - The engine fix touches `wai/wai-quantum/src`, so this pull request fires every wai-quantum workflow. - The CI step downloads about 250 MB of wheels on a runner without `~/.cache`. The full tier is not in CI. - `wai/CONTRIBUTING.md` records how this corpus meets the public-text rule: results per engine as data, projects named only on `allow-name:` lines, no ranking. - Left for the stacked reader-scaling pull request: gates under many controls, and the three behaviours the corpus showed silent (the bit-exact method's global phase, angles below its table, the matrix-product estimate at `max_bond` 256), and `C_XYZ`, `E` and `PAULI_CHANNEL_1`. 🤖 Generated with [Claude Code](https://claude.com/claude-code)

dcharlot added 3 commits 2026-11-05 16:56:41 -05:00

wai-quantum: the surface-code memory experiment at p = 0 writes the reference generator's text ... 112cc14bd3

quantum_frame::surface_code_memory documents that its text matches the
ecosystem's reference generator byte for byte. That held for every p > 0
and not for p = 0: there it wrote each noise instruction with probability
zero (DEPOLARIZE1(0) ..., X_ERROR(0) ...) and the first round's two resets on
two lines, where the reference writes no noise instruction and one reset.
The circuits meant the same thing, so the model and the samples were
unaffected; only the text differed.

It now writes no noise instruction of probability zero, and the two resets
as one instruction. A new fixture test, the reference generator's text at
d = 5 with four rounds and p = 0 (tests/data/memory_z_d5_noiseless.circuit),
failed before the change and passes after it. The fixture count pinned in
wai-quantum-processors.yml and wai-quantum-dsp.yml goes from 2 to 3.

The cross-simulator corpus found it (entry surface_z_d5_noiseless): the
engine's own generator checked against the pinned reference generator.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

wai-quantum: a cross-simulator conformance corpus, reproducible with one command ... ec4f2a5acc

wai/quantum-cross-simulator/ runs the engine beside pinned third-party
simulators on a published set of circuits and holds every answer to an
exact reference within a tolerance derived from the arithmetic that
produced it.

The corpus (tools/corpus.py, corpus/manifest.json with each circuit's
SHA-256): bit order, GHZ, QFT, Grover, phase estimation, random
Clifford+T, dyadic and arbitrary-angle circuits, mirror circuits, angles
at and below the fixed-point table's resolution, every gate of OpenQASM
3's standard library, measurement in the middle with reset and feedback,
stabilizer circuits to 5000 qubits, noise in closed form, wide circuits by
matrix product states and contraction, and error-correction memory
experiments to distance 15.

The references (tools/exact.py, standard library only): a state vector in
integers at scale 2^-256, exact distributions by walking every branch of a
dynamic program, and an integer stabilizer tableau with the reduced form
of a group. The bit-exact method's op-log is rerun by the existing referee
and must give the same integers and hash.

The tolerances (tools/tolerance.py): the bit-exact method's from the
extension's published bound summed over the op-log it ran; floating point
from a stated per-gate rounding model at each engine's fusion width;
sampled counts by Bernstein's inequality at a family-wise rate of 1e-6.
None is fitted to a result.

reproduce.sh fetches a pinned uv, CPython and every wheel by hash, builds
the engine with --locked, runs every check and prints PASS or FAIL.
--self-test injects six faults and passes only when each fails the run on
its own check. --perf measures run time and peak memory per engine.

provider/seeded_statevector_provider.py wraps a third-party simulator as a
process provider of wai.quantum.provider/1; provider/test.sh holds it to
the protocol's conformance vectors through `providers test`, runs it with
`run --provider` and verifies the receipt.

wai-quantum-cli/examples/cross_simulator.rs is the engine's side: one job
in, one answer out, through the calls the command line and the job service
make.

CI: the Linux leg of wai-quantum-cli.yml runs the CI tier, the self-test,
the provider template and the check that the page's results section is
what the committed results say.

Co-Authored-By: Claude Fable 5.1 <[email protected]>

wai-quantum: the cross-simulator corpus's results on two hosts, and its page ... 3e6ab36d2f

results/ holds four runs, each naming its host, every version and the git
trees of the engine and tools that produced it:

- conformance on macOS arm64 (M3 Ultra) and on Linux x86_64 (Ryzen AI Max+
  395), the full tier, from the tools as committed in the previous commit:
  1368 and 1363 checks pass, none fails. Two checks on each host are
  recorded as known disagreements of another engine with the exact
  reference, three are refusals by the engine under test of instructions
  its frame simulator does not implement, and seven (nine on Linux) were
  not run, each with its reason.
- run time and peak memory on the same two hosts, five runs a case, with
  the host's load average beside every case.

The page's results section is written from those files by tools/report.py,
and CI holds it to them. The page also says what disagreed and who was
right, what the corpus shows about conventions, where other engines are
faster (gates under many controls, and single precision on one thread),
and the limits of the comparison.

Three changes to the tools since the previous commit, none of which the
conformance run reads: a timing run on the stabilizer adapter measures one
qubit rather than reading the whole group out through the save
instruction's result, which had been most of its time; the memory baseline
uses a two-qubit job, which every adapter accepts; and the page's tables
print whole seconds and MiB, and a refusal's own words.

Co-Authored-By: Claude Fable 5.1 <[email protected]>

dcharlot referenced this pull request from a commit 2026-11-06 03:23:42 -05:00

ci: the wai-quantum legs build the commit they test, and fail when they did not

dcharlot referenced this pull request 2026-11-06 03:23:54 -05:00

ci: the wai-quantum legs build the commit they test, and fail when they did not #666

dcharlot added 4 commits 2026-11-06 08:04:46 -05:00

wai-quantum: the cross-simulator corpus's checks can fail where the review found they could not ... 3ba49a43ee

Each change below was shown first as a planted fault that the previous tools let
pass, then as a failure of the check it breaks:

- The engine under test is held to answering. A crash, a time-out or a
  refusal that corpus/expected-refusals.json does not list, in its words,
  fails the run; so does an answer to a circuit it must refuse. The bit-exact
  method's refusals are listed by its reader's rule, computed from the exact
  angles. A time-out kills the engine's whole process group.
- The bit-exact method's op-log, run in double precision, must give the
  circuit's exact state: a phase dropped in lowering now fails, where the
  distance and the referee could not see it. Each small-angle gate is checked
  held or dropped, exactly, on every engine.
- repeated_phase_k19 is held to the state with its phase, beside a quarter
  turn that a wrong sign cannot pass, and to the extension's stated figures.
- No closed-form row is exempt for the bit-exact method. QPE, Grover and the
  phase table are checked in integers, mirror_n10 by the referee.
- Matrix product states are charged what each engine reports it truncated;
  a row whose bound cannot see a wrong phase is recorded, not passed.
- Sampled counts are also held as a whole in L1 distance; the 40-qubit
  Clifford outcomes per outcome, per free bit and in number.
- The account of another engine's errors is corrected, and each claim is a
  case of a reproducer committed beside it.
- Timing records where each engine's timer runs (Aer's own experiment time),
  memory the peak of the process that ran the engine, threads what each was
  given, and ops what the engine ran.
- Provenance names the trees as they were on disk, and the toolchain the
  checkout picks. CI covers every kind of check but the wide pairwise one, on
  referee changes too, and its comment says so.
- The self-test injects every fault and requires its own check to fail, on
  its own engine only.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

wai-quantum: the held-or-dropped check decides only where an answer can say ... c5702b44e0

The first full run on the M3 Ultra failed two checks on Aer at
below_grid_k60, whose phase of 2π/2^60 moves the state by 2.7e-18, far
below a double's rounding of an amplitude near 1: the share of the step its
answer took was rounding (−21), not a dropped gate. An engine now holds the
gate when its answer is within half the gate's effect of the ideal state,
drops it when within half of the state without it, and otherwise is recorded
as unable to say, not judged. A dropped gate inside the bound still fails
(the drop_phase fault at k = 48 is within 0 of the state without it).

The known disagreement on measure_twice now names the L1 check too, which
the same fault of the other engine fails.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

wai-quantum: the cross-simulator corpus rerun on both hosts, and its page as data ... fffd228757

Both hosts' full tiers, rerun on the tools of the two commits before this
one (their trees named in each results file):

- macOS arm64 (M3 Ultra): 1638 checks pass, 0 fail; 11 known (the other
  engine's two disagreements, now with their reproducers' cases), 12
  refusals as listed, 11 not run, 12 recorded and not judged.
- Linux x86_64 (Ryzen AI Max+ 395): 1637 pass, 0 fail; 11 known, 12
  refused as listed, 12 not run (the other engine's mirror_n20 timed out
  at 900 s), 12 recorded.
- The self-test catches every planted fault, each on its own check.

Timings rerun on both hosts with each engine's timer stated, its threads,
its ops, and the peak memory of the process that ran it.

The page states results as data: the ranking prose is gone, the account
of the other engine is corrected, repeated_phase_k19's figures are the
modulus and angle the extension states, and the limits say what each
sampled test can and cannot see. report.py prints each reproducer case.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

wai-quantum: the corpus page gives the full tier's time on both hosts ... 5f60cfcf89

Co-Authored-By: Claude Opus 5.5 <[email protected]>

dcharlot referenced this pull request 2026-11-06 09:15:29 -05:00

wai-quantum: a gate under many controls stays one operation on the state vector, the bit-exact method states what its lowering did, and the error-correction reader takes C_XYZ, E and PAULI_CHANNEL_1 #675

dcharlot added 2 commits 2026-11-07 01:26:36 -05:00

Merge main into the cross-simulator corpus ... bdb041e096

The workflow keeps both new steps and both sets of paths: the corpus's CI
tier and main's job receipts by two verifiers.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

wai-quantum: the provider template's run reads as a simulator's ... 90f48f6a75

Main now gives a simulator's result the class `simulated` (the providers
extension, section 8.1); the template's test still held it to `measured`
and failed on the merge.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

dcharlot merged commit 30e70601dc into main 2026-11-07 01:27:16 -05:00

dcharlot referenced this pull request from a commit 2026-11-07 01:27:22 -05:00

Merge pull request 'wai-quantum: a cross-simulator conformance corpus, reproducible with one command, and the surface-code generator's text at p = 0' (#662) from wai/quantum-cross-simulator-corpus into main

Sign in to join this conversation.

Reviewers

No reviewers

Labels

Clear labels

No items

No labels

Milestone

Clear milestone

No items

No milestone

Projects

Clear projects

No items

No project

Assignees

Clear assignees

No assignees

1 participant

Notifications

Subscribe

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference

Transaction-Science/open-standards!662

No description provided.