open-standards/wai/quantum-conformance/landscape/README.md
Transaction Science 40ccfdbc01
Some checks failed
WAI breaking changes and the conformance runner / WAI breaking · does this change touch WAI (push) Successful in 35s
wai-quantum wasmQ / does this change touch it (push) Successful in 19s
wai-quantum energy-landscape jobs / does this change touch it (push) Successful in 22s
WAI operation catalogue / does this change touch it (push) Successful in 24s
joule-moq-remote host agent / joule-moq-remote · does this change touch it (push) Successful in 44s
wai-app and Create / wai-app · does this change touch it (push) Successful in 37s
wai-fleet service / does this change touch it (push) Successful in 29s
joule-moq-web browser client / joule-moq-web · does this change touch it (push) Successful in 39s
wai-audit collector and player integration / does this change touch it (push) Successful in 51s
wai-quantum other formats / does this change touch it (push) Successful in 25s
wai-quantum for JavaScript / does this change touch it (push) Successful in 24s
wai-quantum editors / does this change touch it (push) Successful in 33s
wai-quantum wasmQ / the reference modules rebuild to their modules of record · linux (push) Successful in 39s
wai-quantum-landscape-fpga reference design / does this change touch it (push) Successful in 25s
wai-quantum contract / does this change touch it (push) Successful in 1m16s
joule-moq-room room service / joule-moq-room · does this change touch it (push) Successful in 1m29s
wai-ledger service / does this change touch it (push) Successful in 1m17s
wai-quantum clippy / does this change touch it (push) Successful in 1m18s
wai-quantum resource estimation / does this change touch it (push) Successful in 45s
wai-quantum command line / does this change touch it (push) Successful in 1m16s
wai-quantum for Python / does this change touch it (push) Successful in 21s
wai-quantum docs, course and locales / does this change touch it (push) Successful in 1m17s
wai-quantum error correction and mitigation / does this change touch it (push) Successful in 23s
wai-quantum pulse and dynamics / does this change touch it (push) Successful in 19s
wai-quantum OpenQASM reader / does this change touch it (push) Successful in 35s
wai-app and Create / wai-app · tests, clippy, the line language's verifier (linux) (push) Successful in 44s
wai-quantum qudit jobs / does this change touch it (push) Successful in 20s
wai-sink-android native sink library / does this change touch it (push) Successful in 37s
WAI Studio / does this change touch it (push) Successful in 37s
wai-quantum verdicts and repairs / does this change touch it (push) Successful in 1m7s
wai-quantum views in agent hosts / does this change touch it (push) Successful in 1m5s
wai-quantum extension points / does this change touch it (push) Successful in 1m16s
wai-sink-swift native sink package / does this change touch it (push) Successful in 59s
wai-quantum studio and desktop / does this change touch it (push) Successful in 1m14s
wai-quantum variational and algorithm library / does this change touch it (push) Successful in 1m7s
joule-moq-remote host agent / joule-moq-remote, every target (linux-arm64) (push) Successful in 2m7s
wai-quantum-landscape-fpga reference design / the board's code on the board's processor's architecture · linux-arm64, release (push) Successful in 1m43s
wai-quantum service / does this change touch it (push) Successful in 1m11s
wai-fleet service / service · linux (push) Successful in 2m53s
wai-quantum wasmQ / interpreter and vectors under wasm32-wasip2 (wasmtime) (push) Successful in 3m56s
WAI breaking changes and the conformance runner / WAI breaking · wire, record and API against main · linux (push) Successful in 4m9s
repository checks / lockfiles, advisories, tests in CI, workflow hygiene, spec and section references, public text (push) Failing after 5m15s
wai-quantum wasmQ / profile, engines, reference modules, adversarial, vectors · linux (push) Successful in 4m38s
wai-quantum pulse and dynamics / the same bits · linux-arm64 (push) Successful in 3m39s
wai-quantum other formats / the set under wasm32-wasip2 (wasmtime) (push) Successful in 3m47s
wai-audit collector and player integration / collector · linux (push) Successful in 4m48s
wai-quantum service / service energy · linux packages (push) Successful in 3m48s
wai-quantum variational and algorithm library / parameters, gradients, optimisers, VQE, QAOA and builders · linux-arm64, release (push) Successful in 4m11s
wai-quantum contract / the policy is linked, and the deprecations page reads (push) Successful in 17s
wai-quantum-landscape-fpga reference design / the member against the model of the logic, and the logic against the model · linux, release (push) Failing after 2m54s
wai-quantum energy-landscape jobs / native against WebAssembly, over many seeds (push) Successful in 6m46s
wai-quantum other formats / readers, writers and the set · linux (push) Successful in 5m14s
wai-sink-android native sink library / the JNI layer and the platform-free half on the host JVM · linux-arm64 (push) Successful in 5m6s
wai-quantum editors / editors · linux (push) Successful in 6m35s
wai-quantum for JavaScript / javascript · linux (push) Successful in 4m20s
joule-moq-room room service / joule-moq-room, every target (push) Successful in 7m34s
wai-ledger service / service · linux (push) Successful in 3m29s
wai-quantum extension points / the registry and the example · linux-arm64 (push) Successful in 7m40s
wai-quantum docs, course and locales / the Studio's words in every locale · linux (push) Successful in 20s
WAI operation catalogue / the catalogue and every door · linux (push) Successful in 10m58s
WAI breaking changes and the conformance runner / WAI breaking and conform · their tests, and the independent verifiers through wai conform · linux (push) Successful in 11m25s
wai-quantum pulse and dynamics / reader, simulator and pulse jobs under wasm32-wasip2 (wasmtime) (push) Successful in 1m0s
wai-quantum error correction and mitigation / codes, decoders, texts and mitigation · linux, release (push) Successful in 4m21s
wai-quantum contract / API compatibility · linux (push) Successful in 8m58s
wai-quantum studio and desktop / desktop app · windows (push) Successful in 10m55s
wai-quantum clippy / clippy, warnings denied, every feature set · linux (push) Successful in 6m15s
wai-quantum qudit jobs / the module and the conformance set under wasm32-wasip2 (wasmtime) (push) Successful in 1m13s
wai-quantum for Python / python · linux (push) Successful in 6m5s
wai-quantum OpenQASM reader / corpus under wasm32-wasip2 (wasmtime) (push) Successful in 1m25s
wai-quantum OpenQASM reader / reader and corpus · linux (push) Successful in 2m38s
wai-quantum contract / schemas and the committed tool lists · linux (push) Successful in 10m27s
wai-quantum resource estimation / estimation and calibration · linux, release (push) Successful in 7m40s
wai-quantum pulse and dynamics / reader, simulator and pulse jobs · linux, release (push) Successful in 4m27s
wai-quantum verdicts and repairs / the intent set and the verdict under wasm32-wasip2 (wasmtime) (push) Successful in 1m38s
WAI Studio / the catalogue's actions, the unit tests and the core · linux (push) Successful in 2m8s
wai-quantum docs, course and locales / the course and the command line's languages · linux (push) Successful in 7m14s
wai-quantum verdicts and repairs / verdicts, repairs and the intent set · linux (push) Successful in 2m25s
wai-quantum energy-landscape jobs / the same bits · linux-arm64 (push) Successful in 17m4s
wai-quantum variational and algorithm library / parameters, gradients, optimisers, VQE, QAOA and builders · linux, release (push) Successful in 2m31s
wai-quantum extension points / the registry under wasm32-wasip2 (wasmtime) (push) Successful in 2m32s
wai-quantum energy-landscape jobs / reader, kernels, answers and the stated result · linux, release (push) Successful in 19m19s
wai-quantum views in agent hosts / views · the page and its engine (push) Successful in 4m31s
wai-quantum extension points / the registry and the example · linux (push) Successful in 4m49s
wai-quantum energy-landscape jobs / the module under wasm32-wasip2 (wasmtime) (push) Successful in 21m33s
wai-quantum qudit jobs / the module, the job, the conformance set, the referee and the verb · linux (push) Successful in 10m48s
wai-quantum studio and desktop / desktop app · linux (push) Successful in 7m17s
wai-quantum studio and desktop / studio · browser (push) Successful in 8m21s
wai-quantum views in agent hosts / views · a scripted host (push) Successful in 11m44s
wai-quantum wasmQ / profile, engines, reference modules, adversarial, vectors · macos (push) Successful in 2m4s
joule-moq-remote host agent / joule-moq-remote, every target (macos) (push) Successful in 47s
wai-quantum docs, course and locales / every command and snippet in the docs · linux (push) Successful in 22m55s
wai-fleet service / service · macos (push) Successful in 38s
wai-quantum service / service · linux (push) Successful in 17m40s
wai-quantum energy-landscape jobs / the same bits · macos (push) Successful in 7m26s
wai-app and Create / wai-app · Create, browser end-to-end (macos) (push) Successful in 7m25s
wai-quantum editors / editors · macos (push) Successful in 1m41s
wai-audit collector and player integration / collector, player integration and the browser end to end · macos (push) Successful in 2m8s
wai-ledger service / service · macos (push) Successful in 1m28s
WAI Studio / Studio in WebKit · linux (push) Successful in 27m7s
wai-quantum command line / command line · macos (push) Successful in 3m1s
wai-quantum command line / command line · linux (push) Successful in 34m5s
wai-quantum for Python / python · macos (push) Successful in 3m21s
wai-quantum pulse and dynamics / the same bits · macos (push) Successful in 1m54s
wai-quantum qudit jobs / the same bits · macos (push) Successful in 2m25s
joule-moq-web browser client / joule-moq-web · browser end-to-end (macos) (push) Successful in 16m12s
wai-sink-android native sink library / every ABI, the AAR, and the instrumented tests in a headless emulator · macos (push) Successful in 7m24s
wai-quantum studio and desktop / desktop app · macos (push) Successful in 4m51s
WAI Studio / Studio in Chromium and Firefox · macos (push) Successful in 10m55s
wai-quantum service / service · macos (push) Successful in 7m15s
wai-sink-swift native sink package / libwai slices, desktop tests, simulator tests · macos (push) Successful in 15m21s
wai-quantum: wai.quantum.landscape/2, a cost for each variable, constraints, joules priced only with evidence, and both conformance sets frozen
The second version of the energy-landscape job (extensions/quantum-landscape-2.md), alongside
the first, which answers byte for byte as before (register T1-1 and T4-5; holes H-7, H-9, R-30).

- Each variable has its own cost, of either sign (`cost I C`; `cost C` may be below 0). A
  variable whose cost is below 0 is never 0 in a state of lowest energy, so `exactly K I J ...`
  (one-hot and cardinality) and `atmost 1 I J ...` lower exactly to couplings of -1 and
  whole-number fields; `#! check exactly` and `#! check atmost` grade an answer.
- Kernels wai.quantum.landscape2/settle and /sample: each cell at its own cost; the settle move
  is the cell's lowest energy, a tie to 0 then to +, the first version's move wherever a cost
  is 0 or more. A request names the version with "protocol"; none is the first.
- Joules: a run at temperature 0 priced at about 0.88 pJ an accumulate, the open FPGA member's
  measured settle figure, with its provenance, caveats and files under `evidence`; a run above
  it is "unpriced", with why and no figure. The first version keeps its rates (its replay
  lines hash whole answers) and its section 4.4 now says what stands behind them.
- Ten worked examples: colouring (four regions, an odd ring of five, six regions, and four
  regions that all touch, which must fail), assignment at 3x3, 4x4 and 5x5, two in each row and
  column, four queens, and the assignment at temperature 0. Every lowest state solves each one
  marked as working where all arrangements are checked; seeds 1 to 100 at its own plan all
  solve it, and the two that must fail fail at all 100. The set states each one's reach.
- The vector rule: quantum-conformance/landscape/vectors.json is frozen as it stood on
  2026-11-10 and v2/vectors.json at publication, each BLAKE3 pinned (conformance2); the first
  version's section 9 states how its set changed on 11-06 and 11-09, and why.
- Members: Member::protocols (provided; the first version by default). Only the CPU reference
  runs /2; the WebGPU, CUDA, Metal and FPGA members are certified on /1 alone and passed over
  for a /2 job, saying why. The FPGA member's label names the frozen /1 set.
- Reach: the CLI (--protocol 2), the job service's /v1/landscape routes (stamped with the
  answer's version), Python (protocol=), npm and the desktop app carry /2; verify holds a /2
  answer to its form. The MCP tool keeps to /1 (its output schema is landscape.v1.json) and a
  client sends no /2 job to a provider, whose description offers /1.
- Registry: the two kernel labels; schemas/landscape.v2.json. Workflow pins updated.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-11-10 22:58:02 -05:00

13 KiB
Raw Permalink Blame History

Energy-landscape jobs: conformance and results

For the extension quantum-landscape. Reference: wai_quantum::quantum_landscape and wai-quantum landscape. Everything here is the output of a command named below, on the code at the commit that holds it. Every count is deterministic: the same command gives the same bytes on any machine. Wall-clock time is not, and it is kept apart.

file what made by
vectors.json the conformance set a member is held to, §7 of the extension; frozen as it stood on 2026-11-10 (below) wai-quantum landscape vectors
v2/vectors.json the second version's set, §6 of quantum-landscape-2; frozen wai-quantum landscape vectors --protocol 2
table.json, table.md the scale table, below wai-quantum landscape table --out DIR
timing.json that table's wall-clock seconds a run, on one machine the same command
trial-assignment.json a trial outside the job form, below wai-quantum landscape trial-assignment
native-wasm.txt the browser build against the command line wai-web/demo/verify_landscape_wasm.mjs, verify_landscape_page.mjs
stated-result.md, reproduce.sh a stated result, and the script that checks it

These are classical runs on a classical emulator, and nothing in them is a quantum effect.

Both sets are frozen

vectors.json is frozen as it stood on main on 2026-11-10 (sha256 1da13d2570eb5b1367d3017b465e1b6a199429210def0c4300dbc403413e3a8e), and v2/vectors.json at its publication. The code pins each one's BLAKE3 digest (conformance2::VECTORS_1_BLAKE3, VECTORS_2_BLAKE3), and tests/landscape2.rs fails on any difference: a change of any kind to either is a new version of the protocol. The first set changed its form and grew three times after it was first published, with no expected bit of an earlier vector changing; the first version's §9 lists each change and why. The FPGA member's grade names the set it was earned on.

The second version's worked examples

Ten programs, in wai-quantum/src/quantum_landscape/programs/2/, written with a cost for each variable and the constraints exactly and atmost. Each one's reach in v2/vectors.json states how seeds 1 to 100 end at the program's own plan; a test holds it to the code. Where every arrangement is checked, each one marked as working has every lowest state solving its problem.

example variables lowest, and how many lowest states solve it seeds 1 to 100
colour-four 12 −35: 6 of 6, the six colourings 100 solved
colour-five-ring 15 −40: 30 of 30 100 solved
colour-six 18 not enumerated; an answer that passes its check is a lowest state (quantum-landscape-2 §6) 100 solved
colour-four-all-touching 12 −36: 0 of 24, as it must (four regions that all touch need four colours) 100 failed
assign-3x3 9 −21: 6 of 6 100 solved
assign-4x4 16 −48: 24 of 24 100 solved
assign-5x5 25 not enumerated; as colour-six 100 solved
two-in-each-row-and-column 16 −32: 90 of 90 100 solved
four-queens 16 −84: 2 of 2 100 solved
assign-3x3-settled 9 −21: 6 of 6, at temperature 0 100 failed: never rests

These are a few small instances at one plan each, on the reference: they show that these landscapes express these problems and that this plan reached their lowest states. They say nothing of larger instances or of a device.

The worked examples

Fourteen programs, in wai-quantum/src/quantum_landscape/programs/. At seeds 1 to 20 each one marked as working is solved every time, and each one marked as failing fails every time, for its stated reason (job::tests).

example variables what it shows result
talks 12 three states: room A, room B or left out lowest energy −10, reached at all 1,000 seeds tried
clusters 8 signed clustering with a quarrel no split can settle lowest energy −1, reached
selection 14 six of fourteen that belong together lowest energy −9, and the planted six
chain 8 the deterministic mode with one end held settles to the one lowest arrangement
assign-2x2 4 assignment written as sums, 2×2 both lowest states are assignments
blocks 64 eight blocks of eight: too many arrangements to check the planted pattern, with 200 sweeps a start
assign-3x3 9 the same construction, 3×3 fails: 21 lowest states, only 6 of them assignments
colour-six 6 three colours, six regions fails: none of the 6 lowest states is a colouring
colour-four 4 three colours, four regions fails: neither of the 2 lowest states is a colouring
piles 8 two equal piles fails: quantising to signs loses the sizes, and the answer has energy −7 on the weights as declared, against −130
cold-constraint 4 2×2 assignment at temperature 0 fails: never rests, flipping between two states
cold-start 12 the talks at temperature 0, from all 0 misses: stays at 0, energy 0 against −10
too-costly 14 the six of fourteen at too high a cost fails: leaving everything out is the one lowest state
typo 3 a misspelt statement refused: Line 6: opposis not a statement epuML knows. Did you meanoppose?

Where the truth exists and a program has its own check, every lowest arrangement is graded against it. That is how assign-3x3 fails even when a run's answer happens to be an assignment: the landscape does not express the problem.

Native and WebAssembly

wai-web/demo/verify_landscape_wasm.mjs sends the same request to wai-quantum landscape call and to the browser build, and compares the bytes: every example at its own temperature, at 0, at 0.5 and at a temperature given as text, a replay of 2,000 seeds each, whole answers for 11 seeds, a sweep and a comparison. 700 answers the same bytes, 0 different, 104,000 seeded runs. In Chrome, verify_landscape_page.mjs repeats the replays in the browser's own engine: 84 of 84 answers the same bytes, 19,500 seeded runs (native-wasm.txt). The command line gives the same bytes on x86-64 Linux, arm64 Linux and arm64 macOS (stated-result.md).

Two pinned digests hold this in the repository's checks. job::tests::every_platform_gives_the_same_bits covers every worked example at three temperatures over 20 seeds, on Linux x86-64 and arm64, macOS arm64 and wasm32-wasip2 under wasmtime, in debug and release builds. landscape_jobs_are_the_native_bits_in_wasm runs the browser build's own, over 3 seeds, in wai-web-wasm.yml.

The scale table

Four families, 8 to 128 variables (dense to 64), 5 instances each, 20 runs an instance and method, at 8 starts of 40 sweeps and of 320. Every method in a row has the budget of the sampling mode's run, where one operation is one coupling read, one cell updated or one move considered, whoever does it. The target is the exact lowest energy up to 16 variables, then a planted answer, or the best that long runs of simulated annealing and tabu search found, which proves nothing.

Who reached the target more often, row by row, against the best of the three classical methods:

the sampling mode ahead tied behind of
at the default start 2 45 21 68
at the hot start, the largest row of couplings 0 38 30 68

Simulated annealing is the strongest method here: it is the best of the three, or level with the best, in 64 of the 68 rows, and it is ahead of the default start in 20 where the default start is ahead of it in 2. The two rows where the sampling mode is ahead of all three are ahead by one run of a hundred (clusters at 64 variables and 320 sweeps, sparse-glass at 32 and 40), which chance gives. Simulated annealing leads where it matters, at the larger sizes. At the default budget, at the largest size of each family:

family variables default start hot start deterministic simulated annealing tabu search greedy descent
clusters 128 46 of 100 0 0 79 7 0
sparse-glass 128 36 0 0 71 5 0
dense-glass 64 81 4 0 97 54 91
selection 128 100 100 100 100 100 100

The deterministic mode, at temperature 0 from random starts, is far behind on all but the selection family, which is easy for everyone.

timing.json has wall-clock seconds a run on one shared machine. They depend on how each method here is written, and say nothing about a device, so they are not a result.

What the table does not say. The operation count says nothing about what an operation costs in hardware, or about running cells side by side, which a machine would do and these methods do not.

The starting temperature

A solve with no t0 starts at the square root of the mean couplings a variable has, and at least 1. On instance 99 of three families at 32 and 64 variables, which the table's rows do not use, 40 runs at each start:

held-out instance lowest found default start: runs at the lowest hot start: runs at the lowest start 1: runs at the lowest
clusters, 32 −16 2: 40 of 40 9: 28 40
clusters, 64 −26 2: 28 8: 1 40
sparse-glass, 32 −19 2.449: 33 13: 15 38
sparse-glass, 64 −51 2.449: 40 12: 4 40
dense-glass, 32 −128 5.568: 39 31: 23 39
dense-glass, 64 −362 7.937: 38 63: 0 40

The hot start, the largest row of couplings, is far too hot at 64 variables, and worse at every size here. The default was chosen with this section in view, and it is not the best start: a start of 1 does at least as well on all six, better on three of them (by 2 to 12 runs of 40), and on clusters at 64 by a wide margin. It stays where it is because it depends on the instance, and a fixed start of 1 would be too cold for a landscape of larger fields. No instance here shows that; it is a judgement. wai-quantum landscape sweep tries the temperatures on a program of your own.

Assignment with a {0, +1} domain

wai-quantum landscape trial-assignment, outside the job form: the same assignment landscape, with each cell limited to 0 or +1. 40 runs a size, 8 starts of 60 sweeps.

grid lowest states (assignments among them) three states {0, +1}
2×2 2 (2) 40 of 40 40 of 40
3×3 21 (6) 15 40
4×4 1,344 (24) 1 40
5×5 too many to check 0 40
6×6 0 40
8×8 0 40

So a {0, +1} domain fixes feasibility: the lowest states become exactly the assignments. It does not fix costs. With costs of 0 or 1 a cell, on 80 random instances, it found the cheapest assignment every time the cheapest cost 0, and never found a valid assignment when the cheapest cost more: the couplings are fixed at 1, so the constraint's weight is 1, and a cost of 1 ties with leaving a row empty. Costed assignment would need couplings above 1 or fractional fields, and this model has neither. The domain itself is a change to the cell, so it is a trial and not a kernel.

The joule line

On the twelve talks: 13,440 accumulates × about 0.41 pJ = about 5.5 nJ, priced. The same run's 320 sweeps need at least 320 clocks on that board, which draws about 3.1 W whatever it does: about 9.9 µJ of board power the priced line leaves out. A run's joules are not measured here, because no meter is attached to a run here.

A member's conformance

A member is held to vectors.json as §7 of the extension says, and the host finds its parity by running its kernels there: settle bit for bit for a bit-exact settle kernel, every vector it takes, and it takes every vector whose field bound is at most 2³¹ − 1; chains bit for bit for a bit-exact sampling kernel, the same way; otherwise the statistical battery, thirteen workloads of 20,000 chains each, each block's final states against its exact distribution. wai-quantum/tests/landscape_conformance.rs reads the file as a member's checker would, from the file alone, and holds both parts to kernels that are wrong in one way each. The vectors refuse a dropped carry in a 32 by 32 product, a cut point compared by its high word, fields held in 16 or 32 bits, a check of the fields that forgets the couplings, cells last to first, a draw counter that starts again at each of a device's windows of sweeps, a run that stops after one window, chains past the first 64 left alone, a settle whose cells past 64 or 256 never move or that reports its end a sweep late, and a member that declines below the line. The battery passes the reference's own kernel, one that colours its cells properly and one that takes its cells last to first, and refuses kernels that move cells all at once, even then odd, ignore the schedule or keep only its end, clamp the table at 5, sample the last sweep, hold their fields in 16 or 32 bits, or leave cells from the fifth on where they started.