wai-quantum: an optimisation stops and goes on to the same bytes #597

Closed
dcharlot wants to merge 1 commit from wai/quantum-optimize-resume into main
11 changed files with 599 additions and 54 deletions

View file

@ -108,6 +108,8 @@ jobs:
engine::variational::tests::vqe_finds_the_ground_energy_diagonalisation_gives \
engine::variational::tests::qaoa_is_max_cut_and_finds_an_optimal_cut \
engine::variational::tests::the_answers_are_the_same_bytes_on_every_platform \
engine::variational::tests::an_optimisation_stopped_anywhere_and_gone_on_with_is_the_same_bytes \
engine::variational::tests::a_checkpoint_goes_on_only_from_its_own_run \
engine::algorithms::tests::phase_estimation_reads_the_phase_with_the_fejer_distribution \
engine::algorithms::tests::amplitude_estimation_reads_the_amplitude \
engine::algorithms::tests::the_adder_is_its_truth_table \

View file

@ -180,6 +180,22 @@ and no `depends_on :macos`. It now has both, so the tap carries it as the releas
invoice of version 6, since version 5 is now defined, and is refused `unknown_version` as before; every other vector
keeps its outcome, and the new vectors are added.
### An optimisation stops and goes on to the same bytes
A long optimisation ran to its end in one process or not at all, and one killed partway lost every step.
- **wai-quantum:** `engine::variational::optimize_resumable` (and `call_optimize`, as `call` answers) goes on from a
checkpoint, `wai.quantum.optimize-checkpoint/1`.
- After every step, the caller gets the run as it stands, and may stop it there.
- The checkpoint holds the steps taken with their links and the optimiser's whole state, every number as the bits
of its f64.
- Gone on with from any step, a run ends with the same bytes as one that never stopped, for each optimiser.
- A checkpoint of another request, or one whose steps were changed, is refused (`J052`).
- **wai-quantum-cli:** `optimize --checkpoint FILE` keeps the state in FILE as the run goes, at most once a second and
whenever it stops, written whole or not at all. It goes on from FILE when FILE is there, and removes FILE when the
run ends. A run killed and started again with the same arguments ends with the same bytes, receipt and all.
`--slice N` takes at most N steps, then stops with the state kept (exit 2).
### The receipt ledger bills what the work spends: bytes out, a package's work per run and its storage, every hop's energy, the output its hop delivered
- **A job's output is the object its hop delivered** ([extensions/receipt-ledger.md](extensions/receipt-ledger.md)

View file

@ -319,7 +319,7 @@ wai-quantum build qpe --counting 5 --phase 0.3 > qpe.qasm # checked a
wai-quantum run qpe.qasm --shots 1000
```
`--param NAME=VALUE` binds a program's parameter on any job (`run`, `plan`, `submit`, and fixes one in a sweep). `gradient` gives a Hamiltonian's expectation (`--term`, repeated, or `--hamiltonian FILE`) and its exact gradient by each parameter (`--gradient adjoint` or `parameter_shift`), each component with its rounding bound. `optimize` takes `--optimizer gradient_descent`, `adam` (the default) or `spsa` (`--seed`), `--steps` and `--step-size`, every step in the digest chain the receipt seals. `vqe` takes a program or `--layers L` for the built-in ansatz, and sets the energy beside the exact ground energy up to 8 qubits. `qaoa --graph FILE` solves Max-Cut and sets the cut beside the maximum. `build qpe | amplitude_estimation | adder | qsvt` writes the program to standard output and its check against the reference to standard error. Every answer is signed with your key (`--json` prints it sealed), and `verify` checks it and runs it again to the same bytes. Everything each computes is on [the variational page](../wai-quantum/docs/variational.md).
`--param NAME=VALUE` binds a program's parameter on any job (`run`, `plan`, `submit`, and fixes one in a sweep). `gradient` gives a Hamiltonian's expectation (`--term`, repeated, or `--hamiltonian FILE`) and its exact gradient by each parameter (`--gradient adjoint` or `parameter_shift`), each component with its rounding bound. `optimize` takes `--optimizer gradient_descent`, `adam` (the default) or `spsa` (`--seed`), `--steps` and `--step-size`, every step in the digest chain the receipt seals. With `--checkpoint FILE` it keeps its state in FILE as it goes, and a run killed and started again ends with the same bytes as one that never stopped; `--slice N` takes N steps at a time (exit 2 while steps remain). `vqe` takes a program or `--layers L` for the built-in ansatz, and sets the energy beside the exact ground energy up to 8 qubits. `qaoa --graph FILE` solves Max-Cut and sets the cut beside the maximum. `build qpe | amplitude_estimation | adder | qsvt` writes the program to standard output and its check against the reference to standard error. Every answer is signed with your key (`--json` prints it sealed), and `verify` checks it and runs it again to the same bytes. Everything each computes is on [the variational page](../wai-quantum/docs/variational.md).
## Compiling

View file

@ -1605,7 +1605,8 @@ Commands:
each (adjoint differentiation or the parameter-shift rule, in f64 with a stated
rounding bound), signed
optimize Minimise a Hamiltonian's expectation over a program's parameters: gradient descent,
Adam or SPSA from a seed, every step in a digest chain the receipt seals
Adam or SPSA from a seed, every step in a digest chain the receipt seals; with
--checkpoint it stops and goes on to the same bytes (exit 2 when --slice stops it)
vqe The variational eigensolver: an ansatz optimised on a Hamiltonian, the energy found
set beside the exact ground energy by diagonalisation
qaoa Max-Cut on a graph by QAOA: the program and its cost built from the graph, optimised,
@ -2619,7 +2620,8 @@ Options:
```text
Minimise a Hamiltonian's expectation over a program's parameters: gradient descent, Adam or SPSA
from a seed, every step in a digest chain the receipt seals
from a seed, every step in a digest chain the receipt seals; with --checkpoint it stops and goes on
to the same bytes (exit 2 when --slice stops it)
Usage: wai-quantum optimize [OPTIONS] [FILE]
@ -2668,6 +2670,16 @@ Options:
--seed <SEED>
SPSA's seed (default 1)
--checkpoint <FILE>
Keep the run's state in FILE as it goes (at most once a second, and when it stops), and go
on from FILE when it is there: a run stopped, killed or by --slice, and started again with
the same arguments ends with the same bytes as a run that never stopped. FILE is removed
when the run ends
--slice <N>
With --checkpoint: take at most N steps now, then stop with the state kept (exit 2); the
same command again goes on
--json
Print the answer sealed with this user's key, as JSON (`wai-quantum verify` checks it)

View file

@ -49,7 +49,7 @@
"compile": { "source": "Compile a program: passes each checked where they act, Clifford+T within an ε budget (or the standard gates), and a signed certificate of every pass and the distance (`wai-quantum compile --help`)", "text": "编译程序:每一遍处理(pass)都在其作用之处经过检查,在 ε 预算内得到 Clifford+T(或标准门),并为每一遍处理和距离出具签名证书(`wai-quantum compile --help`)" },
"estimate": { "source": "What a fault-tolerant machine would need to run a program: its logical counts (T, Toffolis, rotations at their synthesised T counts, depth, qubits with ancillas), physical qubits, runtime and factories, and the frontier between qubits and time, each estimate signed (`wai-quantum estimate --help`)", "text": "容错机器运行某个程序需要什么:逻辑计数(T 门、Toffoli 门、按综合后 T 门数计的旋转、深度、含辅助量子比特的量子比特数)、物理量子比特、运行时间和工厂,以及量子比特与时间之间的权衡前沿,每份估算都附签名(`wai-quantum estimate --help`)" },
"gradient": { "source": "The expectation of a Hamiltonian at a program's parameters and its exact gradient by each (adjoint differentiation or the parameter-shift rule, in f64 with a stated rounding bound), signed", "text": "哈密顿量在程序参数处的期望值及其对每个参数的精确梯度(伴随微分或参数平移规则,以 f64 计算并给出舍入误差界),附签名" },
"optimize": { "source": "Minimise a Hamiltonian's expectation over a program's parameters: gradient descent, Adam or SPSA from a seed, every step in a digest chain the receipt seals", "text": "在程序参数上最小化哈密顿量的期望值:梯度下降、Adam 或带种子的 SPSA,每一步都记入摘要链,由回执封存" },
"optimize": { "source": "Minimise a Hamiltonian's expectation over a program's parameters: gradient descent, Adam or SPSA from a seed, every step in a digest chain the receipt seals; with --checkpoint it stops and goes on to the same bytes (exit 2 when --slice stops it)", "text": "在程序参数上最小化哈密顿量的期望值:梯度下降、Adam 或带种子的 SPSA,每一步都记入摘要链,由回执封存;用 --checkpoint 时可中途停下、之后接着运行,得到相同的字节(--slice 使其停下时退出码为 2)" },
"vqe": { "source": "The variational eigensolver: an ansatz optimised on a Hamiltonian, the energy found set beside the exact ground energy by diagonalisation", "text": "变分本征求解器:在哈密顿量上优化拟设(ansatz),并把找到的能量与对角化得到的精确基态能量并列" },
"qaoa": { "source": "Max-Cut on a graph by QAOA: the program and its cost built from the graph, optimised, and the cut set beside the maximum cut by brute force", "text": "用 QAOA 求图的最大割:由图构造程序及其代价函数,优化后把所得的割与穷举得到的最大割并列" },
"build": { "source": "A standard algorithm's program (qpe, amplitude_estimation, adder, qsvt) on standard output, checked by the equivalence ladder against a reference built from its definition", "text": "把标准算法的程序(qpe、amplitude_estimation、adder、qsvt)写到标准输出,并由等价性阶梯对照按其定义构造的参考加以检查" },

View file

@ -264,7 +264,7 @@ enum Command {
Estimate(estimate::Args),
/// The expectation of a Hamiltonian at a program's parameters and its exact gradient by each (adjoint differentiation or the parameter-shift rule, in f64 with a stated rounding bound), signed
Gradient(variational::Gradient),
/// Minimise a Hamiltonian's expectation over a program's parameters: gradient descent, Adam or SPSA from a seed, every step in a digest chain the receipt seals
/// Minimise a Hamiltonian's expectation over a program's parameters: gradient descent, Adam or SPSA from a seed, every step in a digest chain the receipt seals; with --checkpoint it stops and goes on to the same bytes (exit 2 when --slice stops it)
Optimize(variational::Optimize),
/// The variational eigensolver: an ansatz optimised on a Hamiltonian, the energy found set beside the exact ground energy by diagonalisation
Vqe(variational::Vqe),

View file

@ -11,6 +11,8 @@
//! wai-quantum gradient ansatz.qasm --param theta=0.3 --term "Z0 Z1" --term "0.5 X0"
//! wai-quantum gradient ansatz.qasm --param theta=0.3 --term Z0 --gradient parameter_shift
//! wai-quantum optimize ansatz.qasm --param theta=0.3 --term Z0 --optimizer adam --steps 200
//! wai-quantum optimize ansatz.qasm --param theta=0.3 --term Z0 --steps 2000 --checkpoint run.json
//! wai-quantum optimize ansatz.qasm --param theta=0.3 --term Z0 --steps 2000 --checkpoint run.json --slice 100
//! wai-quantum vqe --term "X0 X1" --term "Y0 Y1" --term "Z0 Z1" --layers 3 --steps 400
//! wai-quantum qaoa --graph ring.json --layers 2
//! wai-quantum build qpe --counting 5 --phase 0.3 > qpe.qasm
@ -81,10 +83,69 @@ pub struct Optimize {
pub program: Program,
#[command(flatten)]
pub optimiser: Optimiser,
/// Keep the run's state in FILE as it goes (at most once a second, and when it stops), and go on from FILE when it is there: a run stopped, killed or by --slice, and started again with the same arguments ends with the same bytes as a run that never stopped. FILE is removed when the run ends
#[arg(long, value_name = "FILE")]
pub checkpoint: Option<PathBuf>,
/// With --checkpoint: take at most N steps now, then stop with the state kept (exit 2); the same command again goes on
#[arg(long, value_name = "N", requires = "checkpoint", value_parser = clap::value_parser!(u32).range(1..))]
pub slice: Option<u32>,
#[command(flatten)]
pub out: Out,
}
/// How often a running optimisation writes its checkpoint, at most.
const CHECKPOINT_EVERY: std::time::Duration = std::time::Duration::from_secs(1);
/// What an optimisation with a checkpoint did: ended, or stopped after a step.
enum Went {
Ended(Value),
Stopped { next: u32, steps: u32 },
}
/// `req` optimised with its state in `path`: gone on with from the
/// checkpoint there, if there is one, and kept there as it goes (at most once
/// a [`CHECKPOINT_EVERY`], and when it stops), `slice` steps at most. The
/// file is removed once the run ends.
fn optimize_kept(req: &Value, path: &std::path::Path, slice: Option<u32>) -> Result<Went, String> {
let from: Option<Value> = match std::fs::read(path) {
Ok(b) => Some(serde_json::from_slice(&b).map_err(|e| format!("{}: not a checkpoint: {e}", path.display()))?),
Err(e) if e.kind() == std::io::ErrorKind::NotFound => None,
Err(e) => return Err(format!("{}: {e}", path.display())),
};
if let Some(cp) = &from {
eprintln!("going on from step {} of {}, as {} keeps it", cp["next"], cp["steps"], path.display());
}
let (mut taken, mut wrote, mut stopped) = (0u32, std::time::Instant::now(), None);
let done = variational::call_optimize(req, from.as_ref(), |p| {
taken += 1;
let stop = slice.is_some_and(|n| taken >= n);
if stop || wrote.elapsed() >= CHECKPOINT_EVERY {
keep(path, &p.checkpoint())?;
wrote = std::time::Instant::now();
}
if stop {
stopped = Some((p.next(), p.steps()));
}
Ok(!stop)
})?;
match (done, stopped) {
(Some(r), _) => {
// The run is in its answer now; a checkpoint left would only go on to it again.
let _ = std::fs::remove_file(path);
Ok(Went::Ended(r))
}
(None, Some((next, steps))) => Ok(Went::Stopped { next, steps }),
(None, None) => Err("the optimisation stopped with no step to go on from".into()),
}
}
/// Writes `v` to `path` whole or not at all: a run killed while it writes
/// leaves the checkpoint it had.
fn keep(path: &std::path::Path, v: &Value) -> Result<(), String> {
let partial = std::path::PathBuf::from(format!("{}.partial", path.display()));
std::fs::write(&partial, v.to_string()).and_then(|()| std::fs::rename(&partial, path)).map_err(|e| format!("{}: {e}", path.display()))
}
#[derive(clap::Args, Clone)]
pub struct Vqe {
#[command(flatten)]
@ -337,7 +398,16 @@ fn out_of(cmd: &super::Command) -> &Out {
pub fn main(cmd: &super::Command) {
let r = (|| -> Result<(), String> {
let (op, req) = request(cmd)?;
let result = variational::call(op, &req)?;
let result = match cmd {
super::Command::Optimize(Optimize { checkpoint: Some(path), slice, .. }) => match optimize_kept(&req, path, *slice)? {
Went::Ended(r) => r,
Went::Stopped { next, steps } => {
eprintln!("stopped after step {} of {steps}; {} keeps the state: the same command again goes on", next - 1, path.display());
std::process::exit(2);
}
},
_ => variational::call(op, &req)?,
};
let out = out_of(cmd);
let k = super::load_key(&out.key);
let body = variational::seal(&req, result, &k.key);

View file

@ -0,0 +1,131 @@
//! `wai-quantum optimize --checkpoint`: a run stopped and started again ends
//! with the same bytes as one that never stopped, whether it stops by
//! `--slice` or is killed while it runs. Nothing here leaves this machine.
use std::path::{Path, PathBuf};
use std::process::{Command, Output, Stdio};
use std::time::{Duration, Instant};
fn bin() -> &'static str {
env!("CARGO_BIN_EXE_wai-quantum")
}
/// A scratch directory of the test's own, and the program it optimises:
/// `n` qubits, a layer of `ry` on each, a chain of `cx`, and another layer.
fn scratch(tag: &str, n: usize) -> (PathBuf, PathBuf) {
let dir = std::env::temp_dir().join(format!("wai-q-opt-{tag}-{}", std::process::id()));
let _ = std::fs::remove_dir_all(&dir);
std::fs::create_dir_all(&dir).unwrap();
let mut src = String::from("OPENQASM 3.0;\ninclude \"stdgates.inc\";\n");
for k in 0..2 * n {
src += &format!("input float t{k};\n");
}
src += &format!("qubit[{n}] q;\n");
for k in 0..n {
src += &format!("ry(t{k}) q[{k}];\n");
}
for k in 0..n - 1 {
src += &format!("cx q[{k}], q[{}];\n", k + 1);
}
for k in 0..n {
src += &format!("ry(t{}) q[{k}];\n", n + k);
}
let prog = dir.join("p.qasm");
std::fs::write(&prog, src).unwrap();
(dir, prog)
}
/// The command's arguments: every parameter from 0.1 up, the Hamiltonian
/// Z0 Z1 and X on the last qubit, Adam, `steps` steps, signed with the demo
/// key so that the same answer is the same bytes.
fn args(prog: &Path, n: usize, steps: u32) -> Vec<String> {
let mut a = vec!["optimize".to_string(), prog.display().to_string()];
for k in 0..2 * n {
a.push("--param".into());
a.push(format!("t{k}=0.{}", k % 9 + 1));
}
for t in ["Z0 Z1".to_string(), format!("0.5 X{}", n - 1)] {
a.push("--term".into());
a.push(t);
}
for x in ["--optimizer", "adam", "--steps", &steps.to_string(), "--json", "--demo-key"] {
a.push(x.into());
}
a
}
fn run(a: &[String], extra: &[&str]) -> Output {
Command::new(bin()).args(a).args(extra).env_remove("WAI_QUANTUM_KEY").output().unwrap()
}
#[test]
fn a_run_taken_in_slices_ends_with_the_bytes_of_one_that_never_stopped() {
let (dir, prog) = scratch("slices", 3);
let a = args(&prog, 3, 40);
let whole = run(&a, &[]);
assert!(whole.status.success(), "{}", String::from_utf8_lossy(&whole.stderr));
let cp = dir.join("run.json");
let cps = cp.display().to_string();
let mut slices = 0;
let last = loop {
slices += 1;
let o = run(&a, &["--checkpoint", &cps, "--slice", "7"]);
match o.status.code() {
Some(2) => {
let err = String::from_utf8_lossy(&o.stderr);
assert!(err.contains("the same command again goes on") && o.stdout.is_empty(), "{err}");
assert!(cp.exists(), "a stopped run keeps its state");
}
Some(0) => break o,
other => panic!("{other:?}: {}", String::from_utf8_lossy(&o.stderr)),
}
assert!(slices < 20, "the run never ended");
};
// 41 steps, seven a slice: five slices stop, the sixth ends.
assert_eq!(slices, 6);
assert_eq!(last.stdout, whole.stdout, "the same bytes as a run that never stopped");
assert!(!cp.exists(), "the checkpoint is removed once the run ends");
// A checkpoint of another request is refused, and left as it is.
let o = run(&a, &["--checkpoint", &cps, "--slice", "3"]);
assert_eq!(o.status.code(), Some(2));
let other = args(&prog, 3, 41);
let o = run(&other, &["--checkpoint", &cps]);
assert_eq!(o.status.code(), Some(1));
assert!(String::from_utf8_lossy(&o.stderr).contains("it is of another request"), "{}", String::from_utf8_lossy(&o.stderr));
assert!(cp.exists());
let _ = std::fs::remove_dir_all(&dir);
}
#[test]
fn a_run_killed_while_it_runs_goes_on_to_the_bytes_of_one_that_never_stopped() {
// Big enough that a run outlasts its first checkpoint by seconds.
let (n, steps) = (16, 600);
let (dir, prog) = scratch("killed", n);
let a = args(&prog, n, steps);
let cp = dir.join("run.json");
let cps = cp.display().to_string();
let mut child = Command::new(bin()).args(&a).args(["--checkpoint", &cps]).env_remove("WAI_QUANTUM_KEY").stdout(Stdio::null()).stderr(Stdio::null()).spawn().unwrap();
let t = Instant::now();
// Killed once it has kept a state some steps in, as a crash or a power cut would stop it.
let at = loop {
if let Ok(text) = std::fs::read_to_string(&cp)
&& let Ok(v) = serde_json::from_str::<serde_json::Value>(&text)
&& v["next"].as_u64().is_some_and(|k| k >= 2)
{
break v["next"].as_u64().unwrap();
}
assert!(child.try_wait().unwrap().is_none(), "the run ended before it was killed: make it longer");
assert!(t.elapsed() < Duration::from_secs(120), "no checkpoint was kept");
std::thread::sleep(Duration::from_millis(20));
};
child.kill().unwrap();
child.wait().unwrap();
assert!(at < u64::from(steps), "killed partway");
let resumed = run(&a, &["--checkpoint", &cps]);
assert!(resumed.status.success(), "{}", String::from_utf8_lossy(&resumed.stderr));
assert!(String::from_utf8_lossy(&resumed.stderr).contains("going on from step"), "{}", String::from_utf8_lossy(&resumed.stderr));
let whole = run(&a, &[]);
assert!(whole.status.success());
assert_eq!(resumed.stdout, whole.stdout, "the same bytes as a run that never stopped");
let _ = std::fs::remove_dir_all(&dir);
}

View file

@ -116,6 +116,31 @@ wai-quantum optimize ansatz.qasm --param t=0.4 --term Z1 --optimizer spsa --step
wai-quantum verify opt.json
```
An optimisation can stop between any two steps and go on later, to the same
bytes. Its checkpoint (`wai.quantum.optimize-checkpoint/1`) holds:
- the request it is of, by the chain's `start` and the request's digest;
- the steps taken, each with its `digest`, and the last link;
- the optimiser's whole state: the point, Adam's moments and the powers of its
betas, SPSA's stream, the best point and the counts. Every number is the
hex of its f64's bits, so it reads back exactly.
A checkpoint goes on only from its own request, and only if each step is still
the link its digest says. Any other is refused (`J052`).
On the command line, `--checkpoint FILE` keeps the state in FILE as the run goes,
at most once a second and whenever it stops, and goes on from FILE when it is
there. A run killed and started again with the same arguments loses at most a
second of steps, and ends with the same bytes as a run that never stopped. The
file is removed when the run ends. `--slice N` takes at most N steps, then stops
with the state kept (exit 2), so a long optimisation can be run a slice at a
time.
```sh
wai-quantum optimize ansatz.qasm --param t=0.4 --term Z1 --steps 2000 --checkpoint run.json
wai-quantum optimize ansatz.qasm --param t=0.4 --term Z1 --steps 2000 --checkpoint run.json --slice 100
```
## The variational eigensolver
`vqe` optimises an ansatz on a Hamiltonian: the request's program with its

View file

@ -49,6 +49,14 @@
//! from a seeded stream. The same request gives the same bytes everywhere,
//! and a result carries no clock.
//!
//! **Stopping and going on.** An optimisation can stop between any two steps
//! and go on later ([`optimize_resumable`]): its checkpoint holds the steps
//! taken, each with its link, and the optimiser's whole state (the point,
//! Adam's moments and the powers of its betas, SPSA's stream, the best point
//! and the counts), every number as the bits of its f64. Gone on with, it ends
//! with the same bytes as a run that never stopped. A checkpoint goes on only
//! from its own request, and only with its steps as they were.
//!
//! Refusals end with their codes: `J050` for `params`, `J051` for a gradient
//! (with `Q129` where the reader cannot follow a parameter), `J052` for an
//! optimisation, a VQE or a QAOA run.
@ -856,9 +864,171 @@ pub fn optimize(request: &Value) -> Result<Value, String> {
optimize_from(request, request)
}
/// [`optimize`], stopped and gone on with: from `from`, a checkpoint an
/// earlier run of the same request gave, or from the start. After every
/// step but the last, `each` is given the run as it stands ([`Paused`]):
/// `Ok(true)` goes on, `Ok(false)` stops it there (`Ok(None)`), and an error
/// ends it. Gone on with from any of its checkpoints, a run ends with the
/// same bytes as one that never stopped, since the checkpoint holds the
/// optimiser's whole state as bits.
pub fn optimize_resumable(request: &Value, from: Option<&Value>, each: impl FnMut(&Paused) -> Result<bool, String>) -> Result<Option<Value>, String> {
resume_from(request, request, from, each)
}
/// [`optimize`] on `request`, its chain starting from `origin`: the request
/// as given, where a VQE or a QAOA run derives the optimisation's own from it.
fn optimize_from(request: &Value, origin: &Value) -> Result<Value, String> {
Ok(resume_from(request, origin, None, |_| Ok(true))?.expect("a run nothing stops runs to its end"))
}
/// The format of an optimisation's checkpoint ([`Paused::checkpoint`]).
pub const CHECKPOINT: &str = "wai.quantum.optimize-checkpoint/1";
/// An optimisation between two steps: all its next step needs.
struct State {
/// The step it goes on from.
next: u32,
theta: Vec<f64>,
/// Adam's moment estimates and the powers of its betas.
m: Vec<f64>,
w: Vec<f64>,
b1t: f64,
b2t: f64,
/// SPSA's stream.
rng: u64,
/// The steps taken, each with its link.
trace: Vec<Value>,
best: Option<(f64, Vec<f64>, u32)>,
evaluations: u64,
bound_max: f64,
chain: [u8; 32],
}
/// An optimisation stopped between two steps, as [`optimize_resumable`]
/// gives it to its caller.
pub struct Paused<'a> {
state: &'a State,
request: &'a Value,
start: &'a [u8; 32],
steps: u32,
}
fn bits(x: f64) -> Value {
json!(format!("{:016x}", x.to_bits()))
}
fn bits_of(xs: &[f64]) -> Value {
Value::Array(xs.iter().map(|x| bits(*x)).collect())
}
fn unbits(v: &Value) -> Option<f64> {
v.as_str().filter(|t| t.len() == 16).and_then(|t| u64::from_str_radix(t, 16).ok()).map(f64::from_bits)
}
fn unbits_of(v: &Value, n: usize) -> Option<Vec<f64>> {
v.as_array().filter(|a| a.len() == n)?.iter().map(unbits).collect()
}
/// BLAKE3 of a request's canonical text, in hex.
fn request_digest(request: &Value) -> String {
hex(blake3::hash(&receipt::canonical(request)).as_bytes())
}
impl Paused<'_> {
/// The step the run goes on from: the steps taken so far.
pub fn next(&self) -> u32 {
self.state.next
}
/// The steps the run takes after its first point.
pub fn steps(&self) -> u32 {
self.steps
}
/// The run as it stands, to go on from ([`optimize_resumable`]): the
/// request it is of, by its chain's first link and its digest; the steps
/// taken, each with its link; and the optimiser's state, every number as
/// the bits of its f64, so that it reads back exactly.
pub fn checkpoint(&self) -> Value {
let st = self.state;
json!({
"format": CHECKPOINT,
"start": hex(self.start),
"request": request_digest(self.request),
"next": st.next,
"steps": self.steps,
"trace": st.trace,
"chain": hex(&st.chain),
"state": {
"theta": bits_of(&st.theta),
"m": bits_of(&st.m),
"w": bits_of(&st.w),
"b1t": bits(st.b1t),
"b2t": bits(st.b2t),
"rng": format!("{:016x}", st.rng),
"evaluations": st.evaluations,
"bound_max": bits(st.bound_max),
"best": st.best.as_ref().map(|(v, p, s)| json!({ "value": bits(*v), "params": bits_of(p), "step": s })),
},
})
}
}
/// The state `cp` holds, if it is a checkpoint of this request (its first
/// link `start`, `k` parameters, `steps` steps) whose steps hold their chain.
fn restore(cp: &Value, request: &Value, start: &[u8; 32], k: usize, steps: u32) -> Result<State, String> {
let bad = |why: &str| oerr(format!("the checkpoint does not go on from here: {why}"));
if cp["format"] != CHECKPOINT {
return Err(bad(&format!("it is not {CHECKPOINT}")));
}
if cp["start"].as_str() != Some(hex(start).as_str()) || cp["request"].as_str() != Some(request_digest(request).as_str()) || cp["steps"].as_u64() != Some(u64::from(steps)) {
return Err(bad("it is of another request"));
}
let next = cp["next"].as_u64().filter(|n| (1..=u64::from(steps)).contains(n)).ok_or_else(|| bad("its next step is not one of the request's"))? as u32;
let trace = cp["trace"].as_array().filter(|t| t.len() == next as usize).ok_or_else(|| bad("its steps are not the steps it says it took"))?.clone();
// Each step's link, from the request: a checkpoint changed in any step is refused.
let mut chain = *start;
for (i, rec) in trace.iter().enumerate() {
let mut bare = rec.clone();
let d = bare.as_object_mut().and_then(|o| o.remove("digest"));
chain = link(&chain, &bare);
if rec["step"].as_u64() != Some(i as u64) || d.as_ref().and_then(Value::as_str) != Some(hex(&chain).as_str()) {
return Err(bad(&format!("step {i} is not the link its digest says")));
}
}
if cp["chain"].as_str() != Some(hex(&chain).as_str()) {
return Err(bad("its chain is not its steps' last link"));
}
let o = &cp["state"];
let num = |key: &str| unbits(&o[key]).ok_or_else(|| bad(&format!("its {key} does not read")));
let vec = |key: &str| unbits_of(&o[key], k).ok_or_else(|| bad(&format!("its {key} does not read as {k} numbers")));
let best = match &o["best"] {
Value::Null => None,
b => Some((unbits(&b["value"]).ok_or_else(|| bad("its best value does not read"))?, unbits_of(&b["params"], k).ok_or_else(|| bad("its best point does not read"))?, b["step"].as_u64().filter(|s| *s < u64::from(next)).ok_or_else(|| bad("its best step is not one it took"))? as u32)),
};
let theta = vec("theta")?;
if theta.iter().any(|x| !x.is_finite() || x.abs() > 1e6) {
return Err(bad("its parameters are outside the range a program reads"));
}
Ok(State {
next,
theta,
m: vec("m")?,
w: vec("w")?,
b1t: num("b1t")?,
b2t: num("b2t")?,
rng: o["rng"].as_str().filter(|t| t.len() == 16).and_then(|t| u64::from_str_radix(t, 16).ok()).ok_or_else(|| bad("its rng does not read"))?,
trace,
best,
evaluations: o["evaluations"].as_u64().ok_or_else(|| bad("its evaluations do not read"))?,
bound_max: num("bound_max")?,
chain,
})
}
/// [`optimize_from`], from `from` or the start, `each` given the run after
/// every step but the last ([`optimize_resumable`]).
fn resume_from(request: &Value, origin: &Value, from: Option<&Value>, mut each: impl FnMut(&Paused) -> Result<bool, String>) -> Result<Option<Value>, String> {
let s = Settings::read(request)?;
let (v, p, point) = program_of(request, refusal::OPTIMIZE)?;
let h = hamiltonian(request.get("hamiltonian"), v.qubits, refusal::OPTIMIZE)?;
@ -871,60 +1041,56 @@ fn optimize_from(request: &Value, origin: &Value) -> Result<Value, String> {
return Err(oerr(format!("{} steps would apply about {work:.1e} gate amplitudes, more than one optimisation takes ({MAX_WORK:.0e}): take fewer steps", s.steps)));
}
let start = trace_start(origin);
let mut chain = start;
let mut theta = point;
let k = theta.len();
let (mut m, mut w) = (vec![0.0; k], vec![0.0; k]);
let (mut b1t, mut b2t) = (1.0f64, 1.0f64);
let mut rng = s.seed;
let mut steps = Vec::with_capacity(s.steps as usize + 1);
let mut best: Option<(f64, Vec<f64>, u32)> = None;
let mut evaluations: u64 = 0;
let mut bound_max = 0.0f64;
for step in 0..=s.steps {
let k = point.len();
let mut st = match from {
Some(cp) => restore(cp, request, &start, k, s.steps)?,
None => State { next: 0, theta: point, m: vec![0.0; k], w: vec![0.0; k], b1t: 1.0, b2t: 1.0, rng: s.seed, trace: Vec::with_capacity(s.steps as usize + 1), best: None, evaluations: 0, bound_max: 0.0, chain: start },
};
while st.next <= s.steps {
let step = st.next;
let last = step == s.steps;
let here = theta.clone();
let here = st.theta.clone();
let mut rec = json!({ "step": step, "params": v.named(&here) });
let value = if s.optimizer == "spsa" {
let e = energy_at(&v, &theta, &h)?;
evaluations += 1;
let e = energy_at(&v, &st.theta, &h)?;
st.evaluations += 1;
if !last {
let kk = f64::from(step);
let ak = s.step_size / pow(kk + 1.0 + s.big_a, s.alpha);
let ck = s.c / pow(kk + 1.0, s.gamma);
let delta: Vec<f64> = (0..k).map(|_| if splitmix64(&mut rng) >> 63 == 0 { 1.0 } else { -1.0 }).collect();
let plus: Vec<f64> = theta.iter().zip(&delta).map(|(t, d)| t + ck * d).collect();
let minus: Vec<f64> = theta.iter().zip(&delta).map(|(t, d)| t - ck * d).collect();
let delta: Vec<f64> = (0..k).map(|_| if splitmix64(&mut st.rng) >> 63 == 0 { 1.0 } else { -1.0 }).collect();
let plus: Vec<f64> = st.theta.iter().zip(&delta).map(|(t, d)| t + ck * d).collect();
let minus: Vec<f64> = st.theta.iter().zip(&delta).map(|(t, d)| t - ck * d).collect();
let diff = (energy_at(&v, &plus, &h)? - energy_at(&v, &minus, &h)?) / (2.0 * ck);
evaluations += 2;
st.evaluations += 2;
let est: Vec<f64> = delta.iter().map(|d| diff * d).collect();
rec["perturbation"] = json!(delta);
rec["estimate"] = json!(est);
rec["gains"] = json!({ "a": ak, "c": ck });
for (t, e) in theta.iter_mut().zip(&est) {
for (t, e) in st.theta.iter_mut().zip(&est) {
*t -= ak * e;
}
}
e
} else {
let pt = v.at(&theta, true)?;
let pt = v.at(&st.theta, true)?;
let g = gradient_at(&v, &pt, &h, &s.gradient)?;
evaluations += g.evaluations;
bound_max = bound_max.max(g.value_bound);
st.evaluations += g.evaluations;
st.bound_max = st.bound_max.max(g.value_bound);
if !last {
rec["gradient"] = v.named(&g.gradient);
if s.optimizer == "adam" {
b1t *= s.beta1;
b2t *= s.beta2;
st.b1t *= s.beta1;
st.b2t *= s.beta2;
for j in 0..k {
m[j] = s.beta1 * m[j] + (1.0 - s.beta1) * g.gradient[j];
w[j] = s.beta2 * w[j] + (1.0 - s.beta2) * g.gradient[j] * g.gradient[j];
let mh = m[j] / (1.0 - b1t);
let wh = w[j] / (1.0 - b2t);
theta[j] -= s.step_size * mh / (wh.sqrt() + s.epsilon);
st.m[j] = s.beta1 * st.m[j] + (1.0 - s.beta1) * g.gradient[j];
st.w[j] = s.beta2 * st.w[j] + (1.0 - s.beta2) * g.gradient[j] * g.gradient[j];
let mh = st.m[j] / (1.0 - st.b1t);
let wh = st.w[j] / (1.0 - st.b2t);
st.theta[j] -= s.step_size * mh / (wh.sqrt() + s.epsilon);
}
} else {
for (t, d) in theta.iter_mut().zip(&g.gradient) {
for (t, d) in st.theta.iter_mut().zip(&g.gradient) {
*t -= s.step_size * d;
}
}
@ -932,19 +1098,23 @@ fn optimize_from(request: &Value, origin: &Value) -> Result<Value, String> {
g.value
};
rec["value"] = json!(value);
if best.as_ref().is_none_or(|b| value < b.0) {
best = Some((value, here, step));
if st.best.as_ref().is_none_or(|b| value < b.0) {
st.best = Some((value, here, step));
}
if theta.iter().any(|x| !x.is_finite() || x.abs() > 1e6) {
if st.theta.iter().any(|x| !x.is_finite() || x.abs() > 1e6) {
return Err(oerr(format!("step {step}: the parameters left the range a program reads (|θ| ≤ 10^6); take a smaller step_size")));
}
chain = link(&chain, &rec);
rec["digest"] = json!(hex(&chain));
steps.push(rec);
st.chain = link(&st.chain, &rec);
rec["digest"] = json!(hex(&st.chain));
st.trace.push(rec);
st.next += 1;
if st.next <= s.steps && !each(&Paused { state: &st, request, start: &start, steps: s.steps })? {
return Ok(None);
}
}
let (bv, bp, bs) = best.expect("at least one step");
let (bv, bp, bs) = st.best.expect("at least one step");
let best_params = v.named(&bp);
let fin = steps.last().cloned().unwrap_or(Value::Null);
let fin = st.trace.last().cloned().unwrap_or(Value::Null);
let mut out = json!({
"optimizer": s.json(),
"qubits": v.qubits,
@ -952,17 +1122,17 @@ fn optimize_from(request: &Value, origin: &Value) -> Result<Value, String> {
"parameters": v.names.iter().map(|(n, t)| json!({ "name": n, "type": t })).collect::<Vec<_>>(),
"terms": h.len(),
"start": hex(&start),
"steps": steps,
"chain": hex(&chain),
"steps": st.trace,
"chain": hex(&st.chain),
"best": { "step": bs, "params": best_params, "value": bv },
"final": { "step": fin["step"], "params": fin["params"], "value": fin["value"] },
"evaluations": evaluations,
"precision": precision(if s.optimizer == "spsa" { value_rounding(v.qubits, &h, p.program.ops.iter().map(|o| gate_rounding(o.qubits.len())).sum()) } else { bound_max }, None),
"evaluations": st.evaluations,
"precision": precision(if s.optimizer == "spsa" { value_rounding(v.qubits, &h, p.program.ops.iter().map(|o| gate_rounding(o.qubits.len())).sum()) } else { st.bound_max }, None),
});
if s.optimizer == "spsa" {
out["note"] = json!("SPSA's estimates are of the gradient from two evaluations each; its values are exact for the circuit at each point, its path depends on the seed");
}
Ok(out)
Ok(Some(out))
}
/// Whether a trace's chain holds: its first link from the request, each
@ -1331,10 +1501,8 @@ pub fn qaoa(request: &Value) -> Result<Value, String> {
/// `qaoa`, and `build` (the algorithm builders, [`super::algorithms`]). The
/// answer names the engine and its schema (`wai.quantum.algorithms/1`).
pub fn call(op: &str, request: &Value) -> Result<Value, String> {
if let Some(at) = receipt::inexact_number(request) {
return Err(tag(refusal::LIMIT, format!("`{at}`: a whole number here must be within ±9007199254740991 (2^53 − 1), so that every JSON reader reads it alike and the receipt binds it")));
}
let mut out = match op {
exact_numbers(request)?;
let out = match op {
"gradient" => gradient(request),
"optimize" => optimize(request),
"vqe" => vqe(request),
@ -1342,9 +1510,29 @@ pub fn call(op: &str, request: &Value) -> Result<Value, String> {
"build" => super::algorithms::build(request),
other => Err(oerr(format!("no call {other}: gradient, optimize, vqe, qaoa or build"))),
}?;
Ok(answered(op, out))
}
/// [`call`] for an optimisation stopped and gone on with
/// ([`optimize_resumable`]): the answer once the run ends, `None` while it
/// is stopped.
pub fn call_optimize(request: &Value, from: Option<&Value>, each: impl FnMut(&Paused) -> Result<bool, String>) -> Result<Option<Value>, String> {
exact_numbers(request)?;
Ok(optimize_resumable(request, from, each)?.map(|out| answered("optimize", out)))
}
fn exact_numbers(request: &Value) -> Result<(), String> {
match receipt::inexact_number(request) {
Some(at) => Err(tag(refusal::LIMIT, format!("`{at}`: a whole number here must be within ±9007199254740991 (2^53 − 1), so that every JSON reader reads it alike and the receipt binds it"))),
None => Ok(()),
}
}
/// An answer as [`call`] gives it: its schema, and the call it answers.
fn answered(op: &str, mut out: Value) -> Value {
super::schema::stamp_as(&mut out, super::schema::ALGORITHMS);
out["call"] = json!(op);
Ok(out)
out
}
/// The five MCP tools both servers serve, in one wording: `sealer` names

View file

@ -373,3 +373,104 @@ fn the_answers_are_the_same_bytes_on_every_platform() {
assert_eq!(one, want, "{op}: every digest now: {got:?}");
}
}
/// Stopped after any step and gone on with from its checkpoint, written out
/// as text and read back, a run ends with the same bytes as one that never
/// stopped: for each optimiser, and stopped after every step in turn.
#[test]
fn an_optimisation_stopped_anywhere_and_gone_on_with_is_the_same_bytes() {
let src = header(&["a", "b"], 2) + "ry(a) q[0];\ncx q[0], q[1];\nry(b) q[1];\n";
for extra in [
json!({ "optimizer": "gradient_descent", "steps": 12, "step_size": 0.3 }),
json!({ "optimizer": "adam", "steps": 12 }),
json!({ "optimizer": "spsa", "steps": 12, "seed": 5 }),
json!({ "optimizer": "adam", "steps": 6, "gradient": "parameter_shift" }),
] {
let mut req = json!({ "qasm": src, "params": { "a": 0.2, "b": -0.4 }, "hamiltonian": ["Z1", "0.5 X0"] });
for (k, v) in extra.as_object().unwrap() {
req[k] = v.clone();
}
let whole = optimize(&req).unwrap().to_string();
let steps = req["steps"].as_u64().unwrap() as u32;
// Stopped once, after each step that leaves one to take.
for at in 1..=steps {
let mut kept = None;
let stopped = optimize_resumable(&req, None, |p| {
if p.next() == at {
kept = Some(p.checkpoint().to_string());
return Ok(false);
}
Ok(true)
})
.unwrap();
assert!(stopped.is_none(), "{extra}: stopped at {at}");
let cp: Value = serde_json::from_str(&kept.unwrap()).unwrap();
assert_eq!((cp["next"].as_u64(), cp["trace"].as_array().map(Vec::len)), (Some(u64::from(at)), Some(at as usize)));
let gone_on = optimize_resumable(&req, Some(&cp), |_| Ok(true)).unwrap().unwrap();
assert_eq!(gone_on.to_string(), whole, "{extra}: stopped after step {}", at - 1);
}
// Stopped after every step, one step a slice.
let mut cp: Option<Value> = None;
let mut slices = 0;
let result = loop {
slices += 1;
let mut kept = None;
match optimize_resumable(&req, cp.as_ref(), |p| {
kept = Some(serde_json::from_str::<Value>(&p.checkpoint().to_string()).unwrap());
Ok(false)
})
.unwrap()
{
Some(r) => break r,
None => cp = kept,
}
};
assert_eq!(slices, steps + 1, "{extra}");
assert_eq!(result.to_string(), whole, "{extra}: one step a slice");
}
}
/// A checkpoint goes on only from where it was made: one of another
/// request, one whose steps were changed, and one that does not read are
/// each refused with the optimisation's code.
#[test]
fn a_checkpoint_goes_on_only_from_its_own_run() {
let req = json!({ "qasm": header(&["a"], 1) + "ry(a) q[0];\n", "params": { "a": 0.3 }, "hamiltonian": ["Z0"], "optimizer": "adam", "steps": 8 });
let mut cp = Value::Null;
optimize_resumable(&req, None, |p| {
if p.next() == 4 {
cp = p.checkpoint();
return Ok(false);
}
Ok(true)
})
.unwrap();
assert_eq!(cp["format"], CHECKPOINT);
let refused = |r: &Value, c: &Value| optimize_resumable(r, Some(c), |_| Ok(true)).unwrap_err();
let mut other = req.clone();
other["params"]["a"] = json!(0.31);
let mut longer = req.clone();
longer["steps"] = json!(9);
for r in [&other, &longer] {
let e = refused(r, &cp);
assert!(e.contains("another request") && e.ends_with("[J052]"), "{e}");
}
let edits: Vec<(&str, Box<dyn Fn(&mut Value)>)> = vec![
("a step's value", Box::new(|c| c["trace"][2]["value"] = json!(-1.0))),
("a step dropped", Box::new(|c| drop(c["trace"].as_array_mut().unwrap().pop()))),
("the chain", Box::new(|c| c["chain"] = json!("00".repeat(32)))),
("the next step", Box::new(|c| c["next"] = json!(9))),
("the parameters", Box::new(|c| c["state"]["theta"] = json!(["zz"]))),
("the stream", Box::new(|c| c["state"]["rng"] = json!(7))),
("the format", Box::new(|c| c["format"] = json!("something else"))),
("the parameters' range", Box::new(|c| c["state"]["theta"] = json!([format!("{:016x}", f64::NAN.to_bits())]))),
];
for (what, edit) in edits {
let mut c = cp.clone();
edit(&mut c);
let e = refused(&req, &c);
assert!(e.starts_with("the checkpoint does not go on from here") && e.ends_with("[J052]"), "{what}: {e}");
}
// Its own goes on, and ends as a run that never stopped.
assert_eq!(optimize_resumable(&req, Some(&cp), |_| Ok(true)).unwrap().unwrap(), optimize(&req).unwrap());
}